Abstract
Background:
Studies show that a large proportion of healthcare offered may be inappropriate or unnecessary. Magnetic resonance imaging (MRI) is a new and expensive diagnostic technology which has been increasingly used all over the world. Moreover, this trend has been more rapidly increasing in Iran. Low back pain is a common disorder all over the world and MRI technique is one of the several ways to assess its cause.Objectives:
The present study aims to develop scenarios for lumbar spine MRI.Materials and Methods:
In the present study, the RAND Appropriateness Method (RAM) was used in order to reach consensus regarding developing scenarios for lumbar spine MRI. We generated scenarios from valid clinical guidelines as well as the experts’ opinion. The panel members included nine specialists from various medical specialties that had scored scenarios in two rounds, the first of which was without interaction, while the second one was with interaction.Results:
We extracted 97 scenarios for the lumbar spine MRI in the scenario extracting phase of the study and the panel members added 18 scenarios. After implementation of two rounds, the scenarios were categorized into three ranges. Sixty seven (58%) of the scenarios were considered as appropriate, 45 (39%) as uncertain, and three (2.6%) as inappropriate.Conclusions:
RAM is useful for identifying stakeholder views in settings with limited resources. Since RAM has precise instructions for consensus developing, a large number of scenarios were considered as uncertain. Therefore, more research has to be conducted on the issue.Keywords
Lumbar Vertebrae Magnetic Resonance Imaging RAND RAM Clinical Scenario
1. Background
Healthcare systems face significant and rapid changes in response to changes in population needs and increasing costs (1, 2). Studies show that a large proportion of the healthcare offered is inappropriate or unnecessary, ranging from 15 to 30 percent in many countries and as much as 40 percent in some private clinics (3, 4). The quality of and access to healthcare are not related to high expenditures in the health system (5). Medical imaging costs have had an incremental trend worldwide which might be due to overuse. In fact, among all medical imaging technologies, magnetic resonance imaging (MRI) usage has noticeably had a more incremental trend (6, 7, 8, 9, 10). In case physicians order inappropriately demanded medical imaging procedures, it may result in serious problems in terms of both healthcare economy and quality (11). Moreover, medical imaging overuse might be unable to improve the patients’ health status. As studies have shown, 20% - 50% of high-tech imaging procedures have been unnecessary. (12, 13, 14, 15). Various methods have been developed in order to measure the appropriateness of health care and to develop valid criteria for the physicians (1, 2, 16, 17). Financial and human resources are needed in order to develop the clinical guidelines and these resources may not be available for low and middle income countries (18). Due to these limitations, we aimed to identify the criteria for using available valid guidelines.
2. Objectives
Since low back pain (LBP) is one of the most prevalent disorders (19, 20) and there is little consensus among the specialists about appropriate LBP procedures (21, 22), we aimed to identify the appropriate criteria for utilization of MRI in LBP patients based on the viewpoints of the experts in the country. The result of this study may be useful for policymakers as well as clinicians in Iran and in other low and middle income countries.
3. Patients and Methods
RAND Appropriateness Method (RAM) was selected for the present study since it allows the development of the appropriateness criteria based on the available evidence which is supplemented by the expert panels’ opinion. This method was designed in the 1980s by the RAND institute and the University of California in Los Angeles (UCLA) and has been used in many studies in North America and Europe. RAM involves generating clinical scenarios or criteria. RAM has been used in many studies (2, 23) especially for development of the appropriateness criteria in surgical care as well as investigative procedures (2, 24, 25, 26). The present study was conducted in Shiraz in 2011. Besides, followed is a modified version of RAM which is explained in the next section.
3.1. Generating Scenarios
We started the study by searching for available evidence. In the process of the literature review, we focused on clinical practice guidelines and evidence reviews. In this way,10 relevant documents were identified (27, 28, 29, 30, 31, 32, 33, 34, 35). Then, we reviewed and verified the identified documents and selected four guidelines and evidence reviews which provided a relatively comprehensive coverage of the issues related to MRI for LBP (34, 35). Two panel members as well as two authors used the Persian translation of the AGREE (Appraisal of Guidelines for Research and Evaluation) tool (36) in order to assess the guidelines and finally selected two of them from which the indications and the scenarios were extracted. Each indication was divided into several scenarios and each scenario included a few words which described the patients’ signs. First, we identified the major specialists who were involved in the prescription of MRI for low back pain. we found that six majors were involved in this issue. Then, nine specialists were invited to participate in the panel.
3.2. Panel Members
The panel consisted of nine specialists: two neurosurgeons, two orthopedic surgeons, two radiologists, one neurologist, one rheumatologist and one physiatrist, all of whom were faculty members and worked in educational hospitals. Before the process of consensus making, we talked to all the panelists and described the objectives of the study as well as the process of consensus making. If they agreed, we invited them to participate in the study.
3.3. Consensus Making
The process of consensus making among the specialists was implemented in two rounds: without interaction and with interaction. In the first round, we sent the first form (including the list of scenarios, summaries of the clinical guidelines and the scoring system) to each expert panelist. We asked them to select the best score for each scenario based on their professional judgment and summary of evidence. The scores ranged from one (absolutely inappropriate) to nine (absolutely appropriate). Then, they sent the scored scenarios back to us. We entered all the specialists’ scores to the second form and prepared it for the second round of consensus development. The second forms provided an opportunity for each panelist to be informed about the other panelists’ scores anonymously. In the second round, the members were invited to a meeting. In the meeting, we gave the second form to all the panel members. One author (MHB) was selected as the panel leader. He read each scenario, asked the panelists about each scenario and if necessary, changed their first round scores. The meeting lasted for six hours. Then, the new indications presented by the specialists were collected and discussed. In the meeting, it was emphasized that the panelists should score the scenarios independently while considering the panel discussions.
3.4. Statistical Analysis
Median scores were used to judge the appropriateness of the scenarios. The scores were categorized into three groups: appropriate (score = 7-9), uncertain (score = 4-6), and inappropriate (score = 1-3). Of course, in order to reach agreement, another condition had to be met: if minimum and maximum scores are ignored, all the other scores must fall in the same scoring group.
4. Results
We extracted 97 scenarios for the lumbar spine MRI in the scenario extracting phase of the study. We divided the scenarios into sixteen indications, as used in the selected guidelines (34, 35), in order to make the assessment easier for the panel members (23). Totally, over eighty nine percent of the scenarios were considered as appropriate, but the agreement was reached on 72 (74%) of them. Only three scenarios were considered inappropriate at the first round. According to the results, the second and the fifth indications had the highest extracted scenarios. Comparing the indications, the tenth indication contained the highest number of appropriate scenarios with agreement (Table 1). In the second round of consensus making, the panel members added 18 indications to the first list, all of which were appropriate, but the agreement on them was zero. After discussion during the second panel, four indications were ignored from the list and the consensus process was done on 30 indications. The rate of appropriateness and agreement of scenarios in the second round is presented in Table 2. It is seen that among the 111 scenarios of lumbar spine MRI, 103 were considered as appropriate, amongst which 68 (66%) scenarios were agreed upon. We prepared a table including some examples of appropriateness of the scenarios which are shown in (Table 3).
Agreement on Appropriateness of the Scenarios for Lumbar Spine MRI According to the 16 Main Indications: First Round of Consensus Development
| MRI Indications | Total Number of Scenarios | Appropriateness of Scenarios | |||
| 1 | Acute low back pain (onset within the past 6 weeks), spinal infection | 16 | 0 | 3 (18.75) | 13 (81) |
| 2 | Acute low back pain ( onset within the past 6 weeks), progressive (objective) neurological signs | 2 | 0 (0) | 1 (50) | 1 (50) |
| 3 | Acute low back pain (onset within the past 6 weeks), suspicion of cauda equina syndrome | 4 | 0 (0) | 2 (50) | 2 (50) |
| 4 | Acute low back pain,( onset within past 6 weeks), suspicion of cancer | 6 | 0 (0) | 2 (34) | 4 (66.6) |
| 5 | Sub-acute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and spinal infection: | 16 | 0 (0) | 6 (38) | 10 (62.5) |
| 6 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and progressive (objective) neurological signs: | 1 | 0 (0) | 0 (0) | 1 (100) |
| 7 | Sub-acute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspected cauda equina syndrome | 2 | 0 (0) | 0 (0) | 2 (100) |
| 8 | Sub-acute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspicion of cancer | 3 | 0 (0) | 3 (100) | 0 (0) |
| 9 | Sub-acute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspected radiculopathy | 3 | 0 (0) | 0 (0) | 3 (100) |
| 10 | Chronic low back pain (> 3 months), spinal infection | 18 | 0 (0) | 0 (0) | 18 (100) |
| 11 | Chronic low back pain (> 3 months), progressive (objective) neurological signs | 3 | 0 (0) | 1 (33.3) | 2 (66.6) |
| 12 | Chronic low back pain (> 3 months), suspected cauda equina syndrome | 6 | 2 (33.3) | 2 (33.3) | 2 (33.3) |
| 13 | Chronic low back pain (> 3 months), suspicion of cancer | 9 | 1 (11.11) | 6 (66) | 2 (22.2) |
| 14 | Chronic low back pain (> 3 months), suspected radiculopathy | 3 | 0 (0) | 0 (0) | 3 (100) |
| 15 | Chronic low back pain (> 3 months) | 2 | 0 (0) | 0 (0) | 2 (100) |
| 16 | Chronic low back pain (> 3 months), considered a candidate for spine surgery | 3 | 0 (0) | 1 (33.3) | 2 (66.6) |
| Total | 97 | 3 (3) | 27 (28) | 67 (69) | |
Agreement on Appropriateness of the Scenarios for Lumbar Spine MRI According to the 30 Main Indications: Second Round of Consensus Development
| MRI Indications | Total Number of Scenarios | Appropriateness of Scenarios | |||
| Inappropriate, No. (%) | Appropriate, No. (%) | Equivocal, No. (%) | |||
| 1 | Acute low back pain (onset within past 6 weeks), spinal infection | 16 | 0 | 0 | 16 (100) |
| 2 | Acute low back pain,( onset within past 6 weeks),Progressive (objective) neurological signs | 2 | 0 (0) | 1 (50) | 1 (50) |
| 3 | Acute low back pain,( onset within past 6 weeks), suspected cauda equina syndrome | 4 | 0 (0) | 0 (0) | 4 (100) |
| 4 | Acute low back pain, (onset within past 6 weeks), suspicion of cancer | 6 | 0 (0) | 0 (0) | 6 (100) |
| 5 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and spinal infection: | 16 | 0 (0) | 0 (0) | 16 (100) |
| 6 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and progressive (objective) neurological signs: | 1 | 0 (0) | 0 (0) | 1 (100) |
| 7 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspected cauda equina syndrome: | 2 | 0 (0) | 0 (0) | 2 (100) |
| 8 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspicion of cancer | 3 | 0 (0) | 0 (0) | 3 (100) |
| 9 | Subacute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment and suspected radiculopathy | 3 | 0 (0) | 0 (0) | 3 (100) |
| 10 | Chronic low back pain (> 3 months), spinal infection | 18 | 0 (0) | 0 (0) | 18 (100) |
| 11 | Chronic low back pain (> 3 months), progressive (objective) neurological signs | 3 | 0 (0) | 1 (33.3) | 2 (66.6) |
| 12 | Chronic low back pain (> 3 months), suspected cauda equina syndrome | 6 | 2 (33.3) | 2 (33.3) | 2 (33.3) |
| 13 | Chronic low back pain (> 3 months), suspicion of cancer | 9 | 1 (11.1) | 2 (22.2) | 6 (66.6) |
| 14 | Chronic low back pain (> 3 months), suspected radiculopathy | 3 | 0 (0) | 0 (0) | 3 (100) |
| 15 | Chronic low back pain (> 3 months) | 2 | 0 (0) | 0 (0) | 2 (100) |
| 16 | Chronic low back pain (> 3 months), the patient is considered a candidate for spine surgery: | 3 | 0 (0) | 1 (33.3) | 2 (66.6) |
| 17 | Traumatic acute disc herniations | 1 | 0 (0) | 1 (100) | 0 (0) |
| 18 | Evaluation posterior ligamentous complexity of spine in type B (distruction) spine injuries | 1 | 0 (0) | 1 (100) | 0 (0) |
| 19 | SCI a, normal X ray, normal CT a and neurological deficit need to MRI a evaluation of spinal cord. | 1 | 0 (0) | 1 (100) | 0 (0) |
| 20 | Postoperative evaluation of failed back syndrome (postoperation residual symptoms). | 1 | 0 (0) | 1 (100) | 0 (0) |
| 21 | Unhealed osteoprotic vertebral compression fracture (MRI without contrast is indicated) | 1 | 0 (0) | 1 (100) | 0 (0) |
| 22 | Kummel's disease of vertebra (MRI without contrast) | 1 | 0 (0) | 1 (100) | 0 (0) |
| 23 | Spinal instability secondary to ligament laxity or torn ligament | 1 | 0 (0) | 1 (100) | 0 (0) |
| 24 | Central cord syndrome (trauma- tumor) syringomyelia | 1 | 0 (0) | 1 (100) | 0 (0) |
| 25 | Traumatic cord injuries | 1 | 0 (0) | 1 (100) | 0 (0) |
| 26 | Myelo neuropathy (vit B12 deficiency, HTLV, neurosyphilis) | 1 | 0 (0) | 1 (100) | 0 (0) |
| 27 | Discitis (MRI with contrast) | 1 | 0 (0) | 1 (100) | 0 (0) |
| 28 | Spinal cord infarction | 1 | 0 (0) | 1 (100) | 0 (0) |
| 29 | Traumatic spine injury with LBP spine MRI without contrast | 1 | 0 (0) | 1 (100) | 0 (0) |
| 30 | Spine surgery with LBP & discopathy & radiculopathy: rule out recurrent discopathy: MRI without and with contrast. | 1 | 0 (0) | 1 (100) | 0 (0) |
| Total | 111 | 3 (2.7) | 41 (36.9) | 67 (60) | |
Examples of the Lumbar Spine MRI Scenario Under Each Indication that Extracted to Identify Appropriateness a by the Expert Panel
| A | Acute low back pain (onset within past 6 weeks) |
| Spinal infection | |
| 1-having fever, needed to MRI without contrast. A+ | |
| 2- Suspicion of systemic or spinal infection, needed to MRI without contrast A? | |
| B | Acute low back pain ( onset within past 6 weeks), |
| Progressive (objective) neurological signs | |
| 1-Progressive motor weakness present, needed to MRI without contrast A+ | |
| 2- Progressive motor weakness present, needed to MRI with contrast U? | |
| C | Acute low back pain,( onset within past 6 weeks) |
| Suspected cauda equina syndrome | |
| 1- Bilateral neurologic signs and symptoms, needed to MRI without contrast: A+ | |
| 2- Bilateral neurologic signs and symptoms, needed to MRI with contrast: A? | |
| D | Acute low back pain (onset within past 6 weeks) |
| Suspicion of cancer | |
| 1- Unexplained weight loss & failure to improve after one month, needed to MRI without contrast: A+ | |
| 2- Unexplained weight loss & failure to improve after one month, needed to MRI with contrast: A+ | |
| E | Sub-acute Low back pain > 6 weeks, at least 6 weeks medical/conservative treatment |
| Spinal infection | |
| 1- having fever, needed to MRI without contrast: A+ | |
| 2- having fever, needed to MRI with contrast: A+ | |
| F | Sub-acute low back pain > 6 weeks, At least 6 weeks medical/conservative treatment |
| Progressive (objective) neurological signs | |
| 1- Progressive motor weakness present, needed to MRI without contrast: A+ | |
| G | Sub-acute low back pain > 6 weeks, At least 6 weeks medical/conservative treatment and |
| Suspected cauda equina syndrome | |
| 1-Bilateral neurologic signs and symptoms, needed to MRI without contrast: A+ | |
| 2- Acute bladder or bowel dysfunction, needed to MRI without contrast: A+ | |
| H | Sub-acute low back pain > 6 weeks, at least 6 weeks medical/conservative treatment |
| Suspicion of cancer | |
| 1- Unexplained weight loss & failure to improve after one month, needed to MRI without contrast: A? | |
| I | Sub-acute low back pain > 6 weeks,= At least 6 weeks medical/conservative treatment |
| Suspected radiculopathy | |
| 1- Leg pain is > than back pain & pain present in nerve root distribution & positive straight leg raising test < 45º OR positive crossed straight leg raising test, MRI without contrast: A+ | |
| J | Chronic low back pain (> 3 months) |
| Spinal infection | |
| 1-Suspicion of systemic or spinal infection, MRI with & without contrast: A+ | |
| 2-IV drug use , MRI with & without contrast: A+ | |
| K | Chronic low back pain (> 3 months) |
| Progressive (objective) neurological signs | |
| 1- Progressive motor weakness present, MRI without contrast: A+ | |
| 2-Progressive motor weakness present, MRI with contrast: U- | |
| L | Chronic low back pain (> 3 months) |
| Suspected cauda equina syndrome | |
| 1-Bilateral neurologic signs and symptoms, MRI without contrast: A+ | |
| 2- Acute bladder or bowel dysfunction, MRI without contrast: A+ | |
| M | Chronic low back pain (> 3 months) |
| Suspicion of cancer | |
| 1-Unexplained weight loss & failure to improve after one month, MRI with & without contrast: A+ | |
| 2-Failure to improve after one month & age over 50, MRI with & without contrast: A? | |
| N | Chronic low back pain (> 3 months) |
| Suspected radiculopathy | |
| 1-Leg pain is > than back pain& pain present in nerve root distribution & positive straight leg raising test < 45º OR positive crossed straight leg raising test, MRI without contrast: A+ | |
| O | Chronic low back pain ( > 3 months): |
| 1-Suspicion of substantial spinal stenosis on another imaging procedure, MRI without contrast: A+ | |
| P | Chronic low back pain ( > 3 months) |
| Patient is considered a candidate for spine surgery | |
| 1- Progressive changes in objective neurological findings, MRI without contrast: A+ | |
| 2- At least 1 year since last lumbar MRI (without objective change in neurological signs): A? MRI without contrast |
5. Discussion
To the best of our knowledge, this is the first study that has used the RAND Appropriateness Method for lumbar spine MRI indications. In our study, lumbar spine MRI was considered in 67 (58%) of the scenarios as appropriate, 45 (39%) as uncertain, and three (2.6%) as inappropriate. After two rounds of consensus development, there were still 67 scenarios in which no agreement was reached or the results remained equivocal. This may demonstrate the ambiguous nature of decision making on whether the MRI is indicated for a patient. It may partly explain why it has been so difficult to curb or slow the growth in unnecessary services provided such as the lumbar spine MRI rate around the world (3, 4, 12, 13, 14, 15). Our results will help decision makers in identifying appropriate procedures and focusing their efforts on decreasing unnecessary care. In evidence-based medicine, the question arises that what should be done if there is insufficient evidence for a procedure routinely performed in practice (37). In developed countries, evidence-based clinical guidelines have been helpful, but in middle and low income countries there are serious limitations in preparing these materials (38, 39). Moreover, evidence-based clinical guidelines often lack flexibility and may not provide enough details for clinicians when making decisions about individual patients (38, 40). Formal consensus development (including RAM) provides a timely and efficient solution when evidence is insufficient (40). In our study, we overcame this limitation by developing scenarios representative of the patients seen by clinicians in practice. It should be noted that consensus methods are most widely used in procedures that there is no agreement on. Moreover, no studies were found on using these methods in the lumbar spine MRI. Rashidian (40) performed a review of the evidence in order to identify the indicators of the lumbar spine MRI. General similarities may be observed between the results of the study carried out by Roudsari and the present study. However, our method was more effective since each of the indicators was divided into detailed and accurate scenarios. Accurate and detailed scenarios can be useful for physicians in practice. In some studies, questions remain about the validity of recommendations based on such methods (39). Tan et al.(41) have described the complexities and limitations of using RAM. Many of their criticisms equally apply to other consensus development methods. It should be noted that such methods are useful when there are disagreements or variation in practice and reliable evidence is limited. In these circumstances, formal consensus methods are valuable and their use is inevitable. We spent time on familiarizing the panel members with the method and attracting their valued cooperation. Membership of the panel involves open discussions of personal views and practices and that certain practices might not be supported by evidence or by other panel members. As an advantage of our study, we used the AGREE tool for selection of the evidence sources. This provided a chance for the panel to reach a shared understanding of the evidence before embarking towards consensus building. We also selected the members from different backgrounds and settings to improve comprehensiveness of the views (42, 43). RAM usually results in a long list of scenarios. To ease the use of its results, it may be possible to develop user-friendly software, or to categorize the scenarios into indications and packages in the format of clinical guidelines. We demonstrated that the RAM is useful for development of scenarios for appropriateness of lumbar spine MRI in a developing country. The method should be used more widely in such settings in other areas of healthcare where controversies exist or the practice varies. It also has the added value of developing a level of ownership by the providers if they see their peers and relevant stakeholders are adequately represented in the process.
The findings of this study can be used for developing national guidelines, conducting research to assess whether the criteria are followed in practice and whether their application can curb the growing rate of unnecessary care in all countries. In Iran, a limited amount of resources are allocated to the health sector. Since MRI is an expensive method of medical diagnosis, doctors have to prescribe it after the easier ways of diagnosis are not useful.
Acknowledgements
References
-
1.
Hicks NR. Some observations on attempts to measure appropriateness of care. BMJ. 1994;309(6956):730-3. [PubMed ID: 7950529]. https://doi.org/10.1136/bmj.309.6956.730.
-
2.
Nicollier-Fahrni A, Vader JP, Froehlich F, Gonvers JJ, Burnand B. Development of appropriateness criteria for colonoscopy: comparison between a standardized expert panel and an evidencebased medicine approach. Int J Qual Health Care. 2003;15(1):15-22. [PubMed ID: 12630797]. https://doi.org/10.1093/intqhc/15.1.15.
-
3.
Phelps CE. The methodologic foundations of studies of the appropriateness of medical care. N Engl J Med. 1993;329(17):1241-5. [PubMed ID: 8413392]. https://doi.org/10.1056/NEJM199310213291707.
-
4.
Borowitz M, Sheldon T. Controlling health care: from economic Keshtkaran A et al. incentives to micro-clinical regulation. Health Econ. 1993;2(3):2014. [PubMed ID: 8275164]. https://doi.org/10.1002/hec.4730020302.
-
5.
Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending. Part 1: the content, quality, and accessibility of care. Ann Intern Med. 2003;138(4):273-87. [PubMed ID: 12585825].
-
6.
Fuchs VR, Sox HC, Jr. Physicians’ views of the relative importance of thirty medical innovations. Health Aff (Millwood). 2001;20(5):30-42. [PubMed ID: 11558715]. https://doi.org/10.1377/hlthaff.20.5.30.
-
7.
Greenberg D, Peterburg Y, Vekstein D, Pliskin JS. Decisions to adopt new technologies at the hospital level: insights from Israeli medical centers. Int J Technol Assess Health Care. 2005;21(2):21927.
-
8.
Hillman AL, Schwartz JS. The diffusion of MRI: patterns of siting and ownership in an era of changing incentives. AJR Am J Roentgenol. 1986;146(5):963-9. [PubMed ID: 3485912].
-
9.
Oh EH, Imanaka Y, Evans E. Determinants of the diffusion of computed tomography and magnetic resonance imaging. Int J Technol Assess Health Care. 2005;21(1):73-80. [PubMed ID: 15736517]. https://doi.org/10.1017/S0266462305050099.
-
10.
Passariello R. Cost containment and diffusion of MRI: Oil and water? The situation in Europe. Eur Radiol. 1997;(Suppl 5):259-62. https://doi.org/10.1007/PL00006905.
-
11.
Palesh M, Fredrikson S, Jamshidi H, Jonsson PM, Tomson G. Diffusion of magnetic resonance imaging in Iran. Int J Technol Assess Health Care. 2007;23(2):278-85. [PubMed ID: 17493315]. https://doi.org/10.1017/S0266462307070377.
-
12.
Picano E. Sustainability of medical imaging. BMJ. 2004;328(7439). [PubMed ID: 15001510]. https://doi.org/10.1136/bmj.328.7439.578.
-
13.
Angrisano C, Farrell D, Kocher B, Laboissiere M, Parker S. Accounting for the cost of health care in the United States. McKinsey Global Institute; 2007. Available from: http://www.cows.org/healthcarecharrette/McKinsey-Healthcare.pdf.
-
14.
Beever C, Karbe M. The cost of medical technologies: maximizing the value of innovation. McLean, Va:: Booz Allen Hamilton; 2003. Available from: http://www.booz.com/media/file/137991.pdf.
-
15.
Brenner DJ, Hall EJ. Computed tomography--an increasing source of radiation exposure. N Engl J Med. 2007;657(22):2277-84. [PubMed ID: 18046031]. https://doi.org/10.1056/NEJMra072149.
-
16.
Liu X, Mills A. Evaluating payment mechanisms: how can we measure unnecessary care? Health Policy Plan. 1999;14(4):409-13. [PubMed ID: 10787657]. https://doi.org/10.1093/heapol/14.4.409.
-
17.
Rashidian A. Adapting valid clinical guidelines for use in primary care in low and middle income countries. Prim Care Respir J. 2008;17(3):136-7. [PubMed ID: 18695845]. https://doi.org/10.3132/pcrj.2008.00055.
-
18.
Hart LG, Deyo RA, Cherkin DC. Physician office visits for low back pain. Frequency, clinical evaluation, and treatment patterns from a U.S. national survey. Spine (Phila Pa 1976). 1995;20(1):11-9.
-
19.
Deyo RA, Mirza SK, Martin BI. Back pain prevalence and visit rates: estimates from U.S. national surveys, 2002. Spine (Phila Pa 1976). 2006;31(23):2724-7.
-
20.
Cherkin DC, Deyo RA, Wheeler K, Ciol MA. Physician variation in diagnostic testing for low back pain. Who you see is what you get. Arthritis Rheum. 1994;37(1):15-22. https://doi.org/10.1002/art.1780370104.
-
21.
Ostovar R, Rashidian A, Pourreza A, Rashidi BH, Hantooshzadeh S, Ardebili HE. Developing criteria for cesarean section using the RAND appropriateness method. BMC Pregnancy Childbirth. 2010;10:52. [PubMed ID: 20840776]. https://doi.org/10.1186/1471-2393-10-52.
-
22.
Cherkin DC, Deyo RA, Wheeler K, Ciol M. Physician views about treating low back pain. The results of a national survey. Spine (Phila Pa 1976). 1995;20(1):1-9. discussion -10.
-
23.
Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR. 2001. The RAND/UCLA Appropriateness Method: Users Manual. Available from: http://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf.
-
24.
Porchet F, Vader JP, Larequi-Lauber T, Costanza MC, Burnand B, Dubois RW. The assessment of appropriate indications for laminectomy. J Bone Joint Surg Br. 1999;81(2):234-9. [PubMed ID: 10204927]. https://doi.org/10.1302/0301-620X.81B2.8871.
-
25.
McDonnell J, Stoevelaar HJ, Bosch JL, Kahan JP. The appropriateness of treatment of benign prostatic hyperplasia: a comparison of Dutch and multinational criteria. Health Policy. 2001;57(1):4556. https://doi.org/10.1016/S0168-8510(01)00127-0.
-
26.
Quintana JM, Arostegui I, Azkarate J, Goenaga JI, Guisasola I, Alfageme A, et al. Evaluation by explicit criteria of the use of total hip joint replacement. Rheumatology (Oxford). 2000;39(11):123441. [PubMed ID: 11085803]. https://doi.org/10.1093/rheumatology/39.11.1234.
-
27.
Management AI. Program Guidelines. 2010. Diagnostic Imaging Utilization Management. Available from: http://www.bcbsnc.com/assets/providers/public/pdfs/aim_guidelines_2010-2011.pdf.
-
28.
Pathways DI. 2012. A Clinical Decision Support Tool and Educational Resource for Diagnostic Imaging. Available from: http://www.imagingpathways.health.wa.gov.au.
-
29.
Association AM. Clinical UM Guideline. Available from: http://www.anthem.com/anthem404.html.
-
30.
Healthcare U. 2010. Criteria for Imaging Version 1. Available from: https://www.unitedhealthcareonline.com/ccmcontent/ProviderII/UHC/en-US/Assets/ProviderStaticFiles/ProviderStaticFilesPdf/Clinician%20Resources/Radiology%20Notification%20Program/Radiology%20Notification%20docs/Evidence_based_Clinical_Guidelines.pdf.
-
31.
Care NCCfP. 2009, [updated May]. Low back pain: early management of persistent non-specific low back pain Full guideline. 2010/08/13:[Available from: http://www.ncbi.nlm.nih.gov/pubmed/20704057.
-
32.
Radiologists CAo. Diagnostic Imaging Referral Guidelines: A Guide for Physicians. Cdn. Assoc. of Radiologists; 2005.
-
33.
National Institute for Health and Clinical Excellence. 2009. Quick reference guide. Early management of persistent non-specific low back pain Issue date. Available from: http://www.nice.org.uk/nicemedia/pdf/CG88QuickRefGuide.pdf.
-
34.
American College of Physicians. 2007. ASP Clinical Guidelines. Available from: http://www.acponline.org/clinical_information/guidelines/guidelines/.
-
35.
American College of Radiology ACR Appropriateness Criteria. 2008. Low Back Pain 35. Available from: http://www.acr.org/~/media/ACR/Documents/AppCriteria/Diagnostic/LowBackPain.pdf.
-
36.
Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1). [PubMed ID: 12571340]. https://doi.org/10.1136/qhc.12.1.18.
-
37.
Black N. Evidence-based surgery: A passing fad? World J Surg. 1999;23(8). [PubMed ID: 10415204]. https://doi.org/10.1007/s002689900581.
-
38.
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, et al. Consensus Development Methods and their use in clinical guidelines development. Health Technol Assess. 1998;2(3).
-
39.
Schill WB. Some disturbances of acrosomal development and function in human spermatozoa. Hum Reprod. 1991;6(7):969-78. [PubMed ID: 1761668].
-
40.
Rashidian A, Eccles MP, Russell I. Falling on stony ground? A qualitative study of implementation of clinical guidelines’ prescribing recommendations in primary care. Health Policy. 2008;85(2):148-61. [PubMed ID: 17767976]. https://doi.org/10.1016/j.healthpol.2007.07.011.
-
41.
Tan C, Treasure T, Browne J, Utley M, Davies CW, Hemingway H. Seeking consensus by formal methods: a health warning. J R Soc Med. 2007;10(1):10-4. [PubMed ID: 17197680]. https://doi.org/10.1258/jrsm.100.1.10.
-
42.
Coulter I, Adams A, Shekelle P. Impact of varying panel membership on ratings of appropriateness in consensus panels: a comparison of a multi- and single disciplinary panel. Health Serv Res. 1995;30(4):577-91. [PubMed ID: 7591782].
-
43.
Kahan JP, Park RE, Leape LL, Bernstein SJ, Hilborne LH, Parker L, et al. Variations by specialty in physician ratings of the appropriateness and necessity of indications for procedures. Med Care. 1996;34(6):512-23. [PubMed ID: 8656718]. https://doi.org/10.1097/00005650-199606000-00002.