Accuracy of Brain MRI Interpretation of Emergency Medicine Specialists Versus Radiologist and Neurologist in Suspected Acute Ischemic Stroke Patients; a Diagnostic Accuracy Study

1. Background

Magnetic resonance imaging (MRI) is one of the advanced technologies used in medical diagnostic procedures. Today, it has many applications and is widely used, especially in educational medical centers. As one of the uses, we can mention its diagnostic importance in dealing with suspected stroke patients in their therapeutic golden time. Although in the proposed algorithm for thrombolytic therapy in patients with acute ischemic stroke (AIS), there is no place for this diagnostic procedure (1); however, there may be an indication in wake up stroke and unknown time stroke cases. Such patients who were previously considered contraindicated for thrombolytic therapy may benefit from this treatment if there is a mismatch between fluid-attenuated inversion recovery (FLAIR) and diffusion-weighted imaging (DWI) images of brain MRI (2). However, in some instances, differentiation of AIS from other differential diagnoses may be promptly required in which brain MRI is done. So, there is a need for early reports of such imaging via telemedicine or available physicians in the emergency department (ED). On the one hand, quick and easy access to the radiologist is still not possible in most EDs; and on the other hand, telemedicine has not been established widely in developing countries like Iran. But the need for making quick and correct decisions is still present. Therefore, emergency medicine physicians (EMPs) also need to be able to interpret these types of radiological imaging (3). Such reasons have led to the addition of practical interpretation skills of brain MRI images in the curriculum of emergency medicine residency. Although some studies investigated the accuracy of the interpretation of various imaging types, such as plain X-ray, ultrasonography, and computed tomography (CT) scan conducted by EMPs (4-6), to the best of our knowledge, there is no study available in the literature, in which EMPs ability has been assessed in terms of interpretation of brain MRI of probable AIS patients. Therefore, we decided to investigate the accuracy of interpreting brain MRI of suspected stroke patients conducted by EMPs in comparison with a radiologist and a neurologist.

2. Methods

2.1. Study Design

This diagnostic accuracy study was conducted from April to November 2019 in Tehran, Iran. The study proposal was approved by the ethics committee of Tehran University of Medical Sciences (code: IR.TUMS.MEDICINE.REC.1396.4640). All data were gathered, analyzed, and interpreted anonymously. The eligible individuals were formally invited and participated in the study if they gave their verbal consent. Due to the nature of the study, no interfere was made in the diagnosis and treatment process of the patients, and only the available information was used in this study.

2.2. Study Participants

All emergency medicine attending physicians (including 11 faculty members) of one major educational medical center affiliated to Tehran University of Medical Sciences participated. A radiologist and a neurologist that both were experienced faculty members of the same center also participated. Considering previous studies in which imaging interpretation skills of EMPs were compared with other specialists, the least sample size was calculated as 148 stereotypes.

2.3. Data Gathering

Using Picture archiving and communication system (PACS) of the hospital, a set of brain MRI stereotypes related to patients suspected of having a stroke was selected in a fixed period of time to complete the required sample size. Thereafter, the patients’ presentation key points on admission to the ED were extracted from their file records. A 2-part checklist was designed in this regard in which the first part included a brief history of the patients (age, sex, level of consciousness, focal neurologic deficits, if any), and the second part belonged to the brain MRI interpretation of the patient that had to be filled by the participating faculty members of emergency medicine, radiology, and neurology, separately.

2.4. Statistical Analysis

By reviewing the brain MRI interpretation of EMPs, once in comparison with the radiologist and once in comparison with the neurologist, misinterpretations (presence or absence of findings compatible with the diagnosis of AIS) were extracted. First, according to the nature of the variables, frequency (percentage) or average (standard deviation) as well as tables and graphs were used to describe the data. In data analysis, diagnostic agreement between groups of experts was evaluated using kappa coefficient. To evaluate the accuracy of diagnosis of EMPs with neurologist and radiologist as the gold standard, a decision matrix table was presented for the groups. In these tables, true positive and true negative cases, as well as false positive and false negative cases, were presented in comparison with the gold standard. Accuracy of diagnosis and indicators of sensitivity, specificity, likelihood ratios, and positive and negative predictive values along with 95% confidence interval were calculated. All analyzes were performed using Stata software version 15.

3. Results

Brain MRI stereotypes of 287 suspected AIS patients were interpreted in this study by the participated EMPs; of these patients, 160 cases (55.7%) were male. The mean age of the study patients was 65.0 ± 14.1 (range of 18 to 98) years, and mostly (> 80%) were in age range of 48 - 88 years.

3.1. Agreement of the Participants on the Diagnosis of Stroke

To examine the agreement between the diagnosis of AIS among three types of specialists, 115 cases were reviewed by all three groups. Of these, 54 (0.47) were positively diagnosed by at least one specialist. The percentage of the agreement to diagnose positive cases between the three groups of specialists was 53.7% (29 people). Also, 5 patients (9.3%) in the positive diagnosis in the neurology group, and eight patients (14.8%) in the positive diagnosis in the emergency medicine group were not approved by either of the other two groups. The participating radiologist did not have any positive diagnosis that was not confirmed by at least one of the other two groups (Figure 1).

Distribution of positively diagnosed acute ischemic stroke cases in the diagnosis of three groups of specialists in study cases

Figure 1.

Distribution of positively diagnosed acute ischemic stroke cases in the diagnosis of three groups of specialists in study cases

3.2. Agreement and Accuracy of Experts' Diagnosis

In order to check the agreement and accuracy of the diagnosis of the EMPs with the neurologist and radiologist, the data of the cases that existed for the two groups of specialists were analyzed. The accuracy of the neurologist's diagnosis was also compared with radiology.

Of the total patients, 200 were jointly evaluated by an EMP and neurologist to diagnose AIS. The value of the agreement for diagnosis based on kappa coefficient in these two groups was 0.684 (95% CI: 0.580 to 0.787). Considering the neurologist as the gold standard, the accuracy of AIS diagnosis by the EMPs was 0.85% (95% CI: 79.3 to 89.6). Also, the sensitivity and specificity of the EMPs for AIS diagnosis were 84.9 and 0.85%, respectively. Also, the positive and negative predictive value of EMPs for AIS diagnosis was 76.5% and 90.8%, respectively (Table 1).

154 patients (MRI) suspected to have an ischemic stroke evaluated by both emergency physician and radiologist. The agreement value for diagnosis in these two groups was 0.673 (95% CI: 0.553 to 0.794). Considering the radiologist as the gold standard, the accuracy of AIS diagnosis by the EMPs was 86.3% (95% CI: 79.8 to 91.3). Also, the sensitivity and specificity of the EMPs for diagnosing AIS were 89.4 and 83.2%, respectively. Also, the positive and negative predictive values of emergency medicine specialists for stroke diagnosis were 0.70 and 94.7%, respectively (Table 1).

It should be mentioned that of the total number of cases, 116 were jointly evaluated by both neurologist and radiologist in terms of AIS diagnosis. The agreement value for diagnosis in these two groups was 0.752 (95% CI: 0.627 to 0.877).

4. Discussion

The findings of the current study revealed that the accuracy of brain MRI interpretation performed by the EMPs was proper (more than 85%). Therefore, when there is a need for early reports of such imaging, the interpretation performed by the EMPs can be acceptable.

Radiology, undoubtedly, is the most helpful para-clinical assessment in terms of accurate diagnosis in EDs. In order to make the best use of radiology, physicians must have the skills to interpret imaging (4). In addition to various types of plain X-ray and CT scan, MRI has become an available method in our country (Iran) today. Although there is no need for detailed interpretation, it seems that some specific aspects must be learned during emergency medicine residency. In the viewpoint of the authors of this paper, AIS diagnosis is the most important in this regard, which can be helpful in terms of reducing mortality and morbidity of the affected patients. AIS treatment outcome is highly dependent on the start point, and every 1-minute delay leads to the destruction of a considerable number of neurons. It is more important because based on recent researches, some of wake up stroke and unknown time stroke cases may benefit from thrombolytic therapy (2, 7, 8). MRI-guided thrombolytic therapy in AIS patients with unknown time of onset has become a topic of ongoing researches. It is likely that in those patients who have an ischemic lesion that was visible on DWI stereotypes of brain MRI, which is not accompanied by correlated parenchymal hyperintensity on FLAIR, the stroke had occurred approximately within the previous 4.5 hours (2, 9).

The importance of the subject from one side and the length of time required to perform a standard brain MRI lead to developing new ideas such as what was introduced by Kazmierczak et al. as “ultrafast brain MRI” that they assessed its usefulness in dealing with acute neurological emergencies (10).

Besides what has mentioned above, transient ischemic attach (TIA) triage in ED, its usefulness for assessing traumatic brain injury in young children, reducing the number of CT scan use in the pediatric emergency department are among other possible brain MRI applications in EDs (11-14).

Despite numerous brain MRI utilization in ED, certainly, imaging alone is not enough, and it is necessary to have a correct interpretation by a physician who has sufficient skills in this regard. So, there is a need for early reports of such imaging via telemedicine or available physicians in the emergency department (ED) (15). Traditionally, radiologists were considered the only qualified physicians in this regard, but quick and easy access to the radiologist is still not possible in most EDs (4). Therefore, it seems that the EMPs should be trained in brain MRI interpretation. Fortunately, this issue has already been considered, and brain MRI interpretation is included in the curriculum of emergency medicine residency in some universities.

4.1. Limitations

Unfortunately, due to some problems that occurred during the study process, it was not possible that all imagings should be interpreted by all three groups of physicians. Also, in this study, only the presence or absence of AIS evidence was considered a contradiction and so many details could be considered in further studies.

4.2. Conclusion

The findings of the current study revealed that the accuracy of brain MRI interpretation performed by the EMPs, compared with both neurologist and radiologist was proper. We suggest that, along with the expansion of the fields of using brain MRI in the EDs, the skills of EMPs should also be upgraded, and this important issue should be considered by related policymakers.