SS 1805 - Clinical decision support and structured reporting
SS 1805 - Clinical decision support and structured reportingSunday, March 5, 10:30 - 12:00 Room: M 4 Session Type: Scientific Session Topics: Professional Issues, Computer Applications Moderators: M. Fatehi (Tehran/IR), N. Pyatigorskaya (Paris/FR) Add session to my schedule In your schedule (remove)
Integrating clinical decision support for pulmonary embolism in the emergency department: a pilot study of feasibility and provider perspective
Purpose: In the context of a commercial EMR, we implemented a clinical decision support (CDS) intervention for CTA for pulmonary embolism (CTA-PE) in the emergency department (ED) and quantified its impact on ordering practices.
Methods and Materials: In a survey of 231 radiologists and clinicians in the ED, 78% identified CTA-PE as “overutilised”. We developed an algorithm that combined established risk scores and local practice patterns to risk stratify PE workup. We integrated a mandated questionnaire for every CTA-PE study requested within the EMR. If answers were concordant with the clinical pathway, the study order was placed; if answers were discordant, alternative scenarios were recommended.
Results: Among the 853 studies conducted, 8.2% were positive for PE. The algorithm was highly accurate, with 10.4% and 10.0% positivity among studies that were recommended to proceed with CT or pursue D-dimer, respectively. In cases where cancellation was recommended, 2.6% of studies were positive, and one of these showed a clinically significant PE. Among the 879 studies requested, 479 (55%) were recommended to change their order: 6 (1.3%) studies were subsequently cancelled; 13 (2.7%) changed to a D-dimer, 460 (51%) proceeded with CTA despite the recommendation. Among providers who ordered ≥10 studies, concordance with the CDS recommendation ranged from 12 - 68% (mean 45%) with little (0.10) correlation between compliance and positivity rates.
Conclusion: While the CDS algorithm for PE was accurate, it had only a modest impact on ordering practices, in part due to substantial heterogeneity in physician adherence to the CDS intervention.
Purpose: The aim of this study is to present the results of the Italian survey on dematerialisation of informed consent in radiology (DIC).
Methods and Materials: Two radiologists created an online survey using Survey Monkey. The survey consisted of 15 multiple-choice questions. Members of Italian Society of Medical Radiology (SIRM) were given 1 week to perform the survey.
Results: A total of 1791 radiologists, 18% of all SIRM members, participated. In Italy the radiological informed consent is used for CT and MRI (93.33%), interventional procedures (70.95%), ultrasound examinations with i.v. contrast media (42.40%), and for the exclusion of pregnancy in young women before x-ray examinations (70.22%). Seventy-two percent of radiologists evaluate the appropriateness of the diagnostic procedures before deciding to accept or reject the request. Ninety-five percent (n=1684) of responders have a positive opinion on DIC, while only 5% (n=93) have a negative one. The advantages are: storage and conservation of DIC is safer according to majority of responders (94.54%), its recovery is easier and faster (96.53%) in case of medico-legal disputes respectively and the reduction of paper used leads to reduced costs (90.67%). The disadvantages are: implementation of DIC is complex, in particular to obtain the preliminary approval for the utilization of advanced digital sign from each patients (51.76%) and to provide required dedicated area inside radiological unit (63.97%).
Conclusion: The majority of Italian radiologists are favourable on DIC. However, they have concerns that the implementation of DIC could be complex.
How to prescribe imaging tests and to make clinical decisions? - effectiveness of a virtual classroom for undergraduate students in radiology
Purpose: To develop a virtual learning environment for teaching to medical students the following skills: prescribing imaging tests and making decisions based on their results. To assess its feasibility, effectiveness and students´ satisfaction.
Methods and Materials: An ad-hoc virtual classroom was implemented in the Moodle platform consisting on videolectures, practical exercises and e-learning resources. Sixth-grade students were recruited and gave informed consent prior to randomization into the experimental or the control group. Research Ethics Committee approved this project. Performance (ability to prescribe imaging tests and to take diagnostic-therapeutic decisions based on their results) was evaluated with objective structured clinical exams. Weighted scores from 0 to 100 were compared between experimental and control groups with the non-parametric Wilcoxon test (SAS System for Windows, version 9.2). Satisfaction was assessed with an eleven-items, five-level Likert scale.
Results: Twenty-six students were included (8M/18F, median age 23y) and randomized (13 experimental/13 control group). 76.9% students in the experimental group completed the practical assignments. Mean self-reported time devoted to the virtual course 7.17 hours. Satisfaction: median satisfaction was high or very high in 10 out of 11 items. Efficacy: students in the experimental group performed better at prescribing imaging tests (mean score 55.9% vs 37.1%, p=0.0042) and at making clinical decisions (mean score 40.0% vs 29.5%, p=0.0403).
Conclusion: The abilities to prescribe imaging tests and to make clinical decisions with their results can be taught effectively and satisfactorily to undergraduate medical students through a virtual learning environment. Its use will be generalized to the remaining students.
Structured reporting: using the voice of the customer method to settle an ongoing debate about the future of radiology reporting
Purpose: The presentation and clarity of a radiology report do not always meet the expectations of referring physicians. Meanwhile a debate about the future of radiological reporting is taking place. The purpose of this study is to assess the perception, preferences and expectations of recipients of radiology reports in terms of style and content.
Methods and Materials: A survey was conducted among general practitioner (GP) and hospital based physicians (HP) in north-western Switzerland. The questionnaire consisted of a demographic section, a part addressing current satisfaction and a section addressing expectations in content and structure. The participants were presented with four layouts of radiology reports (text, structured text, tables, images) and asked to rate each in comprehensibility and efficiency (range 1-10; 10 highest score).
Results: 434 participants (121GPs, 313HPs, 4 different hospitals) with 114 residents and 320 board-certified physicians completed the survey. Both GP and HP were equally satisfied with radiology reports with a mean of 7.2. Regarding layout preferences, structured text (mean 5.8-7.9;) and images (mean 6.6-8) rated highest in terms of readability, time savings and helpfulness in the communication with patients, when compared with tables (mean 4.5-5.4) and unstructured text (mean 3.3-4.6). Of all participants, 81% stated a report should allow for fast and efficient reading.
Conclusion: The voice of the customer approach offers valuable feedback and an indisputable argument in favor of structured reporting. Radiology has to facilitate easy communication while delivering comprehensive information. The form of this communication should be tailored to the referring physicians’ preferences.
Guideline based query of conventional narrative "free text" radiological reports and structured reports: a solution for objective comparison
Purpose: Data on how well structured reports (SR) and conventional narrative reports (cFTR) comply with clinically relevant findings are scarce. A feasibility study of a text mining based scoring algorithm was performed to provide an objective intra-individual comparison of SR and cFTR by means of guideline-based key terms.
Methods and Materials: 25 suspected stroke patients with consecutive cMRI stroke protocol were re-assessed by two independent,blinded readers (experience:>2 [unxR];>6yrs[exR]). SRs were generated using an online template-tool (www.smart-radiology.com) with additional free text (uxR:20/;exR:22/25). Corresponding pre-existing cFTR were retrieved from local database. A query-vector of key terms based on imaging recommendations for acute stroke and transient ischemic attack patients by the ASNR and the ACR was defined. Following automatic text retrieval, SR and cFTR were compared with guideline-query using term frequency-inverse document frequency based similarity index and Wilcoxon signed-rank test.
Results: All 18 (72%) cases with ischemia were identified by SR and cFTR (rhosp=1,Cohen’s Kappa=1). UnxR using SR had the highest median (0.82) and maximal (7.57) guideline similarity scores (GSS). SRs of exR had significantly (pW=0.0020) higher GSS (median:0.72,range:0-5.4), than cFTR (median:0.57,range:0-5.7). Although SRs of unxU were not corrected by senior radiologist, they had similar GSS (pW=0.50) like the reviewed cFTR,and were comparable to the SR of exR (Z=1.59,p=0.11).
Conclusion: An objective guideline-based comparison of SRs and cFTRs using term frequency is feasible and provides a scalable quality measure. In spite of additional free text, SR improved the adherence to guidelines of both senior- and junior radiologist when evaluating stroke suspected brain MRIs.
Comparison of report characteristics of un- and experienced radiologists using an online-based structured reporting tool and conventional "free text" reports
Purpose: Online reporting tools (ORT) with predefined text blocks are gaining popularity to create structured reports (SR). However, their effects on report structure and reporting process are obscure. We compared SR of un- (uxR;>2yrs) and experienced radiologists (exR;>6yrs),and conventional narrative -“free text”- reports (cFTR) in the setting of MRI stroke diagnostics.
Methods and Materials: 25 patients with cMRI were randomly sampled from a population of suspected stroke cases (n=780). They were re-assessed blindly by uxR/exR using ORT (www.smart-radiology.com) and SR were created. Parameters of interest were: correctness, reporting time (RT), word count (Wc) of SR, and pre-existing cFTR. Factors like free text in SR (uxR|exR:20|22), extra MRI-sequence (13/25) and subjective case complexity (SCC; 5-point-Likert scale) were also considered. Inter-rater reliability, Spearman’s rank correlation, Wilcoxon signed-rank test, and linear mixed models with outcomes RT, Wc including aforementioned factors were applied.
Results: There was perfect agreement (kappa=1) of ischemia (18/25) in all SR and cFTR. SCC was similar (rsp=0.79,p<0.001) between users showing substantial association with RT (ruxR=0.70;rexR=0.67,pboth<0.01). ExR was significantly (p<0.01) faster (median=6.75min;range:2-12min) than uxR (8min;range:2-13.5min) using SR. SR of exR were significantly shorter (Wc) than SR of uxR (p<0.001) and cFTR (p<0.01) corrected by senior radiologists. SR of uxR had comparable Wc to supervised cFTR (pfindings=0.37;pimpresssion=0.70). SCC had the most decisive effect on Wc by both SR and cFTR when adjusted for aforementioned factors.
Conclusion: ORT enables a standardized, valid and time-effective SR creation, while improving RT and Wc of both senior- and junior radiologist when evaluating stroke suspected brain MRIs.
Purpose: To measure recall by clinicians immediately after reading a structured compared to an unstructured radiology report.
Methods and Materials: Institutional review board approval was obtained. A structured radiology report is a uniform report template that uses headings and subheadings followed by standardized statements to create uniformity and improve communication with referring physicians. A structured radiology report differs from the "freetext" radiology report, which lacks headings or a standardised format. Four hypothetical radiology reports were devised based on a review of common cross sectional imaging studies. Two structured and two unstructured reports followed immediately by a 7-response cloud-based multiple-choice questionnaire were used to survey physicians at multiple institutions whose demographics and training data were collected. The proportion of all correct answers, correct critical findings, sensitivity and specificity were recorded. A paired t-test was used to compare results.
Results: 148 physicians completed the survey. The frequency of incorrect diagnoses was 6.2% for structured reports compared to 18.2% for unstructured reports (p<0.001). Critical diagnoses were missed in 17.4 % of structured reports compared to 35% of unstructured reports (p <0.001). Average physician sensitivity and specificity was greater for structured reports (64.2% and 95.5%, respectively) compared to unstructured reports (59% and 83%, respectively) (p≤0.001).
Conclusion: Physician recall of unstructured radiology reports is significantly inferior to recall of structured reports immediately after reading a radiology report. Structured reports are a more accurate means of conveying the salient findings of a radiology study.
Eye-tracking analysis of interpretation characteristics in detection of alimentary tract lesions on body CT
Purpose: To clarify interpretation characteristics of radiologists in detecting alimentary tract lesions on body CT using eye-tracking technique.
Methods and Materials: 3 radiologists (1, 2, 12 years of experience) and 3 non-radiologists (2 residents and 1 medical student) were asked to detect lesions suspected as malignant tumours within 3 minutes without any clinical information on thoracic and abdominal CT obtained from 10 patients with histologically proven malignant tumours in the alimentary tract. Gaze path was recorded by eye-tracking technique during interpretation to calculate gaze time at each voxel (GTV) on CT. The lesion detection rate (LDR) and interpretation characteristics in eye movement were statistically analysed.
Results: GTV fitted to the Burr distribution (P<0.001); however, the observed GTV was relatively longer than estimated GTV from the Burr distribution above the 95th percentile. Therefore, the intentional gaze was defined where GTV was longer than 95th percentile GTV (GTV95%) estimated from the Burr distribution in this study. The mean value of LDR, GTV95%, and the percentage of gazed area in patient’s body were 46.7%, 0.18 s, 25.0% in radiologists, 0%, 0.36 s, 16.1% in non-radiologists (P<0.001, P=0.002, P<0.001), respectively. The intentionally gazed lesions were significantly fewer than unintentionally gazed lesions in undetected lesions by radiologists (P=0.049).
Conclusion: Radiologists can detect more lesions than non-radiologists due to effective gazes and the diagnostic ability for intentionally gazed lesions. Undetected lesions were located more in unintentionally gazed area; therefore, making radiologists pay attention to the area utilizing gaze path information may improve their diagnostic performance.
Purpose: The aim of the study is to analise the results of the Radiological Counseling (RC) service introduced in C.D.I. Diagnostic Imaging Department. By using this service the patients can meet the radiologist to get information about radiological exams and medical report, and to discuss diagnostic imaging issues.
Methods and Materials: The RC service started in September 2015 and the patients were informed about RC by a notice in the radiological report and in the institutional website. The service outcome has been investigated by a survey on the first 50 patients that used the RC: afterwards the meeting with the radiologist, they answered to a questionary about their age and sex, the discussion satisfaction, as well as the preferred comunication modality in normal or pathologic findings.
Results: The survey shown complete satisfaction of the meeting with the radiologist, with a score of 4,5 of 5. The 60% of the cases are under 60 years-old. The majority of the patients answer that they prefer to receive the radiological report by a meeting with the radiologist: 50% if there are normal findings and 80% if pathological.
Conclusion: The RC is a useful service for the patient and can improve the relationship between the patient and the radiologist, who gains a new clinical role. We are satisfied about this results that will help us to improve this service to raise our patient’s trust in our institution.
Frequency & analysis of non-clinical errors made in radiology reports using the national integrated medical imaging system (NIMIS) voice recognition dictation software
Purpose: Voice recognition (VR) dictation of radiology reports has become the mainstay of reporting in many institutions worldwide. Despite benefit, such software is not without limitations, and transcription errors have been widely report. We evaluated the frequency and nature of transcription error using VR software.
Methods and Materials: Retrospective audit of 378 finalised radiology reports. Errors were counted and categorised by significance, error type and sub-type. Data regarding imaging modality, report length and dictation time was collected.
Results: 67 (17.72%) reports contained ≥1 errors, with 7 (1.85%) containing ‘significant’ and 9 (2.38%) containing ‘very significant’ errors. A total of 90 errors were identified from the 378 reports analysed, with 74 (82.22%) classified as ‘insignificant’, 7 (7.78%) as ‘significant’ and 9 (10%) as ‘very significant’. 68 (75.56%) errors were ‘spelling and grammar’, 20 (22.22%) ‘missense’ and 2 (2.22%) ‘nonsense’. ‘Punctuation’ error was most common sub-type, accounting for 27 errors (30%). On average, complex imaging modalities had higher error rates, with computed tomography containing 0.57 errors per report compared to plain film with 0.08. Longer reports had a higher error rate, with reports >25 sentences containing an average of 1.23 errors per report compared to 0-5 sentences containing 0.09.
Conclusion: These findings highlight the limitations of VR dictation software. While most error was deemed insignificant, there were occurrences of error with potential to alter report interpretation and patient management. Longer reports and reports on more complex imaging had higher error rates and this should be taken into account by the reporting radiologist.
Evaluation of a patient safety tool: the preferred reading of the radiology report - has it been useful?
Purpose: To submit a software tool that allows us to warn about the preferred reading of those reports showing unexpected radiological findings, emergency and incidental findings that imply a change in clinical management of patients. To analyse its use and effectiveness in a radiological hospital service since its implementation.
Methods and Materials: A computer tool of radiological alert reports was implemented in 2012. We review its use, analysing its distribution by modality and requesting service. We analyse the most frequent causes that have produced these warnings and the implications for the management of patients. We also evaluated the problems found in its use.
Results: We review the alert filed in the 574,006 reports performed from January 2012 to September 2016 in our department. We send alerts in 0.13% of the reports, (CT 35,5% , MR 19,8%,US 15,1% and mammography 0,25% ) , 94.2% of them were in out patients. The discovery of unsuspected tumour pathology is the main cause of that alert (45.2%). We found some problems, like the communication with primary care.
Conclusion: The radiological alert of preferred reading of a report is part of the radiology daily activity, both for, transmits unexpected or incidental findings and to enable rapid action on the patient. It is an important element for patient safety. It should be a useful, simple, and effective tool known by all physicians involved. 0.13% of our reports were subsidiaries of prioritising reading mainly CT and MR.