Accuracy validation parameters for diffusion weighted imaging infarct volume visual assessment, with RAPID as the reference standard, and inter-rater agreement, for the cut-off points as target conditions: diffusion weighted imaging infarct volume <21 mL,<31 mL,<51 mL, and <71 mL
| Sensitivity (%) | Specificity (%) | Accuracy (%) | Inter-rater agreement (Fleiss kappa) | |
| Target condition: DWI infarct volume <21 mL according to reference standard RAPID | ||||
| All raters (n=18) | 61% (32–83) | 93% (75–98) | 83% (66–94) | 0.46 (0.31–0.64) |
| Vascular neurologists (n=6) | 48% (21–77) | 96% (80–99) | 81% (64–93) | 0.53 (0.34–0.77) |
| Interventional neuroradiologists (n=6) | 53% (26–76) | 95% (78–99) | 82% (65–93) | 0.40 (0.23–0.64) |
| Diagnostic neuroradiologists (n=6) | 80% (48–93) | 87% (67–96) | 85% (68–95) | 0.57 (0.42–0.76) |
| Senior raters (n=9) | 69% (37–89) | 92% (74–98) | 85% (68–95) | 0.53 (0.34–0.73) |
| Junior raters (n=9) | 52% (26–77) | 93% (76–98) | 81% (63–92) | 0.38 (0.22–0.63) |
| Physicians working in a comprehensive stroke center (n=14) | 58% (30–81) | 93% (75–98) | 82% (65–93) | 0.43 (0.27–0.63) |
| Physicians working in a referring stroke center (n=4) | 70% (37–91) | 92% (73–99%) | 85% (68–95) | 0.54 (0.32–0.75) |
| Target condition: DWI infarct volume <31 mL according to reference standard RAPID | ||||
| All raters (n=18) | 82% (54–94) | 90% (69–97) | 87% (71–95) | 0.59 (0.46–0.73) |
| Vascular neurologists (n=6) | 79% (51–93) | 94% (75–99) | 89% (73–96) | 0.60 (0.40–0.77) |
| Interventional neuroradiologists (n=6) | 74% (45–90) | 94% (75–99) | 86% (70–95) | 0.56 (0.35–0.75) |
| Diagnostic neuroradiologists (n=6) | 93% (65–99) | 81% (59–93) | 86% (69–95) | 0.61 (0.43–0.78) |
| Senior raters (n=9) | 83% (55–94) | 89% (67–98) | 87% (70–96) | 0.60 (0.46–0.75) |
| Junior raters (n=9) | 81% (52–93) | 91% (71–96) | 87% (71–95) | 0.54 (0.40–0.74) |
| Physicians working in a comprehensive stroke center (n=14) | 79% (50–92) | 90% (70–97) | 86% (70–95) | 0.55 (0.41–0.71) |
| Physicians working in a referring stroke center (n=4) | 94% (65–100) | 88% (66–96) | 87% (75–97) | 0.73 (0.53–0.91) |
| Target condition: DWI infarct volume <51 mL according to reference standard RAPID | ||||
| All raters (n=18) | 89% (65–98) | 74% (49–90) | 82% (65–93) | 0.59 (0.46–0.71) |
| Vascular neurologists (n=6) | 88% (64–98) | 78% (51–93) | 83% (66–94) | 0.63 (0.48–0.79) |
| Interventional neuroradiologists (n=6) | 86% (62–97) | 84% (58–96) | 85% (69–95) | 0.70 (0.55–0.83) |
| Diagnostic neuroradiologists (n=6) | 92% (69–99) | 61% (37–80) | 78% (60–90) | 0.45 (0.27–0.66) |
| Senior raters (n=9) | 89% (66–99) | 76% (50–92) | 83% (66–94) | 0.62 (0.47–0.77) |
| Junior raters (n=9) | 88% (65–98) | 73% (48–87) | 81% (64–92) | 0.52 (0.39–0.67) |
| Physicians working in a comprehensive stroke center (n=14) | 87% (63–98) | 76% (50–90) | 82% (65–93) | 0.59 (0.45–0.74) |
| Physicians working in a referring stroke center (n=4) | 94% (72–100) | 70% (45–87) | 83% (66–93) | 0.57 (0.38–0.74) |
| Target condition: DWI infarct volume <71 mL according to reference standard RAPID | ||||
| All raters (n=18) | 96% (77–100) | 47% (24–70) | 76% (58–89) | 0.45 (0.26–0.63) |
| Vascular neurologists (n=6) | 96% (76–100) | 42% (18–70) | 74% (56–88) | 0.49 (0.26–0.76) |
| Interventional neuroradiologists (n=6) | 94% (73–99) | 51% (25–76) | 77% (59–90) | 0.47 (0.25–0.70) |
| Diagnostic neuroradiologists (n=6) | 98% (83–100) | 48% (28–65) | 78% (61–89) | 0.35 (0.25–0.50) |
| Senior raters (n=9) | 97% (78–100) | 54% (28–78) | 80% (62–91) | 0.50 (0.31–0.73) |
| Junior raters (n=9) | 95% (76–99) | 40% (20–63) | 73% (54–86) | 0.39 (0.24–0.57) |
| Physicians working in a comprehensive stroke center (n=14) | 96% (76–100) | 93% (75–98) | 47% (23–72) | 0.46 (0.27–0.70) |
| Physicians working in a referring stroke center (n=4) | 96% (81–100) | 92% (73–99) | 48% (26–65) | 0.30 (0.15–0.42) |
Values are presented as % or kappa value (95% CI).
DWI, diffusion weighted imaging.