PURPOSE: Lumbar disc herniation is associated with substantial morbidity, including low back pain, radicular leg pain (sciatica), sensory disturbance, and motor deficit. Magnetic resonance imaging (MRI) is central to confirming the diagnosis in symptomatic patients and to planning surgical or interventional management. Recent advances in artificial intelligence (AI) raise the possibility of automating aspects of image interpretation to improve consistency and reduce radiologist workload. This study evaluates a general-purpose large multimodal model, Gemini 3.1 Pro, for identifying disc herniations on sagittal lumbar spine MRI in a zero-shot setting.
METHODS: We hypothesized that, without task-specific training, the model would show high sensitivity but limited specificity, and that paired T1- and T2-weighted input (T1 + T2) would outperform T1 alone. Using SPIDER, a public multi-center dataset of sagittal T1- and T2-weighted lumbar MRI from patients with low back pain with expert level-by-level labels, we evaluated a single paired cohort of 119 cases under two input conditions on the same cases: T1-only and T1 + T2. The mid-sagittal slice was extracted from each series and a standardized prompt forced a binary classification (herniation present vs. absent). Performance was summarized as sensitivity, specificity, accuracy, precision, and F1 with 95% confidence intervals (CIs); the paired difference in overall accuracy was tested with the exact McNemar test. A supplementary evaluation was then performed using 3-slice image stacks, testing the model's ability to process 3-D images.
RESULTS: In the paired cohort (31 herniation-positive, 88 negative; prevalence 26%), T1-only input yielded sensitivity 0.58 (95% CI 0.41-0.74), specificity 0.74 (0.64-0.82), accuracy 0.70 (0.61-0.77), precision 0.44, and F1 0.50. Paired T1 + T2 input yielded sensitivity 0.77 (0.60-0.89), specificity 0.51 (0.41-0.61), accuracy 0.58 (0.49-0.67), precision 0.36, and F1 0.49. Contrary to our hypothesis, adding the T2 sequence reduced specificity and overall accuracy: T1-only was significantly more accurate than T1 + T2 in the paired comparison (exact McNemar p = 0.044), driven by a near-doubling of false positives (43 vs. 23). The false-positive burden was substantial, equivalent to 48.9 unnecessary reviews per 100 negative scans with T1 + T2.
CONCLUSION: Gemini 3.1 Pro showed only modest, input-dependent accuracy for lumbar disc herniation detection and a pronounced tendency to over-call pathology, most markedly when a T2 sequence was provided. Specificity fell well below the ≈ 80% typically required of clinical triage tools, so the model is not currently suitable as a triage or screening aid. We present this as an exploratory, proof-of-concept evaluation; it characterizes the failure modes of an off-the-shelf generalist model on a specialized radiological task and motivates future work on volumetric/multi-slice input, task-specific fine-tuning, and decision calibration.
Read the article: PubMed · Publisher (DOI)