This study observed a significant decline-rebound pattern in scan-level pulmonary nodule detection after discontinuation of routine AI assistance. Among 980 chest CT examinations reread by the same group of radiologists, the proportion of scans with at least one reported pulmonary nodule decreased from 37.8% in phase I to 26.5% in phase II and then increased to 43.2% in phase III. The GEE analysis showed lower adjusted odds of detection in phase II than in phase I, followed by higher odds in phase III than in both earlier phases. Because no external lesion-level reference standard was available, these findings should be interpreted as phase-associated changes in scan-level detection patterns rather than definitive changes in lesion-level sensitivity or specificity.
The phase-related difference was mainly driven by nodules with a maximum reported diameter of ≤5 mm. Consistent with this pattern, the median maximum reported nodule diameter increased from 4 mm in phase I to 5 mm in phase II and then returned to 4 mm in phase III. This pattern suggests that the short-term decline after AI withdrawal was concentrated in low-salience targets, whereas larger nodules were less affected. However, because no lesion-level reference standard was available, this finding should be interpreted cautiously: It may reflect reduced reporting of subtle nodules, altered reporting thresholds, or both, rather than directly measured false-negative lesions. Prior computer-aided detection (CAD)/AI studies have similarly shown that AI support is particularly helpful for subtle and very small nodules (
9).
Several factors may explain this short-term decline after AI withdrawal. Prolonged exposure to AI prompts may induce automation bias or imperfect trust calibration, reducing the intensity of independent verification when AI is abruptly removed (
10,
11). Artificial intelligence use may also alter visual search behavior, shifting readers toward more cue-driven or confirmatory reading, while the absence of AI may transiently raise the reporting threshold for subtle findings. At the same time, non-AI-related explanations must also be considered. phase I consisted of routine clinical reports, whereas phases II and III were structured rereads performed without clinical context or prior imaging, and these setup differences alone may have affected both perception and reporting behavior (
7). Hawthorne effects, motivational differences, and practice effects during the 3-month AI-free washout period may also have contributed (
8,
12,
13). Thus, the present findings support an association between AI withdrawal and short-term performance shifts, but they do not establish a single causal mechanism.
The rebound in phase III is also noteworthy. It may reflect a combination of implicit learning, perceptual recalibration, and threshold adjustment, although these interpretations remain speculative (
9,
12-
14). Clinically, this finding suggests a potential vulnerability window during AI downtime, software migration, hardware replacement, or workflow transitions, when subtle nodules may be less likely to be reported. Although nodules < 6 mm usually have low malignant potential in low-risk adults, stable detection of such findings may still be important in high-risk populations, screening settings, and longitudinal follow-up (
15).
Several limitations should be acknowledged. First, phase I data were derived from routine clinical reports, whereas phases II and III were research rereads performed under controlled conditions without access to the original reports, prior imaging, or additional clinical context; these setup differences may have influenced both perception and reporting thresholds (
7). Second, the same case set was re-evaluated in phases II and III; despite the 3-month interval and re-randomization before phase III, residual familiarity, Hawthorne effects, and practice effects cannot be fully excluded (
8). Third, this was a single-center study involving only three senior general radiologists, one commercial AI system, and a single local workflow, which may limit generalizability to other institutions, reader populations, AI tools, workflows, or screening settings. In particular, these findings may not be directly generalizable to residents or early-career readers exposed to AI from the beginning of training, whose adaptation to AI withdrawal may differ. Fourth, no external lesion-level reference standard was established; accordingly, the primary endpoint was scan-level reporting of at least one pulmonary nodule rather than lesion-level sensitivity or specificity. Fifth, a small number of patients contributed more than one scan, although sensitivity analyses supported the robustness of the main findings. Finally, the subgroup analyses were exploratory in nature and should therefore be interpreted with caution, even though correction for multiple comparisons was applied.
In conclusion, discontinuation of routine AI assistance was associated with a short-term decrease in scan-level pulmonary nodule detection, particularly for small nodules, followed by recovery after an AI-free washout period. These findings suggest a potential vulnerability window during AI withdrawal or workflow transitions and highlight the need for resilient implementation strategies, reader monitoring, and further prospective studies incorporating lesion-level reference standards.