part of a clinical study. The FDA also presents several methodologies that have been used in the past to measure uncertainties in ROC estimates; however, it is noted that MRMC methodology is the only approach that accounts for reader and case variability.2 As a result, through November 2002, all successful submissions to the FDA of a system for digital mammography utilized the MRMC ROC paradigm.
1. Beiden SV, Wagner RF, Doi K, Nishikawa RM, Freedman M, Lo SC, Xu XW. 2002. Independent versus sequential reading in ROC studies of computer-assist modalities: analysis of components of variance. Acad Radiol 9(9):1036-1043.
2. Center for Devices and Radiological Health. 2001. Premarket Applications for Digital Mammography Systems; Final Guidance for Industry and FDA. Washington, DC: Food and Drug Administration.
3. Giger M. 2003. Computer Assisted Diagnosis. Workshop on New Technologies for the Early Detection and Diagnosis of Breast Cancer. Washington, DC: Institute of Medicine of the National Academies.
4. Metz CE. 1978. Basic principles of ROC analysis. Semin Nucl Med 8(4):283-298.
5. Wagner RF, Beiden SV. 2003. Independent versus sequential reading in ROC studies of computer-assist modalities: analysis of components of variance. Acad Radiol 10(2):211-212; author reply 212.
6. Wagner RF, Beiden SV, Campbell G, Metz CE, Sacks WM. 2002. Assessment of medical imaging and computer-assist systems: lessons from recent experience. Acad Radiol 9(11):1264-1277.
7. Wagner R. 2003. CDRH Research Perspectives. Fourth National Forum on Biomedical Imaging in Oncology. Bethesda, MD: National Cancer Institute.