Evaluation, Agreement and Interpretation of Independent Radiologist Assessments - ROC Analysis

ROC curves are useful for comparing two independent observations and the outcomes of their assessments for the same disease in a given study.  In general, the test with the higher AUC may be considered better and is an effective way to summarize the overall diagnostic accuracy of the test. AUC scores are convenient to compare multiple classifiers. Nonetheless, it is also important to check the actual curves especially when evaluating the final model. In the ROC curve analysis, the choice of the optimal cutoff value depends both on probabilistic and clinical considerations. From a probabilistic standpoint, one can use the coordinates of the ROC curve to identify the cutoff that maximizes the discrimination between true-positive rate and false-positive rate(1).

The name "Receiver Operating Characteristic" came from. ROC analysis is part of a field called "Signal Detection Theory" (3) developed during World War II for the analysis of radar images. It was not until the 1970's that signal detection theory(4) was recognized as useful for interpreting medical test results.