Interobserver agreement for colposcopy quality control using digitized colposcopic images during the ALTS trial

Objective. To estimate interobserver agreement among colposcopy quality control reviewers viewing digitized cervical images during the Atypical Squamous Cells of Undetermined Significance/Low-Grade Squamous Intraepithelial Lesion Triage Study (ALTS). Materials and Methods. Three colposcopy quality control reviewers independently examined modem-transferred digitized colposcopic images from subjects examined at four clinical centers. Reviewers indicated colposcopic impression, Reid colposcopic index scores, lesion size, and the technical quality of the image. Rates of agreement were evaluated using the κ statistic and McNemar and Bowker tests of symmetry. Results. Regarding colposcopic impressions, the average weighted κ for pairs of initial reviewers was 0.36 (95% confidence interval, 0.33-0.39). κ scores with respect to Reid colposcopic index, cervical image quality, and lesion size ranged from 0.23 to 0.28, 0.18 to 0.27, and 0.33 to 0.42, respectively. Conclusions. Fair rates of agreement and poor to fair κ scores among ALTS colposcopy quality control reviewers were noted for colposcopic impression, Reid colposcopic index scores, image quality, and lesion size. Great latitude exists in the interpretation of digitized cervical images. Poor image quality partially may explain these suboptimal results.