Colposcopy quality control: Establishing colposcopy criterion standards for the national cancer institute ALTS trial using cervigrams

Objectives. The colposcopy quality control (QC) group of the Atypical Squamous Cells of Undetermined Significance-Low-Grade Squamous Intraepithelial Lesions (ASCUS/ LSIL) Triage Study (ALTS) monitors colposcopists at four clinical centers by direct observation and indirectly by assessment of digitized computer colposcopic images transferred nightly by modem. The purpose of this preclinical study was to determine the agreement among colposcopy QC monitors' colposcopic impressions and biopsy site placement through evaluation of cervical photographic images. Materials and Methods. A nonparticipant selected 100 Cervigram slides representing four pathologically confirmed categories: cervical intraepi-thelial neoplasia 1(CIN), CIN2 and CIN3, normal, and cancer. Four colposcopy QC monitors and two certified Cervigram evaluators independently interpreted the images for colposcopic impression on two separate occasions. Measures of agreement were calculated for intraobserver, pairwise inter-observer, and observer versus histological diagnoses. Participants also indicated the most severe biopsy site by marking an X on a 4 × 6-inch color print duplication. Mean distances between biopsy site annotations for each pair of evaluators were calculated. Results. The ALTS observer agreement with histology ranged from 75.3% (κ = .66) to 47.4% (κ = .36) for the first evaluation and 71.1 % (κ = .63) to 50.5% (κ = .38) for the second evaluation. ALTS interobserver agreement varied between 71% and 57% for the first interpretation and 76% and 52% for the second evaluation. ALTS intraobserver agreement varied from 86% (κ = .86) to 68.0% (κ = .60). The mean distances between biopsy site annotations for each pair of ALTS observers varied from 0.99 ± 0.93 mm to 1.44 ± 1.19 mm. Conclusions. The ALTS colposcopy QC monitors achieved poor to good observer agreement with histology, fair to excellent interobserver agreement, and good to excellent intraobserver agreement in assessing the severity of cervical images. These levels of agreement are similar to many reported by pathologists for cervical cytology and histology interpretations. Monitors also demonstrated a very narrow variation of distance (< 2 mm) for ideal biopsy site placement.