Interobserver and intraobserver variability in the calculation of the lipid-laden macrophage index: Implications for its use in the evaluation of aspiration in children

The lipid-laden macrophage index (LLMI) is a semiquantitative test used to evaluate aspiration in children. We assessed the reliability and reproducibility of LLMI by calculating interobserver and intraobserver variability among pathologists, with and without expertise in cytopathology. Forty-nine bronchoalveolar washes/lavages were blindly reviewed by four reviewers and assigned an LLMI. Three pathologists (two cytopathologists, one pathology fellow) reviewed slides twice and one cytotechnologist reviewed them once. Intraclass correlation coefficient (ICC) with 95% confidence interval (C.I.) was used to measure overall intraobserver and interobserver agreement. Interobserver agreement was also calculated separately for each pair of reviewers. ICC values did not indicate an acceptable level of interobserver agreement among pathologists, with (ICC = 0.67, 95% C.I.: 0.56-0.77) and without (ICC = 0.77, 95% C.I.: 0.61-0.84) the cytotechnologist included in the analysis. An ICC of 0.84 (95% C.I.: 0.78-0.89) indicated an acceptable level of intraobserver agreement among pathologists. When calculated separately for each pair of reviewers, all but two ICC values for interobserver agreement were less than 0.75 (the minimally acceptable value for a reliable clinical measurement), and the lower confidence limit of each of the 95% C.I. was far below the 0.75 cutoff. Using Lin's coefficient, intraobserver variability was only acceptable for two pathologists. Our study highlights the lack of precision and subjectivity of the LLMI, as well as the significant inter and intraobserver bias that may occur among experienced and inexperienced pathologists, and cytotechnologists. Clinicians and cytopathologists alike should be mindful of this potential pitfall and interpret LLMI scores with caution.