Confidence intervals for weighted proportions

We investigate methods for the construction of confidence intervals for a proportion in a stratified two‐stage sampling design with few events occurring in a small number of large, unequal size strata. The critical aspect is the incorporation of the weighting scheme into the construction of a single overall confidence interval. With small numbers of events, the binomial based methods may be inadequate since the normal approximation is not valid. Computer simulations compare coverage probability and bias for five methods of obtaining confidence intervals for proportions by combining: (1) binomial variances; (2) confidence intervals based on the F‐distribution approximation to the cumulative binomial; (3) the binomial variance method with exact confidence limits when a zero prevalence occurs in any stratum; (4) confidence intervals based on the F‐distribution using a rescaling factor; and (5) the binomial variance method with exact confidence limits using a rescaling factor. The method that performs best in terms of coverage probability is the combination of stratum specific confidence intervals based on the F‐distribution using a rescaling factor. The methods involving the binomial variance tend to be negatively biased and the methods based on the F‐distribution tend to be positively biased. Application of these methods with data from a study of adolescent depression that employs a stratified two‐stage sampling design is consistent with these results.