TY - JOUR
T1 - Impact of risk of generalizability biases in adult obesity interventions
T2 - A meta-epidemiological review and meta-analysis
AU - Beets, Michael W.
AU - von Klinggraeff, Lauren
AU - Burkart, Sarah
AU - Jones, Alexis
AU - Ioannidis, John P.A.
AU - Weaver, R. Glenn
AU - Okely, Anthony D.
AU - Lubans, David
AU - van Sluijs, Esther
AU - Jago, Russell
AU - Turner-McGrievy, Gabrielle
AU - Thrasher, James
AU - Li, Xiaoming
N1 - Publisher Copyright:
© 2021 The Authors. Obesity Reviews published by John Wiley & Sons Ltd on behalf of World Obesity Federation.
PY - 2022/2
Y1 - 2022/2
N2 - Biases introduced in early-stage studies can lead to inflated early discoveries. The risk of generalizability biases (RGBs) identifies key features of feasibility studies that, when present, lead to reduced impact in a larger trial. This meta-study examined the influence of RGBs in adult obesity interventions. Behavioral interventions with a published feasibility study and a larger scale trial of the same intervention (e.g., pairs) were identified. Each pair was coded for the presence of RGBs. Quantitative outcomes were extracted. Multilevel meta-regression models were used to examine the impact of RGBs on the difference in the effect size (ES, standardized mean difference) from pilot to larger scale trial. A total of 114 pairs, representing 230 studies, were identified. Overall, 75% of the pairs had at least one RGB present. The four most prevalent RGBs were duration (33%), delivery agent (30%), implementation support (23%), and target audience (22%) bias. The largest reductions in the ES were observed in pairs where an RGB was present in the pilot and removed in the larger scale trial (average reduction ES −0.41, range −1.06 to 0.01), compared with pairs without an RGB (average reduction ES −0.15, range −0.18 to −0.14). Eliminating RGBs during early-stage testing may result in improved evidence.
AB - Biases introduced in early-stage studies can lead to inflated early discoveries. The risk of generalizability biases (RGBs) identifies key features of feasibility studies that, when present, lead to reduced impact in a larger trial. This meta-study examined the influence of RGBs in adult obesity interventions. Behavioral interventions with a published feasibility study and a larger scale trial of the same intervention (e.g., pairs) were identified. Each pair was coded for the presence of RGBs. Quantitative outcomes were extracted. Multilevel meta-regression models were used to examine the impact of RGBs on the difference in the effect size (ES, standardized mean difference) from pilot to larger scale trial. A total of 114 pairs, representing 230 studies, were identified. Overall, 75% of the pairs had at least one RGB present. The four most prevalent RGBs were duration (33%), delivery agent (30%), implementation support (23%), and target audience (22%) bias. The largest reductions in the ES were observed in pairs where an RGB was present in the pilot and removed in the larger scale trial (average reduction ES −0.41, range −1.06 to 0.01), compared with pairs without an RGB (average reduction ES −0.15, range −0.18 to −0.14). Eliminating RGBs during early-stage testing may result in improved evidence.
UR - http://www.scopus.com/inward/record.url?scp=85119302727&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85119302727&partnerID=8YFLogxK
U2 - 10.1111/obr.13369
DO - 10.1111/obr.13369
M3 - Review article
C2 - 34779122
AN - SCOPUS:85119302727
SN - 1467-7881
VL - 23
JO - Obesity Reviews
JF - Obesity Reviews
IS - 2
M1 - e13369
ER -