Risk factors and prevention of defective results in biomedical research

PROJECT SUMMARY Finding and implementing evidence-based and effective practices to avoid defects of rigor and reproducibility remain substantial challenges in biomedical research. Notably, the interaction, propagation, and impact of research errors have not been studied well (e.g., importation of coding errors, pre-clinical research defects misguiding clinical research). The prevention of non-reproducible and uninformative results is of paramount interest to every researcher. Improvement efforts should consider that the vast majority of defects occur in well-intended research. The long-term goal of this project is to identify sources and risks of research defects, their implications on the validity and usefulness of results, and advance effective defect prevention practices. We plan a scientifically rigorous collection of research defects and develop the Prevent Error in Research Management (PERM) knowledge base and corresponding ontology. Starting from our already collected, sizeable amount of structured information on defects, quality improvement expertise, and available resources, we are well-positioned to advance the understanding of sources and prevention of non-reproducibility and uninformative results in biomedical research. Several automated curation methods will be developed to facilitate the long-term sustainability of the knowledge base using programmed searches and natural language processing technologies. Three specific aims will guide this project: (1) to identify and catalog common types of research defects, the contexts in which they occur, and their root causes, and on this basis, to develop a sustainable, curated knowledge base of biomedical research defects, and explore potential preventive actions; (2) to elucidate and model the relationship between research defects and consequential societal impact, including changes in health care, public health and progress of a scientific field; and (3) to validate our collection of research defects, assess researchers' understanding of the terms and axioms used in the ontology, and development of new research defect prevention methods. Our Prevent Error in Research Management (PERM) knowledge will be publicly available from the first year of the proposed project to provide access and develop improvements to support the research community. This study is innovative by creating methods and pipelines for the collection of research defects and by leading to changes in the concepts, level of understanding, laboratory methods, decision-support technologies, preventive actions, and quality measures that can advance quality control in biomedical research. The proposed project is significant as it should develop and disseminate actionable knowledge regarding the conduct of rigorous pre-clinical and clinical research. By the end of the grant period, we expect to have developed a greatly improved detection of the most consequential risks to quality research and enabled researchers and research leaders to increase value and trustworthiness.