Researchers at MIT and Dana-Farber Cancer Institute have leveraged artificial intelligence to investigate how medical studies share their analytical code, critical for verifying and building upon clinical prediction models. These models estimate patient outcomes, guiding medical decisions, and thus require transparent documentation to ensure trustworthiness.
The team applied large language models (LLMs) to screen almost 4,000 open-access clinical prediction papers, extracting statements about code availability and retrieving hundreds of associated code repositories. The LLMs' ability to process unstructured text allowed for detailed assessment beyond simple link extraction—evaluating documentation completeness, software requirements, and reuse permissions.
Results showed that only about 12.5 percent of studies reported sharing code, and among those, the quality and clarity of provided materials varied widely. Some repositories were well-organized with thorough documentation enabling reproducibility, while others lacked basic guidance, hindering verification or reuse efforts.
This variability extends across journals and countries; code-sharing rates ranged from zero to over 70 percent depending on publication venue, with existing journal policies on code sharing not consistently translating into actual practice. Some researchers did mitigate privacy constraints by sharing synthetic datasets to facilitate code reuse.
The investigation is informing TRIPOD-Code, an emerging reporting guideline building on existing TRIPOD and TRIPOD+AI frameworks. The new guideline aims to standardize what code and supporting information should be shared to improve reproducibility and clarity in clinical prediction research integrating medicine, statistics, and computer science.
Human reviewers validated the LLM assessments to ensure reliability, though the authors caution that classification errors remain possible and emphasize broader trends over precise counts. The study highlights the potential of AI in evaluating research practices at scale, identifying gaps, and guiding improvements.
Overall, the work underscores that while sharing analytical code is vital for scientific transparency, current practices are insufficiently standardized and inconsistent. By providing concrete guidance through TRIPOD-Code, the research community can foster greater openness and reliability in predictive medical modeling.
Source & further reading
AI helps researchers investigate how thousands of medical studies share code ↗Source published 2026-11-19. Claims and limitations should be checked against the original announcement or research.