The abstract is read at the publisher; the summary is JClub's.
ClinDataBench shows that, on a CDISC pilot with 22 SDTM domains producing 13 ADaM datasets, closed models and DeepSeek delivered complete packages in every setting, while open-weight models did not.
- Why it matters: Industry adoption needs evidence of what LLM agents get right and wrong in end-to-end, conformant clinical trial analysis packages, beyond validators that only check structure.
- What they did: Agents received SDTM domains plus the SAP and specifications, then had to output 13 ADaM datasets, Define-XML, and an offline rebuild program in containers with only model-API access; a host evaluator scored P21 conformance, selected derivations, clean reconstruction, and effort across Base, Self-Refine, and CRITIC.
- The result: Validator findings ranged from 15 to 64 dataset-rule pairs; CRITIC reduced finding counts in each usable comparison (4 of 6 workflows) but did not reduce failed derivation counts, and the lowest-finding package omitted 234 efficacy summary records as specified.
The findingWhy it mattersWhat they didThe result
AI summary of the abstract · may contain errors · not medical advice · read the paper.