The abstract is read at the publisher; the summary is JClub's.
RoB2-Evaluator, a rule-constrained GPT-5.2 system for Cochrane RoB 2 assessment, achieved 90.0% temporal agreement (weighted kappa = 0.93) across repeated RCT ratings after one week.
- Why it matters: RoB 2 scoring is resource intensive and can be inconsistent, so a system that maintains stable judgments over time and aligns with human reviewers is needed.
- What they did: The system used fixed RoB 2 guidance with a locked instruction framework, was calibrated on seven RCTs, then evaluated on 10 separate RCTs with re-assessment after one week versus consensus from two independent human reviewers.
- The result: Agreement with human consensus was 90.0% (weighted kappa = 0.93), methodological rationales stayed stable in 94.3% of repeated comparisons, and full AI-human conceptual alignment occurred in 85.7%.
The findingWhy it mattersWhat they didThe result
AI summary of the abstract · may contain errors · not medical advice · read the paper.