Questioning the Benchmarks: Hidden Circularity in AQ-10 Autism Screening Datasets

High ML accuracy on UCI AQ-10 autism screening variants is linked to circularity, with a mean Circularity Index of 0.947 and item-only accuracy 93.2–100% after removing a precomputed score column.

• Why it matters: Reported 95–99% performance may reflect dataset construction rather than diagnostic learning, so it is critical to test whether models learn behavioral label items or demographic signals.
• What they did: Six model families were evaluated on AQ-10 Child (n=290), Adolescent (n=103), and Adult (n=699) using normalized Circularity Index plus item-only and demographics-only accuracy, and the pattern was checked on DASS-42 (n=35,481) and Rosenberg Self-Esteem (n=39,076).
• The result: Item-only accuracy remained high while demographics-only accuracy dropped to 50.7–71.5% (at or below majority baseline), and AutoCirc reproduced Circularity Index on 3 of 5 datasets, enabling an eight-of-ten AQ-10 short form.

AI summary of the abstract · may contain errors · not medical advice · read the paper.