Assessment of LLMs on basic bioinformatics tasks requiring multi-step reasoning and code writing.

GPT-5.2 achieved the highest accuracy on 104 Rosalind bioinformatics questions, answering 80 correctly (77%), with Claude Sonnet 4.6 next best among frontier LLMs.

• Why it matters: The work addresses uncertainty about how general-purpose LLMs perform on basic bioinformatics tasks that require both multi-step reasoning and code writing, relative to expert and human ground truth.
• What they did: The authors evaluated frontier LLMs (GPT-5.2, GPT-4o, GPT-3.5, Llama-3-70B, Llama-4, Claude Sonnet 4.6) on 104 Rosalind questions and compared results to expert answers and 68,760 humans across expertise levels.
• The result: LLM accuracy varied by biological question category: overall gains did not translate uniformly, and model performance broadly mirrored human patterns while still showing category-specific failures and occasional cases where LLMs excelled.

AI summary of the abstract · may contain errors · not medical advice · read the paper.