Benchmarking DNA Foundation Models: Biological Blind Spots in Evo2 Variant-Effect Prediction

Evo2 attains strong AUROC = 0.927 for mitochondrial variant-effect prediction, but shows systematic blind spots in short-range signals like codon usage bias and context sensitivity.

• Why it matters: Existing benchmarks for DNA foundation models are limited in testing whether they capture established biological constraints, which is critical for trustworthy zero-shot clinical use.
• What they did: The authors built controlled benchmarks using well-characterized nuclear and mitochondrial sequence features, then analyzed Evo2’s performance and failure modes across variant and flanking-sequence settings.
• The result: Flanking-sequence changes can make pathogenic tRNA variants be misclassified as benign, implying Evo2 may rely on learned regularities rather than biological constraints.

AI summary of the abstract · may contain errors · not medical advice · read the paper.