Ground-truth evaluation of sparse autoencoder features on spatial, single-cell and language-model embeddings

Consensus features from five sparse autoencoder fits are highly reproducible (78–98% across sources) and remain as pure as single fits, outperforming varimax sparsification on concept coverage.

• Why it matters: Reproducibility across random seeds is often used to validate sparse autoencoder features, but it is unclear whether reproducible features are also meaningful.
• What they did: The study evaluated SAEs on five spatial transcriptomics embeddings of two colorectal cancer sections, Geneformer embeddings with lineage concepts, synthetic data with planted cell states, and a small language model residual stream with token-class concepts, comparing varimax rotations and single versus consensus SAE fits.
• The result: Varimax rotations were almost perfectly reproducible yet covered at most three of seven lineages and three of eight token classes, while single SAEs covered most concepts but reproduced only 27–46% of features; consensus balanced both strengths but covered fewer lineages than single fits and missed weak planted states.