Evaluation design shapes estimates of cross-donor generalization in single-cell classification of cell-type

Random cell-level splitting in single-cell PBMC and lung cell-type classification yields consistently higher macro F1 than donor-held-out evaluation, even after batch adjustment and donor-level mixed-effects modeling.

• Why it matters: Cell-level splits treat same-donor cells as independent, potentially biasing estimates of cross-donor generalization—critical when models are meant to generalize to unseen individuals.
• What they did: The study compared evaluation designs across PBMC and lung datasets using multiple expression representations, then tested robustness with batch covariate adjustment and donor-level mixed-effects modeling.
• The result: The macro F1 gap persisted under these controls, while evaluation gaps were minimal in near-ceiling kidney, pancreas, and blood benchmarks and in a randomized-label floor control.