Beyond Sequence and Structure: Pretraining PLMs on Continuous Biophysical Fields

The Biophysical Transformer improves Spearman correlation on the FLIP Meltome thermostability benchmark on every trained head for both human splits by adding electrostatic biophysical fields as a pretraining signal.

• Why it matters: Protein thermostability prediction can benefit from physical context beyond sequence and discrete structure, but it is unclear whether continuous biophysical fields add measurable value.
• What they did: They pretrain a protein language model where a Poisson-Boltzmann solver computes electrostatic potential around each three-residue token, and a rotationally invariant spherical-harmonic descriptor is predicted from sequence alone.
• The result: Compared with a matched one-flag sequence-only control, the field objective yields Spearman gains that persist after residualizing dipeptide composition and after removing overlapping pretraining proteins, but not on the organism-mixed split.

AI summary of the abstract · may contain errors · not medical advice · read the paper.