Large language models poorly capture how word order shapes representations of sentence meaning in the human brain.

Transformer-based large language models match human brain sentence representations better than models that ignore word order, but all transformers poorly match brain activity for sentences requiring semantic-role structure.

• Why it matters: It remains unclear whether transformer models encode sentence meaning in a human-like way, especially when meaning depends on structural relations like who did what to whom.
• What they did: Using representational similarity analysis on 7T fMRI from 30 participants reading 108 carefully structured sentences, the study compared eight transformers and eleven semantic models in language regions of interest.
• The result: Transformers were significantly inferior to models explicitly designed to encode structural relations, implying that sentence structure plays a critical role in human sentence representation that transformers capture only weakly.