Can dental EHR data support a learning health system? Completeness, consistency, and accuracy of structured versus narrative diagnoses

In 4,542 qualifying periodontal encounters (2021) across two academic dental centers, structured and narrative diagnoses each matched expert adjudication only about half the time, with completeness ranging 34.7%–84.5%.

• Why it matters: Dental EHRs are used for quality monitoring and research, but periodontal diagnosis reliability is uncertain, risking biased analytics and decision support when data are incomplete or inconsistent.
• What they did: Structured diagnoses were extracted via SQL and narrative diagnoses via a fine-tuned RoBERTa NLP model; ~20% were independently reviewed by two board-certified periodontists to create a gold standard, then compared using quadratic weighted kappa.
• The result: Format agreement was substantial but imperfect (kappa = 0.86 and 0.80), accuracy was modest (near 50%), narrative notes were numerically higher and favored by severity-weighted kappa, and structured fields carried the milder stage in 67.9% of disagreements.

AI summary of the abstract · may contain errors · not medical advice · read the paper.