Toward Standards for Responsible AI in Health Communication: A Dual Method Pre-Deployment Risk Evaluation Framework and Case Study

A sequential dual-method pre-deployment evaluation of the Trusted Health Information Assistant (THIA) identified distinct risk categories, with 28 structured red-team prompts and six user testers validating complementary coverage.

• Why it matters: Public health adoption of generative AI lacks shared, replicable standards to assess platform safety, audience fit, and user readiness before deployment.
• What they did: NORC at the University of Chicago applied adversarial red-teaming first (five independent testers, 28 structured prompts), remediated issues, then ran user testing with six participants using interaction, pre/post surveys, interviews, and backend analytics.
• The result: Red-teaming surfaced platform integrity risks like PII handling and citation reliability, while user testing assessed readiness via trust and usability patterns, and convergent findings supported the framework’s actionable, multi-dimensional assessment.

AI summary of the abstract · may contain errors · not medical advice · read the paper.