Tracey CadetPeople · Privacy · Technology
Menu
← Back to Field Notes

Every System Has a Language · AI

Model Advice in the Wrong Language

A dated research abstract raises a practical question about AI health advice across languages.

At the American College of Gastroenterology’s Annual Scientific Meeting in Nashville, October 12 to 14, 2026, researchers presented an abstract comparing four AI models across seven languages on 50 simulated gastroenterology questions. The questions covered English and six other languages: Spanish, Chinese, Tagalog, Vietnamese, Arabic and French.

The abstract reported guideline concordance of 91.3 percent for English responses, compared with 63.8 to 87.5 percent across the other languages. In emergency scenarios, the models failed to advise patients to seek immediate treatment in 15 percent of English responses, 37.5 percent of Chinese responses and 40.8 percent of Vietnamese responses.

One important limitation belongs up front: the prompts were translated with a language model, and the responses were translated back with the same method. So this study measures the AI models and the translation process together, not the models alone. The findings come from a 2026 conference abstract, not a trial with real patients. Expert reviewers used a study rubric rather than physician adjudication, and clinical outcomes were not evaluated.

Still, the question stays with me. If a tool is offered for health questions, has anyone checked how it responds in the languages people use? Who reviewed not only what the tool says, but what it leaves out and when it tells someone to seek urgent care?

I grew up in Haiti and later came to America. I speak English, French and Haitian Creole. Moving between languages makes me attentive to what can shift in translation, and to who is expected to do extra work when a tool assumes English is enough.

I consider myself a student of responsible evaluation. This abstract is a reminder to ask about language before calling a tool ready for people.

Sources