Case study · Healthcare
A domain model that orthodontists rated above GPT-4o.
How we built OrthoAtlas, a clinical knowledge assistant grounded in orthodontic literature, and how we tested it against the general models clinicians already use.
The problem
Orthodontists already ask general chatbots clinical questions. The answers read well. They come from the open web, carry no sources and sound equally sure when they are wrong. A clinician has no fast way to check them, so the tool either gets trusted too much or not used at all.
What we built
OrthoAtlas answers only from an indexed corpus of orthodontic textbooks and journal literature. Every claim in an answer is tied to the passage it came from. When the retrieved evidence is weak or the sources disagree, the answer says so instead of filling the gap.
- Retrieval. Clinical texts are split into passages, embedded with Voyage AI and searched in Qdrant.
- Generation. Claude writes the answer from the retrieved passages only, with a citation on each claim.
- Abstention. Low-support answers are flagged rather than smoothed over.
- Product. Chat, patient records and case analysis, in English and Turkish, behind clinician sign-in.
- Boundary. It is a decision-support tool. The clinician always has the final say.
How we measured it
We did not grade our own homework. In a blinded, within-subjects study, fourteen orthodontic questions were put to OrthoAtlas (evaluated under its research name, AlimGPT), GPT-4o, Gemini and Llama. Clinicians scored every answer without knowing which model wrote it, on reliability, usefulness, modified DISCERN and a modified Global Quality Score.
| OrthoAtlas vs | GPT-4o | Gemini | Llama |
|---|---|---|---|
| Usefulness | higher | higher | higher |
| Reliability | n.s. | higher | higher |
What did not move
The modified DISCERN score differed overall, but no single pairwise contrast survived correction. The modified Global Quality Score showed no significant difference. Rater agreement on DISCERN was poor to moderate. The study used a modest number of raters and questions, and the paper says a larger panel is the next step. We publish these limits with the result, not after it.
What this means for your team
If your organization runs on expert knowledge that has to be cited, such as clinical protocols, regulations, contracts or technical manuals, the same approach applies: a grounded assistant over your documents, an evaluation set built with your own experts, and a measured comparison against the tools your people use today.
Reference: S. Aksakallı, B. Giray, C. Temel. Domain-Specific vs. General-Purpose Large Language Models in Orthodontics: A Blinded Comparison of AlimGPT, GPT-4o, Gemini, and Llama. Dentistry Journal, 2026, 14(4):219. doi 10.3390/dj14040219 ↗