Why my medical AI's emergency detector doesn't use an LLM

When I built DocuMed, a medical question-answering system for my dissertation, the first requirement I wrote down wasn't about answering questions at all. It was this: if someone describes an emergency, tell them to get help before anything else happens.

The obvious way to do that in 2026 is to ask the language model. I deliberately didn't. The emergency screen is 45 plain, deterministic rules that run before the model is ever called.

The failure costs aren't symmetric

A system that sends someone with a mild headache to a hospital has caused an inconvenience. A system that tells someone with stroke symptoms to rest and drink water has caused harm. I wrote this asymmetry into the design as a rule: telling someone their problem is less urgent than it is counts as a safety event, while telling them it is more urgent counts as an inconvenience. That makes the emergency layer's job narrow and strict: never miss, and accept some over-caution.

Why rules beat a model for this one job

• Predictable. The same message always gets the same answer. A model with any sampling temperature can't promise that.
• Reviewable. Each rule is a few lines of configuration with a clinical source attached, so a doctor can read and challenge it. Nobody can review a model's weights.
• Testable. Every rule has labelled examples that must trigger it, and look-alikes that must not.
• Fast and available. No network call, no rate limit, no outage. The emergency path works even when the AI provider is down.

What the rules cover

Stroke signs, cardiac chest pain, severe breathing difficulty, meningitis patterns, fever in infants under three months, snake bites, carbon-monoxide poisoning when several people in a room feel ill, altitude sickness at the levels that kill (very relevant in Nepal), diabetic emergencies and more. When one fires, the user sees an emergency banner with a one-tap call to 102, Nepal's ambulance number, and a clear statement that this is not a diagnosis.

How I knew it worked

I wrote the acceptance test before the rules: a labelled set of emergency and non-emergency messages, run automatically on every change. The final rule set caught 69 out of 69 emergency cases with zero false alarms on that set. When I found a gap (diabetic ketoacidosis in someone who doesn't know they're diabetic, for example), the new rule went in together with test cases that prove it fires, and look-alikes that prove it doesn't.

The general lesson

This isn't really about AI. It's the same thing I do as a system analyst: identify the requirement whose failure is unacceptable, give it the simplest mechanism that can be fully specified and tested, and keep the clever parts of the system away from it. The language model still does what it's good at, which is explaining things in plain language from cited sources. It just isn't trusted with the decision that matters most.