01 · THE DEMO PROBLEM
Every AI demo works. That is the problem.
The demo room is quiet. One veterinarian, one cooperative Labrador, one tidy history, and a SOAP note that appears seconds after the exam ends. It is impressive, and it tells you almost nothing about Tuesday afternoon.
Tuesday afternoon is a fractious cat in a carrier, a pet parent on speakerphone, two dogs from the same household on one appointment, a technician calling out a weight in pounds, and a recheck that started as a vaccine visit. That is where clinical AI earns its place in the medical record, or quietly corrupts it.
AI is no longer an experiment in veterinary practice. In the 2026 Digitail and AAHA study of 1,730 veterinary professionals, 83.7% reported using AI, up from 39.2% in 2024.¹ Reliability and accuracy was the top concern. Nearly a third reported no AI governance support, and close to half said their practice has no AI plan or owner.¹
Governance has not kept up.
- ~1 in 3 report no AI governance support: no guidelines, training, AI lead or tool review.
- ~1 in 2 say their practice has no AI plan or owner.
- 38.9% have already tried building their own AI agents.
Source: Digitail and AAHA, The State of AI in Veterinary Medicine 2026 (n = 1,730; fielded July–August 2026).¹
Our position is simple. Accuracy is not a feeling you get from a demo. It is a number you can measure, and your practice should insist on that number before any AI tool writes a word into a patient's record.
02 · WHO CHECKS THE TOOL
In veterinary medicine, nobody else is checking.
Human healthcare has at least a scaffolding around clinical AI: FDA review pathways for many diagnostic algorithms, HIPAA, and peer-reviewed reporting standards such as TRIPOD+AI and DECIDE-AI that set out how to report evidence that a model works.²˒³
Veterinary medicine has far less. FDA does not require premarket approval or clearance for devices used in animals, software included; its Center for Veterinary Medicine acts after a product is on the market.⁴ Veterinary records are not protected health information under HIPAA. And the buyer guides practices rely on ask where the audio goes, how long it is kept, who owns the note and whether it connects to the PIMS. Accuracy is not on the list.⁵
That leaves one regulator in the room: your clinical team. The medical record is a legal document, and the licensed veterinarian who signs it owns every line, including the lines a model drafted. That is not a reason to avoid AI. It is a reason to validate it the way you would validate any new diagnostic: against known answers, before you trust it with patients.
Treat AI like a new in-house analyzer. You would never report results from a chemistry analyzer you had not run against known controls. A model that writes in the chart deserves the same discipline, and the same documentation.
03 · HOW CLINICAL AI FAILS
Know the five ways clinical AI fails.
Large language models have one property that makes them risky in a clinic: they are just as fluent when they are wrong as when they are right.⁶ A wrong note does not look wrong. It looks finished. These five failure modes cover most of what goes wrong in AI documentation and decision support.
| Failure mode | What it looks like in the chart | How to test for it |
|---|---|---|
| 1 · Omission | The plan said “recheck in 14 days” and the note dropped it. A discharge instruction or a declined diagnostic disappears. | Count the critical items in the reference note and confirm each one reached the draft. |
| 2 · Fabrication | A grade II/VI murmur nobody auscultated. “Abdominal palpation unremarkable” for an exam that never included it. | Every statement must trace to the recording, the record or a cited source. Flag anything that does not. |
| 3 · Misattribution | The pet parent's “she's been vomiting since Tuesday” recorded as a clinical finding. One dog's findings land in the other dog's chart. | Include multi-speaker and multi-pet visits. Check who said what, about which patient. |
| 4 · Specification error | Left versus right limb. mg versus mg/kg. A canine dose in a feline chart. A weight in pounds read as kilograms. | Seed cases with laterality, dose, unit and species traps. Accept zero errors. |
| 5 · Overconfidence | A fluent answer with no source. An exotic or off-label dosing question answered as if it were routine. | Add questions the tool should decline, and measure how often it does. |
Modes four and five are the ones that hurt patients. Treat any specification error as a critical error, and any confident answer without support as a design flaw, not a quirk.
04 · BUILD THE TEST
Build a test set from your own exam rooms.
You do not need a data science team for this. You need a medical director, a practice manager with PIMS access and about two weeks. The principles regulators apply to medical AI translate directly: test data that represents your real patients, kept independent of anything used to tune the tool, and performance measured under clinically relevant conditions.⁷
- Pull 40 to 60 recent visits. De-identify them. Cover your real case mix: wellness, sick visits, surgical discharges, dentals, euthanasia consults and rechecks. Then add the hard ones: multi-pet households, exotics, controlled drugs, interpreters and speakerphone.
- Write the gold standard. A DVM writes or approves the reference note or answer for each case and tags the critical items: diagnoses, doses, laterality, follow-ups and anything the pet parent was told to do.
- Add should-refuse cases. Make 10–15% of the set questions the tool should decline: a diagnosis from a pet parent's description alone, an exotic dose with no reference, anything outside the tool's intended use.
- Keep it sealed. Never hand the test set to a vendor for tuning. A tool that has seen the exam questions will pass the exam.
- Version every run. Record the model version, prompt or template and date for each run, so this month's result can be compared with next month's.
Why 50 cases is a floor, not a finish line. Zero errors in a small sample is reassuring, but it is not proof. The statistician's rule of three says that if you observe no events in n cases, the true rate could still be as high as about 3 ÷ n, with 95% confidence.⁸ Fifty clean cases still allow a critical-error rate of up to 5.8%. Three hundred clean cases bring that bound down to 1.0%.
That is why the test set is a gate, not a guarantee. Use it to reject tools that fail, then keep measuring in production, where your case count grows every day.
05 · MEASURE
Measure what matters clinically.
“It's 95% accurate” is a sales line, not a metric. Accurate at what, on which cases, by whose standard? Ask for these numbers instead, and track them yourself.
| Metric | What it tells you | Example starting target |
|---|---|---|
| Critical error rate | Specification errors, fabricated findings or wrong-patient content per note | Zero in the test set; any one is a stop |
| Omission rate | Share of the reference note's critical items missing from the draft | Set by your DVM lead, trending down |
| Unsupported statements | Statements that cannot be traced to the recording, the record or a source | Zero on clinical findings |
| Appropriate refusals | Share of should-refuse cases the tool declined | As close to 100% as possible |
| Unnecessary refusals | Share of answerable cases the tool declined | Low enough that clinicians keep using it |
| Clinician edit burden | Edits per note and minutes to sign | Falling, not rising, after go-live |
| Calibration | For scores: does a 90 mean right about nine times in ten?⁹ | Checked on every release |
| Subgroup performance | The same metrics split by species, visit type and number of speakers | No group far worse than average |
Targets are starting points for a conversation. Your medical director sets the bar.
Subgroups matter more in veterinary medicine than almost anywhere else. A tool tuned on canine wellness visits can look excellent on average while failing every rabbit, every ferret and every multi-cat household. An average hides exactly the patients who need you most.
06 · REFUSAL
Treat “I don't know” as a safety feature.
Here is the single best question to ask any AI vendor: how often does your tool refuse to answer? If the answer is never, worry. No model is right about everything, so a model that always answers is a model that sometimes guesses.
Machine learning researchers call the alternative selective prediction: a system allowed to abstain when its evidence is weak makes fewer errors on the questions it does answer.¹⁰ In a clinic, that trade is almost always worth making. A blank field gets a clinician's attention. A confident wrong answer gets signed.
Measure refusals in both directions. A tool that declines every should-refuse case but also refuses half of routine questions will be switched off by Friday. The goal is a tool that knows where its evidence ends, and says so in plain language.
07 · MONITOR
Test continuously, not once.
A model that passed in March can fail in August without anyone in your building touching it. Vendors update the underlying model. Your case mix shifts from kitten season to tick season. New drugs arrive, PIMS templates change and a new associate dictates differently. This is drift, and regulators of human medical AI expect deployed models to be monitored for exactly this reason.⁷
- Every release: re-run the sealed test set before clinicians rely on the new version. If the vendor cannot tell you when the model changed, that is your answer.
- Every month: audit 10 signed AI-drafted notes against the recording or the record, rotating clinicians and species.
- Every incident: log it with the model version, case type and fix, the same way you would log a medication error.
- Every quarter: review the metrics with your medical director and decide whether the tool stays, changes scope or goes.
08 · GOVERNANCE AND DOCUMENTATION
Document it like it will outlive the demo.
The AI tools in your practice will outlast the people who chose them. Associates move on, vendors are acquired, models are swapped and insurers start asking how a record was produced. Documentation is what lets a tool survive all of that. Without it, every staff change resets your institutional knowledge to zero.
The NIST AI Risk Management Framework gives a clinic-sized structure: Govern, Map, Measure and Manage.¹¹ Its Generative AI Profile extends that structure to the language-model tools now common in practice.¹² Groups that want certification-grade rigor can go further with ISO/IEC 42001, the international standard for AI management systems.¹³
| NIST function | What it means in a practice | What you keep on file |
|---|---|---|
| Govern | A named AI lead, a one-page use policy, and a rule that a clinician signs every record | AI policy, named owner, sign-off rule, training log |
| Map | Every AI tool, what it does, what data it touches and where that data goes | AI register and a data-flow map for each tool |
| Measure | The sealed test set, the metrics and the monthly audits | Versioned test results, audit sheets, model cards¹⁴ |
| Manage | What happens when something goes wrong, and who decides | Incident log, change log, rollback and retirement plan |
For each tool, keep a one-page model card: intended use, what it is not for, model and version, test results by subgroup, known limits and the date of the last review.¹⁴ Pair it with the security facts that matter just as much: where inference runs, which sub-processors see your data, how it is encrypted, how long it is kept, and whether it is ever used for training.
Ten questions to ask any AI vendor
- What is the intended use, and what is out of scope?
- Which model and version is running today, and how will we know when it changes?
- Which cases was it tested on, by species and visit type, and can we see results for each group?
- What is your critical error rate, and how do you define a critical error?
- When does the tool refuse to answer, and how often?
- Can every output be traced to its source in the recording, the record or a reference?
- Where does inference run, who are your sub-processors, and is our data used for training?
- What is logged, for how long, and can we export the audit trail?
- How do you monitor performance after go-live, and what triggers a rollback?
- Does a licensed veterinarian remain the final authority on every record?
09 · HOW QUILL BUILDS FOR THE TEST
We built QuillsFlow to be checked.
We designed QuillsFlow on the assumption that every AI answer will be checked, because in a clinic it should be. QuillsFlow is in pilot ahead of general availability, so each capability below carries its status.
- No source, no answer (pilot). Clinical questions are answered from a curated veterinary library, using hybrid retrieval across structured record fields and free-text notes. When confidence or citations fall short, the system withholds the answer instead of guessing.¹⁵˒¹⁶
- Evaluation that includes refusals (pilot). AI quality is measured against known cases, including cases where the right answer is to refuse.¹⁵
- Every decision can be rebuilt (pilot). Each AI-assisted decision is designed to be reconstructed from its inputs, sources, model and reviewer.¹⁵˒¹⁶
- Scores come with reasons (pilot). The AI Confidence Engine returns a 0–100 claim-readiness score, always paired with a plain-language reason. It is a readiness signal, not an approval, and it does not determine coverage.¹⁵˒¹⁷
- The clinician signs (current design). A licensed veterinarian reviews and signs every clinical record before it is final or used in a claim.¹⁶
- Your data stays scoped (current design). Prompts include only the data needed for that task and that clinic, and clinic data is not used for training without written agreement.¹⁶
- One gateway, one policy (design). Every AI feature is designed to route through a single gateway with shared policy, logging and cost control, with models running in Quill's own cloud environment behind private networking.¹⁵
- Governed by framework (current design). Quill's AI governance is aligned to the NIST AI RMF, and its controls are mapped to SOC 2 Trust Services Criteria. An independent SOC 2 audit is planned; no report has been issued yet.¹⁶
Work with Quill — put your AI to the test. We'll help you build it.
Whether your AI comes from Quill or from another vendor, our team can help you prove it works and keep it that way. We work with clinics and multi-location groups to make sure AI and ML integrations are documented, governed and secured for the long run.
- An AI inventory and data-flow map for every tool that touches your records
- A sealed test set and scorecard built from your own case mix
- Vendor reviews against the ten questions in this briefing
- Governance documentation aligned to the NIST AI RMF: policy, model cards, audit and incident logs
- A security review of where your data goes, who can see it and how long it stays
Contact Quill → · Visit the Trust Center → · Read the Technical Docs →
Now enrolling pilot clinics for the November 2026 cohort · hello@quillsdc.com. When you contact us, do not include patient names, medical records, policy numbers, or other sensitive information.
APPENDIX
Your one-page AI test plan.
Print it, assign an owner to each line, and review it with your medical director every quarter.
- Name an AI lead and write a one-page AI use policy.
- List every AI tool, its intended use and the data it touches.
- Pull 40–60 de-identified visits that reflect your real case mix.
- Have a DVM write or approve the reference note for each case, with critical items tagged.
- Add should-refuse cases, about 10–15% of the set.
- Score critical errors, omissions, unsupported statements, refusals, edit burden and subgroups.
- Gate go-live on zero critical errors.
- Record the model version on every test run and every AI-drafted note.
- Audit 10 signed AI-drafted notes every month.
- Re-run the test set on every model, prompt or template change.
- Log AI incidents the way you log medication errors.
- Keep a model card and vendor security facts on file for each tool.
SOURCES
References.
- Digitail and AAHA. The State of AI in Veterinary Medicine 2026. Published October 1, 2026. digitail.com
- Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. doi.org/10.1136/bmj-2023-078378
- Vasey B, Nagendran M, Campbell B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine. 2022;28:924–933.
- U.S. Food and Drug Administration, Center for Veterinary Medicine. Animal Medical Devices. fda.gov
- Wysocki A. 5 Questions to Ask Before You Choose an AI Scribe. Today's Veterinary Business. September 2026. todaysveterinarybusiness.com
- Ji Z, Lee N, Frieske R, et al. Survey of hallucination in natural language generation. ACM Computing Surveys. 2023;55(12):1–38.
- U.S. FDA, Health Canada and UK MHRA. Good Machine Learning Practice for Medical Device Development: Guiding Principles. October 2021.
- Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all right? Interpreting zero numerators. JAMA. 1983;249(13):1743–1745.
- Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning (ICML). 2017.
- Geifman Y, El-Yaniv R. Selective classification for deep neural networks. Advances in Neural Information Processing Systems 30 (NeurIPS). 2017.
- National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. January 2023. doi.org/10.6028/NIST.AI.100-1
- National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. July 2024. doi.org/10.6028/NIST.AI.600-1
- ISO/IEC 42001:2023. Information technology — Artificial intelligence — Management system. International Organization for Standardization. 2023.
- Mitchell M, Wu S, Zaldivar A, et al. Model cards for model reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*). 2019:220–229.
- Quill's Direct Vet Care. AI and analytics. Accessed October 2026. quillsdc.com/intelligence
- Quill's Direct Vet Care. Trust Center. Accessed October 2026. quillsdc.com/security
- Quill's Direct Vet Care. Technical Docs. Accessed October 2026. quillsdc.com/resources/technical
QuillsFlow is in pilot ahead of general availability. Capabilities described as pilot, design or planned are not generally available, and screens are illustrative previews. Statistics carry their source and year. This briefing is general guidance and does not replace your practice's own clinical, legal or compliance review.
