Clinician-Graded Evaluationfor Medical AI.
Find the unsafe answers before patients do. Our clinical teams grade your medical AI's answers against a clinical rubric. Every delivery comes with written reasoning, licensed clinician audit and measured inter-rater agreement. De-identified data only.
✓ Start with 50 cases to check the rubric and agreement before you scale.
Are you a clinician? Join the Medical Network
Sample batch · 500 cases
Paediatric Triage Assistant
Inter-rater agreement (Krippendorff's α)
The Problem
Generic annotators and AI judges miss clinical errors.
Labellers without clinical training can't tell a safe dose from a dangerous one, and an AI judge shares many of the blind spots of the model it grades. In medicine, a fluent answer can still be the wrong one. Errors like these:
A missed interaction
Recommends ibuprofen for back pain to a patient whose history says they take warfarin, with no warning about bleeding risk.
A missed red flag
Tells a parent that a two-year-old with fast breathing and chest indrawing can be cared for at home with fluids and rest.
The wrong context
Treats fever and body aches in a malaria-endemic area as flu without advising a malaria test, or recommends a drug that isn't on the national essential medicines list.
What We Deliver
Clinical evaluation, judged by clinicians.
Clinical Answer Grading
Your model's answers scored 1–5 for clinical safety, diagnostic accuracy, communication and completeness, or your own rubric, with written clinical reasoning for every score.
Clinical Safety Red-Teaming
Our clinical teams try to make your model miss red flags, give unsafe doses or skip escalation, and record each failure with the prompt that caused it.
African Clinical Context
If your model will be used in Africa, it has to know African clinical practice. We check answers against national guidelines, local disease patterns and the medicines actually available, and review patient messages in local languages such as Swahili.
Clinical Test Sets
Realistic, de-identified cases with reference answers, written and approved by licensed clinicians in the specialty, so you can rerun the same benchmark on every release.
How It Works
From model answers to a clinical scorecard
Send De-identified Pairs
Send prompt and answer pairs as CSV or JSONL. We screen every row for patient identifiers and refuse any that contain them, then agree the clinical rubric with you.
Graded by Specialty
Cases go only to evaluators approved in the batch's specialty, across 16 specialties. Calibrated trainees do first-pass grading; licensed clinicians re-grade a share and grade high-risk cases themselves.
Review, Approve, Export
Review the batch in your client portal, then approve it or ask for changes. The export has every score, the written reasoning, each grader's tier and the agreement statistics.
How We Measure Quality
We don't ask you to trust our graders. We measure them.
Vetted before they grade
Every evaluator uploads a credential, and licences are checked by hand on the national council's public register. Then a timed clinical field test: 80% to pass.
Calibrated on known answers
An evaluator's first 10 cases have reference scores set by licensed clinicians. They start on client work only after 70% of their scores land within one point.
Hidden checks on live work
Known-answer cases are mixed into live batches. Evaluators whose accuracy falls below 70% are paused until our team reviews their work.
Licensed re-grading
A licensed clinician independently re-grades about 10% of every batch, and high-risk cases, such as dosing, are graded by licensed clinicians only.
Lead review of disagreements
When two grades of the same case are two or more points apart, a licensed lead reviewer settles the score before delivery.
Agreement on every delivery
Every delivery reports inter-rater agreement as Krippendorff's alpha and weighted kappa, overall and per criterion, read against the standard 0.8 threshold for reliable data.
Security
Security, stated plainly.
No patient data
We accept de-identified data only. Uploads are attested as de-identified, every row is screened for identifiers, and rows that contain them are refused.
Need-to-know access
Your team sees only your batches, and evaluators see only the cases assigned to them. Access is checked on every request.
Evaluators under NDA
Every evaluator signs a non-disclosure agreement covering client work.
Pseudonymous exports
Exports identify each grader by a pseudonymous ID and their tier, never by name.
Licence numbers stay private
Licences are checked against the council registers by our team and never appear in exports.
Short-lived links
Uploaded documents sit in private storage and open only through links that expire after five minutes.
Simple, Per-Case Pricing
Pay for Delivered Cases. Nothing Else.
Every batch is priced per graded case by complexity, quality checks are included, and you approve every delivery.
Approve Every Delivery
In your client portal
Invoiced only for delivered cases.
Per-Case Pricing
Quoted per batch · No seat fees
No seat fees or subscriptions.
Are you a clinician or senior medical student? Grade medical AI remotely, paid per case.
Ready to see how your model handles real clinical cases?
Start with a 50-case pilot batch, graded by clinicians and measured for agreement, before you scale.