AI
Charles Trinh — Ambient Scribe Evaluation | Las Vegas, NV · Las Vegas, Nevada · Henderson · Clark County, Nevada · North Las Vegas, Nevada

A vendor's demo note is not an evaluation. What matters is how a scribe behaves on your encounters, in your specialties, with your clinicians and your patients' words. I build the capture, the review and the scoring that answer that.
Why vendor scores do not transfer
Ambient scribe vendors publish accuracy figures against their own corpora, and those figures are honest in the narrow sense: they describe performance on the encounters the vendor chose. Your encounters differ in the ways that matter. Your specialties speak differently, your patients bring different interpreters and accents, your exam rooms have different acoustics, and your template conventions encode house rules the vendor has never seen. An evaluation has to be run on your recordings, against your reference notes, by your clinicians, and it has to be repeatable.
The most common failure in a scribe pilot is selection bias in the test set. Encounters are recorded for the pilot that the clinician is comfortable having recorded, in specialties where the clinician expects the tool to shine. A pilot can therefore report strong results and fail the first time it meets a rapid, interrupted encounter with a family member participating. I select the evaluation set by stratified sample from real encounter statistics before the pilot begins.
The second failure is the reference note. A reference note is the note a clinician would have written without the scribe, and if the scribe's draft is used to start it, the reference is contaminated. I require references written without access to the draft and signed by a clinician not on the pilot, and I measure the inter-reviewer disagreement on the references before scoring any draft against them.
The evaluation corpus
A usable corpus is maybe forty to eighty encounters for a first read, stratified across the three or four specialties in scope, the common encounter lengths, and the difficult cases that recur — the interpreter, the multi-speaker exam, the patient who is subdued or upset, the visit with a handheld tablet on a shared room. Recording a corpus like that requires consent, a retention rule and a deletion schedule, and all three are part of the project, not a prerequisite someone else provides.
Each encounter in the corpus has metadata that makes scoring possible: specialty, encounter length, speakers, interpreter presence, recording device and room type. Without that metadata every aggregate score masks the conditions where the scribe is failing. On several evaluations I have run, the aggregate recognition rate looked adequate and the rate in interpreter-led visits was below what the clinician needed to trust the draft for that population.
Audio quality deserves its own measurement line. A scribe on clinician phones in try as you record, not in a quiet conference room, because that is where the draft will be produced. I compute word error rate on transcripts against clinician-corrected transcripts per acoustic condition, and score the draft independently of the transcript: poor transcription with a serviceable draft is more usable than accurate words and a bad draft.
Scoring the draft
Scoring separates four things: content present in the draft, content missing from it, content attributed to the wrong speaker, and structure that does not match the template. Each has a different remediation. Weakness on missing content often means the summarisation prompt needs work; weakness on attribution usually needs the vendor to change speaker labelling. Collapsing them into one accuracy score hides where the fix belongs.
Omission harms a clinician through lost information; fabrication harms through false information. Both should be scored per encounter, not by counting words or entities, because the harm in an omission or fabrication depends on the role of the fact in the visit. I ask reviewers to score each encounter on a five-point scale for safe-to-sign without edits, and I report the share that reach the top two points alongside the per-class rates, since those two views almost never agree on a vendor's ranking.
Structure and house style are manageable through the vendor's templates, but only after the substantive issues are resolved. Scoring structure early leads teams into arguments about formatting while unsupported claims remain. I keep a separate formatting score in the report and tell the evaluation team explicitly not to weight it in the go-or-no-go decision, so the decision cannot drift on cosmetics.
The clinician edit as a signal
Once a scribe is deployed, the edits clinicians make to its drafts are the most representative signal available. Running a controlled comparison over four weeks with representative case mix produces a rate of edit types: additions, deletions, changes to the problem list, changes to assessment. That is the operational picture, and it replaces offline scores for deciding whether to keep or replace the tool. The edit log is also more honest than user surveys, which drift the first time the tool becomes part of routine.
The edit signal must be disaggregated to be useful. Aggregate edit volume tells the team whether they are editing a lot; disaggregated edit volume tells them what to change. A scribe may have a high edit count driven almost entirely by the closing summary section, which is often fixed with one template change. Averages hide that, and the vendor dashboard that reports them by section usually only reports sections the vendor chose to report.
Edits also have a ceiling. Once a clinician has edited a particular sentence pattern the same way many times, the scribe learns nothing without vendor intervention, and the clinician learns to dread the section. I monitor for this and frequently the right decision is to take the section away from the scribe and let a template fill it, rather than continue asking the scribe to draft it and continue fixing it.
Consent, recording and retention
Recording a patient encounter requires informed consent in most jurisdictions and the consent has to cover the retention, use for model improvement, and third-party processing explicitly. I do not accept a vendor's claim that its recording arrangement is covered by their terms; the organisation's own notice has to say what is happening. That is a legal question I cannot answer, but I can make sure the pipeline records which consent version was active and the user was recorded under, and that the store can find every recording by consent state.
Retention rules are part of the design, not a policy someone promises. Recordings retained for a pilot need a defined deletion schedule and a command that can demonstrate that the deletion ran. I test the deletion against a controlled record and confirm it is gone from both the primary store and any cache or backup the vendor operates, and if the vendor cannot show that, the retention rule is a wish rather than a control.
Recording devices and rooms present their own risk. A pocket recorder in a clinic with no visible indicator, or a tablet in a room without an indicator light, is a notice problem. I do not provide devices; I check that the deployment has one the patient can see, and that staff training explains the notice and the opt-out. This is one of the few places where a technology decision should be made entirely by the clinical and privacy leadership.
Stop criteria
Every scribe evaluation needs written stop criteria, and the decision has to be made against them at a set point rather than by a meeting where a vendor's account manager has given a presentation. I write the criteria with the clinical leadership before the pilot begins: the minimum safe-to-sign rate overall, the same rate in the hardest stratum, and the maximum per-encounter fabrication rate in clinical-significant fields. A vendor whose account team is excellent should not get a pilot extended because of their account team.
Fail-on-criteria is a real outcome and I have recommended it. Replacing a scribe is a cheap decision compared with letting a bad one become the practice standard and then having a safety event attributed to it. The evaluation report I write names the criteria and the measured values, and it says plainly whether the tool meets them. If the criteria were not met, that is the report's conclusion and it is mine.
My background covers backend systems and data engineering; the clinical judgement in every
score comes from the clinicians who work on the evaluation, and that division is stated in the
report. Where you are also evaluating governance for the vendor, the model records in
ai-model-governance-healthcare and the evidence pack in
ai-vendor-diligence extend the same trail to the data side. None of the clinical scores are mine to sign, and the report says so on its first page.
Deliverables and expected effort
The deliverable is a reusable evaluation harness: a corpus specification, a recording and consent process, a reviewer interface that reviews drafts in isolation from the vendor's branding, a scoring procedure and a report. It is designed to run a second pilot in weeks with a new vendor, and to be run again if the first vendor ships a substantial model update. The harness belongs to you and runs in your repository.
Effort is usually four to eight weeks end to end for the first evaluation, most of it in corpus collection and clinician review rather than engineering. The engineering piece is the harness and the storage; the corpus and reviewer time are clinician costs that I can plan with but not absorb. I quote the engineering portion and estimate the clinician portion with the clinical leadership so the total is visible before the start.
The report answers three questions with measured values rather than impressions: is the scribe safe enough to sign from on your notes; on which encounters does it fail most; and what is the operational edit profile after four weeks of controlled use. Those three answers are usually enough to decide, and where they are not, the gaps are named in the report rather than papered over.
Proof
- ONC (g)(10) self-test PASS, 317 of 317 subtests, April 2026 — public suite URL
- 67 public hostnames, 51 serving, one AWS account
- 104 clearinghouse enrolments across 53 jurisdictions
- USCDI v7 proposal — Patient Digital Identity Credential — listed at isp.healthit.gov
- SAM.gov UEI MVKQW9DKKGB8 · CAGE 20KM5 · NPI 1194645564
- AWS Business Associate Addendum active since 18 August 2026
Where this work is delivered
Engagements run from Las Vegas and reach across the valley: clients in Las Vegas, NV; Las Vegas, Nevada; Henderson; Clark County, Nevada; and North Las Vegas, Nevada — plus remote delivery nationwide.
How an engagement runs
1. Trace one real case
Follow a single patient or claim end to end with a timestamp at every hop, before any document is written.
2. Written findings
A findings list naming the defect, the system, the owner and the date.
3. Monthly review
Review the list monthly and check whether the dates held.
Frequently asked
What is this service?
This page describes charles trinh — ambient scribe evaluation — the scope, the failure modes it prevents and how the work is measured. It sits in the Identity and Consent category at trinh.io.
Where is the work delivered?
Engagements run from Las Vegas, Nevada across the valley — Las Vegas, Henderson, North Las Vegas, Spring Valley, Paradise, Winchester, Sunrise Manor, Enterprise, Summerlin, Clark County — and remotely anywhere in the United States. On-site work in the Las Vegas valley, remote delivery elsewhere.
What does it cost?
Standard rate $195 per hour, committed $165 per hour at twenty hours or more per week for three months or more; $1,600 standard day, $1,350 committed day. Scoped in writing before work starts, fixed fee where the deliverable is defined.
Who is Charles Trinh?
A technology architect in Las Vegas, Nevada, working in healthcare identity, consent and interoperability. Not a physician, not a medical practice, and not affiliated with any other Charles Trinh.
How does an engagement start?
With one trace of a real case and a written findings list naming the defect, the system, the owner and the date. The first call is twenty minutes and free: 702-559-3489.
Engagement
Scoped in writing before work starts; fixed fee where the deliverable is defined. First call is 20 minutes and free. 702-559-3489 · charles@trinh.io
Talk to Charles
- Phone: 702-559-3489
- Email: charles@trinh.io
- Based in Las Vegas, Nevada · working remotely across the United States
Record
Profile
Healthcare technology architect working where federal regulation becomes production infrastructure. Built and operates a FHIR R4 clinical platform — SMART App Launch authorization server, patient and provider portals, SQLite — on a hardened single-host AWS estate of 67 public hostnames. The platform returned a self-test PASS on the ONC (g)(10) Standardized API test suite in April 2026, 317 of 317 subtests, publicly verifiable. Designed a consent architecture in which a permissioned ledger records patient grant, refusal and revocation as immutable events while all clinical data stays off-chain and encrypted per patient — no PHI on the ledger by design. Filed a proposed USCDI v7 data element with ONC on verifiable patient identity. Translates the Federal Register, HHS/CMS guidance and HITRUST requirements into architecture, controls and audit evidence.
Selected engineering scale
- Self-managed estate of 67 public hostnames, 51 serving, on one AWS account.
- FHIR R4 API with a SMART App Launch authorization server; patient and provider portals; an AI service layer; Hyperledger Fabric peer, orderer and CA; IPFS; a 337-table SQLite clinical store.
- Two permissioned distributed ledgers in production, one operating a consent registry that records grant, refusal and revocation with no PHI on chain.
- X12 005010 pipelines across eight transaction sets.
- Designed, built, deployed and operated end to end by one person.
Credentials and standards work
- Inferno (g)(10) self-test PASS, 317 of 317 subtests, April 2026 — the ONC Standardized API test suite. Standards validated: US Core 6.1.0, USCDI v3, SMART App Launch 2.0.0, Bulk Data 2.0.0. Public suite URL.
- USCDI v7 public comment submission, April 2026 — proposed Patient Digital Identity Credential under the Provenance data class, listed publicly on the ONC Interoperability Standards Platform.
- SAM.gov UEI MVKQW9DKKGB8 · CAGE 20KM5 · NPI 1194645564. AWS Business Associate Addendum active 18 August 2026.
Technical environment
AWS us-east-1 (compute, storage, messaging, SMS, email, voice) · Linux and systemd hardening · nginx · Fabric and EVM ledgers · FHIR R4 · X12 005010 · HL7 · SQLite · self-hosted mail, drive, search and monitoring · Git-based change control and CI release gates.