Gateway · agent identity · policy · spend limits · kill switch
Charles Trinh — Agent Gateway and Fleet Control

Running a fleet of agents under one gateway: identity per agent, policy at the boundary, rate and spend limits, and a single place to see what every agent did.
What this is
One agent is a project. Twelve agents is an estate, and an estate needs a control plane: who may call what, on whose behalf, at what rate, at what cost, and where the record lives.
The gateway is that plane. Put it in front of the models and the tools, and the questions that used to need an audit become a query against one table.
It also changes the failure mode. Without a gateway, an agent going wrong is discovered when someone notices the output; with one, it is discovered when the limit trips.
Identity per agent
An agent is a principal; it needs its own credentials, its own permissions and its own audit trail.
Shared service accounts make every incident unattributable. Issue per-agent credentials with a short life, and rotate them on a schedule you have actually tested.
Policy at the boundary
Enforce tool access, data scope and rate limits at the gateway so one place answers what the fleet may do.
Policy as configuration, versioned like code, with a recorded decision for every request. A policy that lives only in a prompt is a suggestion the model is free to ignore.
Spend and rate limits
Unbounded spend is unbounded risk; budgets belong at the gateway with hard stops.
Set per-agent and per-tenant ceilings, alert at eighty percent and stop at a hundred. The point of the stop is the conversation it forces, not the money it saves.
Observability across the fleet
One view of what ran, what it cost and what it produced, keyed by agent and task.
Every request carries an agent id, a task id and a trace id, so a single failing run can be followed through the tools it called. Without the trace, fleet dashboards are decoration.
Versioning and rollout
Models and prompts are releases; canary them and keep the ability to pin a previous behaviour.
A version is a tuple of model, prompt and tool set, deployed together. Pinning matters most the week a vendor silently updates a model you depend on.
Kill switch
A way to stop everything that works in seconds and has been tested, because a switch nobody tested is a diagram.
Test it during a quiet week and record the time to stop. A stop that takes twenty minutes is a stop that will be argued about while it runs.
Secrets and tool credentials
Tools carry credentials; a compromised agent is a compromised tool set unless scoped tightly.
Keep secrets at the gateway and inject per call, so no agent ever holds a long-lived key. Narrow scopes turn a compromised agent into a contained incident.
Review cadence
Read a sample of fleet output weekly; aggregate dashboards hide the failure that matters.
Ten outputs read by a person every week will surface drift that a hundred alerts did not. Put the reading on a calendar with a named owner.
How I do it
I inventory the fleet first: every agent, the tools it can call, the credentials it holds and the person who owns it. The inventory is usually shorter and messier than expected, and it is the document every later decision refers back to.
The gateway then goes in front of the models and tools with identity per agent, policy as versioned configuration, and budgets with a hard stop. Existing agents are moved through it one at a time, with the trace id carried forward so nothing is lost in the transition.
We close by exercising the controls: kill switch, budget stop, credential rotation and a policy change. Each is timed and recorded, because the value of a control is what it does on the day nobody wants to test it.
Artifacts you keep
- A per-agent identity with short-lived credentials and a rotation schedule
- Policy as versioned configuration, with a recorded decision per request
- Per-agent and per-tenant budgets with alert and hard stop
- A trace view joining request, tools called, model version and cost
- A tested kill switch with a recorded time to stop
Why a gateway rather than direct calls
A gateway centralises credentials, rate limits, routing and the audit of every model call. The alternative, each application holding its own key, produces an estate where nobody can answer which models are in use or what they cost.
The gateway is also the place a policy is enforced: an allowlist of models, a maximum context, a redaction pass. A policy that lives in the gateway applies to every caller; a policy that lives in a document applies to the callers who read it.
Fleet operations
Treat models as a fleet with versions, deprecation dates and a fallback. When a provider deprecates a model, the gateway is where the substitution is made and where the regression is measured, rather than in eleven applications each discovering it separately.
Track per-model latency, error rate and cost daily. The day a provider changes its behaviour is visible there first, usually before it is visible in a user complaint.
Keys, quotas and tenancy
Keys belong in a vault the gateway reads at runtime, with rotation and per-tenant quotas. A shared key with no quota is a cost incident waiting for a loop, and every gateway eventually sees a loop.
Set hard ceilings per tenant and alert at eighty percent. The alert is the useful part, because the conversation about a limit is cheaper before it is enforced.
Audit and retention
Log the caller, the model, the task, the tokens and the outcome, and keep prompt payloads under a shorter retention with a documented reason. The audit answers governance questions; the payload retention is a privacy decision.
Reconcile the gateway log against provider invoices monthly. Differences point at calls that bypassed the gateway, which is a finding about architecture rather than about accounting.
T
he build-versus-consume line
A
gateway is small enough to build and important enough to own. Consuming a vendor gateway moves the audit trail and the routing policy outside the estate, which is usually the wrong trade for a regulated workload, though it is sometimes the right one for speed.
W
hatever the choice, keep the audit in the estate. A log you cannot query independently is a log you cannot rely on during an incident.
Testing the fleet
Run a fixed battery of calls against each model on a schedule, so a provider change is detected by your own measurements rather than by a user. The battery is small: a schema-constrained extraction, a refusal case and a long-context case.
Keep the outputs and compare against the previous run. The comparison is the drift detection, and it costs minutes a week.
Write the routing table down as data rather than as code logic. A table is reviewable by the people who own the cost centre, and it changes without a deployment when a model is deprecated.
Where two tenants call the same model, keep the cache keyed by tenant as well as by prompt. A shared cache is a cross-tenant disclosure with a performance benefit, and the benefit is always discovered before the disclosure.
Proof
- ONC (g)(10) self-test PASS, 317 of 317 subtests, April 2026 — public suite URL
- 67 public hostnames, 51 serving, one AWS account
- 104 clearinghouse enrolments across 53 jurisdictions
- USCDI v7 proposal — Patient Digital Identity Credential — listed at isp.healthit.gov
- SAM.gov UEI MVKQW9DKKGB8 · CAGE 20KM5 · NPI 1194645564
- AWS Business Associate Addendum active since 18 August 2026
Where this work is delivered
Engagements run from Las Vegas and reach across the valley: clients in Las Vegas, NV; Las Vegas, Nevada; Henderson; Clark County, Nevada; and North Las Vegas, Nevada, plus remote delivery nationwide.
How an engagement runs
1. Trace one real case
Follow a single patient or claim end to end with a timestamp at every hop, before any document is written.
2. Written findings
A findings list naming the defect, the system, the owner and the date.
3. Monthly review
Review the list monthly and check whether the dates held.
Frequently asked
What is this service?
This page describes charles trinh — agent gateway and fleet control — the scope, the failure modes it prevents and how the work is measured. It sits in the AI and Automation category at trinh.io.
Where is the work delivered?
Engagements run from Las Vegas, Nevada across the valley — Las Vegas, Henderson, North Las Vegas, Spring Valley, Paradise, Winchester, Sunrise Manor, Enterprise, Summerlin, Clark County — and remotely anywhere in the United States. On-site work in the Las Vegas valley, remote delivery elsewhere.
What does it cost?
Standard rate $195 per hour, committed $165 per hour at twenty hours or more per week for three months or more; $1,600 standard day, $1,350 committed day. Scoped in writing before work starts, fixed fee where the deliverable is defined.
Who is Charles Trinh?
A technology architect in Las Vegas, Nevada, working in healthcare identity, consent and interoperability. Not a physician, not a medical practice, and not affiliated with any other Charles Trinh.
How does an engagement start?
With one trace of a real case and a written findings list naming the defect, the system, the owner and the date. The first call is twenty minutes and free: 702-559-3489.
Engagement
Scoped in writing before work starts; fixed fee where the deliverable is defined. First call is 20 minutes and free. 702-559-3489 · charles@trinh.io
Talk to Charles
- Phone: 702-559-3489
- Email: charles@trinh.io
- Based in Las Vegas, Nevada · working remotely across the United States