Gateway · agent identity · policy · spend limits · kill switch

Charles Trinh — Agent Gateway and Fleet Control

A service card reading AGENT GATEWAY — identity, policy and limits for an agent fleet
Charles Trinh — Agent Gateway and Fleet Control

Running a fleet of agents under one gateway: identity per agent, policy at the boundary, rate and spend limits, and a single place to see what every agent did.

What this is

One agent is a project. Twelve agents is an estate, and an estate needs a control plane: who may call what, on whose behalf, at what rate, at what cost, and where the record lives.

The gateway is that plane. Put it in front of the models and the tools, and the questions that used to need an audit become a query against one table.

It also changes the failure mode. Without a gateway, an agent going wrong is discovered when someone notices the output; with one, it is discovered when the limit trips.

Identity per agent

An agent is a principal; it needs its own credentials, its own permissions and its own audit trail.

Shared service accounts make every incident unattributable. Issue per-agent credentials with a short life, and rotate them on a schedule you have actually tested.

Policy at the boundary

Enforce tool access, data scope and rate limits at the gateway so one place answers what the fleet may do.

Policy as configuration, versioned like code, with a recorded decision for every request. A policy that lives only in a prompt is a suggestion the model is free to ignore.

Spend and rate limits

Unbounded spend is unbounded risk; budgets belong at the gateway with hard stops.

Set per-agent and per-tenant ceilings, alert at eighty percent and stop at a hundred. The point of the stop is the conversation it forces, not the money it saves.

Observability across the fleet

One view of what ran, what it cost and what it produced, keyed by agent and task.

Every request carries an agent id, a task id and a trace id, so a single failing run can be followed through the tools it called. Without the trace, fleet dashboards are decoration.

Versioning and rollout

Models and prompts are releases; canary them and keep the ability to pin a previous behaviour.

A version is a tuple of model, prompt and tool set, deployed together. Pinning matters most the week a vendor silently updates a model you depend on.

Kill switch

A way to stop everything that works in seconds and has been tested, because a switch nobody tested is a diagram.

Test it during a quiet week and record the time to stop. A stop that takes twenty minutes is a stop that will be argued about while it runs.

Secrets and tool credentials

Tools carry credentials; a compromised agent is a compromised tool set unless scoped tightly.

Keep secrets at the gateway and inject per call, so no agent ever holds a long-lived key. Narrow scopes turn a compromised agent into a contained incident.

Review cadence

Read a sample of fleet output weekly; aggregate dashboards hide the failure that matters.

Ten outputs read by a person every week will surface drift that a hundred alerts did not. Put the reading on a calendar with a named owner.

How I do it

I inventory the fleet first: every agent, the tools it can call, the credentials it holds and the person who owns it. The inventory is usually shorter and messier than expected, and it is the document every later decision refers back to.

The gateway then goes in front of the models and tools with identity per agent, policy as versioned configuration, and budgets with a hard stop. Existing agents are moved through it one at a time, with the trace id carried forward so nothing is lost in the transition.

We close by exercising the controls: kill switch, budget stop, credential rotation and a policy change. Each is timed and recorded, because the value of a control is what it does on the day nobody wants to test it.

Artifacts you keep

  • A per-agent identity with short-lived credentials and a rotation schedule
  • Policy as versioned configuration, with a recorded decision per request
  • Per-agent and per-tenant budgets with alert and hard stop
  • A trace view joining request, tools called, model version and cost
  • A tested kill switch with a recorded time to stop

Why a gateway rather than direct calls

A gateway centralises credentials, rate limits, routing and the audit of every model call. The alternative, each application holding its own key, produces an estate where nobody can answer which models are in use or what they cost.

The gateway is also the place a policy is enforced: an allowlist of models, a maximum context, a redaction pass. A policy that lives in the gateway applies to every caller; a policy that lives in a document applies to the callers who read it.

Fleet operations

Treat models as a fleet with versions, deprecation dates and a fallback. When a provider deprecates a model, the gateway is where the substitution is made and where the regression is measured, rather than in eleven applications each discovering it separately.

Track per-model latency, error rate and cost daily. The day a provider changes its behaviour is visible there first, usually before it is visible in a user complaint.

Keys, quotas and tenancy

Keys belong in a vault the gateway reads at runtime, with rotation and per-tenant quotas. A shared key with no quota is a cost incident waiting for a loop, and every gateway eventually sees a loop.

Set hard ceilings per tenant and alert at eighty percent. The alert is the useful part, because the conversation about a limit is cheaper before it is enforced.

Audit and retention

Log the caller, the model, the task, the tokens and the outcome, and keep prompt payloads under a shorter retention with a documented reason. The audit answers governance questions; the payload retention is a privacy decision.

Reconcile the gateway log against provider invoices monthly. Differences point at calls that bypassed the gateway, which is a finding about architecture rather than about accounting.

T

he build-versus-consume line

A

gateway is small enough to build and important enough to own. Consuming a vendor gateway moves the audit trail and the routing policy outside the estate, which is usually the wrong trade for a regulated workload, though it is sometimes the right one for speed.

W

hatever the choice, keep the audit in the estate. A log you cannot query independently is a log you cannot rely on during an incident.

Testing the fleet

Run a fixed battery of calls against each model on a schedule, so a provider change is detected by your own measurements rather than by a user. The battery is small: a schema-constrained extraction, a refusal case and a long-context case.

Keep the outputs and compare against the previous run. The comparison is the drift detection, and it costs minutes a week.

Write the routing table down as data rather than as code logic. A table is reviewable by the people who own the cost centre, and it changes without a deployment when a model is deprecated.

Where two tenants call the same model, keep the cache keyed by tenant as well as by prompt. A shared cache is a cross-tenant disclosure with a performance benefit, and the benefit is always discovered before the disclosure.

Proof

  • ONC (g)(10) self-test PASS, 317 of 317 subtests, April 2026 — public suite URL
  • 67 public hostnames, 51 serving, one AWS account
  • 104 clearinghouse enrolments across 53 jurisdictions
  • USCDI v7 proposal — Patient Digital Identity Credential — listed at isp.healthit.gov
  • SAM.gov UEI MVKQW9DKKGB8 · CAGE 20KM5 · NPI 1194645564
  • AWS Business Associate Addendum active since 18 August 2026

Where this work is delivered

Engagements run from Las Vegas and reach across the valley: clients in Las Vegas, NV; Las Vegas, Nevada; Henderson; Clark County, Nevada; and North Las Vegas, Nevada, plus remote delivery nationwide.

How an engagement runs

1. Trace one real case

Follow a single patient or claim end to end with a timestamp at every hop, before any document is written.

2. Written findings

A findings list naming the defect, the system, the owner and the date.

3. Monthly review

Review the list monthly and check whether the dates held.

Frequently asked

What is this service?

This page describes charles trinh — agent gateway and fleet control — the scope, the failure modes it prevents and how the work is measured. It sits in the AI and Automation category at trinh.io.

Where is the work delivered?

Engagements run from Las Vegas, Nevada across the valley — Las Vegas, Henderson, North Las Vegas, Spring Valley, Paradise, Winchester, Sunrise Manor, Enterprise, Summerlin, Clark County — and remotely anywhere in the United States. On-site work in the Las Vegas valley, remote delivery elsewhere.

What does it cost?

Standard rate $195 per hour, committed $165 per hour at twenty hours or more per week for three months or more; $1,600 standard day, $1,350 committed day. Scoped in writing before work starts, fixed fee where the deliverable is defined.

Who is Charles Trinh?

A technology architect in Las Vegas, Nevada, working in healthcare identity, consent and interoperability. Not a physician, not a medical practice, and not affiliated with any other Charles Trinh.

How does an engagement start?

With one trace of a real case and a written findings list naming the defect, the system, the owner and the date. The first call is twenty minutes and free: 702-559-3489.

Engagement

Scoped in writing before work starts; fixed fee where the deliverable is defined. First call is 20 minutes and free. 702-559-3489 · charles@trinh.io

Talk to Charles

TRINH — Charles Trinh, technology architect
TRINH

Charles Trinh

Fractional CTO/CIO and Technology Architect — Healthcare Data, Identity & Regulated Infrastructure · Las Vegas, NV

Las Vegas, Nevada · US citizen · Fractional and project engagements

(702) 559-3489 Email Charles Trinh