Model Gateway¶

The generation interface keeps all three existing OpenAI-compatible paths: model discovery, Chat Completions, and the stateless Responses subset. Clients name a configured virtual model; Mission Control authorizes the call, selects an eligible deployment, isolates the provider credential, records Trace-native evidence, and returns the virtual model name.

For embedding generation and reranking, see Embed and rerank text. Native provider adapters serve the same generation endpoints with explicitly declared capabilities.

Model input/output guardrails apply before provider dispatch and before output release. Configured output checks buffer streaming until inspection completes; policy rejection never triggers provider fallback, and reported provider usage remains billable.

Prerequisites¶

An active virtual model route and resolved provider credential.
The model enabled in the key's Team.
model.generate in the actual virtual-key policy.
HTTPS outside loopback development.

export MISSION_CONTROL_URL="https://gateway.example.test"
export MISSION_CONTROL_API_KEY="mck_example_not_a_real_key"

List models¶

curl --fail-with-body \
  --header "Authorization: Bearer $MISSION_CONTROL_API_KEY" \
  "$MISSION_CONTROL_URL/v1/models"

Only currently enabled and authorized virtual names appear. Listing does not consume product request limits.

Each model's capabilities.operations lists its resolvable generation, embedding and/or rerank deployments. protocols, streaming, structured_output and function_calls describe generation routes only. A retrieval-only model has no generation protocols, false generation flags and text-only input, even when its vendor also offers chat models. Mixed models retain their generation capabilities alongside their additional operations. The list is capability metadata, not a live health or credential test; each invocation still resolves a route that supports the complete request.

Chat Completions¶

curl --fail-with-body \
  --request POST \
  --header "Authorization: Bearer $MISSION_CONTROL_API_KEY" \
  --header "Content-Type: application/json" \
  --header "traceparent: 00-0123456789abcdef0123456789abcdef-0123456789abcdef-01" \
  --data '{"model":"support","messages":[{"role":"user","content":"Summarize this case."}],"max_completion_tokens":128}' \
  "$MISSION_CONTROL_URL/v1/chat/completions"

Supported fields are model, messages, function tools, tool_choice, text or JSON-schema response_format, bounded string metadata, stream, store: false, temperature, and one of max_tokens or max_completion_tokens. Streaming clients may send the exact standard stream_options: {"include_usage": true} object; other stream options are rejected. Unknown or hosted/stateful fields fail with unsupported_feature.

For existing OpenAI-compatible deployments, Mission Control forwards a field to the provider only when the request sends it and adds no defaults of its own: a request without tools carries no tool_choice upstream, a request without metadata carries none, and store and stream travel only when the client set them. tool_choice without tools fails with invalid_request instead of being forwarded. Whether metadata or store is accepted depends on the deployment's vendor profile; a sent field the profile does not accept makes that deployment ineligible, and with no eligible deployment the call fails with capability_unavailable.

Stateless Responses subset¶

curl --fail-with-body \
  --request POST \
  --header "Authorization: Bearer $MISSION_CONTROL_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"model":"support","instructions":"Be concise.","input":"Summarize this case.","store":false,"max_output_tokens":128}' \
  "$MISSION_CONTROL_URL/v1/responses"

Supported fields are model, instructions, input, function tools, tool_choice, text format, bounded string metadata, stream, store: false, temperature, and max_output_tokens. Mission Control does not store Responses state. previous_response_id, provider-hosted tools other than Mission Control's own mcp tool object (below), background execution, and store: true are rejected instead of being approximated. The same forwarding rule applies: only fields the request sends reach the provider.

Governed tools on a model call¶

A model call can carry the caller's governed MCP tools, and the gateway can run some of them inside the call. The provider sees each tool as a function tool named <server-id>__<tool-name> with its discovered description and input schema; the credential that reaches the upstream never leaves the gateway. Nothing changes for a request that does not opt in.

The three public x-mc-* controls are x-mc-tools, x-mc-auto-execute and x-mc-max-tool-rounds. They select governed-tool behavior but confer no tool authority: current key/Team policy, eligibility, rules, approvals and budgets still apply. All other x-mc-* identity, service or execution headers are forbidden with an ordinary bearer key and fail authentication before dispatch. Invalid round values outside 1–20 return 400 invalid_request; a private identity header is not a way to select another caller.

Opt in per request:

Chat Completions with headers. x-mc-tools: governed attaches the tools; x-mc-auto-execute names the tools the gateway may run (* for every tool an administrator marked autoExecute, a comma-separated list of public names, or absent for none); x-mc-max-tool-rounds bounds the loop (1–20, default 5).
Responses with one mcp tool object beside your function tools: {"type":"mcp","server_label":"mission-control","require_approval":…,"max_tool_rounds":…}. require_approval: "always" (or absent) runs nothing, "never" runs every autoExecute tool, and {"never":{"tool_names":[…]}} names them. Any other server_label, a second mcp object, or any other hosted tool is rejected.

curl --fail-with-body \
  --request POST \
  --header "Authorization: Bearer $MISSION_CONTROL_API_KEY" \
  --header "Content-Type: application/json" \
  --header "x-mc-tools: governed" \
  --header "x-mc-auto-execute: crm__lookup-customer" \
  --data '{"model":"support","messages":[{"role":"user","content":"Which plan is customer 42 on?"}],"max_completion_tokens":128}' \
  "$MISSION_CONTROL_URL/v1/chat/completions"

The tools attached are exactly what tools/list would return for the same key: eligible, enabled in the key's Team, and covered by the caller's access. The automatic set is the intersection of three lists: the names the request asked for, the tools whose YAML declaration says autoExecute: true, and the tools the caller may see. A requested name outside that set is not an error; a call to it simply comes back for you to run. Classification never puts a tool into the automatic set: an Action tool runs automatically only when an administrator marked it and the request named it.

Each round asks the provider once. Calls in the automatic set run through the same tool kernel as a /mcp tools/call, in the order the model emitted them, and their results join the conversation as tool messages before the next round. The loop stops when a round returns no call, when any call in a round is outside the automatic set (that round's automatic calls still run; the rest come back pending with finish_reason: "tool_calls"), or when the round limit leaves no round to answer in (that round's calls all come back pending and none runs). A pending call is yours: run it or refuse it, then continue the conversation as with any function tool. tool_choice still needs your own tools and then applies to the combined list.

Every executed call is one mcp decision with origin model_loop on the same Trace as the model decision, under the model request's span, and with its own correlation ID (<model correlation id>:tool:<call id>). The model decision records the usage summed over every round. A hop the kernel refuses (denied, not enabled, drifted schema, invalid arguments, upstream failure) feeds the fixed text Tool invocation failed back to the model with is_error: true; the refused hop's own decision carries the reason, and the model decision still succeeds. A per-user Connector the caller has not connected feeds Connect your account in Mission Control to use this tool. back instead of a credential. Tool output reaching the model is the text items of the MCP result joined by newlines, with non-text items named rather than forwarded, bounded at 32,000 characters.

Rules and approvals apply to every hop as they do on /mcp. A rule that blocks feeds Tool invocation blocked by policy back; a redaction changes what the upstream receives or what the model reads. An Action tool whose declaration keeps approval: required does not run even when it is in the automatic set: the hop pauses as a durable approval, its decision records approval_required, and the model reads This action needs a person's approval in Mission Control before it runs; ask again once it is approved. (after a denial This action was denied in Mission Control. followed by the note; after a lost race with a concurrent identical hop This action already ran after its approval; a new approval is needed to run it again.). Once a person approves in the Console, the identical hop on a later request runs exactly once. See Rules on arguments and results and Approvals.

The response reports what ran:

Chat Completions adds mission_control when at least one call ran: tool_rounds and executed_tool_calls, each with id, name, arguments, content, and is_error. choices[0].message.tool_calls holds only the pending calls. To continue the conversation, append an assistant message carrying the executed calls as tool_calls and one tool message per call with its content, then your own turn.
Responses emits one mcp_call output item per executed call (id: mcpc_<call id>, server_label: "mission-control", name, arguments, output, and error: null or {"message":"Tool invocation failed"}) before the message or the pending function_call items. Send those mcp_call items back as input items to continue.

stream: true together with governed tools is rejected with unsupported_feature before any tool runs. A function tool of your own whose name matches a governed tool is rejected with governed_tool_conflict. Governed tools inside a streamed call, and provider-hosted tools that the provider itself executes, are not approximated.

Streaming¶

Set stream: true. Chat Completions emits compatible SSE chunks and [DONE]; Responses emits named Responses SSE events. The gateway records successful completion and attempts usage accounting before exposing the completed event. A canonical provider failure event records failed; an unexpected end of stream without a terminal event records indeterminate.

A canonical stream read error terminates the public stream with the original failure and attempts to record cancelled when the request signal is aborted, otherwise indeterminate. Consumer cancellation propagates to the selected provider with the same distinction: an unconfirmed disconnect is indeterminate. Cancellation starts terminal evidence and lease cleanup without waiting for provider shutdown to finish. A provider that hangs or fails during cancellation cannot retain local request capacity.

Completion and cancellation share one terminal operation. Once an outcome is being recorded, a competing outcome does not replace it or dispatch another provider. A missing public completion event does not prove the provider failed: completion may have been recorded before a client disconnected or local completion work failed. Check the Trace and provider evidence before retrying.

Deployment timeouts, skips and instance limits¶

Every deployment carries three timeouts, set in Organization YAML under models[].timeouts and defaulted when absent: firstTokenMs (60000: from dispatch to the response headers, which for a non-streaming call is the whole answer, or to the first stream event), idleMs (30000: the longest silence between stream events) and totalMs (600000: the whole call, stream included). A first-token timeout is recorded as an unavailable attempt with the detail timeout first_token after N ms, and the deployment's fallback rule chooses the next candidate as for any unavailable provider; when no candidate answers, the call fails 503 provider_unavailable. An idle or total timeout on a stream cancels the provider read and ends the public stream the way a broken stream ends, with the usage so far. Vertex reranking keeps rankingTimeoutMs and Connectors keep timeoutMs.

Each gateway instance also remembers how each deployment has been answering. Three consecutive unavailable outcomes, timeouts included, skip the deployment for 30 seconds, doubling on each failed probe up to five minutes; a rate-limited answer skips it for the provider's Retry-After, at most 60 seconds. A skipped deployment moves behind the other candidates of its virtual model and is tried only when it is the only one, in which case further failures extend the skip without reopening it; when the skip elapses, the next request probes it while every other request keeps the deployment behind the others; an answer of any kind other than unavailable closes the skip, an unavailable answer doubles it, and a probe that reports nothing within a minute lets the next request probe again. Nothing is stored and every instance learns on its own, so a provider that recovers is found within one cool-down. The instance logs Upstream skipped and Upstream recovered, and exposes mission_control_upstream_skips_total{kind} and mission_control_upstream_skipped{kind} (Metrics and alerts).

An instance admits at most GATEWAY_MAX_IN_FLIGHT governed requests at a time (256 by default), counted from the end of authorization to the end of the request. A request that finds the instance full is refused 503 instance_saturated with retry-after: 1, and once one Team holds half the maximum its next request is refused 429 team_share_exceeded with retry-after: 1, so one tenant's burst leaves the other half of the instance to everyone else. Both refusals happen before any decision is recorded, like a 401. Retry after the header's delay; behind a load balancer the retry reaches another instance, because the full instance's traffic probe answers saturated until it is back under 80% of its maximum (Run several gateway instances). The harness's hung profile measures the gate these give: Capacity.

Trace response¶

The response carries traceparent, X-Mission-Control-Trace-Id, and X-Mission-Control-Correlation-Id. The caller may continue the same Trace on related model or MCP requests by sending a new parent Span ID with the same Trace ID. No separate Trace lifecycle call exists.

Configuration during a call¶

The gateway resolves the Organization baseline plus the authorized virtual model's named guardrail policies before dispatch and keeps that policy across provider retries, fallback and governed-tool loop rounds. Two virtual names may use the same provider model with different additional checks. Clients cannot choose or disable policies with request fields or headers; model access still requires the existing Team/key grants. Any effective output rule, including detect-only, buffers generation streams. Embeddings and reranking use input rules only. See policy assignments.

Each invocation uses loaded process configuration for model routes, provider settings, prepared credentials, base capture policy and operational concurrency. File edits take effect only after gateway/Console restart. Historical health never removes a configured candidate from generation, embedding or reranking, and an instance's skip only reorders the candidates it still tries; capability requirements and bounded fallback remain enforced. Current key policy and Team allowance are checked at authorization, and current product limits at admission. GET /v1/models uses the process configuration load time for each model's created value, not a saved revision creation time.

Evidence and limits¶

inference_decision records the decision each request received — key and owner, operation, requested resource, allow or denial code, policy and funding evidence, configuration hash and Trace IDs — and, on completion, its terminal state, safe error code and finish time. Each physical provider attempt is a separate usage-ledger fact with its provider route, tokens, estimated or actual cost and cost status. Prompt and output content are stored in neither. When the Organization enables telemetry.standardOtlp, the gateway's request span and any standard OTLP spans land in the span store under the same Trace; with it off, neither is stored. A governed tool the gateway ran inside the call is a separate child tool.invoke decision on that Trace; each provider round is its own ledger attempt.

The decision, its admissions and its first attempt are written in one transaction before the provider is called; the settlement, the releases and the terminal state are written in one transaction after the response. Nothing reaches a provider before the first transaction commits, and a request refused before or inside it is recorded as a rejected or failed decision. Fallback applies only to provider invocation, upstream response classification, and normalization failures under the deployment's retry-safety rules; a fallback settles the failed attempt and opens the next one in one more transaction. A failure of the terminal transaction rejects the operation even when the provider succeeded, and no other provider is contacted.

A generic failure does not prove that no upstream work occurred. Check the Trace and provider evidence before retrying. The gateway still ends the request's spans after a failed terminal transaction, but an unavailable evidence store leaves the decision open for the existing stale-decision recovery. Durable permit leases retain their expiry recovery; a lease the gateway can no longer renew aborts the upstream call and records the attempt as indeterminate.

Model requests apply applicable Organization, Team, owner, key and customer spend/request/token/concurrency budgets. An admission denial contacts no provider and remains visible under Observe → Requests, in Requests, and on the Trace.

Function-tool descriptions are optional free-form strings of at most 1,000 characters. Empty or multiline descriptions are accepted; identifier restrictions still apply to tool names. Generation under finite spending caps requires an explicit supported output bound and configured pricing.

Explicit reasoning requests accept reasoning_effort: "low" | "medium" | "high" on Chat Completions and recognize reasoning: {"effort":"high"} on the Responses-facing API. Eligible routes must explicitly declare the effort and use the OpenAI profile with Chat Completions upstream; Responses upstream and other profiles refuse it with capability_unavailable. The field is retained through normalization and each tool-continuation request. Unknown effort values, extra reasoning fields and null values are refused, not dropped. See capability configuration.