Connect native model providers

Purpose

Connect native Anthropic, AWS Bedrock, Azure OpenAI and Google Vertex AI to governed virtual models. Clients continue using Mission Control's public generation, embedding and reranking endpoints. Native provider support describes the upstream connection; it does not add public Anthropic or Gemini protocol endpoints.

Prerequisites

• An organization owner or administrator with organization.settings.manage.
• A gateway deployment containing the native adapters and current database migrations.
• A provider account, approved model/deployment identifiers and the provider permissions required for the selected operation.
• Gateway-only credentials: an Anthropic API key, an AWS workload role/default credential chain, an Azure API key or supported default credential, or Google application default credentials.

The examples below extend an existing organization YAML. Provider resource names and model identifiers are fictional unless explicitly identified otherwise. Replace them with your approved resources before making inference calls. Do not place raw keys in YAML.

Procedure

1. Configure the provider connection in the organization YAML. Select the appropriate example below.
2. Add model deployments with a virtual name and the appropriate operation and profile. Omitted operation means generation, preserving existing configurations.
3. Make the required credentials available to the gateway. Restart it after environment changes. Neither the Console nor the managed Agent runtime needs provider credentials.
4. Select operator preflight and a coordinated gateway/Console restart. Invalid family/profile/operation combinations or fallbacks across operations are rejected.
5. Enable each virtual model on the intended Team and allow model.generate, model.embed or model.rerank as appropriate in its virtual-key policy. A provider credential alone grants no Mission Control access.
6. Make a synthetic request and inspect Observe → Requests and the authorized Trace. Test both streaming and tool calls if the deployment will use them.

Anthropic

providers:
  anthropic:
    type: anthropic
    credential: env/ANTHROPIC_API_KEY

models:
  - id: assistant-anthropic
    provider: anthropic
    providerModel: claude-your-approved-model
    virtualModel: assistant
    priority: 1

This uses Anthropic's native Messages API through its SDK. Anthropic requires an output-token limit: when the request omits one, this adapter uses 4096 tokens; send an explicit lower limit when appropriate. The existing anthropic-compat profile remains available under openai-compatible for deployments that intentionally use that compatibility layer. Native Anthropic does not provide embedding or reranking operations.

AWS Bedrock

providers:
  bedrock:
    type: aws-bedrock
    region: eu-west-1

models:
  - id: assistant-bedrock
    provider: bedrock
    providerModel: approved-bedrock-model-or-inference-profile
    virtualModel: assistant
    operation: generation
    profile: aws-bedrock-converse
    priority: 1
  - id: knowledge-bedrock
    provider: bedrock
    providerModel: amazon.titan-embed-text-v2:0
    virtualModel: knowledge
    operation: embedding
    profile: aws-bedrock-titan-embed-v2
    priority: 1
  - id: ranking-bedrock
    provider: bedrock
    providerModel: approved-bedrock-reranker-model-arn
    virtualModel: search-ranking
    operation: rerank
    profile: aws-bedrock-rerank
    priority: 1

The AWS SDK uses its default credential chain in the gateway process. Prefer a workload role appropriate to the deployment. Grant access only to approved models and the required Converse/streaming, InvokeModel or reranking operations. Model access and regional availability must also be enabled in AWS.

Generation uses Converse; embedding uses the explicit Titan Text Embeddings V2 dialect. Other Bedrock embedding dialects are not inferred from a model name. A Titan batch becomes bounded individual calls; see Embed and rerank text for partial-batch accounting.

Azure OpenAI

providers:
  azure:
    type: azure-openai
    endpoint: https://example-resource.openai.azure.com
    apiVersion: your-supported-api-version
    authentication:
      type: default-credential

models:
  - id: assistant-azure
    provider: azure
    providerModel: chat-deployment
    virtualModel: assistant
    priority: 1
  - id: knowledge-azure
    provider: azure
    providerModel: embedding-deployment
    virtualModel: knowledge
    operation: embedding
    priority: 1

Use the deployment name registered in Azure as providerModel and an API version supported by that resource. default-credential uses Azure's credential chain in the gateway; assign the identity the required inference role on the resource. For API-key authentication, replace the authentication block with:

authentication:
  type: api-key
  credential: env/AZURE_OPENAI_API_KEY

The credential chains' own identity variables, such as AZURE_CLIENT_SECRET, GOOGLE_APPLICATION_CREDENTIALS and every AWS_ name, belong to the gateway and are reserved: a YAML reference to one refuses startup.

Azure OpenAI has no native reranking operation in this adapter.

Google Vertex AI

providers:
  vertex:
    type: google-vertex
    team: example-ai-team
    location: europe-west4
    rankingLocation: global

models:
  - id: assistant-vertex
    provider: vertex
    providerModel: approved-gemini-model
    virtualModel: assistant
    priority: 1
  - id: knowledge-vertex
    provider: vertex
    providerModel: approved-embedding-model
    virtualModel: knowledge
    operation: embedding
    priority: 1
  - id: ranking-vertex
    provider: vertex
    providerModel: approved-ranking-model
    virtualModel: search-ranking
    operation: rerank
    profile: google-vertex-rerank
    priority: 1

Generation and embeddings use the Google Gen AI SDK in Vertex mode. Reranking uses Discovery Engine's ranking API. rankingLocation defaults to global independently of the generation/embedding location. Configure Google application default credentials for the gateway and enable the corresponding APIs and least-privilege resource permissions. The model and location must be supported together. Do not copy service-account key JSON into organization YAML.

Optional connection overrides

Provider	Setting	Behavior
Anthropic	baseUrl, defaultHeaders	Override the SDK base URL or add ordinary provider headers. Configured Authorization/API-key headers cannot replace the resolved credential.
AWS Bedrock	runtimeEndpoint	Override the Bedrock Runtime endpoint for generation/embedding.
AWS Bedrock	controlEndpoint	Override the Bedrock control endpoint for non-inference metadata checks.
AWS Bedrock	agentRuntimeEndpoint	Override the separate Bedrock Agent Runtime endpoint for reranking.
Vertex	apiEndpoint	Gen AI base URL for generation/embedding; not a Discovery Engine endpoint.
Vertex	rankingApiEndpoint	Discovery Engine service hostname for ranking.
Vertex	rankingLocation	Ranking resource location; defaults to global independently of location.
Vertex	rankingTimeoutMs	Finite ranking RPC timeout; defaults to 10000 milliseconds.

Use overrides only for a reviewed provider/proxy endpoint. Leave them absent to use the SDK's normal service endpoints. Configuration changes require preflight and restart; environment credential changes also require a gateway restart.

Expected result

Provider family/profile	Generation	Embeddings	Reranking
openai-compatible	Existing supported subset	Compatible embedding API	Not inferred
anthropic	Native Messages	No	No
aws-bedrock-converse	Converse	No	No
aws-bedrock-titan-embed-v2	No	Titan Text Embeddings V2	No
aws-bedrock-rerank	No	No	Bedrock Rerank
azure-openai	Native Azure OpenAI	Native Azure OpenAI	No
google-vertex	Vertex generation	Vertex embeddings	No
google-vertex-rerank	No	No	Discovery Engine ranking

Each adapter declares the fields, modalities, structured output, streaming and function calls it can actually carry. Capability overrides cannot make an unsupported wire feature available. Bedrock and Vertex generation currently accept text input; they do not download arbitrary image URLs. Requests without an eligible deployment fail with capability_unavailable.

SDK automatic retries are disabled so the gateway remains the retry/fallback authority. Existing retry-safety rules still apply. Client cancellation does not prove that a cloud provider stopped work or waived charges. In particular, the current Discovery Engine SDK may continue a ranking RPC until its configured finite deadline even after the gateway has cancelled its local response. Each physical attempt's usage-ledger fact records provider-reported usage and any configured token-price estimate, not a reconciled invoice.

Test connection remains non-inference. Native metadata probes may need additional read permissions. A provider/operation without a supported non-inference probe displays Connection test unavailable; it never silently runs billable generation to appear healthy. Validate a real inference call separately before making the deployment available to users.

A virtual model used by a managed Agent must retain a generation route. Builder excludes retrieval-only choices, and Save/Publish/Roll back check current generation support. See Build, test and publish.

Verification

1. Confirm the loaded model and provider family/profile on Models.
2. Verify that the gateway identity can access the specific provider resource.
3. Enable the virtual model in a test Team and send an authorized synthetic call.
4. Confirm the requested virtual name is returned, the correct deployment is recorded, and reported usage matches the operation.
5. Remove Team allowance and confirm the next call reaches no provider.
6. Test errors and cancellation with deterministic fixtures before relying on provider failover.

Failure recovery

Symptom	Recovery
Invalid profile or fallback	Match the profile to its provider family and operation. Keep fallbacks within one virtual model and operation.
Credential unavailable	Restore the gateway's environment reference or cloud credential chain; restart when environment changes, then preflight and restart.
Provider account rejected	Check identity permissions, resource endpoint, deployment/model name, billing and regional access.
Capability unavailable	Remove unsupported request fields or select a deployment that explicitly supports them.
Non-inference probe unsupported	Use the documented provider setup and an explicit synthetic inference call; do not interpret the probe limitation as a successful health check.
Unknown cost	Supply approved input/output token prices where applicable. Reranking query/document prices are not represented as guessed token prices.

Related links

• Model administration
• Organization YAML
• Embed and rerank text
• Model guardrails
• Environment variables