Current Release¶
This page summarizes the capabilities available in the current release. Availability in your installation depends on its version, configuration and your permissions. Start with Get started, or use the guides below for setup and detailed behavior.
Supported capabilities¶
| Capability | Available behavior | Guide |
|---|---|---|
| Operational Overview | Organization and Team context, request activity, spend and budget reporting | Overview |
| Gateway access | User and service-account virtual keys with scoped access, expiry and rotation | Virtual keys |
| Models and retrieval | Chat Completions, streaming, structured output, a stateless Responses subset, embeddings and reranking; native Anthropic, Bedrock, Azure and Vertex providers | Models, native providers, retrieval |
| Governed tools on model calls | Let models use explicitly selected, permitted MCP tools | Governed tools |
| Connector operations | Tool discovery, tool sets, five connection states and schema drift inspection | Connectors |
| Connector authentication | Static credentials, Shared MCP OAuth and each person's own connection | Shared MCP OAuth, personal connections |
| Rules and approvals | Model guardrails, tool argument/result rules, administrator approvals and conversation consent; accepted tool dispatch is consumed exactly once | Guardrails, Approvals |
| Managed Agents | Build, preview and publish Agents; manage live versions, Chat audiences and application access | Builder, Chat |
| External integrations | Connect existing model/MCP clients and export application traces through OpenTelemetry | Integrations |
| Human administration | Eight built-in role presets, bounded custom Organization/Team roles, invitations, scoped management tokens and optional enterprise sign-in/provisioning | Permissions, Enterprise identity |
| Budgets and usage | Spend, rate and concurrency limits, with separate known, estimated and unknown costs | Budgets |
| Observability | Requests, Traces, Sessions and Usage, with separate metadata, captured-content and export permissions | Observe activity |
| Evaluation | Scores, versioned datasets, experiments, model comparisons, graders, review and publication gates | Evaluate |
| Self-hosting | Source, release-artifact and signed container deployment; PostgreSQL storage, backups and upgrades | Self-hosting |
Identity and funding¶
API access follows catalog ∩ Team allowance ∩ key policy. Human management roles and virtual-key inference access are separate. Removing a person's membership invalidates its keys; rejoining does not restore them.
Managed production Agents use the publisher's Team funding. Chat audiences and API access are configured independently. An API caller needs a permitted virtual key with deployment.invoke. See Agent access and roles and permissions.
Application and management APIs¶
The inference listener exposes exactly seven paths for application work:
| Method and path | Purpose |
|---|---|
GET /v1/models |
Authorized model discovery |
POST /v1/chat/completions |
Chat Completions |
POST /v1/responses |
Stateless Responses |
POST /v1/embeddings |
Embeddings |
POST /v1/rerank |
Reranking |
POST /mcp |
MCP discovery and tool calls |
POST /v1/deployments/{deploymentId}:invoke |
Invoke a published Agent |
Chat Completions remains supported. Responses requires store: false; stored response chains, background execution and provider-hosted tools are unsupported. Governed-tool streaming is unavailable. MCP supports revision 2026-07-28 with stateless POST.
Management APIs use scoped human tokens on a separate private listener. OAuth discovery metadata is public. See the API reference for authentication, compatibility and errors.
Builder and Chat¶
Agent workspaces provide Build, Live, Evaluate, Activity and Settings. Save a snapshot for private Preview, then use Review & publish to review execution access, accounts, funding and audiences before publication. Live deployments support access changes, account management, limits and rollback.
Chat keeps each person's conversation history private. Preview and live testing are separate, and model/tool execution can incur charges. See Build and publish an Agent and Chat.
Observe and capture¶
Inspect requests and usage, follow model/tool activity in Traces, and group related work into Sessions. Captured-content access (observability.content.read) and export access (observability.export) are separate permissions. External trace export requires telemetry.ingest.
Capture and retention follow your Organization's policy. Credentials are excluded from captured content. Disabling trace collection preserves request and usage records. See Observe activity and Security.
Evaluation evidence¶
Evaluation supports these workflows:
- Versioned datasets with versions, curation from Traces, Sessions and Requests, and JSON/CSV portability. Explicit case identity, context and execution settings preserve source provenance. Versioned dataset slices select repeatable subsets. Evaluation suite bundle import reconciles datasets, slices, graders and templates together. See Datasets.
- Run experiments on Agent snapshots and published versions, with readiness checks, paid smoke tests, immutable run manifests, live progress and cost coverage. Comparison groups and reusable versioned templates keep model comparisons repeatable. See Experiments, comparisons and run summaries.
- Guided grader setup for LLM judge evaluators, deterministic assertions and read-only tool validators. Test and calibrate graders before relying on their scores. See Graders.
- Retry selected failures or regrade saved answers while retaining earlier evidence. See Retry and Regrade.
- Configure auto-evaluations that sample production Traces and review queues that grade a judged experiment or sampled activity by hand. See Auto-evaluation and Review.
- Require a publish gate on the deployment preflight, using evidence from the exact version being promoted. JSON export of an evaluator and an experiment supports sharing authorized evidence. See Publication gates and Export and import.
Generation, grading, reliability, quality and cost coverage remain distinct. Missing evidence or unknown cost does not become a successful score or zero spend.
Configuration, readiness and storage¶
Configuration uses reviewed YAML loaded when services start. Reusable model guardrail policies can set an Organization baseline and additional virtual-model rules. Changes require validation and coordinated restart; permissions and access restrictions are checked against current authority.
The services use one PostgreSQL database, with migrations required before startup. Operators can monitor health and metrics, back up data and run several gateway instances. See Configuration, Health and Scaling.
Verified release artifacts¶
Deploy from source, verified release artifacts or signed container images with software bills of materials. Upgrades support backups and compatible rollback; database restoration is a separate operation. Older SQLite installations require the one-time migration procedure.
See Self-hosting, Upgrade and rollback, Release notes and Migrate from SQLite.
Not included in the current release¶
- External evaluation application runners and authenticated application-chat verification.
- Provider-hosted tool execution or parsing as governed sub-operations.
- Stateful/background Responses or provider fallback that bypasses declared capabilities.
- Trace/Session lifecycle APIs, live tailing and automatic general migration of old inference data.
- Strict distributed no-overrun budgets, billing reconciliation and arbitrary workflow/checkpoint execution.
- Delegated token exchange for upstream authentication.
See Planned capabilities for capabilities outside this release.