Skip to content

Export and import evaluation evidence

Purpose

Move an evaluator definition between Teams or export an experiment for analysis. For a complete reusable suite of cases, graders, slices and templates, use suite import.

Prerequisites

Exports require evaluation.read; prompts and answer bodies additionally require evaluation.content.read. Evaluator import requires evaluation.manage in the destination Team, a matching active score definition, and permitted models. Use the selected Team; a management client uses an expiring scoped mcm_… token on the private control listener.

Procedure

  1. Choose Export evaluator on an evaluator to download a mission-control.evaluator/v1 file: the judge's name and description, the score definition it answers by name and data type, and every version with its model, its variables, its hashes and its prompt. The file names no id, no Team, no status and no agreement — an agreement was measured against a queue in the Team that measured it. Without evaluation.content.read each version carries its hashes and no prompt at all.
  2. Choose Import evaluator under the Evaluators list and pick a File. The target Team must already hold an active score definition of the same name and data type, and every version's model must be inside that Team's allowance; the judge is created there and its versions are saved again in order, renumbering from 1. Choose Export experiment on an experiment for a mission-control.experiment/v1 file: the record, its items with their outputs, every live score on them, the per-definition summary and, when the address names a baseline, the comparison aggregates. There is no import for it. A job with a token does all of this over the API — see Evaluate a change in CI.

Expected result

Both portability documents are read whole; their identity and score-selection contracts differ. mission-control.evaluator/v1 carries the evaluator, the definition it answers by name and data type, and every version in ascending order; ids, the Team, the status, the current version, the revision, the author and the stored agreement stay behind, so an import creates a judge the target Team could have written itself. An export taken without evaluation.content.read carries the hashes and no prompt, and importing that file is refused rather than half replayed. mission-control.experiment/v1 carries the experiment, its items, the selected pass's live automated scores together with independent manual/API scores, the summary and the comparison a baseline names. Omitted pass selectors use original grading. Pass/evaluator IDs, selected pass kind, creation time and evaluator versions travel as provenance; deterministic original-pass IDs contain the experiment ID and allow correlation, so this document is not anonymized. Separate experiment/item ID fields, the generation starter and thread/decision identities remain omitted. Run costs include all passes. The experiment document carries output hashes and exposes output bodies only with evaluation.content.read, and it has no import at all: an experiment records work that was executed in this Organization; it cannot be imported. Score comments never travel in it — Export CSV on the Scores page is where those live. Both files are written with their keys sorted at every depth, so two exports of the same state are the same bytes.

The documented management API supports evaluation automation with a scoped mcm_… token on the private control listener. Console sample tests and human review/calibration flows do not have equivalent public routes. Management mutations are attributed to the token’s human owner; there is no machine-only writer, and revoking that person’s membership revokes their tokens.

Verification

Inspect the exported format and whether it contains bodies or only hashes. For an evaluator import, compare the version count, model and score definition in the destination. Do not assume calibration transfers. Exporting does not execute work; importing a definition needs no restart or new session.

Failure recovery

Import format must be mission-control.evaluator/v1 means the file is a dataset export, an experiment export or something else; Import needs the judge prompt; export again with evaluation content access. means the file was exported by somebody who could not read prompts, so export it again as somebody who can. Import needs an active score definition named "…" with data type … in this Team. names exactly what the target Team is missing: create that definition with that data type first, or archive nothing and rename nothing — the binding is the name and the type, never an id. An evaluator with this name exists counts the archived ones. A judge model refusal names the model and the reason the Team allowance gives, and no evaluator is created when it is raised. The same holds for a prompt the file carries that this Team would refuse on its own — A judge needs a user prompt for a blank user half, Unknown variable for a placeholder outside the supported judge variables, or a prompt over its size — each is settled before the first row is written, so the corrected file imports under the same name. Should an import stop after that — the Team's allowance narrows or somebody edits the new judge while the file is still replaying — you are left with an evaluator holding fewer versions than the file: archive that one, rename the evaluator in the file, and import it again, since an archived judge keeps its name. An experiment records work that was executed in this Organization; it cannot be imported. is the whole answer for the experiment document: bring its numbers in as a dataset or read them where they were produced.