Data dictionary

Column-by-column reference for the files LLM-Archive publishes. Generated by core/plumbing/render.py from core/measure/schema.py's closed field list — if this file and the code ever disagree, the code is correct and this file needs fixing. For what each term means rather than its exact column name, see glossary.md; for hands-on loading examples, see for-researchers.md.

Every field with _pct in its name is on a 0–100 scale. This is a hard, checked convention (SPEC.md §6) — no published field or threshold mixes the 0–1 and 0–100 scales.

`stability.csv`

One row per measurement. One column per key below, in the order they appear in MEASUREMENT_FIELDS.

Column	Type	Meaning
`record_type`	enum	Always `"measurement"` in this file.
`schema_version`	int	Which version of this schema produced the row (currently 2).
`run_id`	text	Unique id for this run, e.g. `2026-09-20__risky_choice_framing__wording__v0__claude-sonnet-5__r0`.
`run_date`	date	The (absolute, never relative) date the run happened.
`protocol_id`	text	Which protocol this is, e.g. `risky_choice_framing__wording__v0`.
`rewording_type`	enum	One of `wording`, `anchoring`, `order`, `default`.
`subject_model_id`	text	The exact model id that was measured.
`stability_pct`	pct	The headline number. `100 − gap_pct`.
`ci_low_pct` / `ci_high_pct`	pct	95% bootstrap interval on `stability_pct`, conditional on this panel (see glossary).
`gap_pct`	pct	Total variation distance between the A and B response distributions.
`gap_null_pct`	pct	Gap between A and A′ — the null-change control.
`gap_positive_pct`	pct	Gap between A and C — the positive control.
`null_floor_pct`	pct	Expected gap from sampling noise alone, at this panel's `n` and `k`.
`at_noise_floor`	bool	True if `stability_pct` is indistinguishable from perfect invariance.
`n`	count	Responses collected per version, frozen per protocol — 30 in the `v0` series, 120 in the `v1` series.
`n_valid`	count, per version	How many of the `n` responses per version were `valid`.
`n_items`	count	Number of scenarios in the panel (15 in v0).
`k`	count	Number of options in the decision (usually 2).
`entropy_a` / `entropy_b`	0–1	Normalized entropy of the A and B answer distributions.
`u_a` / `u_a_prime` / `u_b` / `u_c`	0–1	Fraction of lost responses, per version.
`drop_bound_pct_ab` / `_aa` / `_ac`	pct	Drop bound — worst-case error from lost responses, per pair (A↔B, A↔A′, A↔C).
`drop_asymmetry_pct_ab` / `_aa` / `_ac`	pct	How unevenly the two sides of each pair lost responses.
`scenario_gaps_pct`	list of pct	One gap value per scenario (`item_id`), as a JSON array in this cell.
`scenario_spread_pct`	JSON object	`{min, max, median, iqr}` of the 15 scenario gaps.
`scenario_concentration_pct`	pct	Share of the total gap produced by the two worst scenarios.
`outcomes`	JSON object	Full outcome counts (`valid`, `unparseable`, `refused`, ...) per version, as one JSON cell.
`decisions`	JSON object	The extracted decision counts per version, as one JSON cell.
`flags`	list	Which of the closed flag set applied to this measurement (see glossary — empty list if none).
`gates`	JSON object	Pass/fail detail for every check, one entry each.
`on_curve`	bool	Whether this measurement counts toward the main published time series.
`panel_sha256` / `protocol_sha256`	sha256	Fingerprints proving the scenario content and full protocol definition weren't edited after the fact.
`extractor_sha`	sha256	Fingerprint (sha256) of the file that holds the decision grammar used.
`analysis_code_sha`	sha256	Fingerprint of the exact `core/measure/` code that computed this row.
`schema_sha256`	sha256	Fingerprint of the closed schema itself.
`grammar_version`	int	Version of the decision grammar (the `DECISION:` line and how it is read) — 1 in the `v0` series, 2 in the `v1` series (version 2 counts a line as a decision line only if what follows the colon is already a valid option; version 1's behaviour is unchanged).
`series`	enum	`commercial` or `open_weights` — never mixed on one chart.
`lane`	enum	`open`, `guard`, or `sealed`.
`twin_id`	text	The paired protocol's id, if this protocol has a twin (see glossary); empty otherwise.
`model_family`	text	The wider product line this model belongs to.
`returned_model_id`	text	The model id the API actually reported back, for cross-checking against `subject_model_id`.
`api_surface_sha`	sha256	Fingerprint of the exact API call shape used (parameters, endpoint version).
`condition_profile`	enum	Whether this run used a bare API call, a system instruction, or tools — recorded so a future reader can separate eras of measurement conditions.
`replicate_index`	int	Which repeat this is, for occasional same-window re-scans used to empirically check `null_floor_pct`.

A field that isn't a plain scalar (n_valid, outcomes, decisions, scenario_gaps_pct, scenario_spread_pct, gates, flags) is written as one JSON cell, not split across several CSV columns — parse that cell with any JSON library once loaded.

`outcomes.csv`

The same outcomes field stability.csv already carries, reshaped into one row per (run_id, version, outcome, count) — easier to pivot, filter, or chart without a JSON-parsing step.

Column	Meaning
`run_id`	Same as `stability.csv`.
`version`	`A`, `A_prime`, `B`, or `C`.
`outcome`	One of the closed outcome list: `valid`, `unparseable`, `refused`, `blocked_upstream`, `truncated`, `empty`, `off_format`.
`count`	How many of that version's responses had that outcome.

`open-lane/<year>.jsonl`

For every measurement whose lane is open, the full, unmodified per-trial records, one JSON object per line, grouped by the run's calendar year. Nothing from guard or sealed runs ever appears here, at any stage — those trials aren't even read when building this file, not just filtered out afterward.

What a trial record contains (exactly these keys, no others): record_type, run_id, index, version, scenario_id, rep, prompt_sha256, response_text, stop_reason, returned_model_id, error, outcome, token, reason.

What it does not contain: the prompt text itself. Each record carries prompt_sha256 — a one-way hash that lets you verify a prompt you already hold, but which cannot reveal the wording — not the sent prompt. So these records are enough to re-run the decision grammar over response_text and recompute every published statistic, and not enough to reconstruct the questions.

Where an open protocol's wording is published today: the full four-version text of the panel's first scenario only, embedded in the published results page (open the protocol's row on the site). The other 14 scenarios' wording is in no published artifact yet, and the protocol definitions are not published, even for open protocols, per SPEC.md §10. See reproducing.md for what this does and doesn't let an outside reader do.

`croissant.json`

A MLCommons Croissant metadata file describing stability.csv, outcomes.csv and open-lane/<year>.jsonl (the last as a cr:FileSet over the per-year glob, with one field per trial key, extracted by jsonPath rather than by CSV column) so that dataset-discovery tools can read this project's schema automatically instead of needing it explained by hand. Every column's declared Croissant type is derived mechanically from the same field-type label this dictionary uses: pct/unit_interval → sc:Float, int/count → sc:Integer, bool → sc:Boolean, everything else → sc:Text.

`coverage.csv`

Which scheduled jobs ran, which didn't, and — when one didn't — the single named cause from the closed list (see the glossary's "gap causes"). One row per recorded observation, written the day it happened from core/schedule/coverage.py's append-only log; it is deliberately not backfilled, so it starts when that tracking started rather than reconstructing history by guessing. Scoped to the four jobs the scheduler actually knows how to run (full_sweep, subject_fingerprint, runtime_fingerprint, guard_margin_rotation), not to every cadence in cadence.yaml.

Column	Meaning
`job`	Which scheduled job the row describes.
`date`	The day the observation was recorded.
`ran`	Whether it actually ran that day.
`cause`	Empty when it ran; otherwise one gap cause from the closed list.

Where the reference model's measurements live

There is no separate open-weights/stability.csv, and no instrument.csv — earlier drafts of SPEC.md §10 named both, and neither was built. Both series share stability.csv, separated by its series column (commercial or open_weights). Filter on it before charting: the two are never meant to be drawn as one line, and the project's own site keeps them in two separate tables for exactly that reason.