DateKeys/testdata/README.md

# DateKeys test data

Official vectors, fixtures and corpora of the DateKeys Protocol Specification
v0.8.2, generated by the reference implementation. Another implementation
consumes them as they are: this file documents every format, so that no Go code
has to be read. The rules that decide each verdict are in the specification;
this file points to them, and states only what belongs to the files
themselves.

```
go run ./internal/testkit/genfixtures -out testdata
```

regenerates everything except the `.dkc` and `.dkk` fixtures, which are
generated once and frozen (spec §67). The local gate (`scripts/check.sh`) and CI
run it and fail if any committed file changes: every file below is exactly what
the implementation computes today.

Conventions for every file:

- JSON in UTF-8, with LF line endings. Binary values are lowercase hex strings.
- `error`, `result` and similar fields hold the normative codes of spec §69, such
  as `ERR_NON_CANONICAL_CBOR`.
- `step` is a step of the reading flow of spec §63, 1 to 18. Steps 1 to 8 are the
  pre-unlock checks (`datekeys inspect`, `capsule.Inspect`): no network, no
  secret.
- The `spec` field names the version of the specification.

| File | Content | Spec |
|---|---|---|
| `vectors/profile_quicknet.json` | Quicknet Provider Profile: its canonical CBOR and `profile_hash` | §11, §12 |
| `vectors/quicknet_rounds.json` | date → round resolution | §15, §16, §65 |
| `vectors/dk1.json` | canonical `dk1_` strings, and rejected encodings with their code | §18, §19, §66 |
| `vectors/cbor.json` | the CBOR profile, and one block of vectors per schema | §58, CDDL |
| `vectors/mutations.json` | the mutation corpus: 23 mutations of §64 and further cases | §63, §64 |
| `vectors/inspect_differential.json` | 1825 mutations of the fixtures with the verdict of steps 1 to 8 | §63 |
| `fixtures/<name>.dkc`, `<name>.json` | official capsules and every intermediate value | §67 |
| `fixtures/<name>.dkk`, `<name>.dkk.json` | official access keys | §68 |
| `fixtures/<name>.plaintext` | the plaintext of each capsule | §67 |
| `fixtures/<name>.inspect.json` | the exact output of `datekeys inspect -json` for each `.dkc` | §63 |

The five official capsules are `time_only`, `time_only_extensions`,
`time_and_key_portable`, `time_and_key_recipients` and `empty_payload`. The
release that opens each one, a published Quicknet signature, is in its
`<name>.json`, so they all decrypt offline.

## Edited files

`mutations.json` and `inspect_differential.json` give each mutated `.dkc` as
edits of a base file, not as its full bytes:

```json
{ "base": "time_only.dkc", "edits": [[4, 1, "02"]] }
```

- `base` is a file of `testdata/fixtures`. In `mutations.json` it may be absent:
  the base is then the empty file, and the single edit holds the whole capsule.
- An edit is `[at, delete, insert]`: the `delete` bytes at offset `at` of the
  base are replaced by the bytes of the hex string `insert`.
- The edits of one file refer to offsets of the unmodified base, are sorted by
  `at` and do not overlap. The result is therefore built in one pass: copy the
  base up to `at`, append `insert`, skip `delete` bytes of the base, go on with
  the next edit, and copy the rest of the base.
- `[0, 78799, ""]` on `time_only.dkc` is the empty file; `"edits": []` is the
  base unchanged.

## `vectors/cbor.json`

```json
{
  "spec": "0.8.2",
  "walk": { "max_depth": 3, "max_len": 64 },
  "accept": [ { "name": "uint 2^53 eight bytes", "hex": "1b0020000000000000", "value": "9007199254740992" } ],
  "reject": [ { "name": "tag", "hex": "c101", "error": "ERR_NON_CANONICAL_CBOR" } ],
  "schemas": [ { "block": "extension", "schema": "public_header", "name": "65 extensions", "hex": "a600…", "result": "ERR_NON_CANONICAL_CBOR" } ]
}
```

### Generic vectors: `accept` and `reject`

Each `hex` is checked as exactly one data item of the CBOR profile of spec §58:
major types 0, 2, 3, 4 and 5 only; integers and lengths in their shortest form;
definite lengths; map keys that are unsigned integers in strictly ascending
order; valid UTF-8 text; nothing after the item. `accept` holds the inputs that
pass, `reject` those that fail, all with `ERR_NON_CANONICAL_CBOR`.

The `walk` limits apply as well, as in the reference `codec.Walk`: containers
nest at most `max_depth` deep (a scalar has depth 0, `81818100` has depth 3),
and every byte string and text string has at most `max_len` bytes, every array
at most `max_len` items and every map at most `max_len` entries. The vectors
named "above max_len" or "above max_depth" fail on these limits only.

`value` is present for an accepted unsigned integer: a JSON number up to
2^53 − 1, and a decimal string above, so that no reader loses precision.

Among them: shortest-form boundaries, `a200010101` (the two-key map
`{0: 1, 1: 1}`), a text with a leading BOM, keys out of order or repeated,
non-integer keys, indefinite lengths, tags, floats, simple values, negative
integers, truncation, lengths beyond the input, trailing bytes, overlong UTF-8
and surrogates.

### Schema vectors: `schemas`

Each vector is one encoded object:

- `schema` names the object and the decoder to run on `hex`:
  - `provider_profile`: a Provider Profile (§11), decoded and then validated
    as a profile to pin, by rules 1 to 3 of spec §12.1 in their order: the
    CDDL with the `period` limit of the reference, the field rules
    (`ERR_UNKNOWN_PROFILE`) and the chain-hash self-check
    (`ERR_PROFILE_MISMATCH`), whose formula §12.1 gives. No `profile_hash` is
    expected (rule 4). A vector that changes a hashed key recomputes
    `chain_hash`, unless its name says that the chain hash no longer matches.
  - `public_header`: PUBLIC_HEADER (§24). No profile registry is consulted and
    no extension is known: a header naming an unpinned profile is valid here
    (`ERR_UNKNOWN_PROFILE` comes from the registry at step 4), and critical
    extensions are not checked here.
  - `control_cbor`: CONTROL_CBOR (§31).
  - `dkk_body`: BODY_CBOR of a `.dkk` (§41), without the 12-byte DKK1
    prelude.
- `block` names the schema the vector exercises: the same as `schema`, or
  `verification_metadata` (the `.dkk` body's key 6 varies) or `extension` (the
  PUBLIC_HEADER's key 6, `noncritical_extensions`, varies).
- `result` is `ok` or the error code.

When bytes break several rules, the code is the one of the first failing
layer of spec §69.1: the type tag and the schema version, read first from keys
0 and 1 (layer 2), then the CDDL, the implementation limits included, and the
re-encoding (layer 3), and only then the fields with codes of their own in
ascending key order (layer 4). The objects of these vectors have no frame, so
layer 1 does not apply, and their critical extensions are not checked (see
above).

Each block has a minimal valid object, an unknown key, a missing required key, a
wrong type and values out of size or range. The `extension` block has, besides:
data `40` (empty) and `5801xx` (length not in its shortest form), data of every
other type (text, `null`, integer, map, array, tag, indefinite length), 64 and
65 extensions, an `extension_id` starting with a BOM, the pair U+FF61 and
U+10000 in UTF-8 byte order (valid) and in UTF-16 order (invalid), and
`extension_version` 2^32 − 1 (valid), 2^32 and 2^53 (invalid).

#### Implementation limits

The vectors apply the implementation limits of the reference that spec §74
lists: the maximum `extension_id` length, the Provider Profile `period` of one
day, the maximum name lengths and the `public_key` length. The vectors named
"the implementation limit" sit at a limit and are valid; those named "above
the implementation limit" go one past it and are otherwise valid, so that an
implementation without the limit accepts them. The name alphabets, the
`genesis_time` range and the drand rules of the `provider_profile`
validation are not limits: they are rules of spec §12.1. Neither is the
minimum of one byte of `extension_id` (spec §31).

## `vectors/mutations.json`

The mutation corpus of spec §64, as frozen data. Each case is a `.dkc`, what
the reader is given to open it, and the exact error and step at which the full
reading flow (`capsule.Open`, §63) must fail.

```json
{
  "name": "version changed",
  "spec": true,
  "dkc": { "base": "time_only.dkc", "edits": [[4, 1, "02"]] },
  "release": { "round": 1000, "signature": "b446…" },
  "now": "2023-08-23T15:59:24Z",
  "registry": "default",
  "network": false,
  "frozen": false,
  "error": "ERR_UNSUPPORTED_VERSION",
  "step": 2
}
```

- `name`: unique, stable.
- `spec`: true for the 23 mutations listed in spec §64 (the first 23 cases),
  false for the further cases of the reference.
- `dkc`: the capsule, as edits of a fixture (see above). The reader gets it as a
  seekable file, so that the `capsule_digest` of an offered `.dkk` is checked
  before any release request (spec §63 step 9.a).
- `dkk`: the hex of a complete `.dkk` file (prelude and body) offered as the
  access credential; absent when none is offered.
- `identities`: age X25519 identities (`AGE-SECRET-KEY-1…`) offered as access
  credentials; absent when none.
- `release`: what the release source answers to every request, whatever round
  is asked for. The reader must verify it (§51): a case may serve a release of
  another round, or a round with the signature of another. `null` means that
  no release is available (`ERR_RELEASE_UNAVAILABLE`).
- `now`: the reader's clock, RFC 3339. No release is requested before the round
  time of the DateKey.
- `registry`: `default` pins exactly the Quicknet profile of
  `profile_quicknet.json`; `empty` pins none.
- `extensions`: the extensions the application implements. Each entry is known
  at `(id, version)`, and its data is valid only when it equals the bytes of
  `valid_data`. Absent: the application knows no extension, the state of the
  base protocol V1.
- `network`: whether the failure may come after a release request. When false,
  the reader must fail without requesting any release (§27, §63): every failure
  of steps 1 to 8 and of step 9 before the request (9.a to 9.c).
- `frozen`: the capsule was built once with age randomness; its bytes are kept
  and never regenerated. These cases have no `base`.
- `error`, `step`: the expected code and the step of §63 that fails.

Every case reproduces offline: the recorded release stands in for the network.
A reader that implements only steps 1 to 8 can replay every case whose `step` is
at most 8: 31 cases, 13 of them from §64. Steps 1 to 8 are summarised in "The
checks of steps 1 to 8" below.

### Credentials and the release: step 9

What happens between step 8 and the release request is spec §63 step 9: for
`time_and_key` only, an offered `.dkk` is checked as an object and then bound
to the capsule (9.a), at least one credential must be offered (9.b), and for
either policy a `now` before the round time of the DateKey fails without a
request (9.c); only then is the release requested. For `time_only` the
credentials play no part: the §64 case "access_policy=time_only with
time_and_key structure" offers a `.dkk` whose `capsule_digest` is that of the
unmutated capsule, and fails at step 12, not at step 9. The codes after the
request, for the release (step 10) and for the identities that open each age
file (steps 11, 13 and 17), are those of spec §63 as well. In this corpus:

- identities are not examined before the release (spec §63 step 13);
- a `release` of `null` is `ERR_RELEASE_UNAVAILABLE` at step 9;
- every `.dkk` offered decodes: the corpus checks step 9.a, not the decoding
  of a `.dkk`, whose errors spec §63 also places at step 9.a.

## The checks of steps 1 to 8

`mutations.json` and `inspect_differential.json` follow the rules of the
specification for steps 1 to 8, with the `default` registry (Quicknet pinned),
no extension known unless a case lists some, and no secret. The first failure
ends the flow, and within one object the first failing layer decides the code
(spec §69.1):

| Step | Rules | Spec |
|---|---|---|
| 1, 2 | magic, truncated prelude, framing version, FLAGS and RESERVED, `PUBLIC_HEADER_LEN` in 1 to 1048576 and `SEALED_CONTROL_LEN` in 1 to 67108864 | §22, §23 |
| 3 | the PUBLIC_HEADER bytes are present | §23 |
| 4 | PUBLIC_HEADER: layers 2 to 4, the `public_header` decoder of the schema vectors, then the pinned profile of the DateKey and the critical extensions; with no extension known, every `critical_extensions` array fails | §63, §69.1 |
| 5 | the SEALED_CONTROL bytes are present, then its age header, parsed within `SEALED_CONTROL_LEN` bytes: malformed is `ERR_INTEGRITY`, one tlock stanza is required | §28.1 |
| 6 | the age header of PAYLOAD_AGE, from its offset to the end of the file: malformed is `ERR_INTEGRITY`, one X25519 stanza is required | §22, §28.1 |
| 7 | the round time of the DateKey is at most 9999-12-31T23:59:59Z (Quicknet: round 83903165811 at most); a `dk1_` round above it passes step 4 and fails here | §15 |
| 8 | the tlock stanza has exactly two arguments, the canonical decimal round and the lowercase hex chain hash, compared as strings | §63 step 8 |

The reference parses age headers with `filippo.io/age` v1.3.2, whose parser
limits (1024 stanzas, 128 arguments after the type, 2 MiB) spec §74 lists as
implementation limits. No case of these corpora depends on them.

## `vectors/inspect_differential.json`

A differential corpus of the pre-unlock checks: 1825 deterministic mutations of
the five official `.dkc` fixtures, with the verdict of steps 1 to 8 of §63 as
the reference computes it (`capsule.Inspect` with the `default` registry, no
extension known, no network, no secret), by the rules that "The checks of
steps 1 to 8" above points to. The file repeats the format below in its
`format` field.

```json
{
  "seed": 20260925,
  "bases": [ { "file": "time_only.dkc", "sha256": "99e9…" } ],
  "mutations": [
    {"base":0,"kind":"flip","edits":[[121,1,"b3"]],"result":"ERR_NON_CANONICAL_CBOR","step":4},
    {"base":0,"kind":"flip","edits":[[725,1,"4d"]],"result":"ok"}
  ]
}
```

- `bases`: the fixtures, with the SHA-256 of their exact bytes. `base` in a
  mutation is an index into this list.
- `edits`: see "Edited files".
- `result`: `ok` when steps 1 to 8 pass, or the error code; `step` is the step
  that failed, absent when `ok`.
- `kind` names the generator and is informative: `flip` (one bit), `byte` (one
  byte replaced), `truncate`, `insert` and `delete` (one to four bytes),
  `length` (PUBLIC_HEADER_LEN and SEALED_CONTROL_LEN), `header` (PUBLIC_HEADER
  re-encoded with one CBOR-aware change: a key removed, added or retyped, the
  type tag, version, `capsule_id`, DateKey or `access_policy` changed, an
  extension array added, a head not in its shortest form, keys out of order or
  repeated, the whole item tagged, wrapped, made indefinite, truncated or
  followed by bytes), `datekey` (the DateKey string alone), `age` (the age
  header of SEALED_CONTROL or PAYLOAD_AGE edited: intro line, stanza type,
  arguments, stanzas added or removed, body lines, MAC line, line endings).
  Most `header`, `datekey` and SEALED_CONTROL `age` mutations also rewrite the
  prelude lengths to match; some keep the old ones on purpose.
- `seed` seeds the generator of the reference and is informative too: every
  mutation is stored explicitly.

## `fixtures/<name>.inspect.json`

For each official `.dkc`, the exact bytes that `datekeys inspect -json -in
<name>.dkc` prints when run in `testdata/fixtures`: JSON indented with two
spaces, fields in this order, and a final newline. The pre-unlock checks use
the `default` registry.

| Field | Content |
|---|---|
| `file` | the `-in` argument, `<name>.dkc` |
| `capsule_id` | hex, once step 4 has decoded the header |
| `datekey`, `profile`, `round` | the canonical `dk1_` string, its profile and round |
| `unlock_at` | the round time of the DateKey, RFC 3339 in UTC, once step 7 passes |
| `access_policy` | `time_only` or `time_and_key` |
| `valid` | true when steps 1 to 8 pass |
| `error` | the code of the failure, absent when valid |
| `checks` | one entry per step run: `step`, `name`, `ok`, `detail` (free text) and, for a failed step, `error` |

`detail` is informative text of the reference; a second implementation compares
at least `step`, `name`, `ok` and `error`, and every other field.

## Existing vectors and fixtures

- `vectors/profile_quicknet.json`: the Quicknet profile fields,
  `canonical_cbor` (hex) and `profile_hash`.
- `vectors/quicknet_rounds.json`: `vectors` of `requested` instants (RFC 3339
  with nanoseconds) and the resolved `round` and `effective` time, or `error`.
- `vectors/dk1.json`: valid `vectors` with `network`, `round`,
  `canonical_json`, `base64url` and `dk1`; invalid ones with `input` and the
  `error` code (`accepted` would mean the input decodes).
- `fixtures/<name>.json`: for each `.dkc`, its SHA-256, the release that opens
  it, the hex of the prelude, PUBLIC_HEADER and CONTROL_CBOR, the DateKey,
  `capsule_id`, `header_binding`, `payload_identity` (I_PAYLOAD, a test
  secret), the visible stanzas of each age file, the identities or `.dkk` that
  open it, the exact extension data, and the result of every step of §63.
- `fixtures/<name>.dkk.json`: for each `.dkk`, its SHA-256, `credential_id`,
  `capsule_id`, `access_type`, `access_material` (a test secret),
  `capsule_digest`, extensions and the capsule it opens.