DateKeys/CHANGELOG.md

# Changelog

All notable changes to this module are documented here. The project follows
semantic versioning; `v0.x` versions make no API stability promise.

## Unreleased — specification v0.8.2

Moves the module to the DateKeys Protocol Specification v0.8.2, whose one
normative change closes the extension format (spec §76), refined before
release (see "Specification refinements"). Framing and schema versions do not
change.

The CBOR library is replaced by a codec of the module's own, without
reflection or dependencies. Every valid object encodes to the same bytes as
before: the official vectors and fixtures are unchanged, and every error code
and inspection step of the test suite and the mutation corpus is the same.

### Specification refinements

v0.8.2 is unreleased, so these refinements amend it without a version change;
spec §76 records each with its reproducible cases. No valid object changes,
nor the verdict of any existing official vector or fixture; `dk1.json` gains
three vectors for the reading rules of §19.

- Layered error precedence (spec §69.1, with §57 and §63): within one object
  the code of the first failing layer is reported: frame, then type tag and
  schema version (keys 0 and 1), then the CBOR profile and the CDDL, then the
  fields with codes of their own in ascending key order; the rules with
  codes of their own belong to the last layer, not to the CDDL layer. Across
  objects and steps, the step order of §63 decides; step 9 now spells out its
  order (the `.dkk` as an object, then its binding to the capsule, then
  credentials, then the round time, then the request), and the errors of a
  `.dkk` come at step 9.a even when it is decoded earlier, and never under
  `time_only`. Only the optional steps 5, 6 and 8 and the `capsule_digest`
  check can change the code. Steps 10, 11, 13 and 17 give the codes of the
  release verification (round, then signature) and of each identity.
- Trust model (spec §55.1): who can write PUBLIC_HEADER, CONTROL_CBOR,
  PAYLOAD_AGE and the `.dkk` body, from which step each is bound and by what,
  and what none of them proves. Spec §72: an extension with security-relevant
  claims lives in CONTROL_CBOR or is signed by a signature extension; the
  data of `.dkk` extensions is advisory for its holder only.
- One ordering rule for extension arrays (spec §31, §54, CDDL): strictly
  ascending unsigned bytewise order of the UTF-8 bytes of `extension_id`, a
  proper prefix first, never UTF-16 code units or a locale collation.
- Rules that only `testdata/README.md` stated are now normative text: the age
  header grammar of C2SP and its codes (§28.1, §36), the round-time bound of
  9999-12-31T23:59:59Z and pre-genesis instants (§15), the reading rules of
  `dk1_` (§19: either Base64 alphabet but no CR or LF, one JSON object
  without a byte order mark, numbers by their exact decimal value), lengths
  of at least 1 (§22, §23, §40, §57), the comparison of the tlock stanza
  arguments (§35, §63 step 8), the Provider Profile rules and the chain-hash
  formula (§12.1, with `period` at most 2^32 − 1 and the name alphabets
  normative; the name lengths stay implementation limits), `extension_id` of
  at least one byte (§31), a repeated X25519 ephemeral share in
  INNER_ACCESS_AGE (§36), and the implementation limits of the reference
  (§74).

These error codes of the reference change, for inputs that no existing
official vector holds:

- `capsule.Open` checks an offered `.dkk` as an object before binding it to
  the capsule: `access_type` and `access_material`, then its critical
  extensions, then `capsule_id`, then `capsule_digest`. A `.dkk` for another
  capsule with an unknown critical extension is now
  `ERR_EXTENSION_CRITICAL_UNKNOWN` (was `ERR_ACCESS_INVALID`), and one with
  an unsupported `access_type` and an unknown critical extension is
  `ERR_ACCESS_INVALID` (was `ERR_EXTENSION_CRITICAL_UNKNOWN`).
- `accesskey.Decode` rejects a `BODY_LEN` of 0 with `ERR_INTEGRITY`, like a
  `PUBLIC_HEADER_LEN` of 0; it was `ERR_NON_CANONICAL_CBOR`, the empty body
  failing to decode.
- `datekey.Parse` rejects CR and LF in a `dk1_` string with
  `ERR_DATEKEY_INVALID`; the Go Base64 decoders skipped them, and the
  result was `ERR_DATEKEY_NON_CANONICAL`.
- `datekeys decrypt -dkk` hands the `.dkk` to `capsule.Open` still encoded,
  through the new `OpenOptions.AccessKeyFile`, so its decoding errors come at
  step 9 of a `time_and_key` capsule, after any failure of steps 1 to 8, and
  a `time_only` capsule ignores it; the CLI used to fail on it before
  reading the capsule.
- `capsule.Open` ignores nil entries of `OpenOptions.Identities`: with no
  other credential, the result is `ERR_ACCESS_REQUIRED` at step 9, before
  the clock is consulted, instead of `ERR_RELEASE_UNAVAILABLE` or a release
  request.
- `agewrap.AccessIdentity` tries every identity on every stanza: an identity
  that unwraps two INNER_ACCESS_AGE stanzas is
  `ERR_POLICY_STRUCTURE_MISMATCH` even when another identity unwraps exactly
  one, whatever their order; the file used to open when the other came
  first.
- `profile.NewRegistry` encodes and decodes each profile before comparing its
  `profile_hash`, so a profile gets the code `profile.Decode` reports: a
  `period` of 86401 s is `ERR_NON_CANONICAL_CBOR`, as in `Decode`, not
  `ERR_UNKNOWN_PROFILE`. `Profile.Validate` checks the schema rules a value
  can break (a period that is not a whole number of seconds in 1..86400, a
  genesis time outside 0..2^53−1, a name that is not valid UTF-8) first,
  with `ERR_NON_CANONICAL_CBOR`.

Tests: `capsule.TestPrecedenceWithinPublicHeader`, `TestPrecedenceAcrossSteps`,
`TestFrameLengthLowerBounds`, `TestMalformedAgeHeaders`,
`TestTlockStanzaArgumentComparison`, `TestAccessKeyCheckOrder`,
`TestAccessKeyFileAtStep9`, `TestControlCriticalBeforeHeaderBinding` and
`TestTrustModel`; `accesskey.TestDecodePrecedence`;
`agewrap.TestAccessIdentityStrictness`; `datekey.TestReadingRules`;
`profile.TestDecodePrecedence` (a G1 point outside the prime-order
subgroup), `TestPinPathMatchesDecode` and `TestChainHashFormula`;
`provider.TestVerifyRejects`; `extension.TestOrderIsUnsignedBytewise`;
`cmd/datekeys.TestDecryptAccessKeyOrder`.

### Breaking changes

- `extension.New(id, version, data []byte)` takes the opaque data bytes instead
  of a value that it encoded as CBOR, and rejects nil or empty data. An
  extension without data is the literal `extension.Extension{ID, Version}`,
  which omits key 2.
- Extension data (key 2) must be a byte string of at least one byte. Any other
  CBOR type, `null` or `h''` at key 2 is now `ERR_NON_CANONICAL_CBOR`, so a
  v0.8.1 object with such data no longer decodes. The base protocol never
  decodes the content (§54).
- `codec.Valid` and its fuzz target `codec.FuzzValid` are removed: nothing
  decodes extension data any more.
- Package `codec` is rewritten without reflection, struct tags or dependencies
  (§58). Removed: `Marshal`, the reflection-based `Unmarshal(data, v)` and
  `Peek(data, v)`, and `MaxNestedLevels`, `MaxArrayElements` and
  `MaxMapPairs`. Each schema now writes its encoding with a `codec.Encoder`
  (`Map`, `Array`, `Uint`, `Bstr`, `Text`, `Out`, with a sticky first error,
  and `Fail`, which records an error of the schema so that `Out` never returns
  bytes its decoder rejects) and reads it with a strict `codec.Decoder`
  (`NewDecoder`, `Map`, `Key`, `EndMap`, `Array`, `Uint`, `Bstr`, `Text`,
  `Done`). `codec.Unmarshal(in, decode, encode)` runs the decoder of a schema
  and requires that its re-encoding reproduces the input; `codec.Peek(in)`
  returns the type tag, of at most `codec.MaxTypeTagLen` (64) bytes, and the
  schema version; `codec.Walk(in, maxDepth, maxLen)` checks that bytes are one
  item of the §58 profile, for vectors, fuzzing and diagnostics.
  `CheckSchema` and `MaxSafeUint` keep their names.
- `extension.Wire` and its `UnmarshalCBOR` are removed. `extension.Encode`
  becomes `extension.Canonical`, which returns the validated array in
  canonical order as `[]Extension`; `extension.Decode([]Wire)` becomes
  `extension.DecodeArray(*codec.Decoder)`, which reads and validates one array
  and rejects more than 64 entries from the array head, before reading any;
  `extension.EncodeArray(*codec.Encoder, []Extension)` writes one, and
  records in the Encoder, instead of writing it, an array that `DecodeArray`
  would reject.
- `codec.CheckSchema` reads keys 0 and 1 only, and nothing after them (§70):
  the map head, key 0, a type tag of at most 64 bytes, key 1 and the version
  must be in the profile, each head in its shortest form, and the version at
  most 2^53−1. A schema version other than the expected one read that way is
  `ERR_UNSUPPORTED_VERSION` whatever follows it; it was
  `ERR_NON_CANONICAL_CBOR` when the rest of the object was malformed. Every
  other form of the version is now `ERR_NON_CANONICAL_CBOR`, where it was
  `ERR_UNSUPPORTED_VERSION` whenever the value read was not the expected one:
  a missing version, `null` or `undefined`, a version not in its shortest
  form, a version above 2^53−1 (up to 2^64−1), a version that is not the
  second key (placed before key 0 or after another key), and a version
  behind a map head or a type tag head not in its shortest form. `true` and
  `false` were already `ERR_NON_CANONICAL_CBOR`.
- `capsule.DecodeHeader` checks every CDDL rule of PUBLIC_HEADER, including
  `access_policy`, the extension arrays and the cross-array rule, before it
  parses the DateKey (§57, §63 step 4). A header that breaks both reports
  `ERR_NON_CANONICAL_CBOR` where it reported `ERR_DATEKEY_INVALID` or
  `ERR_DATEKEY_NON_CANONICAL`; a header with one fault keeps its code.
- `profile.Profile.CanonicalCBOR` and `Hash` refuse a `profile_id`,
  `provider`, `network` or `scheme` that is not valid UTF-8, with
  `ERR_NON_CANONICAL_CBOR`: they wrote it as an invalid text string. Every
  encoder refuses such text.
- At most 64 extensions per array and `extension_version` at most 2^32−1, on
  encode and decode (`ERR_NON_CANONICAL_CBOR`). An `extension_id` appears at
  most once per object, and arrays are ordered by `extension_id` only.
- Provider Profile: `genesis_time` is an unsigned integer, and `period` and
  `genesis_time` are at most 2^53−1; a negative or larger value is
  `ERR_NON_CANONICAL_CBOR`.
- `null` in a byte-string field is `ERR_NON_CANONICAL_CBOR` (for example a
  `null` `access_material` was `ERR_ACCESS_INVALID`): nil byte strings, arrays
  and maps now encode as empty ones, never as `null`.
- `capsule.EncodeHeader` and `capsule.DecodeHeader` enforce the 1 MiB
  PUBLIC_HEADER limit and `accesskey.DecodeBody` the 16 MiB BODY limit. Every
  frame-limit refusal, including those of `accesskey.MarshalBody` and of
  `capsule.Encrypt` for SEALED_CONTROL, now wraps `ERR_INTEGRITY` (§57).
- `profile.Profile.CanonicalCBOR` refuses a `period` or `genesis_time` outside
  the schema with `ERR_NON_CANONICAL_CBOR`, as `profile.Decode` does.
- The official fixture `time_only_extensions` is regenerated: its header data
  is the raw UTF-8 bytes of "public label" and its control data is
  `{0: 7, 1: "sealed"}` (`a2000701667365616c6564`). Every other `.dkc` and
  `.dkk` keeps its bytes; the fixture and vector metadata name spec 0.8.2.

### Added

- `capsule.OpenOptions.AccessKeyFile`, a `.dkk` still encoded, which `Open`
  decodes at step 9.a and only for a `time_and_key` capsule (§63, §69.1).
- `ErrExtensionDataInvalid` (`ERR_EXTENSION_DATA_INVALID`, §69).
- `extension.DataValidator`, an optional interface of a `Registry` that
  validates the data of the extensions it knows: a known critical extension
  with invalid data fails with `ErrExtensionDataInvalid` (§63 steps 4 and 14,
  and the `.dkk` check); a known noncritical one is reported in
  `capsule.Inspection.UnusableExtensions`, `capsule.Opened.UnusableControlExtensions`
  or `capsule.Opened.UnusableAccessKeyExtensions` (`extension.CheckNoncritical`,
  `extension.Unusable`) and does not fail.
- Encoder self-checks: `capsule.Encrypt` decodes its PUBLIC_HEADER and
  CONTROL_CBOR, and `accesskey.MarshalBody` its body, with the readers'
  decoders before sealing or writing (§72).
- `extension.MaxExtensions`, `MaxVersion`, `MaxDataLen`, `codec.MaxSafeUint`
  and `codec.MaxTypeTagLen`.
- The `.dkk` fixture `time_and_key_portable_extension`, which carries a
  noncritical extension with data (§68).
- `genfixtures -only NAME[,NAME...]` regenerates the named fixtures only.
- Tests: the three new §64 mutations (data that is not a byte string, `h''`
  data, 65 extensions), regression tests for the cases of §76, conformance
  checks on the exact data bytes, and the fuzz target
  `capsule.FuzzEncodeImpliesDecode` (header, control and `.dkk`).
- The mutation corpus moves from `capsule/mutation_test.go` to
  `internal/testkit.Mutations`, shared by the test and by `genfixtures`. Its
  third-party X25519 identity is now fixed (`testkit.Stranger`), and every
  release source answers with one recorded release, as the exported corpus
  describes it. The CLI's inspect view moves to `internal/inspectview`, which
  `genfixtures` uses to freeze the outputs.
- `internal/cbortest`, an encoder and decoder of generic CBOR values written
  independently of `codec`: tests build with it inputs outside the profile
  and check `codec` against it.
- Tests of the map structure of every schema (key order, required and unknown
  keys, entry counts) and of the codec, whose statement coverage is 100 %.
- `access_policy` values whose low byte is 0 or 1 (256, 257, 65536, 2^32,
  2^53−256…) are tested as undefined, as `FuzzDecodeHeader` seeds and as two
  mutations with a consistent `header_binding` (`testkit.Build.RawPolicy`).
- Tests `extension.TestEncodeArrayRejects`,
  `accesskey.TestEncodeAndDecodeLeaveNoStaleMaterial`,
  `accesskey.TestDecodeShortBodyAllocatesLittle` and `capsule.TestDecryptAll`.
- Shared test data for a second implementation, generated by `genfixtures`,
  regenerated by the gate and documented in `testdata/README.md`:
  - `testdata/vectors/cbor.json`: 36 accepted and 67 rejected items of the
    §58 profile, walked with `codec.Walk` (integers above 2^53−1 as decimal
    strings), and 135 schema vectors: minimal valid object, unknown key,
    missing key, wrong type, size and range for the Provider Profile,
    PUBLIC_HEADER, CONTROL_CBOR, the `.dkk` body, `verification_metadata` and
    extensions (data `40` and `5801xx`, data of every other type, 64 and 65
    extensions, a leading BOM, the U+FF61/U+10000 order, `extension_version`
    2^32−1 and 2^32); schema versions 2^53 and 2^64−1 and the order of type
    tag and version; and the Provider Profile validation (names, public key,
    `genesis_time`, drand scheme, the chain-hash self-check with its formula,
    the `period` limit), each vector keeping the chain hash consistent unless
    it tests the self-check.
  - `testdata/vectors/mutations.json`: the mutation corpus as frozen data, 55
    cases (the 23 of §64 first), each a `.dkc` given as edits of a fixture and
    what the reader is given (`.dkk`, identities, the recorded release, clock,
    registry, known extensions), with the expected error and step. The 16
    capsules built with age randomness are kept from the committed file;
    `genfixtures -only mutations` rebuilds them.
  - `testdata/vectors/inspect_differential.json`: 1825 deterministic mutations
    of the five `.dkc` fixtures (bit flips, byte changes, truncations,
    insertions, deletions, length fields, CBOR-aware header edits, DateKey
    edits, age header edits) with the verdict of steps 1 to 8.
  - `testdata/fixtures/<name>.inspect.json`: the exact output of
    `datekeys inspect -json -in <name>.dkc` for each official capsule.
  - `testdata/README.md` points to the sections of the specification that
    decide each verdict (§12.1, §15, §19, §22, §23, §28.1, §63, §69.1, §74),
    which the refinements above moved into normative text.
- `spec/datekeys.cddl` marks the one-day `period` limit of the Provider
  Profile, which `profile.Decode` already applied, as an implementation limit
  of the reference (§57, §74); spec §11 allows up to 2^53−1.
- Tests that replay them: `codec.TestSharedVectors`,
  `internal/testkit.TestSchemaVectors`, `capsule.TestExportedMutationCorpus`,
  `capsule.TestInspectDifferentialCorpus` and
  `cmd/datekeys.TestInspectJSONGoldens`.
- Fuzz targets `codec.FuzzDecoder` (the Decoder primitives),
  `codec.FuzzWalk` (against the independent decoder), `codec.FuzzPeek`,
  `codec.FuzzEncodeImpliesWalk` and `extension.FuzzDecodeArray`, run by
  `scripts/fuzz.sh`; `codec.FuzzUnmarshal` now fuzzes a hand-written schema.

### Removed

- The dependencies `github.com/fxamacker/cbor/v2` and
  `github.com/x448/float16`. `go.mod` requires nothing new.

### Fixed

- `datekey.Parse` rejects invalid UTF-8 in the `dk1_` JSON at step 2, as
  §19 requires. `encoding/json` replaced it with U+FFFD, so a member that a
  repeated name overwrites passed steps 2 and 3 and ended as
  `ERR_DATEKEY_NON_CANONICAL` instead of `ERR_DATEKEY_INVALID`. Found by the
  second implementation's differential; new `dk1.json` vector.
- `extension.CheckDisjoint` is a linear merge of the two sorted arrays; a
  PUBLIC_HEADER with 40 000 + 40 000 extensions took 8.3 s in the pairwise
  check (§76, case 6).
- Control data made of 14 or 15 nested arrays was sealed by `Encrypt` and
  rejected by `Open` at step 14, after the unlock (§76, case 5).
- Header data `{NaN: 0, NaN: 1}` gave a nondeterministic verdict (§76, case 3).
- `extension.New(id, v, nil)` wrote `null` as data (§76, case 2).
- `codec.Unmarshal` wipes its re-encoding, which after the new self-checks
  held a copy of I_PAYLOAD or `access_material`, and `accesskey.DecodeBody`
  wipes the material on its error paths. The `codec.Encoder` also wipes every
  buffer it outgrows, and `codec.Unmarshal` sizes its re-encoding for the
  input; the former library's internal buffers could keep a copy.
- `accesskey.Encode` wipes the body it wrote, and `accesskey.Decode` reads the
  body into a buffer that grows with the data read, wiping every buffer it
  outgrows, and wipes the body once decoded or on error: both left copies of
  `access_material` behind. The in-memory age decryption of SEALED_CONTROL and
  INNER_ACCESS_AGE in `capsule.Open` reads the plaintext into one buffer of
  the ciphertext's size instead of a growing one, so that no outgrown buffer
  keeps a copy of I_PAYLOAD. Buffers internal to `filippo.io/age` and copies
  made by the Go runtime stay out of reach (SECURITY.md).

## Unreleased — v0.1.0

First implementation of the DateKeys Protocol Specification v0.8.1.

### Added

- `datekey`: local date → round resolution at full precision (§15), canonical
  `dk1_` encoding and strict parsing (§18, §19).
- `profile`: Provider Profile Deterministic CBOR and `profile_hash` (§11), the
  pinned Quicknet profile with its chain-hash self-check (§12), and pinned
  registries (§13).
- `provider`: release sources and local BLS verification (§51);
  `provider/drand`: racing public relays, verifying every answer (§48, §49, §52).
- `codec`: Deterministic CBOR with a re-encoding canonicality check (§58, §58.1).
- `extension`: the generic extension mechanism (§54).
- `agewrap`: strict tlock and X25519 age identities that enforce the stanza
  rules (§27, §29, §32, §33, §35), and a secret-free header probe.
- `capsule`: `.dkc` framing, `Encrypt` for `time_only` and `time_and_key`
  (§61, §62), `Inspect` (§63 steps 1–8) and `Open` (§63 steps 9–18).
- `accesskey`: `.dkk` encoding and decoding (§40–§44).
- `cmd/datekeys`: `encrypt`, `decrypt`, `inspect`, `datekey resolve`,
  `profile hash`, with atomic, non-overwriting outputs.
- Official vectors (§65, §66), `.dkc`/`.dkk` fixtures (§67, §68), the mutation
  corpus (§64), fuzz targets for every parser, interoperability tests with the
  official `age` and `tle` CLIs, and live Quicknet integration tests.
- `spec/datekeys.cddl` and `docs/traceability.md`.