lib/src/cbor.dart is the port of codec/codec.go: CborEncoder, CborDecoder, unmarshalCbor, peekSchema, checkSchema and walkCbor, with the same reads, the same checks in the same order and the same error texts, such as "codec: offset 0: 23 is not in its shortest form (initial byte 0x18): ERR_NON_CANONICAL_CBOR". Integers are exact on the VM and on the web, where an int is a double and the bit operators work on 32 bits. An argument of eight bytes is read as two halves of 32 bits, and is an int up to 2^53-1 and a BigInt above, map keys and the numbers of the error texts included. uint returns an int, since every schema bounds its integers at 2^53-1, and uint64 a BigInt. The map that peekSchema reads is bounded at 2^63-1, Go's math.MaxInt, on the web too. Two kinds of text are of Dart only. CborEncoder.uint refuses an int outside 0..2^53-1 and uint64 a BigInt outside 0..2^64-1, with the text of datekeys-ts, where Go's uint64 cannot hold such a value. And the only invalid text that a Dart String holds is a lone surrogate: the error quotes it as Go quotes its bytes in generalized UTF-8. The tests port codec_test.go, internal_test.go and vectors_test.go, with the texts that Go prints, and cbor.test.ts. The fuzz targets are properties over seeded inputs, checked against a reference encoder and decoder written apart, as internal/cbortest. cbor.json runs its 36 accept and 67 reject vectors with the walk limits and the values; its 172 schema vectors are read and wait for the schema decoders of stage 4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>v0.11
parent
da63bc394c
commit
e2c70f50c1
@ -0,0 +1,682 @@
|
||||
/// The CBOR profile of the DateKeys protocol (spec §58, §58.1), as package
|
||||
/// codec of datekeys-go: the same reads, the same checks in the same order
|
||||
/// and the same error texts.
|
||||
///
|
||||
/// The profile is Deterministic CBOR (RFC 8949 §4.2.1) restricted to major
|
||||
/// types 0 (unsigned integer), 2 (byte string), 3 (text string), 4 (array)
|
||||
/// and 5 (map), with unsigned integer map keys in strictly ascending order,
|
||||
/// integers and lengths in their shortest form, definite lengths only and
|
||||
/// valid UTF-8 text. Negative integers, tags, floats, simple values (false,
|
||||
/// true, null, undefined), indefinite lengths and every other map key are
|
||||
/// rejected with ERR_NON_CANONICAL_CBOR.
|
||||
///
|
||||
/// Each schema writes its own encoding with a [CborEncoder] and reads it with
|
||||
/// a [CborDecoder], a strict cursor that reads exactly what the schema asks
|
||||
/// for. [unmarshalCbor] runs the decoder of a schema and then re-encodes what
|
||||
/// it decoded: the input must be reproduced byte for byte, or it is
|
||||
/// ERR_NON_CANONICAL_CBOR. The same principle as dk1_ canonicality (spec
|
||||
/// §19): canonicality does not depend on the decoder rejecting every
|
||||
/// non-canonical form.
|
||||
///
|
||||
/// [peekSchema] reads the type tag and the schema version of an object
|
||||
/// before strict decoding (spec §70). [walkCbor] checks that bytes are one
|
||||
/// data item of the profile; it is a helper for vectors and diagnostics, and
|
||||
/// never decides whether an object of the protocol is valid.
|
||||
///
|
||||
/// Integers. An unsigned integer of CBOR goes up to 2^64-1, and Dart's int
|
||||
/// does not hold that much on every platform: on the VM it has 64 bits with a
|
||||
/// sign, and on the web it is a double, exact up to 2^53, whose bit operators
|
||||
/// work on 32 bits. So this library never relies on an int above 2^53-1, nor
|
||||
/// on a shift or a bit operation beyond 31 bits:
|
||||
///
|
||||
/// - the decoder reads an argument of eight bytes as two halves of 32 bits;
|
||||
/// it is an [int] up to 2^53-1 ([maxSafeUint]) and a [BigInt] above, as
|
||||
/// datekeys-ts reads a `number | bigint`. Comparisons with the bounds of
|
||||
/// the caller, the order of the map keys and the numbers in the error texts
|
||||
/// are exact on both platforms;
|
||||
/// - every schema of the protocol bounds its integers at 2^53-1 (spec §58),
|
||||
/// so [CborDecoder.uint] returns an int, and [CborEncoder.uint] takes one;
|
||||
/// - what may be larger is exact: [CborDecoder.uint64] returns a BigInt,
|
||||
/// [CborEncoder.uint64] takes one, and [CborDecoder.key] returns a map key
|
||||
/// as an int up to 2^53-1 and as a BigInt above, so that a schema compares
|
||||
/// it with its keys in a switch and prints any other exactly.
|
||||
library;
|
||||
|
||||
import 'dart:typed_data';
|
||||
|
||||
import 'bytes.dart';
|
||||
import 'errors.dart';
|
||||
|
||||
/// 2^53-1, the largest unsigned integer any schema of the protocol allows,
|
||||
/// so that every integer is exact as an IEEE 754 double (spec §58), and as
|
||||
/// an int of Dart on every platform.
|
||||
const int maxSafeUint = 9007199254740991;
|
||||
|
||||
/// The bound of the type tag that [peekSchema] reads, in bytes. Every type
|
||||
/// tag of V1 is at most 25 bytes, so a longer one is of no known schema; the
|
||||
/// bound also keeps an input-sized tag out of the errors. It is an
|
||||
/// implementation limit (spec §74).
|
||||
const int maxTypeTagLen = 64;
|
||||
|
||||
/// 2^64-1, the largest unsigned integer of CBOR.
|
||||
final BigInt maxUint64 = (BigInt.one << 64) - BigInt.one;
|
||||
|
||||
// Go's math.MaxInt, 2^63-1: the bound of the map that Peek reads.
|
||||
final BigInt _maxInt64 = (BigInt.one << 63) - BigInt.one;
|
||||
|
||||
// 2^32, the weight of the high half of an argument of eight bytes.
|
||||
const _twoPow32 = 0x100000000;
|
||||
|
||||
// The low half of an argument of eight bytes, as a mask of a BigInt.
|
||||
final BigInt _low32 = BigInt.from(0xffffffff);
|
||||
|
||||
// The largest high half of an argument that is at most 2^53-1.
|
||||
const _maxSafeHigh = 0x1fffff;
|
||||
|
||||
// Major types of the profile (spec §58).
|
||||
const _majorUint = 0;
|
||||
const _majorBytes = 2;
|
||||
const _majorText = 3;
|
||||
const _majorArray = 4;
|
||||
const _majorMap = 5;
|
||||
|
||||
const _majorNames = [
|
||||
'an unsigned integer',
|
||||
'a negative integer',
|
||||
'a byte string',
|
||||
'a text string',
|
||||
'an array',
|
||||
'a map',
|
||||
'a tag',
|
||||
'a float or simple value',
|
||||
];
|
||||
|
||||
/// The ERR_NON_CANONICAL_CBOR error `codec: <detail>`, as errorf of the
|
||||
/// reference.
|
||||
DateKeysException _cborError(String detail) =>
|
||||
DateKeysException(ErrorCode.nonCanonicalCbor, 'codec: $detail');
|
||||
|
||||
// Go's %#02x of an initial byte.
|
||||
String _initialByte(int b) => '0x${b.toRadixString(16).padLeft(2, '0')}';
|
||||
|
||||
// Whether a > b, for two unsigned integers that are each an int or a BigInt.
|
||||
bool _above(Object a, Object b) =>
|
||||
a is int && b is int ? a > b : _bigOf(a) > _bigOf(b);
|
||||
|
||||
BigInt _bigOf(Object v) => v is BigInt ? v : BigInt.from(v as int);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Encoder
|
||||
|
||||
/// Writes the deterministic encoding of data items of the profile: every
|
||||
/// integer and length in its shortest form, definite lengths only. The first
|
||||
/// error is kept and later calls do nothing; [out] throws it.
|
||||
///
|
||||
/// An encoder does not know the schema: the caller writes the map keys, as
|
||||
/// unsigned integers in ascending order, and as many entries and items as it
|
||||
/// announced. The decoder of the schema checks both on the output (spec §72).
|
||||
///
|
||||
/// An encoding may hold secrets, such as I_PAYLOAD or access_material: the
|
||||
/// encoder wipes every buffer it outgrows, so that the output is the only
|
||||
/// copy, and the caller wipes the output.
|
||||
final class CborEncoder {
|
||||
/// An encoder whose buffer has room for [capacity] bytes before it grows.
|
||||
CborEncoder({int capacity = 0}) : _buf = Uint8List(capacity);
|
||||
|
||||
Uint8List _buf;
|
||||
int _len = 0;
|
||||
Exception? _err;
|
||||
|
||||
// Makes room for n more bytes, wiping the buffer it outgrows.
|
||||
void _grow(int n) {
|
||||
if (_buf.length - _len >= n) return;
|
||||
final b = Uint8List(2 * _buf.length + n)..setRange(0, _len, _buf);
|
||||
_buf.fillRange(0, _len, 0);
|
||||
_buf = b;
|
||||
}
|
||||
|
||||
// Appends the head of a data item: its major type and the argument
|
||||
// hi * 2^32 + lo, each half below 2^32.
|
||||
void _head(int major, int hi, int lo) {
|
||||
if (_err != null) return;
|
||||
final ib = major << 5;
|
||||
if (hi == 0 && lo < 24) {
|
||||
_grow(1);
|
||||
_buf[_len++] = ib | lo;
|
||||
} else if (hi == 0 && lo <= 0xff) {
|
||||
_grow(2);
|
||||
_buf[_len++] = ib | 24;
|
||||
_buf[_len++] = lo;
|
||||
} else if (hi == 0 && lo <= 0xffff) {
|
||||
_grow(3);
|
||||
_buf[_len++] = ib | 25;
|
||||
_buf[_len++] = lo >> 8;
|
||||
_buf[_len++] = lo & 0xff;
|
||||
} else if (hi == 0) {
|
||||
_grow(5);
|
||||
_buf[_len++] = ib | 26;
|
||||
_put32(lo);
|
||||
} else {
|
||||
_grow(9);
|
||||
_buf[_len++] = ib | 27;
|
||||
_put32(hi);
|
||||
_put32(lo);
|
||||
}
|
||||
}
|
||||
|
||||
// Appends v, below 2^32, in four bytes, big-endian, without a bit
|
||||
// operation on more than 31 bits.
|
||||
void _put32(int v) {
|
||||
final top = v ~/ 0x1000000;
|
||||
final rest = v - top * 0x1000000;
|
||||
_buf[_len++] = top;
|
||||
_buf[_len++] = rest >> 16;
|
||||
_buf[_len++] = rest >> 8 & 0xff;
|
||||
_buf[_len++] = rest & 0xff;
|
||||
}
|
||||
|
||||
// Appends the head of an argument that is a non-negative int.
|
||||
void _headInt(int major, int v) {
|
||||
if (v < _twoPow32) {
|
||||
_head(major, 0, v);
|
||||
} else {
|
||||
final hi = v ~/ _twoPow32;
|
||||
_head(major, hi, v - hi * _twoPow32);
|
||||
}
|
||||
}
|
||||
|
||||
void _append(List<int> b) {
|
||||
if (_err != null) return;
|
||||
_grow(b.length);
|
||||
_buf.setRange(_len, _len + b.length, b);
|
||||
_len += b.length;
|
||||
}
|
||||
|
||||
/// Records [error] as the error of the encoding unless one is recorded
|
||||
/// already. Every later call does nothing, and [out] throws the first
|
||||
/// error. The encoder of a schema calls it when its value breaks a rule of
|
||||
/// the schema, so that bytes the decoder rejects are never returned.
|
||||
void fail(Exception error) {
|
||||
_err ??= error;
|
||||
}
|
||||
|
||||
/// Writes the head of a map of [pairs] entries. The caller then writes each
|
||||
/// key, with [uint], followed by its value.
|
||||
void map(int pairs) {
|
||||
if (pairs < 0) {
|
||||
fail(_cborError('map of $pairs entries'));
|
||||
return;
|
||||
}
|
||||
_headInt(_majorMap, pairs);
|
||||
}
|
||||
|
||||
/// Writes the head of an array of [items] items. The caller then writes
|
||||
/// each item.
|
||||
void array(int items) {
|
||||
if (items < 0) {
|
||||
fail(_cborError('array of $items items'));
|
||||
return;
|
||||
}
|
||||
_headInt(_majorArray, items);
|
||||
}
|
||||
|
||||
/// Writes the unsigned integer [v], which must be in 0..2^53-1, the range of
|
||||
/// every schema of the protocol and of an exact int on every platform.
|
||||
/// [uint64] writes any unsigned integer of CBOR.
|
||||
void uint(int v) {
|
||||
if (v < 0 || v > maxSafeUint) {
|
||||
fail(_cborError('$v is not an unsigned integer in 0..2^53-1'));
|
||||
return;
|
||||
}
|
||||
_headInt(_majorUint, v);
|
||||
}
|
||||
|
||||
/// Writes the unsigned integer [v], which must be in 0..2^64-1.
|
||||
void uint64(BigInt v) {
|
||||
if (v.isNegative || v > maxUint64) {
|
||||
fail(_cborError('$v is not an unsigned integer in 0..2^64-1'));
|
||||
return;
|
||||
}
|
||||
_head(_majorUint, (v >> 32).toInt(), (v & _low32).toInt());
|
||||
}
|
||||
|
||||
/// Writes a byte string.
|
||||
void bstr(List<int> b) {
|
||||
_headInt(_majorBytes, b.length);
|
||||
_append(b);
|
||||
}
|
||||
|
||||
/// Writes a text string, which must be valid Unicode: a lone surrogate,
|
||||
/// which a Dart String may hold, has no UTF-8 encoding. The error quotes the
|
||||
/// string as Go's `%q` quotes the bytes of the generalized UTF-8 of such a
|
||||
/// string, as [utf8Bytes] writes them.
|
||||
void text(String s) {
|
||||
final b = utf8Bytes(s);
|
||||
if (!isWellFormedUtf16(s)) {
|
||||
fail(_cborError('text string ${goQuote(b)} is not valid UTF-8'));
|
||||
return;
|
||||
}
|
||||
_headInt(_majorText, b.length);
|
||||
_append(b);
|
||||
}
|
||||
|
||||
/// Returns the encoding, or throws the first error. The encoding is a view
|
||||
/// of the buffer of the encoder, not a copy, so that no other copy of it
|
||||
/// remains: the caller wipes it, and writes nothing more with this encoder.
|
||||
/// On error the partial output is wiped.
|
||||
Uint8List out() {
|
||||
final err = _err;
|
||||
if (err != null) {
|
||||
_buf.fillRange(0, _len, 0);
|
||||
_buf = Uint8List(0);
|
||||
_len = 0;
|
||||
throw err;
|
||||
}
|
||||
return Uint8List.sublistView(_buf, 0, _len);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Decoder
|
||||
|
||||
/// A strict cursor over the encoding of one data item of the profile. Each
|
||||
/// method reads one data item, or one head, and throws ERR_NON_CANONICAL_CBOR
|
||||
/// for a major type outside the profile, a major type other than the one
|
||||
/// asked for, an indefinite length, an integer or length not in its shortest
|
||||
/// form, a length beyond the remaining input and a value outside the bounds
|
||||
/// the caller gives. Within each open map the keys are unsigned integers in
|
||||
/// strictly ascending order. Every error names the offset of the input where
|
||||
/// it was found: `codec: offset 3: …: ERR_NON_CANONICAL_CBOR`.
|
||||
///
|
||||
/// The first error is kept: every later call throws it again.
|
||||
final class CborDecoder {
|
||||
/// A decoder positioned at the start of [input].
|
||||
CborDecoder(List<int> input)
|
||||
: _in = input is Uint8List ? input : Uint8List.fromList(input);
|
||||
|
||||
final Uint8List _in;
|
||||
int _off = 0;
|
||||
final _maps = <_OpenMap>[];
|
||||
DateKeysException? _err;
|
||||
|
||||
int get _remaining => _in.length - _off;
|
||||
|
||||
// Records the first error, at the current offset, and returns it.
|
||||
DateKeysException _fail(String detail) =>
|
||||
_err ??= _cborError('offset $_off: $detail');
|
||||
|
||||
void _throwIfFailed() {
|
||||
final err = _err;
|
||||
if (err != null) throw err;
|
||||
}
|
||||
|
||||
// Reads the head of the next data item and returns its major type and
|
||||
// argument, an int up to 2^53-1 and a BigInt above. It rejects the major
|
||||
// types outside the profile, reserved values, indefinite lengths,
|
||||
// arguments not in their shortest form and truncation.
|
||||
(int, Object) _head() {
|
||||
_throwIfFailed();
|
||||
if (_off >= _in.length) throw _fail('truncated input');
|
||||
final b = _in[_off];
|
||||
final major = b >> 5;
|
||||
final info = b & 0x1f;
|
||||
if (major == 1 || major == 6 || major == 7) {
|
||||
throw _fail(
|
||||
'${_majorNames[major]} (initial byte ${_initialByte(b)}) '
|
||||
'is outside the CBOR profile',
|
||||
);
|
||||
}
|
||||
if (info < 24) {
|
||||
_off++;
|
||||
return (major, info);
|
||||
}
|
||||
if (info == 31) {
|
||||
throw _fail('indefinite length (initial byte ${_initialByte(b)})');
|
||||
}
|
||||
if (info > 27) {
|
||||
throw _fail(
|
||||
'reserved additional information (initial byte ${_initialByte(b)})',
|
||||
);
|
||||
}
|
||||
final n = 1 << (info - 24);
|
||||
if (_remaining < 1 + n) throw _fail('truncated input');
|
||||
final p = _off + 1;
|
||||
final Object arg;
|
||||
final int min;
|
||||
switch (n) {
|
||||
case 1:
|
||||
arg = _in[p];
|
||||
min = 24;
|
||||
case 2:
|
||||
arg = _in[p] << 8 | _in[p + 1];
|
||||
min = 0x100;
|
||||
case 4:
|
||||
arg = _uint32At(p);
|
||||
min = 0x10000;
|
||||
default:
|
||||
final hi = _uint32At(p);
|
||||
final lo = _uint32At(p + 4);
|
||||
arg = hi <= _maxSafeHigh
|
||||
? hi * _twoPow32 + lo
|
||||
: BigInt.from(hi) << 32 | BigInt.from(lo);
|
||||
min = _twoPow32;
|
||||
}
|
||||
// A BigInt is above 2^53-1, and so in its shortest form.
|
||||
if (arg is int && arg < min) {
|
||||
throw _fail(
|
||||
'$arg is not in its shortest form (initial byte ${_initialByte(b)})',
|
||||
);
|
||||
}
|
||||
_off = p + n;
|
||||
return (major, arg);
|
||||
}
|
||||
|
||||
// The big-endian uint32 at p, without a bit operation on more than 31 bits.
|
||||
int _uint32At(int p) =>
|
||||
_in[p] * 0x1000000 + (_in[p + 1] << 16 | _in[p + 2] << 8 | _in[p + 3]);
|
||||
|
||||
// Reads the head of a data item of major type want.
|
||||
Object _expect(int want) {
|
||||
final start = _off;
|
||||
final (major, arg) = _head();
|
||||
if (major != want) {
|
||||
_off = start;
|
||||
throw _fail(
|
||||
'${_majorNames[major]} where ${_majorNames[want]} was expected',
|
||||
);
|
||||
}
|
||||
return arg;
|
||||
}
|
||||
|
||||
/// Reads the head of a map of at most [max] entries and returns the number
|
||||
/// of entries. The caller reads each entry with [key] and a value, then
|
||||
/// calls [endMap].
|
||||
int map(int max) => _map(max);
|
||||
|
||||
// map with a bound that is an int or a BigInt, as Go's MaxInt of Peek.
|
||||
int _map(Object max) {
|
||||
final n = _expect(_majorMap);
|
||||
if ((max is int && max < 0) || _above(n, max)) {
|
||||
throw _fail('map of $n entries, at most $max');
|
||||
}
|
||||
if (_above(n, _remaining ~/ 2)) {
|
||||
throw _fail('truncated input: map of $n entries');
|
||||
}
|
||||
// At most half the remaining input, so an int.
|
||||
final pairs = n as int;
|
||||
_maps.add(_OpenMap(pairs));
|
||||
return pairs;
|
||||
}
|
||||
|
||||
/// Reads the key of the next entry of the innermost open map: an unsigned
|
||||
/// integer greater than the previous key of that map. The key is exact: an
|
||||
/// [int] up to 2^53-1 and a [BigInt] above, which no schema defines.
|
||||
Object key() {
|
||||
_throwIfFailed();
|
||||
if (_maps.isEmpty) throw _fail('map key outside a map');
|
||||
final m = _maps.last;
|
||||
if (m.left == 0) throw _fail('map key after the last entry');
|
||||
final start = _off;
|
||||
final k = _expect(_majorUint);
|
||||
if (m.started && !_above(k, m.last)) {
|
||||
_off = start;
|
||||
throw _fail(
|
||||
'map key $k after key ${m.last}: keys must be strictly ascending',
|
||||
);
|
||||
}
|
||||
m.left--;
|
||||
m.last = k;
|
||||
m.started = true;
|
||||
return k;
|
||||
}
|
||||
|
||||
/// Closes the innermost open map, all of whose entries must have been
|
||||
/// read.
|
||||
void endMap() {
|
||||
_throwIfFailed();
|
||||
if (_maps.isEmpty) throw _fail('end of a map outside a map');
|
||||
final left = _maps.last.left;
|
||||
if (left != 0) throw _fail('$left map entries not read');
|
||||
_maps.removeLast();
|
||||
}
|
||||
|
||||
/// Reads the head of an array of at most [max] items and returns the number
|
||||
/// of items, which the caller then reads.
|
||||
int array(int max) {
|
||||
final n = _expect(_majorArray);
|
||||
if (max < 0 || _above(n, max)) {
|
||||
throw _fail('array of $n items, at most $max');
|
||||
}
|
||||
if (_above(n, _remaining)) {
|
||||
throw _fail('truncated input: array of $n items');
|
||||
}
|
||||
return n as int;
|
||||
}
|
||||
|
||||
/// Reads an unsigned integer of at most [max], which must be in 0..2^53-1,
|
||||
/// as the bounds of every schema of the protocol: the integer is then an
|
||||
/// exact int on every platform. [uint64] reads up to 2^64-1.
|
||||
int uint([int max = maxSafeUint]) {
|
||||
if (max < 0 || max > maxSafeUint) {
|
||||
throw ArgumentError.value(max, 'max', 'not in 0..2^53-1');
|
||||
}
|
||||
final v = _expect(_majorUint);
|
||||
if (_above(v, max)) throw _fail('unsigned integer $v above $max');
|
||||
return v as int;
|
||||
}
|
||||
|
||||
/// Reads an unsigned integer of at most [max], 2^64-1 when absent.
|
||||
BigInt uint64([BigInt? max]) {
|
||||
final bound = max ?? maxUint64;
|
||||
if (bound.isNegative || bound > maxUint64) {
|
||||
throw ArgumentError.value(max, 'max', 'not in 0..2^64-1');
|
||||
}
|
||||
final v = _expect(_majorUint);
|
||||
if (_above(v, bound)) throw _fail('unsigned integer $v above $bound');
|
||||
return _bigOf(v);
|
||||
}
|
||||
|
||||
// Reads a string of major type want and returns its content, a view of the
|
||||
// input. The length is checked against the remaining input and then
|
||||
// against min and max.
|
||||
Uint8List _content(int want, int min, int max) {
|
||||
final n = _expect(want);
|
||||
if (_above(n, _remaining)) {
|
||||
throw _fail('truncated input: ${_majorNames[want]} of $n bytes');
|
||||
}
|
||||
final length = n as int;
|
||||
if (length < min || length > max) {
|
||||
throw _fail('${_majorNames[want]} of $length bytes outside $min..$max');
|
||||
}
|
||||
final b = Uint8List.sublistView(_in, _off, _off + length);
|
||||
_off += length;
|
||||
return b;
|
||||
}
|
||||
|
||||
/// Reads a byte string of [min] to [max] bytes and returns a copy of its
|
||||
/// content. The length is checked before anything is copied.
|
||||
Uint8List bstr(int min, int max) =>
|
||||
Uint8List.fromList(_content(_majorBytes, min, max));
|
||||
|
||||
/// Reads a text string of at most [max] bytes of valid UTF-8. A leading
|
||||
/// U+FEFF is part of the text.
|
||||
String text(int max) {
|
||||
final start = _off;
|
||||
final s = decodeUtf8(_content(_majorText, 0, max));
|
||||
if (s == null) {
|
||||
_off = start;
|
||||
throw _fail('text string is not valid UTF-8');
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
/// Checks that every map was closed and that no byte follows the data
|
||||
/// item.
|
||||
void done() {
|
||||
_throwIfFailed();
|
||||
if (_maps.isNotEmpty) throw _fail('${_maps.length} maps not closed');
|
||||
if (_off != _in.length) throw _fail('${_in.length - _off} trailing bytes');
|
||||
}
|
||||
|
||||
// The major type of the next data item, or an unsigned integer at the end
|
||||
// of the input, where reading it reports the truncation.
|
||||
int _next() => _off >= _in.length ? _majorUint : _in[_off] >> 5;
|
||||
|
||||
// Reads one data item for walkCbor: a scalar, or the head of a container,
|
||||
// which it pushes on open.
|
||||
void _walkItem(List<_WalkLevel> open, int maxDepth, int maxLen) {
|
||||
switch (_next()) {
|
||||
case final major when major == _majorMap || major == _majorArray:
|
||||
if (open.length >= maxDepth) {
|
||||
throw _fail('containers nested deeper than $maxDepth');
|
||||
}
|
||||
final isMap = major == _majorMap;
|
||||
open.add(_WalkLevel(isMap ? map(maxLen) : array(maxLen), isMap));
|
||||
case _majorBytes:
|
||||
_content(_majorBytes, 0, maxLen);
|
||||
case _majorText:
|
||||
text(maxLen);
|
||||
default:
|
||||
// An unsigned integer, any up to 2^64-1, or the error of whatever is
|
||||
// there.
|
||||
_expect(_majorUint);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The state of a map between map and endMap.
|
||||
final class _OpenMap {
|
||||
_OpenMap(this.left);
|
||||
|
||||
// Entries not read yet.
|
||||
int left;
|
||||
|
||||
// The last key read, an int or a BigInt.
|
||||
Object last = 0;
|
||||
|
||||
// Whether at least one key was read.
|
||||
bool started = false;
|
||||
}
|
||||
|
||||
// An open container of walkCbor.
|
||||
final class _WalkLevel {
|
||||
_WalkLevel(this.left, this.isMap);
|
||||
|
||||
// Entries of a map or items of an array not read yet.
|
||||
int left;
|
||||
final bool isMap;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Objects
|
||||
|
||||
/// Decodes one object from [input] with [decode], checks that the whole
|
||||
/// input was read, and re-encodes the decoded value with [encode]: the result
|
||||
/// must reproduce [input] byte for byte. [decode] and [encode] are the two
|
||||
/// halves of one schema and work on the same value.
|
||||
///
|
||||
/// Errors of the decoder and of the re-encoding are ERR_NON_CANONICAL_CBOR;
|
||||
/// any other exception of [decode] passes as it is.
|
||||
///
|
||||
/// The re-encoding equals [input] on success, so it may hold secrets such as
|
||||
/// I_PAYLOAD or access_material; it is wiped on every path, and so is every
|
||||
/// buffer the encoder outgrows.
|
||||
void unmarshalCbor(
|
||||
List<int> input,
|
||||
void Function(CborDecoder d) decode,
|
||||
void Function(CborEncoder e) encode,
|
||||
) {
|
||||
final d = CborDecoder(input);
|
||||
decode(d);
|
||||
d.done();
|
||||
final e = CborEncoder(capacity: input.length);
|
||||
encode(e);
|
||||
Uint8List? re;
|
||||
try {
|
||||
re = e.out();
|
||||
} on Exception {
|
||||
// The error of the encoder is not the error of the input: the input is
|
||||
// not what the encoder writes, whatever the reason.
|
||||
}
|
||||
try {
|
||||
if (re == null || !equalBytes(re, input)) {
|
||||
throw _cborError('input is not the deterministic encoding of its value');
|
||||
}
|
||||
} finally {
|
||||
re?.fillRange(0, re.length, 0);
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the type tag (key 0, a text string of at most [maxTypeTagLen]
|
||||
/// bytes) and the schema version (key 1, an unsigned integer of at most
|
||||
/// 2^53-1) of the map at the start of [input], before strict decoding, so
|
||||
/// that an unknown schema version is reported as such (spec §70). The map
|
||||
/// must start with keys 0 and 1, in the profile; nothing after them is read.
|
||||
/// The result must never be used as the decoded object.
|
||||
({String typeTag, int version}) peekSchema(List<int> input) {
|
||||
final d = CborDecoder(input);
|
||||
// The input bounds the map.
|
||||
final pairs = d._map(_maxInt64);
|
||||
if (pairs < 2) {
|
||||
throw d._fail('map without a type tag and a schema version');
|
||||
}
|
||||
var typeTag = '';
|
||||
var version = 0;
|
||||
for (var want = 0; want < 2; want++) {
|
||||
final k = d.key();
|
||||
if (k != want) throw d._fail('map key $k where key $want was expected');
|
||||
if (want == 0) {
|
||||
typeTag = d.text(maxTypeTagLen);
|
||||
} else {
|
||||
version = d.uint();
|
||||
}
|
||||
}
|
||||
return (typeTag: typeTag, version: version);
|
||||
}
|
||||
|
||||
/// Reads the type tag and the schema version of an object with [peekSchema]
|
||||
/// and requires [typeTag] and [version]. A different type tag is
|
||||
/// ERR_NON_CANONICAL_CBOR; a different version is ERR_UNSUPPORTED_VERSION.
|
||||
void checkSchema(List<int> input, String typeTag, int version) {
|
||||
final (typeTag: tag, version: v) = peekSchema(input);
|
||||
if (tag != typeTag) {
|
||||
throw _cborError(
|
||||
'type ${goQuote(utf8Bytes(tag))}, want ${goQuote(utf8Bytes(typeTag))}',
|
||||
);
|
||||
}
|
||||
if (v != version) {
|
||||
throw DateKeysException(
|
||||
ErrorCode.unsupportedVersion,
|
||||
'codec: $typeTag schema version $v, want $version',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Checks that [input] is exactly one data item of the profile, with
|
||||
/// containers nested at most [maxDepth] deep (a scalar has depth 0) and every
|
||||
/// string and container at most [maxLen] long: bytes of a string, items of
|
||||
/// an array, entries of a map. Unsigned integers take any value up to
|
||||
/// 2^64-1. It reads iteratively, so deep input cannot exhaust the stack.
|
||||
///
|
||||
/// A helper for vectors and diagnostics, and for registered extensions whose
|
||||
/// data is CBOR (spec §72). It never decides whether an object of the
|
||||
/// protocol is valid: the decoder of its schema does.
|
||||
void walkCbor(List<int> input, int maxDepth, int maxLen) {
|
||||
final d = CborDecoder(input);
|
||||
final open = <_WalkLevel>[];
|
||||
for (var first = true; first || open.isNotEmpty; first = false) {
|
||||
final top = open.isEmpty ? null : open.last;
|
||||
if (top != null && top.left == 0) {
|
||||
// The innermost container is complete.
|
||||
if (top.isMap) d.endMap();
|
||||
open.removeLast();
|
||||
continue;
|
||||
}
|
||||
if (top != null) {
|
||||
top.left--;
|
||||
if (top.isMap) d.key();
|
||||
}
|
||||
d._walkItem(open, maxDepth, maxLen);
|
||||
}
|
||||
d.done();
|
||||
}
|
||||
@ -0,0 +1,223 @@
|
||||
// An encoder and a decoder of generic CBOR values for the tests, as
|
||||
// internal/cbortest of datekeys-go. They are written independently of
|
||||
// lib/src/cbor.dart on purpose, with BigInt arithmetic and dart:convert:
|
||||
// tests use them to build inputs that the codec cannot write, such as null,
|
||||
// negative integers or a float inside an otherwise valid object, and as a
|
||||
// second reading of the CBOR profile of spec §58 to check the codec against.
|
||||
|
||||
import 'dart:convert';
|
||||
import 'dart:typed_data';
|
||||
|
||||
/// An encoded data item that [marshal] writes verbatim.
|
||||
final class Raw {
|
||||
const Raw(this.bytes);
|
||||
|
||||
final List<int> bytes;
|
||||
}
|
||||
|
||||
/// A text string held as its bytes, as [unmarshal] returns it: dart:convert
|
||||
/// would drop a leading U+FEFF.
|
||||
final class Text {
|
||||
const Text(this.bytes);
|
||||
|
||||
final Uint8List bytes;
|
||||
}
|
||||
|
||||
/// The deterministic encoding of [v], which is one of: an int (a negative one
|
||||
/// as major type 1) or a BigInt (unsigned), a String or a [Text], a
|
||||
/// Uint8List, a bool, null, a List of values, a Map whose keys are ints or
|
||||
/// BigInts (written in ascending order), or a [Raw].
|
||||
Uint8List marshal(Object? v) {
|
||||
final out = <int>[];
|
||||
_append(out, v);
|
||||
return Uint8List.fromList(out);
|
||||
}
|
||||
|
||||
void _append(List<int> out, Object? v) {
|
||||
switch (v) {
|
||||
case null:
|
||||
out.add(0xf6);
|
||||
case final bool b:
|
||||
out.add(b ? 0xf5 : 0xf4);
|
||||
case final int i when i < 0:
|
||||
_head(out, 1, BigInt.from(-(i + 1)));
|
||||
case final int i:
|
||||
_head(out, 0, BigInt.from(i));
|
||||
case final BigInt i:
|
||||
_head(out, 0, i);
|
||||
case final Uint8List b:
|
||||
_head(out, 2, BigInt.from(b.length));
|
||||
out.addAll(b);
|
||||
case final String s:
|
||||
final b = utf8.encode(s);
|
||||
_head(out, 3, BigInt.from(b.length));
|
||||
out.addAll(b);
|
||||
case final Text t:
|
||||
_head(out, 3, BigInt.from(t.bytes.length));
|
||||
out.addAll(t.bytes);
|
||||
case final Raw r:
|
||||
out.addAll(r.bytes);
|
||||
case final List<Object?> l:
|
||||
_head(out, 4, BigInt.from(l.length));
|
||||
for (final x in l) {
|
||||
_append(out, x);
|
||||
}
|
||||
case final Map<Object, Object?> m:
|
||||
final keys = m.keys.toList()..sort((a, b) => _big(a).compareTo(_big(b)));
|
||||
_head(out, 5, BigInt.from(m.length));
|
||||
for (final k in keys) {
|
||||
_head(out, 0, _big(k));
|
||||
_append(out, m[k]);
|
||||
}
|
||||
default:
|
||||
throw ArgumentError('cannot encode ${v.runtimeType}');
|
||||
}
|
||||
}
|
||||
|
||||
BigInt _big(Object k) => k is BigInt ? k : BigInt.from(k as int);
|
||||
|
||||
void _head(List<int> out, int major, BigInt arg) {
|
||||
final m = major << 5;
|
||||
if (arg < BigInt.from(24)) {
|
||||
out.add(m | arg.toInt());
|
||||
return;
|
||||
}
|
||||
final int n;
|
||||
if (arg < BigInt.from(0x100)) {
|
||||
out.add(m | 24);
|
||||
n = 1;
|
||||
} else if (arg < BigInt.from(0x10000)) {
|
||||
out.add(m | 25);
|
||||
n = 2;
|
||||
} else if (arg < BigInt.one << 32) {
|
||||
out.add(m | 26);
|
||||
n = 4;
|
||||
} else {
|
||||
out.add(m | 27);
|
||||
n = 8;
|
||||
}
|
||||
for (var i = n - 1; i >= 0; i--) {
|
||||
out.add(((arg >> (8 * i)) & BigInt.from(0xff)).toInt());
|
||||
}
|
||||
}
|
||||
|
||||
/// The nesting that [unmarshal] follows at most.
|
||||
const maxDepth = 1000;
|
||||
|
||||
/// Decodes exactly one data item of the CBOR profile of spec §58 into BigInt,
|
||||
/// Uint8List, [Text], List and Map with BigInt keys in their order. It throws
|
||||
/// a FormatException for every other major type, an indefinite length, a
|
||||
/// head not in its shortest form, a map key that is not an unsigned integer
|
||||
/// greater than the previous one, invalid UTF-8, truncation, trailing bytes
|
||||
/// and nesting deeper than [maxDepth].
|
||||
Object? unmarshal(List<int> b) {
|
||||
final r = _Reader(b);
|
||||
final v = r.value(0);
|
||||
if (r.off != b.length) {
|
||||
throw FormatException('${b.length - r.off} trailing bytes');
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
final class _Reader {
|
||||
_Reader(this.b);
|
||||
|
||||
final List<int> b;
|
||||
int off = 0;
|
||||
|
||||
(int, BigInt) head() {
|
||||
if (off >= b.length) throw const FormatException('truncated');
|
||||
final ib = b[off++];
|
||||
final major = ib >> 5;
|
||||
final info = ib & 0x1f;
|
||||
if (major == 1 || major >= 6) throw FormatException('major type $major');
|
||||
if (info < 24) return (major, BigInt.from(info));
|
||||
if (info > 27) throw FormatException('additional information $info');
|
||||
final n = 1 << (info - 24);
|
||||
if (b.length - off < n) throw const FormatException('truncated');
|
||||
var arg = BigInt.zero;
|
||||
for (var i = 0; i < n; i++) {
|
||||
arg = arg << 8 | BigInt.from(b[off + i]);
|
||||
}
|
||||
off += n;
|
||||
if ((n == 1 && arg < BigInt.from(24)) ||
|
||||
(n > 1 && arg >> (4 * n) == BigInt.zero)) {
|
||||
throw FormatException('$arg not in its shortest form');
|
||||
}
|
||||
return (major, arg);
|
||||
}
|
||||
|
||||
Uint8List bytes(BigInt n) {
|
||||
if (n > BigInt.from(b.length - off)) {
|
||||
throw const FormatException('truncated');
|
||||
}
|
||||
final s = Uint8List.fromList(b.sublist(off, off + n.toInt()));
|
||||
off += n.toInt();
|
||||
return s;
|
||||
}
|
||||
|
||||
Object? value(int depth) {
|
||||
final (major, arg) = head();
|
||||
switch (major) {
|
||||
case 0:
|
||||
return arg;
|
||||
case 2:
|
||||
return bytes(arg);
|
||||
case 3:
|
||||
final s = bytes(arg);
|
||||
// dart:convert is strict about everything but a leading U+FEFF,
|
||||
// which it drops: the text keeps its bytes.
|
||||
const Utf8Decoder().convert(s);
|
||||
return Text(s);
|
||||
}
|
||||
if (depth >= maxDepth) throw const FormatException('nested too deep');
|
||||
if (arg > BigInt.from(b.length - off)) {
|
||||
throw const FormatException('truncated');
|
||||
}
|
||||
final count = arg.toInt();
|
||||
if (major == 4) {
|
||||
return [for (var i = 0; i < count; i++) value(depth + 1)];
|
||||
}
|
||||
final out = <BigInt, Object?>{};
|
||||
BigInt? last;
|
||||
for (var i = 0; i < count; i++) {
|
||||
final (km, k) = head();
|
||||
if (km != 0) throw FormatException('map key of major type $km');
|
||||
if (last != null && k <= last) {
|
||||
throw FormatException('map key $k after $last');
|
||||
}
|
||||
last = k;
|
||||
out[k] = value(depth + 1);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
}
|
||||
|
||||
/// The nesting depth of [v], a value of [marshal] or [unmarshal], and the
|
||||
/// length of its longest string or container: bytes of a string, items of an
|
||||
/// array, entries of a map.
|
||||
(int, int) shape(Object? v) {
|
||||
final Iterable<Object?> items;
|
||||
switch (v) {
|
||||
case final Uint8List b:
|
||||
return (0, b.length);
|
||||
case final String s:
|
||||
return (0, utf8.encode(s).length);
|
||||
case final Text t:
|
||||
return (0, t.bytes.length);
|
||||
case final List<Object?> l:
|
||||
items = l;
|
||||
case final Map<Object, Object?> m:
|
||||
items = m.values;
|
||||
default:
|
||||
return (0, 0);
|
||||
}
|
||||
var depth = 0;
|
||||
var length = items.length;
|
||||
for (final x in items) {
|
||||
final (d, l) = shape(x);
|
||||
if (d > depth) depth = d;
|
||||
if (l > length) length = l;
|
||||
}
|
||||
return (depth + 1, length);
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,120 @@
|
||||
// The shared vectors of the CBOR profile of spec §58,
|
||||
// testdata/vectors/cbor.json, as vectors_test.go of package codec of
|
||||
// datekeys-go: walkCbor accepts exactly the accept list, with the value
|
||||
// recorded for unsigned integers, and rejects the reject list with the
|
||||
// recorded code.
|
||||
@TestOn('vm')
|
||||
library;
|
||||
|
||||
import 'dart:convert';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:datekeys/datekeys.dart';
|
||||
import 'package:test/test.dart';
|
||||
|
||||
void main() {
|
||||
final file = jsonDecode(
|
||||
File('testdata/vectors/cbor.json').readAsStringSync(),
|
||||
) as Map<String, Object?>;
|
||||
final walk = file['walk']! as Map<String, Object?>;
|
||||
final maxDepth = walk['max_depth']! as int;
|
||||
final maxLen = walk['max_len']! as int;
|
||||
List<Map<String, Object?>> list(String key) =>
|
||||
(file[key]! as List).cast<Map<String, Object?>>();
|
||||
final accept = list('accept');
|
||||
final reject = list('reject');
|
||||
final schemas = list('schemas');
|
||||
|
||||
test('the vector file is complete', () {
|
||||
expect(file['spec'], specVersion);
|
||||
expect(accept, isNotEmpty);
|
||||
expect(reject, isNotEmpty);
|
||||
expect(maxDepth, isPositive);
|
||||
expect(maxLen, isPositive);
|
||||
});
|
||||
|
||||
group('accept', () {
|
||||
for (final v in accept) {
|
||||
test(v['name'], () {
|
||||
final input = fromHex(v['hex']! as String);
|
||||
walkCbor(input, maxDepth, maxLen);
|
||||
// An unsigned integer records its value: a JSON number up to 2^53-1
|
||||
// and a decimal string above, so that no reader loses precision.
|
||||
final isUint = input.isNotEmpty && input[0] >> 5 == 0;
|
||||
expect(v.containsKey('value'), isUint);
|
||||
if (!isUint) return;
|
||||
final n = CborDecoder(input).uint64();
|
||||
if (n > BigInt.from(maxSafeUint)) {
|
||||
expect(v['value'], '$n');
|
||||
} else {
|
||||
expect(v['value'], n.toInt());
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
group('reject', () {
|
||||
for (final v in reject) {
|
||||
test(v['name'], () {
|
||||
final input = fromHex(v['hex']! as String);
|
||||
var code = '';
|
||||
try {
|
||||
walkCbor(input, maxDepth, maxLen);
|
||||
} on DateKeysException catch (e) {
|
||||
code = e.code.code;
|
||||
}
|
||||
expect(code, v['error']);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// The schemas block is decoded with the decoder of each schema: Provider
|
||||
// Profile, PUBLIC_HEADER, CONTROL_CBOR and the body of a .dkk, which arrive
|
||||
// with stage 4 of docs/PLAN_dart.md. Until then the block is only read: its
|
||||
// shape is checked, and the objects it holds as valid are items of the
|
||||
// profile whose type tag peekSchema reads.
|
||||
group('schemas', () {
|
||||
const typeTags = {
|
||||
'provider_profile': 'datekeys-provider-profile',
|
||||
'public_header': 'datekeycap',
|
||||
'control_cbor': 'datekeys-control',
|
||||
'dkk_body': 'datekeys-access-key',
|
||||
};
|
||||
final codes = {for (final c in ErrorCode.values) c.code};
|
||||
|
||||
test('the block is well formed', () {
|
||||
expect(schemas, isNotEmpty);
|
||||
for (final v in schemas) {
|
||||
expect(typeTags.keys, contains(v['schema']), reason: '${v['name']}');
|
||||
expect(v['block'], isA<String>());
|
||||
expect(v['name'], isA<String>());
|
||||
expect(fromHex(v['hex']! as String), isNotEmpty);
|
||||
expect(
|
||||
v['result'] == 'ok' || codes.contains(v['result']),
|
||||
isTrue,
|
||||
reason: '${v['name']}: ${v['result']}',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('the valid objects are items of the profile with their type tag', () {
|
||||
final valid = schemas.where((v) => v['result'] == 'ok').toList();
|
||||
expect(valid, isNotEmpty);
|
||||
for (final v in valid) {
|
||||
final input = fromHex(v['hex']! as String);
|
||||
walkCbor(input, 16, input.length);
|
||||
expect(
|
||||
peekSchema(input).typeTag,
|
||||
typeTags[v['schema']],
|
||||
reason: '${v['name']}',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test(
|
||||
'the vectors decode with the decoders of their schemas',
|
||||
() {},
|
||||
skip: 'the decoders of the schemas arrive with stage 4',
|
||||
);
|
||||
});
|
||||
}
|
||||
Loading…
Reference in new issue