You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
dateKeys-dart/test/pathrule_planes.dart

96 lines
2.9 KiB

Stage 4a: the rules of paths and texts on the Unicode tables lib/src/pathrule.dart ports internal/pathrule of datekeys-go at c531e93 (spec §29.5, §29.5.1, §29.6): NFD with the canonical ordering and Hangul, the case folding, the simple lowercase, Default_Ignorable with the whitelist of R4, the best-fit projections of R6c, every rule of a path (R2 to R6c and R10), the tree (R7 with its key and the two paths it names, and R9), and the texts of the comment and the declared author, with the texts of Go. canonicalTables is pathrule.Canonical, and a test recomputes tablesDigest from the lists. Go reads a string as bytes, and so does this port: the functions whose name ends in Utf8 take the bytes of a Go string, where a byte that is not valid UTF-8 is the rune U+FFFD, and the limits count bytes; the others take a String as utf8Bytes writes it. Each rule returns its violation, as in Go, and only the public functions throw. tool/pathrule_go_vectors.go runs in an export of datekeys-go, since internal/pathrule cannot be imported from outside its tree, and writes test/vectors/pathrule_vectors.json and its Dart copy: the cases of the tests of Go and of datekeys-ts, 1300 strings and 350 trees drawn from a fixed seed (marks, Hangul, ignorables, emoji, best-fit look-alikes, device names, 8.3 aliases, limits, texts and invalid UTF-8), the cases of R9, code points, and for each plane the SHA-256 of one line per code point of each function. Every code point of every plane gives the results of Go: planes 0, 1 and 14 also on Node.js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 days ago
// The exhaustive comparison of the rules of paths and texts with Go: for each
// code point of a plane, one line per function, hashed with SHA-256, against
// the digests of test/vectors/pathrule_vectors.json, which
// tool/pathrule_go_vectors.go computes with the same lines.
import 'dart:typed_data';
import 'package:datekeys/src/bytes.dart';
import 'package:datekeys/src/pathrule.dart';
import 'package:datekeys/src/sha256.dart';
import 'package:test/test.dart';
/// Lines of ASCII added to a SHA-256 through a buffer of 64 KiB.
final class LineHash {
final _hash = Sha256();
final _buffer = Uint8List(1 << 16);
var _length = 0;
void add(String line) {
if (_length + line.length > _buffer.length) _flush();
for (var i = 0; i < line.length; i++) {
final c = line.codeUnitAt(i);
if (c >= 0x80) throw ArgumentError.value(line, 'line', 'not ASCII');
_buffer[_length++] = c;
}
}
void _flush() {
_hash.add(_buffer, 0, _length);
_length = 0;
}
String finish() {
_flush();
return toHex(_hash.finish());
}
}
String _hexUpper(int r) => r.toRadixString(16).toUpperCase();
String _outcome(void Function() f) {
try {
f();
return 'ok';
} on PathRuleException catch (e) {
return e.message;
}
}
/// The planes whose code points pathrule_test.dart checks, also compiled to
/// JavaScript: the BMP, the SMP and plane 14, of the tags and of the
/// variation selectors. pathrule_vm_test.dart checks the others.
const sharedPlanes = [0, 1, 14];
/// The names of the digests of a plane, in the order of the lines.
const planeFunctions = [
'props',
'nfd',
'fold',
'key',
'path',
'comment',
'author',
];
/// Checks every code point of [plane] against [digests], the entry of the
/// plane in the vectors: for each code point r, its string is its UTF-8, or
/// U+FFFD for a surrogate, as Go's string(rune(r)) writes it.
void checkPlane(int plane, Map<String, Object?> digests) {
final replacement = Uint8List.fromList([0xef, 0xbf, 0xbd]);
final h = [for (final _ in planeFunctions) LineHash()];
for (var r = plane << 16; r <= (plane << 16 | 0xffff); r++) {
final s = r >= 0xd800 && r <= 0xdfff
? replacement
: utf8Bytes(String.fromCharCode(r));
final x = _hexUpper(r);
h[0].add(
'$x ${isAssigned(r)} ${isDefaultIgnorable(r)} ${_hexUpper(lower(r))}\n',
);
h[1].add('$x ${toHex(nfdUtf8(s))}\n');
h[2].add('$x ${toHex(foldUtf8(s))}\n');
h[3].add('$x ${toHex(pathKeyUtf8(s))}\n');
h[4].add('$x ${_outcome(() => checkPathUtf8(s))}\n');
h[5].add('$x ${_outcome(() => checkCommentUtf8(s))}\n');
h[6].add('$x ${_outcome(() => checkAuthorUtf8(s))}\n');
}
expect(digests['plane'], plane);
for (var i = 0; i < planeFunctions.length; i++) {
expect(
h[i].finish(),
digests[planeFunctions[i]],
reason: 'plane $plane, ${planeFunctions[i]}',
);
}
}

Powered by TurnKey Linux.