Format 3, step 6c: the vectors of paths, keys, heads and security
The vector files of spec 67 for format 3, generated with the result
each case is written for, so that the generator fails when the
implementation or the tables change:
- paths.json: 83 paths with the result of the rules of one entry, the
violation worded as every implementation must word it, and 16 trees,
the paths of a head and the result of decoding it: U+00A0, accepted,
and U+3000, R6c, at both ends of a segment; best-fit, full-width
forms, U+00B4 of cp1253; 8.3 aliases with ~1; Cn; U+206A to U+206F,
tags and other ignorables; a dot and ZWJ, and ZWJ alone; 127 and 85
times U+0390; U+F03A; .datekeys-x at two levels; U+FF5E and U+1F600
in both orders; ab with and without ZWNJ; U+00BF, U+00A7 and U+2665,
accepted; VS16 after U+2764 and after a; ZWJ at the start, at the end
and twice; the rainbow flag and the flag of Scotland; b/.. and a.
- path_fold.json: 22 segments with their NFD and their key of R7,
among them the entries F of CaseFolding, the dotless i, the Kelvin
and Angstrom signs, Cherokee, Hangul and the whitelist dropped
before NFD.
- head_schema.json: 63 heads through layers 2, 3 and 4, in key order,
with the violation of each ERR_HEAD_INVALID; they add comments with
tags and with loose variation selectors.
- security.json: 21 areas with their verdicts, X, F0, F1, S0, S1 and
S2, among them key 2 that is not a byte string, a key 4, a byte more,
alg 0, an empty key with the seal intact, and a seal that breaks its
schema with an unknown seal_type.
- cbor.json gains the control of schema version 3.
A test replays every committed vector through the implementation, and
another checks that the files are current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 week ago
{
"spec" : "0.16" ,
Format 3, step 6c: the vectors of paths, keys, heads and security
The vector files of spec 67 for format 3, generated with the result
each case is written for, so that the generator fails when the
implementation or the tables change:
- paths.json: 83 paths with the result of the rules of one entry, the
violation worded as every implementation must word it, and 16 trees,
the paths of a head and the result of decoding it: U+00A0, accepted,
and U+3000, R6c, at both ends of a segment; best-fit, full-width
forms, U+00B4 of cp1253; 8.3 aliases with ~1; Cn; U+206A to U+206F,
tags and other ignorables; a dot and ZWJ, and ZWJ alone; 127 and 85
times U+0390; U+F03A; .datekeys-x at two levels; U+FF5E and U+1F600
in both orders; ab with and without ZWNJ; U+00BF, U+00A7 and U+2665,
accepted; VS16 after U+2764 and after a; ZWJ at the start, at the end
and twice; the rainbow flag and the flag of Scotland; b/.. and a.
- path_fold.json: 22 segments with their NFD and their key of R7,
among them the entries F of CaseFolding, the dotless i, the Kelvin
and Angstrom signs, Cherokee, Hangul and the whitelist dropped
before NFD.
- head_schema.json: 63 heads through layers 2, 3 and 4, in key order,
with the violation of each ERR_HEAD_INVALID; they add comments with
tags and with loose variation selectors.
- security.json: 21 areas with their verdicts, X, F0, F1, S0, S1 and
S2, among them key 2 that is not a byte string, a key 4, a byte more,
alg 0, an empty key with the seal intact, and a seal that breaks its
schema with an unknown seal_type.
- cbor.json gains the control of schema version 3.
A test replays every committed vector through the implementation, and
another checks that the files are current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 week ago
"description" : "The key of R7 (spec §29.5) of segments, with the Unicode 18.0.0 tables of §29.5.1, generated by the reference implementation: nfd is NFD(segment) and key is NFD(fold(NFD(s'))), s' the segment without ZWNJ, ZWJ, VS15 and VS16. See testdata/README.md." ,
"unicode_version" : "18.0.0" ,
pathrule: the simple lower case of Unicode 18.0.0
The key of words lowers each code point with the simple mapping of
UnicodeData.txt, field 13, of Unicode 18.0.0 (spec v0.11, 38.1), not
with unicode.ToLower, which carries Unicode 15.0.0: U+A7CB lowers to
U+0264 in Unicode 16.0 and later, and a key made in the page, whose
platform knows it, would not open in the CLI.
The generator writes the table for Go and for datekeys-ts, and the
canonical text gains its lines, so TablesDigest changes; paths.json and
path_fold.json record the new one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6 days ago
"tables_digest" : "07cf5d54aea1cd13a3ecef14a06976cc49a3cdad755cf9bc10395178b93aeb07" ,
Format 3, step 6c: the vectors of paths, keys, heads and security
The vector files of spec 67 for format 3, generated with the result
each case is written for, so that the generator fails when the
implementation or the tables change:
- paths.json: 83 paths with the result of the rules of one entry, the
violation worded as every implementation must word it, and 16 trees,
the paths of a head and the result of decoding it: U+00A0, accepted,
and U+3000, R6c, at both ends of a segment; best-fit, full-width
forms, U+00B4 of cp1253; 8.3 aliases with ~1; Cn; U+206A to U+206F,
tags and other ignorables; a dot and ZWJ, and ZWJ alone; 127 and 85
times U+0390; U+F03A; .datekeys-x at two levels; U+FF5E and U+1F600
in both orders; ab with and without ZWNJ; U+00BF, U+00A7 and U+2665,
accepted; VS16 after U+2764 and after a; ZWJ at the start, at the end
and twice; the rainbow flag and the flag of Scotland; b/.. and a.
- path_fold.json: 22 segments with their NFD and their key of R7,
among them the entries F of CaseFolding, the dotless i, the Kelvin
and Angstrom signs, Cherokee, Hangul and the whitelist dropped
before NFD.
- head_schema.json: 63 heads through layers 2, 3 and 4, in key order,
with the violation of each ERR_HEAD_INVALID; they add comments with
tags and with loose variation selectors.
- security.json: 21 areas with their verdicts, X, F0, F1, S0, S1 and
S2, among them key 2 that is not a byte string, a key 4, a byte more,
alg 0, an empty key with the seal intact, and a seal that breaks its
schema with an unknown seal_type.
- cbor.json gains the control of schema version 3.
A test replays every committed vector through the implementation, and
another checks that the files are current.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 week ago
"keys" : [
{
"name" : "ASCII capitals" ,
"segment" : "A.txt" ,
"nfd" : "A.txt" ,
"key" : "a.txt"
} ,
{
"name" : "sharp s, entry F" ,
"segment" : "Straße" ,
"nfd" : "Straße" ,
"key" : "strasse"
} ,
{
"name" : "capital sharp s" ,
"segment" : "ẞ" ,
"nfd" : "ẞ" ,
"key" : "ss"
} ,
{
"name" : "the Kelvin sign, a singleton decomposition" ,
"segment" : "K ,
"nfd" : "K" ,
"key" : "k"
} ,
{
"name" : "the Angstrom sign" ,
"segment" : "Å" ,
"nfd" : "Å" ,
"key" : "å"
} ,
{
"name" : "dotless i, the extra folding of NTFS" ,
"segment" : "ı ,
"nfd" : "ı ,
"key" : "i"
} ,
{
"name" : "capital I with a dot, entry F" ,
"segment" : "İ" ,
"nfd" : "İ" ,
"key" : "i̇"
} ,
{
"name" : "precomposed e acute" ,
"segment" : "é" ,
"nfd" : "é" ,
"key" : "é"
} ,
{
"name" : "decomposed e acute" ,
"segment" : "é" ,
"nfd" : "é" ,
"key" : "é"
} ,
{
"name" : "final sigma" ,
"segment" : "ς" ,
"nfd" : "ς" ,
"key" : "σ
} ,
{
"name" : "U+0390, recursive decomposition" ,
"segment" : "ΐ" ,
"nfd" : "ΐ" ,
"key" : "ΐ"
} ,
{
"name" : "a Cherokee small letter folds to the capital" ,
"segment" : "ꭰ" ,
"nfd" : "ꭰ" ,
"key" : "Ꭰ
} ,
{
"name" : "a Cherokee capital stays" ,
"segment" : "Ꭰ ,
"nfd" : "Ꭰ ,
"key" : "Ꭰ
} ,
{
"name" : "ligature ff, entry F" ,
"segment" : "ff" ,
"nfd" : "ff" ,
"key" : "ff"
} ,
{
"name" : "DZ with caron, titlecase" ,
"segment" : "Dž" ,
"nfd" : "Dž" ,
"key" : "dž"
} ,
{
"name" : "Hangul syllable with a final jamo" ,
"segment" : "한" ,
"nfd" : "한" ,
"key" : "한"
} ,
{
"name" : "canonical order of two marks" ,
"segment" : "ạ́" ,
"nfd" : "ạ́" ,
"key" : "ạ́"
} ,
{
"name" : "ZWJ between marks is dropped before NFD" ,
"segment" : "á ,
"nfd" : "á ,
"key" : "ạ́"
} ,
{
"name" : "ZWNJ is dropped" ,
"segment" : "a ,
"nfd" : "a ,
"key" : "ab"
} ,
{
"name" : "VS16 and ZWJ of the rainbow flag are dropped" ,
"segment" : "🏳️🌈" ,
"nfd" : "🏳️🌈" ,
"key" : "🏳🌈"
} ,
{
"name" : "a reserved prefix, compared by its key" ,
"segment" : ".DateKeys-X" ,
"nfd" : ".DateKeys-X" ,
"key" : ".datekeys-x"
} ,
{
"name" : "full-width letters fold, not to ASCII" ,
"segment" : "Ｃ Ｏ Ｎ ,
"nfd" : "Ｃ Ｏ Ｎ ,
"key" : "ｃ ｏ ｎ
}
]
}
