testdata and wordlists at datekeys-go 27a75ee: the word lists, vendored

tool/sync_testdata.dart also copies every blob under wordkey/lists of the
same commit of Go into wordlists/, with a SOURCE.json in the format of
testdata/SOURCE.json, after reading every blob of both trees; a tree missing
at the commit is an error. check verifies both trees with the same rules,
also against the repository, and prints one line per tree, testdata first;
test/testdata_test.dart checks the wordlists tree and its line too.

Synced at 27a75ee, the branch v0.15 after the tag spec-v0.15: the 142 files
of testdata are those of the tag, byte for byte, so that only the commit of
testdata/SOURCE.json changes; wordlists/ holds README.md and es.txt, the
Spanish list, whose SHA-256 is the one that wordlist.dart pins. The list is
CC BY-SA 4.0, an adaptation of FrequencyWords by Hermit Dave, unlike the
code; its README, copied from Go, records its source, method and license.
.gitattributes keeps the exact bytes of wordlists/.

wordlist_vm_test.dart reads the Spanish list with readWordList, as
TestBuiltInLists of Go, draws from it the words of Go, and runs TestGenerate
and TestGenerateUniform with the CSPRNG of the platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v0.15
dev 12 hours ago
parent a770d4d8b0
commit e29f4299b0

4
.gitattributes vendored

@ -4,3 +4,7 @@
# Vendored Go fixtures and vectors must keep their exact bytes: their SHA-256 is # Vendored Go fixtures and vectors must keep their exact bytes: their SHA-256 is
# recorded in testdata/SOURCE.json and in the fixtures themselves. # recorded in testdata/SOURCE.json and in the fixtures themselves.
testdata/** -text testdata/** -text
# So must the vendored word lists of wordkey.Generate: their SHA-256 is
# recorded in wordlists/SOURCE.json and pinned in lib/src/wordlist.dart.
wordlists/** -text

@ -3,7 +3,8 @@
// than pass with fewer cases. The check is the one of tool/sync_testdata.dart: // than pass with fewer cases. The check is the one of tool/sync_testdata.dart:
// every file matches the SHA-256 recorded in testdata/SOURCE.json for the // every file matches the SHA-256 recorded in testdata/SOURCE.json for the
// pinned datekeys-go commit, with none missing and none extra. And every file // pinned datekeys-go commit, with none missing and none extra. And every file
// names the specification that this package implements. // names the specification that this package implements. The same check holds
// for wordlists/, the word lists of wordkey/lists of the same commit.
@TestOn('vm') @TestOn('vm')
library; library;
@ -24,8 +25,29 @@ void main() {
expect(r.module, testdata.module); expect(r.module, testdata.module);
expect(r.commit, matches(RegExp(r'^[0-9a-f]{40}$'))); expect(r.commit, matches(RegExp(r'^[0-9a-f]{40}$')));
expect(r.files, greaterThan(100)); expect(r.files, greaterThan(100));
expect(
r.line,
'testdata: ${r.files} files match ${r.module} at ${r.commit}',
);
}); });
test(
'wordlists/ matches SOURCE.json byte for byte, at the commit of testdata/',
() {
final r = testdata.check(root, tree: testdata.wordlistsTree);
expect(r.errors, isEmpty);
expect(r.module, testdata.module);
expect(r.commit, testdata.check(root).commit);
// The README of the lists, with their source and license, and a list.
expect(r.files, greaterThanOrEqualTo(2));
expect(
r.line,
'wordlists: ${r.files} files match ${r.module} at ${r.commit}',
);
expect(testdata.trees, [testdata.testdataTree, testdata.wordlistsTree]);
},
);
test('every JSON of testdata/ with a spec field names specVersion', () { test('every JSON of testdata/ with a spec field names specVersion', () {
var named = 0; var named = 0;
final files = Directory('${root.path}/testdata') final files = Directory('${root.path}/testdata')

@ -1,18 +1,24 @@
// The word lists against files, and at length: the copy of // The word lists against files, and at length: the copy of
// test/vectors/wordlist_vectors.json that wordlist_test.dart reads, a Dart // test/vectors/wordlist_vectors.json that wordlist_test.dart reads, a Dart
// constant for the tests compiled to JavaScript, is the JSON file byte for // constant for the tests compiled to JavaScript, is the JSON file byte for
// byte; and checkWordList gives the result of Go's CheckList for every code // byte; checkWordList gives the result of Go's CheckList for every code
// point of planes 0, 1 and 14. On the VM only: they read a file or take a // point of planes 0, 1 and 14; readWordList reads wordlists/es.txt, the
// few seconds. // Spanish list of datekeys-go, as Go's List reads the list built into it
// (TestBuiltInLists), and generateWords draws from it the words of Go; and,
// as TestGenerate and TestGenerateUniform of Go, the words drawn with the
// CSPRNG of the platform are a key, and each about as likely. On the VM
// only: they read files, use Random.secure or take a few seconds.
@TestOn('vm') @TestOn('vm')
library; library;
import 'dart:convert'; import 'dart:convert';
import 'dart:io'; import 'dart:io';
import 'dart:typed_data';
import 'package:datekeys/src/bytes.dart'; import 'package:datekeys/src/bytes.dart';
import 'package:datekeys/src/pathrule.dart' show codePointName; import 'package:datekeys/src/pathrule.dart' show codePointName;
import 'package:datekeys/src/sha256.dart'; import 'package:datekeys/src/sha256.dart';
import 'package:datekeys/src/wordkey.dart' show checkWords, normalizeWords;
import 'package:datekeys/src/wordlist.dart'; import 'package:datekeys/src/wordlist.dart';
import 'package:test/test.dart'; import 'package:test/test.dart';
@ -50,4 +56,104 @@ void main() {
expect(toHex(digest.finish()), a['sha256']); expect(toHex(digest.finish()), a['sha256']);
expect(accepted, a['ok']); expect(accepted, a['ok']);
}); });
group('the Spanish list of wordlists/', () {
final file = File('wordlists/es.txt').readAsBytesSync();
final es = readWordList('es', file);
List<Json> cases(String name) => (vectors[name]! as List).cast<Json>();
test('is the list of Go, with its pinned SHA-256 (TestBuiltInLists)', () {
expect(wordListLanguages(), ['es']);
final go = cases('lists').single;
expect(go['lang'], 'es');
expect(toHex(sha256(file)), wordListSha256['es']);
expect(file, hasLength(go['bytes']));
expect(es, hasLength(go['words']));
expect(es, hasLength(7776));
// wordlists/SOURCE.json records the pinned SHA-256 of each list.
final source =
jsonDecode(File('wordlists/SOURCE.json').readAsStringSync()) as Json;
final files = source['files']! as Json;
for (final lang in wordListLanguages()) {
expect(files['$lang.txt'], wordListSha256[lang], reason: lang);
}
// The words are the lines of the file, and they pass checkWordList.
expect(utf8Bytes('${es.join('\n')}\n'), file);
checkWordList('es', es);
expect(
outcome(() => readWordList('xx', file)),
'wordkey: no word list for "xx"; the lists are es',
);
});
test('readWordList refuses the list with a byte changed, more or less', () {
for (final changed in [
for (final i in [0, 1, file.length ~/ 2, file.length - 1])
Uint8List.fromList(file)..[i] ^= 0x20,
file.sublist(0, file.length - 1),
Uint8List.fromList([...file, 0x0a]),
]) {
expect(
outcome(() => readWordList('es', changed)),
'wordkey: the list "es" has the SHA-256 ${toHex(sha256(changed))}, '
'not ${wordListSha256['es']}',
);
}
});
test('generateWords draws from it the words of Go', () {
var drawn = 0;
for (final c in cases('generate')) {
if (c['list'] != 'es' || c['error'] != null) continue;
final name = c['name']! as String;
final words = generateWords(
es,
c['n']! as int,
streamOf(c['stream']! as Json),
);
final indices = (c['indices']! as List).cast<int>();
expect(words, [for (final i in indices) es[i]], reason: name);
if (c['words'] != null) expect(words, c['words'], reason: name);
drawn++;
}
expect(drawn, greaterThan(10));
});
test('the words drawn with the CSPRNG of the platform are a key '
'(TestGenerate)', () {
final words = generateWords(es);
expect(words, hasLength(defaultWordCount));
checkWords(normalizeWords(words.join(' ')));
expect(words.toSet(), hasLength(words.length));
for (final (n, want) in [
(5, 'at least 6 words, not 5'),
(es.length, '7776 words of a list of 7776'),
]) {
expect(outcome(() => generateWords(es, n)), contains(want));
}
});
// Over 7776·40 draws of one word, each index falls in its bucket of 64
// between 0.8 and 1.2 times the mean.
test('every word is about as likely (TestGenerateUniform)', () {
final index = {for (var i = 0; i < es.length; i++) es[i]: i};
const buckets = 64;
final count = List.filled(buckets, 0);
var draws = 0;
while (draws < es.length * 40) {
for (final w in generateWords(es, 6)) {
count[index[w]! * buckets ~/ es.length]++;
draws++;
}
}
final mean = draws / buckets;
for (var i = 0; i < buckets; i++) {
expect(
count[i],
inInclusiveRange(0.8 * mean, 1.2 * mean),
reason: 'bucket $i',
);
}
});
});
} }

@ -1,6 +1,6 @@
{ {
"module": "g.activething.com/go/DateKeys", "module": "g.activething.com/go/DateKeys",
"commit": "fe5088549186465e08d55f89680be986076bf165", "commit": "27a75eefcd2ac2a3b32930d6872f3b5b6e648792",
"files": { "files": {
"README.md": "d26b3f507edf5afe89600f063cb509b9ef62a908b93ed476bff964b5a443bf4b", "README.md": "d26b3f507edf5afe89600f063cb509b9ef62a908b93ed476bff964b5a443bf4b",
"fixtures/empty_payload.dkc": "871e9bf05b52bbae17f3adfbbf97b46e7f0e53aa8f57bcaa506e43f36f53a9d4", "fixtures/empty_payload.dkc": "871e9bf05b52bbae17f3adfbbf97b46e7f0e53aa8f57bcaa506e43f36f53a9d4",

@ -1,8 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Local gate of datekeys-dart, as scripts/check.sh of datekeys-go and npm run # Local gate of datekeys-dart, as scripts/check.sh of datekeys-go and npm run
# verify of datekeys-ts: format, analysis, tests on the VM and compiled to # verify of datekeys-ts: format, analysis, tests on the VM and compiled to
# JavaScript, and the copy of testdata/ against the Go repository next to this # JavaScript, and the copies of testdata/ and wordlists/ against the Go
# one. Run it before every commit. It needs Node.js, as datekeys-ts does. # repository next to this one. Run it before every commit. It needs Node.js, as
# datekeys-ts does.
# #
# tool/check.sh # tool/check.sh
set -euo pipefail set -euo pipefail
@ -23,7 +24,7 @@ dart test
echo "== dart test -p node" echo "== dart test -p node"
dart test -p node dart test -p node
echo "== testdata against datekeys-go" echo "== testdata and wordlists against datekeys-go"
dart run tool/sync_testdata.dart check --against ../datekeys-go dart run tool/sync_testdata.dart check --against ../datekeys-go
echo "all checks passed" echo "all checks passed"

@ -1,37 +1,64 @@
// Vendors testdata/ from the Go reference implementation // Vendors testdata/ and the word lists of wordkey/lists/ from the Go
// (g.activething.com/go/DateKeys) at one pinned commit and verifies it. The Go // reference implementation (g.activething.com/go/DateKeys) at one pinned
// fixtures and vectors are the source of truth: this package never generates // commit and verifies them. The Go fixtures, vectors and lists are the source
// its own. A port of scripts/sync-testdata.mjs of datekeys-ts, with the same // of truth: this package never generates its own. A port of
// testdata/SOURCE.json. // scripts/sync-testdata.mjs of datekeys-ts, with the same testdata/SOURCE.json.
// //
// dart run tool/sync_testdata.dart sync [--repo ../datekeys-go] [--commit HEAD] // dart run tool/sync_testdata.dart sync [--repo ../datekeys-go] [--commit HEAD]
// dart run tool/sync_testdata.dart check [--against ../datekeys-go] // dart run tool/sync_testdata.dart check [--against ../datekeys-go]
// //
// sync copies every blob under testdata/ at the commit (read with git, so // sync copies every blob under testdata/ and under wordkey/lists/ at the
// uncommitted changes in the Go checkout are never picked up) and writes // commit (read with git, so uncommitted changes in the Go checkout are never
// testdata/SOURCE.json with the full commit id and the SHA-256 of each file. // picked up) into testdata/ and wordlists/, and writes the SOURCE.json of
// check verifies that the files on disk match SOURCE.json exactly, with no // each, with the full commit id and the SHA-256 of each file. It reads every
// missing or extra files; with --against it also re-reads every blob from the // blob of both before it touches either copy, and a tree missing at the
// Go repository at the recorded commit. test/testdata_test.dart runs check. // commit is an error. check verifies that the files on disk of each tree
// No dependencies but package:crypto, Dart and git. // match its SOURCE.json exactly, with no missing or extra files; with
// --against it also re-reads every blob from the Go repository at the
// recorded commit. It prints one line per tree, testdata first.
// test/testdata_test.dart runs check. No dependencies but package:crypto,
// Dart and git.
import 'dart:convert'; import 'dart:convert';
import 'dart:io'; import 'dart:io';
import 'package:crypto/crypto.dart'; import 'package:crypto/crypto.dart';
/// The module whose testdata/ is vendored. /// The module whose trees are vendored.
const module = 'g.activething.com/go/DateKeys'; const module = 'g.activething.com/go/DateKeys';
/// What check found: the files, the commit and the module of SOURCE.json, and /// A tree that sync vendors: [dir], a directory of this package, holds the
/// every difference, none when the copy is intact. /// files of [upstream], a directory of the Go repository.
class Tree {
const Tree(this.dir, this.upstream);
final String dir;
final String upstream;
}
/// testdata/, the fixtures and the vectors of the Go reference.
const testdataTree = Tree('testdata', 'testdata');
/// wordlists/, the word lists of wordkey.Generate, whose SHA-256
/// lib/src/wordlist.dart pins.
const wordlistsTree = Tree('wordlists', 'wordkey/lists');
/// The trees, in the order of the lines of check.
const trees = [testdataTree, wordlistsTree];
/// What check found in a tree: the files, the commit and the module of its
/// SOURCE.json, and every difference, none when the copy is intact.
class CheckResult { class CheckResult {
CheckResult(this.module, this.commit, this.files, this.errors); CheckResult(this.tree, this.module, this.commit, this.files, this.errors);
final Tree tree;
final String module; final String module;
final String commit; final String commit;
final int files; final int files;
final List<String> errors; final List<String> errors;
/// The line that check prints for an intact copy.
String get line => '${tree.dir}: $files files match $module at $commit';
} }
void main(List<String> args) { void main(List<String> args) {
@ -46,23 +73,28 @@ void main(List<String> args) {
sync(root, Directory(repo).absolute, commit); sync(root, Directory(repo).absolute, commit);
case 'check': case 'check':
final against = _option(rest, '--against'); final against = _option(rest, '--against');
final r = check(
root,
against: against == null ? null : Directory(against).absolute,
);
if (r.errors.isNotEmpty) {
for (final e in r.errors) {
stderr.writeln('testdata: $e');
}
exitCode = 1;
return;
}
final verified = against == null final verified = against == null
? '' ? ''
: ' (verified against the repository)'; : ' (verified against the repository)';
stdout.writeln( for (final tree in trees) {
'testdata: ${r.files} files match ${r.module} at ${r.commit}$verified', try {
); final r = check(
root,
tree: tree,
against: against == null ? null : Directory(against).absolute,
);
if (r.errors.isEmpty) {
stdout.writeln('${r.line}$verified');
continue;
}
for (final e in r.errors) {
stderr.writeln('${tree.dir}: $e');
}
} catch (e) {
stderr.writeln('${tree.dir}: $e');
}
exitCode = 1;
}
default: default:
stderr.writeln( stderr.writeln(
'usage: sync_testdata.dart sync [--repo PATH] [--commit REV] | check [--against PATH]', 'usage: sync_testdata.dart sync [--repo PATH] [--commit REV] | check [--against PATH]',
@ -70,56 +102,72 @@ void main(List<String> args) {
exitCode = 2; exitCode = 2;
} }
} catch (e) { } catch (e) {
stderr.writeln('testdata: $e'); stderr.writeln('$command: $e');
exitCode = 1; exitCode = 1;
} }
} }
/// Copies testdata/ of the Go repository [repo] at [rev] into [root], and /// Copies each tree of [trees] of the Go repository [repo] at [rev] into
/// writes testdata/SOURCE.json. /// [root], and writes its SOURCE.json.
void sync(Directory root, Directory repo, String rev) { void sync(Directory root, Directory repo, String rev) {
final commit = utf8 final commit = utf8
.decode(_git(repo, ['rev-parse', '--verify', '$rev^{commit}'])) .decode(_git(repo, ['rev-parse', '--verify', '$rev^{commit}']))
.trim(); .trim();
final paths = _listBlobs(repo, commit); final listed = [
if (paths.isEmpty) throw StateError('no testdata/ at $commit'); for (final tree in trees) (tree, _listBlobs(repo, commit, tree)),
// Read everything before touching the local copy, so a failed read leaves it intact. ];
final blobs = [for (final p in paths) (p, _readBlob(repo, commit, p))]; for (final (tree, paths) in listed) {
final testdata = Directory('${root.path}/testdata'); if (paths.isEmpty) throw StateError('no ${tree.upstream}/ at $commit');
if (testdata.existsSync()) testdata.deleteSync(recursive: true); }
final files = <String, String>{}; // Read everything before touching the local copies, so a failed read
for (final (path, data) in blobs) { // leaves them intact.
File('${testdata.path}/$path') final read = [
..createSync(recursive: true) for (final (tree, paths) in listed)
..writeAsBytesSync(data); (tree, [for (final p in paths) (p, _readBlob(repo, commit, tree, p))]),
files[path] = _sha256(data); ];
for (final (tree, blobs) in read) {
final dir = Directory('${root.path}/${tree.dir}');
if (dir.existsSync()) dir.deleteSync(recursive: true);
final files = <String, String>{};
for (final (path, data) in blobs) {
File('${dir.path}/$path')
..createSync(recursive: true)
..writeAsBytesSync(data);
files[path] = _sha256(data);
}
final source = {'module': module, 'commit': commit, 'files': files};
File('${dir.path}/SOURCE.json').writeAsStringSync(
'${const JsonEncoder.withIndent(' ').convert(source)}\n',
);
stdout.writeln(
'${tree.dir}: ${blobs.length} files from $module at $commit',
);
} }
final source = {'module': module, 'commit': commit, 'files': files};
File('${testdata.path}/SOURCE.json').writeAsStringSync(
'${const JsonEncoder.withIndent(' ').convert(source)}\n',
);
stdout.writeln('testdata: ${paths.length} files from $module at $commit');
} }
/// Checks testdata/ of [root] against its SOURCE.json and, with [against], /// Checks [tree] of [root], testdata/ by default, against its SOURCE.json
/// against the Go repository at the recorded commit. /// and, with [against], against the Go repository at the recorded commit.
CheckResult check(Directory root, {Directory? against}) { CheckResult check(
final testdata = Directory('${root.path}/testdata'); Directory root, {
Tree tree = testdataTree,
Directory? against,
}) {
final dir = Directory('${root.path}/${tree.dir}');
final source = jsonDecode( final source = jsonDecode(
File('${testdata.path}/SOURCE.json').readAsStringSync(), File('${dir.path}/SOURCE.json').readAsStringSync(),
) as Map<String, Object?>; ) as Map<String, Object?>;
final recorded = (source['files']! as Map<String, Object?>) final recorded = (source['files']! as Map<String, Object?>)
.cast<String, String>(); .cast<String, String>();
final commit = source['commit']! as String; final commit = source['commit']! as String;
final expected = recorded.keys.toList()..sort(); final expected = recorded.keys.toList()..sort();
final actual = _listLocal(testdata); final actual = _listLocal(dir);
final errors = <String>[]; final errors = <String>[];
for (final path in expected) { for (final path in expected) {
if (!actual.contains(path)) { if (!actual.contains(path)) {
errors.add('missing $path'); errors.add('missing $path');
continue; continue;
} }
final data = File('${testdata.path}/$path').readAsBytesSync(); final data = File('${dir.path}/$path').readAsBytesSync();
if (_sha256(data) != recorded[path]) { if (_sha256(data) != recorded[path]) {
errors.add('modified $path'); errors.add('modified $path');
} }
@ -128,17 +176,18 @@ CheckResult check(Directory root, {Directory? against}) {
if (!recorded.containsKey(path)) errors.add('extra $path'); if (!recorded.containsKey(path)) errors.add('extra $path');
} }
if (against != null) { if (against != null) {
final upstream = _listBlobs(against, commit); final upstream = _listBlobs(against, commit, tree);
if (upstream.join('\n') != expected.join('\n')) { if (upstream.join('\n') != expected.join('\n')) {
errors.add('file list differs from $commit'); errors.add('file list differs from $commit');
} }
for (final path in upstream) { for (final path in upstream) {
if (_sha256(_readBlob(against, commit, path)) != recorded[path]) { if (_sha256(_readBlob(against, commit, tree, path)) != recorded[path]) {
errors.add('differs from upstream $path'); errors.add('differs from upstream $path');
} }
} }
} }
return CheckResult( return CheckResult(
tree,
source['module']! as String, source['module']! as String,
commit, commit,
expected.length, expected.length,
@ -162,8 +211,9 @@ List<int> _git(Directory repo, List<String> args) {
String _sha256(List<int> data) => sha256.convert(data).toString(); String _sha256(List<int> data) => sha256.convert(data).toString();
// Files under testdata/ at commit, as paths relative to testdata/ with forward slashes. // Files under the upstream directory of tree at commit, as paths relative to
List<String> _listBlobs(Directory repo, String commit) { // it with forward slashes.
List<String> _listBlobs(Directory repo, String commit, Tree tree) {
final out = utf8.decode( final out = utf8.decode(
_git(repo, [ _git(repo, [
'ls-tree', 'ls-tree',
@ -172,7 +222,7 @@ List<String> _listBlobs(Directory repo, String commit) {
'--full-tree', '--full-tree',
commit, commit,
'--', '--',
'testdata', tree.upstream,
]), ]),
); );
final paths = <String>[]; final paths = <String>[];
@ -183,20 +233,20 @@ List<String> _listBlobs(Directory repo, String commit) {
if (meta[1] != 'blob' || meta[0] == '120000') { if (meta[1] != 'blob' || meta[0] == '120000') {
throw StateError('unsupported entry $path (${meta[0]} ${meta[1]})'); throw StateError('unsupported entry $path (${meta[0]} ${meta[1]})');
} }
paths.add(path.substring('testdata/'.length)); paths.add(path.substring('${tree.upstream}/'.length));
} }
return paths..sort(); return paths..sort();
} }
List<int> _readBlob(Directory repo, String commit, String path) => List<int> _readBlob(Directory repo, String commit, Tree tree, String path) =>
_git(repo, ['cat-file', 'blob', '$commit:testdata/$path']); _git(repo, ['cat-file', 'blob', '$commit:${tree.upstream}/$path']);
// Files on disk under testdata/, excluding SOURCE.json. // Files on disk under dir, excluding its SOURCE.json.
List<String> _listLocal(Directory testdata) { List<String> _listLocal(Directory dir) {
final prefix = testdata.absolute.path.length + 1; final prefix = dir.absolute.path.length + 1;
final files = <String>[]; final files = <String>[];
for (final f for (final f
in testdata.absolute in dir.absolute
.listSync(recursive: true, followLinks: false) .listSync(recursive: true, followLinks: false)
.whereType<File>()) { .whereType<File>()) {
final path = f.path final path = f.path

@ -0,0 +1,46 @@
# Word lists of `wordkey.Generate`
The lists from which `datekeys encrypt -new-words` draws a key of words at
random (spec §38.1: at least 6 words of a public list of 2048 or more). They
are not normative: a reader does not need them, because the key is derived
from the normalized text of the words, whatever list they came from.
| List | Words | SHA-256 | Status |
|---|---|---|---|
| `es.txt` | 7776 | `ff77b487765c000da97cca58fe94a2cdb947303e7a07460614d7d95d800034fe` | Draft, not yet reviewed by a native speaker |
A list changes only with its hash in this file and in `generate_test.go`:
an application that downloads a list pins its SHA-256 and refuses any other.
`wordkey.CheckList` checks a list against the alphabet of its language,
which the code gives and not the list: a word with a letter of another script
that looks the same, such as a Cyrillic `а` (U+0430) for a Latin `a`, would be
typed again with the letter of the keyboard, and the capsule would not open.
| Language | Alphabet |
|---|---|
| `es` | `a` to `z`, `á`, `é`, `í`, `ó`, `ú`, `ü` and `ñ`, in lower case and NFC |
## `es.txt`
- **Source:** the frequencies of
[FrequencyWords](https://github.com/hermitdave/FrequencyWords) by Hermit
Dave, `content/2018/es/es_50k.txt`, counted on the OpenSubtitles 2018
corpus, licensed under CC BY-SA 4.0. The Spanish Hunspell dictionary of
LibreOffice (RLA-ES, `es_ES.dic` and `es_ES.aff`) is used only as a filter
and is not redistributed.
- **License:** this list is an adaptation of FrequencyWords and is licensed
under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/),
unlike the code of this module (Apache 2.0).
- **Method:** the most frequent words that are, in order:
1. lowercase letters only, of 3 to 9 letters;
2. a base form of the dictionary: an entry with affix flags, or the
feminine its flag `G` makes. Entries without flags, which are
conjugations, plurals and pieces of names, are left out;
3. not in a short list of offensive or unpleasant words;
4. not the other half of a pair that differs only in a final `-o` or `-a`
(`chico` and `chica`): the more frequent stays;
5. not the same as a word already taken once normalized as in §38.1
(`papa` and `papá`): the more frequent stays.
The first 7776 that pass, sorted. `wordkey.CheckList` checks the last rule,
the characters of every word and the alphabet.

@ -0,0 +1,8 @@
{
"module": "g.activething.com/go/DateKeys",
"commit": "27a75eefcd2ac2a3b32930d6872f3b5b6e648792",
"files": {
"README.md": "db71bbd031a335adeeff9e0453fdc2b2378b3bacc0104907efdfbc72fc7e6d6a",
"es.txt": "ff77b487765c000da97cca58fe94a2cdb947303e7a07460614d7d95d800034fe"
}
}

File diff suppressed because it is too large Load Diff
Loading…
Cancel
Save

Powered by TurnKey Linux.