deploy: 871de5b

giellalt · Nov 15, 2024 · 074b3bb · 074b3bb
1 parent 35af84a
commit 074b3bb
Show file tree

Hide file tree

Showing 4 changed files with 37 additions and 3 deletions.
diff --git a/index-header.md b/index-header.md
@@ -10,7 +10,7 @@ This page documents the work on the **Tokelauan language model**.
 
 ## Project documentation
 
-* Add links to project specific documentation here as needed. Keep the documentation in the `docs/` directory.
+* [Test diary](test-diary.md)
 
 ## In-source documentation
 

diff --git a/index.md b/index.md
@@ -10,7 +10,7 @@ This page documents the work on the **Tokelauan language model**.
 
 ## Project documentation
 
-* Add links to project specific documentation here as needed. Keep the documentation in the `docs/` directory.
+* [Test diary](test-diary.md)
 
 ## In-source documentation
 

diff --git a/lemmacount.json b/lemmacount.json
@@ -1 +1 @@
-{ "schemaVersion": 1, "label": "Lemmas", "message": "4.7 K", "color": "red" }
+{ "schemaVersion": 1, "label": "Lemmas", "message": "5.4 K", "color": "red" }
diff --git a/test-diary.md b/test-diary.md
@@ -0,0 +1,34 @@
+Test diary
+==========
+
+
+## Lexical coverage 
+
+Number of words (standing in `lang-tkl`):
+
+```
+cat misc/nt_tkl.txt |hfst-tokenise -mgW tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst |grep ' ?'|wc -l
+```
+
+Number of unknown words:
+
+
+```
+cat misc/nt_tkl.txt |hfst-tokenise -m tools/tokenisers/tokeniser-disamb-gt-desc.pmhfst |wc -l
+```
+
+### Lexical coverage nt
+
+
+Coverage:
+
+```
+241115: 1-(34193/351080) = 0.9026
+
+```
+
+
+
+
+
+
Original file line number	Diff line number	Diff line change
		@@ -1 +1 @@
		{ "schemaVersion": 1, "label": "Lemmas", "message": "4.7 K", "color": "red" }
		{ "schemaVersion": 1, "label": "Lemmas", "message": "5.4 K", "color": "red" }