Code

Calamari2#118

Code

Open

bertsky wants to merge 56 commits intoOCR-D:masterOCR-D/ocrd_calamari:masterfrom bertsky:calamari2bertsky/ocrd_calamari:calamari2Copy head branch name to clipboard

+1,641-805

Commits

Commits on Sep 13, 2024

• adapt to ocrd>=3.0

  bertsky

  committed
  bf755a3

  Copy full SHA for bf755a3

Commits on Sep 15, 2024

• make test: no assumption on OCRD resource location

  bertsky

  committed
  1edd5e7

  Copy full SHA for 1edd5e7

• tests: adapt to v3, overhaul and add caching+threading modes

  bertsky

  committed
  3333cab

  Copy full SHA for 3333cab

Commits on Sep 16, 2024

• require ocrd 3.0 and calamari-ocr 1.0.7

  bertsky

  committed
  7aae9bc

  Copy full SHA for 7aae9bc

Commits on Sep 17, 2024

• aggregate all lines instead of per region to better utilise batched predictor

  bertsky

  committed
  9611e2c

  Copy full SHA for 9611e2c

Commits on Sep 18, 2024

• run prediction in bg thread (shared across pages to interleave CPU/GPU)

  bertsky

  committed
  fb2a680

  Copy full SHA for fb2a680

• let GPU memory grow by demand (instead of exclusive reservation)

  bertsky

  committed
  b9b0e13

  Copy full SHA for b9b0e13

• no more need for model fixup

  bertsky

  committed
  46c2ef6

  Copy full SHA for 46c2ef6

• CI: increase RAM

  bertsky

  committed
  e68ce5f

  Copy full SHA for e68ce5f

Commits on Sep 23, 2024

• 🔥 adapt to (and require) Calamari 2…

  Show description for 4904a00
  bertsky

  committed
  4904a00

  Copy full SHA for 4904a00

Commits on Sep 26, 2024

• wait for executor to fully initialize (so we get input_channels)

  bertsky

  committed
  f7a870e

  Copy full SHA for f7a870e

• convert images to input_channels (required TensorShape)

  bertsky

  committed
  1c13622

  Copy full SHA for 1c13622

• voter cannot be parallel; shmem is faster

  bertsky

  committed
  26ba7f5

  Copy full SHA for 26ba7f5

• test configs: test metsserver independent of pageparallel

  bertsky

  committed
  e99daa4

  Copy full SHA for e99daa4

• test with pytest-isolate so TF memory can be freed between tests

  bertsky

  committed
  27dc858

  Copy full SHA for 27dc858

Commits on Sep 27, 2024

• expose device parameter, default to GPU

  bertsky

  committed
  55c175c

  Copy full SHA for 55c175c

• 📦 2.0.0

  bertsky

  committed
  1a41330

  Copy full SHA for 1a41330

• add C2 models

  bertsky

  committed
  37ff61e

  Copy full SHA for 37ff61e

• use C2 deep3_fraktur19 model for testing

  bertsky

  committed
  260f184

  Copy full SHA for 260f184

• update models (5→6 HDF5→SavedModel)

  bertsky

  committed
  92cd36d

  Copy full SHA for 92cd36d

Commits on Oct 1, 2024

• update requirements (Calamari 2.3, remove implied TF)

  bertsky

  committed
  c24d99c

  Copy full SHA for c24d99c

• update models (2.1→2.2 for Py3.11 re syntax)

  bertsky

  committed
  25ab12b

  Copy full SHA for 25ab12b

Commits on Oct 2, 2024

• 🔥 change license from Apache 2 to GPL 3

  bertsky

  committed
  0e35701

  Copy full SHA for 0e35701

Commits on Oct 9, 2024

• switch from ThreadPoolExecutor to custom mp.Process bg task

  bertsky

  committed
  25e3560

  Copy full SHA for 25e3560

• pass only line IDs instead of full TextLine objects

  bertsky

  committed
  da60d42

  Copy full SHA for da60d42

Commits on Oct 18, 2024

• fix error handling and IPC with page workers…

  Show description for bb6321b
  bertsky

  committed
  bb6321b

  Copy full SHA for bb6321b

• use tf.data generator for model.predict_on_batch instead of tfaip's

  Show description for 3bb6cf4
  bertsky

  committed
  3bb6cf4

  Copy full SHA for 3bb6cf4

• reduce base batch size -- too large for long lines

  Show description for f1c54d0
  bertsky

  committed
  f1c54d0

  Copy full SHA for f1c54d0

• Merge branch 'master' into calamari2

  bertsky

  authored
  d054512

  Copy full SHA for d054512

• CI: test on smaller model

  bertsky

  authored
  5dafcfe

  Copy full SHA for 5dafcfe

Commits on Oct 19, 2024

• ocrd-tool.json: fix 2.2 model URLs (zip→tar.gz)

  bertsky

  authored
  816562a

  Copy full SHA for 816562a

• CI: wait longer for test without output

  bertsky

  authored
  8e37d0d

  Copy full SHA for 8e37d0d

• Makefile: export MODEL variable

  bertsky

  authored
  1fb24af

  Copy full SHA for 1fb24af

• ocrd-tool.json: remove model default

  Show description for 5545337
  bertsky

  authored
  5545337

  Copy full SHA for 5545337

• disallow TF 2.12.0

  bertsky

  authored
  4d46d49

  Copy full SHA for 4d46d49

Commits on Oct 21, 2024

• robust shutdown

  bertsky

  committed
  5af55e3

  Copy full SHA for 5af55e3

Commits on Oct 22, 2024

• configure infinite dataset/pipeline via queue during setup, run one thread per line to send task and receive result

  bertsky

  committed
  9276c8b

  Copy full SHA for 9276c8b

Commits on Oct 30, 2024

• line task threads: catch and raise when joining

  bertsky

  committed
  03b69ec

  Copy full SHA for 03b69ec

• cannot do bucket batching in this mode (occasionally stalls the pipeline)

  bertsky

  committed
  fdb5ef4

  Copy full SHA for fdb5ef4

• don't confuse received result with expected result

  bertsky

  committed
  384f107

  Copy full SHA for 384f107

• fill up batches as long as any workers are receiving results

  bertsky

  committed
  fcb40c1

  Copy full SHA for fcb40c1

• fix termination/interruption

  bertsky

  committed
  05b77c7

  Copy full SHA for 05b77c7

• require ocrd 3.0.0b6 (mp)

  bertsky

  committed
  d1c74fc

  Copy full SHA for d1c74fc

Commits on Oct 31, 2024

• Merge branch 'calamari2-subprocess' into calamari2

  bertsky

  committed
  2dfba15

  Copy full SHA for 2dfba15

Commits on Nov 12, 2024

• replace atexit with weakref, also add destructor

  bertsky

  committed
  f97eb97

  Copy full SHA for f97eb97

• fix word/glyph level

  bertsky

  committed
  fdf452e

  Copy full SHA for fdf452e

• test: proper shutdown w/ no side effects

  bertsky

  committed
  a3142b8

  Copy full SHA for a3142b8

• test: no need for fixture as dict

  bertsky

  committed
  c721161

  Copy full SHA for c721161

• test: default to supported model

  bertsky

  committed
  594b9b7

  Copy full SHA for 594b9b7

• test: no need for --isolate anymore

  bertsky

  committed
  a5f0df5

  Copy full SHA for a5f0df5

• require ocrd 3.0.0b7 (sub logging)

  bertsky

  committed
  175583b

  Copy full SHA for 175583b

• install dev before normal requirements to avoid typing-extensions conflict

  bertsky

  committed
  c00969f

  Copy full SHA for c00969f

• require calamari-ocr 2.3.1 (s11n of results via queue)

  bertsky

  committed
  298cf9a

  Copy full SHA for 298cf9a

Commits on Nov 13, 2024

• coverage: try to include subprocess results

  bertsky

  committed
  d842416

  Copy full SHA for d842416

• CI: store coverage results

  bertsky

  committed
  e9c12c8

  Copy full SHA for e9c12c8

• CI: fix indentation

  bertsky

  committed
  9a749b0

  Copy full SHA for 9a749b0