Limit label OCR to 8 threads that sleep when idle
All checks were successful
Verify and publish Calio container / build-and-push (push) Successful in 27s

ONNX Runtime's default thread count spread each step across the
i7-14700K's efficiency cores, so every step waited on the slowest one:
a label took ~7 s in the production container. Eight threads with
spinning disabled took 1.9 s there and stay at ~1.2 s on the dev CPU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Elijah 2026-10-22 14:17:31 -07:00
parent 80fc48b9f1
commit 40d9bc062b

View file

@ -2,6 +2,7 @@
// PaddleOCR's own pre- and post-processing: DB detection, min-area boxes with
// unclip, then CTC decoding of each line crop.
import { readFileSync } from "node:fs";
import { availableParallelism } from "node:os";
import { join } from "node:path";
import ort from "onnxruntime-node";
import sharp from "sharp";
@ -257,6 +258,12 @@ export async function createOcr(modelDir, { detSide = 1600 } = {}) {
graphOptimizationLevel: "all",
// The arena would keep ~2 GB of detection buffers after the first scan.
enableCpuMemArena: false,
// ONNX Runtime splits each step evenly across its threads, so on hybrid
// CPUs every step waits for the slowest efficiency core: an i7-14700K
// took ~7 s per label with its default 20 threads and 1.9 s with 8.
// Idle threads sleep instead of spinning, which was also faster there.
intraOpNumThreads: Math.min(8, availableParallelism()),
extra: { session: { intra_op: { allow_spinning: "0" } } },
};
const [det, rec] = await Promise.all([
ort.InferenceSession.create(