Add barcode and nutrition label scanning
All checks were successful
Verify and publish Calio container / build-and-push (push) Successful in 1m2s
All checks were successful
Verify and publish Calio container / build-and-push (push) Successful in 1m2s
Barcode scanning - Scan barcode in the food sheet reads UPC-A/E and EAN-13/8 live from the camera with ZXing WebAssembly (bundled, precached, CSP allows wasm), with photo and typed-digit fallbacks - Lookups check saved foods first (offline), then local copies of USDA FoodData Central Branded Foods and Open Food Facts, then the live Open Food Facts API; API answers and misses are cached - Server imports refresh in a low-priority child process: USDA when a new archive is published, Open Food Facts weekly by streaming its CSV export; each import builds a new table and swaps it in atomically - Every available nutrient is kept: foods and entries gain optional barcode (GTIN-14) and extra nutrients, shown under More nutrients and included in JSON exports - Settings shows the size, date and any error for each product copy Label scanning - Scan label sends a downscaled photo to the server, read with PaddleOCR PP-OCRv5 (server detection, English recognition) on ONNX Runtime CPU, about 1.3 s median - Parser handles US/Canadian, linear, Spanish and EU per-100 g panels, tilted photos, OCR misspellings and %DV cross-checks - Models are pinned and checksum-verified at image build time Container - Runtime moves to Debian slim for ONNX Runtime's glibc build; other platforms' binaries are pruned - Smoke tests now read a fixture label inside the container - README and Unraid docs cover disk use, network access and refresh Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
8f6981c8a2
commit
80fc48b9f1
39 changed files with 4434 additions and 33 deletions
|
|
@ -5,3 +5,4 @@ data
|
|||
.env
|
||||
.local
|
||||
test-results
|
||||
models
|
||||
|
|
|
|||
|
|
@ -10,6 +10,8 @@ jobs:
|
|||
runs-on: ubuntu-latest
|
||||
env:
|
||||
JOB_IMAGE: calio-ci:${{ github.run_id }}
|
||||
# CPU-only ONNX Runtime; skip its optional CUDA download.
|
||||
ONNXRUNTIME_NODE_INSTALL: skip
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
|
|
|||
1
.gitignore
vendored
1
.gitignore
vendored
|
|
@ -7,3 +7,4 @@ coverage/
|
|||
test-results/
|
||||
playwright-report/
|
||||
.local/
|
||||
models
|
||||
|
|
|
|||
23
Dockerfile
23
Dockerfile
|
|
@ -1,4 +1,6 @@
|
|||
FROM node:24-alpine AS build
|
||||
# Debian rather than Alpine: ONNX Runtime (label OCR) ships glibc binaries.
|
||||
FROM node:24-bookworm-slim AS build
|
||||
ENV ONNXRUNTIME_NODE_INSTALL=skip
|
||||
WORKDIR /app
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm ci --no-audit --no-fund
|
||||
|
|
@ -9,14 +11,27 @@ COPY public ./public
|
|||
COPY scripts/build-sw.js ./scripts/build-sw.js
|
||||
RUN npm run build
|
||||
|
||||
FROM node:24-alpine AS runtime
|
||||
# Pinned PP-OCRv5 models, verified by SHA-256.
|
||||
FROM node:24-bookworm-slim AS models
|
||||
WORKDIR /app
|
||||
COPY scripts/fetch-models.js ./scripts/fetch-models.js
|
||||
RUN node scripts/fetch-models.js /models
|
||||
|
||||
FROM node:24-bookworm-slim AS runtime
|
||||
LABEL org.opencontainers.image.title="Calio" \
|
||||
org.opencontainers.image.description="Personal offline-first nutrition and weight journal" \
|
||||
org.opencontainers.image.source="https://git.elijahkuntz.com/Elijah/Calio"
|
||||
ENV NODE_ENV=production PORT=3000 DATA_DIR=/data
|
||||
ENV NODE_ENV=production PORT=3000 DATA_DIR=/data MODELS_DIR=/app/models \
|
||||
ONNXRUNTIME_NODE_INSTALL=skip
|
||||
WORKDIR /app
|
||||
COPY package.json package-lock.json ./
|
||||
RUN npm ci --omit=dev --no-audit --no-fund && npm cache clean --force && mkdir /data && chown node:node /data
|
||||
# Keep only this platform's ONNX Runtime library (the package bundles all).
|
||||
RUN npm ci --omit=dev --no-audit --no-fund && npm cache clean --force \
|
||||
&& arch="$(node -p process.arch)" \
|
||||
&& find node_modules/onnxruntime-node/bin -mindepth 2 -maxdepth 2 ! -path "*/linux" -exec rm -rf {} + \
|
||||
&& find node_modules/onnxruntime-node/bin -mindepth 3 -maxdepth 3 ! -name "$arch" -exec rm -rf {} + \
|
||||
&& mkdir /data && chown node:node /data
|
||||
COPY --from=models /models ./models
|
||||
COPY --from=build /app/dist ./dist
|
||||
COPY server ./server
|
||||
COPY shared ./shared
|
||||
|
|
|
|||
34
README.md
34
README.md
|
|
@ -1,6 +1,6 @@
|
|||
# Calio
|
||||
|
||||
A single-user, self-hosted nutrition journal. PWA first, with a responsive desktop layout, light/dark/system themes, configurable accent colors, and no third-party analytics, fonts, or nutrition services. Dark mode uses neutral near-black and charcoal surfaces; choose a preset or custom accent under Settings → Look & feel; it applies immediately. Accent preferences sync between devices and remain available offline.
|
||||
A single-user, self-hosted nutrition journal. PWA first, with a responsive desktop layout, light/dark/system themes, configurable accent colors, and no third-party analytics or fonts. Barcode lookups use local copies of USDA FoodData Central and Open Food Facts kept on your server, with Open Food Facts' public API only as a fallback; nutrition labels are read on your server, never by an outside service. Dark mode uses neutral near-black and charcoal surfaces; choose a preset or custom accent under Settings → Look & feel; it applies immediately. Accent preferences sync between devices and remain available offline.
|
||||
|
||||
## Included
|
||||
|
||||
|
|
@ -24,7 +24,33 @@ Phone controls use larger type, 44–56 px tap targets, and 51×31 px switches.
|
|||
|
||||
The iPhone Home Screen icon uses a dark background and high-contrast mark so automatic tinting has a dark base instead of a bright colored square. The versioned Apple touch-icon URL lets new installations request the revised asset; Android keeps its existing regular and maskable icons. iOS owns the final tinted/clear rendering, so a PWA cannot guarantee the native Icon Composer appearance. If an existing Home Screen shortcut retains its old cached icon, first sync all pending entries, then add Calio to the Home Screen again. The server's database is unchanged.
|
||||
|
||||
Barcode lookup and nutrition-label OCR are deferred. This release uses manual nutrition entry and saved foods. It does not include Apple Watch integration, HealthKit, fasting, reminders or water tracking; those screenshot features were not part of the requested core.
|
||||
## Barcode and label scanning
|
||||
|
||||
The food sheet's **Quick entry** tab starts with **Scan barcode** and **Scan label**.
|
||||
|
||||
**Scan barcode** opens the rear camera and reads UPC-A, UPC-E, EAN-13 and EAN-8 codes continuously in the browser (ZXing compiled to WebAssembly, bundled with the app and cached for offline use; Safari has no built-in barcode reader). Line the code up inside the frame: it is read as soon as it is sharp, with no shutter. If the camera is unavailable or permission is off, **Take a photo instead** or type the digits. A scan is looked up in this order:
|
||||
|
||||
1. Your saved foods, on the device; this also works offline.
|
||||
2. The server's copy of **USDA FoodData Central Branded Foods** (~435,000 US products from manufacturers' label submissions). Its values win when both copies have a product.
|
||||
3. The server's copy of **Open Food Facts** (~3.4 million products worldwide, community-entered).
|
||||
4. The live Open Food Facts API, for products newer than the copies. Answers, including "not found", are cached on the server.
|
||||
|
||||
A find fills the form with the product's name, serving and every nutrient the source has: the seven tracked ones plus extras such as saturated fat, cholesterol, added sugars, vitamins and minerals, under **More nutrients**. **Save this food to my library** is pre-checked so the next scan is instant. When nothing is found, scan the label instead; the barcode stays attached to the food.
|
||||
|
||||
**Scan label** opens the iPhone camera app (full resolution, focus and flash, no permission prompt), or a file picker on desktop. The photo is downscaled on the device and read on the server with PaddleOCR PP-OCRv5 (server detection, English recognition) on ONNX Runtime, CPU only, typically in 1–3 seconds. A parser then pulls out the serving size and each nutrient, undoes common OCR confusions (O/0, l/1, a unit read as a digit), checks amounts against their printed %DV, and handles US/Canadian panels, linear labels, tilted photos, Spanish labels and EU "per 100 g" tables (salt → sodium). Values it found replace the form's values; check them against the package before saving. Fill the frame with the panel and keep it flat and sharp: curved containers and glare are the usual causes of misses.
|
||||
|
||||
Extra nutrients are stored per serving with each food and entry, are scaled by servings, sync like everything else, and appear in JSON exports as `otherNutrients`.
|
||||
|
||||
### Product data refresh
|
||||
|
||||
The server checks for new data a minute after start-up and every six hours, in a low-priority child process so the app stays responsive:
|
||||
|
||||
- **USDA**: when the download page lists a newer Branded Foods archive (published about twice a year), it downloads the ~450 MB zip to `/data/tmp`, imports it in about a minute, and deletes it.
|
||||
- **Open Food Facts**: weekly, it streams the ~1.5 GB daily CSV export straight into the database (about 5 minutes; nothing is written to disk first).
|
||||
|
||||
Each import builds a new table beside the old one and swaps it in at the end, so lookups keep working throughout and a failed download changes nothing. The data lives in `/data/products.sqlite` (~1.6 GB; up to about twice that while a refresh is being built). It is a cache: deleting it is safe, and it is rebuilt by the next check. It does not need backing up. **Settings → Barcode database** shows each copy's size, date and any download error. Until the first import finishes (roughly 10 minutes after the first start), scans fall back to the live API.
|
||||
|
||||
This release does not include Apple Watch integration, HealthKit, fasting, reminders or water tracking; those screenshot features were not part of the requested core.
|
||||
|
||||
## Deploy on Unraid
|
||||
|
||||
|
|
@ -50,7 +76,7 @@ Sessions expire after 30 days and persist in SQLite. Changing credentials does n
|
|||
|
||||
## Persistent data, updates and backups
|
||||
|
||||
SQLite lives at `/data/calio.sqlite`, mapped to `/mnt/user/appdata/calio` on Unraid. Keep this directory and mapping when replacing the image. The optional Compose setup uses a named `calio-data` volume instead. Schema migrations are versioned and transactional; an older app refuses to open a newer schema.
|
||||
SQLite lives at `/data/calio.sqlite` (with the regenerable product cache in `/data/products.sqlite`), mapped to `/mnt/user/appdata/calio` on Unraid. Keep this directory and mapping when replacing the image. The optional Compose setup uses a named `calio-data` volume instead. Schema migrations are versioned and transactional; an older app refuses to open a newer schema.
|
||||
|
||||
Before updating, make a consistent backup while Calio is running:
|
||||
|
||||
|
|
@ -101,7 +127,7 @@ The repository configuration is ready, but a successful live publish still requi
|
|||
|
||||
## Develop and verify
|
||||
|
||||
Node 24+ is required for built-in SQLite. `npm ci` installs the lockfile. `npm test` covers authentication, validation, deterministic merges, tombstones, persistence, and client outbox handling.
|
||||
Node 24+ is required for built-in SQLite. `npm ci` installs the lockfile. `npm test` covers authentication, validation, deterministic merges, tombstones, persistence, client outbox handling, barcode normalization, the product importers and lookup order, and the label parser. `node scripts/fetch-models.js` downloads the pinned, checksummed OCR models into `models/` for local runs (the Docker build does this itself).
|
||||
|
||||
For a production preview, run `npm run build`, configure the same environment variables above, then `npm start`. For live UI editing, run `npm run dev:server` on port 3000 and `npm run dev` on port 5173, with `APP_ORIGIN=http://localhost:5173` and `COOKIE_SECURE=false`. Vite proxies `/api`; the service worker is registered only in production builds.
|
||||
|
||||
|
|
|
|||
|
|
@ -54,7 +54,9 @@ On iPhone, use Safari → Share → **Add to Home Screen**. On Android, use Chro
|
|||
|
||||
## Update without losing data
|
||||
|
||||
Keep `/mnt/user/appdata/calio` mapped to `/data` on every recreation. It holds the SQLite database, persistent sessions and backups; the image holds only application code.
|
||||
Keep `/mnt/user/appdata/calio` mapped to `/data` on every recreation. It holds the SQLite database, persistent sessions and backups; the image holds only application code and the OCR models.
|
||||
|
||||
The same directory also holds `products.sqlite`, the barcode database (about 1.6 GB, briefly up to about 3.5 GB while a refresh is built, plus a 450 MB USDA download in `tmp/` about twice a year). Keep appdata on a share with room for that. The file is a cache and needs no backup. The container needs outbound HTTPS to `fdc.nal.usda.gov`, `static.openfoodfacts.org` (redirects to Amazon S3) and `world.openfoodfacts.org`.
|
||||
|
||||
Before updating, create a consistent backup from the Unraid terminal:
|
||||
|
||||
|
|
|
|||
851
package-lock.json
generated
851
package-lock.json
generated
|
|
@ -8,15 +8,20 @@
|
|||
"name": "calio",
|
||||
"version": "0.1.0",
|
||||
"dependencies": {
|
||||
"barcode-detector": "^3.2.3",
|
||||
"express": "^5.1.0",
|
||||
"express-rate-limit": "^8.1.0",
|
||||
"helmet": "^8.1.0",
|
||||
"idb": "^8.0.3",
|
||||
"lucide-react": "^0.468.0",
|
||||
"matter-js": "^0.20.0",
|
||||
"onnxruntime-node": "^1.30.0",
|
||||
"react": "^19.2.0",
|
||||
"react-dom": "^19.2.0",
|
||||
"zod": "^4.1.12"
|
||||
"sharp": "^0.35.5",
|
||||
"yauzl": "^3.4.0",
|
||||
"zod": "^4.1.12",
|
||||
"zxing-wasm": "^3.1.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/matter-js": "^0.20.2",
|
||||
|
|
@ -315,6 +320,16 @@
|
|||
"node": ">=6.9.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@emnapi/runtime": {
|
||||
"version": "1.11.3",
|
||||
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.3.tgz",
|
||||
"integrity": "sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==",
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@esbuild/aix-ppc64": {
|
||||
"version": "0.28.2",
|
||||
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz",
|
||||
|
|
@ -757,6 +772,506 @@
|
|||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/colour": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@img/colour/-/colour-1.1.0.tgz",
|
||||
"integrity": "sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-darwin-arm64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-arm64/-/sharp-darwin-arm64-0.35.5.tgz",
|
||||
"integrity": "sha512-QRUlFQ0WxvdWyqqG/WtI3iupfD5rBzmCHXSdPsY91sAtVtTo7Q4cb6zOccZ3gqEqkr0f1As1ehLqmEpDsRf+lg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-darwin-arm64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-darwin-x64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-darwin-x64/-/sharp-darwin-x64-0.35.5.tgz",
|
||||
"integrity": "sha512-+BR255RhDlpygUpOc/Jdt1nT6DQ3XG/ERo5wbcdOf5Q320dKtPCKPLR1LJs9VGXRaMa8l1uUa0tkCNOXiAxZUw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-darwin-x64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-freebsd-wasm32": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-freebsd-wasm32/-/sharp-freebsd-wasm32-0.35.5.tgz",
|
||||
"integrity": "sha512-Y/z91nEZ4uIBX5X3nfTovjU9lHNKFYbL2lpHCLVNmXQK03VIZvXBBt0KxbPGp2SdGSF+2mQU4e+hQaWOt86iAw==",
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"freebsd"
|
||||
],
|
||||
"dependencies": {
|
||||
"@img/sharp-wasm32": "0.35.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-darwin-arm64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-arm64/-/sharp-libvips-darwin-arm64-1.3.4.tgz",
|
||||
"integrity": "sha512-5R89nBYiRdUlSWJxPhO+GVtaXzXSxKnRu/xqMn3KTA3L9EB9Oy/P+Nn2f2vlhPuUdy/Zusb2DarbyTpGCfEDuw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-darwin-x64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-darwin-x64/-/sharp-libvips-darwin-x64-1.3.4.tgz",
|
||||
"integrity": "sha512-iR2OKH80yi0U+dUplyh3/xdpFvps6YkCwsXenIJxqxR1v9o+xtKTGbS9H7cps+2Vxjc8B1j96p75NmTGjIhtpQ==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-arm": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm/-/sharp-libvips-linux-arm-1.3.4.tgz",
|
||||
"integrity": "sha512-LmRtTsOHuvM2+wlO2Db37dx5MiZhB0FvSunciw48YjdOkZz9KAiRbm8ujeMOA1INqmei5NapFxYEK1D1ZSidmw==",
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-arm64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-arm64/-/sharp-libvips-linux-arm64-1.3.4.tgz",
|
||||
"integrity": "sha512-Y3dgX/6lE2QhQb+Gxy0WZxfg9MEm/JBjamZpS2IklP7xIQoKN4hzAm7KcMVGtaVDt3neE9OKBC7vAfonA/Lr1A==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-ppc64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-ppc64/-/sharp-libvips-linux-ppc64-1.3.4.tgz",
|
||||
"integrity": "sha512-Le6boB8Tai0Nis+gIxIpKx68UDVVIqdR8Tin5Yf1z2LJJQLDJvCDRqRu+jC2qCoD+eIomonmOwB4smBRxfVpYQ==",
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-riscv64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-riscv64/-/sharp-libvips-linux-riscv64-1.3.4.tgz",
|
||||
"integrity": "sha512-aHkkIEHPRdQEegJN20MLmGtxYD9R2wQr3Cwpddnu5+YKMt6Uzax7S9h5gpZTo8wyrGuZSlfQ63OevL5mTyOC7Q==",
|
||||
"cpu": [
|
||||
"riscv64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-s390x": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-s390x/-/sharp-libvips-linux-s390x-1.3.4.tgz",
|
||||
"integrity": "sha512-ra/mB6MikESDUO7Yg+Mi95bFBb9GsObURuhnOv3OqknjGe9sZrG8tCe9q0xSIGrtLgvgw0gKnFWcK4blSgQOuQ==",
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linux-x64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linux-x64/-/sharp-libvips-linux-x64-1.3.4.tgz",
|
||||
"integrity": "sha512-GJ//SSXbnwSDes02umB3nDJLFcQzw8a18V8fyhqr6tV515tOEMdImjjxj1AoafMRz56F3PHgftnj1QEKSU1zkw==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linuxmusl-arm64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-arm64/-/sharp-libvips-linuxmusl-arm64-1.3.4.tgz",
|
||||
"integrity": "sha512-hvulFwtjUcagsis6BBxHwGFwWoNZjgYmULGVrZcyfNbjA8hKILbRxGg15/7w5HDyXHXUos/j6baAWqnCyQ2DWA==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-libvips-linuxmusl-x64": {
|
||||
"version": "1.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-libvips-linuxmusl-x64/-/sharp-libvips-linuxmusl-x64-1.3.4.tgz",
|
||||
"integrity": "sha512-6zXKeE/p39I1AmA3cJG35eyBGNqNddLnUXjhwBnsGjFPWqf5VKkDBEqaEkPDoTEtkxwi2vv8Tcr2mDyP4So7Fg==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-arm": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm/-/sharp-linux-arm-0.35.5.tgz",
|
||||
"integrity": "sha512-LEaXK2WdXVK5ykcw0buWyPMsmLLL2vpHLD6yrNSW+JGEL3BZPA4tpKN6iaMc4AxTTAoaX/sU1rOL51lcIz48ZQ==",
|
||||
"cpu": [
|
||||
"arm"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-arm": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-arm64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-arm64/-/sharp-linux-arm64-0.35.5.tgz",
|
||||
"integrity": "sha512-LYVx5JTsOM2CBzmxreh+nl64/3H6Xb09iSLknqH47z2T2DFFxDeFLP5y4dJwe6H7uGQlHPyEEtIqyo3DYsRwdQ==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-arm64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-ppc64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-ppc64/-/sharp-linux-ppc64-0.35.5.tgz",
|
||||
"integrity": "sha512-QVxAAq8evVRI9ia2vqgwrmWucn5Dfv+JdWzj75pD8omHLPSP7f8p20O8jxzjCcuCEQEOtYOZUmX1hkiZ0kdevA==",
|
||||
"cpu": [
|
||||
"ppc64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-ppc64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-riscv64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-riscv64/-/sharp-linux-riscv64-0.35.5.tgz",
|
||||
"integrity": "sha512-LtdreXguaavKODPIfzJ4kffx7UNt1omwtK0rch4EBbbSTXPnxWmYSayXdLJw0fJzQ97kHt1gL/yh4tvU+nCyRQ==",
|
||||
"cpu": [
|
||||
"riscv64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-riscv64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-s390x": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-s390x/-/sharp-linux-s390x-0.35.5.tgz",
|
||||
"integrity": "sha512-UZasTOFiYzotTsGOCu42BfUzP6Tu6Do/947iRm1RsLKvlllxwGcn4RN27LibGWceix4Y+Pmw3jsnTcCQIgWjqA==",
|
||||
"cpu": [
|
||||
"s390x"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-s390x": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linux-x64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linux-x64/-/sharp-linux-x64-0.35.5.tgz",
|
||||
"integrity": "sha512-SxFtLTeJInhAA9Q836kux2vZNeOBQEx658qvbboZScr0wIARym3IcGmW7KpVD5sbVg0Ojy+udFQdayYIZyoNog==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linux-x64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linuxmusl-arm64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-arm64/-/sharp-linuxmusl-arm64-0.35.5.tgz",
|
||||
"integrity": "sha512-9HbMclmI1zlNkFRs3z9/eBtDjfD0sGlrX1z6b1qwmiFY5ElDLh4BC0LPBdVp7z1DXFiKlIcznf+ZlsuZzLxQqg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linuxmusl-arm64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-linuxmusl-x64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-linuxmusl-x64/-/sharp-linuxmusl-x64-0.35.5.tgz",
|
||||
"integrity": "sha512-4KOphqB035HrVdqLZfCgMzzERrQkkzOwRhl4OAkRO1YCldbaFjySXMaK534Mo0V+LndnlJk+sbUyLeU0ULyD1A==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-libvips-linuxmusl-x64": "1.3.4"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-wasm32": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-wasm32/-/sharp-wasm32-0.35.5.tgz",
|
||||
"integrity": "sha512-Ptsga1su4tQx+LLF1ECS9U6nz5kmrXKo6XVbtR48Ke3ZRxxgaWBu7IDtEe1quo8hiupwm6WFqxVlXaSf7IINGQ==",
|
||||
"license": "Apache-2.0 AND LGPL-3.0-or-later AND MIT",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"@emnapi/runtime": "^1.11.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-webcontainers-wasm32": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-webcontainers-wasm32/-/sharp-webcontainers-wasm32-0.35.5.tgz",
|
||||
"integrity": "sha512-hfhF/FmoQyTUkA0bIKFOtw536BQSeBMe6BF6QyWlrPxT754+TFLaZ7sKKTfvvM0yJgKgaYTwnFCIZ/GuDw5SUA==",
|
||||
"cpu": [
|
||||
"wasm32"
|
||||
],
|
||||
"license": "Apache-2.0",
|
||||
"optional": true,
|
||||
"dependencies": {
|
||||
"@img/sharp-wasm32": "0.35.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-win32-arm64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-win32-arm64/-/sharp-win32-arm64-0.35.5.tgz",
|
||||
"integrity": "sha512-X4t7g+7ZA5DKblCBEXGjUqqemj4vczING/5viFwAL8h4N3qYeyjwdCvRLHi4EdOUI+2Z7UFlp1VM+p/AuEtm6Q==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
],
|
||||
"license": "Apache-2.0 AND LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"win32"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-win32-ia32": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-win32-ia32/-/sharp-win32-ia32-0.35.5.tgz",
|
||||
"integrity": "sha512-5Zm82LoBc43nhwNybZlG7Y1KO//Zhsn306fQl29ZOuStHLGTo3BWL83q3cznX0poxSAMuYL1On/BHBxkBeKr6A==",
|
||||
"cpu": [
|
||||
"ia32"
|
||||
],
|
||||
"license": "Apache-2.0 AND LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"win32"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@img/sharp-win32-x64": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/@img/sharp-win32-x64/-/sharp-win32-x64-0.35.5.tgz",
|
||||
"integrity": "sha512-x76eH0vEiHlcMQu8Y8IenntaACtddpT6W0wmXtWrnKcnKI7ME5DdgqhAD6SEWOEl1v2zDvkZDhFA9KnURwpfqg==",
|
||||
"cpu": [
|
||||
"x64"
|
||||
],
|
||||
"license": "Apache-2.0 AND LGPL-3.0-or-later",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"win32"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
}
|
||||
},
|
||||
"node_modules/@jridgewell/gen-mapping": {
|
||||
"version": "0.3.13",
|
||||
"resolved": "https://registry.npmjs.org/@jridgewell/gen-mapping/-/gen-mapping-0.3.13.tgz",
|
||||
|
|
@ -1226,6 +1741,12 @@
|
|||
"@babel/types": "^7.28.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/emscripten": {
|
||||
"version": "1.41.6",
|
||||
"resolved": "https://registry.npmjs.org/@types/emscripten/-/emscripten-1.41.6.tgz",
|
||||
"integrity": "sha512-uN+9i8bFT5CUcZfyIEYDrSueACEyKGbUs5kC/72DGlZZoinh84sJfVV0i8UOJD1asdzkvLPBRrKs41kZ8MdEXg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/estree": {
|
||||
"version": "1.0.9",
|
||||
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
|
||||
|
|
@ -1306,6 +1827,24 @@
|
|||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/adm-zip": {
|
||||
"version": "0.6.1",
|
||||
"resolved": "https://registry.npmjs.org/adm-zip/-/adm-zip-0.6.1.tgz",
|
||||
"integrity": "sha512-Xwrja8nx9e5o2N1my4DsKCeKpdrnACyr1wtbPxBDgGzKzKyE9kRtBFA8mWldI+RVlD7CBZNWY/wQ2+ydwOR6kQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=14.0"
|
||||
}
|
||||
},
|
||||
"node_modules/barcode-detector": {
|
||||
"version": "3.2.3",
|
||||
"resolved": "https://registry.npmjs.org/barcode-detector/-/barcode-detector-3.2.3.tgz",
|
||||
"integrity": "sha512-PLinpWGiOQoQH4X+YObWZ6iXeffXKDM3d4vnOD86481SvukIfvkf7jEE/tud9oI4HTDxtDdlHaFilA5Kn7Xa7A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"zxing-wasm": "3.1.6"
|
||||
}
|
||||
},
|
||||
"node_modules/baseline-browser-mapping": {
|
||||
"version": "2.11.30",
|
||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.30.tgz",
|
||||
|
|
@ -1512,6 +2051,40 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"node_modules/define-data-property": {
|
||||
"version": "1.1.4",
|
||||
"resolved": "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz",
|
||||
"integrity": "sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-define-property": "^1.0.0",
|
||||
"es-errors": "^1.3.0",
|
||||
"gopd": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/define-properties": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/define-properties/-/define-properties-1.2.1.tgz",
|
||||
"integrity": "sha512-8QmQKqEASLd5nx0U1B1okLElbUuuttJ/AnYmRXbbbGDWh6uS208EjD4Xqq/I9wK7u0v6O08XhTWnt5XtEbR6Dg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"define-data-property": "^1.0.1",
|
||||
"has-property-descriptors": "^1.0.0",
|
||||
"object-keys": "^1.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/depd": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
|
||||
|
|
@ -1521,6 +2094,15 @@
|
|||
"node": ">= 0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/detect-libc": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
|
||||
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/dunder-proto": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
|
||||
|
|
@ -1645,6 +2227,18 @@
|
|||
"integrity": "sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/escape-string-regexp": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-4.0.0.tgz",
|
||||
"integrity": "sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/etag": {
|
||||
"version": "1.8.1",
|
||||
"resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz",
|
||||
|
|
@ -1855,6 +2449,49 @@
|
|||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/global-agent": {
|
||||
"version": "4.1.3",
|
||||
"resolved": "https://registry.npmjs.org/global-agent/-/global-agent-4.1.3.tgz",
|
||||
"integrity": "sha512-KUJEViiuFT3I97t+GYMikLPJS2Lfo/S2F+DQuBWzuzaMPnvt5yyZePzArx36fBzpGTxZjIpDbXLeySLgh+k76g==",
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"globalthis": "^1.0.2",
|
||||
"matcher": "^4.0.0",
|
||||
"semver": "^7.3.5",
|
||||
"serialize-error": "^8.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/global-agent/node_modules/semver": {
|
||||
"version": "7.8.5",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
|
||||
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/globalthis": {
|
||||
"version": "1.0.4",
|
||||
"resolved": "https://registry.npmjs.org/globalthis/-/globalthis-1.0.4.tgz",
|
||||
"integrity": "sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"define-properties": "^1.2.1",
|
||||
"gopd": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/gopd": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
|
||||
|
|
@ -1867,6 +2504,18 @@
|
|||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/has-property-descriptors": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz",
|
||||
"integrity": "sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-define-property": "^1.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/has-symbols": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
|
||||
|
|
@ -2027,6 +2676,21 @@
|
|||
"react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0-rc"
|
||||
}
|
||||
},
|
||||
"node_modules/matcher": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/matcher/-/matcher-4.0.0.tgz",
|
||||
"integrity": "sha512-S6x5wmcDmsDRRU/c2dkccDwQPXoFczc5+HpQ2lON8pnvHlnvHAHj5WlLVvw6n6vNyHuVugYrFohYxbS+pvFpKQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"escape-string-regexp": "^4.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/math-intrinsics": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
|
||||
|
|
@ -2155,6 +2819,15 @@
|
|||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/object-keys": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/object-keys/-/object-keys-1.1.1.tgz",
|
||||
"integrity": "sha512-NuAESUOUMrlIXOfHKzD6bpPu3tYt3xvjNdRIQ+FeT0lNb4K8WR70CaDxhuNguS2XG+GjkyMwOzsN5ZktImfhLA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/on-finished": {
|
||||
"version": "2.4.1",
|
||||
"resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz",
|
||||
|
|
@ -2176,6 +2849,29 @@
|
|||
"wrappy": "1"
|
||||
}
|
||||
},
|
||||
"node_modules/onnxruntime-common": {
|
||||
"version": "1.30.0",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-common/-/onnxruntime-common-1.30.0.tgz",
|
||||
"integrity": "sha512-7fdVWjAID1dVhH/G8qK3APARunV4VkBFoCQAP7qp4Wkab0mrorvmc+sqiT+mKXOzDqdjN5j+/Z9nb4gzNPWcyA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/onnxruntime-node": {
|
||||
"version": "1.30.0",
|
||||
"resolved": "https://registry.npmjs.org/onnxruntime-node/-/onnxruntime-node-1.30.0.tgz",
|
||||
"integrity": "sha512-twhs1C2C/BFkz1yc5OY0KIU2GUq6DURO7hD4bx5Q2Qy3nAMJwRXW8xU3NVczE29VA9lolLOYepoD8fjTGOfIqw==",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"os": [
|
||||
"win32",
|
||||
"darwin",
|
||||
"linux"
|
||||
],
|
||||
"dependencies": {
|
||||
"adm-zip": "^0.6.0",
|
||||
"global-agent": "^4.1.3",
|
||||
"onnxruntime-common": "1.30.0"
|
||||
}
|
||||
},
|
||||
"node_modules/parseurl": {
|
||||
"version": "1.3.3",
|
||||
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
|
||||
|
|
@ -2195,6 +2891,12 @@
|
|||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/pend": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz",
|
||||
"integrity": "sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/picocolors": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
|
||||
|
|
@ -2448,6 +3150,21 @@
|
|||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/serialize-error": {
|
||||
"version": "8.1.0",
|
||||
"resolved": "https://registry.npmjs.org/serialize-error/-/serialize-error-8.1.0.tgz",
|
||||
"integrity": "sha512-3NnuWfM6vBYoy5gZFvHiYsVbafvI9vZv/+jlIigFn4oP4zjNPK3LhcY0xSCgeb1a5L8jO71Mit9LlNoi2UfDDQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"type-fest": "^0.20.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/serve-static": {
|
||||
"version": "2.2.1",
|
||||
"resolved": "https://registry.npmjs.org/serve-static/-/serve-static-2.2.1.tgz",
|
||||
|
|
@ -2473,6 +3190,67 @@
|
|||
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/sharp": {
|
||||
"version": "0.35.5",
|
||||
"resolved": "https://registry.npmjs.org/sharp/-/sharp-0.35.5.tgz",
|
||||
"integrity": "sha512-Ywn4OnzGukp7CDMrp08RQ50YKmuwG47brZgIVPTvBaaAfQlRlygrRqSrxdCiL9M+LlzLBiJ68IR1QqvzHyjC7g==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@img/colour": "^1.1.0",
|
||||
"detect-libc": "^2.1.2",
|
||||
"semver": "^7.8.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.9.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/libvips"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@img/sharp-darwin-arm64": "0.35.5",
|
||||
"@img/sharp-darwin-x64": "0.35.5",
|
||||
"@img/sharp-freebsd-wasm32": "0.35.5",
|
||||
"@img/sharp-libvips-darwin-arm64": "1.3.4",
|
||||
"@img/sharp-libvips-darwin-x64": "1.3.4",
|
||||
"@img/sharp-libvips-linux-arm": "1.3.4",
|
||||
"@img/sharp-libvips-linux-arm64": "1.3.4",
|
||||
"@img/sharp-libvips-linux-ppc64": "1.3.4",
|
||||
"@img/sharp-libvips-linux-riscv64": "1.3.4",
|
||||
"@img/sharp-libvips-linux-s390x": "1.3.4",
|
||||
"@img/sharp-libvips-linux-x64": "1.3.4",
|
||||
"@img/sharp-libvips-linuxmusl-arm64": "1.3.4",
|
||||
"@img/sharp-libvips-linuxmusl-x64": "1.3.4",
|
||||
"@img/sharp-linux-arm": "0.35.5",
|
||||
"@img/sharp-linux-arm64": "0.35.5",
|
||||
"@img/sharp-linux-ppc64": "0.35.5",
|
||||
"@img/sharp-linux-riscv64": "0.35.5",
|
||||
"@img/sharp-linux-s390x": "0.35.5",
|
||||
"@img/sharp-linux-x64": "0.35.5",
|
||||
"@img/sharp-linuxmusl-arm64": "0.35.5",
|
||||
"@img/sharp-linuxmusl-x64": "0.35.5",
|
||||
"@img/sharp-webcontainers-wasm32": "0.35.5",
|
||||
"@img/sharp-win32-arm64": "0.35.5",
|
||||
"@img/sharp-win32-ia32": "0.35.5",
|
||||
"@img/sharp-win32-x64": "0.35.5"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@types/node": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/sharp/node_modules/semver": {
|
||||
"version": "7.8.5",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
|
||||
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/side-channel": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/side-channel/-/side-channel-1.1.1.tgz",
|
||||
|
|
@ -2564,6 +3342,18 @@
|
|||
"node": ">= 0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/tagged-tag": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/tagged-tag/-/tagged-tag-1.0.0.tgz",
|
||||
"integrity": "sha512-yEFYrVhod+hdNyx7g5Bnkkb0G6si8HJurOoOEgC8B/O0uXLHlaey/65KRv6cuWBNhBgHKAROVpc7QyYqE5gFng==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/tinyglobby": {
|
||||
"version": "0.2.17",
|
||||
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz",
|
||||
|
|
@ -2590,6 +3380,25 @@
|
|||
"node": ">=0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/tslib": {
|
||||
"version": "2.8.1",
|
||||
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
|
||||
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
|
||||
"license": "0BSD",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/type-fest": {
|
||||
"version": "0.20.2",
|
||||
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-0.20.2.tgz",
|
||||
"integrity": "sha512-Ne+eE4r0/iWnpAxD852z3A+N0Bt5RN//NjJwRd2VFHEmrywxf5vsZlh4R6lixl6B+wz/8d+maTSAkN1FIkI3LQ==",
|
||||
"license": "(MIT OR CC0-1.0)",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/type-is": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
|
||||
|
|
@ -2767,6 +3576,18 @@
|
|||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/yauzl": {
|
||||
"version": "3.4.0",
|
||||
"resolved": "https://registry.npmjs.org/yauzl/-/yauzl-3.4.0.tgz",
|
||||
"integrity": "sha512-jIH9yLR9wqr0wOS0TpBvo/g/2UgZH5qePVbjgRliiF0BYvOZyaBknKsF+x9Iht0O6sqgnB93rCICdOZFecJuDw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"pend": "~1.2.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/zod": {
|
||||
"version": "4.6.7",
|
||||
"resolved": "https://registry.npmjs.org/zod/-/zod-4.6.7.tgz",
|
||||
|
|
@ -2775,6 +3596,34 @@
|
|||
"funding": {
|
||||
"url": "https://github.com/sponsors/colinhacks"
|
||||
}
|
||||
},
|
||||
"node_modules/zxing-wasm": {
|
||||
"version": "3.1.6",
|
||||
"resolved": "https://registry.npmjs.org/zxing-wasm/-/zxing-wasm-3.1.6.tgz",
|
||||
"integrity": "sha512-mU7UNuvl+hOT+nlSOQTMHZ2tM8lRMsbjDIp5IO/+HmBl/FOq77q0OSZPJ3cm42A9c1MKU6Uve67FiGdkhsrT9g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/emscripten": "^1.41.6",
|
||||
"type-fest": "^5.10.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@types/emscripten": ">=1.39.6"
|
||||
}
|
||||
},
|
||||
"node_modules/zxing-wasm/node_modules/type-fest": {
|
||||
"version": "5.10.0",
|
||||
"resolved": "https://registry.npmjs.org/type-fest/-/type-fest-5.10.0.tgz",
|
||||
"integrity": "sha512-NoSdpq/WEiAg5sjmBkmV/hfxv6HJH4NqPNrqjtSO5CwRmpsDfaf4begxW34KdJykH/l1yHtwBWQkCRdoXO8mPA==",
|
||||
"license": "(MIT OR CC0-1.0)",
|
||||
"dependencies": {
|
||||
"tagged-tag": "^1.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -15,15 +15,20 @@
|
|||
"password": "node scripts/password.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"barcode-detector": "^3.2.3",
|
||||
"express": "^5.1.0",
|
||||
"express-rate-limit": "^8.1.0",
|
||||
"helmet": "^8.1.0",
|
||||
"idb": "^8.0.3",
|
||||
"lucide-react": "^0.468.0",
|
||||
"matter-js": "^0.20.0",
|
||||
"onnxruntime-node": "^1.30.0",
|
||||
"react": "^19.2.0",
|
||||
"react-dom": "^19.2.0",
|
||||
"zod": "^4.1.12"
|
||||
"sharp": "^0.35.5",
|
||||
"yauzl": "^3.4.0",
|
||||
"zod": "^4.1.12",
|
||||
"zxing-wasm": "^3.1.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/matter-js": "^0.20.2",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import { execFileSync, spawnSync } from "node:child_process";
|
||||
import assert from "node:assert/strict";
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { emptyNutrients } from "../shared/domain.js";
|
||||
const image = process.argv[2] || "calio:local";
|
||||
const unraid = process.argv.includes("--unraid");
|
||||
|
|
@ -66,19 +67,20 @@ async function start() {
|
|||
);
|
||||
}
|
||||
// In-container probes work on socket-mounted Forgejo runners without published ports.
|
||||
const request = async (path, body, cookie) => {
|
||||
// `image` (base64 PNG) is posted as the raw request body.
|
||||
const request = async (path, body, cookie, image) => {
|
||||
const probe = `let input=''; for await (const chunk of process.stdin) input+=chunk;
|
||||
const {path,body,cookie}=JSON.parse(input);
|
||||
const {path,body,cookie,image}=JSON.parse(input);
|
||||
const response=await fetch('http://127.0.0.1:3000'+path,{
|
||||
method:body===undefined?'GET':'POST',
|
||||
headers:{'Content-Type':'application/json',Origin:'http://localhost:3000','X-Calio-Request':'1',Cookie:cookie||''},
|
||||
body:body===undefined?undefined:JSON.stringify(body)
|
||||
method:body===undefined&&!image?'GET':'POST',
|
||||
headers:{'Content-Type':image?'image/png':'application/json',Origin:'http://localhost:3000','X-Calio-Request':'1',Cookie:cookie||''},
|
||||
body:image?Buffer.from(image,'base64'):body===undefined?undefined:JSON.stringify(body)
|
||||
});
|
||||
console.log(JSON.stringify({status:response.status,cookie:response.headers.get('set-cookie'),body:await response.text()}));`;
|
||||
const result = JSON.parse(
|
||||
docker(
|
||||
["exec", "-i", name, "node", "--input-type=module", "-e", probe],
|
||||
JSON.stringify({ path, body, cookie }),
|
||||
JSON.stringify({ path, body, cookie, image }),
|
||||
),
|
||||
);
|
||||
return {
|
||||
|
|
@ -135,6 +137,28 @@ try {
|
|||
"/icons/apple-touch-icon-v2.png",
|
||||
])
|
||||
assert.equal((await request(path)).status, 200, path);
|
||||
// The bundled OCR models load and read a label inside the image.
|
||||
const label = await request(
|
||||
"/api/label",
|
||||
undefined,
|
||||
cookie,
|
||||
readFileSync(
|
||||
new URL("../tests/fixtures/label.png", import.meta.url),
|
||||
).toString("base64"),
|
||||
);
|
||||
assert.equal(label.status, 200, await label.text());
|
||||
const read = (await label.json()).label;
|
||||
assert.deepEqual(read.nutrients, {
|
||||
calories: 230,
|
||||
fat: 8,
|
||||
sodium: 160,
|
||||
carbs: 37,
|
||||
fiber: 4,
|
||||
sugar: 12,
|
||||
protein: 3,
|
||||
});
|
||||
assert.equal(read.serving, "2/3 cup (55 g)");
|
||||
assert.equal((await request("/api/products", undefined, cookie)).status, 200);
|
||||
const html = await (await request("/")).text();
|
||||
const asset = html.match(/src="(\/assets\/[^\"]+)"/)[1];
|
||||
assert.equal((await request(asset)).status, 200);
|
||||
|
|
@ -147,7 +171,7 @@ try {
|
|||
assert.equal(response.status, 200);
|
||||
assert.deepEqual((await response.json()).records, [record]);
|
||||
console.log(
|
||||
`PASS (${unraid ? "Unraid 99:100" : "default 1000"}): non-root container, production assets, authentication, read-only root, persistent volume and sessions after container recreation.`,
|
||||
`PASS (${unraid ? "Unraid 99:100" : "default 1000"}): non-root container, production assets, authentication, label OCR, read-only root, persistent volume and sessions after container recreation.`,
|
||||
);
|
||||
} finally {
|
||||
try {
|
||||
|
|
|
|||
48
scripts/fetch-models.js
Normal file
48
scripts/fetch-models.js
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
// Downloads the pinned PaddleOCR (PP-OCRv5) ONNX models used for label scans.
|
||||
// node scripts/fetch-models.js [directory] (default: models)
|
||||
import { createHash } from "node:crypto";
|
||||
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
||||
import { join, resolve } from "node:path";
|
||||
|
||||
export const models = [
|
||||
{
|
||||
dir: "det",
|
||||
repo: "PaddlePaddle/PP-OCRv5_server_det_onnx",
|
||||
revision: "dcf248c4dcc064c6d030db241255f9e8fbc6733a",
|
||||
sha256: "10803475a591f7dc623e24670fb5752ec94d39a1f8cf069aac1b6f0ce19cfc85",
|
||||
},
|
||||
{
|
||||
dir: "rec",
|
||||
repo: "PaddlePaddle/en_PP-OCRv5_mobile_rec_onnx",
|
||||
revision: "3fafbc3b5dcf93dd72add9f48368be8a3a2cd33b",
|
||||
sha256: "b5f833dfc5d0eb71da397b4efa06ebeee9b431b690a47d6af40d77d8eabc557f",
|
||||
},
|
||||
];
|
||||
|
||||
async function get(repo, revision, file) {
|
||||
const url = `https://huggingface.co/${repo}/resolve/${revision}/${file}`;
|
||||
const response = await fetch(url);
|
||||
if (!response.ok) throw new Error(`${url}: ${response.status}`);
|
||||
return Buffer.from(await response.arrayBuffer());
|
||||
}
|
||||
|
||||
const root = resolve(process.argv[2] || "models");
|
||||
for (const model of models) {
|
||||
const dir = join(root, model.dir);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
const onnx = join(dir, "inference.onnx");
|
||||
const current =
|
||||
existsSync(onnx) &&
|
||||
createHash("sha256").update(readFileSync(onnx)).digest("hex");
|
||||
if (current === model.sha256) continue;
|
||||
const data = await get(model.repo, model.revision, "inference.onnx");
|
||||
const digest = createHash("sha256").update(data).digest("hex");
|
||||
if (digest !== model.sha256)
|
||||
throw new Error(`${model.repo} checksum mismatch: ${digest}`);
|
||||
writeFileSync(onnx, data);
|
||||
writeFileSync(
|
||||
join(dir, "inference.yml"),
|
||||
await get(model.repo, model.revision, "inference.yml"),
|
||||
);
|
||||
console.log(`Fetched ${model.repo}`);
|
||||
}
|
||||
|
|
@ -6,7 +6,11 @@ import { resolve, join } from "node:path";
|
|||
import { existsSync } from "node:fs";
|
||||
import { z } from "zod";
|
||||
import { recordSchema } from "../shared/domain.js";
|
||||
import { normalizeBarcode } from "../shared/barcode.js";
|
||||
import { productToFood } from "../shared/products.js";
|
||||
import { checkPassword, tokenHash } from "./auth.js";
|
||||
import { fetchOffProduct } from "./products/off.js";
|
||||
import { parseLabel } from "./ocr/label.js";
|
||||
|
||||
export function createApp({
|
||||
store,
|
||||
|
|
@ -15,6 +19,9 @@ export function createApp({
|
|||
origin,
|
||||
secureCookies = true,
|
||||
dist = resolve("dist"),
|
||||
products,
|
||||
fetchProduct = fetchOffProduct,
|
||||
ocr,
|
||||
}) {
|
||||
const app = express();
|
||||
app.disable("x-powered-by");
|
||||
|
|
@ -22,9 +29,10 @@ export function createApp({
|
|||
helmet({
|
||||
contentSecurityPolicy: {
|
||||
directives: {
|
||||
"script-src": ["'self'"],
|
||||
// The barcode reader is WebAssembly, served from this origin.
|
||||
"script-src": ["'self'", "'wasm-unsafe-eval'"],
|
||||
"style-src": ["'self'", "'unsafe-inline'"],
|
||||
"img-src": ["'self'", "data:"],
|
||||
"img-src": ["'self'", "data:", "blob:"],
|
||||
"connect-src": ["'self'"],
|
||||
"object-src": ["'none'"],
|
||||
"upgrade-insecure-requests": secureCookies ? [] : null,
|
||||
|
|
@ -146,6 +154,77 @@ export function createApp({
|
|||
});
|
||||
}
|
||||
});
|
||||
app.get("/api/products/:code", requireAuth, async (req, res) => {
|
||||
const gtin = normalizeBarcode(req.params.code);
|
||||
if (!gtin)
|
||||
return res
|
||||
.status(400)
|
||||
.json({ error: "That isn’t a valid product barcode." });
|
||||
if (!products)
|
||||
return res
|
||||
.status(503)
|
||||
.json({ error: "The product database isn’t available." });
|
||||
let product = products.lookup(gtin);
|
||||
if (product === undefined) {
|
||||
try {
|
||||
product = await fetchProduct(gtin);
|
||||
} catch {
|
||||
return res.status(502).json({
|
||||
error:
|
||||
"This product isn’t in the local database yet, and Open Food Facts couldn’t be reached.",
|
||||
gtin,
|
||||
});
|
||||
}
|
||||
products.remember(gtin, product);
|
||||
}
|
||||
if (!product)
|
||||
return res
|
||||
.status(404)
|
||||
.json({ error: "No nutrition information for this barcode.", gtin });
|
||||
res.json({ food: productToFood(product) });
|
||||
});
|
||||
app.get("/api/products", requireAuth, (_, res) =>
|
||||
res.json({ sources: products?.status() || [] }),
|
||||
);
|
||||
// One label at a time: recognition already uses every core.
|
||||
let labelQueue = Promise.resolve();
|
||||
app.post(
|
||||
"/api/label",
|
||||
requireAuth,
|
||||
express.raw({
|
||||
type: ["image/jpeg", "image/png", "image/webp"],
|
||||
limit: "16mb",
|
||||
}),
|
||||
async (req, res) => {
|
||||
if (!Buffer.isBuffer(req.body) || !req.body.length)
|
||||
return res
|
||||
.status(415)
|
||||
.json({ error: "Send a JPEG, PNG or WebP photo of the label." });
|
||||
if (!ocr)
|
||||
return res
|
||||
.status(503)
|
||||
.json({ error: "Label scanning isn’t available on this server." });
|
||||
const run = labelQueue.then(async () => parseLabel(await ocr(req.body)));
|
||||
labelQueue = run.catch(() => {});
|
||||
try {
|
||||
const label = await run;
|
||||
if (!label.found.length)
|
||||
return res.status(422).json({
|
||||
error:
|
||||
"No nutrition facts were found. Fill the frame with the label and keep it sharp.",
|
||||
});
|
||||
res.json({ label });
|
||||
} catch (error) {
|
||||
if (error?.unavailable)
|
||||
return res
|
||||
.status(503)
|
||||
.json({ error: "Label scanning isn’t available on this server." });
|
||||
res
|
||||
.status(422)
|
||||
.json({ error: "That photo couldn’t be read. Try another one." });
|
||||
}
|
||||
},
|
||||
);
|
||||
app.use("/api", (_, res) => res.status(404).json({ error: "Not found" }));
|
||||
if (existsSync(dist)) {
|
||||
// The service worker is also served without a file extension. Reverse
|
||||
|
|
|
|||
|
|
@ -3,6 +3,9 @@ import { resolve } from "node:path";
|
|||
import { openDatabase } from "./db.js";
|
||||
import { createApp } from "./app.js";
|
||||
import { hashPassword } from "./auth.js";
|
||||
import { openProducts } from "./products/db.js";
|
||||
import { scheduleProductImports } from "./products/scheduler.js";
|
||||
import { createOcr } from "./ocr/engine.js";
|
||||
const secret = (name) =>
|
||||
process.env[name + "_FILE"]
|
||||
? readFileSync(process.env[name + "_FILE"], "utf8").trimEnd()
|
||||
|
|
@ -30,12 +33,30 @@ if (
|
|||
const dataDir = resolve(process.env.DATA_DIR || "data");
|
||||
mkdirSync(dataDir, { recursive: true });
|
||||
const store = openDatabase(resolve(dataDir, "calio.sqlite"));
|
||||
const products = openProducts(resolve(dataDir, "products.sqlite"));
|
||||
const stopImports = scheduleProductImports(dataDir);
|
||||
// Models load in the background so the first label scan doesn't wait.
|
||||
const engine = createOcr(resolve(process.env.MODELS_DIR || "models")).catch(
|
||||
(error) => {
|
||||
console.error("Label scanning is unavailable:", error.message);
|
||||
return null;
|
||||
},
|
||||
);
|
||||
const app = createApp({
|
||||
store,
|
||||
username: process.env.CALIO_USERNAME || "calio",
|
||||
passwordHash,
|
||||
origin,
|
||||
secureCookies,
|
||||
products,
|
||||
ocr: async (image) => {
|
||||
const ocr = await engine;
|
||||
if (!ocr)
|
||||
throw Object.assign(new Error("OCR models are not loaded"), {
|
||||
unavailable: true,
|
||||
});
|
||||
return ocr.read(image);
|
||||
},
|
||||
});
|
||||
const server = app.listen(Number(process.env.PORT || 3000), "0.0.0.0", () =>
|
||||
console.log(`Calio listening at ${origin}`),
|
||||
|
|
@ -43,6 +64,8 @@ const server = app.listen(Number(process.env.PORT || 3000), "0.0.0.0", () =>
|
|||
for (const signal of ["SIGTERM", "SIGINT"])
|
||||
process.on(signal, () =>
|
||||
server.close(() => {
|
||||
stopImports();
|
||||
products.close();
|
||||
store.close();
|
||||
process.exit(0);
|
||||
}),
|
||||
|
|
|
|||
434
server/ocr/engine.js
Normal file
434
server/ocr/engine.js
Normal file
|
|
@ -0,0 +1,434 @@
|
|||
// PP-OCRv5 text detection + recognition on ONNX Runtime (CPU). Mirrors
|
||||
// PaddleOCR's own pre- and post-processing: DB detection, min-area boxes with
|
||||
// unclip, then CTC decoding of each line crop.
|
||||
import { readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import ort from "onnxruntime-node";
|
||||
import sharp from "sharp";
|
||||
|
||||
const DET_THRESH = 0.3;
|
||||
const BOX_THRESH = 0.6;
|
||||
const UNCLIP_RATIO = 1.5;
|
||||
const REC_HEIGHT = 48;
|
||||
const REC_BATCH = 8;
|
||||
|
||||
/** Reads the CTC character list out of a PaddleOCR inference.yml. */
|
||||
export function readDictionary(yaml) {
|
||||
const lines = yaml.split(/\r?\n/);
|
||||
const start = lines.findIndex((line) => /^\s*character_dict:\s*$/.test(line));
|
||||
if (start < 0) throw new Error("No character_dict in model config");
|
||||
const chars = [];
|
||||
for (const line of lines.slice(start + 1)) {
|
||||
const match = /^\s*- (.*)$/.exec(line);
|
||||
if (!match) break;
|
||||
let value = match[1];
|
||||
if (value.startsWith("'")) value = value.slice(1, -1).replace(/''/g, "'");
|
||||
else if (value.startsWith('"')) value = JSON.parse(value);
|
||||
chars.push(value);
|
||||
}
|
||||
return ["", ...chars, " "]; // blank, dictionary, then the space character
|
||||
}
|
||||
|
||||
// ---------- geometry ----------
|
||||
|
||||
function convexHull(points) {
|
||||
points.sort((a, b) => a[0] - b[0] || a[1] - b[1]);
|
||||
const cross = (o, a, b) =>
|
||||
(a[0] - o[0]) * (b[1] - o[1]) - (a[1] - o[1]) * (b[0] - o[0]);
|
||||
const lower = [];
|
||||
for (const p of points) {
|
||||
while (lower.length >= 2 && cross(lower.at(-2), lower.at(-1), p) <= 0)
|
||||
lower.pop();
|
||||
lower.push(p);
|
||||
}
|
||||
const upper = [];
|
||||
for (let i = points.length - 1; i >= 0; i--) {
|
||||
const p = points[i];
|
||||
while (upper.length >= 2 && cross(upper.at(-2), upper.at(-1), p) <= 0)
|
||||
upper.pop();
|
||||
upper.push(p);
|
||||
}
|
||||
return lower.slice(0, -1).concat(upper.slice(0, -1));
|
||||
}
|
||||
|
||||
/** Minimum-area rectangle as centre, unit axes and side lengths. */
|
||||
export function minAreaRect(points) {
|
||||
const hull = convexHull(points);
|
||||
if (hull.length < 3) {
|
||||
const [x, y] = hull[0] || [0, 0];
|
||||
return { cx: x, cy: y, ux: 1, uy: 0, w: 1, h: 1 };
|
||||
}
|
||||
let best = null;
|
||||
for (let i = 0; i < hull.length; i++) {
|
||||
const [x1, y1] = hull[i];
|
||||
const [x2, y2] = hull[(i + 1) % hull.length];
|
||||
const length = Math.hypot(x2 - x1, y2 - y1);
|
||||
if (!length) continue;
|
||||
const ux = (x2 - x1) / length;
|
||||
const uy = (y2 - y1) / length;
|
||||
let minU = Infinity;
|
||||
let maxU = -Infinity;
|
||||
let minV = Infinity;
|
||||
let maxV = -Infinity;
|
||||
for (const [x, y] of hull) {
|
||||
const u = x * ux + y * uy;
|
||||
const v = -x * uy + y * ux;
|
||||
minU = Math.min(minU, u);
|
||||
maxU = Math.max(maxU, u);
|
||||
minV = Math.min(minV, v);
|
||||
maxV = Math.max(maxV, v);
|
||||
}
|
||||
const area = (maxU - minU) * (maxV - minV);
|
||||
if (!best || area < best.area) {
|
||||
const cu = (minU + maxU) / 2;
|
||||
const cv = (minV + maxV) / 2;
|
||||
best = {
|
||||
area,
|
||||
cx: cu * ux - cv * uy,
|
||||
cy: cu * uy + cv * ux,
|
||||
ux,
|
||||
uy,
|
||||
w: maxU - minU,
|
||||
h: maxV - minV,
|
||||
};
|
||||
}
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
/**
|
||||
* Orients a rectangle for reading: the axis nearer horizontal runs left to
|
||||
* right, the other top to bottom. Returns the top-left corner and axes.
|
||||
*/
|
||||
function readingFrame({ cx, cy, ux, uy, w, h }) {
|
||||
let ax = [ux, uy, w];
|
||||
let bx = [-uy, ux, h];
|
||||
if (Math.abs(bx[0]) > Math.abs(ax[0])) [ax, bx] = [bx, ax];
|
||||
if (ax[0] < 0) ax = [-ax[0], -ax[1], ax[2]];
|
||||
if (bx[1] < 0) bx = [-bx[0], -bx[1], bx[2]];
|
||||
const [dx, dy, width] = ax;
|
||||
const [ex, ey, height] = bx;
|
||||
return {
|
||||
x: cx - (dx * width) / 2 - (ex * height) / 2,
|
||||
y: cy - (dy * width) / 2 - (ey * height) / 2,
|
||||
dx,
|
||||
dy,
|
||||
ex,
|
||||
ey,
|
||||
width,
|
||||
height,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------- detection ----------
|
||||
|
||||
function detectionBoxes(prob, width, height) {
|
||||
const visited = new Uint8Array(width * height);
|
||||
const boxes = [];
|
||||
const stack = [];
|
||||
for (let start = 0; start < prob.length; start++) {
|
||||
if (visited[start] || prob[start] <= DET_THRESH) continue;
|
||||
// Flood-fill one connected region, keeping each row's extreme pixels as
|
||||
// hull candidates.
|
||||
const rows = new Map();
|
||||
let sum = 0;
|
||||
let count = 0;
|
||||
stack.push(start);
|
||||
visited[start] = 1;
|
||||
while (stack.length) {
|
||||
const index = stack.pop();
|
||||
const x = index % width;
|
||||
const y = (index - x) / width;
|
||||
sum += prob[index];
|
||||
count++;
|
||||
const row = rows.get(y);
|
||||
if (!row) rows.set(y, [x, x]);
|
||||
else {
|
||||
if (x < row[0]) row[0] = x;
|
||||
if (x > row[1]) row[1] = x;
|
||||
}
|
||||
for (const next of [
|
||||
x > 0 ? index - 1 : -1,
|
||||
x < width - 1 ? index + 1 : -1,
|
||||
y > 0 ? index - width : -1,
|
||||
y < height - 1 ? index + width : -1,
|
||||
])
|
||||
if (next >= 0 && !visited[next] && prob[next] > DET_THRESH) {
|
||||
visited[next] = 1;
|
||||
stack.push(next);
|
||||
}
|
||||
}
|
||||
if (count < 4 || sum / count < BOX_THRESH) continue;
|
||||
const points = [];
|
||||
for (const [y, [x0, x1]] of rows)
|
||||
points.push([x0, y], [x1 + 1, y], [x0, y + 1], [x1 + 1, y + 1]);
|
||||
const rect = minAreaRect(points);
|
||||
if (Math.min(rect.w, rect.h) < 3) continue;
|
||||
// Expand by the DB "unclip" distance, as PaddleOCR does with pyclipper.
|
||||
const d = (rect.w * rect.h * UNCLIP_RATIO) / (2 * (rect.w + rect.h));
|
||||
rect.w += 2 * d;
|
||||
rect.h += 2 * d;
|
||||
if (Math.min(rect.w, rect.h) < 5) continue;
|
||||
boxes.push({ ...rect, score: sum / count });
|
||||
}
|
||||
return boxes;
|
||||
}
|
||||
|
||||
// ---------- recognition ----------
|
||||
|
||||
/** Samples a rectangle of the RGB image into a normalised BGR tensor slot. */
|
||||
function sampleCrop(image, frame, out, offset, batchWidth, cropWidth) {
|
||||
const { data, width, height } = image;
|
||||
const plane = REC_HEIGHT * batchWidth;
|
||||
const scale = frame.height / REC_HEIGHT;
|
||||
const taps = Math.min(4, Math.max(1, Math.round(scale)));
|
||||
const pixel = (x, y, c) => {
|
||||
const xi = Math.min(width - 1, Math.max(0, x));
|
||||
const yi = Math.min(height - 1, Math.max(0, y));
|
||||
return data[(yi * width + xi) * 3 + c];
|
||||
};
|
||||
for (let ty = 0; ty < REC_HEIGHT; ty++)
|
||||
for (let tx = 0; tx < cropWidth; tx++) {
|
||||
let r = 0;
|
||||
let g = 0;
|
||||
let b = 0;
|
||||
for (let sy = 0; sy < taps; sy++)
|
||||
for (let sx = 0; sx < taps; sx++) {
|
||||
const u = ((tx + (sx + 0.5) / taps) / cropWidth) * frame.width;
|
||||
const v = ((ty + (sy + 0.5) / taps) / REC_HEIGHT) * frame.height;
|
||||
const x = frame.x + frame.dx * u + frame.ex * v - 0.5;
|
||||
const y = frame.y + frame.dy * u + frame.ey * v - 0.5;
|
||||
const x0 = Math.floor(x);
|
||||
const y0 = Math.floor(y);
|
||||
const fx = x - x0;
|
||||
const fy = y - y0;
|
||||
for (let c = 0; c < 3; c++) {
|
||||
const value =
|
||||
pixel(x0, y0, c) * (1 - fx) * (1 - fy) +
|
||||
pixel(x0 + 1, y0, c) * fx * (1 - fy) +
|
||||
pixel(x0, y0 + 1, c) * (1 - fx) * fy +
|
||||
pixel(x0 + 1, y0 + 1, c) * fx * fy;
|
||||
if (c === 0) r += value;
|
||||
else if (c === 1) g += value;
|
||||
else b += value;
|
||||
}
|
||||
}
|
||||
const n = taps * taps * 127.5;
|
||||
const at = offset + ty * batchWidth + tx;
|
||||
out[at] = b / n - 1;
|
||||
out[at + plane] = g / n - 1;
|
||||
out[at + 2 * plane] = r / n - 1;
|
||||
}
|
||||
}
|
||||
|
||||
function ctcDecode(output, dims, chars) {
|
||||
const [count, steps, classes] = dims;
|
||||
const results = [];
|
||||
for (let n = 0; n < count; n++) {
|
||||
let text = "";
|
||||
let confidence = 0;
|
||||
let kept = 0;
|
||||
let previous = -1;
|
||||
for (let t = 0; t < steps; t++) {
|
||||
const base = (n * steps + t) * classes;
|
||||
let best = 0;
|
||||
let bestValue = output[base];
|
||||
for (let c = 1; c < classes; c++)
|
||||
if (output[base + c] > bestValue) {
|
||||
best = c;
|
||||
bestValue = output[base + c];
|
||||
}
|
||||
if (best !== 0 && best !== previous) {
|
||||
text += chars[best] ?? "";
|
||||
confidence += bestValue;
|
||||
kept++;
|
||||
}
|
||||
previous = best;
|
||||
}
|
||||
results.push({ text, score: kept ? confidence / kept : 0 });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
// ---------- pipeline ----------
|
||||
|
||||
export async function createOcr(modelDir, { detSide = 1600 } = {}) {
|
||||
const options = {
|
||||
graphOptimizationLevel: "all",
|
||||
// The arena would keep ~2 GB of detection buffers after the first scan.
|
||||
enableCpuMemArena: false,
|
||||
};
|
||||
const [det, rec] = await Promise.all([
|
||||
ort.InferenceSession.create(
|
||||
join(modelDir, "det", "inference.onnx"),
|
||||
options,
|
||||
),
|
||||
ort.InferenceSession.create(
|
||||
join(modelDir, "rec", "inference.onnx"),
|
||||
options,
|
||||
),
|
||||
]);
|
||||
const chars = readDictionary(
|
||||
readFileSync(join(modelDir, "rec", "inference.yml"), "utf8"),
|
||||
);
|
||||
|
||||
async function detect(image) {
|
||||
// Small photos are enlarged too: tiny print detects far better.
|
||||
const ratio = Math.min(2, detSide / Math.max(image.width, image.height));
|
||||
const w = Math.max(32, Math.round((image.width * ratio) / 32) * 32);
|
||||
const h = Math.max(32, Math.round((image.height * ratio) / 32) * 32);
|
||||
const { data } = await sharp(image.data, {
|
||||
raw: { width: image.width, height: image.height, channels: 3 },
|
||||
})
|
||||
.resize(w, h, { fit: "fill" })
|
||||
.raw()
|
||||
.toBuffer({ resolveWithObject: true });
|
||||
const input = new Float32Array(3 * w * h);
|
||||
const mean = [0.406, 0.456, 0.485]; // applied to B, G, R
|
||||
const std = [0.225, 0.224, 0.229];
|
||||
for (let i = 0; i < w * h; i++)
|
||||
for (let c = 0; c < 3; c++)
|
||||
// Channel c of the tensor is BGR; data is RGB.
|
||||
input[c * w * h + i] = (data[i * 3 + (2 - c)] / 255 - mean[c]) / std[c];
|
||||
const result = await det.run({
|
||||
[det.inputNames[0]]: new ort.Tensor("float32", input, [1, 3, h, w]),
|
||||
});
|
||||
const prob = result[det.outputNames[0]].data;
|
||||
const sx = image.width / w;
|
||||
const sy = image.height / h;
|
||||
return detectionBoxes(prob, w, h).map((box) => {
|
||||
// Scale the rectangle back to the full image.
|
||||
const ax = box.ux * sx;
|
||||
const ay = box.uy * sy;
|
||||
const length = Math.hypot(ax, ay);
|
||||
const bx = -box.uy * sx;
|
||||
const by = box.ux * sy;
|
||||
return {
|
||||
cx: box.cx * sx,
|
||||
cy: box.cy * sy,
|
||||
ux: ax / length,
|
||||
uy: ay / length,
|
||||
w: box.w * length,
|
||||
h: box.h * Math.hypot(bx, by),
|
||||
score: box.score,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async function recognize(image, boxes) {
|
||||
const frames = boxes.map((box) => {
|
||||
const frame = readingFrame(box);
|
||||
return { frame, ratio: frame.width / Math.max(1, frame.height) };
|
||||
});
|
||||
const order = frames
|
||||
.map((_, i) => i)
|
||||
.sort((a, b) => frames[a].ratio - frames[b].ratio);
|
||||
const texts = new Array(frames.length);
|
||||
for (let start = 0; start < order.length; start += REC_BATCH) {
|
||||
const batch = order.slice(start, start + REC_BATCH);
|
||||
const maxRatio = Math.max(
|
||||
320 / REC_HEIGHT,
|
||||
...batch.map((i) => frames[i].ratio),
|
||||
);
|
||||
const batchWidth = Math.min(3200, Math.ceil(REC_HEIGHT * maxRatio));
|
||||
const plane = REC_HEIGHT * batchWidth;
|
||||
const input = new Float32Array(batch.length * 3 * plane);
|
||||
batch.forEach((i, n) => {
|
||||
const cropWidth = Math.min(
|
||||
batchWidth,
|
||||
Math.max(1, Math.ceil(REC_HEIGHT * frames[i].ratio)),
|
||||
);
|
||||
sampleCrop(
|
||||
image,
|
||||
frames[i].frame,
|
||||
input,
|
||||
n * 3 * plane,
|
||||
batchWidth,
|
||||
cropWidth,
|
||||
);
|
||||
});
|
||||
const result = await rec.run({
|
||||
[rec.inputNames[0]]: new ort.Tensor("float32", input, [
|
||||
batch.length,
|
||||
3,
|
||||
REC_HEIGHT,
|
||||
batchWidth,
|
||||
]),
|
||||
});
|
||||
const output = result[rec.outputNames[0]];
|
||||
ctcDecode(output.data, output.dims, chars).forEach((value, n) => {
|
||||
texts[batch[n]] = value;
|
||||
});
|
||||
}
|
||||
return boxes.map((box, i) => {
|
||||
const { frame } = frames[i];
|
||||
const xs = [
|
||||
frame.x,
|
||||
frame.x + frame.dx * frame.width,
|
||||
frame.x + frame.ex * frame.height,
|
||||
frame.x + frame.dx * frame.width + frame.ex * frame.height,
|
||||
];
|
||||
const ys = [
|
||||
frame.y,
|
||||
frame.y + frame.dy * frame.width,
|
||||
frame.y + frame.ey * frame.height,
|
||||
frame.y + frame.dy * frame.width + frame.ey * frame.height,
|
||||
];
|
||||
return {
|
||||
text: texts[i].text,
|
||||
score: texts[i].score,
|
||||
angle: Math.atan2(frame.dy, frame.dx),
|
||||
cx: (Math.min(...xs) + Math.max(...xs)) / 2,
|
||||
cy: (Math.min(...ys) + Math.max(...ys)) / 2,
|
||||
x0: Math.min(...xs),
|
||||
y0: Math.min(...ys),
|
||||
x1: Math.max(...xs),
|
||||
y1: Math.max(...ys),
|
||||
height: frame.height,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async function decode(buffer, rotate = 0) {
|
||||
const { data, info } = await sharp(buffer, { failOn: "none" })
|
||||
.rotate() // honour EXIF orientation
|
||||
.rotate(rotate)
|
||||
.resize(2400, 2400, { fit: "inside", withoutEnlargement: true })
|
||||
.removeAlpha()
|
||||
.raw()
|
||||
.toBuffer({ resolveWithObject: true });
|
||||
return { data, width: info.width, height: info.height };
|
||||
}
|
||||
|
||||
async function readImage(image) {
|
||||
const boxes = await detect(image);
|
||||
const lines = (await recognize(image, boxes)).filter((line) =>
|
||||
line.text.trim(),
|
||||
);
|
||||
const score = lines.reduce(
|
||||
(sum, line) => sum + line.score * line.text.length,
|
||||
0,
|
||||
);
|
||||
return { lines, score, boxes };
|
||||
}
|
||||
|
||||
/** Returns recognised lines with their bounding boxes, in image pixels. */
|
||||
async function read(buffer) {
|
||||
const image = await decode(buffer);
|
||||
let best = await readImage(image);
|
||||
// A label photographed sideways yields mostly tall, narrow boxes; try
|
||||
// both quarter turns and keep whichever reads more confidently.
|
||||
const tall = best.boxes.filter((box) => {
|
||||
const frame = readingFrame(box);
|
||||
return frame.height > frame.width * 1.5;
|
||||
}).length;
|
||||
if (best.boxes.length && tall > best.boxes.length / 2)
|
||||
for (const angle of [90, 270]) {
|
||||
const attempt = await readImage(await decode(buffer, angle));
|
||||
if (attempt.score > best.score) best = attempt;
|
||||
}
|
||||
return best.lines;
|
||||
}
|
||||
|
||||
return { read };
|
||||
}
|
||||
461
server/ocr/label.js
Normal file
461
server/ocr/label.js
Normal file
|
|
@ -0,0 +1,461 @@
|
|||
// Turns OCR text boxes from a Nutrition Facts panel into a food. Built for
|
||||
// US/Canadian labels, with the common EU "per 100 g" table as a fallback.
|
||||
import { convertMass, nutrientUnit } from "../../shared/nutrients.js";
|
||||
|
||||
// FDA Daily Values (2016 rule) in the unit printed on labels; a %DV next to
|
||||
// an amount is used to catch misread digits and to fill amounts that are only
|
||||
// given as a percentage.
|
||||
const dailyValues = {
|
||||
fat: [78, "g"],
|
||||
"saturated-fat": [20, "g"],
|
||||
cholesterol: [300, "mg"],
|
||||
sodium: [2300, "mg"],
|
||||
carbohydrates: [275, "g"],
|
||||
fiber: [28, "g"],
|
||||
"added-sugars": [50, "g"],
|
||||
proteins: [50, "g"],
|
||||
"vitamin-d": [20, "mcg"],
|
||||
calcium: [1300, "mg"],
|
||||
iron: [18, "mg"],
|
||||
potassium: [4700, "mg"],
|
||||
"vitamin-a": [900, "mcg"],
|
||||
"vitamin-c": [90, "mg"],
|
||||
"vitamin-e": [15, "mg"],
|
||||
"vitamin-k": [120, "mcg"],
|
||||
"vitamin-b1": [1.2, "mg"],
|
||||
"vitamin-b2": [1.3, "mg"],
|
||||
"vitamin-pp": [16, "mg"],
|
||||
"vitamin-b6": [1.7, "mg"],
|
||||
"vitamin-b9": [400, "mcg"],
|
||||
"vitamin-b12": [2.4, "mcg"],
|
||||
biotin: [30, "mcg"],
|
||||
"pantothenic-acid": [5, "mg"],
|
||||
phosphorus: [1250, "mg"],
|
||||
iodine: [150, "mcg"],
|
||||
magnesium: [420, "mg"],
|
||||
zinc: [11, "mg"],
|
||||
selenium: [55, "mcg"],
|
||||
copper: [0.9, "mg"],
|
||||
manganese: [2.3, "mg"],
|
||||
chromium: [35, "mcg"],
|
||||
molybdenum: [45, "mcg"],
|
||||
chloride: [2300, "mg"],
|
||||
choline: [550, "mg"],
|
||||
};
|
||||
|
||||
// Most specific first: "saturated fat" must claim its row before "fat" does.
|
||||
const fields = [
|
||||
["trans-fat", /\btrans\.?\s*fat|\bgrasas?\s*trans\b/],
|
||||
[
|
||||
"saturated-fat",
|
||||
/\b(?:of which\s*)?sat(?:urated|\.)?\s*fat|\bsaturates\b|\bgrasas?\s*saturadas?\b/,
|
||||
],
|
||||
["polyunsaturated-fat", /\bpoly(?:unsat(?:urated|\.)?)?\s*fat/],
|
||||
["monounsaturated-fat", /\bmono(?:unsat(?:urated|\.)?)?\s*fat/],
|
||||
[
|
||||
"fat",
|
||||
/\b(?:total\s*)?fat\b(?!\s*free)|\bgrasas?(?:\s*totales?)?(?:\s*\(?lipidos\)?)?/,
|
||||
],
|
||||
["cholesterol", /\bcholest(?:erol|\.)?|\bcolesterol\b/],
|
||||
["sodium", /\bsodium\b|\bsodio\b/],
|
||||
["salt", /\bsalt\b/],
|
||||
["soluble-fiber", /(?<!in)\bsoluble\s*fib(?:er|re)/],
|
||||
["insoluble-fiber", /\binsoluble\s*fib(?:er|re)/],
|
||||
["fiber", /\b(?:dietary\s*)?fib(?:er|re)\b|\bfibra(?:\s*dietetica)?\b/],
|
||||
["polyols", /\bsugar\s*alcohols?\b|\bpolyols\b/],
|
||||
[
|
||||
"sugars",
|
||||
/\b(?:total\s*|of which\s*)?sugars?\b|\bazucares(?:\s*totales)?\b(?!\s*anadidos)/,
|
||||
],
|
||||
[
|
||||
"carbohydrates",
|
||||
/\b(?:total\s*)?carb(?:ohydrates?|s|\.)?|\bcarbohidratos(?:\s*\(?hidratos\s*(?:\d\s*)?de\s*carbono\)?)?|\bhidratos\s*de\s*carbono\b/,
|
||||
],
|
||||
["proteins", /\bprotein\b|\bproteinas?\b/],
|
||||
["vitamin-d", /\bvit(?:amin|amina|\.)?\s*d\d?\b/],
|
||||
["vitamin-a", /\bvit(?:amin|\.)?\s*a\b/],
|
||||
["vitamin-c", /\bvit(?:amin|\.)?\s*c\b/],
|
||||
["vitamin-e", /\bvit(?:amin|\.)?\s*e\b/],
|
||||
["vitamin-k", /\bvit(?:amin|\.)?\s*k\d?\b/],
|
||||
["vitamin-b6", /\bvit(?:amin|\.)?\s*b\s*-?6\b/],
|
||||
["vitamin-b12", /\bvit(?:amin|\.)?\s*b\s*-?12\b/],
|
||||
["vitamin-b1", /\bthiamine?\b/],
|
||||
["vitamin-b2", /\briboflavin\b/],
|
||||
["vitamin-pp", /\bniacin\b/],
|
||||
["vitamin-b9", /\bfolate\b|\bfolic acid\b/],
|
||||
["biotin", /\bbiotin\b/],
|
||||
["pantothenic-acid", /\bpantothenic\b/],
|
||||
["calcium", /\bcalcium\b|\bcalcio\b/],
|
||||
["iron", /\biron\b|\bhierro\b/],
|
||||
["potassium", /\bpotas(?:sium|\.)?|\bpotasio\b/],
|
||||
["phosphorus", /\bphosphorus\b/],
|
||||
["magnesium", /\bmagnesium\b/],
|
||||
["zinc", /\bzinc\b/],
|
||||
["selenium", /\bselenium\b/],
|
||||
["copper", /\bcopper\b/],
|
||||
["manganese", /\bmanganese\b/],
|
||||
["iodine", /\biodine\b/],
|
||||
["chromium", /\bchromium\b/],
|
||||
["molybdenum", /\bmolybdenum\b/],
|
||||
["chloride", /\bchloride\b/],
|
||||
["choline", /\bcholine\b/],
|
||||
["caffeine", /\bcaffeine\b/],
|
||||
];
|
||||
// The unit printed when OCR loses it.
|
||||
const defaultUnits = {
|
||||
"energy-kcal": "kcal",
|
||||
sodium: "mg",
|
||||
cholesterol: "mg",
|
||||
potassium: "mg",
|
||||
calcium: "mg",
|
||||
iron: "mg",
|
||||
"vitamin-d": "mcg",
|
||||
};
|
||||
|
||||
/**
|
||||
* Groups boxes into reading rows. Coordinates are first turned by the
|
||||
* dominant text angle, so a tilted photo still lines up "Sodium 160mg 7%".
|
||||
*/
|
||||
export function rows(lines) {
|
||||
const angles = lines
|
||||
.filter(
|
||||
(line) => line.angle !== undefined && line.x1 - line.x0 > 3 * line.height,
|
||||
)
|
||||
.map((line) => line.angle)
|
||||
.sort((a, b) => a - b);
|
||||
const angle = angles.length ? angles[Math.floor(angles.length / 2)] : 0;
|
||||
const cos = Math.cos(angle);
|
||||
const sin = Math.sin(angle);
|
||||
const placed = lines.map((line) => {
|
||||
const cx = line.cx ?? (line.x0 + line.x1) / 2;
|
||||
const cy = line.cy ?? (line.y0 + line.y1) / 2;
|
||||
const height = line.height ?? line.y1 - line.y0;
|
||||
const y = -cx * sin + cy * cos;
|
||||
return {
|
||||
text: line.text,
|
||||
x: cx * cos + cy * sin,
|
||||
y0: y - height / 2,
|
||||
y1: y + height / 2,
|
||||
};
|
||||
});
|
||||
placed.sort((a, b) => a.y0 + a.y1 - b.y0 - b.y1);
|
||||
const result = [];
|
||||
for (const line of placed) {
|
||||
const height = line.y1 - line.y0;
|
||||
const row = result.find((candidate) => {
|
||||
const overlap =
|
||||
Math.min(candidate.y1, line.y1) - Math.max(candidate.y0, line.y0);
|
||||
return overlap > 0.5 * Math.min(height, candidate.y1 - candidate.y0);
|
||||
});
|
||||
if (row) row.items.push(line);
|
||||
else result.push({ y0: line.y0, y1: line.y1, items: [line] });
|
||||
}
|
||||
return result
|
||||
.sort((a, b) => a.y0 - b.y0)
|
||||
.map((row) => {
|
||||
row.items.sort((a, b) => a.x - b.x);
|
||||
return clean(row.items.map((item) => item.text).join(" "));
|
||||
});
|
||||
}
|
||||
|
||||
// Label vocabulary; OCR misspellings within a small edit distance of these
|
||||
// ("Sodlum", "Proteln", "Caicium") are snapped back before matching.
|
||||
const vocabulary = `calories total fat saturated trans polyunsaturated
|
||||
monounsaturated cholesterol sodium carbohydrate carbohydrates dietary fiber
|
||||
soluble insoluble sugars sugar added includes alcohols protein vitamin
|
||||
calcium iron potassium serving servings size container amount daily value
|
||||
thiamin riboflavin niacin folate phosphorus magnesium zinc selenium copper
|
||||
manganese biotin pantothenic iodine chromium molybdenum chloride choline
|
||||
caffeine energy saturates fibre salt nutrition facts grasa grasas saturada
|
||||
colesterol sodio fibra dietetica azucares anadidos carbohidratos hidratos
|
||||
carbono proteinas calcio hierro potasio vitamina energetico contenido
|
||||
porcion lipidos`.split(/\s+/);
|
||||
const known = new Set(vocabulary);
|
||||
function distance(a, b, limit) {
|
||||
if (Math.abs(a.length - b.length) > limit) return limit + 1;
|
||||
let previous = Array.from({ length: b.length + 1 }, (_, i) => i);
|
||||
for (let i = 1; i <= a.length; i++) {
|
||||
const current = [i];
|
||||
let best = i;
|
||||
for (let j = 1; j <= b.length; j++) {
|
||||
current[j] = Math.min(
|
||||
previous[j] + 1,
|
||||
current[j - 1] + 1,
|
||||
previous[j - 1] + (a[i - 1] === b[j - 1] ? 0 : 1),
|
||||
);
|
||||
best = Math.min(best, current[j]);
|
||||
}
|
||||
if (best > limit) return limit + 1;
|
||||
previous = current;
|
||||
}
|
||||
return previous[b.length];
|
||||
}
|
||||
function correctWord(word) {
|
||||
const lower = word.toLowerCase();
|
||||
if (known.has(lower)) return word;
|
||||
const limit = lower.length >= 8 ? 2 : 1;
|
||||
let best = null;
|
||||
let bestDistance = limit + 1;
|
||||
for (const candidate of vocabulary) {
|
||||
const d = distance(lower, candidate, limit);
|
||||
if (d < bestDistance) {
|
||||
best = candidate;
|
||||
bestDistance = d;
|
||||
}
|
||||
}
|
||||
return best ?? word;
|
||||
}
|
||||
|
||||
/** Undoes the usual OCR confusions in label text. */
|
||||
export function clean(text) {
|
||||
return (
|
||||
text
|
||||
.normalize("NFD")
|
||||
.replace(/[̀-ͯ]/g, "") // Azúcares → Azucares
|
||||
.replace(/[a-z]{5,}/gi, correctWord)
|
||||
.replace(/[µμ]g\b/gi, "mcg")
|
||||
.replace(/\bug\b/gi, "mcg")
|
||||
.replace(/less than/gi, "<")
|
||||
// Letters standing in for digits next to digits or units: "Og", "1O".
|
||||
.replace(/(?<=\d)[oO]|[oO](?=\d)/g, "0")
|
||||
.replace(/(?<![a-z])[oO](?=\s?(?:m?c?g|%)\b)/g, "0")
|
||||
.replace(/(?<=\d)[lI|](?=\d|\s?m?g\b)|(?<![a-z])[lI|](?=\d)/g, "1")
|
||||
// A footnote mark read as a digit: "Carbohydrate1 13g".
|
||||
.replace(/\b([a-z]{4,})[1†*]\s+(?=<?\d)/gi, "$1 ")
|
||||
// Thousands separators, then decimal commas.
|
||||
.replace(/(\d),(\d{3})(?!\d)/g, "$1$2")
|
||||
.replace(/(\d),(\d)/g, "$1.$2")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim()
|
||||
);
|
||||
}
|
||||
|
||||
// The amount may follow a split word ("Carbohy ydrate 42g"); a number
|
||||
// directly followed by % is a percentage, not an amount.
|
||||
const amountPattern =
|
||||
/^\s*((?:[a-z]{1,10}\s+){0,2})[\s:.\-–—]*(<\s*)?(\d+(?:\.\d+)?|\.\d+)\s*(mcg|mg|g|kcal|kj|iu)?(?![a-z%\d.])(?!\s*%)/i;
|
||||
const percentPattern = /^[^\d%]{0,12}?(\d{1,4})\s*%/;
|
||||
|
||||
function readAmount(rest) {
|
||||
const found = amountPattern.exec(rest);
|
||||
if (!found) return null;
|
||||
// Never skip over another nutrient's name: in "Cholest. Protein 9g" the
|
||||
// 9g is protein's.
|
||||
const skipped = found[1].trim().toLowerCase().split(/\s+/).filter(Boolean);
|
||||
if (skipped.some((word) => known.has(word))) return null;
|
||||
const match = [found[0], ...found.slice(2)];
|
||||
const after = rest.slice(match[0].length);
|
||||
const percent = percentPattern.exec(after);
|
||||
return {
|
||||
less: Boolean(match[1]),
|
||||
raw: match[2],
|
||||
value: Number(match[2]),
|
||||
unit: match[3]?.toLowerCase(),
|
||||
percent: percent ? Number(percent[1]) : null,
|
||||
};
|
||||
}
|
||||
|
||||
const toCatalog = (key, value, unit) =>
|
||||
key === "energy-kcal"
|
||||
? value
|
||||
: convertMass(value, unit === "mcg" ? "µg" : unit, nutrientUnit(key));
|
||||
|
||||
/** Resolves an amount, using its %DV to correct misreads such as 8g → "89". */
|
||||
function resolve(key, reading) {
|
||||
const dv = dailyValues[key];
|
||||
let unit = reading.unit || defaultUnits[key] || dv?.[1] || "g";
|
||||
if (unit === "iu" || unit === "kj" || unit === "kcal") return null;
|
||||
if (unit === "g" && dv?.[1] === "mcg") unit = "mcg"; // "mcg" read as "g"
|
||||
let value = reading.less ? reading.value / 2 : reading.value;
|
||||
if (dv && reading.percent !== null) {
|
||||
const daily = convertMass(dv[0], mass(dv[1]), mass(unit));
|
||||
// %DV is rounded (and may read 0 below 2%), so compare in points.
|
||||
const consistent = (candidate) =>
|
||||
Math.abs((candidate / daily) * 100 - reading.percent) <=
|
||||
Math.max(2, reading.percent * 0.3);
|
||||
if (daily && !consistent(value)) {
|
||||
const candidates = [];
|
||||
// A unit read as a digit: "8g" → "89", "160mg" → "16019".
|
||||
if (!reading.unit && /\d9$/.test(reading.raw))
|
||||
candidates.push(Number(reading.raw.slice(0, -1)));
|
||||
if (!reading.unit && /\d19$/.test(reading.raw))
|
||||
candidates.push(Number(reading.raw.slice(0, -2)));
|
||||
// A lost decimal point: "05" → 0.5, "25" → 2.5.
|
||||
if (/^\d{2,3}$/.test(reading.raw))
|
||||
candidates.push(Number(reading.raw) / 10);
|
||||
const fixed = candidates.find(consistent);
|
||||
if (fixed !== undefined) value = fixed;
|
||||
// Nothing plausible was read; trust the percentage.
|
||||
else if (!reading.unit) value = (reading.percent / 100) * daily;
|
||||
}
|
||||
}
|
||||
const result = toCatalog(key, value, unit);
|
||||
// Far beyond a day's worth in one serving is a misread, not a food.
|
||||
if (dv && result > 10 * toCatalog(key, dv[0], dv[1])) return null;
|
||||
return result;
|
||||
}
|
||||
const mass = (unit) => (unit === "mcg" ? "µg" : unit);
|
||||
|
||||
function percentOnly(key, rest) {
|
||||
const dv = dailyValues[key];
|
||||
const match = /^[^\d]{0,6}(\d{1,4})\s*%/.exec(rest);
|
||||
if (!dv || !match) return null;
|
||||
return toCatalog(key, (Number(match[1]) / 100) * dv[0], dv[1]);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{text: string, x0: number, y0: number, x1: number, y1: number}[]} lines
|
||||
* @returns {{serving?: string, nutrients: Record<string, number>,
|
||||
* extra: Record<string, number>, found: string[], text: string[]}}
|
||||
*/
|
||||
export function parseLabel(lines) {
|
||||
const text = rows(lines);
|
||||
const values = {};
|
||||
const claimed = new Set(); // character spans already used, per row
|
||||
const used = (row, start, end) => {
|
||||
for (let i = start; i < end; i++)
|
||||
if (claimed.has(`${row}:${i}`)) return true;
|
||||
for (let i = start; i < end; i++) claimed.add(`${row}:${i}`);
|
||||
return false;
|
||||
};
|
||||
|
||||
// "Includes 10g Added Sugars" puts the amount before the name.
|
||||
text.forEach((row, index) => {
|
||||
const match =
|
||||
/\bincl(?:udes|\.)?\s*(<\s*)?(\d+(?:\.\d+)?)\s*(g)?\s*(?:of\s*)?added\s*sugars?/i.exec(
|
||||
row,
|
||||
) ||
|
||||
/\b(?:added\s*sugars?|azucares\s*anadidos)[\s:]*(<\s*)?(\d+(?:\.\d+)?)\s*(g)?/i.exec(
|
||||
row,
|
||||
);
|
||||
if (!match || values["added-sugars"] !== undefined) return;
|
||||
used(index, match.index, match.index + match[0].length);
|
||||
const percent = percentPattern.exec(
|
||||
row.slice(match.index + match[0].length),
|
||||
);
|
||||
values["added-sugars"] = resolve("added-sugars", {
|
||||
less: Boolean(match[1]),
|
||||
raw: match[2],
|
||||
value: Number(match[2]),
|
||||
unit: match[3],
|
||||
percent: percent ? Number(percent[1]) : null,
|
||||
});
|
||||
});
|
||||
|
||||
for (const [key, pattern] of fields) {
|
||||
if (values[key] !== undefined) continue;
|
||||
for (
|
||||
let index = 0;
|
||||
index < text.length && values[key] === undefined;
|
||||
index++
|
||||
) {
|
||||
const regex = new RegExp(pattern.source, "gi");
|
||||
for (let match; (match = regex.exec(text[index])); ) {
|
||||
const start = match.index;
|
||||
const end = start + match[0].length;
|
||||
if (used(index, start, end)) continue;
|
||||
let rest = text[index].slice(end);
|
||||
// A linear label can wrap between a name and its amount.
|
||||
if (
|
||||
/^[\s,.:]*$/.test(rest) &&
|
||||
/^<?\s*\.?\d/.test(text[index + 1] || "")
|
||||
)
|
||||
rest = " " + text[index + 1];
|
||||
const reading = readAmount(rest);
|
||||
const value = reading ? resolve(key, reading) : percentOnly(key, rest);
|
||||
if (value === null || value === undefined) continue;
|
||||
values[key] = value;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (values.salt !== undefined) {
|
||||
if (values.sodium === undefined) values.sodium = (values.salt / 2.5) * 1000;
|
||||
delete values.salt;
|
||||
}
|
||||
|
||||
// Calories: the amount follows on the same row, or sits alone on a
|
||||
// neighbouring row when it's printed in a much larger font.
|
||||
for (let index = 0; index < text.length; index++) {
|
||||
const row = text[index];
|
||||
const match =
|
||||
/\bcalories\b(?!\s*from)|\benergy\b|\benerg(?:ia|etico)\b/i.exec(row);
|
||||
if (!match) continue;
|
||||
const rest = row.slice(match.index + match[0].length);
|
||||
const kcal =
|
||||
/(\d+(?:\.\d+)?)\s*kcal/i.exec(rest) ||
|
||||
/^[^\d]{0,24}?(\d{1,4})(?!\s*(?:k?j|%|m?c?g))\b/i.exec(rest);
|
||||
const before = text[index - 1] || "";
|
||||
const after = text[index + 1] || "";
|
||||
const perServing =
|
||||
/amount\s*per\s*serving[:\s]*(\d{1,4})\b(?!\s*(?:m?c?g|%))/i;
|
||||
const alone =
|
||||
// Big type often lands beside "Amount per serving" on the next row.
|
||||
perServing.exec(before) ||
|
||||
perServing.exec(after) ||
|
||||
// A number alone at the end of a neighbouring row.
|
||||
[after, before]
|
||||
.map((neighbour) =>
|
||||
/%|\d\s*m?c?g\b/i.test(neighbour)
|
||||
? null
|
||||
: /(?:^|\s)(\d{1,4})$/.exec(neighbour),
|
||||
)
|
||||
.find(Boolean) ||
|
||||
// A linear label wrapped right after "Calories": "80, Total Fat…".
|
||||
/^(\d{1,4})\b(?!\s*(?:m?c?g|%|\.\d))/.exec(after);
|
||||
const value = Number((kcal || alone)?.[1]);
|
||||
if (Number.isFinite(value) && value <= 5000 && (kcal || alone)) {
|
||||
values["energy-kcal"] = value;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
let serving;
|
||||
const perHundred = text.some((row) => /\bper\s*100\s*(g|ml)\b/i.test(row));
|
||||
for (let index = 0; index < text.length; index++) {
|
||||
const match = /\bserv(?:ing|\.)\s*size\b[\s:]*(.*)$/i.exec(text[index]);
|
||||
if (!match) continue;
|
||||
let size = match[1].trim();
|
||||
if (!/\d/.test(size)) size = `${size} ${text[index + 1] || ""}`.trim();
|
||||
// Two-column layouts can run nutrient rows into the serving text.
|
||||
size = size
|
||||
.split(/\b(?:calories|amount|total|sodium|protein|servings)\b|%/i)[0]
|
||||
.trim()
|
||||
.replace(/\s*\(\s*/g, " (")
|
||||
.replace(/\s*\)\s*/g, ")")
|
||||
.replace(/(\d)\s*(g|ml)\b/gi, "$1 $2")
|
||||
.replace(/\)\s*.*$/, ")")
|
||||
.trim();
|
||||
if (size && /\d/.test(size)) serving = size.slice(0, 100);
|
||||
break;
|
||||
}
|
||||
if (!serving && perHundred) {
|
||||
const ml = text.some((row) => /\bper\s*100\s*ml\b/i.test(row));
|
||||
serving = ml ? "100 ml" : "100 g";
|
||||
}
|
||||
|
||||
const core = {
|
||||
"energy-kcal": "calories",
|
||||
proteins: "protein",
|
||||
carbohydrates: "carbs",
|
||||
fat: "fat",
|
||||
fiber: "fiber",
|
||||
sugars: "sugar",
|
||||
sodium: "sodium",
|
||||
};
|
||||
const nutrients = {};
|
||||
const extra = {};
|
||||
for (const [key, value] of Object.entries(values)) {
|
||||
if (!Number.isFinite(value) || value < 0 || value > 100000) continue;
|
||||
const rounded = Math.round(value * 1000) / 1000;
|
||||
if (core[key]) nutrients[core[key]] = rounded;
|
||||
else extra[key] = rounded;
|
||||
}
|
||||
return {
|
||||
...(serving ? { serving } : {}),
|
||||
nutrients,
|
||||
extra,
|
||||
found: Object.keys(values),
|
||||
text,
|
||||
};
|
||||
}
|
||||
165
server/products/db.js
Normal file
165
server/products/db.js
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
import { DatabaseSync } from "node:sqlite";
|
||||
import { titleCase } from "../../shared/products.js";
|
||||
|
||||
// Each source lives in its own table so a refresh can be built alongside the
|
||||
// old copy and swapped in atomically. The file is a cache: it can be deleted
|
||||
// at any time and is rebuilt on the next import.
|
||||
export const sources = ["usda", "off", "api"];
|
||||
const columns = `gtin TEXT PRIMARY KEY, name TEXT, brand TEXT, serving TEXT,
|
||||
serving_qty REAL, serving_unit TEXT, nutrients TEXT`;
|
||||
|
||||
export function openProducts(filename) {
|
||||
const db = new DatabaseSync(filename);
|
||||
db.exec(`PRAGMA journal_mode=WAL; PRAGMA busy_timeout=15000;
|
||||
CREATE TABLE IF NOT EXISTS imports (source TEXT PRIMARY KEY, version TEXT,
|
||||
imported_at TEXT, rows INTEGER, checked_at TEXT, error TEXT);`);
|
||||
for (const source of ["usda", "off"])
|
||||
db.exec(
|
||||
`CREATE TABLE IF NOT EXISTS ${source}_products (${columns}) WITHOUT ROWID`,
|
||||
);
|
||||
// API answers include misses (nutrients NULL) so repeated scans of an
|
||||
// unknown code don't hit Open Food Facts again.
|
||||
db.exec(
|
||||
`CREATE TABLE IF NOT EXISTS api_products (${columns}, fetched_at INTEGER NOT NULL) WITHOUT ROWID`,
|
||||
);
|
||||
const select = Object.fromEntries(
|
||||
sources.map((source) => [
|
||||
source,
|
||||
db.prepare(`SELECT * FROM ${source}_products WHERE gtin=?`),
|
||||
]),
|
||||
);
|
||||
const toProduct = (row, source) =>
|
||||
row?.nutrients
|
||||
? {
|
||||
gtin: row.gtin,
|
||||
name: source === "usda" ? titleCase(row.name) : row.name,
|
||||
brand: source === "usda" ? titleCase(row.brand) : row.brand,
|
||||
serving: row.serving,
|
||||
servingQty: row.serving_qty,
|
||||
servingUnit: row.serving_unit,
|
||||
nutrients: JSON.parse(row.nutrients),
|
||||
source,
|
||||
}
|
||||
: null;
|
||||
|
||||
/** The best local match, or `undefined` when no source has an answer. */
|
||||
function lookup(
|
||||
gtin,
|
||||
{ apiMaxAge = 30 * 86400000, missMaxAge = 86400000 } = {},
|
||||
) {
|
||||
const found = {};
|
||||
for (const source of sources) found[source] = select[source].get(gtin);
|
||||
const api = found.api;
|
||||
if (
|
||||
api &&
|
||||
Date.now() - api.fetched_at > (api.nutrients ? apiMaxAge : missMaxAge)
|
||||
)
|
||||
found.api = undefined;
|
||||
const usda = toProduct(found.usda, "usda");
|
||||
const off = toProduct(found.off, "off") || toProduct(found.api, "api");
|
||||
// USDA comes from manufacturers' own label submissions, so it wins;
|
||||
// community-entered Open Food Facts fills in what it lacks.
|
||||
const base = usda || off;
|
||||
if (base)
|
||||
return {
|
||||
...base,
|
||||
name: base.name || off?.name,
|
||||
brand: base.brand || off?.brand,
|
||||
};
|
||||
return found.api ? null : undefined;
|
||||
}
|
||||
|
||||
const upsertApi = db.prepare(
|
||||
`INSERT OR REPLACE INTO api_products VALUES (?,?,?,?,?,?,?,?)`,
|
||||
);
|
||||
function remember(gtin, product) {
|
||||
upsertApi.run(
|
||||
gtin,
|
||||
product?.name ?? null,
|
||||
product?.brand ?? null,
|
||||
product?.serving ?? null,
|
||||
product?.servingQty ?? null,
|
||||
product?.servingUnit ?? null,
|
||||
product ? JSON.stringify(product.nutrients) : null,
|
||||
Date.now(),
|
||||
);
|
||||
}
|
||||
|
||||
/** Bulk loader used by the importers: writes `<source>_products_next`. */
|
||||
function beginImport(source) {
|
||||
db.exec(`DROP TABLE IF EXISTS ${source}_products_next;
|
||||
CREATE TABLE ${source}_products_next (${columns}) WITHOUT ROWID;`);
|
||||
const insert = db.prepare(
|
||||
`INSERT OR REPLACE INTO ${source}_products_next VALUES (?,?,?,?,?,?,?)`,
|
||||
);
|
||||
let pending = 0;
|
||||
let rows = 0;
|
||||
db.exec("BEGIN");
|
||||
return {
|
||||
add(product) {
|
||||
insert.run(
|
||||
product.gtin,
|
||||
product.name || null,
|
||||
product.brand || null,
|
||||
product.serving || null,
|
||||
product.servingQty ?? null,
|
||||
product.servingUnit || null,
|
||||
JSON.stringify(product.nutrients),
|
||||
);
|
||||
rows++;
|
||||
if (++pending >= 20000) {
|
||||
db.exec("COMMIT; BEGIN");
|
||||
pending = 0;
|
||||
}
|
||||
},
|
||||
commit(version) {
|
||||
db.exec(`COMMIT; BEGIN IMMEDIATE;
|
||||
DROP TABLE IF EXISTS ${source}_products;
|
||||
ALTER TABLE ${source}_products_next RENAME TO ${source}_products;`);
|
||||
db.prepare(
|
||||
`INSERT OR REPLACE INTO imports(source,version,imported_at,rows,checked_at,error)
|
||||
VALUES (?,?,?,?,?,NULL)`,
|
||||
).run(
|
||||
source,
|
||||
version,
|
||||
new Date().toISOString(),
|
||||
rows,
|
||||
new Date().toISOString(),
|
||||
);
|
||||
db.exec("COMMIT");
|
||||
// Old pages are reused by the next import; reclaiming them would
|
||||
// rewrite the whole file.
|
||||
db.exec("PRAGMA wal_checkpoint(TRUNCATE)");
|
||||
return rows;
|
||||
},
|
||||
abort() {
|
||||
if (db.isTransaction) db.exec("ROLLBACK");
|
||||
db.exec(`DROP TABLE IF EXISTS ${source}_products_next`);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const state = (source) =>
|
||||
db.prepare("SELECT * FROM imports WHERE source=?").get(source);
|
||||
function recordCheck(source, error = null) {
|
||||
db.prepare(
|
||||
`INSERT INTO imports(source,checked_at,error) VALUES (?,?,?)
|
||||
ON CONFLICT(source) DO UPDATE SET checked_at=excluded.checked_at, error=excluded.error`,
|
||||
).run(source, new Date().toISOString(), error);
|
||||
}
|
||||
const status = () =>
|
||||
db
|
||||
.prepare("SELECT source, version, imported_at, rows, error FROM imports")
|
||||
.all();
|
||||
|
||||
return {
|
||||
db,
|
||||
lookup,
|
||||
remember,
|
||||
beginImport,
|
||||
state,
|
||||
recordCheck,
|
||||
status,
|
||||
close: () => db.close(),
|
||||
};
|
||||
}
|
||||
94
server/products/import.js
Normal file
94
server/products/import.js
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
// Refreshes the local product databases. Runs as a low-priority child process
|
||||
// (see scheduler.js) so parsing several gigabytes never stalls the app.
|
||||
// node server/products/import.js <data dir> [--force]
|
||||
import { mkdirSync, rmSync } from "node:fs";
|
||||
import { setPriority } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { openProducts } from "./db.js";
|
||||
import { OFF_CSV, importOffCsv, userAgent } from "./off.js";
|
||||
import { download, importUsdaZip, latestUsdaArchive } from "./usda.js";
|
||||
|
||||
const dataDir = resolve(process.argv[2] || "data");
|
||||
const force = process.argv.includes("--force");
|
||||
const OFF_MAX_AGE = 7 * 86400000;
|
||||
const log = (...args) => console.log("[products]", ...args);
|
||||
try {
|
||||
setPriority(19);
|
||||
} catch {
|
||||
// Lowering priority is best effort.
|
||||
}
|
||||
const products = openProducts(join(dataDir, "products.sqlite"));
|
||||
const tmp = join(dataDir, "tmp");
|
||||
mkdirSync(tmp, { recursive: true });
|
||||
|
||||
async function refreshUsda() {
|
||||
const latest = await latestUsdaArchive();
|
||||
if (!force && products.state("usda")?.version === latest.version)
|
||||
return products.recordCheck("usda");
|
||||
const file = join(tmp, "usda-branded.zip");
|
||||
log(`downloading USDA Branded Foods ${latest.version}`);
|
||||
try {
|
||||
await download(latest.url, file);
|
||||
const sink = products.beginImport("usda");
|
||||
try {
|
||||
await importUsdaZip(file, sink, {
|
||||
onProgress: (n) => log(`USDA: ${n.toLocaleString()} products`),
|
||||
});
|
||||
log(
|
||||
`USDA: imported ${sink.commit(latest.version).toLocaleString()} products`,
|
||||
);
|
||||
} catch (error) {
|
||||
sink.abort();
|
||||
throw error;
|
||||
}
|
||||
} finally {
|
||||
rmSync(file, { force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshOff() {
|
||||
const state = products.state("off");
|
||||
if (
|
||||
!force &&
|
||||
state?.imported_at &&
|
||||
Date.now() - Date.parse(state.imported_at) < OFF_MAX_AGE
|
||||
)
|
||||
return;
|
||||
const response = await fetch(OFF_CSV, {
|
||||
headers: { "User-Agent": userAgent },
|
||||
});
|
||||
if (!response.ok)
|
||||
throw new Error(`Open Food Facts answered ${response.status}`);
|
||||
const version =
|
||||
response.headers.get("last-modified") || new Date().toUTCString();
|
||||
if (!force && state?.version === version) {
|
||||
await response.body?.cancel();
|
||||
return products.recordCheck("off");
|
||||
}
|
||||
log("streaming the Open Food Facts export");
|
||||
const sink = products.beginImport("off");
|
||||
try {
|
||||
await importOffCsv(response, sink, {
|
||||
onProgress: (n) => log(`Open Food Facts: ${n.toLocaleString()} products`),
|
||||
});
|
||||
log(
|
||||
`Open Food Facts: imported ${sink.commit(version).toLocaleString()} products`,
|
||||
);
|
||||
} catch (error) {
|
||||
sink.abort();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
for (const [source, refresh] of [
|
||||
["usda", refreshUsda],
|
||||
["off", refreshOff],
|
||||
]) {
|
||||
try {
|
||||
await refresh();
|
||||
} catch (error) {
|
||||
console.error(`[products] ${source} refresh failed:`, error.message);
|
||||
products.recordCheck(source, String(error.message).slice(0, 300));
|
||||
}
|
||||
}
|
||||
products.close();
|
||||
175
server/products/off.js
Normal file
175
server/products/off.js
Normal file
|
|
@ -0,0 +1,175 @@
|
|||
import { createGunzip } from "node:zlib";
|
||||
import { Readable } from "node:stream";
|
||||
import { databaseGtin } from "../../shared/barcode.js";
|
||||
import { fromGrams, nutrientKeyPattern } from "../../shared/nutrients.js";
|
||||
|
||||
export const OFF_CSV =
|
||||
"https://static.openfoodfacts.org/data/en.openfoodfacts.org.products.csv.gz";
|
||||
export const userAgent =
|
||||
"Calio/0.1 (self-hosted personal nutrition journal; https://git.elijahkuntz.com/Elijah/Calio)";
|
||||
|
||||
// Columns that are not nutrient amounts, or duplicate one handled separately.
|
||||
const skipped = new Set([
|
||||
"energy",
|
||||
"energy-kj",
|
||||
"energy-from-fat",
|
||||
"salt",
|
||||
"added-salt",
|
||||
"ph",
|
||||
"alcohol", // % vol in Open Food Facts, not grams
|
||||
"cocoa",
|
||||
"chlorophyl",
|
||||
"collagen-meat-protein-ratio",
|
||||
"glycemic-index",
|
||||
"water-hardness",
|
||||
"acidity",
|
||||
"carbohydrates-total",
|
||||
]);
|
||||
const notNutrient = /score|estimate|footprint|fruits-vegetables|nova|prepared/;
|
||||
|
||||
/**
|
||||
* Converts Open Food Facts per-100 g values (grams, kcal) into Calio's
|
||||
* stored units.
|
||||
* @param {Iterable<[string, unknown]>} entries `[key, value]` without `_100g`
|
||||
*/
|
||||
export function offNutrients(entries) {
|
||||
const raw = new Map();
|
||||
for (const [key, value] of entries) {
|
||||
const number = typeof value === "number" ? value : Number(value);
|
||||
if (value === "" || value == null || !Number.isFinite(number) || number < 0)
|
||||
continue;
|
||||
raw.set(key, number);
|
||||
}
|
||||
const nutrients = {};
|
||||
let kcal = raw.get("energy-kcal");
|
||||
if (kcal === undefined) {
|
||||
const kj = raw.get("energy-kj") ?? raw.get("energy");
|
||||
if (kj !== undefined) kcal = kj / 4.184;
|
||||
}
|
||||
if (kcal !== undefined && kcal <= 950) nutrients["energy-kcal"] = kcal;
|
||||
let sodium = raw.get("sodium");
|
||||
if (sodium === undefined && raw.has("salt")) sodium = raw.get("salt") / 2.5;
|
||||
if (sodium !== undefined && sodium <= 100) nutrients.sodium = sodium * 1000;
|
||||
for (const [key, grams] of raw) {
|
||||
if (
|
||||
key === "energy-kcal" ||
|
||||
key === "sodium" ||
|
||||
skipped.has(key) ||
|
||||
notNutrient.test(key) ||
|
||||
!nutrientKeyPattern.test(key) ||
|
||||
grams > 100 // more than 100 g per 100 g is a data-entry error
|
||||
)
|
||||
continue;
|
||||
const value = fromGrams(key, grams);
|
||||
if (value !== null) nutrients[key] = value;
|
||||
}
|
||||
return nutrients;
|
||||
}
|
||||
|
||||
function servingUnit(text) {
|
||||
return /\d\s*(ml|millilit|fl\.?\s*oz|cl\b|l\b)/i.test(text || "")
|
||||
? "ml"
|
||||
: "g";
|
||||
}
|
||||
|
||||
/** Shapes a CSV row or API product into a products-table row. */
|
||||
export function offProduct({
|
||||
code,
|
||||
name,
|
||||
brands,
|
||||
serving,
|
||||
servingQty,
|
||||
nutrients,
|
||||
}) {
|
||||
const gtin = databaseGtin(code);
|
||||
if (!gtin || nutrients["energy-kcal"] === undefined) return null;
|
||||
const qty = Number(servingQty);
|
||||
return {
|
||||
gtin,
|
||||
name: (name || "").trim().slice(0, 200) || null,
|
||||
brand: (brands || "").split(",")[0].trim().slice(0, 100) || null,
|
||||
serving: (serving || "").trim().slice(0, 100) || null,
|
||||
servingQty: Number.isFinite(qty) && qty > 0 ? qty : null,
|
||||
servingUnit: servingUnit(serving),
|
||||
nutrients,
|
||||
};
|
||||
}
|
||||
|
||||
async function* lines(stream) {
|
||||
let rest = "";
|
||||
stream.setEncoding("utf8");
|
||||
for await (const chunk of stream) {
|
||||
const parts = (rest + chunk).split("\n");
|
||||
rest = parts.pop();
|
||||
yield* parts;
|
||||
}
|
||||
if (rest) yield rest;
|
||||
}
|
||||
|
||||
/** Streams the daily CSV export (~1.5 GB gzipped) straight into `sink`. */
|
||||
export async function importOffCsv(response, sink, { onProgress } = {}) {
|
||||
const gunzip = Readable.fromWeb(response.body).pipe(createGunzip());
|
||||
let header;
|
||||
let count = 0;
|
||||
for await (const line of lines(gunzip)) {
|
||||
const fields = line.split("\t");
|
||||
if (!header) {
|
||||
header = Object.fromEntries(fields.map((name, i) => [name, i]));
|
||||
for (const column of ["code", "product_name", "energy-kcal_100g"])
|
||||
if (!(column in header))
|
||||
throw new Error(`Open Food Facts export is missing ${column}`);
|
||||
header.nutrientColumns = fields
|
||||
.map((name, i) => [name, i])
|
||||
.filter(([name]) => name.endsWith("_100g"))
|
||||
.map(([name, i]) => [name.slice(0, -5), i]);
|
||||
continue;
|
||||
}
|
||||
if (fields[header.no_nutrition_data] === "on") continue;
|
||||
const product = offProduct({
|
||||
code: fields[header.code],
|
||||
name: fields[header.product_name],
|
||||
brands: fields[header.brands],
|
||||
serving: fields[header.serving_size],
|
||||
servingQty: fields[header.serving_quantity],
|
||||
nutrients: offNutrients(
|
||||
header.nutrientColumns.map(([key, i]) => [key, fields[i]]),
|
||||
),
|
||||
});
|
||||
if (!product) continue;
|
||||
sink.add(product);
|
||||
if (++count % 250000 === 0) onProgress?.(count);
|
||||
}
|
||||
if (!header) throw new Error("Open Food Facts export was empty");
|
||||
return count;
|
||||
}
|
||||
|
||||
const apiFields =
|
||||
"code,product_name,product_name_en,brands,serving_size,serving_quantity,nutriments";
|
||||
/** Live lookup, used only when no local copy has the product. */
|
||||
export async function fetchOffProduct(gtin, fetchImpl = fetch) {
|
||||
// Open Food Facts keys products by EAN-13 (or EAN-8).
|
||||
const stripped = gtin.replace(/^0+/, "");
|
||||
const code = stripped.padStart(stripped.length <= 8 ? 8 : 13, "0");
|
||||
const response = await fetchImpl(
|
||||
`https://world.openfoodfacts.org/api/v2/product/${code}.json?fields=${apiFields}`,
|
||||
{ headers: { "User-Agent": userAgent }, signal: AbortSignal.timeout(8000) },
|
||||
);
|
||||
if (response.status === 404) return null;
|
||||
if (!response.ok)
|
||||
throw new Error(`Open Food Facts answered ${response.status}`);
|
||||
const data = await response.json();
|
||||
if (data.status !== 1 || !data.product) return null;
|
||||
const p = data.product;
|
||||
const entries = Object.entries(p.nutriments || {})
|
||||
.filter(([key]) => key.endsWith("_100g"))
|
||||
.map(([key, value]) => [key.slice(0, -5), value]);
|
||||
const product = offProduct({
|
||||
code: gtin,
|
||||
name: p.product_name_en || p.product_name,
|
||||
brands: p.brands,
|
||||
serving: p.serving_size,
|
||||
servingQty: p.serving_quantity,
|
||||
nutrients: offNutrients(entries),
|
||||
});
|
||||
return product && { ...product, source: "api" };
|
||||
}
|
||||
27
server/products/scheduler.js
Normal file
27
server/products/scheduler.js
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
import { fork } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const script = fileURLToPath(new URL("./import.js", import.meta.url));
|
||||
const HOUR = 3600000;
|
||||
|
||||
/**
|
||||
* Checks for new product data shortly after start-up and every six hours.
|
||||
* The import itself decides whether anything needs downloading.
|
||||
*/
|
||||
export function scheduleProductImports(dataDir, { firstDelay = 60000 } = {}) {
|
||||
let child = null;
|
||||
const run = () => {
|
||||
if (child) return;
|
||||
child = fork(script, [dataDir], {
|
||||
execArgv: ["--max-old-space-size=1024"],
|
||||
stdio: "inherit",
|
||||
});
|
||||
child.on("exit", () => (child = null));
|
||||
};
|
||||
const timers = [setTimeout(run, firstDelay), setInterval(run, 6 * HOUR)];
|
||||
for (const timer of timers) timer.unref();
|
||||
return () => {
|
||||
for (const timer of timers) clearInterval(timer);
|
||||
child?.kill();
|
||||
};
|
||||
}
|
||||
398
server/products/usda.js
Normal file
398
server/products/usda.js
Normal file
|
|
@ -0,0 +1,398 @@
|
|||
import { createWriteStream } from "node:fs";
|
||||
import { Readable } from "node:stream";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import yauzl from "yauzl";
|
||||
import { databaseGtin } from "../../shared/barcode.js";
|
||||
import { convertMass, nutrientUnit } from "../../shared/nutrients.js";
|
||||
|
||||
export const USDA_PAGE = "https://fdc.nal.usda.gov/download-datasets";
|
||||
const archiveName =
|
||||
/FoodData_Central_branded_food_csv_(\d{4}-\d{2}-\d{2})\.zip/g;
|
||||
|
||||
/** The newest Branded Foods CSV archive linked from the download page. */
|
||||
export async function latestUsdaArchive(fetchImpl = fetch) {
|
||||
const response = await fetchImpl(USDA_PAGE, {
|
||||
signal: AbortSignal.timeout(30000),
|
||||
});
|
||||
if (!response.ok) throw new Error(`USDA answered ${response.status}`);
|
||||
const versions = [...(await response.text()).matchAll(archiveName)].map(
|
||||
(match) => match[1],
|
||||
);
|
||||
if (!versions.length) throw new Error("No USDA branded archive is listed");
|
||||
const version = versions.sort().at(-1);
|
||||
return {
|
||||
version,
|
||||
url: `https://fdc.nal.usda.gov/fdc-datasets/FoodData_Central_branded_food_csv_${version}.zip`,
|
||||
};
|
||||
}
|
||||
|
||||
export async function download(url, file, fetchImpl = fetch) {
|
||||
const response = await fetchImpl(url);
|
||||
if (!response.ok) throw new Error(`Download failed: ${response.status}`);
|
||||
await pipeline(Readable.fromWeb(response.body), createWriteStream(file));
|
||||
}
|
||||
|
||||
// FoodData Central nutrient IDs mapped to Calio keys. Unlisted mass nutrients
|
||||
// are kept under a slug of their USDA name.
|
||||
const nutrientIds = {
|
||||
1008: "energy-kcal",
|
||||
1003: "proteins",
|
||||
1004: "fat",
|
||||
1005: "carbohydrates",
|
||||
1079: "fiber",
|
||||
2000: "sugars",
|
||||
1093: "sodium",
|
||||
1258: "saturated-fat",
|
||||
1257: "trans-fat",
|
||||
1292: "monounsaturated-fat",
|
||||
1293: "polyunsaturated-fat",
|
||||
1253: "cholesterol",
|
||||
1235: "added-sugars",
|
||||
1086: "polyols",
|
||||
1082: "soluble-fiber",
|
||||
1084: "insoluble-fiber",
|
||||
1009: "starch",
|
||||
1010: "sucrose",
|
||||
1011: "glucose",
|
||||
1012: "fructose",
|
||||
1013: "lactose",
|
||||
1014: "maltose",
|
||||
1075: "galactose",
|
||||
1056: "sorbitol",
|
||||
1078: "xylitol",
|
||||
1055: "mannitol",
|
||||
1051: "water",
|
||||
1018: "alcohol",
|
||||
1057: "caffeine",
|
||||
1234: "taurine",
|
||||
1068: "beta-glucan",
|
||||
1181: "inositol",
|
||||
1106: "vitamin-a",
|
||||
1104: "vitamin-a-iu",
|
||||
1162: "vitamin-c",
|
||||
1114: "vitamin-d",
|
||||
1109: "vitamin-e",
|
||||
1185: "vitamin-k",
|
||||
1165: "vitamin-b1",
|
||||
1166: "vitamin-b2",
|
||||
1167: "vitamin-pp",
|
||||
1175: "vitamin-b6",
|
||||
1190: "vitamin-b9",
|
||||
1177: "folates",
|
||||
1186: "folic-acid",
|
||||
1178: "vitamin-b12",
|
||||
1176: "biotin",
|
||||
1170: "pantothenic-acid",
|
||||
1180: "choline",
|
||||
1087: "calcium",
|
||||
1089: "iron",
|
||||
1092: "potassium",
|
||||
1090: "magnesium",
|
||||
1091: "phosphorus",
|
||||
1095: "zinc",
|
||||
1098: "copper",
|
||||
1101: "manganese",
|
||||
1103: "selenium",
|
||||
1100: "iodine",
|
||||
1096: "chromium",
|
||||
1102: "molybdenum",
|
||||
1088: "chloride",
|
||||
1099: "fluoride",
|
||||
};
|
||||
// Fallbacks used only when the preferred ID is absent.
|
||||
const fallbacks = {
|
||||
1062: ["energy-kcal", (kj) => kj / 4.184], // Energy, kJ
|
||||
1085: ["fat", (g) => g], // Total fat (NLEA)
|
||||
1063: ["sugars", (g) => g], // Sugars, Total
|
||||
1110: ["vitamin-d", (iu) => iu * 0.025], // Vitamin D, IU
|
||||
};
|
||||
const usdaUnits = { G: "g", MG: "mg", UG: "µg", KCAL: "kcal", IU: "IU" };
|
||||
const slug = (name) =>
|
||||
name
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-|-$/g, "")
|
||||
.slice(0, 48);
|
||||
|
||||
/** Builds a converter from a nutrient row (id, amount) to [key, value]. */
|
||||
export function nutrientMapper(nutrientRows) {
|
||||
const map = new Map();
|
||||
for (const { id, name, unit } of nutrientRows) {
|
||||
const from = usdaUnits[unit];
|
||||
let key = nutrientIds[id];
|
||||
if (fallbacks[id]) {
|
||||
const [target, convert] = fallbacks[id];
|
||||
map.set(id, { key: target, convert, fallback: true });
|
||||
continue;
|
||||
}
|
||||
if (!key) {
|
||||
if (!["g", "mg", "µg"].includes(from) || /do not use/i.test(name))
|
||||
continue;
|
||||
key = slug(name);
|
||||
if (!key) continue;
|
||||
}
|
||||
const to = nutrientUnit(key);
|
||||
const convert =
|
||||
from === to ? (value) => value : (value) => convertMass(value, from, to);
|
||||
if (from !== to && convertMass(1, from, to) === null) continue;
|
||||
map.set(id, { key, convert, fallback: false });
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
// Parses one record starting at `pos`; null when `text` ends mid-record.
|
||||
function parseRecord(text, pos) {
|
||||
const fields = [];
|
||||
for (;;) {
|
||||
if (pos >= text.length) return null;
|
||||
if (text[pos] === '"') {
|
||||
let value = "";
|
||||
let i = pos + 1;
|
||||
for (;;) {
|
||||
const quote = text.indexOf('"', i);
|
||||
if (quote === -1 || quote + 1 >= text.length) return null;
|
||||
value += text.slice(i, quote);
|
||||
if (text[quote + 1] !== '"') {
|
||||
pos = quote + 1;
|
||||
break;
|
||||
}
|
||||
value += '"'; // "" inside a quoted field
|
||||
i = quote + 2;
|
||||
}
|
||||
fields.push(value);
|
||||
} else {
|
||||
let end = pos;
|
||||
while (end < text.length && text[end] !== "," && text[end] !== "\n")
|
||||
end++;
|
||||
if (end >= text.length) return null;
|
||||
fields.push(text.slice(pos, end).replace(/\r$/, ""));
|
||||
pos = end;
|
||||
}
|
||||
const c = text[pos];
|
||||
if (c === ",") pos++;
|
||||
else if (c === "\n") return { fields, next: pos + 1 };
|
||||
else if (c === "\r" && text[pos + 1] === "\n")
|
||||
return { fields, next: pos + 2 };
|
||||
else if (pos + 1 >= text.length) return null;
|
||||
else pos++; // stray character after a closing quote
|
||||
}
|
||||
}
|
||||
|
||||
/** Minimal RFC 4180 reader: yields one array of fields per record. */
|
||||
export async function* csvRecords(stream) {
|
||||
stream.setEncoding("utf8");
|
||||
let buffer = "";
|
||||
for await (const chunk of stream) {
|
||||
buffer += chunk;
|
||||
let start = 0;
|
||||
for (let record; (record = parseRecord(buffer, start)); start = record.next)
|
||||
yield record.fields;
|
||||
buffer = buffer.slice(start);
|
||||
}
|
||||
if (buffer.trim()) {
|
||||
const record = parseRecord(buffer + "\n", 0);
|
||||
if (record) yield record.fields;
|
||||
}
|
||||
}
|
||||
|
||||
function openZip(file) {
|
||||
return new Promise((resolve, reject) =>
|
||||
yauzl.open(file, { lazyEntries: false, autoClose: false }, (error, zip) => {
|
||||
if (error) return reject(error);
|
||||
const entries = new Map();
|
||||
zip.on("entry", (entry) =>
|
||||
entries.set(entry.fileName.split("/").at(-1), entry),
|
||||
);
|
||||
zip.on("end", () => resolve({ zip, entries }));
|
||||
zip.on("error", reject);
|
||||
}),
|
||||
);
|
||||
}
|
||||
// Yields the requested columns of each row, in the order asked for.
|
||||
async function* table(zip, entries, name, columns) {
|
||||
const entry = entries.get(name);
|
||||
if (!entry) throw new Error(`USDA archive is missing ${name}`);
|
||||
const stream = await new Promise((resolve, reject) =>
|
||||
zip.openReadStream(entry, (error, s) =>
|
||||
error ? reject(error) : resolve(s),
|
||||
),
|
||||
);
|
||||
let indexes;
|
||||
for await (const fields of csvRecords(stream)) {
|
||||
if (!indexes) {
|
||||
indexes = columns.map((column) => fields.indexOf(column));
|
||||
const missing = columns.filter((_, i) => indexes[i] < 0);
|
||||
if (missing.length)
|
||||
throw new Error(`USDA ${name} is missing ${missing.join(", ")}`);
|
||||
continue;
|
||||
}
|
||||
yield indexes.map((i) => fields[i]);
|
||||
}
|
||||
}
|
||||
|
||||
const servingUnits = { g: "g", grm: "g", ml: "ml", mlt: "ml" };
|
||||
function servingText(household, qty, unit) {
|
||||
const amount = `${+qty.toFixed(1)} ${unit}`;
|
||||
if (!household || /^(amount|per) /i.test(household) || !/\d/.test(household))
|
||||
return amount;
|
||||
// Some submissions already include the weight: "3/4 cup (20g)".
|
||||
return /\d\s*(g|ml|grams?)\b/i.test(household)
|
||||
? household
|
||||
: `${household} (${amount})`;
|
||||
}
|
||||
|
||||
/** Reads a Branded Foods CSV archive into `sink`. */
|
||||
export async function importUsdaZip(file, sink, { onProgress } = {}) {
|
||||
const { zip, entries } = await openZip(file);
|
||||
try {
|
||||
const nutrientRows = [];
|
||||
for await (const [id, name, unit] of table(zip, entries, "nutrient.csv", [
|
||||
"id",
|
||||
"name",
|
||||
"unit_name",
|
||||
]))
|
||||
nutrientRows.push({ id: Number(id), name, unit });
|
||||
const mapper = nutrientMapper(nutrientRows);
|
||||
|
||||
// A GTIN can appear several times as a product is resubmitted; keep the
|
||||
// newest submission.
|
||||
const byGtin = new Map();
|
||||
const strings = new Map();
|
||||
// Substrings keep their whole parse buffer alive; store fresh copies,
|
||||
// shared across the many products with the same brand or serving.
|
||||
const keep = (text) => {
|
||||
if (!text) return "";
|
||||
let copy = strings.get(text);
|
||||
if (copy === undefined) {
|
||||
copy = Buffer.from(text).toString();
|
||||
strings.set(copy, copy);
|
||||
}
|
||||
return copy;
|
||||
};
|
||||
for await (const [
|
||||
id,
|
||||
gtinUpc,
|
||||
brandName,
|
||||
brandOwner,
|
||||
servingSize,
|
||||
servingSizeUnit,
|
||||
householdText,
|
||||
available,
|
||||
modified,
|
||||
] of table(zip, entries, "branded_food.csv", [
|
||||
"fdc_id",
|
||||
"gtin_upc",
|
||||
"brand_name",
|
||||
"brand_owner",
|
||||
"serving_size",
|
||||
"serving_size_unit",
|
||||
"household_serving_fulltext",
|
||||
"available_date",
|
||||
"modified_date",
|
||||
])) {
|
||||
const gtin = databaseGtin(gtinUpc);
|
||||
if (!gtin) continue;
|
||||
const date = available || modified || "";
|
||||
const fdcId = Number(id);
|
||||
const current = byGtin.get(gtin);
|
||||
if (
|
||||
current &&
|
||||
(current.date > date ||
|
||||
(current.date === date && current.fdcId > fdcId))
|
||||
)
|
||||
continue;
|
||||
const qty = Number(servingSize);
|
||||
const household = (householdText || "").trim();
|
||||
const unit = servingUnits[(servingSizeUnit || "").toLowerCase()];
|
||||
byGtin.set(keep(gtin), {
|
||||
fdcId,
|
||||
date: keep(date),
|
||||
brand: keep((brandName || brandOwner || "").trim().slice(0, 100)),
|
||||
servingQty: unit && qty > 0 ? qty : null,
|
||||
servingUnit: unit || "g",
|
||||
serving:
|
||||
unit && qty > 0 ? keep(servingText(household, qty, unit)) : null,
|
||||
name: "",
|
||||
gtin: "",
|
||||
});
|
||||
}
|
||||
strings.clear();
|
||||
const byFdc = new Map();
|
||||
for (const [gtin, product] of byGtin) {
|
||||
product.gtin = gtin;
|
||||
byFdc.set(product.fdcId, product);
|
||||
}
|
||||
byGtin.clear();
|
||||
|
||||
for await (const [id, description] of table(zip, entries, "food.csv", [
|
||||
"fdc_id",
|
||||
"description",
|
||||
])) {
|
||||
const product = byFdc.get(Number(id));
|
||||
if (product)
|
||||
product.name = Buffer.from(description.slice(0, 200)).toString();
|
||||
}
|
||||
|
||||
let count = 0;
|
||||
const finish = (product, values) => {
|
||||
const nutrients = {};
|
||||
const late = [];
|
||||
for (let i = 0; i < values.length; i += 2) {
|
||||
const mapping = mapper.get(values[i]);
|
||||
const number = values[i + 1];
|
||||
if (!mapping || !Number.isFinite(number) || number < 0) continue;
|
||||
if (mapping.fallback) late.push([mapping, number]);
|
||||
else {
|
||||
const value = mapping.convert(number);
|
||||
if (value !== null) nutrients[mapping.key] = value;
|
||||
}
|
||||
}
|
||||
for (const [mapping, number] of late)
|
||||
if (nutrients[mapping.key] === undefined)
|
||||
nutrients[mapping.key] = mapping.convert(number);
|
||||
if (
|
||||
nutrients["energy-kcal"] === undefined ||
|
||||
nutrients["energy-kcal"] > 950
|
||||
)
|
||||
return;
|
||||
sink.add({
|
||||
gtin: product.gtin,
|
||||
name: product.name,
|
||||
brand: product.brand,
|
||||
serving: product.serving,
|
||||
servingQty: product.servingQty,
|
||||
servingUnit: product.servingUnit,
|
||||
nutrients,
|
||||
});
|
||||
if (++count % 250000 === 0) onProgress?.(count);
|
||||
};
|
||||
// ~30 million rows, grouped by food: each product is finished as soon as
|
||||
// the next one starts, so only one food's nutrients are held at a time.
|
||||
let currentId = null;
|
||||
let values = [];
|
||||
for await (const [id, nutrientId, amount] of table(
|
||||
zip,
|
||||
entries,
|
||||
"food_nutrient.csv",
|
||||
["fdc_id", "nutrient_id", "amount"],
|
||||
)) {
|
||||
const fdcId = Number(id);
|
||||
if (fdcId !== currentId) {
|
||||
const done = byFdc.get(currentId);
|
||||
if (done) {
|
||||
finish(done, values);
|
||||
byFdc.delete(currentId);
|
||||
}
|
||||
currentId = fdcId;
|
||||
values = [];
|
||||
}
|
||||
if (byFdc.has(fdcId) && amount !== "")
|
||||
values.push(Number(nutrientId), Number(amount));
|
||||
}
|
||||
const last = byFdc.get(currentId);
|
||||
if (last) finish(last, values);
|
||||
return count;
|
||||
} finally {
|
||||
zip.close();
|
||||
}
|
||||
}
|
||||
65
shared/barcode.js
Normal file
65
shared/barcode.js
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
// Barcodes are stored as 14-digit GTINs so UPC-A, EAN-13 and EAN-8 forms of
|
||||
// the same product always match.
|
||||
|
||||
export function checkDigitValid(digits) {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < digits.length - 1; i++)
|
||||
sum += Number(digits[digits.length - 2 - i]) * (i % 2 === 0 ? 3 : 1);
|
||||
return (10 - (sum % 10)) % 10 === Number(digits[digits.length - 1]);
|
||||
}
|
||||
|
||||
/** Expands an 8-digit UPC-E code (number system, 6 digits, check) to UPC-A. */
|
||||
export function expandUpcE(code) {
|
||||
if (!/^[01]\d{7}$/.test(code)) return null;
|
||||
const [s, d1, d2, d3, d4, d5, d6, check] = code;
|
||||
const body =
|
||||
d6 <= "2"
|
||||
? `${d1}${d2}${d6}0000${d3}${d4}${d5}`
|
||||
: d6 === "3"
|
||||
? `${d1}${d2}${d3}00000${d4}${d5}`
|
||||
: d6 === "4"
|
||||
? `${d1}${d2}${d3}${d4}00000${d5}`
|
||||
: `${d1}${d2}${d3}${d4}${d5}0000${d6}`;
|
||||
return s + body + check;
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalizes a scanned or typed barcode to GTIN-14, or returns null when it
|
||||
* isn't a valid retail product code.
|
||||
* @param {string} raw
|
||||
* @param {string} [format] BarcodeDetector format, e.g. "upc_e"
|
||||
*/
|
||||
export function normalizeBarcode(raw, format) {
|
||||
let digits = String(raw).replace(/\D/g, "");
|
||||
if (format === "upc_e" || (digits.length === 6 && format === undefined)) {
|
||||
if (digits.length === 6) {
|
||||
// Six printed digits omit the number system (0) and check digit.
|
||||
for (let check = 0; check < 10; check++) {
|
||||
const upcA = expandUpcE("0" + digits + check);
|
||||
if (upcA && checkDigitValid(upcA)) {
|
||||
digits = upcA;
|
||||
break;
|
||||
}
|
||||
}
|
||||
} else digits = expandUpcE(digits) || "";
|
||||
}
|
||||
if (![8, 12, 13, 14].includes(digits.length) || !checkDigitValid(digits))
|
||||
return null;
|
||||
return digits.padStart(14, "0");
|
||||
}
|
||||
|
||||
/** Pads any 8–14 digit database code to GTIN-14 without checksum validation. */
|
||||
export function databaseGtin(code) {
|
||||
const digits = String(code).trim();
|
||||
if (!/^\d{8,14}$/.test(digits)) return null;
|
||||
const gtin = digits.padStart(14, "0");
|
||||
return /^0+$/.test(gtin) ? null : gtin;
|
||||
}
|
||||
|
||||
/** The shortest standard form: EAN-8, UPC-A or EAN-13/GTIN-14. */
|
||||
export function displayBarcode(gtin) {
|
||||
if (gtin.startsWith("000000")) return gtin.slice(6);
|
||||
if (gtin.startsWith("00")) return gtin.slice(2);
|
||||
if (gtin.startsWith("0")) return gtin.slice(1);
|
||||
return gtin;
|
||||
}
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
import { z } from "zod";
|
||||
import { nutrientKeyPattern } from "./nutrients.js";
|
||||
|
||||
export const metricKeys = [
|
||||
"calories",
|
||||
|
|
@ -27,6 +28,15 @@ const food = z.object({
|
|||
name,
|
||||
serving: z.string().trim().min(1).max(100),
|
||||
nutrients: nutrientsSchema,
|
||||
barcode: z
|
||||
.string()
|
||||
.regex(/^\d{14}$/)
|
||||
.optional(),
|
||||
// Further nutrients per serving, keyed as in shared/nutrients.js.
|
||||
extra: z
|
||||
.record(z.string().regex(nutrientKeyPattern), amount)
|
||||
.refine((value) => Object.keys(value).length <= 200, "Too many nutrients")
|
||||
.optional(),
|
||||
});
|
||||
const entry = food
|
||||
.extend({
|
||||
|
|
|
|||
115
shared/nutrients.js
Normal file
115
shared/nutrients.js
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
// Nutrients beyond the seven tracked metrics. Keys follow Open Food Facts
|
||||
// naming, so both product databases and label scans share one vocabulary.
|
||||
// Amounts are stored in the listed unit; keys not listed here are stored in
|
||||
// grams.
|
||||
export const nutrientCatalog = [
|
||||
{ key: "saturated-fat", label: "Saturated fat", unit: "g" },
|
||||
{ key: "trans-fat", label: "Trans fat", unit: "g" },
|
||||
{ key: "monounsaturated-fat", label: "Monounsaturated fat", unit: "g" },
|
||||
{ key: "polyunsaturated-fat", label: "Polyunsaturated fat", unit: "g" },
|
||||
{ key: "omega-3-fat", label: "Omega-3 fat", unit: "g" },
|
||||
{ key: "omega-6-fat", label: "Omega-6 fat", unit: "g" },
|
||||
{ key: "omega-9-fat", label: "Omega-9 fat", unit: "g" },
|
||||
{ key: "cholesterol", label: "Cholesterol", unit: "mg" },
|
||||
{ key: "added-sugars", label: "Added sugars", unit: "g" },
|
||||
{ key: "polyols", label: "Sugar alcohols", unit: "g" },
|
||||
{ key: "soluble-fiber", label: "Soluble fiber", unit: "g" },
|
||||
{ key: "insoluble-fiber", label: "Insoluble fiber", unit: "g" },
|
||||
{ key: "starch", label: "Starch", unit: "g" },
|
||||
{ key: "vitamin-a", label: "Vitamin A", unit: "µg" },
|
||||
{ key: "vitamin-a-iu", label: "Vitamin A (IU)", unit: "IU" },
|
||||
{ key: "vitamin-c", label: "Vitamin C", unit: "mg" },
|
||||
{ key: "vitamin-d", label: "Vitamin D", unit: "µg" },
|
||||
{ key: "vitamin-e", label: "Vitamin E", unit: "mg" },
|
||||
{ key: "vitamin-k", label: "Vitamin K", unit: "µg" },
|
||||
{ key: "vitamin-b1", label: "Thiamin", unit: "mg" },
|
||||
{ key: "vitamin-b2", label: "Riboflavin", unit: "mg" },
|
||||
{ key: "vitamin-pp", label: "Niacin", unit: "mg" },
|
||||
{ key: "vitamin-b6", label: "Vitamin B6", unit: "mg" },
|
||||
{ key: "vitamin-b9", label: "Folate", unit: "µg" },
|
||||
{ key: "folates", label: "Folate, total", unit: "µg" },
|
||||
{ key: "folic-acid", label: "Folic acid", unit: "µg" },
|
||||
{ key: "vitamin-b12", label: "Vitamin B12", unit: "µg" },
|
||||
{ key: "biotin", label: "Biotin", unit: "µg" },
|
||||
{ key: "pantothenic-acid", label: "Pantothenic acid", unit: "mg" },
|
||||
{ key: "choline", label: "Choline", unit: "mg" },
|
||||
{ key: "calcium", label: "Calcium", unit: "mg" },
|
||||
{ key: "iron", label: "Iron", unit: "mg" },
|
||||
{ key: "potassium", label: "Potassium", unit: "mg" },
|
||||
{ key: "magnesium", label: "Magnesium", unit: "mg" },
|
||||
{ key: "phosphorus", label: "Phosphorus", unit: "mg" },
|
||||
{ key: "zinc", label: "Zinc", unit: "mg" },
|
||||
{ key: "copper", label: "Copper", unit: "mg" },
|
||||
{ key: "manganese", label: "Manganese", unit: "mg" },
|
||||
{ key: "selenium", label: "Selenium", unit: "µg" },
|
||||
{ key: "iodine", label: "Iodine", unit: "µg" },
|
||||
{ key: "chromium", label: "Chromium", unit: "µg" },
|
||||
{ key: "molybdenum", label: "Molybdenum", unit: "µg" },
|
||||
{ key: "chloride", label: "Chloride", unit: "mg" },
|
||||
{ key: "fluoride", label: "Fluoride", unit: "mg" },
|
||||
{ key: "caffeine", label: "Caffeine", unit: "mg" },
|
||||
{ key: "taurine", label: "Taurine", unit: "mg" },
|
||||
{ key: "alcohol", label: "Alcohol", unit: "g" },
|
||||
];
|
||||
|
||||
// Where each tracked metric lives in a product's nutrient map.
|
||||
export const coreNutrientKeys = {
|
||||
calories: "energy-kcal",
|
||||
protein: "proteins",
|
||||
carbs: "carbohydrates",
|
||||
fat: "fat",
|
||||
fiber: "fiber",
|
||||
sugar: "sugars",
|
||||
sodium: "sodium",
|
||||
};
|
||||
// Stored units for the tracked metrics inside a product's nutrient map.
|
||||
export const coreUnits = {
|
||||
"energy-kcal": "kcal",
|
||||
proteins: "g",
|
||||
carbohydrates: "g",
|
||||
fat: "g",
|
||||
fiber: "g",
|
||||
sugars: "g",
|
||||
sodium: "mg",
|
||||
};
|
||||
|
||||
const byKey = new Map(nutrientCatalog.map((item) => [item.key, item]));
|
||||
const gramsPer = { g: 1, mg: 1e-3, µg: 1e-6 };
|
||||
|
||||
export const nutrientKeyPattern = /^[a-z0-9][a-z0-9-]{0,47}$/;
|
||||
export function nutrientUnit(key) {
|
||||
return coreUnits[key] || byKey.get(key)?.unit || "g";
|
||||
}
|
||||
/** Converts grams to the stored unit for `key`; null when it isn't a mass. */
|
||||
export function fromGrams(key, grams) {
|
||||
const factor = gramsPer[nutrientUnit(key)];
|
||||
return factor ? grams / factor : null;
|
||||
}
|
||||
export function convertMass(amount, from, to) {
|
||||
const a = gramsPer[from];
|
||||
const b = gramsPer[to];
|
||||
return a && b ? (amount * a) / b : null;
|
||||
}
|
||||
export function nutrientLabel(key) {
|
||||
if (byKey.has(key)) return byKey.get(key).label;
|
||||
const words = key.replace(/-/g, " ");
|
||||
return words.charAt(0).toUpperCase() + words.slice(1);
|
||||
}
|
||||
/** Catalog order first, then any other keys alphabetically. */
|
||||
export function sortNutrientKeys(keys) {
|
||||
const order = new Map(nutrientCatalog.map((item, i) => [item.key, i]));
|
||||
return [...keys].sort(
|
||||
(a, b) =>
|
||||
(order.get(a) ?? 1e3) - (order.get(b) ?? 1e3) || a.localeCompare(b),
|
||||
);
|
||||
}
|
||||
/** A readable amount: grams of uncatalogued nutrients scale to mg or µg. */
|
||||
export function displayAmount(key, amount) {
|
||||
let unit = nutrientUnit(key);
|
||||
let value = amount;
|
||||
if (!byKey.has(key) && !coreUnits[key] && value > 0 && value < 1) {
|
||||
unit = value < 0.001 ? "µg" : "mg";
|
||||
value = convertMass(amount, "g", unit);
|
||||
}
|
||||
return { value, unit };
|
||||
}
|
||||
74
shared/products.js
Normal file
74
shared/products.js
Normal file
|
|
@ -0,0 +1,74 @@
|
|||
import { coreNutrientKeys } from "./nutrients.js";
|
||||
|
||||
// Label precision: 1 decimal from 10 up, 2 from 0.1, finer below.
|
||||
const round = (value) => {
|
||||
const places = value >= 10 ? 10 : value >= 0.1 ? 100 : 1e4;
|
||||
return Math.round(value * places) / places;
|
||||
};
|
||||
|
||||
/**
|
||||
* A product database row describes nutrition per 100 g (or 100 ml) plus a
|
||||
* serving size. This turns it into a per-serving food for the editor.
|
||||
* @param {{gtin: string, name?: string|null, brand?: string|null,
|
||||
* serving?: string|null, servingQty?: number|null, servingUnit?: string|null,
|
||||
* nutrients: Record<string, number>, source?: string}} product
|
||||
*/
|
||||
export function productToFood(product) {
|
||||
const qty =
|
||||
product.servingQty > 0 && product.servingQty <= 5000
|
||||
? product.servingQty
|
||||
: null;
|
||||
const unit = product.servingUnit === "ml" ? "ml" : "g";
|
||||
const factor = qty ? qty / 100 : 1;
|
||||
let serving = `100 ${unit}`;
|
||||
if (qty) {
|
||||
const text = (product.serving || "").trim();
|
||||
// Prefer the label's own wording ("2/3 cup (55 g)") when it has any.
|
||||
serving =
|
||||
text && /[a-z]/i.test(text) && text.length <= 100
|
||||
? text
|
||||
: `${round(qty)} ${unit}`;
|
||||
}
|
||||
const nutrients = {};
|
||||
const core = new Set(Object.values(coreNutrientKeys));
|
||||
for (const [metric, key] of Object.entries(coreNutrientKeys))
|
||||
nutrients[metric] = round((product.nutrients[key] || 0) * factor);
|
||||
const extra = {};
|
||||
for (const [key, value] of Object.entries(product.nutrients))
|
||||
if (!core.has(key) && Number.isFinite(value) && value >= 0)
|
||||
extra[key] = Math.min(round(value * factor), 100000);
|
||||
for (const metric of Object.keys(nutrients))
|
||||
nutrients[metric] = Math.min(nutrients[metric], 100000);
|
||||
const brand = (product.brand || "").trim();
|
||||
const title = (product.name || "").trim();
|
||||
// "Cheerios" + "Cheerios Cereal" reads as "Cheerios Cereal", and a brand
|
||||
// owner such as "Coca-Cola Services SA/NV" never replaces "Coca-Cola".
|
||||
const overlaps =
|
||||
title.toLowerCase().includes(brand.toLowerCase()) ||
|
||||
brand.toLowerCase().includes(title.toLowerCase());
|
||||
const name = !title
|
||||
? brand
|
||||
: !brand || overlaps
|
||||
? title
|
||||
: `${brand} ${title}`;
|
||||
return {
|
||||
name: name.slice(0, 160) || undefined,
|
||||
brand: product.brand || undefined,
|
||||
serving,
|
||||
nutrients,
|
||||
...(Object.keys(extra).length ? { extra } : {}),
|
||||
barcode: product.gtin,
|
||||
source: product.source,
|
||||
};
|
||||
}
|
||||
|
||||
/** USDA descriptions are often upper case; make them readable. */
|
||||
export function titleCase(text) {
|
||||
if (!text || /[a-z]/.test(text)) return text;
|
||||
return text
|
||||
.toLowerCase()
|
||||
.replace(
|
||||
/(^|[\s(/&-])([a-z])/g,
|
||||
(_, gap, letter) => gap + letter.toUpperCase(),
|
||||
);
|
||||
}
|
||||
228
src/Scanner.tsx
Normal file
228
src/Scanner.tsx
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
import { useEffect, useRef, useState } from "react";
|
||||
import { Camera, Flashlight, Search } from "lucide-react";
|
||||
import { normalizeBarcode } from "../shared/barcode.js";
|
||||
import { Modal } from "./components";
|
||||
import { barcodeDetector, detectBarcode } from "./scan";
|
||||
|
||||
type Status = "starting" | "scanning" | "denied" | "unavailable";
|
||||
|
||||
export function BarcodeScanner({
|
||||
close,
|
||||
detected,
|
||||
}: {
|
||||
close: () => void;
|
||||
detected: (gtin: string) => void;
|
||||
}) {
|
||||
const video = useRef<HTMLVideoElement>(null);
|
||||
const track = useRef<MediaStreamTrack | null>(null);
|
||||
const done = useRef(false);
|
||||
const [status, setStatus] = useState<Status>("starting");
|
||||
const [torch, setTorch] = useState<boolean | null>(null);
|
||||
const [digits, setDigits] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const finish = (gtin: string) => {
|
||||
if (done.current) return;
|
||||
done.current = true;
|
||||
navigator.vibrate?.(30);
|
||||
detected(gtin);
|
||||
};
|
||||
const finishRef = useRef(finish);
|
||||
finishRef.current = finish;
|
||||
|
||||
useEffect(() => {
|
||||
let stream: MediaStream | undefined;
|
||||
let stopped = false;
|
||||
let timer = 0;
|
||||
const reader = barcodeDetector();
|
||||
(async () => {
|
||||
try {
|
||||
stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: false,
|
||||
video: {
|
||||
facingMode: { ideal: "environment" },
|
||||
width: { ideal: 1920 },
|
||||
height: { ideal: 1080 },
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
if (!stopped)
|
||||
setStatus(
|
||||
(err as Error).name === "NotAllowedError"
|
||||
? "denied"
|
||||
: "unavailable",
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (stopped) return stream.getTracks().forEach((t) => t.stop());
|
||||
const element = video.current!;
|
||||
element.srcObject = stream;
|
||||
await element.play().catch(() => {});
|
||||
const camera = stream.getVideoTracks()[0];
|
||||
track.current = camera;
|
||||
const capabilities = camera.getCapabilities?.() as
|
||||
| (MediaTrackCapabilities & { torch?: boolean; focusMode?: string[] })
|
||||
| undefined;
|
||||
if (capabilities?.torch) setTorch(false);
|
||||
if (capabilities?.focusMode?.includes("continuous"))
|
||||
camera
|
||||
.applyConstraints({
|
||||
advanced: [{ focusMode: "continuous" } as MediaTrackConstraintSet],
|
||||
})
|
||||
.catch(() => {});
|
||||
setStatus("scanning");
|
||||
const detector = await reader.catch(() => null);
|
||||
if (!detector) return setStatus("unavailable");
|
||||
const canvas = document.createElement("canvas");
|
||||
const context = canvas.getContext("2d", { willReadFrequently: true })!;
|
||||
const tick = async () => {
|
||||
if (stopped || done.current) return;
|
||||
const { videoWidth: w, videoHeight: h } = element;
|
||||
if (w && h) {
|
||||
// Read only the band under the on-screen guide: it's faster and
|
||||
// ignores other barcodes in view.
|
||||
const cw = Math.round(w * 0.86);
|
||||
const ch = Math.round(h * 0.56);
|
||||
const scale = Math.min(1, 1280 / cw);
|
||||
canvas.width = Math.round(cw * scale);
|
||||
canvas.height = Math.round(ch * scale);
|
||||
context.drawImage(
|
||||
element,
|
||||
(w - cw) / 2,
|
||||
(h - ch) / 2,
|
||||
cw,
|
||||
ch,
|
||||
0,
|
||||
0,
|
||||
canvas.width,
|
||||
canvas.height,
|
||||
);
|
||||
try {
|
||||
const frame = context.getImageData(
|
||||
0,
|
||||
0,
|
||||
canvas.width,
|
||||
canvas.height,
|
||||
);
|
||||
const gtin = await detectBarcode(frame);
|
||||
if (gtin && !stopped) return finishRef.current(gtin);
|
||||
} catch {
|
||||
// A dropped frame; try the next one.
|
||||
}
|
||||
}
|
||||
timer = window.setTimeout(tick, 100);
|
||||
};
|
||||
void tick();
|
||||
})();
|
||||
return () => {
|
||||
stopped = true;
|
||||
clearTimeout(timer);
|
||||
stream?.getTracks().forEach((t) => t.stop());
|
||||
};
|
||||
}, []);
|
||||
|
||||
async function photo(file: File | undefined) {
|
||||
if (!file) return;
|
||||
setError("");
|
||||
try {
|
||||
const bitmap = await createImageBitmap(file, {
|
||||
imageOrientation: "from-image",
|
||||
});
|
||||
const gtin = await detectBarcode(bitmap);
|
||||
bitmap.close();
|
||||
if (gtin) finish(gtin);
|
||||
else
|
||||
setError("No barcode found in that photo. Get closer and try again.");
|
||||
} catch {
|
||||
setError("That photo couldn’t be read.");
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Modal
|
||||
title="Scan a barcode"
|
||||
subtitle="Line the barcode up inside the frame."
|
||||
close={close}
|
||||
>
|
||||
<div className={"scanner-view" + (status === "scanning" ? " live" : "")}>
|
||||
<video ref={video} playsInline muted aria-hidden="true" />
|
||||
{status === "scanning" && <span className="scanner-guide" />}
|
||||
{status !== "scanning" && (
|
||||
<p>
|
||||
{status === "starting"
|
||||
? "Starting the camera…"
|
||||
: status === "denied"
|
||||
? "Camera access is off for Calio. Allow it in Settings, or use a photo or the digits below."
|
||||
: "The camera isn’t available here. Use a photo or the digits below."}
|
||||
</p>
|
||||
)}
|
||||
{torch !== null && (
|
||||
<button
|
||||
type="button"
|
||||
className="scanner-torch"
|
||||
aria-label={torch ? "Turn light off" : "Turn light on"}
|
||||
aria-pressed={torch}
|
||||
onClick={() => {
|
||||
const next = !torch;
|
||||
track.current
|
||||
?.applyConstraints({
|
||||
advanced: [{ torch: next } as MediaTrackConstraintSet],
|
||||
})
|
||||
.then(() => setTorch(next))
|
||||
.catch(() => setTorch(null));
|
||||
}}
|
||||
>
|
||||
<Flashlight size={18} />
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
<div className="scanner-actions">
|
||||
<label className="secondary">
|
||||
<Camera size={17} />
|
||||
Take a photo instead
|
||||
<input
|
||||
type="file"
|
||||
accept="image/*"
|
||||
capture="environment"
|
||||
hidden
|
||||
onChange={(e) => {
|
||||
void photo(e.target.files?.[0]);
|
||||
e.target.value = "";
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
<form
|
||||
className="scanner-manual"
|
||||
onSubmit={(event) => {
|
||||
event.preventDefault();
|
||||
const gtin = normalizeBarcode(digits);
|
||||
if (gtin) finish(gtin);
|
||||
else setError("Check the digits: that isn’t a valid UPC or EAN.");
|
||||
}}
|
||||
>
|
||||
<label>
|
||||
Or type the numbers under the barcode
|
||||
<span className="scanner-manual-row">
|
||||
<input
|
||||
inputMode="numeric"
|
||||
autoComplete="off"
|
||||
maxLength={18}
|
||||
placeholder="e.g. 016000275287"
|
||||
value={digits}
|
||||
onChange={(e) => setDigits(e.target.value)}
|
||||
/>
|
||||
<button className="primary" disabled={!digits.trim()}>
|
||||
<Search size={17} />
|
||||
Look up
|
||||
</button>
|
||||
</span>
|
||||
</label>
|
||||
</form>
|
||||
{error && (
|
||||
<p className="error" role="alert">
|
||||
{error}
|
||||
</p>
|
||||
)}
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
|
|
@ -11,11 +11,12 @@ import {
|
|||
Moon,
|
||||
Monitor,
|
||||
ArrowUpRight,
|
||||
ScanBarcode,
|
||||
} from "lucide-react";
|
||||
import type { JournalRecord, Metric, Settings as SettingsData } from "./types";
|
||||
import { metrics } from "./types";
|
||||
import { downloadExport, offsetDay, today } from "./utils";
|
||||
import { logout, saveRecord, sync, useSyncStatus } from "./store";
|
||||
import { api, logout, saveRecord, sync, useSyncStatus } from "./store";
|
||||
import {
|
||||
accentPalette,
|
||||
accentPresets,
|
||||
|
|
@ -23,6 +24,63 @@ import {
|
|||
} from "../shared/theme.js";
|
||||
import { useTilt } from "./useTilt";
|
||||
|
||||
type ProductSource = {
|
||||
source: "usda" | "off";
|
||||
imported_at: string | null;
|
||||
rows: number | null;
|
||||
error: string | null;
|
||||
};
|
||||
const sourceNames = {
|
||||
usda: "USDA FoodData Central",
|
||||
off: "Open Food Facts",
|
||||
};
|
||||
// Status of the server's product databases, refreshed in the background.
|
||||
function ProductDatabases() {
|
||||
const [sources, setSources] = useState<ProductSource[] | null>(null);
|
||||
const [failed, setFailed] = useState(false);
|
||||
useEffect(() => {
|
||||
api("/products")
|
||||
.then((data) => setSources(data.sources))
|
||||
.catch(() => setFailed(true));
|
||||
}, []);
|
||||
return (
|
||||
<section className="card settings-card">
|
||||
<div className="card-title">
|
||||
<div>
|
||||
<span className="eyebrow">SCAN AND GO</span>
|
||||
<h2>Barcode database</h2>
|
||||
</div>
|
||||
<ScanBarcode size={22} className="muted" />
|
||||
</div>
|
||||
{(["usda", "off"] as const).map((key) => {
|
||||
const source = sources?.find((item) => item.source === key);
|
||||
return (
|
||||
<div className="status-detail" key={key}>
|
||||
<div>
|
||||
<strong>{sourceNames[key]}</strong>
|
||||
<small>
|
||||
{failed
|
||||
? "Status unavailable while offline."
|
||||
: !sources
|
||||
? "Checking…"
|
||||
: source?.imported_at
|
||||
? `${(source.rows ?? 0).toLocaleString()} products · updated ${new Date(source.imported_at).toLocaleDateString()}`
|
||||
: source?.error
|
||||
? `Download failed: ${source.error}`
|
||||
: "First download in progress on the server."}
|
||||
</small>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
<p className="small muted">
|
||||
Scans check your saved foods first, then these copies on your server,
|
||||
then Open Food Facts online. Both copies refresh automatically.
|
||||
</p>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
// Edits a number locally and commits once it is valid: after a short pause, on
|
||||
// blur, or on Enter. External changes (sync) apply only while it is not focused.
|
||||
function NumberField({
|
||||
|
|
@ -580,6 +638,7 @@ export function Settings({
|
|||
</button>
|
||||
)}
|
||||
</section>
|
||||
<ProductDatabases />
|
||||
<section className="account-row">
|
||||
<div>
|
||||
<ShieldCheck size={20} />
|
||||
|
|
|
|||
|
|
@ -1,5 +1,13 @@
|
|||
import { useEffect, useId, useRef, useState, type ReactNode } from "react";
|
||||
import { X, ArrowUpRight, Plus, Minus, Leaf } from "lucide-react";
|
||||
import {
|
||||
X,
|
||||
ArrowUpRight,
|
||||
Plus,
|
||||
Minus,
|
||||
Leaf,
|
||||
ScanBarcode,
|
||||
ScanText,
|
||||
} from "lucide-react";
|
||||
import { bubbleCount } from "../shared/bubble-physics.js";
|
||||
import { useBubblePhysics } from "./useBubblePhysics";
|
||||
import {
|
||||
|
|
@ -11,10 +19,20 @@ import {
|
|||
type Meal,
|
||||
type Nutrients,
|
||||
type Plate,
|
||||
type ScannedFood,
|
||||
type Settings,
|
||||
type Weight,
|
||||
} from "./types";
|
||||
import { emptyNutrients } from "../shared/domain.js";
|
||||
import { displayBarcode } from "../shared/barcode.js";
|
||||
import {
|
||||
nutrientLabel,
|
||||
nutrientUnit,
|
||||
sortNutrientKeys,
|
||||
} from "../shared/nutrients.js";
|
||||
import { BarcodeScanner } from "./Scanner";
|
||||
import { lookupBarcode, readLabel } from "./scan";
|
||||
import { Presence } from "./Presence";
|
||||
import { saveRecord, putRecords, useJournal } from "./store";
|
||||
import { localTime, nextFoodName } from "../shared/entry.js";
|
||||
import { displayDate, number, today, weightValue } from "./utils";
|
||||
|
|
@ -326,6 +344,16 @@ export function FoodEditor({
|
|||
const [time, setTime] = useState(
|
||||
existing?.type === "entry" ? (initial as Entry).time || "" : localTime(),
|
||||
);
|
||||
const [barcode, setBarcode] = useState(initial?.barcode);
|
||||
const [extra, setExtra] = useState<Record<string, number | "">>(
|
||||
initial?.extra || {},
|
||||
);
|
||||
const [scanning, setScanning] = useState(false);
|
||||
const [reading, setReading] = useState<"" | "barcode" | "label">("");
|
||||
const [scanNote, setScanNote] = useState<{
|
||||
text: string;
|
||||
tone?: "error";
|
||||
} | null>(null);
|
||||
const fallbackName = nextFoodName(
|
||||
mode === "food" ? "Food" : selectedMeal,
|
||||
records
|
||||
|
|
@ -343,6 +371,63 @@ export function FoodEditor({
|
|||
);
|
||||
const mealLabel = useId();
|
||||
const [saveFood, setSaveFood] = useState(false);
|
||||
const fill = (food: Food | ScannedFood) => {
|
||||
setName(food.name || "");
|
||||
if (food.serving) setServing(food.serving);
|
||||
if (food.nutrients) setNutrients(food.nutrients);
|
||||
setExtra(food.extra || {});
|
||||
setBarcode(food.barcode);
|
||||
};
|
||||
async function scanned(gtin: string) {
|
||||
setScanning(false);
|
||||
setReading("barcode");
|
||||
setScanNote(null);
|
||||
setBarcode(gtin);
|
||||
try {
|
||||
const food = await lookupBarcode(gtin, foods);
|
||||
fill(food);
|
||||
// Saving a database find makes the next scan instant and offline.
|
||||
if (mode === "log" && food.source !== "library") setSaveFood(true);
|
||||
setScanNote({
|
||||
text:
|
||||
food.source === "library"
|
||||
? "Found in your library."
|
||||
: `Found ${food.source === "usda" ? "in USDA FoodData Central" : "on Open Food Facts"}. Check the serving before adding.`,
|
||||
});
|
||||
} catch (err) {
|
||||
const status = (err as { status?: number }).status;
|
||||
setScanNote({
|
||||
tone: "error",
|
||||
text:
|
||||
status === 404 || status === 502
|
||||
? `${(err as Error).message} Scan the nutrition label instead; barcode ${displayBarcode(gtin)} will be saved with this food.`
|
||||
: (err as Error).message,
|
||||
});
|
||||
} finally {
|
||||
setReading("");
|
||||
}
|
||||
}
|
||||
async function label(file: File | undefined) {
|
||||
if (!file) return;
|
||||
setReading("label");
|
||||
setScanNote(null);
|
||||
try {
|
||||
const result = await readLabel(file);
|
||||
if (result.serving) setServing(result.serving);
|
||||
// The label is the authority: anything it didn't show stays as typed,
|
||||
// but nothing carries over from an earlier lookup's extra nutrients.
|
||||
setNutrients((current) => ({ ...current, ...result.nutrients }));
|
||||
setExtra(result.extra);
|
||||
if (mode === "log" && barcode) setSaveFood(true);
|
||||
setScanNote({
|
||||
text: `Read ${result.found.length} ${result.found.length === 1 ? "value" : "values"} from the label. Check them against the package before saving.`,
|
||||
});
|
||||
} catch (err) {
|
||||
setScanNote({ tone: "error", text: (err as Error).message });
|
||||
} finally {
|
||||
setReading("");
|
||||
}
|
||||
}
|
||||
const [search, setSearch] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
const [busy, setBusy] = useState(false);
|
||||
|
|
@ -355,7 +440,12 @@ export function FoodEditor({
|
|||
throw new Error("Choose an entry date and time.");
|
||||
if (mode === "log" && !(Number(quantity) > 0))
|
||||
throw new Error("Enter how many servings you had.");
|
||||
const food = {
|
||||
const extras = Object.fromEntries(
|
||||
Object.entries(extra)
|
||||
.filter(([, value]) => value !== "" && Number.isFinite(Number(value)))
|
||||
.map(([key, value]) => [key, Number(value)]),
|
||||
);
|
||||
const food: Food = {
|
||||
name: name.trim() || fallbackName,
|
||||
serving: serving.trim(),
|
||||
nutrients: Object.fromEntries(
|
||||
|
|
@ -364,6 +454,8 @@ export function FoodEditor({
|
|||
Number(value) || 0,
|
||||
]),
|
||||
) as Nutrients,
|
||||
...(barcode ? { barcode } : {}),
|
||||
...(Object.keys(extras).length ? { extra: extras } : {}),
|
||||
};
|
||||
const items: Parameters<typeof putRecords>[0] = [
|
||||
{
|
||||
|
|
@ -522,9 +614,7 @@ export function FoodEditor({
|
|||
className="picker-row"
|
||||
key={r.id}
|
||||
onClick={() => {
|
||||
setName(r.data.name);
|
||||
setServing(r.data.serving);
|
||||
setNutrients(r.data.nutrients);
|
||||
fill(r.data);
|
||||
setTab("manual");
|
||||
}}
|
||||
>
|
||||
|
|
@ -577,6 +667,45 @@ export function FoodEditor({
|
|||
</div>
|
||||
) : (
|
||||
<form className="panel-arrive" onSubmit={submit}>
|
||||
<div className="scan-actions">
|
||||
<button
|
||||
type="button"
|
||||
className="secondary"
|
||||
disabled={Boolean(reading)}
|
||||
onClick={() => setScanning(true)}
|
||||
>
|
||||
<ScanBarcode size={18} />
|
||||
{reading === "barcode" ? "Looking up…" : "Scan barcode"}
|
||||
</button>
|
||||
<label
|
||||
className={"secondary" + (reading ? " disabled" : "")}
|
||||
aria-disabled={Boolean(reading)}
|
||||
>
|
||||
<ScanText size={18} />
|
||||
{reading === "label" ? "Reading label…" : "Scan label"}
|
||||
<input
|
||||
type="file"
|
||||
accept="image/*"
|
||||
capture="environment"
|
||||
hidden
|
||||
disabled={Boolean(reading)}
|
||||
onChange={(e) => {
|
||||
void label(e.target.files?.[0]);
|
||||
e.target.value = "";
|
||||
}}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
{scanNote && (
|
||||
<p
|
||||
className={
|
||||
"scan-note" + (scanNote.tone ? " " + scanNote.tone : "")
|
||||
}
|
||||
role="status"
|
||||
>
|
||||
{scanNote.text}
|
||||
</p>
|
||||
)}
|
||||
<label>
|
||||
Food name <span className="muted">(optional)</span>
|
||||
<input
|
||||
|
|
@ -620,6 +749,51 @@ export function FoodEditor({
|
|||
<span>From your label or recipe</span>
|
||||
</div>
|
||||
<NutrientFields nutrients={nutrients} set={setNutrients} />
|
||||
{Object.keys(extra).length > 0 && (
|
||||
<details className="extra-nutrients">
|
||||
<summary>
|
||||
More nutrients
|
||||
<span className="muted">{Object.keys(extra).length}</span>
|
||||
</summary>
|
||||
<div className="extra-fields">
|
||||
{sortNutrientKeys(Object.keys(extra)).map((key) => (
|
||||
<label key={key}>
|
||||
{nutrientLabel(key)}
|
||||
<span className="unit-input">
|
||||
<input
|
||||
type="number"
|
||||
inputMode="decimal"
|
||||
min="0"
|
||||
max="100000"
|
||||
step="any"
|
||||
placeholder="0"
|
||||
value={extra[key]}
|
||||
onChange={(e) =>
|
||||
setExtra({
|
||||
...extra,
|
||||
[key]: e.target.value === "" ? "" : +e.target.value,
|
||||
})
|
||||
}
|
||||
/>
|
||||
<span>{nutrientUnit(key)}</span>
|
||||
</span>
|
||||
</label>
|
||||
))}
|
||||
</div>
|
||||
</details>
|
||||
)}
|
||||
{barcode && (
|
||||
<p className="barcode-line muted">
|
||||
Barcode {displayBarcode(barcode)}
|
||||
<button
|
||||
type="button"
|
||||
className="text-button"
|
||||
onClick={() => setBarcode(undefined)}
|
||||
>
|
||||
Remove
|
||||
</button>
|
||||
</p>
|
||||
)}
|
||||
{mode === "log" && !existing && (
|
||||
<label className="check-row">
|
||||
<input
|
||||
|
|
@ -652,6 +826,14 @@ export function FoodEditor({
|
|||
{error}
|
||||
</p>
|
||||
)}
|
||||
<Presence>
|
||||
{scanning && (
|
||||
<BarcodeScanner
|
||||
close={() => setScanning(false)}
|
||||
detected={(gtin) => void scanned(gtin)}
|
||||
/>
|
||||
)}
|
||||
</Presence>
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -159,6 +159,17 @@ textarea {
|
|||
.check-row {
|
||||
font-size: 14px;
|
||||
}
|
||||
.scan-note,
|
||||
.barcode-line,
|
||||
.scanner-view > p {
|
||||
font-size: 13px;
|
||||
}
|
||||
.extra-nutrients summary {
|
||||
font-size: 14px;
|
||||
}
|
||||
.extra-fields label {
|
||||
font-size: 12px;
|
||||
}
|
||||
.error {
|
||||
font-size: 13px;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -534,10 +534,27 @@
|
|||
.sheet .section-caption > span:last-child {
|
||||
display: none;
|
||||
}
|
||||
.nutrient-fields {
|
||||
.nutrient-fields,
|
||||
.extra-fields {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 16px;
|
||||
}
|
||||
.scan-note,
|
||||
.barcode-line,
|
||||
.scanner-view > p {
|
||||
font-size: 14px;
|
||||
}
|
||||
.extra-nutrients summary {
|
||||
font-size: 16px;
|
||||
}
|
||||
.scanner-view {
|
||||
aspect-ratio: 1;
|
||||
}
|
||||
.scan-actions .secondary {
|
||||
padding-inline: 10px;
|
||||
gap: 7px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.nutrient-fields label:first-child {
|
||||
font-size: 16px;
|
||||
grid-column: 1 / -1;
|
||||
|
|
|
|||
86
src/scan.ts
Normal file
86
src/scan.ts
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
import { normalizeBarcode } from "../shared/barcode.js";
|
||||
import { api } from "./store";
|
||||
import type { Food, JournalRecord, ScannedFood } from "./types";
|
||||
|
||||
type Detected = { rawValue: string; format: string };
|
||||
type Detector = { detect(source: ImageBitmapSource): Promise<Detected[]> };
|
||||
|
||||
let detector: Promise<Detector> | null = null;
|
||||
/**
|
||||
* The ZXing WebAssembly reader, loaded on first use. Safari has no native
|
||||
* BarcodeDetector, so every browser uses the same reader; its .wasm is
|
||||
* bundled and precached rather than fetched from a CDN.
|
||||
*/
|
||||
export function barcodeDetector() {
|
||||
detector ||= (async () => {
|
||||
const [{ BarcodeDetector, prepareZXingModule }, { default: wasm }] =
|
||||
await Promise.all([
|
||||
import("barcode-detector/ponyfill"),
|
||||
import("zxing-wasm/reader/zxing_reader.wasm?url"),
|
||||
]);
|
||||
prepareZXingModule({
|
||||
overrides: {
|
||||
locateFile: (path: string, prefix: string) =>
|
||||
path.endsWith(".wasm") ? wasm : prefix + path,
|
||||
},
|
||||
});
|
||||
return new BarcodeDetector({
|
||||
formats: ["ean_13", "ean_8", "upc_a", "upc_e"],
|
||||
}) as Detector;
|
||||
})();
|
||||
detector.catch(() => (detector = null));
|
||||
return detector;
|
||||
}
|
||||
|
||||
/** The first valid retail barcode in an image, as GTIN-14. */
|
||||
export async function detectBarcode(source: ImageBitmapSource) {
|
||||
for (const code of await (await barcodeDetector()).detect(source)) {
|
||||
const gtin = normalizeBarcode(code.rawValue, code.format);
|
||||
if (gtin) return gtin;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Saved foods first (works offline), then the server's product databases. */
|
||||
export async function lookupBarcode(
|
||||
gtin: string,
|
||||
foods: JournalRecord<Food>[],
|
||||
): Promise<ScannedFood> {
|
||||
const saved = foods.find((record) => record.data.barcode === gtin);
|
||||
if (saved) return { ...saved.data, source: "library" };
|
||||
const { food } = (await api("/products/" + gtin)) as { food: ScannedFood };
|
||||
return food;
|
||||
}
|
||||
|
||||
export type LabelResult = {
|
||||
serving?: string;
|
||||
nutrients: Partial<Food["nutrients"]>;
|
||||
extra: Record<string, number>;
|
||||
found: string[];
|
||||
};
|
||||
|
||||
/** Sends a label photo, downscaled and re-encoded, to be read on the server. */
|
||||
export async function readLabel(file: Blob): Promise<LabelResult> {
|
||||
let photo: Blob = file;
|
||||
try {
|
||||
const bitmap = await createImageBitmap(file, {
|
||||
imageOrientation: "from-image",
|
||||
});
|
||||
const scale = Math.min(1, 2560 / Math.max(bitmap.width, bitmap.height));
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = Math.round(bitmap.width * scale);
|
||||
canvas.height = Math.round(bitmap.height * scale);
|
||||
canvas
|
||||
.getContext("2d")!
|
||||
.drawImage(bitmap, 0, 0, canvas.width, canvas.height);
|
||||
bitmap.close();
|
||||
photo =
|
||||
(await new Promise<Blob | null>((resolve) =>
|
||||
canvas.toBlob(resolve, "image/jpeg", 0.92),
|
||||
)) || file;
|
||||
} catch {
|
||||
// Unsupported formats (e.g. HEIC on desktop) go up unchanged.
|
||||
}
|
||||
const { label } = (await api("/label", photo)) as { label: LabelResult };
|
||||
return label;
|
||||
}
|
||||
11
src/store.ts
11
src/store.ts
|
|
@ -102,13 +102,18 @@ export const saveRecord = (type: RecordType, data: unknown, id?: string) =>
|
|||
export const deleteRecord = (record: JournalRecord) =>
|
||||
putRecords([{ ...record, deleted: true }]);
|
||||
export async function api(path: string, body?: unknown) {
|
||||
// Photos upload as-is and get longer to be read.
|
||||
const blob = body instanceof Blob;
|
||||
let response: Response;
|
||||
try {
|
||||
response = await fetch("/api" + path, {
|
||||
method: body === undefined ? "GET" : "POST",
|
||||
headers: { "Content-Type": "application/json", "X-Calio-Request": "1" },
|
||||
body: body === undefined ? undefined : JSON.stringify(body),
|
||||
signal: AbortSignal.timeout(15000),
|
||||
headers: {
|
||||
"Content-Type": blob ? body.type : "application/json",
|
||||
"X-Calio-Request": "1",
|
||||
},
|
||||
body: body === undefined || blob ? body : JSON.stringify(body),
|
||||
signal: AbortSignal.timeout(blob ? 45000 : 15000),
|
||||
});
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
|
|
|
|||
141
src/styles.css
141
src/styles.css
|
|
@ -1147,6 +1147,8 @@ textarea {
|
|||
font-size: 10px;
|
||||
color: var(--muted);
|
||||
padding-right: 12px;
|
||||
flex-shrink: 0;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.section-caption {
|
||||
display: flex;
|
||||
|
|
@ -1205,6 +1207,145 @@ textarea {
|
|||
.sheet form > .primary {
|
||||
margin-top: 22px;
|
||||
}
|
||||
/* Barcode and label scanning. The camera inputs are hidden inside their
|
||||
buttons; [hidden] must win over the phone layout's block inputs. */
|
||||
[hidden] {
|
||||
display: none !important;
|
||||
}
|
||||
.scan-actions {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 10px;
|
||||
}
|
||||
.scan-actions .secondary {
|
||||
cursor: pointer;
|
||||
margin: 0;
|
||||
}
|
||||
.scan-actions .disabled,
|
||||
.scan-actions :disabled {
|
||||
opacity: 0.55;
|
||||
pointer-events: none;
|
||||
}
|
||||
.scan-note {
|
||||
margin-top: 12px;
|
||||
padding: 10px 12px;
|
||||
border-radius: 10px;
|
||||
background: var(--accent-soft);
|
||||
color: var(--accent-text);
|
||||
font-size: 11px;
|
||||
line-height: 1.45;
|
||||
}
|
||||
.scan-note.error {
|
||||
background: color-mix(in srgb, var(--danger) 10%, transparent);
|
||||
color: var(--danger);
|
||||
}
|
||||
.extra-nutrients {
|
||||
margin-top: 16px;
|
||||
border-top: 1px solid var(--line);
|
||||
padding-top: 12px;
|
||||
}
|
||||
.extra-nutrients summary {
|
||||
cursor: pointer;
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
min-height: 32px;
|
||||
font-size: 11px;
|
||||
font-weight: 550;
|
||||
}
|
||||
.extra-fields {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||
gap: 13px 15px;
|
||||
margin-top: 8px;
|
||||
}
|
||||
.extra-fields label {
|
||||
min-width: 0;
|
||||
font-size: 10px;
|
||||
color: var(--muted);
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.barcode-line {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
margin-top: 12px;
|
||||
font-size: 11px;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.scanner-view {
|
||||
position: relative;
|
||||
aspect-ratio: 4 / 3;
|
||||
border-radius: 16px;
|
||||
overflow: hidden;
|
||||
background: #111216;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
}
|
||||
.scanner-view video {
|
||||
position: absolute;
|
||||
inset: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
opacity: 0;
|
||||
}
|
||||
.scanner-view.live video {
|
||||
opacity: 1;
|
||||
}
|
||||
.scanner-view > p {
|
||||
position: relative;
|
||||
max-width: 30ch;
|
||||
padding: 20px;
|
||||
text-align: center;
|
||||
color: #e8e9f0;
|
||||
font-size: 12px;
|
||||
}
|
||||
.scanner-guide {
|
||||
position: absolute;
|
||||
inset: 22% 7%;
|
||||
border: 2px solid #ffffffd0;
|
||||
border-radius: 14px;
|
||||
box-shadow: 0 0 0 100vmax #0006;
|
||||
}
|
||||
.scanner-torch {
|
||||
position: absolute;
|
||||
right: 12px;
|
||||
bottom: 12px;
|
||||
width: 44px;
|
||||
height: 44px;
|
||||
border-radius: 50%;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
background: #0008;
|
||||
color: #fff;
|
||||
}
|
||||
.scanner-torch[aria-pressed="true"] {
|
||||
background: #fff;
|
||||
color: #111;
|
||||
}
|
||||
.scanner-actions {
|
||||
margin-top: 14px;
|
||||
}
|
||||
.scanner-actions .secondary {
|
||||
width: 100%;
|
||||
cursor: pointer;
|
||||
}
|
||||
.scanner-manual {
|
||||
margin-top: 16px;
|
||||
}
|
||||
.scanner-manual-row {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
align-items: stretch;
|
||||
}
|
||||
.scanner-manual-row input {
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
.scanner-manual-row .primary {
|
||||
margin-top: 8px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.error {
|
||||
color: var(--danger);
|
||||
padding: 10px 0;
|
||||
|
|
|
|||
15
src/types.ts
15
src/types.ts
|
|
@ -7,7 +7,20 @@ export type Metric =
|
|||
| "sugar"
|
||||
| "sodium";
|
||||
export type Nutrients = Record<Metric, number>;
|
||||
export type Food = { name: string; serving: string; nutrients: Nutrients };
|
||||
export type Food = {
|
||||
name: string;
|
||||
serving: string;
|
||||
nutrients: Nutrients;
|
||||
/** GTIN-14, see shared/barcode.js. */
|
||||
barcode?: string;
|
||||
/** Further nutrients per serving, see shared/nutrients.js. */
|
||||
extra?: Record<string, number>;
|
||||
};
|
||||
/** A barcode lookup or label scan, ready to prefill the food editor. */
|
||||
export type ScannedFood = Partial<Food> & {
|
||||
brand?: string;
|
||||
source?: string;
|
||||
};
|
||||
export type Meal = "Breakfast" | "Lunch" | "Dinner" | "Snacks";
|
||||
export type Entry = Food & {
|
||||
date: string;
|
||||
|
|
|
|||
20
src/utils.ts
20
src/utils.ts
|
|
@ -1,4 +1,5 @@
|
|||
import { totalNutrients } from "../shared/domain.js";
|
||||
import { nutrientUnit } from "../shared/nutrients.js";
|
||||
import type { Entry, JournalRecord, Metric, Settings, Weight } from "./types";
|
||||
import { metrics } from "./types.ts";
|
||||
export const today = () => localDate(new Date());
|
||||
|
|
@ -117,7 +118,24 @@ export function downloadExport(
|
|||
{
|
||||
...metadata,
|
||||
dailyTotals: days,
|
||||
entries: logs,
|
||||
// Nutrients beyond the tracked metrics, from barcode and label scans.
|
||||
entries: logs.map((log, i) => {
|
||||
const extra = entries[i].data.extra;
|
||||
if (!extra) return log;
|
||||
const quantity = entries[i].data.quantity;
|
||||
return {
|
||||
...log,
|
||||
otherNutrients: Object.fromEntries(
|
||||
Object.entries(extra).map(([key, amount]) => [
|
||||
key,
|
||||
{
|
||||
amount: +(amount * quantity).toFixed(4),
|
||||
unit: nutrientUnit(key),
|
||||
},
|
||||
]),
|
||||
),
|
||||
};
|
||||
}),
|
||||
weights: weights.map((r) => ({ ...r.data, id: r.id })),
|
||||
},
|
||||
null,
|
||||
|
|
|
|||
37
tests/barcode.test.js
Normal file
37
tests/barcode.test.js
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
databaseGtin,
|
||||
displayBarcode,
|
||||
expandUpcE,
|
||||
normalizeBarcode,
|
||||
} from "../shared/barcode.js";
|
||||
|
||||
test("UPC-A, EAN-13 and GTIN-14 forms normalize to one GTIN", () => {
|
||||
assert.equal(normalizeBarcode("016000275287"), "00016000275287");
|
||||
assert.equal(normalizeBarcode("0016000275287"), "00016000275287");
|
||||
assert.equal(normalizeBarcode("00016000275287"), "00016000275287");
|
||||
assert.equal(normalizeBarcode("3017620422003"), "03017620422003");
|
||||
assert.equal(normalizeBarcode("96385074"), "00000096385074");
|
||||
});
|
||||
|
||||
test("invalid check digits and lengths are rejected", () => {
|
||||
assert.equal(normalizeBarcode("016000275288"), null);
|
||||
assert.equal(normalizeBarcode("12345"), null);
|
||||
assert.equal(normalizeBarcode("abc"), null);
|
||||
});
|
||||
|
||||
test("UPC-E expands to UPC-A", () => {
|
||||
assert.equal(expandUpcE("04252614"), "042100005264");
|
||||
assert.equal(normalizeBarcode("04252614", "upc_e"), "00042100005264");
|
||||
assert.equal(normalizeBarcode("425261"), "00042100005264");
|
||||
});
|
||||
|
||||
test("database codes pad without validation; display uses the short form", () => {
|
||||
assert.equal(databaseGtin("28400090858"), "00028400090858");
|
||||
assert.equal(databaseGtin("123"), null);
|
||||
assert.equal(databaseGtin("00000000000000"), null);
|
||||
assert.equal(displayBarcode("00016000275287"), "016000275287");
|
||||
assert.equal(displayBarcode("03017620422003"), "3017620422003");
|
||||
assert.equal(displayBarcode("00000096385074"), "96385074");
|
||||
});
|
||||
BIN
tests/fixtures/label.png
vendored
Normal file
BIN
tests/fixtures/label.png
vendored
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
142
tests/label.test.js
Normal file
142
tests/label.test.js
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { clean, parseLabel, rows } from "../server/ocr/label.js";
|
||||
|
||||
// Lays out label rows as OCR boxes: each row is [text, ...more boxes].
|
||||
function boxes(lines, { angle = 0 } = {}) {
|
||||
const result = [];
|
||||
lines.forEach((parts, row) => {
|
||||
let x = 0;
|
||||
for (const text of [].concat(parts)) {
|
||||
const width = text.length * 10;
|
||||
const cx = x + width / 2;
|
||||
const cy = row * 30 + 10 + cx * Math.tan(angle);
|
||||
result.push({
|
||||
text,
|
||||
angle,
|
||||
cx,
|
||||
cy,
|
||||
height: 20,
|
||||
x0: x,
|
||||
x1: x + width,
|
||||
y0: cy - 10,
|
||||
y1: cy + 10,
|
||||
});
|
||||
x += width + 40;
|
||||
}
|
||||
});
|
||||
return result;
|
||||
}
|
||||
|
||||
test("a US Nutrition Facts panel is read in full", () => {
|
||||
const label = parseLabel(
|
||||
boxes([
|
||||
"Nutrition Facts",
|
||||
"About 31 servings per container",
|
||||
["Serving size", "1 cup (54g)"],
|
||||
"Amount per serving",
|
||||
["Calories", "230"],
|
||||
"% Daily Value*",
|
||||
["Total Fat 5g", "6%"],
|
||||
["Saturated Fat 0.5g", "3%"],
|
||||
"Trans Fat 0g",
|
||||
["Cholesterol Omg", "0%"],
|
||||
["Sodium 1,230mg", "53%"],
|
||||
["Total Carbohy ydrate 42g", "15%"],
|
||||
["Dietary Fiber <1g", "2%"],
|
||||
"Total Sugars 15g",
|
||||
["Includes 10g Added Sugars", "20%"],
|
||||
"Protein 4g",
|
||||
["Vitamin D 2mcg 10%", "Calcium 260mg 20%"],
|
||||
["Iron 8mg 45%", "Potassium 235mg 6%"],
|
||||
["Thiamin", "60%"],
|
||||
]),
|
||||
);
|
||||
assert.equal(label.serving, "1 cup (54 g)");
|
||||
assert.deepEqual(label.nutrients, {
|
||||
calories: 230,
|
||||
fat: 5,
|
||||
sodium: 1230,
|
||||
carbs: 42,
|
||||
fiber: 0.5,
|
||||
sugar: 15,
|
||||
protein: 4,
|
||||
});
|
||||
assert.equal(label.extra["saturated-fat"], 0.5);
|
||||
assert.equal(label.extra["trans-fat"], 0);
|
||||
assert.equal(label.extra.cholesterol, 0);
|
||||
assert.equal(label.extra["added-sugars"], 10);
|
||||
assert.equal(label.extra["vitamin-d"], 2);
|
||||
assert.equal(label.extra.calcium, 260);
|
||||
assert.equal(label.extra.iron, 8);
|
||||
assert.equal(label.extra.potassium, 235);
|
||||
assert.equal(label.extra["vitamin-b1"], 0.72); // 60% of 1.2 mg
|
||||
});
|
||||
|
||||
test("misread units are corrected by the %DV", () => {
|
||||
const label = parseLabel(
|
||||
boxes([
|
||||
["Total Fat 89", "10%"], // 8g with the g read as 9
|
||||
["Sodium 160", "7%"],
|
||||
["Calcium 20mg", "0%"], // under 2% legitimately rounds to 0
|
||||
]),
|
||||
);
|
||||
assert.equal(label.nutrients.fat, 8);
|
||||
assert.equal(label.nutrients.sodium, 160);
|
||||
assert.equal(label.extra.calcium, 20);
|
||||
});
|
||||
|
||||
test("a large calorie figure on its own row is found", () => {
|
||||
const label = parseLabel(
|
||||
boxes(["Amount per serving 130", "Calories", ["Total Fat 6g", "8%"]]),
|
||||
);
|
||||
assert.equal(label.nutrients.calories, 130);
|
||||
});
|
||||
|
||||
test("rows follow the text angle of a tilted photo", () => {
|
||||
const tilted = boxes(
|
||||
[
|
||||
["Sodium", "160mg", "7%"],
|
||||
["Total Carbohydrate", "37g", "13%"],
|
||||
],
|
||||
{ angle: 0.12 },
|
||||
);
|
||||
assert.deepEqual(rows(tilted), [
|
||||
"Sodium 160mg 7%",
|
||||
"Total Carbohydrate 37g 13%",
|
||||
]);
|
||||
});
|
||||
|
||||
test("an EU per-100 g table converts salt to sodium", () => {
|
||||
const label = parseLabel(
|
||||
boxes([
|
||||
"Nutrition per 100g",
|
||||
"Energy 1046kJ / 250kcal",
|
||||
"Fat 8,0g",
|
||||
"of which saturates 1,2g",
|
||||
"Carbohydrate 37g",
|
||||
"of which sugars 12g",
|
||||
"Fibre 4g",
|
||||
"Protein 3g",
|
||||
"Salt 0,5g",
|
||||
]),
|
||||
);
|
||||
assert.equal(label.serving, "100 g");
|
||||
assert.deepEqual(label.nutrients, {
|
||||
calories: 250,
|
||||
fat: 8,
|
||||
carbs: 37,
|
||||
sugar: 12,
|
||||
fiber: 4,
|
||||
protein: 3,
|
||||
sodium: 200,
|
||||
});
|
||||
assert.equal(label.extra["saturated-fat"], 1.2);
|
||||
});
|
||||
|
||||
test("OCR letter/digit confusions are undone", () => {
|
||||
assert.equal(clean("Trans Fat Og"), "Trans Fat 0g");
|
||||
assert.equal(clean("Iron 1O.8mg"), "Iron 10.8mg");
|
||||
assert.equal(clean("Vitamin D 2µg"), "Vitamin D 2mcg");
|
||||
assert.equal(clean("Sugars less than 1g"), "Sugars < 1g");
|
||||
});
|
||||
209
tests/products.test.js
Normal file
209
tests/products.test.js
Normal file
|
|
@ -0,0 +1,209 @@
|
|||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { Readable } from "node:stream";
|
||||
import { gzipSync } from "node:zlib";
|
||||
import { openProducts } from "../server/products/db.js";
|
||||
import {
|
||||
fetchOffProduct,
|
||||
importOffCsv,
|
||||
offNutrients,
|
||||
} from "../server/products/off.js";
|
||||
import { csvRecords, nutrientMapper } from "../server/products/usda.js";
|
||||
import { productToFood, titleCase } from "../shared/products.js";
|
||||
import { recordSchema } from "../shared/domain.js";
|
||||
|
||||
test("Open Food Facts grams become Calio units", () => {
|
||||
const nutrients = offNutrients([
|
||||
["energy-kj", "1046"],
|
||||
["fat", "8"],
|
||||
["salt", "1"],
|
||||
["vitamin-c", "0.0036"],
|
||||
["vitamin-d", "0.000005"],
|
||||
["cholesterol", "0.012"],
|
||||
["nutrition-score-fr", "12"],
|
||||
["alcohol", "5"],
|
||||
["proteins", "250"],
|
||||
["fiber", ""],
|
||||
]);
|
||||
assert.equal(Math.round(nutrients["energy-kcal"]), 250);
|
||||
assert.equal(nutrients.fat, 8);
|
||||
assert.equal(nutrients.sodium, 400);
|
||||
assert.ok(Math.abs(nutrients["vitamin-c"] - 3.6) < 1e-9);
|
||||
assert.ok(Math.abs(nutrients["vitamin-d"] - 5) < 1e-9);
|
||||
assert.ok(Math.abs(nutrients.cholesterol - 12) < 1e-9);
|
||||
for (const key of ["nutrition-score-fr", "alcohol", "proteins", "fiber"])
|
||||
assert.equal(nutrients[key], undefined, key);
|
||||
});
|
||||
|
||||
test("products become per-serving foods with every nutrient kept", () => {
|
||||
const food = productToFood({
|
||||
gtin: "00016000275287",
|
||||
name: "Cheerios",
|
||||
brand: "General Mills",
|
||||
serving: "1 cup (28 g)",
|
||||
servingQty: 28,
|
||||
servingUnit: "g",
|
||||
nutrients: {
|
||||
"energy-kcal": 357,
|
||||
proteins: 12.5,
|
||||
sodium: 500,
|
||||
calcium: 357,
|
||||
"palmitic-acid": 0.5,
|
||||
},
|
||||
source: "usda",
|
||||
});
|
||||
assert.equal(food.name, "General Mills Cheerios");
|
||||
assert.equal(food.serving, "1 cup (28 g)");
|
||||
assert.equal(food.nutrients.calories, 100);
|
||||
assert.equal(food.nutrients.sodium, 140);
|
||||
assert.equal(food.nutrients.fat, 0);
|
||||
assert.deepEqual(food.extra, { calcium: 100, "palmitic-acid": 0.14 });
|
||||
const record = {
|
||||
id: "food-1",
|
||||
type: "food",
|
||||
updatedAt: new Date().toISOString(),
|
||||
deleted: false,
|
||||
data: {
|
||||
name: food.name,
|
||||
serving: food.serving,
|
||||
nutrients: food.nutrients,
|
||||
barcode: food.barcode,
|
||||
extra: food.extra,
|
||||
},
|
||||
};
|
||||
assert.equal(recordSchema.safeParse(record).success, true);
|
||||
record.data.barcode = "123";
|
||||
assert.equal(recordSchema.safeParse(record).success, false);
|
||||
});
|
||||
|
||||
test("products without a serving size are shown per 100 g", () => {
|
||||
const food = productToFood({
|
||||
gtin: "03017620422003",
|
||||
name: "Nutella",
|
||||
brand: "Nutella",
|
||||
nutrients: { "energy-kcal": 539 },
|
||||
});
|
||||
assert.equal(food.name, "Nutella");
|
||||
const coke = productToFood({
|
||||
gtin: "05449000000996",
|
||||
name: "Coca-Cola",
|
||||
brand: "COCA-COLA SERVICES SA/NV",
|
||||
nutrients: {},
|
||||
});
|
||||
assert.equal(coke.name, "Coca-Cola");
|
||||
assert.equal(food.serving, "100 g");
|
||||
assert.equal(food.nutrients.calories, 539);
|
||||
assert.equal(titleCase("CHEERIOS CEREAL"), "Cheerios Cereal");
|
||||
assert.equal(titleCase("Mixed case"), "Mixed case");
|
||||
});
|
||||
|
||||
test("the CSV reader handles quotes, escapes and multi-line fields", async () => {
|
||||
const stream = Readable.from(['a,"b ""c""",', '"multi\nline"\r\n1,', "2,3"]);
|
||||
const records = [];
|
||||
for await (const record of csvRecords(stream)) records.push(record);
|
||||
assert.deepEqual(records, [
|
||||
["a", 'b "c"', "multi\nline"],
|
||||
["1", "2", "3"],
|
||||
]);
|
||||
});
|
||||
|
||||
test("USDA nutrients map to catalog keys and units", () => {
|
||||
const map = nutrientMapper([
|
||||
{ id: 1008, name: "Energy", unit: "KCAL" },
|
||||
{ id: 1099, name: "Fluoride, F", unit: "UG" },
|
||||
{ id: 1110, name: "Vitamin D (D2 + D3), International Units", unit: "IU" },
|
||||
{ id: 1210, name: "Tryptophan", unit: "G" },
|
||||
{ id: 1023, name: "pH", unit: "PH" },
|
||||
]);
|
||||
assert.equal(map.get(1008).key, "energy-kcal");
|
||||
assert.equal(map.get(1099).convert(500), 0.5); // µg → mg
|
||||
assert.equal(map.get(1110).convert(400), 10); // IU → µg
|
||||
assert.equal(map.get(1110).fallback, true);
|
||||
assert.equal(map.get(1210).key, "tryptophan");
|
||||
assert.equal(map.has(1023), false);
|
||||
});
|
||||
|
||||
test("imports swap in atomically and USDA wins over Open Food Facts", async () => {
|
||||
const products = openProducts(":memory:");
|
||||
const header = [
|
||||
"code",
|
||||
"product_name",
|
||||
"brands",
|
||||
"serving_size",
|
||||
"serving_quantity",
|
||||
"no_nutrition_data",
|
||||
"energy-kcal_100g",
|
||||
"proteins_100g",
|
||||
];
|
||||
const rows = [
|
||||
header,
|
||||
["16000275287", "My Bff", "Cheerios", "1 cup (28g)", "28", "", "350", "12"],
|
||||
["5449000000996", "Coca-Cola", "Coca-Cola", "330 ml", "330", "", "42", "0"],
|
||||
["123", "Too short", "", "", "", "", "100", "1"],
|
||||
["4000000000006", "No data", "", "", "", "on", "", ""],
|
||||
];
|
||||
const body = gzipSync(rows.map((row) => row.join("\t")).join("\n"));
|
||||
const off = products.beginImport("off");
|
||||
assert.equal(await importOffCsv(new Response(body), off), 2);
|
||||
off.commit("v1");
|
||||
const usda = products.beginImport("usda");
|
||||
usda.add({
|
||||
gtin: "00016000275287",
|
||||
name: "CHEERIOS CEREAL",
|
||||
brand: "Cheerios",
|
||||
serving: "1 cup (28 g)",
|
||||
servingQty: 28,
|
||||
servingUnit: "g",
|
||||
nutrients: { "energy-kcal": 357 },
|
||||
});
|
||||
usda.commit("2026-04-30");
|
||||
const cheerios = products.lookup("00016000275287");
|
||||
assert.equal(cheerios.source, "usda");
|
||||
assert.equal(cheerios.name, "Cheerios Cereal");
|
||||
assert.equal(products.lookup("05449000000996").servingUnit, "ml");
|
||||
assert.equal(products.lookup("00000000000123"), undefined);
|
||||
products.remember("00000000000123", null);
|
||||
assert.equal(products.lookup("00000000000123"), null);
|
||||
assert.deepEqual(
|
||||
products
|
||||
.status()
|
||||
.map((row) => [row.source, row.rows])
|
||||
.sort(),
|
||||
[
|
||||
["off", 2],
|
||||
["usda", 1],
|
||||
],
|
||||
);
|
||||
products.close();
|
||||
});
|
||||
|
||||
test("the Open Food Facts API answer is shaped like an import row", async () => {
|
||||
let url;
|
||||
const fetchStub = async (address) => {
|
||||
url = address;
|
||||
return Response.json({
|
||||
status: 1,
|
||||
product: {
|
||||
product_name: "Diet Coke",
|
||||
brands: "Coca-Cola",
|
||||
serving_size: "1 can (355 ml)",
|
||||
serving_quantity: 355,
|
||||
nutriments: {
|
||||
"energy-kcal_100g": 0.4,
|
||||
sodium_100g: 0.012,
|
||||
caffeine_100g: 0.0129,
|
||||
},
|
||||
},
|
||||
});
|
||||
};
|
||||
const product = await fetchOffProduct("00049000028911", fetchStub);
|
||||
assert.match(url, /\/product\/0049000028911\.json/);
|
||||
assert.equal(product.source, "api");
|
||||
assert.equal(product.servingUnit, "ml");
|
||||
assert.equal(product.nutrients.sodium, 12);
|
||||
assert.ok(Math.abs(product.nutrients.caffeine - 12.9) < 1e-9);
|
||||
const missing = await fetchOffProduct("00049000028911", async () =>
|
||||
Response.json({ status: 0 }),
|
||||
);
|
||||
assert.equal(missing, null);
|
||||
});
|
||||
160
tests/scan-api.test.js
Normal file
160
tests/scan-api.test.js
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { createApp } from "../server/app.js";
|
||||
import { openDatabase } from "../server/db.js";
|
||||
import { hashPassword } from "../server/auth.js";
|
||||
import { openProducts } from "../server/products/db.js";
|
||||
|
||||
const password = "test-only-password-123";
|
||||
const origin = "http://localhost:3000";
|
||||
async function fixture(t, { fetchProduct, ocr } = {}) {
|
||||
const store = openDatabase(":memory:");
|
||||
const products = openProducts(":memory:");
|
||||
const app = createApp({
|
||||
store,
|
||||
username: "test",
|
||||
passwordHash: hashPassword(password),
|
||||
origin,
|
||||
secureCookies: false,
|
||||
dist: ".local/nonexistent",
|
||||
products,
|
||||
fetchProduct,
|
||||
ocr,
|
||||
});
|
||||
const server = app.listen(0, "127.0.0.1");
|
||||
await new Promise((resolve) => server.once("listening", resolve));
|
||||
const base = `http://127.0.0.1:${server.address().port}/api`;
|
||||
t.after(
|
||||
() =>
|
||||
new Promise((resolve) =>
|
||||
server.close(() => {
|
||||
store.close();
|
||||
products.close();
|
||||
resolve();
|
||||
}),
|
||||
),
|
||||
);
|
||||
const login = await fetch(base + "/login", {
|
||||
method: "POST",
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
Origin: origin,
|
||||
"X-Calio-Request": "1",
|
||||
},
|
||||
body: JSON.stringify({ username: "test", password }),
|
||||
});
|
||||
const cookie = login.headers.get("set-cookie").split(";")[0];
|
||||
const request = (path, init = {}) =>
|
||||
fetch(base + path, {
|
||||
...init,
|
||||
headers: {
|
||||
Origin: origin,
|
||||
"X-Calio-Request": "1",
|
||||
Cookie: cookie,
|
||||
...init.headers,
|
||||
},
|
||||
});
|
||||
return { products, request };
|
||||
}
|
||||
|
||||
test("barcode lookups use local data first, then Open Food Facts once", async (t) => {
|
||||
let calls = 0;
|
||||
const { products, request } = await fixture(t, {
|
||||
fetchProduct: async (gtin) => {
|
||||
calls++;
|
||||
return gtin === "05449000000996"
|
||||
? {
|
||||
gtin,
|
||||
name: "Coca-Cola",
|
||||
brand: "Coca-Cola",
|
||||
serving: "1 can (330 ml)",
|
||||
servingQty: 330,
|
||||
servingUnit: "ml",
|
||||
nutrients: { "energy-kcal": 42, sugars: 10.6 },
|
||||
source: "api",
|
||||
}
|
||||
: null;
|
||||
},
|
||||
});
|
||||
const usda = products.beginImport("usda");
|
||||
usda.add({
|
||||
gtin: "00016000275287",
|
||||
name: "CHEERIOS",
|
||||
brand: "General Mills",
|
||||
serving: "1 cup (28 g)",
|
||||
servingQty: 28,
|
||||
servingUnit: "g",
|
||||
nutrients: { "energy-kcal": 357, calcium: 357 },
|
||||
});
|
||||
usda.commit("test");
|
||||
|
||||
const local = await request("/products/016000275287");
|
||||
assert.equal(local.status, 200);
|
||||
const { food } = await local.json();
|
||||
assert.equal(food.barcode, "00016000275287");
|
||||
assert.equal(food.source, "usda");
|
||||
assert.equal(food.nutrients.calories, 100);
|
||||
assert.equal(food.extra.calcium, 100);
|
||||
assert.equal(calls, 0);
|
||||
|
||||
const remote = await request("/products/5449000000996");
|
||||
assert.equal((await remote.json()).food.serving, "1 can (330 ml)");
|
||||
await request("/products/5449000000996");
|
||||
assert.equal(calls, 1, "the API answer is cached");
|
||||
|
||||
assert.equal((await request("/products/4006381333931")).status, 404);
|
||||
assert.equal((await request("/products/4006381333931")).status, 404);
|
||||
assert.equal(calls, 2, "misses are cached too");
|
||||
assert.equal((await request("/products/12345")).status, 400);
|
||||
assert.equal(
|
||||
(
|
||||
await fetch(
|
||||
new URL("/api/products/016000275287", (await request("/health")).url),
|
||||
)
|
||||
).status,
|
||||
401,
|
||||
);
|
||||
});
|
||||
|
||||
test("an unreachable Open Food Facts is reported, not cached", async (t) => {
|
||||
const { request } = await fixture(t, {
|
||||
fetchProduct: async () => {
|
||||
throw new Error("offline");
|
||||
},
|
||||
});
|
||||
const response = await request("/products/5449000000996");
|
||||
assert.equal(response.status, 502);
|
||||
assert.match((await response.json()).error, /couldn’t be reached/);
|
||||
});
|
||||
|
||||
test("label photos are read and parsed", async (t) => {
|
||||
let received;
|
||||
const { request } = await fixture(t, {
|
||||
ocr: async (image) => {
|
||||
received = image;
|
||||
const row = (text, y) => ({ text, x0: 0, x1: 300, y0: y, y1: y + 20 });
|
||||
return [
|
||||
row("Serving size 1 cup (55g)", 0),
|
||||
row("Calories 230", 30),
|
||||
row("Total Fat 8g 10%", 60),
|
||||
row("Sodium 160mg 7%", 90),
|
||||
];
|
||||
},
|
||||
});
|
||||
const response = await request("/label", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "image/jpeg" },
|
||||
body: new Uint8Array([0xff, 0xd8, 0xff]),
|
||||
});
|
||||
assert.equal(response.status, 200);
|
||||
const { label } = await response.json();
|
||||
assert.equal(received.length, 3);
|
||||
assert.equal(label.serving, "1 cup (55 g)");
|
||||
assert.deepEqual(label.nutrients, { calories: 230, fat: 8, sodium: 160 });
|
||||
const wrongType = await request("/label", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "text/plain" },
|
||||
body: "hello",
|
||||
});
|
||||
assert.equal(wrongType.status, 415);
|
||||
});
|
||||
|
|
@ -15,7 +15,7 @@
|
|||
<Icon>https://git.elijahkuntz.com/Elijah/Calio/raw/branch/main/public/icons/icon-192.png</Icon>
|
||||
<Category>Home: Other:</Category>
|
||||
<Config Name="Web port" Target="3000" Default="3080" Mode="tcp" Description="Container HTTP port. Forward the public HTTPS reverse proxy to this host port." Type="Port" Display="always" Required="true" Mask="false"/>
|
||||
<Config Name="Appdata" Target="/data" Default="/mnt/user/appdata/calio" Mode="rw" Description="Persistent SQLite journal, sessions and backups. Keep this mapping on updates. Must be writable by 99:100." Type="Path" Display="always" Required="true" Mask="false"/>
|
||||
<Config Name="Appdata" Target="/data" Default="/mnt/user/appdata/calio" Mode="rw" Description="Persistent SQLite journal, sessions, backups and the regenerable barcode database (~1.6 GB). Keep this mapping on updates. Must be writable by 99:100." Type="Path" Display="always" Required="true" Mask="false"/>
|
||||
<Config Name="Public HTTPS origin" Target="APP_ORIGIN" Default="" Description="Required: exact public HTTPS origin, for example https://calio.example.com. No trailing slash." Type="Variable" Display="always" Required="true" Mask="false"/>
|
||||
<Config Name="Username" Target="CALIO_USERNAME" Default="calio" Description="The one account used to sign in." Type="Variable" Display="always" Required="true" Mask="false"/>
|
||||
<Config Name="Password" Target="CALIO_PASSWORD" Default="" Description="Required: a unique password of at least 12 characters. Set only at runtime, never in the image or repository." Type="Variable" Display="always" Required="true" Mask="true"/>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue