Study/docs/AI.md
Elijah 031f7bec31
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
Second revision: MCP servers, shared-link AI, flashcard modes, document previews
Generation
- Every quiz is NCLEX style; "Practice exam" writes 50 questions.
- Calculation questions only when the material involves calculations (max 5 per exam).
- Broad topics: the class's topics are listed in the prompt, a quick grouping call maps
  narrow labels onto broad ones, and leftover variants are merged.
- Exact select-all count per batch; option order shuffled.
- Python check: at most 3 runs, fixed problems count as passed, cosmetic remarks ignored.

Assistant
- Text, tool steps and cards appear in the order they happen, with paragraph breaks
  between steps.
- Optional thinking box: fixed height, follows the newest text, can be scrolled back.
- "+" opens a material picker listing recent items; items can be dragged in from class
  pages; shorter placeholder.

MCP servers
- Streamable HTTP client; encrypted headers.
- Per-server instructions and switches for chat, Explain, generation and the quiz check.
- Settings UI.

Share links
- Optional "Explain this" and "Ask about this material" per link, off by default.
- Items are loaded on the server and checked against the share; no tools; rate-limited.
- Migration 4 stores the switches.

Study and documents
- Standard flashcard mode (optionally repeat missed cards until known) next to spaced
  repetition.
- Streak visuals restyled in the app palette.
- Side by side / Original document views, with sandboxed previews of Word and PowerPoint
  files.

Other
- Built-in embedding model baked into the image; sharp stubbed if it cannot load; real
  load error shown in Settings.
- Python runner blocks network and process modules in JavaScript too.
- "View questions and answers" removed from the quiz start screen.
- Docs updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 15:48:54 -07:00

159 lines
9.9 KiB
Markdown

# AI connections and verification
Settings → AI manages provider connections and model routing. Add a connection, test it,
discover models where supported (or add a custom model), enable models and select defaults
for generation, chat, explanations and vision. Output limits, capabilities, pricing and
reasoning settings can be edited per model. Catalog prices are estimates, not billing records.
Adapters use OpenAI Responses, Anthropic Messages, Gemini generateContent, DeepSeek and
OpenAI-compatible Chat Completions, and Ollama's native chat API.
Reasoning effort has seven levels (Off, Minimal, Low, Medium, High, Extra high, Max); each
adapter maps them onto the nearest value its API accepts. DeepSeek accepts low / high / max
and turns thinking off at "Off". DeepSeek rejects forced tool calls in thinking mode, so
structured generation keeps thinking on and asks for the tool call in the prompt; answer keys
were noticeably less reliable with thinking off. Its `reasoning_content` is replayed between
tool steps, as the API requires.
Cost estimates split input into fresh, cache-hit and cache-write tokens. A model's "Cached
input price" is used when set; otherwise the provider's published discount applies (10% of
input for GPT-5+, Claude and Gemini; DeepSeek's catalog cache-hit price). DeepSeek's
off-peak half price is applied by time of call. Configure the endpoint as shown in Settings.
Local models are reached **from the server/container**: `localhost` inside Docker is the
container itself, not your workstation or Unraid host. Use a reachable LAN address instead.
Keys are encrypted with AES-256-GCM in SQLite using `/data/secret.key` or `APP_SECRET`.
Only the server decrypts them. Keep the key with backups; changing it breaks decryption.
Documents are sent to the selected provider when you request AI processing; extraction and
full-text indexing otherwise run locally. Review generated nursing content and source
references before using it for study.
## Reading scanned PDFs (OCR)
A PDF with little selectable text is read by the "Vision and OCR" model, which must accept
PDFs (OpenAI, Anthropic, Gemini; OpenAI-compatible servers when the model is marked "PDF").
Long files are split with pdf-lib into batches of 8 pages (smaller if a batch exceeds 24 MB)
and sent one batch per request, so there is no whole-file size cap. Each batch keeps its real
page numbers for citations. Finished batches are cached beside the upload
(`uploads/<file>.ocr.json`), so a restart or a failed batch does not repeat paid work; "Read
with AI" on the document forces a fresh read. A batch that fails twice is skipped with a
warning on the document. At most 600 pages are read per file.
## Semantic search
Settings → AI → Semantic search chooses an embeddings provider and model (suggested:
OpenAI `text-embedding-3-small`, Gemini `gemini-embedding-001`, Ollama `nomic-embed-text`;
Anthropic and DeepSeek have no embeddings API). Until something else is chosen, the built-in
model is used: BAAI bge-small-en-v1.5 (8-bit ONNX, 384 dimensions) running on the server's CPU
through transformers.js. The Docker image ships the model in `/app/models` (`LOCAL_MODELS_DIR`),
so it works without internet access; outside Docker it is downloaded once (about 35 MB) from
huggingface.co into `DATA_DIR/models`. It costs nothing. If the model cannot load, Settings shows
the actual error. `LOCAL_EMBEDDINGS=off` disables the default. Every document passage is embedded in the background and
stored as a vector in SQLite (`chunk_vectors`). Search, the assistant's document search and
focused generation then merge keyword (FTS5) and meaning-based rankings with reciprocal-rank
fusion, so exact terms still match while paraphrased questions find the right passage. If the
embedding provider fails, search silently falls back to keywords. Changing the model clears
and rebuilds the index. Costs appear under Usage as "semantic indexing" / "semantic search".
Quiz questions, flashcards and notes are still searched by keyword.
## Python sandbox and the quiz check
The assistant has a `run_python` tool, and generated quizzes are checked with it. Code runs in
Pyodide (CPython compiled to WebAssembly) inside a separate Node process started with
`--permission`. It can read only the Pyodide files, gets an empty environment, cannot reach
the network or start processes (blocked in Python and again in JavaScript), and has no access
to `/data`. One warm process is reused and recycled every 50 runs; a run that times out kills
it.
After a quiz is generated (unless "Check answers" is off), the generation model gets the draft
as Python data plus the request (count, select-all share, calculation allowance). In at most
three Python runs it checks the count, question types, number of correct answers per type,
numeric answers, the calculation allowance and topic breadth, then reports with `submit_review`.
Fixes are applied only when the question stays valid. Remarks the model files as cosmetic or
informational (answer position, quotes) are not counted as failed checks; answer order is
shuffled whenever a quiz is taken.
## Quiz generation rules
Every quiz is NCLEX style. "Practice exam" writes 50 questions. Calculation (numeric) questions
are written only when the source material involves calculations (dosage, drip rates, mg/kg,
intake and output...): up to 5 in an exam, about one in ten otherwise, none when the material
has no calculations. Topics are kept broad ("Stroke", not "Right hemisphere stroke"): the
prompt lists the class's existing topics, and narrower labels that contain an existing one are
merged into it after generation.
## MCP servers
Settings → AI → MCP servers connects Model Context Protocol servers over Streamable HTTP
(JSON-RPC over POST, answered as JSON or SSE). Headers (usually an `Authorization` token) are
encrypted like API keys and never returned to the browser. "Test and list tools" connects
afresh and lists the tools.
Each server has:
- **When to use it**: your instructions, added to the model's prompt next to the server's tools.
- **Where it can be used**: Assistant chat, Explain this, Generating material (quizzes,
flashcards, study guides) and Quiz check, each switched separately. New servers are on for
chat and Explain only.
Tools are offered as `mcp_<server>__<tool>`, at most 48 per request. A server that cannot be
reached is skipped for that request, and the assistant is told it is unavailable. During
generation the model may make up to four rounds of tool calls before it must submit its
answer; the quiz check allows four external calls. Tool output is treated as data, not
instructions. MCP tools are never offered on share links.
## AI on share links
Each share link has two switches, both off by default:
- **Explain this**: visitors can ask for an explanation of a shared question or flashcard.
The request names the item by id; the server loads its text (and checks it belongs to the
share), so a link cannot be used to run arbitrary prompts.
- **Ask about this material**: a chat that answers from the shared item (or, for a shared
class or folder, the item being viewed, the list of materials and passages from its
documents). It has no tools, cannot read the rest of the library and saves nothing.
Both use the owner's models and keys, and are rate-limited to 20 requests per visitor per 10
minutes and 150 per link per hour. They show only when an AI model is configured.
## Original documents
A document's page offers Extracted text, Side by side and Original. PDFs and images show the
file itself. Word and PowerPoint files are shown as a generated page
(`GET /api/documents/:id/preview`): the document's content, or each slide's text, pictures and
notes. Pictures are inlined; formats a browser cannot show (EMF/WMF) are listed as omitted. The
page is served with `default-src 'none'; sandbox`, so it cannot run scripts or load anything.
Pictures inside Word and PowerPoint files are **not read** for search or generation: only
their text, tables and speaker notes are extracted. PDFs are read by OCR only when they have
almost no selectable text, so pictures inside a text PDF are not read either. Material that
lives only in pictures needs to be in the text, or uploaded as an image or scanned PDF and
read with AI.
## ChatGPT account investigation (2026-11-11)
The official [Sign in with ChatGPT quickstart](https://developers.openai.com/siwc/quickstart)
now describes plan access for eligible integrations. The
[official implementation guide](https://developers.openai.com/cookbook/articles/sign-in-with-chatgpt)
distinguishes local personal/open-source tools from remotely hosted apps, which require
access approval. This Unraid/reverse-proxy deployment is a hosted web application; eligibility
for its private client has not been established. Therefore V2 currently uses conventional API
connections, and does not advertise ChatGPT plan access or reuse another client's credentials.
To add this later, obtain the appropriate approval/registration and implement the documented
OAuth/PKCE flow, verified identity, encrypted refresh tokens, approved callbacks and plan-scoped
model discovery. Keep this separate from the single administrator's local login. Never copy
ChatGPT browser session tokens or a Codex client's registration/credentials into Study Desk.
## What has been tested
Mock protocol tests cover request translation, streams, tools, usage and error handling.
Integration tests use a local OpenAI-compatible mock to generate/edit/save drafts, search
documents with citations and exercise confirmed assistant actions, batched OCR of a 20-page
scanned PDF and semantic indexing/search (the mock returns deterministic word-hash vectors).
They cost no API credits.
No successful real-provider inference or real ChatGPT OAuth authorization is claimed.
For manual local AI testing: `npx tsx scripts/mock-ai.ts` starts a scripted mock at
`http://localhost:4010/v1` with key `mock-key`. It returns clearly labeled synthetic material.
Do not use mock output as nursing guidance. Stop the mock and remove its connection after QA.