Add sandboxed Python for the assistant and a Python check of generated quizzes
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 51s

- Pyodide (Python 3.14, WebAssembly) in a separate Node process under the permission
  model: read-only access to Pyodide's files, no writes, processes or network, empty
  environment, JS bridges removed, time limit with kill and restart.
- Assistant tool run_python, which can load a draft or quiz as Python data.
- After quiz generation the model checks question count, type mix, correct answers per
  type, duplicates and recomputed calculations with Python, then submits fixes that are
  applied only if the question still validates. Switch in the Generate dialog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Elijah 2026-11-15 13:03:24 -07:00
parent f0aabe4d5d
commit 5ea61c882c
13 changed files with 800 additions and 6 deletions

View file

@ -21,7 +21,9 @@ import { dueCounts } from '../services/srs.ts';
import { topicStats, totals } from '../services/stats.ts';
import { newId, now, parseJson } from '../util.ts';
import { cancelGeneration, createDraft, getDraft, startGeneration, startImprove } from './generate.ts';
import { runPython } from './sandbox.ts';
import { resolveModel, runChat } from './service.ts';
import { pythonOutput, pythonPrelude, quizForPython, RUN_PYTHON_DESCRIPTION } from './verify.ts';
import { AIError, type ChatMessage, type ContentPart, type ToolDef } from './types.ts';
/* ------------------------------ UI event model ------------------------------ */
@ -605,6 +607,36 @@ const TOOLS: ToolSpec[] = [
return { output: { folder_id: folder.id, name: folder.name }, summary: folder.name };
},
},
{
def: {
name: 'run_python',
description: `${RUN_PYTHON_DESCRIPTION} Pass draft_id or quiz_id to load that draft or quiz as the variable \`draft\` / \`quiz\` (dict with "questions": [{"number", "type", "stem", "options": [{"number", "text", "correct"}], "rationale"}]); decks load as {"cards": [{"front", "back"}]}.`,
parameters: obj(
{
code: { ...S, description: 'Python 3 source. Print the results you need.' },
draft_id: { ...S, description: 'Optional: load this draft as `draft`.' },
quiz_id: { ...S, description: 'Optional: load this saved quiz as `quiz`.' },
},
['code'],
),
},
label: () => 'Running Python',
run: async (input, env) => {
const { db } = env.app;
const vars: Record<string, unknown> = {};
if (typeof input.draft_id === 'string' && input.draft_id) {
const d = getDraft(db, input.draft_id);
vars.draft = d.payload.kind === 'quiz' ? { ...quizForPython(d.payload), request: d.meta.request ?? {} } : d.payload.kind === 'deck' ? { title: d.payload.title, cards: d.payload.cards.map((c) => ({ front: c.front, back: c.back })) } : { title: d.payload.title, content: d.payload.content };
}
if (typeof input.quiz_id === 'string' && input.quiz_id) {
const quiz = getQuiz(db, input.quiz_id);
vars.quiz = quizForPython({ title: quiz.title, questions: listQuestions(db, quiz.id).map((q) => ({ ...q, options: q.options.map((o) => ({ text: o.text, correct: o.correct, why: o.why })) })) });
}
const r = await runPython((Object.keys(vars).length ? pythonPrelude(vars) : '') + String(input.code ?? ''));
const firstLine = (r.error ?? '').trim().split('\n').pop() ?? '';
return { output: pythonOutput(r), summary: r.ok ? (r.stdout.trim().split('\n').pop() ?? r.result ?? 'done').slice(0, 120) : firstLine.slice(0, 160) };
},
},
{
def: {
name: 'propose_changes',
@ -896,7 +928,8 @@ function systemPrompt(app: AppContext, context: AssistantContext): string {
- For "what should I study" questions, call get_performance and give specific, prioritised advice.
- When a request is clear, act on it without asking for permission. Ask one short question only when a missing detail would change the result (for example which class, when it is ambiguous).
- Be concise and concrete. Use Markdown. Teach like a good clinical instructor: explain the why, connect to patient care, and prefer current standard practice.
- After a tool produces a draft or a session link, reply with one or two sentences; the card already shows the details.`,
- After a tool produces a draft or a session link, reply with one or two sentences; the card already shows the details.
- Use run_python whenever exactness matters: dosage and rate calculations, unit conversions, counting questions, options or correct answers, checking that a quiz meets what was asked for. Quizzes from generate_quiz are already checked this way; when you write questions yourself with create_draft, check the draft with run_python (pass its draft_id) before telling the user it is ready, and fix problems you find.`,
);
return lines.join('\n\n');
}