Varied question openers, readable rationales, plain practice score
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s

- Stems no longer all open the same way: the writing rules, batch prompt and
  planner ask for varied openings (age openings stay part of the mix), and when
  one opener covers more than a quarter of a quiz the excess stems have only
  their opening reworded, keeping every number, before the check.
- Rationales are asked for as Markdown (a bold key point, then short bullets);
  older one-paragraph rationales are laid out the same way on display.
- The practice score in the quiz header is a plain "x/y correct", with no
  percentage or colour.
- Fix a flaky assertion: the mock's answer-key fix is a no-op when the shuffle
  already put the answer in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Elijah 2026-11-16 12:05:46 -07:00
parent 10be5c39da
commit 517278be71
15 changed files with 382 additions and 24 deletions

View file

@ -54,7 +54,7 @@ async function sse(url: string, body: unknown): Promise<any[]> {
const isCheck = (r: { body: any }) => (r.body?.tools ?? []).some((t: any) => t.function?.name === 'submit_review');
/** The quick call that groups topic labels after a quiz is written. */
const isGrouping = (r: { body: any }) => (r.body?.tools ?? []).some((t: any) => t.function?.name === 'submit_topics');
const isNaming = (r: { body: any }) => ['name_set', 'submit_proofread', 'plan_questions'].includes(r.body?.tool_choice?.function?.name);
const isNaming = (r: { body: any }) => ['name_set', 'submit_proofread', 'plan_questions', 'submit_stems'].includes(r.body?.tool_choice?.function?.name);
async function waitForDraft(id: string) {
for (let i = 0; i < 200; i += 1) {
@ -213,7 +213,12 @@ describe('generation', () => {
expect(pythonResult.ok).toBe(true);
expect(pythonResult.stdout).toMatch(/^12 12 \d+ \[(?:[12], ){11}[12]\]\n$/);
expect(draft.payload.questions[1].options.map((o: any) => o.correct)).toEqual([false, true, false, false]);
expect(draft.meta.notes.join(' ')).toMatch(/Checked with Python \(1 run\): 2 of 2 checks passed, 1 fix applied/);
// The mock's fix keys option 2; when the shuffle already put the answer there it changes
// nothing and is skipped, so "1 fix applied" only shows when it did something.
expect(draft.meta.notes.join(' ')).toMatch(/Checked with Python \(1 run\): 2 of 2 checks passed(?:, 1 fix applied|\.)/);
// Every mock stem opens "Mock question…": the excess ones were reworded (stem only) before the check.
expect(draft.meta.notes.join(' ')).toMatch(/Reworded how \d+ questions open/);
expect(draft.payload.questions.filter((q: any) => q.stem.startsWith('On the unit, Mock question')).length).toBeGreaterThan(4);
expect(JSON.stringify(calls[0].body.messages)).toMatch(/SOURCE EXCERPTS/);
// The questions are planned first; both batches then run at the same time, each with its own planned angles.
expect(all.filter((r) => r.body?.tool_choice?.function?.name === 'plan_questions')).toHaveLength(1);