Varied question openers, readable rationales, plain practice score
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
- Stems no longer all open the same way: the writing rules, batch prompt and planner ask for varied openings (age openings stay part of the mix), and when one opener covers more than a quarter of a quiz the excess stems have only their opening reworded, keeping every number, before the check. - Rationales are asked for as Markdown (a bold key point, then short bullets); older one-paragraph rationales are laid out the same way on display. - The practice score in the quiz header is a plain "x/y correct", with no percentage or colour. - Fix a flaky assertion: the mock's answer-key fix is a no-op when the shuffle already put the answer in place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
10be5c39da
commit
517278be71
15 changed files with 382 additions and 24 deletions
|
|
@ -54,7 +54,7 @@ async function sse(url: string, body: unknown): Promise<any[]> {
|
|||
const isCheck = (r: { body: any }) => (r.body?.tools ?? []).some((t: any) => t.function?.name === 'submit_review');
|
||||
/** The quick call that groups topic labels after a quiz is written. */
|
||||
const isGrouping = (r: { body: any }) => (r.body?.tools ?? []).some((t: any) => t.function?.name === 'submit_topics');
|
||||
const isNaming = (r: { body: any }) => ['name_set', 'submit_proofread', 'plan_questions'].includes(r.body?.tool_choice?.function?.name);
|
||||
const isNaming = (r: { body: any }) => ['name_set', 'submit_proofread', 'plan_questions', 'submit_stems'].includes(r.body?.tool_choice?.function?.name);
|
||||
|
||||
async function waitForDraft(id: string) {
|
||||
for (let i = 0; i < 200; i += 1) {
|
||||
|
|
@ -213,7 +213,12 @@ describe('generation', () => {
|
|||
expect(pythonResult.ok).toBe(true);
|
||||
expect(pythonResult.stdout).toMatch(/^12 12 \d+ \[(?:[12], ){11}[12]\]\n$/);
|
||||
expect(draft.payload.questions[1].options.map((o: any) => o.correct)).toEqual([false, true, false, false]);
|
||||
expect(draft.meta.notes.join(' ')).toMatch(/Checked with Python \(1 run\): 2 of 2 checks passed, 1 fix applied/);
|
||||
// The mock's fix keys option 2; when the shuffle already put the answer there it changes
|
||||
// nothing and is skipped, so "1 fix applied" only shows when it did something.
|
||||
expect(draft.meta.notes.join(' ')).toMatch(/Checked with Python \(1 run\): 2 of 2 checks passed(?:, 1 fix applied|\.)/);
|
||||
// Every mock stem opens "Mock question…": the excess ones were reworded (stem only) before the check.
|
||||
expect(draft.meta.notes.join(' ')).toMatch(/Reworded how \d+ questions open/);
|
||||
expect(draft.payload.questions.filter((q: any) => q.stem.startsWith('On the unit, Mock question')).length).toBeGreaterThan(4);
|
||||
expect(JSON.stringify(calls[0].body.messages)).toMatch(/SOURCE EXCERPTS/);
|
||||
// The questions are planned first; both batches then run at the same time, each with its own planned angles.
|
||||
expect(all.filter((r) => r.body?.tool_choice?.function?.name === 'plan_questions')).toHaveLength(1);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue