Natural reworded openers and exam formatting on finished quizzes
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 29s

Checked against a quiz generated from four neuro and mental health
lectures:
- The opener rewording now asks for the plain, active openings a real
  exam uses and rules out inverted sentences ("Two days of vomiting have
  left a client weak", "An order is received for…"). It steps in only
  above 30% of a quiz, since the exam's most common opener was 24%.
- The finished set is tidied after review: plain stems, no closing
  period on phrase choices, and first-person choices to a "Which
  statement…" question in quotes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Elijah 2026-11-16 15:17:24 -07:00
parent 39a85ebb0c
commit f738fbe7be
5 changed files with 68 additions and 15 deletions

View file

@ -219,7 +219,7 @@ stored DeepSeek connection still holding the old key; it was updated from `.env`
## Verification status
Baseline: 162 tests passed, TypeScript passed and production web/server builds passed.
Added six regression tests. Full `npm run check` passes: **291 tests** (exam-style questions, 2026-11-16; 290 after V1 importer removed; 291 after full-backup restore; 289 after rationale layout; 285 after varied openers; 281 after generation steps; 280 after the check loop and second review; 274 after the fifth revision; 261 after share-link generation, 2026-11-15; 257 after the fourth revision; 247 after the third, 208 after the second, 181 on 2026-11-12), TypeScript
Added six regression tests. Full `npm run check` passes: **293 tests** (exam formatting tidy and natural openers, 2026-11-16; 291 after exam-style questions; 290 after V1 importer removed; 291 after full-backup restore; 289 after rationale layout; 285 after varied openers; 281 after generation steps; 280 after the check loop and second review; 274 after the fifth revision; 261 after share-link generation, 2026-11-15; 257 after the fourth revision; 247 after the third, 208 after the second, 181 on 2026-11-12), TypeScript
and both production builds. The Docker `test` stage runs the same 181 tests on Node 24. Tests were run with loopback access for the mock provider.
The earlier agent manually exercised quiz completion/results, flashcards, streak effects,

View file

@ -86,8 +86,8 @@ checks; answer order is shuffled whenever a quiz is taken.
Before the check, **openers are varied** (`server/ai/openers.ts`). Models tend to open nearly
every stem the same way ("A 72-year-old client…"). The first two words of each stem are compared,
with ages and numbers folded together. When one opener covers more than a quarter of a quiz of 8 or
more, a spread-out quarter is kept. The rest have only their opening reworded (`submit_stems`); the
with ages and numbers folded together. When one opener covers more than 30% of a quiz of 8 or
more, a spread-out 30% is kept (a real course exam's most common opener covered about a quarter). The rest have only their opening reworded (`submit_stems`); the
options, answer and rationale stay the same. A rewording is kept only when it holds every number
from the original, keeps "Select all that apply." and no longer opens the repeated way.
Calculations keep their wording. The writing rules, the batch prompt (which names openers already
@ -98,8 +98,10 @@ Templates as "NCLEX style"). Stems are short and plain, usually 15 to 30 words:
needed, then one direct question in the exam's own phrasing ("Which action should the nurse take
first?", "Which statement by the parent indicates a need for further teaching?"), in one
paragraph with no scene-setting that does not change the answer. About a third may ask directly
about a fact the nurse uses (an injection site, a pain tool, expected lab results). Bold or
underlined words are removed from generated stems (`plainStem`). Choices are concise and
about a fact the nurse uses (an injection site, a pain tool, expected lab results). The finished
set is tidied to match the exam's formatting: bold or underlined words are removed from stems
(`plainStem`), a phrase choice loses its closing period (`plainChoice`), and first-person choices
to a "Which statement…" question are put in quotes (`quoteStatement`). Choices are concise and
parallel: a term or value when that is what is asked, a 4 to 12 word action or finding, or quoted
speech for client statements and nurse responses. Explanations go in each choice's "why" (1 or
2 sentences), not on the end of the choice. Distractors come from related facts in the material.

View file

@ -327,6 +327,23 @@ export function plainStem(stem: string): string {
return stem.replace(/\*\*(\S(?:[^*]*?\S)?)\*\*/g, '$1').replace(/__(\S(?:[^_]*?\S)?)__/g, '$1');
}
/** A first-person choice to a "Which statement…" question is quoted speech, as on the exam. */
export function quoteStatement(text: string): string {
const t = text.trim();
if (/^["“‘']/.test(t) || !/^(?:I|I'm|I'll|I've|I'd|We|We'll|We're|My|Our)\b/.test(t)) return t;
return `"${/[.!?]$/.test(t) ? t : `${t}.`}"`;
}
/**
* Exam choices are phrases without a closing period ("Place the child in a knee-chest position").
* Quoted speech and choices of more than one sentence keep their punctuation.
*/
export function plainChoice(text: string): string {
const t = plainStem(text.trim());
if (/^["“‘']/.test(t) || /[.!?]\s/.test(t) || /\b(?:etc|e\.g|i\.e|vs)\.$/i.test(t)) return t;
return t.replace(/(?<!\.)\.$/, '');
}
/** How a rationale is laid out, so it reads at a glance instead of as one block of text. */
const RATIONALE_FORMAT =
'Format the rationale in Markdown for quick reading: one bold sentence with the key point, then 2 to 4 short bullet points (one idea each, such as the principle or framework, the physiology or drug action, the safety point). No long paragraphs, and do not repeat the choices\' "why" text, which is shown beside each choice.';
@ -1217,6 +1234,14 @@ async function runGeneration(app: AppContext, draftId: string, req: GenerateRequ
}
}
// Exam formatting on the finished set, including questions the review rewrote.
if (draft.kind === 'quiz') {
draft.questions = draft.questions.map((q) => {
const speech = /\bstatements?\b/i.test(q.stem);
return { ...q, stem: plainStem(q.stem), options: q.type === 'numeric' ? q.options : q.options.map((o) => ({ ...o, text: plainChoice(speech ? quoteStatement(o.text) : o.text) })) };
});
}
// A short name and description of what this particular set covers.
if (!req.title) {
setProgress(db, draftId, 0.99, 'Naming the set', draft);

View file

@ -22,8 +22,11 @@ export function stemOpener(stem: string): string {
.join(' ');
}
/** Openers used by more than a quarter of the questions (at least 4), and which of them to reword. */
export function repeatedOpeners(questions: Pick<DraftQuestion, 'stem' | 'type'>[], share = 0.25): { opener: string; example: string; total: number; reword: number[] }[] {
/**
* Openers used by more than 30% of the questions (at least 4), and which of them to reword. A real
* course exam's most common opener ("A 4-year-old…") covered about a quarter of its items.
*/
export function repeatedOpeners(questions: Pick<DraftQuestion, 'stem' | 'type'>[], share = 0.3): { opener: string; example: string; total: number; reword: number[] }[] {
if (questions.length < 8) return [];
const allow = Math.max(3, Math.floor(questions.length * share));
const groups = new Map<string, number[]>();
@ -71,11 +74,12 @@ export function acceptRewording(original: Pick<DraftQuestion, 'stem' | 'type'>,
return next;
}
const OPENER_SYSTEM = `You edit NCLEX-style questions for variety. Too many stems in this quiz open the same way. Reword only the opening of each listed stem so the quiz reads like a real exam, where items open in many ways, for example:
- with the nurse's situation: "The nurse is caring for…", "While reviewing the morning laboratory results, the nurse notes…", "After receiving report, the nurse…"
- with the setting or role: "On a medical-surgical unit…", "In the emergency department…", "The charge nurse is making assignments…", "During discharge teaching…"
- with a finding, order or report: "A client with heart failure reports…", "The health care provider prescribes…", "Two hours after surgery, a client's…"
- or with the question itself when the scenario is short: "Which finding in a client receiving… requires immediate follow-up?"
const OPENER_SYSTEM = `You edit NCLEX-style questions for variety. Too many stems in this quiz open the same way. Reword only the opening of each listed stem so the quiz reads like a real nursing exam. Real exam items open in plain, natural ways, for example:
- with the nurse: "The nurse is caring for a client who…", "The nurse is teaching a client who…", "The nurse assesses a client…", "The nurse receives report on…"
- with a family member, when the original involves one: "The parent of a client with…", "A family member asks…"
- with a time or event: "Two days after a stroke, a client…", "Immediately after a seizure, the client…", "During discharge teaching, a client…"
- with the question itself when the situation is short: "Which finding in a client receiving phenytoin requires immediate follow-up?"
Keep each stem the way a person writing an exam would say it, in the active voice with a person as the subject. Never invert the sentence to put a symptom, order or test result first ("Two days of vomiting have left a client weak", "An order is received for…", "Assessment of a client reveals…", "Expressive aphasia affects a client…", "Clear fluid is noted in…"). Do not add a setting or details that are not in the original.
Openings with the client's age are fine elsewhere in the quiz; these stems just need a different opening from the repeated one.
Keep everything the question tests: the same client facts, every number exactly (age, vital signs, lab values, doses), the same question asked, and an ending of "Select all that apply." where there is one. Include the age only where it is in the original, but it need not come first. Use "client", never "patient". Make the listed stems open differently from one another too. Call submit_stems once with every listed question.`;

View file

@ -10,7 +10,7 @@ describe('question openers', () => {
expect(stemOpener('The nurse is caring for…')).toBe('the nurse');
});
it('flags an opener used by more than a quarter of the quiz and keeps a spread-out share of it', () => {
it('flags an opener used by more than 30% of the quiz and keeps a spread-out share of it', () => {
const stems = [
...Array.from({ length: 16 }, (_, i) => q(`A ${40 + i}-year-old client with condition ${i} reports pain. Which action is first?`)),
...Array.from({ length: 4 }, (_, i) => q(`The nurse is caring for client ${i}. Which finding requires follow-up?`)),
@ -19,8 +19,8 @@ describe('question openers', () => {
expect(groups).toHaveLength(1);
expect(groups[0]!.opener).toBe('a #-year-old');
expect(groups[0]!.total).toBe(16);
// 20 questions allow 5 of one opener: 11 are reworded, the 5 kept are spread out.
expect(groups[0]!.reword).toHaveLength(11);
// 20 questions allow 6 of one opener: 10 are reworded, the 6 kept are spread out.
expect(groups[0]!.reword).toHaveLength(10);
expect(groups[0]!.reword).not.toContain(0);
expect(openerAdvice(stems)).toMatch(/"A 40-year-old client…"/);
});
@ -54,3 +54,25 @@ describe('plain stems', () => {
expect(plainStem('A dose of 2 * 3 mg is ordered.')).toBe('A dose of 2 * 3 mg is ordered.');
});
});
describe('plain choices', () => {
it('drops the closing period of a phrase but keeps quoted speech and longer text as written', async () => {
const { plainChoice } = await import('../server/ai/generate.ts');
expect(plainChoice('Sit the client upright and lower the legs.')).toBe('Sit the client upright and lower the legs');
expect(plainChoice('0.9% sodium chloride')).toBe('0.9% sodium chloride');
expect(plainChoice('"I will take my anticoagulant every day as prescribed."')).toBe('"I will take my anticoagulant every day as prescribed."');
expect(plainChoice('Stop the drug. Notify the provider.')).toBe('Stop the drug. Notify the provider.');
expect(plainChoice('Wait...')).toBe('Wait...');
});
});
describe('quoted statements', () => {
it('quotes first-person choices and leaves the rest alone', async () => {
const { quoteStatement } = await import('../server/ai/generate.ts');
expect(quoteStatement('I will turn him on his side and time how long it lasts')).toBe('"I will turn him on his side and time how long it lasts."');
expect(quoteStatement("We'll keep the follow-up appointment.")).toBe('"We\'ll keep the follow-up appointment."');
expect(quoteStatement('"I will call my provider."')).toBe('"I will call my provider."');
expect(quoteStatement('Increased appetite with weight gain')).toBe('Increased appetite with weight gain');
expect(quoteStatement('IV fluid boluses of normal saline')).toBe('IV fluid boluses of normal saline');
});
});