Keep rationales on what the question tests
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
Med math rationales were padded with unrelated drug facts (TSH and heart rate for a levothyroxine tablet calculation) because the math prompt asked for "one safety check" and the rationale format asked for at least two bullets. - Rationales take 1 to 4 bullets, each about what the question tests, with fewer bullets preferred to padding. - Med math rationales explain only the calculation: the conversion, the method and an arithmetic check. - The second review rewrites a rationale or "why" that brings in facts the question does not test. - The outside-chatbot instructions say the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
0ad6290250
commit
fe94dbf876
4 changed files with 11 additions and 6 deletions
|
|
@ -109,9 +109,13 @@ Select-all items have exactly 6 choices with 2 to 5 correct, most often 3 or 4.
|
|||
appear in the batch prompt, the planner, the review and proofread rewrites, "improve questions"
|
||||
and the outside-chatbot instructions.
|
||||
|
||||
**Rationales** are asked for as Markdown: one bold key sentence, then 2 to 4 short bullet points,
|
||||
**Rationales** are asked for as Markdown: one bold key sentence, then 1 to 4 short bullet points,
|
||||
without repeating the per-choice "why" (`RATIONALE_FORMAT`, also in the question schema, the
|
||||
review's rewrite schema, "improve questions" and the outside-chatbot instructions). The web app
|
||||
review's rewrite schema, "improve questions" and the outside-chatbot instructions). Every bullet
|
||||
must be about what the question tests; fewer bullets are preferred to padding with monitoring or
|
||||
side effects. Med math rationales explain only the calculation (conversion, method and an
|
||||
arithmetic check), and the second review rewrites a rationale that brings in facts the question
|
||||
does not test (such as TSH monitoring in a levothyroxine tablet calculation). The web app
|
||||
lays out older one-paragraph rationales the same way when it shows them (`formatRationale`).
|
||||
|
||||
Then a **second review**: a fresh model call with no Python reads the whole quiz as a careful
|
||||
|
|
|
|||
|
|
@ -346,7 +346,7 @@ export function plainChoice(text: string): string {
|
|||
|
||||
/** How a rationale is laid out, so it reads at a glance instead of as one block of text. */
|
||||
const RATIONALE_FORMAT =
|
||||
'Format the rationale in Markdown for quick reading: one bold sentence with the key point, then 2 to 4 short bullet points (one idea each, such as the principle or framework, the physiology or drug action, the safety point). No long paragraphs, and do not repeat the choices\' "why" text, which is shown beside each choice.';
|
||||
'Format the rationale in Markdown for quick reading: one bold sentence with the key point, then 1 to 4 short bullet points (one idea each, such as the principle or framework, the physiology or drug action, the safety point). Every bullet must be about what this question tests: never add monitoring, lab values, side effects or other facts the question does not ask about just to fill out the list, and use fewer bullets rather than pad. No long paragraphs, and do not repeat the choices\' "why" text, which is shown beside each choice.';
|
||||
|
||||
const QUESTION_SCHEMA = {
|
||||
type: 'object',
|
||||
|
|
@ -955,7 +955,7 @@ async function runGeneration(app: AppContext, draftId: string, req: GenerateRequ
|
|||
`Write ${want} medication math (dosage calculation) questions, all of type "numeric": the student types the number.`,
|
||||
`Difficulty ${difficulty}: ${MATH_LEVELS[difficulty] ?? MATH_LEVELS.mixed}.`,
|
||||
'These do not need to come from the course material: write standard nursing med math with realistic orders, labels and client weights, using generic drug names (drugs from the material are welcome where they fit).',
|
||||
'Each stem gives the order and what is on hand, and states the unit and the rounding rule ("Round to the nearest tenth."). Exactly one option: the correct answer with its unit, marked correct, with "why" showing the working step by step (dimensional analysis or desired over have). The rationale explains the method and one safety check.',
|
||||
'Each stem gives the order and what is on hand, and states the unit and the rounding rule ("Round to the nearest tenth."). Exactly one option: the correct answer with its unit, marked correct, with "why" showing the working step by step (dimensional analysis or desired over have). The rationale explains only the calculation: the conversion, the method (dimensional analysis or desired over have) and a quick check that the answer is reasonable for the arithmetic (for example "75 mcg is three 25 mcg tablets"). Do not add drug monitoring, lab values, side effects or dose-limit checks unless the stem asks about them.',
|
||||
'Set the topic to "Dosage calculation". Vary the problem types.',
|
||||
]
|
||||
: isQuiz
|
||||
|
|
|
|||
|
|
@ -460,7 +460,8 @@ Flag a question only for a real problem:
|
|||
- the writer changed its mind or left a note in the text (for example "A 78 kg? No — an 80 kg client", "Actually,", "wait"), or the wording is garbled, truncated or repeated;
|
||||
- data needed to answer is missing, or the stem contradicts itself, the options or the rationale;
|
||||
- two options are defensible, or none is correct; the rationale disagrees with the keyed answer;
|
||||
- an option gives the answer away, or the content is clinically wrong or outdated.
|
||||
- an option gives the answer away, or the content is clinically wrong or outdated;
|
||||
- the rationale or a choice's "why" brings in facts the question does not test (for example lab monitoring or dose limits in a plain dosage calculation): rewrite it to explain only what the question asks.
|
||||
For each problem give the complete corrected question with rewrite (same type, the same clinical point, following the rules for its type: single = 4 options with exactly one correct; multi = 6 options with 2 to 5 correct and a stem ending "Select all that apply."; a short plain stem; choices concise and parallel, with no explanation tacked on; numeric = one option, the correct answer with its unit, and "why" showing the working), or remove it if it cannot be saved. Do not flag style preferences or answer order. Call submit_proofread once; an empty list is the right answer when every question is sound.`;
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -638,7 +638,7 @@ Rules:
|
|||
- Use "numeric" for calculations (dosages, IV rates, drip rates, conversions, intake and output). Its "options" holds only the correct answer, e.g. [{ "text": "12.5 mL", "correct": true, "why": "Show the calculation" }]. The stem states the unit and how to round, e.g. "Round to the nearest tenth." Only the number is checked, so units in the answer are optional.
|
||||
- Write NCLEX-style questions that test clinical judgment: a short, plain stem of usually 15 to 30 words with only the cues needed (vary how stems open so they do not all start the same way, and use no bold in stems), then one direct question such as "Which action should the nurse take first?" or "Which statement by the parent indicates a need for further teaching?". About a third may ask directly about a fact the nurse uses ("Which injection site is most appropriate?"). Never refer to the material itself in a question.
|
||||
- Answer choices are concise and parallel: a term or value when the question asks for one, otherwise a short action or finding (about 4 to 12 words), or one or two sentences of quoted speech for client statements and nurse responses. No explanation tacked on, and the correct one never stands out as the longest or most detailed. Build distractors from related facts (a similar condition, a different stage, another drug or test) so they are plausible but wrong for a specific reason. Never use "all of the above" or "none of the above".
|
||||
- Every option needs a "why" of 1 or 2 sentences with its specific clinical reason. The "rationale" explains the principle behind the correct answer, laid out for quick reading: one bold key sentence, then 2 to 4 short bullet points (use \\n between lines).
|
||||
- Every option needs a "why" of 1 or 2 sentences with its specific clinical reason. The "rationale" explains the principle behind the correct answer, laid out for quick reading: one bold key sentence, then 1 to 4 short bullet points (use \\n between lines), each about what the question tests. Do not pad it with monitoring or facts the question does not ask about.
|
||||
- "topic" is a short label. Reuse the same label for related questions and keep the number of different topics small.
|
||||
- "difficulty" is "easy", "medium" or "hard".
|
||||
- Markdown is allowed inside strings (bold, lists, tables). Write line breaks inside strings as \\n.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue