Keep rationales on what the question tests
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
All checks were successful
Verify and publish V2 container / build-and-push (push) Successful in 30s
Med math rationales were padded with unrelated drug facts (TSH and heart rate for a levothyroxine tablet calculation) because the math prompt asked for "one safety check" and the rationale format asked for at least two bullets. - Rationales take 1 to 4 bullets, each about what the question tests, with fewer bullets preferred to padding. - Med math rationales explain only the calculation: the conversion, the method and an arithmetic check. - The second review rewrites a rationale or "why" that brings in facts the question does not test. - The outside-chatbot instructions say the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
0ad6290250
commit
fe94dbf876
4 changed files with 11 additions and 6 deletions
|
|
@ -109,9 +109,13 @@ Select-all items have exactly 6 choices with 2 to 5 correct, most often 3 or 4.
|
|||
appear in the batch prompt, the planner, the review and proofread rewrites, "improve questions"
|
||||
and the outside-chatbot instructions.
|
||||
|
||||
**Rationales** are asked for as Markdown: one bold key sentence, then 2 to 4 short bullet points,
|
||||
**Rationales** are asked for as Markdown: one bold key sentence, then 1 to 4 short bullet points,
|
||||
without repeating the per-choice "why" (`RATIONALE_FORMAT`, also in the question schema, the
|
||||
review's rewrite schema, "improve questions" and the outside-chatbot instructions). The web app
|
||||
review's rewrite schema, "improve questions" and the outside-chatbot instructions). Every bullet
|
||||
must be about what the question tests; fewer bullets are preferred to padding with monitoring or
|
||||
side effects. Med math rationales explain only the calculation (conversion, method and an
|
||||
arithmetic check), and the second review rewrites a rationale that brings in facts the question
|
||||
does not test (such as TSH monitoring in a levothyroxine tablet calculation). The web app
|
||||
lays out older one-paragraph rationales the same way when it shows them (`formatRationale`).
|
||||
|
||||
Then a **second review**: a fresh model call with no Python reads the whole quiz as a careful
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue