Brief
Program-solve interface improves clinical math for 32B model, not 7B
A preprint tests whether a language model can write case-specific Python for a restricted executor instead of doing clinical arithmetic itself. The handoff helped a 32B open-weight model but not a 7B one.
Researchers tested an alternative to hardcoding clinical calculators: the model writes case-specific Python that a restricted local executor runs as a deterministic solver. On MedCalc-Bench Verified (1,100 cases, 55 calculators), with formulas and gold variables supplied, the approach helped Qwen2.5-32B-AWQ (90.53% vs 83.47%, +7.05 points, 95% CI [0.47, 14.60]) but not Qwen2.5-7B (75.31% vs 72.02%, +3.29 points, 95% CI [-3.49, 10.38]).
A hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, scoring 40.0% overall. The authors audited the benchmark's formulas against current clinical guidelines and flagged 16 of 55 with version, use or coefficient concerns.
Our reading
Our reading is that this approach may help larger models but does not substitute for verified formulas or reliable variable extraction.
What to do or watch
The result only holds under matched formula and gold-variable access, so watch for follow-up tests where the model must extract variables and supply its own formulas — that, not the executor handoff itself, is the unresolved question. Readers should also treat the 16 of 55 benchmark formulas flagged for version, use or coefficient concerns as a reason to verify any calculator's math against current guidelines before relying on it.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by arXiv
- The study evaluated a Program-Solve interface on MedCalc-Bench Verified, covering 1,100 cases and 55 calculators.
- With formulas and gold variables supplied, handing off to the solver was not a reliable advantage at 7B (75.31% vs 72.02%, +3.29 points, 95% CI [-3.49, 10.38]).
- At 32B, handing off to the solver was a reliable advantage (90.53% vs 83.47%, +7.05 points, 95% CI [0.47, 14.60]).
- The hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, scoring 40.0% overall.
- The benchmark's formulas were audited against current clinical guidelines, flagging 16 of 55 with version, use or coefficient concerns.
Sources
- arXivText stored 13 September 2026
How this story was checked. Written from the 1 page listed above, stored 13 September 2026; claims checked against that stored text on 14 September 2026.
What that means
- 5 of 5 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.