Brief
Handing arithmetic to a solver helps a 32B model, not a 7B one
A preprint tests whether a model can write case-specific Python for a restricted executor instead of doing clinical arithmetic itself. The handoff helped a 32B open-weight model but not a 7B one, and the authors flagged 16 of the 55 benchmark calculators for version, use or coefficient concerns.
Researchers tested an alternative to hardcoding clinical calculators: the model writes case-specific Python that a restricted local executor runs as a deterministic solver. On MedCalc-Bench Verified (1,100 cases, 55 calculators), with formulas and gold variables supplied, the approach helped Qwen2.5-32B-AWQ (90.53% vs 83.47%, +7.05 points, 95% CI [0.47, 14.60]) but not Qwen2.5-7B (75.31% vs 72.02%, +3.29 points, 95% CI [-3.49, 10.38]).
A hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, scoring 40.0% overall. The authors audited the benchmark's formulas against current clinical guidelines and flagged 16 of 55 with version, use or coefficient concerns.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by arXiv
- The study evaluated a Program-Solve interface on MedCalc-Bench Verified, covering 1,100 cases and 55 calculators.
- With formulas and gold variables supplied, handing off to the solver was not a reliable advantage at 7B (75.31% vs 72.02%, +3.29 points, 95% CI [-3.49, 10.38]).
- At 32B, handing off to the solver was a reliable advantage (90.53% vs 83.47%, +7.05 points, 95% CI [0.47, 14.60]).
- The hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, scoring 40.0% overall.
- The benchmark's formulas were audited against current clinical guidelines, flagging 16 of 55 with version, use or coefficient concerns.
Sources
- arXivText stored 13 September 2026
How this story was checked. Written from the 1 page listed above, stored 13 September 2026; claims checked against that stored text on 14 September 2026.
What that means
- 5 of 5 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.