Report
A coding model taught to paint: open RL recipe puts the reward in a hand-rated reference pool
A Hugging Face blog post reproduces, with every artifact published, a viral demonstration in which a language model paints watercolours by writing JavaScript, and compares three reward mixes. The original video's own artifacts are still not open, and the post says a full technical report is coming.
A Hugging Face blog post published on 3 September 2026 by Sergio Paniego reproduces, in the open, a viral demonstration in which a language model paints watercolours by writing JavaScript. The model's output is a program of about 150 lines using p5.brush, described in the post as a library that "adds natural drawing tools to p5.js". Because the output is code, it can be read, edited and run again.
The change is the lever, not the medium. The post traces the project to Surya Narreddi's video of 23 August, which passed 1.5M views at the time of writing, and to earlier work in which the prompt was the only control a user could pull. Training the model itself goes further than prompting a text-to-image system, and here style is constrained deliberately: the model is allowed only ten of the library's methods.
The reward is taste rather than correctness. Four terms are weighted: a gate at 0.05 for code that compiles, paints something and does not cheat; a length term at 0.05; a pairwise style judge at 0.60 comparing against references drawn from a pool; and HPSv3 at 0.30. HPSv3 is described as an open 7B preference model trained on human choices between pairs of images, returning a score for how much a person would prefer an image given a text description.
Everything is published. The pipeline runs end to end on Hugging Face: training on Jobs, the RL environment and scorer model as Spaces, the pairwise judge through Inference Providers, and the reference pool dataset, scripts and trained models on the Hub in one collection. Once the two Spaces are up, the recipe is a single command with two environment variables for the reward mix; the published run uses Qwen/Qwen3.5-35B-A3B with LoRA, 110 steps and 240 episodes. Three runs, one per reward mix, were trained and compared.
The author notes he followed the original blog step by step, changed something only when strictly needed, and put his own ideas into a "What I would try next" list rather than the experiment.
Our reading
This desk follows implementation patterns and evaluation harnesses, and the reusable part here is the assembled reward: a compiled-output gate, a hand-rated pool and an open preference model, wired so a subjective output can be optimised. Teams scoring work that has no correct answer — design, copy, layout — should read it as a template rather than a watercolour story.
What to do or watch
The reproducible starting point is the published collection and its one-command run after duplicating the environment and scorer Spaces; watch the post's "What it actually learned" and "What it costs" sections, and the original author's forthcoming technical report, for measured outcomes.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by Hugging Face
- Surya Narreddi posted a video of watercolours painted by a language model on 23 August, which passed 1.5M views at the time of writing.
- The model writes JavaScript through p5.brush, a library the post describes as adding natural drawing tools to p5.js.
- The reward mix weights a pairwise judge at 0.60, HPSv3 at 0.30, and gate and length terms at 0.05 each.
- HPSv3 is an open 7B preference model trained on human choices between image pairs, returning a score for how much a person would prefer an image given a text description.
- The pipeline runs on Hugging Face end to end: training on Jobs, the RL environment and scorer model as Spaces, the pairwise judge through Inference Providers, and all artifacts on the Hub in one collection.
- Once the two Spaces are up, the recipe is one command, with two environment variables set for the reward mix.
Sources
- Hugging FaceText stored 16 September 2026
How this story was checked. Written from the 1 page listed above, stored 16 September 2026; claims checked against that stored text on 16 September 2026.
What that means
- 6 of 6 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.