A benchmark of brushwork for code agents

The studioA robot arm, one brush, sixteen paints.

Ground · strokes 1–95First the ground, in broad rows.

Forms · strokes 96–255Then the forms: the hills, the cypress, the sky.

Details · strokes 256–455Then the details and the stars: 455 strokes in all.

RoboPaint

Can your code agent hold a brush?

82 works of calligraphy, lettering and painting. A code agent plans every stroke; a simulated robot arm makes it; the sheet and the hand are judged.

Scroll to paint

From a single brush stroke to a van Gogh. Each work is a picture to copy with a robot's brush or pen, stroke by stroke, the way a person would.

The collection

Five disciplines, each with its own brush. Hover a work for the reference robot's copy; open it to compare and to watch it made.

Written, not coloured in

A fine tip can trace round and round a character until it is filled, and the picture alone would pass. RoboPaint also judges the hand: brush strokes, made as a calligrapher or a painter makes them.

How a work is judged

Aligned first, then compared. Writing is judged by its ink, painting by its colour; both by their strokes.

Open book, closed book

Every work is set twice. In the studio the agent may look at the exact sheet, ask for its score and start over; on stage it sees only what a camera sees, and paint is permanent.

Results

Two code agents, every work, both modes. A point is a work solved: the picture matches and the hand passes.

At the easel

Every copy the two agents handed in, hung beside its exemplar. A red seal marks a work solved; a note says what failed, the picture or the hand. Open one to compare it with the exemplar.