$ cat blog/our-agent-writes-code-now.md
Our agent writes code now, and it can finally count
Ask an LLM agent “how many maths lessons do I have left before Christmas, bank holidays excluded?” and you’ll get a confident number. It’ll probably be wrong, too. The agent reads ten timetables one tool call at a time, holds them all in context, and counts in its head.
At Teetsh, our agent now writes a small program for that. The program walks every school day, skips holidays, counts the lessons and adds up the minutes. The machine does the maths, so the number is exact.
That’s code mode. I took it from research to 20 % of prod in under two weeks, and I didn’t have to wait for anyone to ship it.
The idea
Instead of calling tools one by one, the model writes ~20 lines of JavaScript where our tools are plain async functions. The script loops, filters, joins and counts. Only its return value goes back to the model.
// simplified
const days = teetsh.helpers.school_dates({ to: "2026-12-18" });
const edt = await teetsh.find_emplois_du_temps({ periode: 2 });
let sessions = 0, minutes = 0;
for (const d of days) {
for (const slot of edt.slots_on(d)) {
if (slot.subject === "maths") { sessions++; minutes += slot.duration; }
}
}
return { sessions, hours: minutes / 60 };
None of this is my invention. Cloudflare called it Code Mode, Anthropic and OpenAI both offer “programmatic tool calling” in their APIs, Pydantic has its own take, and the CodeAct paper did it in 2024. When I looked at it end of October, the concept was clear and so were the numbers. What was missing was a version that fit us.
Why it matters for us
Two thirds of what our agent read was raw tool output. Every step re-read the whole conversation, and the hard requests (prepare next week’s cahier journal, check programme coverage, replan after a strike day) were exactly the long ones. That’s where the model lost track mid-loop, copied the wrong ID from one result into the next call, or averaged 300 grades “in its head”.
With code mode, the gradebooks never enter the context. The model gets back “5 pupils under 50 %, and the 3 skills they struggle with”, computed by code.
How it’s built
The whole thing is one new tool, run_script, and it lives inside our Go API process:
- Sandbox: QuickJS (quickjs-ng) compiled to WebAssembly, run by wazero. Pure Go, no cgo, no new service, a few milliseconds to start.
- Nothing to reach: no network, no file system, no env. The only way out is our own tool functions, and the teacher’s identity and school are bound by the server before the script starts.
- Hard caps: 15 s of CPU, 64 MB of memory, 50 tool calls, 20 KB of result.
- Writes, carefully: reads are open, writes are declared up front and capped, and deletes are never run in a script. They stop and wait for the teacher’s approval, like before.
- Helpers: the school calendar (periods, holidays, school days) is available in the script, so the model doesn’t reinvent date maths.
- Observability: every script, every inner call, its result and duration are stored and replayable in our admin.
Wasm is what makes this cheap. Not long ago, “run code the LLM wrote, safely, in your API” meant containers, microVMs or a whole new service. Today it’s a Go library and a pinned .wasm file.
Yo dawg
Code is great at counting, but bad at meaning. Ask “which of my lessons this year were about water?” and a script searching for “eau” will happily pick up the phonics lesson on the au / eau sound. No regex can tell those apart.
So scripts can call a model too. teetsh.ai.classify sends up to 100 short texts to a small, cheap classifier and gets a label back for each one, and teetsh.ai.check asks a yes/no question about each. The script loops, the small model judges each item, the code counts, and the agent only reads the total.
On the bench, the “meaning” cases went from 44 % to 67 % strict success, at a third of the cost per case. In prod the agent hasn’t reached for it yet, so for now this one is a bench story.
What it changed
On our bench (45 cases written for code mode, 5 runs each, same day, only the flag differs):
- +15.6 points of strict success (66.8 % → 82.3 %)
- −38 % cost per case
- Calendar questions went from 66 % to 96 %, bulk writes from 70 % to 96 %
It’s not magic, though. The model still sometimes mis-copies the script’s numbers into its answer: the code says 35, the reply says 33. The maths is right now, but the rewording isn’t always. And on short requests it doesn’t help, a single tool call is still cheaper than a script. In prod the agent only reaches for it in about one conversation out of five, which is about what you’d want.
Data the model never sees
There’s a side effect I like even more than the maths. Whatever stays inside the sandbox never reaches the model. A script can read 26 pupils’ grades, compute who’s struggling and on which skill, and return only what’s needed.
For us, that matters. These are children’s data. Our goal is simple to state: the LLM should never be able to recognise a pupil. Grades and averages can reach it, names, remarks, birth dates and health notes can’t.
Code mode gives us the place to do that. Pupils come into the conversation as mentions, which the server turns into opaque tags. Tools hand the model tags instead of names, and anything sensitive is only readable from inside a script. The script can join, filter and count on the real data, and the model reasons on élève_7f3a without ever knowing who that is. The teacher still sees the real name in the app.
That part isn’t shipped yet, we’re building it now. But it’s the reason I wanted the sandbox in our own process rather than in someone else’s container.
Six days later
We merged on November 6 and rolled out to 20 % of teachers on November 10. On November 12, Claude Code shipped its own Code Mode: Claude writes JavaScript that calls its tools. Same idea, almost the same shape.
I’m not claiming we beat Anthropic to anything. They had programmatic tool calling in their API a year ago. But I enjoyed the timing, because of what it says about how we can build now.
Take the recipe, not the framework
The hosted version wasn’t an option for us anyway. It runs in Anthropic’s container, it isn’t eligible for zero data retention (and these are pupils’ data), and it only works with Claude, while we route to several models. Waiting for a framework to add it would have meant waiting for one that fits our Go harness, our scopes, our approval gate and our tracing.
So I didn’t wait. I took the concept, the architecture everyone converged on and the gotchas they published, and built it into what we already had. The bridge reuses the exact dispatcher our agent already calls, with the same auth checks and the same traces.
That’s the point I made in Share the recipe, not the code, from the other side. When code is cheap, what you want from others is the idea, the shape and the traps, not their package. A good write-up from Cloudflare or Anthropic was worth more to me than any SDK, because I could build it our way in a week.
Now it’s just a matter of getting the idea, understanding the architecture, and going for it.