Colex / build log

I stopped writing code.
Now I run a factory.

Agents write the code here. My job is to decide what gets built, and to catch what they got wrong. This is how that setup came together, including the parts that are still broken.

· 10 min read

The problem

Writing code was never the bottleneck

I had one skill that took a plan and built it test-first. It worked. Not sometimes. Reliably, on real work, often enough that I stopped reading the code it produced line by line.

So I used it more. An agent comes back with a red test. Is that a real bug or a broken container? Nobody knows, so I go find out. Twenty minutes gone. An agent says it's done, and I have no idea what "done" meant to it, so I go check. Twenty more. Multiply by every piece of work, every day.

Code generation was the solved part. Everything on either side of it wasn't, and that's where all my time went.

Then reliability created its own problem. Once each piece lands, the limit becomes how fast I can hand out the next one. I was the queue. Sitting there dispatching work one item at a time, which is a bad job for a person and a worse one for the person meant to be deciding what gets built.

I call it a factory and it is not really one. Nothing here is a stamped part. The code is bespoke every time, and when something bigger goes wrong I still have to open the thing up, think like a developer, and guide it out. What actually resembles a factory is the discipline around the work: every run measures itself, every bit of friction gets written down, and each one becomes a fix before the next run starts. That is kaizen, not mass production. The line is worth copying. The idea that the parts are interchangeable is not.

How I got to the fix

Every fix moved the problem somewhere else

I didn't design this. Each part exists because the part before it worked and the bottleneck slid somewhere new.

  • Kill the environment questions. Every workspace gets built fresh and runs its test suite before any agent touches it. Whatever is already failing gets written down by name. Now a red test is either on that list or it's real. No investigation.
  • Stop arguing about proof. What counts as evidence comes from what the change touched, decided before the work starts. The agent doing the work doesn't get to pick. Sounds bureaucratic. It killed a conversation I had several times a day.
  • Get myself out of the dispatch loop. Runs, manifests, a ledger that survives between sessions. A coordinator hands out the work now. It helped that plans got good enough to write for every open issue, which turned a pile of vague bugs into an actual queue.

Three fixes in. All three are working, and together they are the thing below.

The fix, exactly

The whole thing, drawn

This is what replaced me sitting in the loop. Nine stages. Click any one to open it and see what happens inside. Reviews are burgundy, stops are red, my calls are green.

Drag to pan · hover a node for detail

Machine work
Gate
Review
Stops work
Me
  1. Start the run. Check that the machinery matches. Then resume the run, or plan its shape. Steps: I start the run, Version assert, Stop. I rule, Ledger exists?, Resume from state header, Compile the run shape only, Bounce the plan-set, Report to me.
  2. Provision. Build a certified workspace. Without one, no work starts. Steps: Build the workspace, Failed. Retry?, Stop the run report systemic, Freeze read-only, Certified + warm spare.
  3. Dispatch. Write the spec for this unit. Then give it to a new executor. Steps: Compile THIS unit's spec, Dispatch fresh executor.
  4. Execution. Review the plan. Write the code test-first. Then review the diff with a different model. Steps: REVIEW 1 Plan review, TDD loop red → green, REVIEW 3 Diff review, Evidence bundle + signed verdict, ENV-FAULT, Drift or design conflict.
  5. Gate the work. Check the bundle is complete. Work out the real risk level. Both steps are mechanical. Steps: GATE 1 Completeness, Bounce to executor, Re-derive risk level, Escalate one-way.
  6. Audit the evidence. Did this really happen? This stage re-runs the claims. It does not judge the code. Steps: REVIEW 2 Evidence audit, Pick methods from the changed surface, Independent failure modes?, Add a method that fails differently, Non-deterministic method used?, Claims reproducible?, Bounce to executor.
  7. Merge & record. Merge the work. Prove that nothing broke. Write it down. Steps: Merge + full suite, WORK STOPS nothing dispatches, Ledger + telemetry, Seed the server write the brief.
  8. My verdict. I check a unit when its risk calls for it. I check every run without exception. Steps: My verdict on the unit, Fast-repair at queue front, Units remaining?, MY SMOKE every run.
  9. Factory feedback. Turn the friction from this run into the next version of the workshop. Steps: Collect every FRICTION tag, Draft the fixes they imply, I keep, strike or amend.

Still the fix

One run is never the answer

That's the part I had wrong for a while. A run doesn't produce a finished thing. It produces something worth testing. I smoke it, that finds real problems, those become the next run. Repeat until it holds.

How many rounds depends on the work. Logic settles fast. Screens don't. I find them hard to specify well, so they take more passes.

  • Round one. Full run. Many units, a manifest, a ledger.
  • Then I smoke it. I use the real thing and find what the gates missed. Smoke isn't the last gate. It's what generates the next round.
  • Round two onward. Mostly single units. I talk one fix through, dispatch it, same gates, no manifest.

What is next

Testing is the bottleneck now

Code, setup and dispatch are all handled. What is left is the part where I find out whether any of it is good. Single units still have no ledger entry either, so the mode I use most is the one leaving the least trace.

Testing is the part I still picked by hand, out of memory, before every run. I wrote the rules down and ran them properly for the first time in July. The run found real problems and three failures that were not real, which turned out to be the more useful result.

A test bench I can rely on →

None of this makes agents smarter. It makes their output checkable, which turns out to be the part that was missing. Every fix here started as something I was doing by hand several times a day until I got tired enough to build the thing that does it instead.

I expect most of this to look wrong in six months. The certified workspaces and the prescribed proof I would keep. The rest is scaffolding I built to get past a specific problem, and it will change when the problem does. That's the whole method, really. Fix the slow thing, watch where the slowness goes, fix that.