I've been reworking the Software Factory. It became brittle enough that it was getting in the way of the work I built it to do.
Things I expected to take a couple of hours to ship were starting to take two days. An orchestrator agent would run the factory, hit something it needed to fix, stop, and trigger cascading reruns. I was spending time getting the system through its own process rather than getting useful software out of it.
One example was a single PR of roughly 240 lines that took about two days. Another was a 17-PR stack that took a week to build and amounted to about 2,000 lines of code. Line count isn't a measure of difficulty, but the time I was spending getting these changes through the factory made me question whether this was the wrong approach altogether.
Agents are incredibly useful. It's also easy to trust them too quickly and let them do too much. I have that tendency myself. I'll be methodical for the first few iterations, then start handing over more because the early results were good.
Agents take what your tendencies are and multiply them. In this case, that included letting agents keep building the Software Factory itself.
The factory started asking for the wrong thing
I built the factory to let agents cook, then verify the results. Over time, we overindexed on receipts, contracts, and verification. Receipts are the records of what ran and what it checked; contracts define what the factory will accept before continuing.
I probably helped steer it there. I'd encouraged the agents to adopt Swamp and its principles, but without enough specificity about what I wanted those principles to accomplish. Swamp gives me typed model methods, workflows to compose them, and durable data about what happened. I wanted that to help the agents do useful work and help me understand the results.
What I got started feeling more like a Costco door watcher checking every receipt than a Walmart greeter helping me get where I was going.
Those things have a purpose. I want to know that a test actually ran, that it checked the code I'm looking at, and that an agent hasn't treated permission to open a draft as permission to merge it.
But iteration and improvisation started becoming red flags for the factory. An agent could make a useful adjustment and then spend more effort getting that adjustment through the machinery. My read is that I'd let the agents optimize the factory toward perfect outputs and exhaustive proof rather than the handoff I actually wanted.
If an agent gets ninety percent of the work right, I can take it through the last ten percent. That's the target, not a measured success rate. I don't need every task to arrive as a finished, autonomous delivery. I need something useful that I can inspect, change, and finish.
So I'm scaling it back. The Frink rework is an attempt to get back to that: work toward a reviewable draft, accept feedback, and check what changed without treating every adjustment as a reason to start over.
What changed in the flow
The old factory wasn't just an agent followed by some tests. The stack-first route made the shape of the delivery part of the construction process: choose PR-sized units up front, build each in its own worktree, and certify each unit before moving on.
There really was a model named packet-certifier. It took a snapshot of ignored files before implementation, then checked the resulting changes against the permitted files, ignored-file policy, and change budget. In the historical stack workflow below, the configured ceiling was 20 changed files and 550 changed lines per unit. Separate runtime methods checked the reported file list and the one-commit-over-parent rule. Then another workflow assertion checked that the certification record itself had passed for the correct invocation.
This is the old stack-first route, with the actual model and method names. Each box groups calls so it stays readable; it is not a list of every bookkeeping step.
┌────────────────────────────────────────────────────────────────┐
│ frink-stack: approved plan → preselected PR units │
└─────────────────────────────┬──────────────────────────────────┘
▼
┌────────────────────────────────────────────────────────────────┐
│ A. frink-stack-implement workflow — REPEAT FOR EACH UNIT │
│ │
│ frink-runtime.prepareStackUnitBuild │
│ packet-certifier.snapshotIgnoredState │
│ cli-agent-frink-implement.invokeAndParse │
│ frink-runtime.finalizeImplementation │
│ frink-runtime.finalizeStackUnitCommit │
│ packet-certifier.certify → assert certification passed │
│ frink-runtime.runUnitValidationCommands │
│ frink-runtime.completeStackUnitBuild → assert recorded state │
│ │
│ Own worktree; one commit; file/line budget; pinned parent │
└─────────────────────────────┬──────────────────────────────────┘
▼ all units built
┌────────────────────────────────────────────────────────────────┐
│ B. frink-runtime.validateComposedHead + deriveChangeSurface │
└─────────────────────────────┬──────────────────────────────────┘
▼
┌────────────────────────────────────────────────────────────────┐
│ C. frink-stack-review workflow │
│ │
│ cli-agent-frink-review-{correctness,security,design,testing} │
│ + selected specialist lenses │
│ frink-runtime.validateReviewPanel │
│ cli-agent-frink-review-verifier │
│ frink-runtime.validateFindingVerification │
│ frink-stack.record_artifact / record_evidence + assertions │
└─────────────────────────────┬──────────────────────────────────┘
│
blocker → patch → frink-runtime.restackAbove
→ return through build/validation/review
│ findings cleared/accepted
▼
┌────────────────────────────────────────────────────────────────┐
│ Submission approval → frink-runtime.submitStack │
│ → draft PRs, identity readback, recorded submission evidence │
└────────────────────────────────────────────────────────────────┘
The four core reviewers were separate cli-agent model instances. The specialist selection included object-oriented design, query performance, AI-slop, extensibility, visual, and completeness lenses. The runtime checked the panel; another agent verified its findings; the runtime then checked that verification. The managed single-feature route had its own focused/full test and applicable browser gates. These were related routes, not one universal chain that every item traversed.
The new route removes that mandatory per-unit certification and panel/verifier chain from ordinary draft delivery. It doesn't just hide it behind a shorter workflow name. Work stays on one integrated candidate while the agent builds and repairs it. The actual frink-prototype-to-draft workflow has two model-method steps: validatePrototypeDraft, then openPrototypeDraft.
┌────────────────────────────────────────────────────────────────┐
│ Direction approval → frink-runtime.beginPrototypeDraft │
│ Configured cli-agent planner + independent checker │
│ → frink-runtime.compilePrototypeTasks │
└─────────────────────────────┬──────────────────────────────────┘
▼
┌────────────────────────────────────────────────────────────────┐
│ One cli-agent implementation owner │
│ One integrated candidate; practical checkpoint commits │
│ In-scope implementation, adaptation, and repair │
└─────────────────────────────┬──────────────────────────────────┘
▼
┌────────────────────────────────────────────────────────────────┐
│ frink-prototype-to-draft workflow │
│ │
│ 1. frink-runtime.validatePrototypeDraft │
│ Check candidate; execute/reuse checks; retain gaps │
│ 2. frink-runtime.openPrototypeDraft │
│ Check authority; submit/read back draft; observe CI │
└─────────────────────────────┬──────────────────────────────────┘
▼
┌────────────────────────────────────────────────────────────────┐
│ WORKING DRAFT — I can inspect it now │
│ │
│ My feedback + broad independent cli-agent review │
│ + risk-relevant specialists / available browser QA │
│ + frink-runtime.refreshPrototypeDraftVerification (CI) │
└─────────────────────────────┬──────────────────────────────────┘
│
feedback → frink-runtime.prototypeDraftAction
→ same owner/candidate; affected rechecks
│ direction settles
▼
Keep one PR or package a useful stack
There is still real work inside those two methods. Validation checks repository/candidate identity and command evidence. Draft creation checks scoped publication authority and known security exposure, and reuses the existing submitStack adapter as a one-unit submission before reading back the PR's head, base, draft state, and body. I haven't thrown away the Git and PR checks that establish what actually exists.
What changes is what I have to wait for:
| Before, in the managed stack route | Now, in ordinary draft-first delivery |
|---|---|
| Partition before building; certify every unit against commit and size rules. | Build one candidate; checkpoint when useful; package after feedback. |
| Core review panel, selected specialists, finding verifier, and verification gates before submission. | Inspect the draft while broad review and available CI/QA proceed; add lenses for risk. |
| A repair can cause downstream restacking and a return through build, composed validation, and review. | Keep the candidate and run; repair and recheck affected evidence. |
| Passing the managed delivery gates determines when a draft can be submitted. | A coherent draft can expose missing evidence without claiming those checks passed. |
The current runner still needs an orchestrating agent/operator. This is fewer mandatory handoffs before I can give useful feedback, not a claim that two model methods replace all engineering judgment or make every repair cheap. Unsafe exposure still blocks, and merge and deployment remain separate decisions.
The simplification is also concrete: five general factory definitions and 50 delivery workflows were taken out of active discovery and retained as historical material. Ideation, reproduction-first bugs, and surgical hotfixes remain distinct paths. Ordinary feature work uses the draft runtime instead of choosing among the old managed factories.
Stacking now comes after a working integrated draft and product feedback, when splitting at meaningful seams helps review or delivery. It isn't a requirement to turn every implementation task into its own polished commit and PR before I've seen the result.
What I've been testing
On September 12, I ran two disposable local simulations, then a draft-only trial in a real application repository. The local runs used real agent invocations, Git, and tests, but simulated the external PR and CI behavior. Product feedback and approvals in these trials were explicitly synthetic test inputs, not actual product decisions.
One useful example came from the second simulation. After a local draft stack existed, the run paused and resumed with a small product change: reduce the default page size from 100 to 50. It retained the ideation, approval, plan, and unaffected evidence, then reran the affected validation.
Review still found a gap in the new tests. The follow-up covered the boundary cases, and the final implementation passed all 25 tests. That is the kind of iteration I want: keep the work that is still valid, change the part that needs changing, and verify it. This was a local test, not proof that the whole factory is now easy to use.
The real-repository trial supplied another example. The new tests passed locally, but an independent reviewer found that they weren't included in hosted CI. The repair wired them into CI. All six CircleCI backend and JavaScript workflows across the three draft PRs then completed successfully.
That's verification earning its place. Passing local tests wouldn't have caught the missing CI integration by itself.
The same trial also reached draft-ready while the aggregate technicallyReady flag remained false: the factory treated intentionally skipped or cancelled auxiliary GitHub checks as non-passing. The deployment and database migration jobs were intentionally skipped. Nothing was merged or deployed, and the trial PRs were closed afterward.
That doesn't mean skipped checks should always count as success. It does illustrate the distinction I need the factory to make: enough evidence for me to review a draft is not the same requirement as enough evidence to ship it.
Swamp is still part of the answer
Swamp was useful in troubleshooting this, too. The runs left durable records of agent invocations, workflow results, failures, and candidate versions. We could inspect those records rather than rely on an agent's account of what it had done. Pause and resume could retain the candidate and earlier work instead of starting a new run and losing the failure history.
One local trial found a particularly revealing problem: a planner's trailing punctuation tripped an incorrect text-matching check on intent. The fix kept the approved direction in the task constraints and used the independent checker to assess whether the plan meant the right thing. Deterministic code can check that required fields exist. Matching strings isn't the same as understanding whether a plan satisfies the request.
Even the reporting needed repair. The audit and flow-metrics reports initially produced no output for the factory's installed type because they recognized a different type name. Local report copies corrected that mismatch, making stage visits and approvals visible again. The accounting also exposed missing run tags on downstream invocations; those tags were subsequently fixed. Token and billing coverage still aren't complete.
We also isolated a real Swamp bug: automatic upgrades could turn timestamp-shaped strings into dates because the YAML writer and reader used different schemas. That became Swamp issue #2120. By September 14, the adopted build passed both the small round-trip reproducer and an isolated real ideation upgrade. That was a platform defect to reproduce and fix, not another rule to add to the factory's product process.
The better use of Swamp here is to execute repeatable operations, retain what happened, and make recovery specific. Reuse checks only when their inputs and environment still justify it. Rerun affected evidence when something changes. Let agents reconcile ordinary implementation details and ask me about genuine product or authority decisions.
I still want receipts. I don't want checking receipts to become the product.
Where that leaves me
I'm still at a decision point. I'm giving this another couple of days to see whether I can make it useful enough to keep investing in. Otherwise, I need to lean more directly on frontier models for what they're good at.
I suspect the answer is somewhere in between. Use deterministic code for the things that need repeatable behavior. Give agents bounded work and room to solve it. Verify the results, but don't ask for too much autonomy or too much certainty too early.
The trials show some of the behavior I want. They don't yet show that I've turned two-day tasks back into two-hour tasks, and the available accounting doesn't establish the total cost. The next test is whether this helps me finish actual work with less intervention in the factory itself.