One of the interesting things about building software with coding agents is that the time taken to implement a feature is increasingly becoming a poor indicator of how long it takes to actually finish it.
For one of my recent iterations, the implementation was done in a day. The code was in place, the tests were passing, and the system could be run end-to-end. However, there was still one criterion in the checklist that hadn't been satisfied. The system had to demonstrate that it could run unattended, repeatedly, and behave as expected.
And that was going to take time.
Not time to write more code, but time for the system to run and accumulate enough evidence to satisfy the criterion.
This created an interesting problem with the way I had structured my roadmap. My approach to spec-driven development involves maintaining a roadmap and a checklist for every iteration, with the idea that an iteration must clear its exit criteria before the next one can begin. This is a fairly reasonable way to maintain a working understanding of the codebase, especially when coding agents can implement changes much faster than I can manually inspect every line of code.
But I had treated every item on that checklist as the same kind of gate. And here, that assumption was beginning to cost me.
The implementation was ready, but the iteration wasn't complete. The next iteration was waiting, and the only way to clear the remaining criterion was to let the system run for a few more days.
The obvious solutions weren't good enough
There were two ways to approach this.
The first was to wait. The iteration wasn't complete, so the next one would have to wait as well. This would preserve the integrity of the roadmap, but it would also mean that I was effectively leaving the codebase idle while waiting for something that didn't require any further implementation.
With coding agents, this becomes a particularly interesting bottleneck. If implementation takes a day but validation requires a week of observation, then a strict, iteration-by-iteration workflow could spend more time waiting than actually building.
The second option was to remove the time-based criterion altogether. After all, the code was working, the tests were passing, and the feature had been implemented as intended.
But that would have been dishonest. There is a meaningful difference between a system that works when I run it and a system that continues to work without my intervention. If the requirement is to demonstrate unattended operation, passing a test suite cannot substitute for that evidence.
I didn't want to sacrifice the integrity of the checklist just to keep the roadmap moving. At the same time, I didn't want a criterion that required waiting to block work that could safely proceed.
The solution, as it turned out, was to stop treating completion as a single event.
Implementation and evidence are two different things
Consider the difference between these two statements:
- The code runs successfully.
- The system has behaved correctly five times without intervention.
The first is something I can generally establish on demand. I can run the code, execute the tests, inspect the results, and determine whether the implementation satisfies its immediate requirements.
The second is something I cannot produce on demand in the same way. I can configure the system to run, observe it, and collect evidence, but I cannot legitimately claim five successful unattended runs before those runs have happened.
The distinction is not just about how long each activity takes. It is about the nature of the evidence required to make each claim.
For implementation, the evidence might live in a code diff, a passing test suite, or a successful end-to-end execution. For unattended operation, the evidence lives in the run history: what actually happened, whether the system required intervention, and whether the expected behaviour held across repeated executions.
A test can be rerun. Time cannot be skipped simply because the implementation is ready.
This is also why I don't think the problem is specific to scheduling. Consider a system with a latency requirement. I can implement the changes intended to improve latency in a day, but demonstrating that the system consistently meets its latency target under representative load is a different exercise. The code may be ready long before the evidence is.
The mistake is to treat both activities as if they were the same kind of work, simply because they belong to the same feature.
Splitting the exits
I decided to separate the implementation exit from the evidence exit.
The implementation exit requires the code to be complete, the relevant tests to pass, and the feature to work end-to-end. Once these conditions are satisfied, the implementation can be considered ready for the next iteration to build upon.
The evidence exit requires the time-based criterion to be satisfied. In this case, that means the required unattended runs have occurred and the resulting behaviour has been verified.
The distinction matters because passing the first exit does not mean the second can be ignored. It means the second no longer has to block everything else.
There is, however, an important qualification. An open evidence criterion cannot be used as an excuse to proceed when the underlying implementation is still suspect. If the system encounters a failure during the observation period, or if the evidence reveals that the expected behaviour was based on an incorrect assumption, the next iteration must stop where it depends on that assumption. The issue has to be investigated and fixed before proceeding.
In other words, waiting for evidence is not the same as assuming the evidence will be favourable.
This also comes with a cost. Splitting the exits introduces another piece of state to maintain, another distinction to document, and another way for the roadmap to become confusing. There is also a risk that a supposedly non-blocking criterion will be forgotten once the next iteration starts. The approach only works if the outstanding evidence remains visible and the next iteration respects any dependencies on it.
It is a little more complicated than a single checklist, but that complexity reflects a distinction that was already present in the work.
The problem with a single status field
I initially tried to represent this using a single status in my status.md.
The difficulty was that none of the obvious values was quite right.
Marking the iteration as complete, with a green tick, would suggest that all the exit criteria had been satisfied. That wasn't true. Marking it as in progress would imply that implementation was still underway, which wasn't true either. And leaving it blocked didn't distinguish between work that was waiting for implementation and work that was waiting for evidence.
The problem was not that I needed a more sophisticated status label. It was that I was trying to represent two independent states with one field.
An iteration can have its implementation complete while its evidence is still being collected. The next iteration can be allowed to start while that evidence remains outstanding, provided the unfinished criterion does not invalidate the assumptions on which the next iteration depends.
That is a much more accurate description of the situation than either complete or in progress.
It also forces me to be more deliberate about what a status actually communicates. A green tick should mean something specific. It should not simply mean that the work feels finished, or that the remaining item is inconvenient to wait for.
What this is really about
The more I work with coding agents, the more I think that maintaining an accurate understanding of a codebase is not just about understanding the code itself. It is also about understanding what has and has not been established about the system.
The code can be complete while the claim about the code is not yet justified.
That distinction becomes particularly important when implementation is cheap relative to validation. The faster we can produce software, the easier it becomes to confuse the ability to build something with the ability to demonstrate that it works as intended.
I don't think the answer is to make every workflow more elaborate, or to introduce more status fields for the sake of it. The goal is to represent the actual state of the work as honestly as possible, even when that state is inconvenient.
For my project, this means keeping the roadmap moving where it is safe to do so, without pretending that an outstanding acceptance criterion has disappeared.
The work was done in a day. It took a week to be true.
And I think the distinction between those two things is worth preserving.