Every team we worked on before Velo had a definition of done pinned to a wiki page nobody read. Ours did too, until a Wednesday standup in March when Priya asked whether the invoice retry work was finished, and four engineers gave four different answers. That morning cost us a customer refund and a two hour incident review. We decided the Notion page was not the problem. The definition was.
Three tries that did not stick
Our first attempt was a paragraph in Notion titled “shipping standards” that said tasks should be “merged, tested, and reviewed.” It read fine on the page. In practice, “tested” meant whatever the author felt like: a unit test, a manual walkthrough, or nothing if the diff was under twenty lines. We shipped a race condition in the billing worker three weeks later because the author had run the change against a fresh database and assumed that counted.
The second attempt was a Linear checklist template with nine items. Everyone checked every box, because the boxes were reported by the author and the reviewer had no way to verify half of them without opening five other tabs. The checklist became a ritual, then a joke, then a template we quietly stopped applying to new tickets.
The third attempt was strict: a task was done when a designated QA engineer signed off in a Slack thread. This lasted eleven days. Our QA lead, Ruth, went on holiday, and the queue backed up to forty two tickets. When she came back, half the context was gone and she had to re verify work from memory. We had traded ambiguity for a bottleneck.
The four criteria we settled on
After the third failure, we spent a Friday afternoon working through what we needed the definition to do. It had to be verifiable by someone other than the author, it had to survive one person being out, and it had to answer the question Priya asked in March without a debate. We landed on four criteria, in this order:
- Works: the change does what the ticket says, verified against the acceptance criteria written before the branch was cut. If those criteria were vague, that gets fixed before the ticket moves to review, not after.
- Tested: automated coverage exists for the new behavior, and the tests fail without the change. The reviewer runs the suite locally or points at a green CI badge tied to the merge commit.
- Deployed: the change is live in production, not staging, not behind a flag that has never been flipped on for a real user. If the work sits behind a flag, done waits until the flag is on for the intended audience.
- Observed: a human has confirmed the change behaves as expected in production, using logs, a Datadog dashboard, or a real user event. Not a synthetic ping. A trace of the feature being used, or a metric moving in the direction we predicted.
The order matters. If “works” is unclear, testing the wrong thing is worse than not testing. If we skip “deployed” and call something done at merge, we hide half our incidents in the gap between main and production.
The compromise on observed
Observed was the criterion that almost killed the whole definition. Half the team pointed out, correctly, that internal only changes have no production traffic to watch. A new admin report, a migration script, an internal CLI: none of these throw off metrics on the customer dashboards we use for observability. Waiting for a real user event on an internal tool would mean waiting forever, or fabricating one.
We debated dropping the criterion for internal work. We tried, for a sprint. Two internal tools broke silently and we found out from a support agent who could not load the refunds page. The criterion needed to survive.
The compromise: for internal only changes, observed means the author or a teammate has used the feature in production for its intended purpose, with a Loom or a screenshot posted to the ticket. Not tested it. Used it. If the ticket is a migration, the observation is the query result after the migration ran. If it is a CLI, it is the terminal output from a real invocation on the real database.
The distinction we care about is between “I believe this works” and “this has done its job for a real person, once.” The Loom feels heavy the first time. It stops feeling heavy the second time somebody catches a broken admin page before a customer does.
How the four criteria show up in our week
Every ticket in Linear now has four checkboxes matching the criteria. The author checks the first three. The reviewer, or on internal changes any teammate, checks observed and pastes the evidence. Our Monday planning meeting starts by pulling the list of tickets marked done in the last week and skimming the observation links. It takes eight minutes. In the six months since we adopted this, we have had two rollback situations that a proper observation caught before the on call engineer noticed. We have also had one case where the observation link was a screenshot of the wrong environment, which is a different problem, and one we are still working on.
We do not think this definition is universal. It is what our team of eleven engineers, on a codebase with sixteen deploys a week, needs to keep the wiki page honest. If the shape of the team changes, we expect the definition to change with it.
Leave a Reply