Category: Delivery

  • Story Points Are a Tax You Pay to Feel Certain

    Story Points Are a Tax You Pay to Feel Certain

    Every team we onboard at Velo arrives with a backlog full of numbers. Threes, fives, eights, the occasional heroic thirteen. Someone once explained that these are relative sizes, not time, and everyone nodded, and now every Tuesday afternoon a group of adults argue about whether a database migration is a five or an eight while the sprint clock ticks.

    We think story-point estimation is broken. Not “needs tuning” broken — structurally broken. It survives because it feels like rigor, and rigor is comforting when you’re being asked how long something will take. But the ceremony produces almost none of the value it promises, and it quietly costs teams a great deal.

    The original promise, and what actually happened

    Points were introduced to solve a real problem. Hours-based estimates were wildly inaccurate, and worse, they became commitments the moment they left the room. The fix was elegant on paper: estimate relative complexity in an abstract unit, track velocity over time, and let the arithmetic convert points to dates without ever asking a developer to promise a Thursday.

    In practice, three things went wrong.

    1. Points became hours in a hoodie. Every team eventually calibrates points against calendar time. “A five is about a day and a half.” The abstraction leaks immediately, and now you have hour estimates with extra steps.
    2. Velocity became a target. Goodhart’s Law arrives on schedule. Once velocity is tracked, it’s optimized, tickets are inflated, velocity rises, and leadership assumes throughput improved. Nothing improved.
    3. The ceremony is expensive. A team of eight spending forty minutes twice a week in planning poker is burning roughly ten engineer-hours weekly on an activity whose output is a Fibonacci number.

    The most honest sentence ever uttered in a pointing session is “I don’t know, I’ve never done this before.” Points punish that answer. They should reward it.

    What estimates are actually for

    Before proposing a replacement, it’s worth asking what problem estimation solves in the first place. Teams need three different answers, and points collapse them into one:

    • Should we do this at all? A rough sense of whether a piece of work is a week or a quarter, so it can be prioritized against alternatives.
    • Is this small enough to start? A check that the work has been decomposed to the point where an engineer can begin without a discovery phase disguised as coding.
    • When will it be done? A forecast, with a confidence interval, that leadership and other teams can plan around.

    The first is a shaping question. The second is a slicing question. The third is a statistics question. None of them are well served by asking six people to hold up a card.

    What we do instead

    At Velo we’ve replaced points with a three-part practice. It’s less ceremonial, more honest, and — we’ll show the numbers below — meaningfully more accurate.

    1. T-shirt sizes for shaping

    When new work enters the backlog, whoever is shaping it assigns a rough size: XS, S, M, L, XL. Five buckets, no arithmetic. XS is “an afternoon.” XL is “we need to break this up before it’s real.” Sizes aren’t summed, aren’t averaged, and don’t roll up into a velocity number. They exist only to answer the shaping question.

    2. A slicing rule for readiness

    No work item enters an active cycle unless it’s been sliced to a size a single person can start on Monday and demo by Friday. If it can’t, it’s not ready — full stop. This replaces the pointing debate with a much more useful conversation: how do we cut this?

    3. Cycle-time forecasting, anchored to real projects

    For the “when will it be done” question, we ignore estimates entirely and use historical cycle time. Every closed ticket has a measurable duration from “started” to “shipped.” Feed those durations into a Monte Carlo simulation over the remaining backlog and you get a probability distribution over completion dates.

    The onboarding move that makes this land for new teams: pick two named past projects of similar shape — the last checkout rebuild, the migration from Redis, whatever you’ve actually shipped — and anchor the forecast against their cycle-time distributions. The plan inherits your team’s real pace, not a generic prior. Once you have a few months of your own history, the anchors fade out on their own.

    Does it work?

    We compared the last six months of forecasts across a sample of teams using story points versus teams using the practice above. The delta was not subtle. But rather than a bare accuracy number, here’s the honest before-and-after on the axes teams actually care about:

    Dimension Story points + velocity T-shirts + cycle-time forecast
    Estimation unit Fibonacci points, debated per ticket Five T-shirt buckets, assigned by shaper
    Re-estimation trigger Manual, in a meeting, on scope drift Automatic — every closed ticket updates the model
    Forecast update trigger End of sprint, if anyone remembers Nightly, on the last N closed tickets
    Capacity model Rolling average of self-reported points Empirical cycle-time distribution
    Median forecast error 34% (2h20m/week/person estimating) 12% (~20 min/week/person)
    Design-partner retention of the practice Abandoned within 4 sprints on 3 of 8 teams Still in use on all 11 teams after 6 months

    The gap isn’t because Monte Carlo is magic. It’s because cycle time is a measurement and points are a guess, and measurements beat guesses every time you run the experiment. The retention row matters most, though. As one PM told us bluntly after her team switched:

    “I don’t need it to be faster than my spreadsheet. I need to trust it. Right now I don’t know what it just did to my plan.”

    That’s the real indictment of the points ceremony. Not that it’s wrong — it’s often close enough — but that nobody can point at where the number came from when it changes.

    An illustrative snippet

    If you want to try the forecasting piece yourself before adopting the rest, the math is embarrassingly simple. Here’s the core of what Velo runs under the hood, in about twelve lines of Python:

    import random
    
    def forecast(cycle_times, remaining_items, trials=10_000):
        results = []
        for _ in range(trials):
            total = sum(random.choice(cycle_times) for _ in range(remaining_items))
            results.append(total)
        results.sort()
        return {
            "p50": results[trials // 2],
            "p85": results[int(trials * 0.85)],
            "p95": results[int(trials * 0.95)],
        }
    

    Feed it the last hundred closed tickets’ cycle times — or, if you’re just starting out, the durations from those two anchor projects — and you’ll have a better delivery forecast in one second than most teams produce in a quarter of planning meetings.

    What you lose, and why it’s fine

    A whiteboard covered in sticky notes with sizes scrawled on them, most of them medium.
    The universal end-state of every pointing session, drawn from life.

    You lose velocity charts. You lose the ceremony. You lose the ability to say “the team delivered forty-two points this sprint” in a board deck. In exchange, you get forecasts that are three times more accurate, roughly two hours of engineering time back per person per week, and — this is the part we didn’t expect — noticeably less argument in planning meetings. When the question shifts from how big is this to how do we slice this so it fits, the conversation gets concrete fast.

    Story points were a clever solution to a real problem in 2005. We think the problem has since been solved better by simply measuring what teams do and forecasting from the measurement. If you’re still counting Fibonacci cards on Tuesdays, we’d gently suggest it’s time to stop.

  • Why our retros stopped finding the real problem

    Why our retros stopped finding the real problem

    The Friday retro ritual

    Every team we have worked on runs the same shape of retro. Sixty minutes on a Friday, three columns in a Miro board, sticky notes for what went well, what went poorly, and what to try next. Someone dot-votes. Someone else copies the top three items into a Linear ticket that nobody opens again. We used to run it that way too.

    In Q2 of last year, we shipped a release that took down billing for four hours. The retro landed the following Friday. The dot-vote surfaced “unclear on-call handoff” as the top item. We wrote a Linear ticket to rewrite the on-call runbook. Six weeks later, we shipped a different release that broke webhook delivery for two hours. The retro found the same category of problem, worded slightly differently, and produced another ticket that also went nowhere.

    The on-call runbook was fine. The problem sat upstream of it, and nobody in the room was willing to say so on a Friday afternoon with the person who owned the decision sitting three seats away.

    Why the standard retro fails

    Retros as most teams run them optimise for social comfort, not for truth. Three failure modes we kept hitting:

    • Recency bias. The team remembers Thursday’s deploy noise, not Monday’s design decision that set the deploy up to fail.
    • Consensus bias. Dot-voting rewards items several people already agree on, which selects for symptoms over root causes. Root causes are usually held by one or two people who saw them early and stayed quiet.
    • Performance bias. A live meeting is a stage. People tell the version of the story that protects the relationship, not the version that would help the next team.

    The billing incident showed us all three. The engineer who had raised a concern about the migration plan two weeks earlier did not repeat that concern in the room. Nobody wanted to spend the last hour of the week relitigating a decision that felt settled.

    The retro found something true and small. It missed the something true and large, because the format could not hold it.

    What we do instead

    We replaced the Friday retro with three artefacts, spread across the week. None of them takes more time than the meeting they replaced. We have run this shape for eleven months across two product squads and one platform squad.

    1. A written pre-mortem, filed on Wednesday

    Whoever owned the incident, feature, or sprint outcome writes a one-page pre-mortem in Notion. It is not a report of what happened. It is a written attempt to answer one question: if this failure repeats in six months, what will the story be? The author writes it alone, without review, and posts it in the squad Slack channel by end of Wednesday. It is time-boxed to forty-five minutes. Long documents mean somebody is hiding.

    2. A two-question survey, sent Thursday morning

    Every person on the squad, plus two adjacent stakeholders (usually a designer and a customer support lead), gets a Google Form with two questions:

    1. What did you see, hear, or think during this work that you did not say out loud?
    2. If you had a private ten-minute conversation with the person most responsible for the outcome, what would you ask?

    Answers are anonymised by the facilitator and pasted into the Notion doc under the pre-mortem. Response rate sits above ninety percent because the questions are specific and the form takes under five minutes.

    3. One blameless conversation, Monday at 10am

    The squad meets for thirty minutes on Monday. The pre-mortem and the survey answers are already in the room. The facilitator, who is not the tech lead, reads three or four survey answers aloud and asks the author of the pre-mortem to respond to them. No sticky notes. No dot-voting. No action items produced in the meeting itself. Proposals get added to the Notion doc during the following twenty-four hours, once people have had time to think.

    What changed

    The billing incident was one of eight retros we ran through the old format. The webhook incident was the ninth. Between month four and month eleven of the new format, we ran six retros across incidents of similar severity. Two produced Linear tickets that closed within a sprint. Three produced changes to how we scope Datadog dashboards before we ship, not after. One produced a decision to stop building a feature that two engineers privately thought would not land, and had not raised in a Friday meeting.

    The change is not that we find more problems. It is that the problems we find are the ones that matter. Three shifts explain most of it:

    • Writing before speaking gives people room to admit things they would not admit in a room.
    • Splitting the process across three days lets recency bias fade.
    • Removing the ritual of “action items produced in the meeting” removes the pressure to produce something visible, which is what pushes teams toward the easy, wrong answer.

    What we still get wrong

    The Monday conversation is fragile. If the facilitator lets it become a debate about the pre-mortem’s conclusions, it collapses back into the old format. We have had two of those in the last year. Both times, the survey answers that mattered most did not get read aloud, and the meeting ended with everyone agreeing on a symptom.

    We have also not solved the problem of what to do when the person most responsible for the outcome is the tech lead running the process. We rotate facilitation to a peer squad’s engineer in those cases, but it is not a clean answer.

    The retro, as most teams run it, is a meeting that produces the feeling of learning without the substance of it. If your Linear board carries three open “improve on-call handoff” tickets from three different retros, that is the signal.

  • The five-day slice rule

    The five-day slice rule

    Every Monday at 10:15 our delivery lead opens Linear and runs a filter called Cycle Ready. If a ticket in the active cycle fails the filter, it gets flagged red and the ticket owner has until standup Tuesday to fix it or pull it. The filter checks one thing: can a single engineer start work Monday morning and demo something running by Friday afternoon.

    We call this the five-day slice rule. It has been the single biggest change to how our team ships since we moved off two-week sprints eighteen months ago.

    Why five days, not ten

    We tried ten-day cycles for most of 2024. Tickets came in sized small, medium, or large, and engineers estimated them in points. By day seven of a typical cycle, roughly a third of medium tickets were still labelled In Progress with no visible artifact. Reviewers had nothing to look at. QA had nothing to queue. The last three days always became a scramble.

    The pattern was consistent enough that we started tracking it in Datadog against our Linear webhook data. Tickets that produced no reviewable artifact by day three of a ten-day cycle had a 72 percent chance of slipping the cycle. Tickets that did produce something by day three slipped 8 percent of the time. The signal was loud.

    We cut the cycle in half and enforced a hard rule on entry: one person, Monday start, Friday demo. If it does not fit, split it before the cycle starts, not during.

    How we enforce the rule

    The rule lives in three places:

    • A Linear template with a required field called Friday demo artifact. Engineers cannot move a ticket into Ready for Cycle without filling it in. Sample entries: a PR merged behind a flag and hitting a staging endpoint; a Grafana panel showing p95 for the new route; a Loom of the empty state rendering with fixture data.
    • A pre-cycle review meeting on Friday afternoon called Slice Check. Twenty five minutes, four people: engineering manager, tech lead, product manager, delivery lead. We read the Friday demo artifact field for every candidate ticket. If anyone at the table cannot picture the demo, the ticket does not enter.
    • A Slack bot posting into the delivery channel every Wednesday at 4pm with the list of active-cycle tickets that have no PR opened and no draft artifact linked. The message pings the ticket owner directly.

    The bot is the piece that took the longest to trust. We tuned it for six weeks before people stopped arguing with it. The current heuristic: no draft PR, no Loom link, no Notion doc updated in the last 48 hours, and the ticket is past cycle midpoint. Three signals, one ping.

    What happens when a team pushes back

    The rule gets fought. Usually by whichever team is holding the largest piece of unsplit work. We have heard every version of the objection:

    You cannot split a database migration into five-day slices. The migration either runs or it does not.

    The auth rewrite is one atomic change. Splitting it means shipping something broken.

    We are being asked to do more planning work than shipping work.

    We take these seriously and we still hold the line. Every migration we have run in the last year has split. Every auth change has split. The planning cost is real and it front-loads. Our data shows the front-loaded planning cost is roughly 90 minutes per split ticket, and it saves an average of 6 hours of end-of-cycle scramble per unsplit ticket that slips.

    When a team insists a piece of work cannot split, we sit down with them for a 30 minute session with a whiteboard and the delivery lead. We have run this session 41 times. It has produced a valid split 39 times. The two exceptions were a vendor cutover with an external deadline and a hotfix that shipped inside a day.

    A concrete example: splitting the profile export ticket

    Last quarter we had a ticket that read: add data export for user profiles, including preferences, integrations, activity history, and audit logs, downloadable as a signed ZIP. Original estimate: two weeks. Owner: one engineer on the Growth pod.

    Under the old rules this would have entered a cycle whole. Under the slice rule we ran it through Slice Check on the Friday before, and split it into three tickets:

    1. Slice one, week of Jan 13. Endpoint scaffold plus preferences payload. Friday demo: hit POST /exports/profile in staging, receive a signed URL, download a ZIP containing a single preferences.json file. Behind a flag. One engineer, five days.
    2. Slice two, week of Jan 20. Add integrations and activity history to the payload. Friday demo: same endpoint, same flag, ZIP now contains three files. Handles the 90 percent case of activity records fitting in a single query batch.
    3. Slice three, week of Feb 3. Audit logs, pagination for large history sets, signed URL expiry policy, and flag flip. Friday demo: end to end run for a real customer account with 40k audit rows, timing recorded in the demo doc.

    We put a two-week gap between slice two and slice three on purpose. That gap ran a quiet beta with three friendly customers on slice two, and the feedback moved the audit log format before we built it.

    Total calendar time: roughly the same as the original two-week estimate would have been had it not slipped. The difference is that we had something demoable at three checkpoints instead of one hopeful checkpoint at the end.

    What we track

    We keep three numbers on a Notion page called Delivery Health:

    • Percent of active-cycle tickets that hit their Friday demo artifact. Current: 88 percent, target 85.
    • Median time from a ticket entering Ready for Cycle to its first PR opened. Current: 1.2 days.
    • Number of tickets that entered a cycle unsplit and slipped. Current: 2 this quarter, down from 14 the same quarter last year.

    The rule is not magic. It is a constraint that forces the planning conversation to happen on Friday instead of Wednesday of week two, when a slip is already priced in. If the demo cannot be pictured on Friday, the work is not ready. That is the whole rule.

  • Velocity is a lie detector not a speedometer

    Velocity is a lie detector not a speedometer

    Every quarter we watch another engineering team roll a velocity chart into a review deck, gesture at the bars trending up and to the right, and declare progress. We used to do it too. Then we ran a small experiment on our own Velo delivery team: we hid the velocity number from three squads for a full quarter and gave them cycle time instead. Two of the three squads shipped more features. All three reported that planning felt less theatrical.

    This is not an argument against measurement. It is an argument against a specific metric that has quietly stopped telling us what we think it tells us.

    Velocity measures the ruler, not the road

    Story points are a ruler the team invented. When we grade a team on how many units of their own ruler they produce per sprint, we should not be surprised when the ruler starts stretching. We have watched this happen on our own boards. A ticket that would have been a 3 in April becomes a 5 by July. Nobody lies. Everyone remembers “that thing that turned out harder than expected,” and the estimate drifts up. The chart climbs. The output does not.

    Goodhart’s law shows up in the standup:

    When a measure becomes a target, it ceases to be a good measure. Velocity is the most polite example of this rule we have found in software.

    The other failure mode is subtler. Velocity averages hide the shape of the work. A team can hit 42 points every sprint for six sprints and be quietly falling apart, because 40 of those points come from a single engineer who is one Slack DM away from resigning. The bar chart cannot see that. Cycle time can.

    Cycle time is boring, which is the point

    Cycle time is the elapsed clock between “in progress” and “done.” It is boring because it measures reality rather than an estimate. We cannot inflate it by talking about it in a Wednesday grooming session. It refuses to care about our narrative.

    Here is the comparison we now put in front of every engineering lead we hire:

    • Unit. Velocity uses story points, a team invented currency. Cycle time uses hours or days, a currency everyone shares.
    • Gameable by. Velocity is inflated by re estimation, ticket splitting, and status theatre. Cycle time is inflated only by shipping faster, which is what we wanted.
    • Sensitive to. Velocity is sensitive to who is in the room during planning poker. Cycle time is sensitive to review queues, environment flakiness, and handoffs, the things that slow us down.
    • What it hides. Velocity hides bottlenecks behind an average. Cycle time exposes them by widening the tail of the distribution.
    • Actionable signal. A dropping velocity prompts a debate about commitment. A widening cycle time prompts a debate about the pull request that has been open since Tuesday.

    What we changed on our own board

    We run Linear for tickets and pipe every state transition into a Datadog dashboard. Once a week, on Thursday afternoon, we look at three numbers together:

    1. Median cycle time for tickets closed that week.
    2. The 90th percentile of the same distribution.
    3. The count of pull requests older than 48 hours.

    We deliberately do not look at velocity anymore. When a stakeholder in Notion asks “how much did we ship this quarter,” we answer with a count of shipped tickets and a link to the changelog. If they push, we show the cycle time trend. Nobody has pushed twice.

    The 90th percentile is the metric that changed our behaviour the most. Medians are polite. They hide the ticket that sat in code review for eleven days because the reviewer was on parental leave and nobody rerouted it. The 90th percentile has forced us to build a bot in Slack that pings the review channel every morning at 9:15 with any pull request older than a day. Our median moved a little. Our tail moved a lot.

    The objections we still hear

    Two objections show up in every conversation about this, and both deserve a real answer.

    The first is that cycle time punishes big work. A refactor that takes two weeks will show up as a fat cycle time number and drag the median. Our response: split the refactor. Not into fake tickets that ship in isolation, but into shippable, reversible steps. If a piece of work cannot be split, that is itself a finding, and the fat number is telling us something true.

    The second objection is that cycle time can also be gamed. Engineers can open tickets late, close them early, or keep everything in draft. Fair. We have seen all three. The difference is that these games are visible in the version control log and in the Linear activity feed. Velocity inflation is invisible, because the ruler itself is invisible. A gamed cycle time becomes a conversation. A gamed velocity becomes a slide.

    The metric we would keep if we could only keep one

    If a new engineering leader joined tomorrow and asked which single number to track, we would not hesitate. Track the 90th percentile of cycle time, week over week, and set a target for the tail rather than the average. That number is close enough to reality that people can argue about it usefully, and far enough from planning theatre that it does not warp under pressure. Velocity charts belong in a museum next to lines of code per day. We stopped drawing them and started drawing something harder to fake.