A lead time story in the digital world

Three months ago, Michael Ballé visited us at Theodo and, once again, asked us where he could see the lead time of our digital products. Embarrassed, we answered: “Well… um… you know, it's actually very hard to see how long it takes, because…”. We really had no good reason not to measure it.

At Theodo, we build and deploy web-based applications for our clients. Companies come to us when they need a technical team to build their application quickly. To start a project, we have to convince our clients that we deliver value faster than others. Isn't that a good reason to measure how fast we deliver?

DEFINE WHAT AND WHEN TO MEASURE

First of all, let me give you some context on how we work. Day after day, our developers build what is written on a Trello card - what we call user stories. They represent the smallest piece of a feature that can be shipped, and they cannot take more than a day to build. A group of user stories makes up an epic, which represents a complete feature. In turn, a project is made up of several epics.

For example, an epic could be: “As a salesperson, I can send an invoice to the customer for their order”. And one of the stories to achieve this might be: "As a salesperson, in the customer's order list, when I click the 'Send invoice' button, I see a preview of the invoice before sending it".

Lead time is the time between the moment you receive an order from a customer and the moment it is delivered to them. In our environment, customers order applications without knowing exactly what their components will be; then, week after week, they add epics. Finally, they decide when to start building those features, and the team creates stories for that epic.

So our initial problem was understanding which lead time to track. Should we focus on the project lead time? On the epic lead time? Or on the user story lead time? We couldn't choose, so we decided to track all of them, defining each one as follows:

• The lead time of a project starts at the first contact with the client and ends when the first feature is publicly released.
• The lead time of an epic starts when the epic is created by the product owner and ends when the last user story of the epic is released to production.
• The lead time of a user story starts when it is created and ends when it is released to production.

I started working with my colleagues Frédéric and Grégoire (both are lead developers on different projects) on the last two lead times. Frédéric focused on the lead time of his team's user stories, while Gregoire set out to understand the lead time of his team's epics. Our first results showed that we took between 27 and 86 days to deliver epics and between 1 and 43 days to deliver user stories.

IS THERE A PROBLEM TO SOLVE?

At that point, we wondered whether or not we had a problem on our hands. Without a clear standard, we couldn't tell whether those numbers were bad or not. So both Frédéric and Grégoire set out to define the lead time they thought we should aim for and confirmed with the client that they were acceptable.

Frédéric learned that the ideal lead time for user stories is less than 10 days:

• 7 days in the analysis phase: between creation and the first line of code.
• 1 day in the development phase: between the first and the last line of code in the user story.
• 1 day in the validation phase: between the last line of code and the moment the product owner validates the user story.
• 1 hour in the deployment phase: from validation until the user story is deployed online.

We found that 7 user stories out of 19 in total were late:

• 5 took more than 7 days in the analysis phase.
• 3 took more than 1 day in the development phase.
• 3 took more than 1 day in the validation phase.
• 18 took more than 1h in the deployment phase.

For his part, Grégoire learned that epics should be done in less than 28 days:

• 14 days in the analysis phase: between the creation of the epic and the creation of the user stories.
• 7 days in the ready-for-development phase: between the creation of the user stories and the first line of code.
• 7 days in the development phase: between the first line of code and the release of the epic's last user story.

Based on this, we found that 8 epics out of 10 in total were late:

• 8 took more than 14 days in the analysis phase.
• 3 took more than 7 days in the ready-for-development phase.
• 1 took more than 7 days in the development phase.

The cost of inventory

With a clear understanding of the first steps needed to get closer to the standard lead time, we can now tackle the analysis phase: the inventory of stories the team will have to develop. We dug into Frédéric's project to see the “age” of the stories. Eighty-eight percent of them were more than 7 days old.

This phase can be divided into three steps. First, when created, the user story is stored in the backlog. At some point, the product owner prioritizes the backlog. He moves what he wants the team to develop into another column called "to be specified". Once a week, the team specifies how to code all the tickets in that column. At the end, they move the story to the “specified” column before developing it. Here is the breakdown:

• In the backlog: 65 of the 65 stories were more than 7 days old.
• “To be specified”: 14 of the 17 stories were more than 7 days old.
• “Specified”: 6 out of a total of 14 were more than 7 days old.

In Gregoire's project, this exercise was even more important, since 11 epics out of 11 in total had been created more than 14 days earlier.

We know that inventory has many consequences, even if we have not yet estimated its cost. However, we were aware that the team went over each item, week after week, asking themselves whether they should do it right away or later. For his part, the product owner spent a lot of time managing his backlog, having to find the best way to prioritize stories and epics. In addition, our clients were missing business opportunities. Users waiting for features to help improve their productivity are disappointed not to see them released, and clients still suffer from features for which there are improvements in the backlog, and so they see their sales drop little by little.

To reduce lead time, we realized we needed to reduce the inventory of epics and stories. The first step was to identify the sources of the stories and how many stories they generated.

1. 18.7% of stories are created from end-user feedback and the problems they run into in the application.
2. 28% of stories are created by the team when they find something to improve.
3. 53.3% is work-in-process inventory: what we couldn't finish in previous weeks, the investigations we started and couldn't complete, and the stories the product owner didn't prioritize.

This revealed all the waste we have in our process. The analysis phase is our inventory of stories and epics. All week long, the team moves stories and epics up and down the backlog to prioritize them. They read each story more than once to make sure whether or not it is the right time to work on it - a kind of overprocessing. And we also overproduce, adding new stories to the backlog, identifying technical strategies for them while they are defective.

NO PULL AND BROKEN FLOW

We focused on overproduction and defects. Years ago, we set a minimum inventory of stories that the product owner needs to create to make sure the team always has something to do, but not a maximum. But we found, among other things, that one product owner kept creating stories even though the backlog was already full. Here, the pull system seemed to be broken, with the product owner pushing stories without considering the team's capacity. Ironically, the product owner complained that he didn't have enough time to work on important things, such as getting user feedback, running user experiments, and so on. It seems we found a solution to his lack of time!

There is another interesting thing. We don't distinguish between designing the feature and splitting the epics into user stories. This happens at the same time, which results in a lot of missing or incomplete information. It also results in other departments (designers, translators, marketing, infrastructure team) discovering at the last moment that they need to do something critical for the feature's delivery. In such cases, the flow is interrupted.

That's where we are today. The window is a little cleaner, and although I believe the future will remain challenging, we are now aware of what we are missing. More importantly, we are now facing our problems, which means our tomorrow can only be brighter and full of new learning and innovation. I'll keep you posted on our progress in a few months ...