How to Prevent Your Agile Project from Turning into a Waterfall

To date, nearly every project has been Agile. But what is the success rate of the Agile Projects around you? Or how can you evaluate the success of an Agile Project?

To date, nearly every project has been Agile. But what is the success rate of the Agile Projects around you? Or how can you evaluate the success of an Agile Project? To answer those questions, let’s recall some of the reasons to choose Agile Methodology over Waterfall:

• To adopt alterations better due to changes in requirements or miscommunication
• To identify and solve problems on the delivery line quickly
• To enable the squad to perform better
• To gather analysis and development teams as one
• Higher Customer Involvement
• …

In most recent cases, “Squads” fail to achieve those goals. Instead, we observe:

• Teams that are waiting for dependencies
• Continuously growing backlog
• Unclear objectives and metrics
• Tasks spanning multiple sprints
• Increasing inefficiency times
• Postponed delivery
• War rooms and hives towards deadlines
• Lack of a broader roadmap and documentation

We aim for agility for better execution but encounter waterfalls within sprints. We observe that “waterfall” is encoded in most organizations’ genes. But what is the reason for that, and why can’t many projects achieve their intended goals? How can we reduce the risk of failure?

Waterfall within Sprints (Iterfalls)

During most of the daily standup meetings this morning, the squad said: “I dealt with this task yesterday and I will continue dealing with it today.”

This sentence is the mother of all evil in a project. It implies uncertainty, and most of the time, it means “I will finish it sometime”. For instance, if you assign the effort of the whole sprint to this product backlog item, you can almost be sure that it will not be completed in this sprint, and you will only be aware of that at the end.

A substantial number of dependencies are uncovered during sprint planning or after development. And once you discover them, it’s almost impossible for the dependent teams to solve them in time. So, you hear a sentence that many of us are familiar with: “We have our own task for this sprint. We can deliver your request in the 3rd sprint after this one.”

When you tell them that we need the dependent task to be done immediately at the very last minute, can you blame them?

We call those projects waterfalls with sprints. They may succeed, but we cannot consider them successful because they exceed the prospective budget and calendar. The blame is on a common misunderstanding about Agility. Let’s remember the frequently recited motto of the agile methodology: “Most of the decisions we will make today will change eventually. Therefore, we don’t have to decide on everything in the beginning.” This is an excellent point, right? But in practice, this is what people understand: “Most of the decisions that we are going to make today will change eventually. Therefore, before we start, we don’t have to decide on anything.”

Last but not least, have any of your sponsors ever dictated a deadline for the project? Did you end up promising something? Or did the sponsor negotiate for it? Most probably, you said: “We have a due date. So, we must do overtime for the next month, guys.”

Recalling the Discovery Phase

Let’s recall the typical agile delivery lifecycle. In this lifecycle, we start with project inception. This inception is generally done by conducting a requirement analysis and constructing the product backlog. We only go into details about some things. Instead, we create initiatives and epic stories and leave them to be detailed later in refinement ceremonies. We aim to shrink the initiative and epics systematically and have clear user stories once we start the sprint.

Figure 1 Typical Agile Delivery Lifecycle

Another aim of project inception is to balance the workforce, the scope, and the calendar. But we will not go into the details of that here.

Figure 2 Our Intention: Shrinking Epic Stories Before the Sprint

Humans tend to postpone complicated tasks and only see the rock once it hits. When it comes to our projects, what we continuously delay is shrinking significant stories. So, when the time comes, we hit epics and initiatives in sprint planning as depicted in the following figure:

Figure 3 Our Action: Trying to Clarify Postponed Epic Stories in Sprint Planning

Moreover, frequently, when we try to shrink a story, we realize that it needs to be less tangled to clarify as an atomic unit. Most of the time, we understand that we must clarify many other issues before finding a solution to the problem at hand.

Figure 4 Most of the stories are tangled and have dependencies

One significant advantage of a waterfall over agile methodology is that you can see your goals. Therefore, you can foresee the big rocks on the track. But you must keep pursuing many other advantages of an agile methodology over a waterfall, even if you encounter a critical problem.

Believe it or not, the problem is not Agile Methodology. The problem is you are more agile than it’s meant to be.

CAREFULLY PLAN YOUR DISCOVERY PHASE

Discovery Study is a partial analysis. It’s a sketch of what you need to do and when you need to do it. At DefineX, we carefully plan the project inception and do not rush to start the project since we know that skipping the Discovery Study is the highway to hell. We start the Discovery Study by drawing the as-is functional flow with different roles and environments as lanes. This functional flow has few details. We primarily focus on the interaction with other domains and functionality.

Draw the as-is functional flow. The second phase is the optimization of flow, which includes the elimination of unnecessary steps, determination of steps that you can run asynchronously, domains that you can decouple, etc. Then we redraw the flow accordingly.

Refactor the flow. After handshaking with the PO, we draw the architecture diagram, which indicates the interaction between the domains and technology layers to emphasize dependencies.

Draw the architecture diagram. Then as a detailed version of the solution, we redraw the functional flow, which indicates every technology stack as a lane—MQs, APIs, In-Memory Grids, NoSQL DBs, etc.

Support functional flow with technology stack. This final discovery output also serves as a basis for the technical inventory and initial backlog with epics and features.When you complete the discovery study and initiate the first sprint, you have a backlog supported with technical inventory and a dependency matrix. This stage means you will not have epic stories or initiatives in Sprint Planning. So, you will not guess but know what to do next, when to do it next, and what to expect.

Now all you must do is break down the user stories into single-day-or-less tasks in Sprint Planning. Hence you should aim to hear these words in daily planning: “I completed this task yesterday and I will complete this task today.” You will discover that this sentence can save your day.

The Waterfall is not a complete waste, and Agile Methodology is not a total savior. We must do more than hit the road and start a new project. We must sketch the project first. If we have this sketch, you’ll improve the chances of making promises that you can keep.