Your project managers are now certified scrum masters. You’ve divided your development teams into groups of six who track their work on a Kanban board and subdivide their effort into two-week sprints. The certified scrum masters are translating business requirements into stories. The development teams conduct regular estimation sessions during which they assign story points to those stories. Work is taken from the top of the backlog into each sprint. Burndown charts show the progress being made to the next release. And yet … progress is still slow. Projects still stall. Should we have listened to the people who said “This agile thing won’t work here”?
OK, I can’t specifically tell you why your agile initiative is failing. Every organization is different and has its own challenges and opportunities. However, there are many misconceptions about what agile is and how to apply it, and these misunderstandings have grown in number within organizations alongside the agile industry that sells related training and certificates.
How agile development evolved
Software used to be planned and delivered like civil engineering projects. There were project managers, Gannt charts, resource allocations, and detailed timetables. Analysts gathered requirements, architects drew up plans, lead programmers made designs, then coding started, followed by testing and release.
This method of software development became known as “waterfall.” Waterfall worked well in the early days of software development because applications were simpler, scarce computer resources needed careful allocation, and end users were trained. But 40 years after the personal computing revolution started, applications are complex, computer resources are ubiquitous, and everyone is an end user.
When everyone is an end user, usability becomes a key nonfunctional requirement, particularly when people expect applications to be intuitive and delightful. It’s hard to design good usability up front, as is required by a waterfall style of project management. Modern software needs to be developed in a more iterative and experimental way, gathering feedback from the potential audience as it is created. It’s why designers are now an important part of successful software development teams.
Agile software development offers many benefits:
-
Minimizing risk (in particular, the risk of not delivering something useful)
-
Ability to change direction quickly
-
Better alignment between delivery teams and user needs
-
Sustainable work for development teams
In practice, many organizations fail to realize these benefits of agile. An entire industry has grown up around agile, with prescriptive methodologies about how to do it right. However, it’s hard to achieve true agility by following a set of rules. Instead, you need to cultivate openness and curiosity, prioritize effectively, and frame success in terms of “what has been learned” as well as “what has been delivered.”
Software development projects are often framed too narrowly—e.g., We need to build X. Not enough time is spent thinking about the value delivered by X once it is built. For example: What is the problem it solves? How does it help someone? Who will use it? And what benefits will it bring? A software development lifecycle (SDLC) that doesn’t give proper consideration to these questions will not be agile because it won’t be clear when to pivot to meet those deeper goals.
In this article, we look at some common agile antipatterns and give guidance on proven agile practices. Although the agile manifesto stands as an important marker in the paradigm shift from waterfall to modern development practices, it is starting to show its age. Here, we draw on the evolution of agile practices pioneered by VMware Tanzu Labs (formerly Pivotal Labs).
Agile antipatterns
As a developer and consultant for more than 20 years, I’ve observed several agile antipatterns—methodologies that go against recommended agile practices. These antipatterns have appeared at different organizations and in many different kinds of businesses. They are not a deliberate attempt to frustrate an agile SDLC. More commonly, they are an attempt to squeeze agile ways of working into organizations that have not yet fully embraced agility. Here are some of the most common agile antipatterns I’ve encountered.
Big planning up front
This is the antipattern I see most frequently, also known as waterscrum, water-scrum-fall, or wagile.
A typical development pattern
Even before the “requirements capture” stage, portfolio planners are designing products years ahead. Then, requirements are gathered and development plans are made up front. The enterprise architects provide an approved architecture pattern, and plans are turned into epics on a backlog. Next, the development team starts breaking down the epics into stories and working in sprints toward a delivery milestone.
The scope and end date are more or less fixed, so where are the opportunities for the delivery team to be agile? What happens when early user feedback (if the team is fortunate enough to be able to gather it) reveals that the requirements capture hasn’t fully understood an actual workflow?
Long sprints
Each iteration or sprint in your agile project is a chance to show what you have done and adjust course if needed. Three- or four-week sprints are likely to have more waste than one-week sprints. They are also an indicator that stories are not being broken down into small enough pieces. Large stories are inherently riskier; they often require more discovery work and might hit more blockers.
It’s not easy to break a large story into small ones, each of which will deliver some value. But it is something that teams can learn to do, particularly with sympathetic coaching. The better teams can divide the work into small, useful pieces, the faster they can deliver software increments that show real progress.
Story points as proxies for time
After naming and cache invalidation, the next hardest problem in software is accurate estimation. Agile attempted to sidestep this by getting teams to estimate stories using points. Points are not supposed to map directly to time but rather to represent the complexity of a story, particularly in terms of “things we don’t fully understand yet.”
However, not having a definite timetable can be uncomfortable for many organizations, so there is a continual pressure to make a link between story points and time so that they know when something will be finished. This pressure gets stronger when organizations use the Scaled Agile Framework (SAFe), since it attempts to synchronize the work of different teams across the enterprise. Instead, product managers can use tools like outcome-oriented roadmaps to reassure stakeholders that there is a structure and a plan without committing to deliverables within arbitrary timeframes that might not actually be achievable.
When teams start being held accountable for these estimates, they are likely to pick bigger numbers so they have some headroom. In turn, this erodes some of the value of the estimation process. In a genuinely agile environment, estimates give the product manager some idea of the cost of implementation, which in turn helps them decide the priority order. For example, if feature X is twice as expensive as feature Y, is it twice as valuable?
When story points are misused as proxies for time, they are less useful for deciding priority order, and in many years of experience, I’ve rarely seen them being particularly useful for predicting how long something will take.
Ineffective retrospectives
Retrospectives are a chance for teams to reflect on what is going well and what is not. Retrospectives are valuable when the insights provided are used to improve processes and make it easier for teams to make progress. However, retrospectives can be ineffective for a number of reasons, including the following:
-
Lack of psychological safety (e.g., people are afraid to raise problems or suggest different ways of doing things)
-
Too much time between retrospectives
-
No remediations or actions identified to fix problems that are uncovered
-
No evaluation to see whether remediations were useful
Sometimes deep cultural change is needed to enable retrospectives to be useful, particularly if the problems are rooted in issues of psychological safety. However, even small changes can start to build toward that culture—this is an area where management can have a big impact by enabling teams to be more autonomous, to highlight remediations, or to ask for resolutions.
Schedule retrospectives regularly. Start with the habit of not missing an end-of-sprint retrospective, and always try to capture at least one or two actionable items to improve for the next sprint. If you can show small increments of improvement as the direct result of each retrospective, teams will start to take them more seriously. If you need a starting point, we’ve documented some of our favorite retrospective templates on Miro.
Four pillars of effective agile
At Tanzu Labs, our approach to agile is based on a set of guiding principles and core practices, honed from years of experience, and centered around four primary pillars:
-
Balanced teams
-
Extreme programming
-
Lean thinking
-
User-centered design
Each pillar is part of the supporting structure for agile development practice and reinforces the others.
Balanced teams
A team should have people with a mix of skills and perspectives. The classic balanced team for product development is a product manager, a product designer, and several engineers. The product stakeholders or sponsors establish the goals for a product. The product designer’s importance is often overlooked, but their role is critical because they are the person who investigates user needs and translates them into product requirements. The designer’s role is to articulate what users need. The engineer’s role is to translate wishes into working code, as well as to evaluate the relative cost. The product manager has to balance what the user wants against how that meets the goals for the product and the overall budget.
How balanced teams relate to a product
A balanced team works best when it has some autonomy and freedom of movement. The stakeholders have set the goals, but the balanced team will conduct research to determine how best to meet those goals, and perhaps experiment with different approaches to see what works best. The balanced team can also change direction when the thing being built does not meet the original goals.
Extreme programming
Many practices are associated with extreme programming (XP). The goals of XP are to improve software quality and responsiveness to changing requirements. Both of these are important in achieving an agile SDLC. At Tanzu Labs, we strongly advocate test-driven development (TDD), pair programming, and frequent refactoring—three practices that can improve software quality.
Software quality is important, not least because poor-quality software is brittle and hard to change. It’s hard to be agile when it’s hard to change your software. Frequent refactoring keeps technical debt low, and it’s easier and safer when you have a comprehensive test suite that you developed along with the software itself. Pair programming ensures that knowledge is spread evenly throughout the team. It enables you to avoid time-consuming, ineffective code reviews. It also makes it easier to onboard new team members.
Lean thinking
Lean thinking is about avoiding waste; that is, effort that does not contribute directly to reaching a goal. All large software projects will include some missteps; places where an assumption turned out to be incorrect, where a proposed solution did not work as expected. One way to view the development process is like this:
-
You have a list of stories on a backlog.
-
Each software increment is a hypothesis in the form of If we deliver this code, it will meet the acceptance criteria for this story.
-
Keeping increments small and frequent enables you to learn fast and keep waste low.
With big planning up front (antipattern no. 1), you have a lot of assumptions and hypotheses that are not going to be validated until the team starts writing code. Your assumptions are everything you believe to be true about the problem, as well as proposed solutions that are not directly backed by evidence.
Some assumptions are about your product. For example, you assume that your users want video guides to explain how to use the product. But when you start doing testing, you find that video guides are hard to follow when you are outside and accessing both an application and guide on a phone.
Other assumptions are technical. For example, you think that it will be straightforward to consume data held on your legacy platform. Then you discover that the data is held in a form not directly usable in your target environment.
Assumptions are not bad things! You need to make assumptions before you can start designing and planning because you don’t know everything when you start. Some of them, perhaps most of them, will be correct or close enough. But the earlier you can validate them, the lower the risk.
The longer you spend planning without writing code, the more waste there will be in your process. When you embark on any new project, you are forced to make assumptions for everything you don’t have actual data about. When you start the actual work, some assumptions will be proved correct and some will turn out to be wrong.
At Tanzu Labs, we prefer to plan just enough to get started, and many of our practices are designed as ways to gather and synthesize information quickly so that we can keep the planning phase short. This applies whether you are developing a brand-new consumer-focused application or modernizing a large back-end system in an enterprise.
One way to keep your thinking lean is to pick a single problem and fix it. This focuses the team’s attention on delivering something useful early on. This is particularly effective when starting a project to modernize existing software. What’s the most complained-about part of the current system? Can you improve it? Slice that piece off and work out how to deliver it and make it work alongside the existing software.
This gets you started on modernization far quicker than months of planning, and it gives you a chance to deliver value early in the project.
User-centered design
User-centered design puts the needs and objectives of users at the heart of the design process. By treating users as important stakeholders in the product, you ensure that what you are building is relevant, helpful, and delightful to use. User research at the start helps establish what your users need. As you start to build the product, you ask for regular feedback on designs, prototypes, and early versions so the outcome aligns better with your target audience.
Get started with agile
Having an agile culture and SDLC requires more than creating a backlog of stories and working in sprints or iterations. It’s about being ready to accept that there is uncertainty in any worthwhile project, and it’s about being ready to manage that uncertainty in useful ways.
Critics of agile practice worry that not fixing the scope of work in advance is undisciplined and chaotic. In fact, a successful agile practice requires rigorous discipline and planning; but the early planning is used to determine desired outcomes rather than the scope of work. Planning continues throughout the project in small increments, so learnings from each increment of delivery can be applied directly to the next planning exercise.
Not having a plan that says “Milestone A will contain X, Y, and Z and be delivered on date T” can be uncomfortable for organizations and can make coordination across teams difficult. But the history of software development shows that, often, Milestone A either doesn’t arrive on date T or it only contains X because Y and Z were harder to produce than expected. Agile development is simply being realistic about any team’s ability to accurately forecast more than a few days ahead.
To learn more, watch this webcast about how to scale agile from teams to enterprises, download our white paper about using balanced teams to make better software, and read this lightning summary of agile development. If you want to speak directly with our own product managers, engineers, and designers, we run free virtual office hour sessions that you can sign up for here.
