Tyranny of the Timebox Revisited

Oct 13, 2020 10 min read
Tyranny of the Timebox Revisited

Over the past 15 years, I have observed that managing dependencies has been a constant source of anxiety and a huge challenge for organizations adopting Agile methods. Meanwhile, dependency management has largely been a non-issue for organizations using the Kanban Method.

One of the most important, anti-Agile articles I´ve written is titled, The Tyranny of the Ever-decreasing Timebox (published in 2014, based on a speech I gave in 2009). This article  “The Tyranny of Timebox Revised” is it`s new updated and revised  version. Over the past 15 years, I have observed that managing  dependencies has been a constant source of anxiety and a huge challenge  for organizations adopting Agile methods. Meanwhile, over the same time  period, dependency management has largely been a non-issue for  organizations using the Kanban Method. This week we release our general solution for dependency management using Kanban, and in a series of articles, starting with this one, we look at why it is so much easier to manage dependencies with Kanban than  with Agile methods based on time-boxed increments (sprints) such as  Scrum and SAFe. This article revisits, revises and extends the original text from 2014, updating it to include the topic of dependencies.

Timeboxes

Agile software development methods, with the odd long since forgotten  exception¹, adopt the use of fixed time increments, often wrongly  called “iterations”². In Scrum, these are known as Sprints. A Sprint is a  fixed period of time with a defined scope and a commitment³ to complete  that scope within that time window. Originally, Scrum defined 4-week  sprints. This was changed later, circa 2004, to a recommendation for  2-week sprints as the default. In general, it is recognized that agility  is related to the frequency of interaction with the customers or  business owners, and the frequency of deliveries. Hence, smaller  timeboxes are more desirable, if you wish to pursue greater agility.

Batches

Software quality is often related to batch size and the time  taken to complete a piece of work. The relationship is known to be  non-linear, i.e. the defect rate rises faster and accelerates away as  the batch size or length of time extends. Hence, smaller batches,  completed in short periods of time lead to dramatically fewer defects.  So, in theory, small batches of work are desirable.

Two ways to constrain small batches of work – timeboxes or WIP constraints

There are two ways to constrain the batch size of work:  constraining the amount of time available to do the work, resulting in  scoping to small numbers of requests that can be completed in the given  time; or, simply constraining the number of work items, constraining the  size of the batch of requests, also known as WIP constraints. All  mainstream Agile methods adopt the use of timeboxes to constrain batch  size by scoping work to fit in the available time. The Kanban Method  instead adopts WIP constraints to directly constrain batch size.

At a small scale and with lower maturity organizations, it  doesn´t much matter which of these two strategies you pick – they are  both effective. Scrum is a perfectly good methodology to take a team  from maturity level 0 to 1. However, as you both scale your ambition and  the size of the organization, and improve the organizational maturity,  with the goal of consistently delivering customer demand, within  customer expectations, at larger scales (a product unit of 150 people,  to a business unit of 600 to 1500 people or more), the timebox strategy  breaks down. You cannot scale agility and organizational maturity with  team-level timeboxes. The remainder of this article explains why not…

The Pressure for Shorter Timeboxes

As a result of all the advantages of smaller batches – higher  quality, more frequent interaction between customers and the delivery  organization, and the potential gains from earlier delivery of valuable,  working software (also known as the avoidance of opportunity cost of  delay), organizations have been under pressure to pursue shorter and  shorter sprint lengths.

The Challenges of Shorter Timeboxes On the face of it, smaller timeboxes and hence smaller batch sizes  for the sprint backlog, are a good thing. However, smaller timeboxes  create three types of pressure that are often difficult to cope with and  adjust to: firstly, smaller batches require an ever more detailed  approach to requirements analysis and development – the need to write  ever more fine-grained story which can be completed within the smaller  time window; secondly, the need for an ever more accurate approach to  estimation, so that a realistic commitment can be made; and finally, the  need to know whether a piece of work will be affected by dependencies,  and if so, a need to track and manage such dependencies, between teams  and multiple sprint backlogs, and potentially, across sprint boundaries.  A failure at any of these three things causes the timebox approach to  break down and fail spectacularly. The timebox approach is inherently  fragile on a large scale. Let´s examine why…

Requirements Analysis

Developing capability with a new and hopefully better  requirements analysis technique, designed to provide the granularity to  ensure work items are sufficiently small as to fit neatly into a short  timebox, is hard, even when you know with confidence which technique you  should adopt. The challenge with very short sprints is that there is,  still after 20 years of Agile, little or no solid guidance on writing  fine-grained, consistently small user stories. Even if there is a good  method that your team is comfortable using, then (a) you now have a lot  of analysis upfront, before sprint planning, and (b) you have introduced  a peer-to-peer, or parent-child dependency management problem, that  becomes especially acute as soon as you are unable to select all of the  peers for the same sprint, with the same team.

Fine-grained requirements analysis coupled to short timeboxed sprints introduces a dependency management problem to Agile methodologies.

Alternatively, in the absence of strong guidance on  requirements analysis, anxiety leads to undesirable effects such as  breaking stories up into functional units based on information discovery  activities such as “architecture story”, “design story”, “development  story,” “test story” where the real customer value may now span across  multiple sprints and peer-to-peer dependencies between stories across  sprints is now a tracking requirement. This type of breakdown defeats  the purpose of the ever-smaller timeboxes and creates a false sense of  agility when in fact, customer value and quality are not improving,  perhaps the opposite may even be true. A failure to instrument for and  track customer lead time disguises the problem, and Agile teams focused  locally and only on themselves, merrily ignore that they are not  delivering customer value.

The concept of a ¨Design Sprint¨ (or other similar, upstream,  pre-commitment information discovery) is now a recognized thing and you  can take classes in it. Be under no illusion, design sprints are an  anti`-pattern, they are an indicator that timeboxed increments in Agile  methods are a dead-end – they do not scale! To be Agile you need to be  small – small teams, small products, small codebase. To scale, you need  to abandon agility and use much larger timeboxes.

Estimation

As sprints become shorter and shorter, the challenge of knowing  whether a work item can be completed within the time limit becomes more  and more important. Consequently, over the last 10-15 years, I have  watched on the sidelines as the Agile community has pursued ever more  elaborate means of estimation. The shorter the timebox, the more upfront  effort is required to estimate whether the work will fit into the  available time. Big estimation up front, is an anti-pattern and destroys  the concept of agility – you need to estimate everything in your  backlog first to play some magical game of Tetris trying to fit  odd-shaped, fine-grained requirements into the sprint backlog. As we  shrink the timebox, we increase the economic costs (the transaction  costs of sprint planning) and decrease the efficiency of sprints. It is  neither Agile or Lean.

Dependencies

What if a story in our sprint backlog gets blocked because of a  dependency? That might prevent it from completing on time. And hence,  the shorter the sprint timebox, the greater the need to identify  dependencies upfront. This requires yet more analysis of as yet  uncommitted items in the product backlog. And what if there are  dependencies? Agile methodologies resort to organization design to solve  this – the magical cross-functional team. The idea that everyone on the  team can do everything that might be required to produce a top-quality  product. For any firm that aspires to market leadership, this has never  been true – you don´t win at anything with generalists, winning requires  specialists. A small band of elite generalists can and does happen but  again it doesn´t scale. You don´t staff a 500 person IT department, or a  600-person business unit with elite generalists.

All too often we see organizations, like one of those in a case  study we use daily in our training classes – they had 5 development  teams, 3 data-base administrators, and 2 user-experience designers. So  either, you put the two UX guys in each of 2 teams, and the 3 database  guys in the other 3 teams and now you have 2 front-end teams and 3  back-end teams, and you try to plan the sprints such that those teams  only pick front-end or back-end stories, and oh whoops, you are  disconnected from the customer requested functionality and have created  dependencies, or you accept the truth and you use the dba´s and the UX  people as shared services, and once again you have dependencies.

Dependencies are a fact of life in all but very small-scale  software development, or mediocre development done by largely dilettante  generalists. If you are doing anything at scale, or anything that  aspires to market leadership then you have specialists, and you have  dependencies. The Agile movement has been in denial of this basic truth –  to be Agile you have to aspire to be mediocre!

Scaling, Business-agility and Timeboxes

Scaled Agile Framework (SAFe) approach to dependencies, is to  reverse agility and instead use 3-month timeboxes called product  increments. Effectively rolling the clock back to 1994, a big planning  up-front approach, involving big analysis and big estimation up front –  and lots of red string – to produce incredibly brittle plans that are  unlikely to survive the first few of the 13 weeks intended. Scaled Agile  Framework and its product increments and PI Planning quite simply  aren´t Agile. To cope with the challenges and anxieties of agilists  everywhere, it simply gives them permission to stop being Agile and do  everything in quarterly time-boxes. It is an elegant solution but it  isn´t Agile, rather it is the antithesis of Agile and the antithesis of  the motivation for the lightweight methods movement of the late 1990s  that evolved into the Agile movement.

Dependency Management Anxiety is Rooted in the Sprint Constraint

If you want to be Agile beyond the team level, beyond maturity  level 1, then you have to remove the constraint of timeboxes to control  batch size. A time constraint is a fantastic way to take a chaotic  organization from the anarchy of maturity level 0 to the control and  predictability of maturity level 1, but that is where it ends.  Time-constrained sprints are an evolutionary cul-de-sac, a dead-end. You  cannot scale Agile using timeboxed sprints!

The answer is to focus on quality and short delivery times, by  using a WIP constraint, rather than a time constraint. Only the Kanban  Method offers this for organizations struggling to scale Agile.

How to Start your Journey to Large-Scale Business Agility

I am now convinced that this pattern, taken from Kanban Maturity  Model 2nd edition page 133, is a necessary step to large scale agility.

For each Scrum team, leave everything else alone, except you remove  the sprint timebox – stories started dont have to finish within 2  weeks. Instead of limiting the time, limit the WIP, use a constant  work-in-progress (CONWIP) limit. Initially, replenishment cadence and  delivery cadence can be left synchronized with the former sprint  cadence, eventually, you are likely to tune these to the needs of the  business domain, the transaction, and coordination costs of holding the  meetings, and the arrival rate of new information, that affects  selection, sequencing, scheduling, and priority. This is your first step  on a journey of evolutionary change, and your first step on the road to  large scale business agility.

Relief from the Tyranny of the Timebox This change frees you from the tyranny of the timebox, and its  three dysfunctions of upfront analysis, excessive and costly estimation,  and heavy-weight dependency management.

Avoid: Will it fit within the timebox?

If things take, however long they take, then there is no need  for upfront analysis to break an item into smaller items, no need to  track complicated dependencies between sprints or across teams. Instead,  let the work break down happen naturally once the work is committed for  delivery. Track that work with a 2-tiered kanban board, parking lots,  and avatars showing the involvement of shared services.

There is no need for awkward elaborate, big analysis upfront,  estimation efforts. Simply track historical lead time through your  workflow and use the lead time distribution chart to determine  probabilistically how long something might take to complete.

Avoid: Will it get delayed by a dependency?

If things take, however long they take,  then there is no need for upfront analysis to determine the need for  specialists or expertise, nor do you  need to constantly reorganize pursuing the Nirvana of the perfect  cross-functional team that never needs assistance from outside. Let  dependencies happen as you discover them, create visibility on to them,  actively track them. Use a service-oriented approach to your  organization design and utilize shared services for elite, specialist,  market leading capabilities.

A General Solution for Dependency Management

If you need to pursue large-scale business  agility, then you need to drop the use of time constraints to control  batch size and improve quality and time-to-market. Instead you need to  use WIP constraints and embrace the specialist shared services in your organization. Stop reorganizing! Start flowing work across a network of interdependent services!

To have a truly effective general solution for dependency management,  you need to understand the opportunity cost of delay. You need to  understand the urgency of a piece of work, and how dependency delays  affect it. We have released that solution as an infographic poster in print-ready PDF format. You can also find it in Appendix F of the new Kanban Maturity Model 2nd edition available now.

If you are serious about enterprise scale business agility, we know how to take you there. Contact us for more information.

___________________________________

¹ Feature-driven Development, from Peter Coad and Jeff De Luca

² Iteration implies that something should be reworked. Iterative  methods in software engineering such as Barry Boehm´s Spiral Method  already existed in the literature long before Agile methodologies. Agile  methods use incremental development with each piece of scope building  on the last rather than reworking that which has gone before, although  reworking to improve the fidelity of functionality, is both possible and  feasible with Agile methods, it is unusual, particularly with Scrum.

³  Since, the original article the Scrum Guide has been modified to remove  ¨commitment¨ and replace it with ¨forecast¨. While most organizations  still use the term Sprint Commitment and customers intepret it as a  promise, the official guide has modified it to what we might consider a  ¨soft commitment¨, an indicator rather than a hard promise.

Great! Next, complete checkout for full access to Kanban Maturity Model Blog.
Welcome back! You've successfully signed in.
You've successfully subscribed to Kanban Maturity Model Blog.
Success! Your account is fully activated, you now have access to all content.
Success! Your billing info has been updated.
Your billing was not updated.