Why it’s so difficult to manage the work of delivering software
One of the classic American TV comedy moments was the chocolate factory episode from the 50’s sitcom I Love Lucy. The titular character Lucy and her best friend Ethel get jobs wrapping candies in a factory. What seems simple at first quickly turns into chaos as they cannot keep up with the endless stream of chocolates along the conveyor belt.
Watching this clip, you cannot help but think of similar moments of being overwhelmed at work. The conveyor belt of tasks piles up in email, Jira tickets, bug reports, and calendar invites for meetings. The constant stream never stops, there is always more work to fill up the empty moments you think you have in your day.
Several weeks back, a debate erupted over Twitter. The CEO of Basecamp had some opinions to share about the state of work:
“If your company requires you to work nights and weekends, your company is broken. This is a managerial problem, not your problem. This is a process problem, not a personal problem. This is an ownership problem, not an individual problem.”
He later clarified that the point was not about nights and weekends, but working additional hours outside of one’s regular work schedule. This sparked backlash from the workaholics contingent of Twitter that espoused the value and morality of hard work.
Whenever anyone advocates for work-life balance or reasonable approaches to work, the hustleporn universe of Twitter goes on attack mode. They lean on two arguments to support workaholism, first is that global competition requires it and second that it is the only path to greatness. These ideas are supported by many investors, executives, and startup founders who work their people and companies to exhaustion for some greater glory.
These arguments are strawmen unsupported by evidence or logic. Competitive advantage comes from market acceptance and product fit. Working harder rarely factors into creating a sustainable advantage, but confirmation bias allows this belief to persist. As for the pursuit of greatness, the idea that you cannot be good at your job unless you exhaust yourself is total bunk.
Alexis Ohanian, co-founder of Reddit, said it best of the “always on” work culture:
“This idea that unless you are suffering, grinding, working every hour of every day, you’re not working hard enough, this is one of the most toxic, dangerous things in tech right now.”
I should know as I managed to burnout a decade ago working on my startup. I calculated that if I reduced my sleep to 4 hours per night, I could have 20 hours per day, 7 days a week to focus on work. I could outdo the Marissa Mayer 130 hour work week schedule. The outcome of my ridiculous pursuit of “greatness” was startup failure, a frayed marriage, near bankruptcy, and ongoing mental turmoil.
It is not just in tech or startups where this culture of workaholism thrives. I see the same thing in banks, insurance firms, airlines, car manufacturers, retailers, etc. It is particularly acute in IT departments though, where deploys happen on weekends, production issues mean all hands on deck, and unreasonable release schedules cause a near death march to the finish line.
The era of Agile and scrum teams was supposed to bring more predictability to software delivery. That was also the same promise however with waterfall where all the work was visualized and planned and tied to staff and timelines in massive Gantt charts. The long wait to get working software however lead to numerous problems, so Agile came along to aid the delivery of software through smaller batches and faster cycles.
To support this new way of delivering software, organizations adopted ideas from lean manufacturing. This led to the use of Kanban, storyboards, and other ways to visualize and estimate work in process. Dominica DeGrandis shares these concepts in her book Making Work Visible and identifies the five “time thieves” that conspire to prevent the flow of work:
Too much work in progress — queue is overflowing with more work than resources
Unknown dependencies — work that depends on other related work or processes
Unplanned work — surprises, firefights, and interruptions that halt current work
Conflicting priorities — competition of resources & budget within an organization
Neglected work — unfinished work usually the result of the other factors above
Yet even with the ability to “see” work in process, this may not result in consistent delivery of software and the effective use of time. Kanban and other visual metaphors cannot account for two broad organizational challenges in the planning of work. First is organizational dysfunction. Second is the perception of technology work as repetitive tasks.
The people doing work typically get their marching orders from stakeholders that only consider the outputs. However, they rarely understand the obstacles and friction that actively fight against the delivery of a new product or feature. Technical debt, infighting between teams, lack of institutional information, increasing technical complexity, and legacy dependencies are more often dismissed as excuses for poor delivery.
Second challenge is related to the first in that our understanding of what work is in the modern age has not changed. We still think of work like a conveyor belt. In concepts such as the Toyota Way, work reflects known processes and regularly executed tasks done at a particular pace. Each car on the assembly line resembles the next one and is predictable. When an anomaly appears, workers can pull on the Andon cord to halt work and fix the problem.
The nature of work however in knowledge companies is not as repetitive and predictable. Knowledge work is variable and the more creative the work, the greater the variability. This is especially true of software development. While reuse of code, adoption of frameworks, and setting common standards can bring more order to the process of building software, software development can still appear chaotic and messy because of the complexity of the problems to be solved and dependencies that need to be considered.
This misunderstanding of the modern software delivery process in turn leads to poor prioritizing. I call this the “software is the easy part” fallacy that arises due to executives simply believing programming is akin to manufacturing software using code widgets and commodity coding labor.
The overwork we observe in organizations is a direct consequence of the lack of delivering quality software due to constraints and the fact that software itself is a creative exercise. Enterprises misallocate resources towards firefighting over the work that would prevent firefighting in the first place. This creates breaks in the flow of work, as we often see in the deployment of new features to production. What should take minutes can often take weeks or months.
Organizations need to stop managing to the expectations of physical production and move towards managing in ways that optimize for creative problem solving. The first step is to visualize the work as Dominica outlines in her excellent book. Once the work is clearly laid out, map out the dependencies and roadblocks in the flow of work.
With this information in hand, you have ample evidence to make the case to stakeholders to reduce overwork as a means to increase productivity. This is the only way to bring order and sanity back from the toxic culture of workaholism infecting companies and the hero culture that is so pervasive in IT organizations. If we cannot get the work done in a sensible 40 hour work week, our default response should be to fix the processes, not add hours.
How are you helping your team and colleagues to find work-life balance? Are there ways that you found to improve the balance of work in queue so you are not overwhelmed?
Time Theft: How Hidden and Unplanned Work Commit the Perfect Crime | Dominica DeGrandis
I had the pleasure of hearing Dominica speak at DevOpsDays NYC 2019 and she truly is one of the foremost experts on helping companies improve the flow of their work.
Look out later this week for our next podcast episode and also check out past episodes on Apple Podcasts, Google Podcast, or wherever you listen to your favorite podcasts. We would appreciate your likes and please do subscribe 😁
We help IT leaders in enterprises solve the cultural challenges involved in digital transformation and move towards a community based culture that delivers innovation and customer value faster. Learn more about our work here.