IT extremism strikes again: the odd resistance to bug tracking

In information technology and software development, how many of our wounds are self-inflicted?

Here’s what I mean.

What I’ve seen happen, recurringly, in IT over the years and decades: idealistic but inexperienced people come along (within IT itself or in other departments within a company), to whom IT systems and issues seem to be easier than they in fact are. They are smart and earnest and oh-so-self-assured, but they also seem blissfully unburdened by much real understanding of past approaches.

They are dismissive of the need for much (if any) rigor. They generalize often quite broadly from very limited experience. Most notably, they fail to understand that what may be effective for an individual or even a small group of developers often doesn’t translate into working well for a team of any size. And, alas, there’s usually a whole host of consultants, book authors, and conference presenters who are willing and eager to feed their idealistic simplifications.

Over the years, I’ve seen a subset of developers in particular argue vehemently against any number of prudent and long-accepted IT practices: variously, things like source code control, scripted builds, reuse of code, and many others. (Oh, yes, and the use of estimates. Or, planning in general.) To be sure: it’s not just developers; we see seasoned industry analysts, for reputable firms, actually proposing that to get better quality, you need to “fire your QA team.” And actually getting applauded by many for “out of the box thinking.” Self-inflicted wounds.

But let’s talk about just one of these “throw out the long-used practice” memes that pops up regularly: dismissing the value of bug tracking.

How can anyone argue against tracking bugs? Unbelievably, they do, and vigorously.

A number of years back, I came into a struggling social networking company as their first CTO. Among other issues, I discovered that the dev team had basically stopped tracking bugs a year or two before, and were proud of that. Did that mean their software had no bugs? Not when I talked to the business stakeholders, who lamented that nearly every system was already bug-riddled, and getting worse by the release.

Why did the development team shun bug tracking? That wasn’t quite so clear.

They told me dismissively that Bugzilla (their former, now-mothballed bug tracking tool of choice) had just “filled up with a lot of text”. They cited all sorts of idealistic Agile justifications: if you really want to address a bug, you should do it right away, they opined, otherwise it’s not really important anyway. Index cards make the bug more visible than burying it in a tracking system. And so on. They showed me with great pride how they’d set up a light to turn green when all the automated tests passed. And hey, the light was green. In short, they were in deep denial about the proliferation of bugs. And no tracking meant no real data existed to contradict that claim, just people’s anecdotal impressions and vocal complaints.

But then, looking around the team’s work area, I found hundreds of index cards for various bugs, some of them many months old, randomly scattered in piles behind various workstations, scribbled with cryptic notes. No one knew for sure which bugs had been fixed, worked on, dismissed, etc. Least of all the stakeholders. And the end customers complained endlessly about bugs to the company’s customer care representatives, who then felt that no one internal really listened to them as the “voice of the customer.”

Recurring, self-inflicted wounds

So it’s déjà vu for me when I read extreme, wrong-headed tweets like this:

Various tweets

This isn’t just a one-individual disconnect or blind spot. It’s distressingly common among software developers and (often) their management: the mistaken but dogged belief that we’ll find all bugs, and we’ll just fix them as we find them. But that simply doesn’t happen on non-trivial systems or projects. As Joel Spolsky noted,  “programmers consistently seem to believe … that they can remember all their bugs or keep them on post-it notes.” But developers can’t, and they don’t. And anyone who has worked in a development shop of any reasonable size knows that developers can’t, and that even if they could, there are numerous compelling reasons to more formally track bugs anyway.

Half an hour of surfing (see the Lagniappe below) and a little brainstorming will get you a list of powerful reasons to do bug tracking more formally than using scattered index cards:

What you lose

Here are just a few of the things a team loses when there’s no use of bug tracking software:

  • Information is lacking that is needed for troubleshooting the issue and confirming a fix. Index cards rarely contain the full scenario needed to understand and resolve the bug: specifically, the detailed steps to reproduce it, the expected result, and the actual result. Instead, you tend to get scribbled notes like “the footer sometimes gets wonky in the checkout module”.
  • Other than leaning on “tribal knowledge,” (essentially, talking to individual heroes and gurus on the team), developers have to start from scratch every time an issue comes in. They can’t easily leverage what’s already been solved.
  • With just “tribal knowledge” and no searchable database, it’s hard to tell if a newly reported issue is a known problem in an unusual guise, or a completely new problem.
  • It’s difficult, lacking a historical database of bugs, to identify a pattern in the bugs that might suggest a more holistic fix than just addressing each individual bug.
  • Lacking a searchable database of bugs and the ensuing record of discovery/research on them, it’s much more difficult to transfer responsibility for a given bug from one developer to another.
  • Users have no clear process for ensuring or confirming that their problem has been recorded.
  • Users noticing software issues can’t easily look to see if those issues have already been reported.
  • In any normal business environment, people tend to submit issues multiple ways: phone, email, paper, even a hallway conversation. When those issues are not consolidated in one place, it becomes remarkably easy for issues to fall off the radar.
  • Having no system of record for problems worsens business relations and causes randomizing of the team: in particular, submitters tend to repeatedly request personal updates from someone they know.
  • With no central place for updates, managers can’t get much of the overview they need to make tactical and strategic decisions. So instead, they can spend a lot of time tracking down individuals to gauge where things stand.
  • Other than shuffling laboriously through a stack of cards, no one can easily understand how many known bugs are left to fix, what the severity of those bugs is, etc.
  • Release readiness evaluations can’t take into account the rate of bug discovery, which can be key to deciding if the software is truly launch-worthy.
  • A scribbled index card can’t be easily consulted later to find out what was the solution.

And to reject all these solid reasons (and yes, there are lots more) out of hand? Or to even opine, as one vocal Twitter poster actually did on this topic, that “memory is waste“? Well, there goes IT again, with another arrogant, amnesiac, self-inflicted wound. Bug tracking as a software development practice is tried and true, with far more upside than downside (in short, not all overhead is waste, contrary to what you’ll be told). Yet some firms inexplicably allow teams to reject bug tracking nonetheless, and that (i.e., misguided management) is the problem. In fact, as cited here,

“Much of the blame for [software project] failures [lies] on management shortcomings, in particular the failure of both customers and suppliers to follow known best practice. In IT, best practice is rarely practiced. Projects are often poorly defined, codes of practice are frequently ignored and there is a woeful inability to learn from past experience’.”

Bottom line: when you hear people dismissing the need for various long-standing, solidly understood processes in the development of software, it’s appropriate to be tremendously skeptical. Unless you want to be part of self-inflicting the next wound.

Lagniappe:

  • IssueTrak, “Top 10 Reasons Why Your Organization Needs Issue Tracking Software”
  • Sifter, “Why Use Bug and Issue Tracking?”
  • Wikipedia, “Bug Tracking System”
  • Joel Spolsky, “Painless Bug Tracking”, November 8, 2000
  • neilj, “Why I Don’t Use Bug Tracking Software”, April 7, 2010.
  • neilj, “Why I Still Don’t Use Bug Tracking Software”, April 8, 2010.
Mastodon