The case against #NoEstimates: the bottom line

I’ve now methodically presented the case against #NoEstimates in three different lights: from a common sense standpoint, from the perspective of the solid reasons why estimates are useful, and by examining the various frequent talking points used by NoEstimates advocates.  Looked at from any of these angles, NoEstimates comes up way short on both its core ideas and business practicality.

Aside from these issues of substance, let’s look briefly at the behavior of the NoEstimates proponents. Blunt as it may be, here’s my summary of the behaviors I’ve seen across most NoEstimates posts and tweets:

  • Presenting, and repeating via redundant tweets month after month, fallacy-riddled arguments consisting primarily of anecdotal horror stories, jibes at evil management, snide cartoons, and vague declarations that “there are better ways.”
  • Providing little or no detail or concrete proposals on their approach; relying (for literally years now) on stating that “we’re just exploring” or “there are better ways”
  • Consistently dodging substantive engagement with critics, and often even openly questioning whether critics should even have a voice in the discussion. If NoEstimates avoids engaging actively in the marketplace of ideas and debate, why should their arguments be taken seriously? Real progress in understanding any controversial topic requires we do more than state and restate our own views, but actually engage with those who disagree.
  • Continuing to use discredited examples and statistics to help “prove” their case.
  • Frequent use of epithets to describe NoEstimates critics: “trolls”, liars, “morons”, “box of rocks”, and more.

I pointed out in my introduction that the lofty claims of the NoEstimates movement (essentially, that software development can and should be an exception to the natural, useful, and pervasive use of estimates in every other walk of life) carry a heavy burden of proof. Not only have they failed to meet that burden, they’ve barely attempted to, at least not the way that most people normally set about justifying a specific stance on anything.

But aside from style, let’s return to the substance of the issue. Here’s my take, as backed by specific examples over the course of these blog posts: estimates are an important part of the process of collaboratively setting reasonable targets, goals, commitments. Indeed, whether estimates are explicit or implicit, they’re a reality. I see them as an unavoidable and indispensable factor in business.

[Read more...]

The case against #NoEstimates, part 3: NoEstimates arguments and their weaknesses

I’ve spent the last two blog posts introducing the #NoEstimates movement, first discussing what it appears to espouse, and presenting some initial reasons why I reject it. I then covered the many solid reasons why it makes sense to use estimates in software development.

This time, let’s go through, in detail, the various arguments put forward commonly by the NoEstimates advocates in their opposition to estimates and in their explanation of their approach. Full disclosure: I’ve attempted to include the major NoEstimates arguments, but this won’t be a balanced presentation by any means; I find these arguments all seriously flawed, and I’ll explain why in each case.

Here we go, point by point:

  • “Estimates aren’t accurate, and can’t be established with certainty”

Let’s use Ron Jeffries’ statement as an example of this stance:

“Estimates are difficult. When requirements are vague — and it seems that they always are — then the best conceivable estimates would also be very vague. Accurate estimation becomes essentially impossible. Even with clear requirements — and it seems that they never are — it is still almost impossible to know how long something will take, because we’ve never done it before. “

“Accurate” is simply the wrong standard to apply to estimates. It’d be great if they could be totally accurate, but it should be understood at all times that by nature they probably are not. They are merely a team’s best shot, using the best knowledge available at the time, and they’re used to establish an initial meaningful plan that can be monitored and adjusted moving forward. They’re a tool, not an outcome. As such, the benefits of estimates, and their contributions to the planning and tracking process, exist even without them being strictly “accurate” per se. These benefits were itemized in my last post.

Knowing the future precisely isn’t what estimating is about, actually. It’s a misunderstanding and a disservice to think it is. Here’s why. [Read more...]

The case against #NoEstimates, part 1: introduction and common sense

In the immortal words of Pogo, “we have met the enemy, and he is us.”

The long-held stereotype of IT portrays us as uncooperative, unable to integrate socially, always arguing over nits, and deeply, intractably immersed in our own tunnel vision and parochial perspective.

That stereotype has held us back, as individuals and as a profession: it’s actually one root cause for the oft-lamented situation of the CIO needing a seat at the executive table and not getting it.

Now a whole movement has arisen that unfortunately reinforces that negative perception of IT people, a movement that coalesces around the Twitter hashtag #NoEstimates (all movements need a Twitter hashtag now, it appears). It started with long screeds about how inaccurate estimates are for software development: they’re nothing more than guesses; “there is nothing about them that makes them necessary, or even beneficial to the actual creation of software”. They’re a “wasteful and deceptive practice“, lies, and needing them is even comparable to how heroin users need their heroin. You can’t predict the future, they insist. The #NoEstimates rhetoric has become increasingly harsh, and replete with drastic imagery: estimates are a “game of fools“; an “inherited disease for the industry“; “we predict like gypsy ladies.” In what seems at times to be some kind of “top the previous hyperbole” competition, estimates have even been referred to as “management by violence”.

This sort of overreaction, this IT resistance to estimates, isn’t really wholly new, of course. More than once, I’ve watched in horror as an IT person earnestly explained to a company’s senior management how predicting systems delivery is so very difficult and so very filled with uncertainty, justifying how (for example) they couldn’t possibly commit to having a system deployed in a particular quarter. One time, I witnessed the VP of Sales in particular having zero patience with that: “Hey, I get asked to set my sales quotas way in advance based on my best professional judgment; what makes you folks in IT think that you should be an exception?” Business runs on goals and commitments, and on the “best judgment” estimates that help create those goals and commitments. And everyone in the room seems to understand that, except IT.

Setting achievable, concrete goals is healthy, in business and in life. Making solid, reasonable commitments is healthy. Taking responsibility for meeting one’s commitments all or at least most of the time is natural and should be encouraged. And if you’re perceived as constantly wailing that you’re different and special, in fact so special that you believe you can’t really be expected to state when you’ll be done? That’s a non-starter. This blog post is an introduction to how some IT practitioners have been pulled in by the seductive but ultimately wrongheaded NoEstimates claims, and what the resulting implications are for our industry.

What’s #NoEstimates about? Well, its proponents are actually quite slippery on providing any solid definition, reflexively pushing back whenever anyone tries to summarize it for them, and they often prefer to vaguely say it’s just “a hashtag for a concept about alternatives to estimates and how they might help make better decisions.” However, it seems to boil down to a virulent and unshakable base conviction that estimates are utterly horrible, for all the reasons stated above and more. Using estimates allegedly leads to chronic abuse of developers by management, cements inflexibility into projects, and disturbs developer “flow”. The sole alternative to estimates that the NoEstimates advocates clearly identify, however, seems to simply restate the age-old Agile principles of small teams, user stories sliced into doable chunks, use of “drip funding” so as to achieve a predictable fixed cost, and software delivery “early and often”. They point out that software project scope tends to change drastically and frequently during such frequent delivery, and argue that this scope variability allows projects to be declared as done and successful, long before and without ever having actually delivered all the upfront-identified desired functionality.

Over the next two blog posts, I’ll first lay out the reasons to see considerable value in a software development estimating process and its outcomes, and then respond to the myriad NoEstimates complaints that are levied against estimates as used for software development. But we’ll start here, as an intro, just with invoking basic common sense about life and business.

[Read more...]

Towards a more balanced list of content about #NoEstimates

Both my readers will have noticed there’s been a fairly large gap between my posts here, as life (picnic, lightning, and all that) has intervened. Like J.D. Salinger, however, I have continued writing drafts on various topics, and I plan to post more in the coming months.

My past posts here have often delved into a favorite theme of mine: that IT people tend to go to extremes, often rejecting something useful (an approach, a technology, a tool) simply because it has downsides. Such rejection is at times emotional and even self-righteous; we can get so caught up in it that we fail to look at a topic at all evenhandedly, let alone dispassionately.

No better case example along these lines has come along in the past year than the active and contentious #NoEstimates debate on Twitter and in the blogosphere. I’ll have a much more detailed post soon about my objections to the #NoEstimates approach overall (full disclosure: I’m one of its most vocal critics), but right now, let’s focus on one aspect of the relentless advocacy I see in the hashtag’s proponents: its lack of evenhandedness.

Specifically, proponents of #NoEstimates insist repeatedly and proudly that they’re “exploring”; recently, one major advocate tweeted out a call for links to posts about the topic (“I’m gathering links to #NoEstimates content”) so that these could be collected and posted. Yet, it turned out that only posts advocating one side of the issue would be included, even though the resulting list of links was then touted to people who might be “interested in exploring some ideas about #NoEstimates.” When challenged on this dubious interpretation of the meaning of “exploring”, the advocate then defiantly attached a disclaimer: “Warning! There are no links to “Estimate-driven” posts”.

Advocates can use their own blog for whatever purposes they want, of course. Yet, there’s an interesting split going on here: staunchly claiming to be “exploring”, while rejecting the inclusion of any summarizing or critical posts, and then sneeringly labeling all such posts as “estimate-driven.” There couldn’t be a clearer case study of IT black-and-white-ism, them vs us. Explore all you want, this behavior says, as long as you’re doing it on my side of the issue and on my terms. What, there’s a post that attempts to summarize both sides of the argument? Not interested.

[Read more...]

IT does the moonwalk: our endless search for absolutes

Scene: I was CTO at a high-traffic social networking site, circa ten years ago. It was one of those times when our site got crushed by unexpected sudden volume, due to being mentioned in an article in a prominent newspaper. My infrastructure manager walked into my office the next morning, ashen-faced. “We’re gonna get killed tomorrow unless we add ten front-end servers to our prod environment,” he proclaimed. A fairly common IT reaction: absolute, adamant, ominous.

Ten new servers? That was a nice round pulled-from-thin-air number, obviously, and by the time we talked through it, we actually found other, more practical, more feasible ways first to estimate and then handle the increased load. But to the infrastructure guy as he walked in, the situation was both dire and absolute, and he saw only one solution that should even be considered.

So now let’s look at another data point on IT psychology. Take the latest iPhone brouhaha: the quick “cracking” of the iPhone 5s Touch ID fingerprint scanning technology.  Amazingly, Touch ID has turned out to be less than perfect. Someone with $1,000 of equipment, plus lots of time, motivation, and patience, could conceivably fool the scanner. Meanwhile, what gets lost in the outrage over this turn of events is the notion that the technology might indeed be “good enough”, or “better than the alternative”. We forget the simple fact that the technology is primarily oriented to people who currently don’t use passcodes at all, and that it vastly improves general security for those sorts of users.  As one article pointed out, “The point of any security system isn’t to be unbreakable – there’s no such thing – but to be fit for purpose.”

My larger point: if there’s a problem or a difficulty or even a nuance to a particular approach’s applicability, a common IT practitioner’s instant reaction is that the approach or practice is absolute junk and should be completely avoided.

Similarly, we often reject fundamental improvements to a situation, simply because they are not perfect. We let “best get in the way of better.” On this general theme, an amusing tweet crossed my screen the other day. @rands wrote, “I find when an engineer says, ‘Less than ideal’, they often mean ‘Complete fucking catastrophe.’”  I laughed at this, of course, but partly because I’ve more often experienced that scenario in reverse: an engineer deciding, and then loudly and profanely proclaiming, that a situation was nothing short of a complete disaster, simply because it was less than ideal.

[Read more...]

IT conferences for the CIO: microcosms of industry trends

I’m back from attending ServiceNow’s Knowledge13 conference last month in Las Vegas, and have a grab bag of random thoughts and reactions to share as a result. As usual, these thoughts reach beyond any particular vendor or product niche.

For anyone not familiar with this company, ServiceNow is slowly and steadily developing a generalized platform (“ERP for IT”) for enterprise IT management, all the way from IT service management (ITSM) to (now, in a new offering) cloud orchestration and management of instances.

My attendance last year at this same conference broke a personal streak of almost 8 years of avoiding conferences altogether. My recap post from last year discusses how I discovered what I’d been missing: exposure to new approaches, new energy, and new perspectives that, like it or not, don’t just come from online.

In fact, it reminds me of the classic Woody Allen line about “I need the eggs”. Conferences are messy, chaotic, overwhelming, sipping from a firehose, and so on. But we keep going, because we need those eggs.

Here are some “eggs,” large and small, that I took away from this year’s experience.

[Read more...]

Novels of IT: The Phoenix Project

Nerd alert: it’s an exciting day for me when someone releases a new “novel of IT”. I’ve made it my mission to find and review several of these (now four) over the past couple of years, and I may be one of the few people out there who has read and reviewed all of them.

To recap: what do I mean by a “novel of IT”? It’s a term I coined to describe a fictionalized depiction of life in a corporate IT environment, usually bearing a number of intended lessons in tow about IT best practices, approaches, pitfalls. They’re generally not works of serious fiction; their audience is usually the lot of IT professionals rather than the broad public. (For example, I don’t include in this category two fine and recommended works that in fact aspire more to literature than to IT didacticism: Douglas Coupland’s Microserfs and Ellen Ullman’s The Bug).

As I’ve traveled through the fictional scenarios depicted in these four books, I’ve evolved criteria for what makes them successful (or not) in my eyes. In a novel of IT, I’m looking for a book that is both reasonably engaging as a novel and one that accurately portrays a broad swath of the inner workings, nuances, and personality types that are typically part of the landscape of IT in today’s world. Reading the book should provide a window into common dilemmas and disagreements regarding IT issues, lending perspective and insight into all parties’ motivations and interests.

I looked forward for many months to the release last week of The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win, by Gene Kim, Kevin Behr, and George Spafford,  after meeting and chatting with Gene Kim at a conference back in May of last year. I was greatly impressed at the time with Gene’s general demeanor, enthusiasm, and articulateness. He gave a rip-roaring presentation at the conference on “ITIL at Ludicrous Speeds: Rugged DevOps”: I recommend seeing him speak if you get the chance. I felt certain that his long-promised “novel of IT” would be a worthy addition to the collection of works in this category.

[Read more...]

The One True Way syndrome exemplified: the overstated case against code comments

I write frequently, and not without some exasperation, about the perennial search for the “silver bullet” in IT: the holy grail, the end-all, be-all solution to preventing IT failure.

The silver bullet has a very close and similarly pernicious internal twin cousin: the One True Way. That’s a technique or practice that is (usually) adopted by its IT aficionados as the key to overall success, with the important insistence that it will work as long as you follow it to the letter, in all cases, no matter what.

So this post will seemingly be about a specific (and low-level) development issue, but it’s only to serve as an example to illustrate this One True Way syndrome that is so prevalent in IT. At core, my takeaway boils down to the same old message I usually have when it comes to IT matters: be wary of something promising to fix all your problems. Be wary of absolutes. And be especially wary of the combination.

[Read more...]