“Refuse to lose”: how executive pressure contributes to IT failure

“We went live before the system was ready”.  It’s a common excuse/explanation that I hear from IT people when they tell war stories about system launches that failed miserably. Implicit (and sometimes explicit) is the add-on statement: “and we told them so beforehand, too.”

There are obviously many things (and many parts of the org chart) that contribute to a failed launch, but here I’d like to focus on what drives this particular kind of launch-before-readiness, where the views of the rank-and-file are unheard or ignored.

In a nutshell: it’s management pressure. Sometimes that pressure comes from middle management, sometimes from the very top, and often from both.

Why do executives apply the pressure? There are a number of reasons:

  • They actually believe it’s the best way to keep the team producing and working hard. They fear that extending the deadline will cause a let-up.
  • Their own bonus and/or job is on the line if there’s a delay. They, or their management, regard a launch delay as a failure, just as much as a bad launch. At the very least, a delay means losing face.
  • Testosterone (for either gender) sets in: a “refuse to lose” mentality. To some extent, that’s what has gotten them where they are.

Deadlines matter, of course, a lot, and people who are lackadaisical about deadlines simply shouldn’t work in IT. That said, if you bull-headedly insist on adhering to deadlines, to the exclusion of facts-based input, it makes you all the more likely to delude yourself that you’re really ready, just because the launch date has arrived and the pressures are heavy. If the stakes aren’t high, that may even be fine. (But, are the stakes ever not high in these situations?)

Let me tell three anecdotes that illustrate management “refuse to lose”:

  • Earlier in my career, I was a middle manager, wrestling with a project that was both controlled by and staffed with a single “Big 5” consulting vendor’s people, with millions of dollars per month of consulting costs being racked up.  The high-profile initial system launch was a week away, and according to the bug tracker, there were still over 100 open major bugs. I sat down with the vendor’s partner-level project head. I don’t remember the precise conversation, of course, but it went something like this from him: “Oh, we’ve already got 39 of those 100 fixed in dev, 44 others are being worked by our best people and will be resolved by tonight, so really, there are only about 15. Four of those we think have workarounds so there are something like 10. I don’t think we’ll be getting many more bugs. We usually resolve bugs in a day or two, so even if each one takes two days, we’ll be able to close those ten and be ready for launch in a week.” I sat in awe (and frustration) at this hand-waving, this magic with math, and at his reluctance to think things through without an agenda. Here, “refuse to lose” had developed blinders, simple math had turned fuzzy and infused with vested interest, and IT best practices (not to mention common sense) had flown out the window.
  • I worked as VP of Operations under a dev-oriented CTO. We had a late night launch, and the CTO was there in person, hovering over my sysadmins who were attempting to install the latest release into production with little success. The installation had failed, starting with the very first run of the script. “No, try this: run just those three lines of the installation script we provided you, then type these two commands, then run these other five lines.”  When that didn’t work, he came up with another idea, and then another. The next day, he sent out a company email declaring victory and congratulating his dev team, because the system had come up finally, even though it had run for a total of only two hours before crashing hard. Installation by seat of the pants. Refuse to lose.
  • I arrived on the scene as the new CTO of a company in the midst of a complete replatforming project, completely retooling and replacing their sole product. The project had been underway for about nine months already; just before the point of my arrival, the team had been told by senior management that their launch date was going to be in six weeks, no ifs, ands, or buts.  I proceeded to do what I call a quick “temperature check”: I asked to see key indicators of readiness, such as bug counts over time and test case execution reports.

As you’ve surely guessed, I quickly determined that the facts didn’t even remotely support the possibility of a launch in a mere six weeks. About 40% of the code was still to be written; less than half of the test cases had been executed at all, with an even lower percentage having passed.  I marshaled my facts and figures, and went in to tell the CEO of this sorry state.  Judging from my experience, I told him, it looked like it would be a number of months, not weeks, before the product could be successfully launched. I remember him sputtering “that’s unacceptable”, and pressuring me on what I could do to, yes, bring off a launch in six weeks. (Cutting to the end of the story: we launched, successfully, nine months later).  In this case, a mixture of testosterone and probably some job-on-the-line aspects were driving this CEO to ignore facts and look for ways to bull through to success despite obvious counter-indicators.

The tragedy of the space shuttle Challenger has become almost cliché along these lines: a combination of executives feeling pressure and ignoring communication from lower ranks about major problems. “They also disregarded warnings from engineers about the dangers of launching posed by the cold temperatures of that morning and had failed to adequately report these technical concerns to their superiors.”  For whatever reasons, NASA management gave the green light for launch despite the adverse weather conditions. In essence, they “refused to lose”, and by doing that, they lost in a major way.

Takeaways:

  • Don’t let your executives turn you, and don’t let them become, riverboat gamblers, where failure is a strong possibility through a premature launch. The company culture must be shaped so that it views a spectacularly failed launch as much more onerous than a delayed launch.
  • Don’t let “groupthink” settle in, where your people are afraid to speak up. Any and every member of the team should have a chance to “pull the emergency brake”.  Especially, holding a formal “go/no-go” meeting is essential, where any stakeholder or project participant can voice concerns and point out facts that speak against readiness.
  • That said, do be careful of the pessimist amid your ranks, who typically voices dire predictions on every project, and who sooner or later will be proved right. Insist on facts, not “Eeyore-ism”.
  • Let your team watchwords for these situations become the following: candor, and facts. Bring both to the meetings, or go home.
  • The best way to avoid letting pressure drive a premature launch: establish your launch criteria clearly and openly, with quantitative measurements, way in advance. If you don’t have legitimate facts-based ways to tell if you’re ready (i.e., not just the launch date arriving, and not your testosterone level that morning), well then, you’re not ready.

Without a doubt, “refuse to lose” can be a great attitude, up to a point. No executive, no team, should ever become blasé about missing a target. But “refuse to lose” can’t ever be allowed to become “refuse to face facts”. When it does, you don’t just lose face: you just plain lose.

Comments

  1. Great perspective on the pressures on IT projects and over pressure before truly being ready to launch leads to disaster. I would bet a review of a failed launch, re-work, then re-launch financials would show noticeable costs compared to delaying the launch date and completing the project successfully the first launch. Have been on my share of recorded director level conference calls where product, project IT, operations IT all state their concerns, yet the launch is still a go … followed by pain backing out a failed launch and more pain avoiding the spinning wheel of blame.

  2. I agree with your comments.

    Early in my career we implemented a major system which was not quite ready but the sponsor browbeat the team into submission. They meekly agreed. The result was a major disaster requiring more than six months of cleanup that nearly brought the company down.

    Take away for me is to listen to the small voices of caution rather than override them. A delay is always better than a disaster.

  3. A significant part of the problem is with the CEO’s attitude that the facts are “not acceptable.” In a funny way he’s literally right: the facts are neither acceptable nor unacceptable, they are either true or false (or possibly in between).

    Some executives are used to bending reality to their wishes. It’s a kind of arrogance that doesn’t work on software projects. I just wrote about this at http://criticalresults.wordpress.com/2009/10/23/essential-humility/

    In short, everyone involved in the process, from the dev team on up, needs to distinguish among wishes, opinions, aspirations, and facts. We also have to accept that we don’t know as much about the project today as we will tomorrow, and that we’ll never know everything.

  4. On the ‘groupthink’ comment, I think it is important to consider diversity of opinions. You need all the parties represented and you actually want to hear from all concerned parties. It might be from a lowly developer in a corner or system administrator or someone in the business, but you’ve got to hear their voices and concerns. There are important and valuable viewpoints that are spoken lightly by someone.

Trackbacks

  1. Social comments and analytics for this post…

    This post was mentioned on Twitter by JohnFMoore: Good post by @PeterKretzman on Exec pressure sometimes leading to #itfail: http://bit.ly/4c5dzk

  2. […] “Refuse to lose”: how executive pressure contributes to IT failure by Peter Kretzman on CTO/CIO perspectives […]

  3. […] Refuse to Lose: how executive pressure contributes to IT failure. A good look at the ways managerial pressure to complete IT projects results in failed cutovers. Been there, done that. There are good principles in here; in this job market, would anyone push back? […]

Speak Your Mind

*