I recently had the privilege of being interviewed, along with other experienced senior technology executives, by CIO magazine for my thoughts on communication mistakes still made by CIOs. Some great ideas came out in the article, but when it comes to communication (see tip #1 below), there’s always more to say. So here goes.

• Communication can always be worked on and improved. I was at one company where we did a semiannual employee satisfaction survey. Even better, the company was admirably dogged about implementing specific measures to address areas of dissatisfaction that emerged from the survey results. But in every single survey, the number one vote-getter was the need to improve intracompany communications, no matter what initiatives were spawned to improve them. Communication is an ongoing challenge and necessity.

• Don’t just communicate upwards. Communication with your team is every bit as vital as your communication to management and to peers. You need your team to have a clear and common understanding of the goals, as well as getting their active contribution and buy-in on the necessary approach to getting there. You can’t go it alone.
• Hierarchy is actually helpful; use it actively. Do your best to avoid providing explicit nuts-and-bolts-level direction to staff members other than your direct reports. Jumping in and telling people at all levels what to do confuses people mightily, undermines the authority of your direct reports, and increases the likelihood of mixed signals. And it’s just plain inefficient. Early in my career, I was a director responsible for a major week-long software rollout in one of my company’s regional offices. The project manager of the initiative was one of my direct reports. But then, the stakes were high enough that the CIO decided to fly down, and of course, the regional VP of IT was there as well. So for a week, we had four management-level people all sitting in every meeting, each trying to actively demonstrate they were in charge and look good to the others. Or, there’s out-and-out, from-the-top, intentional bypassing of the chain of command: Amazingly, I’ve seen cases of a CEO setting up a task force without involving or even informing the people who managed the members of the task force. Talk about undermining!
•  Be clear about what’s actually a directive and what’s just an exploration. Dawn Lepore, CEO of Drugstore.com, recently spoke in an interview about how she couches her direction to her staff as to whether it’s a “a light bulb or a gun”. “A light bulb means this is just an idea I had, so think about it, see if you think it’s a good one. Either follow up or don’t, but it’s just an idea. A gun is, I want you to do this.” Surprisingly, people often can’t tell the difference on their own. As a leader, you typically want most meetings, particularly at a senior level, to feature open brainstorming, a free exchange of out-of-the-box ideas, but that means it’s all the more important to carefully identify what emerges as actionable initiatives and what things continue to be simply thrown around as possibilities.
• Put it in writing. Supplement the directions you give in meetings and one-on-ones with a clear and succinct written summary. Doing this both clarifies and makes an unambiguous record of what you’ve asked for, and accurately communicates it more broadly than just to the people who happened to be in the meeting. It’s a way of demonstrating that you’re declaring yourself accountable. Doing so is vastly preferable to having your people run around claiming “The CIO wants this.” (And with some, once they discover that such a line grants them all sorts of vicarious power, they’ll tend to overuse it, and at inappropriate points). I’ve seen a lot of ineffective leaders follow some kind of old-school “just tell them” mentality, and they cause significant confusion in the ranks by failing to put things in writing. And it’s no wonder that with this approach, which evolves into a kind of “Telephone” game, they’re often disappointed in what they get.

Oral communication is often vague and ambiguous. Written communication is on the record. Strive to go on the record as much as possible.

• Be utterly clear about who owns the execution of the initiative, and who will be making specific lower-level related decisions. Most notably, avoid having people think that everything needs to be cleared with you at all junctures. For most initiatives, your role is to set the overall goal, not to shepherd day-to-day the specific process of getting there. Lack of clarity here is common, even when everything else is done right.

Common sense, all these tips, no? The thing about common sense, though, is that it’s often frightfully uncommon.

How do we (IT executives) get away from being typecast as technologists, unconsulted on core business issues and approaches? Face it, that’s a common situation and dilemma that we all encounter, early and often, and it’s the grist for a constant mill of articles and blog posts and books on business/IT alignment.

Lately, though, a part of that mill has started insisting that focus on technology should be avoided altogether by what they usually cast as the “next generation” of CIO.  So I’m going to (again) be a bit of a contrarian here: it’s possible for the pendulum to swing too far in the wrong direction. I think that we can at times go overboard in our desire to avoid being seen as the geek with the pocket protector.  Examples: some preach outright denial that there might be such a perception problem: don’t even think of using the terms “IT” and “business”, they urge, and they recommend against ever discussing “alignment” as a goal.  Stop referring to the “business” as something separate, they recommend; IT is just as much part of the business as anything else! Similarly, their advice is “avoid discussing the technology itself.” As if a mere shift in language could solve the perception problem and automatically propel the CIO into the inner circle of decision-makers.

Here’s the gist of how I see it, though: in many (I daresay most) companies, the path of IT from high priesthood to strategic key playerdom has not really been fully traversed: in other words, greater alignment IS still needed of IT with “the business.”

In many companies, there’s still a trust problem. IT is arcane. IT can be as resented as a roadblock as the in-house corporate counsel types who veto certain kinds of corner-cutting.  Of course we’re “part of the business” (and increasingly so), but that doesn’t mean it’s wrong to continue to focus on breaking down the walls that linger.  The walls, in many/most places, are a reality, and it’s going to take steady, relentless, active work, not just a shift in language, to scale them. What especially will not help our effort to do that is if we embrace this odd trend of sneering at technology as a lesser-order concern—basically, positing that we should let someone else do that while we do the important (strategic) stuff.

In other words, I differ (vehemently) with those folks who now look down on such things as the need to align IT with the business, or who don’t recognize the benefits that come from regarding people as “internal customers”, or who recommend even avoiding using the terms “IT” and “business”. While the basic spirit behind those sentiments is of course valid—again, it’s undeniable that IT is part of the business, as much as any other department—it is in fact IT that has a problem in how it’s perceived by other elements of the company, going back to the high priesthood. Let’s be frank: IT folks have often been snooty, vague, cagey even, about what we do, why we do it, and how we’re meeting the company’s business needs as a whole. Deprecating the important progress and mentality shift that’s occurred in the last few years (that is, the high priesthood evolving to a service mentality) is exactly the wrong approach.

And somehow, inexplicably, the deprecators are getting louder, despite years of research and general agreement on the value of approaches such as IT governance, the ongoing need for IT/business alignment, etc.

Taken to an extreme, insisting loudly that IT is part of the business can extend to feeling that IT knows just as much about what’s good for certain business processes (say, manufacturing or shipping or collections or whatever) as the people executing those functions on a daily basis.  From there, it’s just a small but perilous step back into the old sort of IT arrogance, where systems were designed and deployed with little or no feedback from the actual users.  I exaggerate? Quite possibly, but I’ve seen it happen.

IT needs to both know its place and insist on its place, not one or the other. Here’s another example of a truly ham-fisted (even though well-intentioned) attempt to align IT more closely with business: I took over as CTO at a high-profile internet company, in the midst of deep, bet-the-company changes to its main product, a consumer-facing web site. I learned in my first few weeks that my predecessor had initiated a robot project.  His idea was apparently that he’d program this robot to walk the halls of the company, greeting people and thus making them feel more comfortable with technology. As Dave Barry is fond of saying, I am not making this up.  Needless to say, this project swiftly became a laughingstock across the company, representative of IT embracing technology for technology’s sake, completely OUT of alignment with business needs. Meanwhile, systems were crashing, projects delayed, etc.  This CTO somehow thought he was being strategic and business-oriented, but he actually was just being plain clueless, and more tech-focused than ever.

With IT’s background and history, the push towards IT/business alignment is ongoing and needs to be continual, rather than dismissed as something we’ve now evolved beyond. Even more, alignment mustn’t be deprecated as somehow beneath the higher goal of strategy. It’s not an either/or, and it most certainly doesn’t happen through mere declaration. And openly striving for alignment with our peers is nothing that anyone should present as a taboo or a sign of weakness. Additionally, at the same time that we serve as service providers to the company, we can and must be strategic in our outlook. This shouldn’t be either a surprise or controversial; in fact, excellence in tactical delivery is the ante that gets you to the strategic table.

It’s taken decades to get where we are—and I’d say (given, for example, the anecdote I cite above) that most of us are not really there yet anyway. Let’s not risk reinstating the “ivory tower” of strategy, devoid of the reality that comes from grappling with the actual pragmatics of implementation.

So, here are MY three tips on how to help convert your CEO’s understanding of where IT fits into the bigger picture. You’ll notice the emphasis of these points is quite different from the similar list in the piece I’m responding to here: in particular, my points stress a healthy balance between strategy and tactics, technology and business.

• Make sure there’s a rock-solid prioritization mechanism for all projects, where the priorities are clearly set, the tradeoffs carefully examined, and tough choices explicitly made and regularly revisited by the group of senior management, including the senior technology executive, who are jointly responsible for the business success of the company.  Focus on IT governance, Project Portfolio Management, etc.

• Speak up visibly and strongly to ensure that “roof projects” are given appropriate attention and priority. Build for the day after tomorrow rather than just for tomorrow. You in IT are a key advocate for this sort of basic corporate responsibility; you have and need to exercise your equal seat at the table.  And don’t let a misguidedly monomaniacal focus on strategy derail your obligation to keep the trains running on time. Nothing undermines the case for IT to be viewed as having a greater strategic role more than if the company is experiencing ongoing crises in basic delivery and systems.

• Be the straw that stirs the drink. IT, given a leader with appropriate vision and perspective, can and should be a natural cross-functional leader in the enterprise, both initiating business innovation AND espousing tactical responsibility.  They go hand in hand.

Let me end by quoting my colleague Steve Romero, who wrote, “Recognize that the “achievement” of alignment does not mean you’re done. This is where the “journey” analogy comes in. Think of IT-Business Alignment as “getting your head above water.” Once you reach the surface, you are not done. You need to keep treading water or you sink again. IT-Business Alignment is something that must be maintained as opposed to achieved.”

Lagniappe:

• Michael Krigsman, “IT failure? Blame your CEO.”  February 16, 2010
• Steve Romero, “IT-Business Alignment is Not a Meaningless Catchphrase“, October 28, 2009
• Michael Pattison, “Is IT – Business Alignment Meaningless?“, November 6, 2009
• Bob Evans, “Global CIO: Suicide Strategy For CIOs: Aligning IT With The Business“, October 13, 2009
• Fred Cummins, “Is Business-IT Alignment Suicide for the CIO?“, October 21, 2009
• Mary Nugent, “The Four Phases of IT/Business Alignment“, December 10, 2004
• Paul A. Strassman, “What is Alignment?  Alignment is The Delivery of the Required Results“, Cutter IT Journal, August 1998

A few years back, I had an extremely surprising and unpleasant experience as CTO. The director of my Program Management Office ran a weekly status meeting for project stakeholders, where we’d all methodically go through the current project portfolio, in order to communicate on issues, gather necessary feedback, and align everyone’s expectations. I typically attended in order to provide input and executive-level decision participation, but left it to the director to actually run the meeting and present the topics.

Unfortunately, immediately before one of these weekly meetings, that director was given bad news (in a brief hallway conversation, no less) about a major bug that had just been discovered in the software for our highest profile project, which was currently in testing and due to launch in just a couple of weeks.  This project, with its strategic and revenue-enhancing potential, was foremost in the minds of everyone in the company.  Stakes were high, in other words.

Without conferring with me or anyone else, the director announced to the entire group of assembled stakeholders that the project had hit an insurmountable snag in terms of our target launch date, and that it would be delayed potentially by weeks. No further information was available on specifics, and people’s concerned questions went unanswered. You could feel the anxiety spread through the room. I actually had to truncate the meeting abruptly and promise to reassemble the group when we knew more.

My director was quite upset with me. Rather than having my action be understood as a necessary attempt to prevent panic until we could get some real facts, I suddenly realized that I was being accused of wanting to “hide information” from the stakeholders. The director even told me, “Well, I like to be honest and tell it like it is, but that obviously just isn’t what is wanted here.” I was stunned. Here was a person who had the wrong idea of transparency, and astonishingly for their position, a warped idea of what project management is all about.

As most experienced project managers know, it makes utterly no sense to drag stakeholders through the entrails of every back-and-forth readjustment that is discussed and worked through in the course of a project’s lifetime.  To do so lowers stakeholder confidence, unnecessarily in many instances.  In this case, the director somehow wanted to be a kind of Paul Revere, forewarning the stakeholders, but instead came off as Chicken Little. Transparency can’t be an act of individual heroism; it needs to be a concerted, institutionalized, collaborative philosophy and approach.

Moreover, good project management isn’t about making everything go smoothly. There’s probably no project plan in the history of the world that has gone completely smoothly from beginning to end.  Instead, project management is about responding appropriately when it doesn’t, and recalibrating to keep everything still on track. In fact, it should be expected that not everything will go as planned, and your whole team should be prepared to take some bumps and glitches in stride without panicking.

It’s not hiding anything to hold back on discussing, in a public forum, a problem that has just been discovered, the ripple effects of which initially can only be guessed at.  When you promulgate potential bad news prematurely, it puts you into the position of trotting out the absolute worst case, before you really have any idea whether that worst case is likely or even plausible. In fact, it’s a fundamentally selfish act: it takes your own personal panic and extends it to as many people as possible.  And if there’s one thing we know about panic, it’s that it’s contagious.

You need to shoot for judicious and organized transparency. That’s where you plan on regular status communications, and you methodically, soberly assemble the facts in a structured manner before you make those announcements. Most of all, you collaborate internally: before you bleat about a potential crisis to the stakeholders, you bring in the rest of your team to talk through the possibilities and “degrees of freedom” in how you’ll act with respect to that crisis. Otherwise, you may think you’re nobly playing Paul Revere, but instead of imparting critical facts, you’re shouting “The British might be coming!”  And that’s not transparency, that’s mere alarmism.  Remember that especially during the dog days of a stressful IT system development effort, what often seems like a dire setback, upon careful and collaborative examination in the light of day, turns out to be merely a minor speed bump that is easily overcome.

Is it “controlling the message” to be cautious about spreading the alarm? You bet it is. And doing that is in everyone’s interest. Remember my adage: show that you’re in control. Your job as project manager (and, by extension, the job of the Director of the PMO, and the CIO) is to be in control and to demonstrate to the users that you’re in control.  That doesn’t mean that you “hide” anything; it does mean that you deliver bad news broadly only when you’re sure of your ground, have weighed the options for how to cope with that bad news, and are ready to present those options soberly instead of in a flailing manner.

Adhering to a philosophy of transparency is a far cry from shouting every blip or misgiving about the project from the rafters. This is particularly so if you’re in a leadership position. You want your communication to be transparent, yes. But without appropriate structure and caution, transparency can be the height of irresponsibility. In those cases, in fact, you’re just building a slightly more institutionalized rumor mill.

Lagniappe:

• Karen Guglielmo, “What is transparency, and how is it relevant to IT?“
• Phil Windley,  “Tools for IT Transparency“
• Peter Kretzman, “Why the CIO should air the dirty laundry“

“We went live before the system was ready”.  It’s a common excuse/explanation that I hear from IT people when they tell war stories about system launches that failed miserably. Implicit (and sometimes explicit) is the add-on statement: “and we told them so beforehand, too.”

There are obviously many things (and many parts of the org chart) that contribute to a failed launch, but here I’d like to focus on what drives this particular kind of launch-before-readiness, where the views of the rank-and-file are unheard or ignored.

In a nutshell: it’s management pressure. Sometimes that pressure comes from middle management, sometimes from the very top, and often from both.

Why do executives apply the pressure? There are a number of reasons:

• They actually believe it’s the best way to keep the team producing and working hard. They fear that extending the deadline will cause a let-up.
• Their own bonus and/or job is on the line if there’s a delay. They, or their management, regard a launch delay as a failure, just as much as a bad launch. At the very least, a delay means losing face.
• Testosterone (for either gender) sets in: a “refuse to lose” mentality. To some extent, that’s what has gotten them where they are.

Deadlines matter, of course, a lot, and people who are lackadaisical about deadlines simply shouldn’t work in IT. That said, if you bull-headedly insist on adhering to deadlines, to the exclusion of facts-based input, it makes you all the more likely to delude yourself that you’re really ready, just because the launch date has arrived and the pressures are heavy. If the stakes aren’t high, that may even be fine. (But, are the stakes ever not high in these situations?)

Let me tell three anecdotes that illustrate management “refuse to lose”:

• Earlier in my career, I was a middle manager, wrestling with a project that was both controlled by and staffed with a single “Big 5” consulting vendor’s people, with millions of dollars per month of consulting costs being racked up.  The high-profile initial system launch was a week away, and according to the bug tracker, there were still over 100 open major bugs. I sat down with the vendor’s partner-level project head. I don’t remember the precise conversation, of course, but it went something like this from him: “Oh, we’ve already got 39 of those 100 fixed in dev, 44 others are being worked by our best people and will be resolved by tonight, so really, there are only about 15. Four of those we think have workarounds so there are something like 10. I don’t think we’ll be getting many more bugs. We usually resolve bugs in a day or two, so even if each one takes two days, we’ll be able to close those ten and be ready for launch in a week.” I sat in awe (and frustration) at this hand-waving, this magic with math, and at his reluctance to think things through without an agenda. Here, “refuse to lose” had developed blinders, simple math had turned fuzzy and infused with vested interest, and IT best practices (not to mention common sense) had flown out the window.

• I worked as VP of Operations under a dev-oriented CTO. We had a late night launch, and the CTO was there in person, hovering over my sysadmins who were attempting to install the latest release into production with little success. The installation had failed, starting with the very first run of the script. “No, try this: run just those three lines of the installation script we provided you, then type these two commands, then run these other five lines.”  When that didn’t work, he came up with another idea, and then another. The next day, he sent out a company email declaring victory and congratulating his dev team, because the system had come up finally, even though it had run for a total of only two hours before crashing hard. Installation by seat of the pants. Refuse to lose.

• I arrived on the scene as the new CTO of a company in the midst of a complete replatforming project, completely retooling and replacing their sole product. The project had been underway for about nine months already; just before the point of my arrival, the team had been told by senior management that their launch date was going to be in six weeks, no ifs, ands, or buts.  I proceeded to do what I call a quick “temperature check”: I asked to see key indicators of readiness, such as bug counts over time and test case execution reports.

As you’ve surely guessed, I quickly determined that the facts didn’t even remotely support the possibility of a launch in a mere six weeks. About 40% of the code was still to be written; less than half of the test cases had been executed at all, with an even lower percentage having passed.  I marshaled my facts and figures, and went in to tell the CEO of this sorry state.  Judging from my experience, I told him, it looked like it would be a number of months, not weeks, before the product could be successfully launched. I remember him sputtering “that’s unacceptable”, and pressuring me on what I could do to, yes, bring off a launch in six weeks. (Cutting to the end of the story: we launched, successfully, nine months later).  In this case, a mixture of testosterone and probably some job-on-the-line aspects were driving this CEO to ignore facts and look for ways to bull through to success despite obvious counter-indicators.

The tragedy of the space shuttle Challenger has become almost cliché along these lines: a combination of executives feeling pressure and ignoring communication from lower ranks about major problems. “They also disregarded warnings from engineers about the dangers of launching posed by the cold temperatures of that morning and had failed to adequately report these technical concerns to their superiors.”  For whatever reasons, NASA management gave the green light for launch despite the adverse weather conditions. In essence, they “refused to lose”, and by doing that, they lost in a major way.

Takeaways:

• Don’t let your executives turn you, and don’t let them become, riverboat gamblers, where failure is a strong possibility through a premature launch. The company culture must be shaped so that it views a spectacularly failed launch as much more onerous than a delayed launch.
• Don’t let “groupthink” settle in, where your people are afraid to speak up. Any and every member of the team should have a chance to “pull the emergency brake”.  Especially, holding a formal “go/no-go” meeting is essential, where any stakeholder or project participant can voice concerns and point out facts that speak against readiness.
• That said, do be careful of the pessimist amid your ranks, who typically voices dire predictions on every project, and who sooner or later will be proved right. Insist on facts, not “Eeyore-ism”.
• Let your team watchwords for these situations become the following: candor, and facts. Bring both to the meetings, or go home.
• The best way to avoid letting pressure drive a premature launch: establish your launch criteria clearly and openly, with quantitative measurements, way in advance. If you don’t have legitimate facts-based ways to tell if you’re ready (i.e., not just the launch date arriving, and not your testosterone level that morning), well then, you’re not ready.

Without a doubt, “refuse to lose” can be a great attitude, up to a point. No executive, no team, should ever become blasé about missing a target. But “refuse to lose” can’t ever be allowed to become “refuse to face facts”. When it does, you don’t just lose face: you just plain lose.

I’ve done several posts featuring what I call “Peterisms”, which are basically aphorisms I’ve adopted that encapsulate hard-earned IT lessons. Let’s turn it around this time, and talk about two sayings that sound equally folksy-sensible, and that I hear again and again, but which I feel are actually dangerous to apply to information technology work. And, of course, I’ll discuss why that’s so.

• If it ain’t broke, don’t fix it

• A camel is a horse designed by a committee

So, not all aphorisms are created equal. If you’re in IT, I advise that you avoid the attitudes and philosophies contained in these two.

If you’ve been in the information technology industry for a number of years, certain jokes tend to pop up again and again. Why? I’d say it’s because their underlying premises, the things that make them applicable and funny, continue to occur. So even if you’ve heard these before (and that’s probably the case), it’s worth taking a few moments here to look at them again and consider what makes them timeless.  Remember, even jokes have morals to the story. Sometimes especially jokes.

1) Just what they asked for but not what they want
Something that’s been drifting around the Internet for decades, it seems, are various IT versions of the famous poem, “Twas the Night Before Christmas,” usually called “Twas the Night Before Implementation.” I won’t include the whole text here; most notably, the poem ends with the “super programmer,” akin to Santa Claus, swinging into action and saving the day.  And yet, the final two lines read,

Who in IT can’t relate to this eleventh-hour disconnect between what the user “wants” and what IT has arduously created and delivered? Why does this phenomenon keep happening? A lengthy serious answer here would derail the intended levity of this post, but consider the following:  the long-standing industry practice of pure, unadulterated “waterfall“-style requirements gathering before design, build, test and deliver tends to lead to such disconnects. That fact certainly presents an argument for greater agility in the process, greater involvement of stakeholders throughout, and resistance of “big bang” approaches to systems delivery.

2) I had a truck like that once:

This joke almost always comes to mind whenever I hear IT people blissfully bragging about the complexity or size of their infrastructure, latest system, etc.  For example, most recently, my friend Gunner was telling me about his current development project, just entering User Acceptance Test: “Our programmers have gotten a total of 8 hours sleep in the last three days!”  “Yes, Gunner,” I replied. “I had a truck like that, once.”

Size, complexity, and lack of sleep are sometimes unfortunate realities, but they are rarely great things to brag about.  Don’t let your enthusiasm for what you’ve accomplished make you forget that simpler is nearly always better than more complex, smaller almost always better than large, and sleeplessness leads to bad decision-making in the trenches.

3) What you just told me is technically correct but of no use to me whatsoever

Rackspace, a prominent managed hosting provider, recently incurred a massive outage. Even though this is a company that prides itself on “fanatical service,” (and to be sure, is praised by many of its customers for providing such), on the day of the outage, it sent out a “customer incident report“, featuring sentences like, “The generators experienced a subsequent excitation failure forcing transition of cluster A back to the primary utility feed prior to the completion of UPS repairs.” Rackspace’s incident report is an amazingly bad example of jargon-laden, un-customer-oriented IT communication. There’s no discussion of real root cause (the breaker tripped, but why?) or specified action plan, just a bunch of seemingly nervous technobabble that lets no one understand what really happened, why it happened, and what they’re doing to ensure that it won’t happen again.

In other words, everything in their report is technically correct, but it’s no use to anyone. Sure, they were still in the throes of diagnosis, but that doesn’t excuse the plethora of excess detail and their failure to address what customers really are interested in. This kind of approach inspires negative confidence among one’s customers, needless to say.

4) Difference between a methodologist and a terrorist
This joke has also been around for decades. I stopped telling it for a few years after 9/11, but it continues to make a needed point:

I’ve frequently noted that some IT folks latch onto a particular methodology (Rational, Agile, Lean, whatever) and come to see it (and it alone) as representing pure, unvarnished guidance and truth.  If not brought back to earth, they can drift into a discipleship attitude that there’s really only “one true way” of approaching systems and projects.  If confronted with a failure in a project using their “one true way,” they’re quick to claim that the methodology wasn’t fully or correctly implemented (“If it doesn’t work, you aren’t doing it right.”).  This “one trick pony” perspective, a sort of tunnel vision that lacks introspection, creates a rigidity that’s almost always detrimental.

In fact, it tends to put you into situations where joke #1 is relevant.

Here’s one series of questions that’s great to ask a candidate for a senior position in IT, be it in project management, development, operations, or whatever: Tell me about a recent project or initiative for which you were responsible. What was the goal, and how did you ensure that the goal was achieved? What were the roadblocks you encountered along the way, and what did you do to address them? What mistakes did you make? What would you do differently if you were faced with a similar project or initiative again?

For this post, I’ll play candidate and outline the kind of full and detailed response I would be looking for as a hiring manager, based on a project from my own experience. Let’s use a relatively simple one: implementing off-site (Software-as-a-Service, known as SaaS) email, replacing a poorly implemented in-house email system. Today, this would loftily be called “going to the cloud.”  (Yes, I’m aware that cloud computing encompasses far more than SaaS, but that’s not my thrust here).

As the incoming CTO at an Internet social networking company several years ago, I quickly observed that an unacceptable percentage of my IT staff’s time was absorbed in administering, troubleshooting, and user handholding related to the in-house email system, a third-party substitute for Microsoft Exchange.  But far more than the high cost factor was at stake: the flakiness of the company’s email and calendaring system had become both a cultural punchline and, frankly, an easy company-wide excuse (a little like “the dog ate my homework”) for people missing meetings and failing to communicate on key initiatives. Undeniably, meetings were indeed disappearing regularly from people’s calendars, and even simple meeting scheduling required using an unintuitive workaround to guarantee that all recipients would get the meeting notice. So, our goal was two-fold: reduce cost and staff distraction significantly, AND (most importantly) give the company a solid, reliable system for internal and external communication.

The mail server package had been implemented several years before my arrival as CTO, apparently without a careful plan or study, and mostly as a gut reaction by a then-VP of IT who simply loathed Microsoft and wanted a non-Microsoft solution. A significant portion of a DBA’s time, not to mention that of the entire help desk staff, was being expended in support of the system, not to mention the high hardware and software maintenance costs. My staff, and just about all users, detested the system. But oddly, roadblocks were present nonetheless:

• staff resistance on changing out a system they regarded as part of their job;
• CFO reluctance to make changes in general, and especially reluctance to go to an external email provider (concern about loss of control).

One lesson here? Things are never so bad that people drop their innate resistance to change!  But I digress.

We countered these roadblocks via several approaches:

• We surveyed all users in the company for their views of the system and tolerance for change in this area, and fed this feedback into the resulting action plan.
• We constructed a thorough cost analysis of the status quo: how much the current solution was costing the company in both hard and soft costs. I used this analysis to obtain executive buy-in, and to specify in detail what the cost and effort would be to fix it.
• We made sure that both junior and senior staff were involved in the methodical evaluation of alternative approaches, including the alternative of keeping the status quo. This helped achieve staff buy-in.
• We brought our “short list” vendors in for week-long user proof-of-concept trial periods, getting both staff and user feedback, thus defusing a lot of the “loss of control” type of uncertainty regarding the change.
• We negotiated hard with our “short list” vendors to obtain a good deal for the company.
• We worked hand-in-hand with our excellent legal staff to make sure we were well protected in terms of vendor SLAs, security, etc.
• We constructed and followed a careful user communication plan, keeping everyone informed of the rollout schedule via FAQ-style emails.

The rollout of the chosen solution went fairly smoothly.  We failed to identify two key risk areas or areas of concern clearly enough, though:

1. Users were much more concerned about calendar migration (old appointments and their attachments) than we had anticipated, and somehow this didn’t come out in surveys etc. before the implementation; we responded not by migrating the old items (expensive and difficult), but by keeping the old system running for a period of time after system rollout of the new solution, and that seemed to be satisfactory.
2. Users had great difficulty adapting to a 250MB email space limitation, since they had effectively had “no limit” (albeit poor performance and terrible stability) under the old system.  Even though we had discussed that limit in advance, done benchmarking across similar companies about their space limits, and gotten clear user buy-in, it still might have been better to consider a higher limit (and hence more cost) as we sought expenditure approval.

Now, just three years or so later, I’d do several things differently:

1. obviously, anticipate and avoid the mistakes itemized above;
2. consider using Google’s business email offering (we ruled it out since it was brand-new as of the time of the implementation, but it offers a much greater storage limit than any other provider I’m aware of); and finally,
3. implement sooner rather than later. Why sooner? The implementation and system was such a success that despite the bumps and concerns I just named, there was no more constant hallway conversation about the email system. It had ceased to become a concern, and also ceased to become an excuse.

And bottom line for the company, besides those productivity benefits? The total cost of running our (now very stable) mail system had dropped to the equivalent of just one FTE, about a third of what it had been.  Happier staff, happier users, and much lower costs: all projects should be so fortunate.

Lagniappe:

• Maria Spínola, “An Essential Guide to Possibilities and Risks of Cloud Computing“
• Michael Coté, “Don’t confuse SaaS with Cloud Computing“

I noted last time, once again, that “IT is hard. In fact, it’s so hard that it seems most people have to learn certain core lessons by themselves.  It seems like everyone needs to burn his or her own hand on the hot stove.”  I went through some examples of this sort of “hot stove” lessons particular to management; this time, let’s talk about similar “hot stove” lessons/myths I’ve observed in other IT areas, most notably development and operations.

• Source code control and release management. One of the traits of a superlative programmer is his or her ability to maintain a complete logical model of the system/program in their head.  The really good ones are in fact really good at this.  Unfortunately, their consummate skill ironically leads to them sometimes resisting tools that help out with some of the logistical pitfalls that arise as system complexity increases.  Source code control is the most important such tool.  Programmers have even told me, “I can just keep track of what I’ve changed and what’s where.”  In a way, I suppose this is a species of the (typically male) trait of refusing to ask for directions—it’s a reluctance to embrace appropriate tools that are designed to avoid common screw-ups and to facilitate overall team success.  Suddenly, though, complexity mushrooms: releases overlap and patches are made in the heat of the moment, and without impeccable source code control and release management, bugs reappear, QA takes longer, and so on.  Typically, I’ve seen this “hot stove” lesson not really get learned by an organization until the source code control failure causes a notable customer-facing issue with a major release.

• Avoiding reporting against production.  Here, the going-in position of many developers is that “it’s fine to run reports against our production data; we’ll change it if it becomes a problem.”  This is another example of development activities incurring implicit technical debt.  What often happens, of course, is that certain ad hoc reports suddenly become regular and popular throughout the business, with business decisions resting on what they reveal.  New variations get created over time, and overall demand (and frequency of access) soars.  Then, unanticipated and unpleasant ripples occur, often without an obvious link to their cause: system performance problems, unpredictable timeouts, disk shortages, and so on.  When it reaches that point, it’s actually quite difficult to step back and re-architect seamlessly, without a hiccup.  In other words, by the time everyone becomes glaringly aware of the problem, you’ve painted yourself into a corner in terms of the ability to fix it quickly.
  

• Separation of development and operations. As companies grow from small numbers of employees to larger enterprises, it’s a common rite of passage for the organization to wrestle with the role of development in ongoing operations [See my post on Speed vs. bureaucracy: management issues confronted by companies in transition.]  Originally in such companies, of course, “everybody did everything,” and people are often proud of and wistful for those halcyon days.  Reducing people’s long-held access to systems is often seen as bureaucratic and authoritarian. But yet it’s necessary. The more you institute a healthy separation of duties and systems, the more likely you ensure clean handoffs from development into production, and the more likely you will avoid the sorts of production problems that ensue when developers make and apply “just one little quick fix” in the heat of the moment.  In truth, minimal developer intervention should ever be needed for production purposes. Developers should have no operational responsibilities and no regular access to production data.  I witnessed one mid-level startup once, where in a crisis moment, the CEO wanted to have a developer code a patch to our enterprise software product in the middle of the night and push it out that night to all other customers as well.  That kind of freewheeling approach and company simply can’t (and didn’t) succeed over time.  In that case, the “hot stove” resulted in the whole company and its investors getting burned.

• Full disclosure of problems and issues.  Maybe it’s human nature, but I’ve seen a common trend among IT staffers who haven’t learned this particular “hot stove” lesson.  It’s a general feeling that they should keep potentially damaging information quiet, and that “what people don’t know won’t hurt us”.  That philosophy shows up when they resist informing the company widely of system outages (often applying an IT kind of twist to the Five Second Rule).  Or when they are aware of but resist wider debate over critical bugs that they believe won’t actually be noticed by the customer.  Or when system performance problems in testing aren’t escalated, again in the belief that it’ll all turn out to be OK in production environments.  In my IT executive experience, there is no lesson more critical than the need for IT to create and actively encourage an atmosphere of full trust and open disclosure about system issues across the enterprise. All it takes to undermine that trust, and undermine it for a very long time, is one unfortunate instance of IT sweeping something under the rug and getting “caught” at it.  Then, a cycle often ensues: the reaction to the error itself is so strong and painful that IT folks are even more reluctant, the next time around, to disclose anything similar.  IT management can best help here by insisting on airing the dirty laundry, as I’ve discussed before.
  

Note here that the underlying theme of these “hot stove” lessons is that people resist learning them. Advice and counsel alone do not seem to hammer them home. IT management, if appropriately aware of these pitfalls, can (and should) insist on instituting mechanisms that will prevent the worst of their impacts, but be aware that doing so generally will be against some resistance, both up and down the management chain.  In essence, it may just be that people sometimes need to, well, get burned.

Regular readers here have certainly noticed the recurring nature of many of my posts: “things about IT that should be obvious, but clearly aren’t.”  Each week, as I set about writing on my chosen topic, it often strikes me that what I have to say is anything but new or radical; rather, it seems to embody fundamental, well-known practices and underpinnings that should be, if not incredibly obvious, at least quite familiar to anyone who’s spent much time in or around IT.  Yet, as I reflect on my actual experiences, I realize anew that I’ve spent much of my career working up and down the executive and worker chain to absorb and impart these basic principles again and again.

So here comes more of the same.  In fact, this dual post encapsulates a lot of what I’ve written about on this blog for the last year: lessons learned and lessons still learnable about IT for the people who work in it and the people who have to deal with it.  In other words, I should hasten to say, these are not only IT management lessons, but lessons related to development, operations, QA, and project management: in other words, the whole spectrum. 

As I’ve observed before, IT is hard. In fact, it’s so hard that it seems most people have to learn certain core lessons by themselves.  It seems like everyone needs to burn his or her own hand on the hot stove.  So here are some of the glowing redhot stove elements that I’ve watched make fingers (including mine, for I’m not immune either from learning some things the hard way) sizzle over the years.  I’ll start here in part I with lessons particular to management, and next time will cover similar lessons/myths relating to other IT areas.

Management “hot stove” lessons/myths:

• If the project is late, can’t we put more people on it. I’m compelled to put this old chestnut first, since it’s (ironically) both the most discussed AND the most frequently still observed meme out there of this nature.  Fred Brooks wrote a whole book on it, The Mythical Man-Month, which should be required (and recurring) reading for every executive who deals with technology.  The same mindset that gravitates towards putting more people on a late project often proves to be equally seducible by the luster of offshoring, or by the notion that simply doubling the number of people will be able to turn out a product in half the time originally estimated.  Everyone in the trenches of IT, at nearly any level, intuitively recognizes the folly and futility of what essentially amounts here to equating software development with ditch digging. The very suggestion (i.e., that elapsed time will simply equal the total level of effort divided by any arbitrary number of resources, where having more resources automatically results in faster product delivery) is, these days, at about the same level of thinking as an executive wondering about the efficiency of double-entry bookkeeping (what a waste of time to enter everything twice, eh?).
• If we outsource a lot of things, then we can do a lot more as an organization. However, people embracing this myth seldom show a good understanding of what it takes (resource-wise, as well as the hit to organizational focus) to select and manage all this outsourced work, much less integrate its results.  Many IT organizations don’t even have the necessary technical foundation/infrastructure in place to allow outsourcing to happen effectively at all.  Outsourcing is indeed a worthy goal and a fine choice, when chosen and planned for prudently, but it doesn’t come for free, and tradeoffs must be discussed and understood up front when these choices are made.
• If there are bugs in the software, it’s because QA didn’t do their job well.  Bugs, however, are a fact of life in software development.  Mitigating them (in quantity and impact and risk) is the issue, not eliminating them.  Almost always, it tends to be schedule pressure and/or resource constraints (both caused, typically, by management impatience and/or inadequate understanding of cause and effect) that have facilitated the inclusion of an unacceptable level of bugs in the released product.
• IT should work on everything that the users say they need; the truly necessary stuff will sift to the top. There’s no real need for messy and time-consuming prioritization efforts. This may be the most dangerous myth and need-to-learn-lesson of all these management-related items.  Prioritization needs to happen on purpose, not by accident or by mere dint of IT championship.  It’s the most important thing that an organization does, at least the organizations that manage to avoid hopeless fragmentation of resources and effort.  Purposeful portfolio management, with heavy stakeholder participation and sponsorship, is the only really effective way of dealing with the firehose that is constantly trained at IT’s eyeball.  Tying this to the items at the top: consider that if you want to do more, one very viable approach is to take on less, and to make sure that what you actually take on are truly the top priorities (the “vital few”) for the company.

One tiny anecdote to frame this discussion: I saw one company where literally dozens of IT contractors were brought in at short notice and with little planning, due to the unfortunately entrenched belief that adding more resources would of course result in faster delivery of badly needed product.  After only a month or two of having these workers on board, the short-term operational results of the company caused budgets to suddenly tighten, and all the contractors were let go.  Due to ramp-up and inadequate knowledge transfer, almost no value was gained to the company from what equated to several person-years of cost/effort.  See the first two lessons/myths above.  People’s fingers came away badly burned on that one, you can be sure.

So again, I’m not just bashing management here as I discuss these “hot stove” lessons.  Stay tuned for next time, when I talk about the myths and lessons-always-needing-relearning that I’ve seen within the IT ranks themselves.

A few months back, I wrote a post on various “Astounding Sayings” that I’ve encountered in my career in information technology.  It turns out that it’s been one of the more popular posts I’ve written, judging from page views, so in true Hollywood fashion, it must be time for a sequel.  I am retitling it slightly, though, to distinguish it from the Peterisms I post from time to time.  The point of writing about the “astounding” sayings was that they usually reflect misguided energy (or, to put it bluntly: wrong-headed thinking); the point of the Peterisms, on the other hand, is to distill and communicate absolute, undeniable, sublime truth and wisdom at every possible turn. (Hopefully it’s unnecessary, but just in case, <insert smiley face here>.)  Hence, I’m now going to call these non-truthful, unwise sayings “astounding utterances” instead.

Here are two more such utterances, with moral-of-the-story observations for each.  Note: as before, these are true stories.  I may have changed some of the facts, lightly, to make them less identifiable.  They also always come from at least several years in the past, to provide a healthy amount of distance for everyone.

The utterances that I consider to be astounding, in their context, are highlighted for you below in bold.

• Looking out for #1, and, um, that would be ME.
  

Regular and formal goal-setting, to my mind, is one of the key managerial ways to focus and align everyone’s efforts in the business world.  Done well, it ensures that people are working on the right objectives, and are incented to achieve things for the common good of the business. The exercise often also brings to light any number of disconnects (teams working at cross-purposes, etc.) that would otherwise go undetected.

Usually, the way I’ve found this kind of goal-setting works most effectively is to actually collaborate with the employee on establishing meaningful goals for their role.  This isn’t exactly a jaw-dropping insight.  Workers themselves usually are quite expert in what needs to be done in their area; managers can help prioritize, push for improvements, and “ripple down” the larger goals of the company at that particular time.  So, the manager will typically draw up a list of suggested goals, ask the employee to do the same, and then they work together on aligning the two so that it all makes sense.  Goals should of course be “SMART”—Specific, Measurable, Attainable, Realistic, and Timely—but I won’t go into the specifics of that here; check out the links at the bottom for more information.

All of that’s pretty obvious, right?  Well, during one such goal-setting exercise a few years back, one of my managers came to me and reported that one of her employees had gotten upset at the draft suggestions the manager had drawn up for possible goals.  The employee didn’t like seeing specific targets for her work, such as measurable throughput, successful completion of projects, improvement of operating results.  Her response to seeing her manager’s suggestions was to snort, “Those aren’t my goals—those are the company’s goals!”

We can of course laugh at how obviously misguided this person was (just why she imagined that the company was paying her, if not to push for improvements and to help achieve certain company goals, I have no idea).  But I think the incident reveals an all-too-frequent attitude in the workforce, sometimes at all levels.  People who aren’t focused and actively driven to working on achieving meaningful overall company goals will simply hold everyone back. They’re in it for themselves, first and foremost.  If you find someone who doesn’t innately understand that having the company succeed will almost certainly tend to augment their own situation (salary, position, etc.), you need to work with that person quite seriously to see if you can bring them around.

• We wanna do what WE wanna do. Why are you asking these annoying questions?
  

You’ll note a theme, as usual, in this pairing of utterances.  For my second story, I’ll describe just a bit (tip-of-the-iceberg style) about a situation I walked into several years ago, where a development team had gone whole hog, tooth and nail, hook-line-sinker-and-the-whole-nine-yards, into Extreme Programming as their operative model.  Now, my purpose here is not to bash Agile or Extreme Programming, because there are actually quite a few positive aspects of those approaches, and discussing them fairly deserves at least a whole separate post. 

But when I started my CTO role at the company, I needed, as always in a new position, to ramp up quickly on current projects, deliverables, time frames.  And I could find nothing to go on.  Far from being Specific, Measurable, Attainable, etc. in their goals, this team had come to a style of work (and had been allowed to do so) where next to nothing was ever written down, and few goals were concrete other than (it seemed) just getting through the day.  In fact, I’d been hired precisely because top management was growing ever more concerned at how little was actually getting delivered.  I sat the project manager down and asked him how they hoped to know when they were really done, or whether what they were delivering would meet the business needs, or how they hoped to test what they had produced if they didn’t have definitions of what the specific goals were.  Where, in short, were the guiding definitions and documents that would serve as their touchstones during such projects?  He looked at me, arched his eyebrows, and proudly stated, “We’re parsimonious on documentation around here.”

Whatever one thinks about Agile and Extreme Programming (and let me reiterate that that is a complex topic that deserves a separate post), this was clearly a situation that had gone to a bad extreme, one where accountability and purpose had gradually drifted out of the picture.  There’s room for a lot of different and viable approaches in software, but almost completely eliminating accountability and direction isn’t an approach that I’ve seen bear a lot of fruit.

And there’s nothing astounding about that.

Lagniappe:

• http://en.wikipedia.org/wiki/SMART_(project_management)
• http://www.goal-setting-guide.com/smart-goals.html
• http://www.topachievement.com/smart.html