Logical and Stupid

I am a big fan of logic of many types. I understand predicate logic, boolean logic and the rigorous application of value functions to alternatives, giving logic a solid place in decision-making.

As big a fan of logic as I am, I always apply the final test: "the proof of the pudding is in the eating." In other words, it matters where you end up, even if your chain of reasoning is flawless. If you end up with a stupid conclusion, it matters little how you reached that stupid conclusion.

This is on my mind because I keep running into a stupid conclusion that was arrived at logically. Mercifully, this particular stupidity rarely ruins my day, but it certainly ruins the days of many others.

The day ruining takes the form of endlessly refusing to address any immediate issues because only big issues can be taken up the chain and local power seems to be non-existent. When I tried to push back, in different contexts in different organizations, here is the reasoning that was offered as an explanation:

1. Premise: the view from 50,000 feet/big picture is always clearer
2. Therefore all decisions should be made by senior staff
3. But senior staff cannot get bogged down in details
4. So only big decisions are worth making
5. Therefore a collection of smaller decisions, even if that collection has an enormous collective impact, should be ignored forever

I can see how the recent recession and shortness of money has caused organizations to tighten purse strings. I realize that pushing decisions up a level is chic. I know all too well that many low-level managers do not have great decision-making skills. But to end up in a place where nothing is possible and ever-more out-of-touch senior staff make an ever-higher percentage of decisions does not appear to be going so well.

On the bright side, the resulting not-to-be-admitted disasters seem to offer decent opportunities for a consultant such as myself, so perhaps I should just keep my big mouth shut.

References for the interested:

• http://www.phrases.org.uk/meanings/proof-of-the-pudding.html
• http://en.wikipedia.org/wiki/Hypothetical_syllogism
• http://en.wikipedia.org/wiki/Predicate_logic
• http://en.wikipedia.org/wiki/Boolean_logic

Hope Spring Eternal: the Upgrade Treadmill

For my text today, I take this famous snippet from Alexander Pope:

Hope springs eternal in the human breast;
Man never Is, but always To be blest.
The soul, uneasy, and confin'd from home,
Rests and expatiates in a life to come.”

I found this quotation here:
http://www.goodreads.com/quotes/10692-hope-springs-eternal-in-the-human-breast-man-never-is

I found this excellent explanation of the quotation here:


It means that no matter the circumstances, man will always hope for the best - thinks that better things will come down the road. We may not always act our best, but we have the potential to be better in the future. No matter how bad things have been, they can always get better.

http://answers.yahoo.com/question/index?qid=20060809101312AANcoOn 

While I believe in the sentiment, that things can always get better, I do not believe that all things always get better, or even that most things get somewhat better.

Specifically, I am appalled by what I call "the upgrade treadmill." By this, I mean the tendency to hope that a major upgrade to a large software package will fix the many inadequacies. To me, this hope can be restated like this:

Perhaps this time the same people and corporate culture which produced bad software last time will produce better software this time, because trying to fix issues in a large code base is easier or more likely than getting it right the first time.

We see our clients jogging along on the upgrade treadmill, mortgaging the present for the hope of a better future, frantically moving forward on the treadmill but not in any other sense.

In some ways this is inspiring and touching: the triumph of hope over experience.

In other ways, it is frustrating and tiresome, because the treadmill tends to slip into a one-size-fits-all excuse: sure, I have a problem in the present, but I do not need to address this problem because this problem might be fixed in a future update. Who knows? It might. It might not. The track record of updates usually does not inspire confidence.

So keep hope alive, because what is life without hope? But don't let the upgrade treadmill distract you from realistic assessment of your situation and periodic review: sometimes the only way to make progress is to stop the mindless forward motion and step off the treadmill.

Managing Up

Today's topic is ticklish: managing up.

As I understand it, this term means trying to influence one's superior's expectations and behavior.

I suppose this term should mean "treating one's superiors as though they were subordinates" but that is overly literal. However, there is a high-level truth to the exaggeration: one is taking responsibility for the attitudes and actions of one's boss(es).

When I first encountered this concept, many moon ago, I found it silly. I was quite comfortable with the idea that those below me should do as I bid and those above me should bid me do whatever it was that I was supposed to do.

Over time, I understood that there was a legitimate "managing up" need: my superiors required context and feedback if they were going to make good decisions. So I grudgingly took on the task of keeping them informed, over and above what was strictly necessary.

Alas, what I see very frequently now in IT is feast or famine: either mid-level IT professionals do nothing but "manage up," leaving their direct reports wandering in the dark, or they do no managing up, leaving their superiors with ever-more-inaccurate world views.

Worse, many newish mid-level IT folks confuse "sucking up" with "managing up." They go out of their way to confirm their bosses prejudices and to avoid correcting mistaken impressions. This turns even well-meaning, talented senior managers into blunder idiots. If you don't know what works and what does not; and if you think the issues are different than they are, you will have great difficulty choosing an effective solution to the most pressing problem.

A sure sign that managing up is not working or not being done is this: there are resources and good staff working hard to accomplish their goals, but things just don't get any better over time. In fact, they get worse, although reorganizations and new software rollouts neither improve or degrade matters.

If you are working hard with decent people but the steady and amazing march of technology is not making your job easier or better, then you may have a managing up problem. If your superiors are reasonable people, start treating them as such. If your superiors punish bearers of bad news, then managing up is out and sucking up is in. May commerce have mercy on your career.

I Don't Get No Respect

WARNING: THE POST YOU ARE ABOUT TO READ IS QUITE BITTER

I have been struck recently by the general decline in status of the techno-guru and particular decline of my status as as techno-guru.

Once upon a time, email hit the general public; for me, that was in the early-to-mid 1990s. I had starting using email about a decade before, so email was a bit old hat to me by the time it reached my mother and other trailing edge technology users. I found myself frequently asked about this new email thing by people who knew me as a technology expert. While answering naive questions is not my favourite pastime, I sighed deeply  and answered with all the patience I could muster.

(Had I known that the tsunami of users would destroy the quiet, thoughtful thing that I knew as email, I would not have been so helpful or reassuring. In those ancient times, I only received few emails and each one was welcome. I corresponded with just about anyone I cared to in the ANSI C/Unix world. I remember clearly when I started getting automated "I can't read all my email, I am only accepting email from colleagues" from the big names in the field. I shortly had to do something similar as even minor players such as myself were overwhelmed by the flood of spam.)

More recently, Facebook hit the scene. This time, the situation was exactly reversed. I found that absolutely ignorant people were comfortable lecturing me about Facebook's pitfalls (always some vague concern about privacy or wasted time) or exhorting me to enjoy Facebook's benefits (usually something about the trivia of the lives of mutual acquaintances).

Whether one is pro-Facebook or anti-Facebook is beside my point, which is this: when did technical expertise become utterly irrelevant to technical discussions? Whenever I pushed back on the lectures, I found that the lecturers were happy to dismiss my requests for more detail, for evidence, for explanation. They waved their hands elegantly: they were not computer people, after all, and we were talking about the technical attributes of a piece of software, a distributed information system.

(For a humorous take on the Facebook privacy issue, watch the satirical Onion look at Facebook: http://www.youtube.com/watch?v=cqggW08BWO0&feature=youtube_gdata_player)

Would I have been interested in their personal experience as a user or non-user of social media? Yes, I would have been. That would have been valid and interesting. But no one wanted to talk to me about their personal experience; instead, they all wanted to talk to me about either the pitfalls of the software's design and policies, or the benefits of using this technology without regard to the risks.

Sadly for me, this phenomenon is not confined to parties or family gatherings; I encounter it in the professional arena as well. I find myself being lectured on software choice and deployment by MBAs who are proud of their technical ignorance and their non-technical backgrounds. Somehow, they have convinced themselves that those of us who get our hands dirty in IT are less capable decision makers. We lack the business perspective. We are like children in a toy store, grabbing the shiny technology without regard to how much it costs or how long we will want to play with it.
I suppose that there are IT experts who can't grasp the concept of a budget. There may be IT experts who want to play with shiny new technologies just because the new technologies are new, without regard to the problems to be solved. But I am not such an IT expert and I don't think that I have ever met one.

But until I figure out how to overcome the disadvantage of years of experience and a track record of successful deployment, it won't matter: I won't be part of the conversation. Maybe I need to get an on-line MBA and lie about the last thirty years of my career.

(Why lecture an expert with only ignorance as raw material? When did this become a good idea? Do people now lecture their doctors, their dentists, their accountants, their garage mechanics? From my very limited survey, the general public does lecture experts with confidence. So perhaps this is not limited to me or to my profession. Sigh.)

Simplicity: Too Much of a Good Thing?

Like many engineering types, I strive for simplicity. I seek to avoid excess complication. I rely on analysis to tame complexity and I rely on rigour to help me manage it.

That said, I am very cranky about simplistic people and their simplistic thinking and their simplistic solutions to complex problems.

To be concise, simple solutions lack complexity while simplistic solutions suffer from being too simple, from being insufficiently complex.

In practice, I find that simplistic thinking is the result of poor analysis, of not digging deeply enough, of accepting apparently simplicity as the whole story.

I feel rather self-indulgent today, so I will put this into doggerel:

Simple is a virtue,
Simplistic is a vice.
Simple might not hurt you,
Simplistic is never nice.

So what is an example of simplistic thinking? Consider patient identification; one of our systems is used to support dealing with the public, which requires that it support patient identification. At first blush, this is simple:

• require that the patient give you their unique patient ID, probably on some kind of card you gave them;

or

• require a photo ID with name, date of birth and sex

Since the client organization already has a patient registration system, they objected to our claim that we needed to build a patient identification support module. We had to do it against their objections and then only bill for it when they saw its value.

What complicates this seemingly simple task? The actual answer is horrifically long and complicated, so I will an abbreviated reply which will still make the point:

• Women often change their names when they get married; their pre-existing paper work, orders, etc, do not magically change with them, so it is helpful to keep previous versions of names, marked as not canonical, to help ensure that the driver's license / printed report / old information mismatches are not evidence of error.
  
• Children do not have photo IDs and are often known by unofficial names. Is young Billy Doe actually Ralph W. Doe? R. William Doe? Bill Doe Jr? Is this the former Babyboy Doe? Perhaps he is the former Boytwin1 Doe--or is he the former Boytwin2 Doe? By keeping a history of these transactions we can support confirming that you have the correct child, even if the caregiver can't remember which kid had the allergy test in July, as opposed to the kid who had the allergy test in June.
  
• Alas, people make mistakes and sometimes these mistakes line up with real data. There can be two John Does, one born 4/1/1941 and one born 1/4/1961 and if you mistype and have to search, you might pick the wrong one, in which case a custom module which says "this patient does not have a relevant order, but the other one does: are you sure?" will really save on dangerous and expensive patient identification mistakes, no matter how plausible those mistakes might be.

As a vendor of custom solutions for medical information process automation (yikes! what a mouthful), I am rarely called in because a situation is simple; if the problem had a simple, obvious, excellent solution, someone would have implemented that solution. I am only called in when things are complicated, hairy, ugly or all of the above. While I do what I can to make my designs as simple as I can, some problems can only be solved with involved and complex solutions.
 
Recently this pet peeve of mine has been elevated to a business issue as
our firm more frequently encounters simplistic thinking in our customer's senior management. These managers, from their lofty perches, are issuing clean, simplistic edicts which do not match the reality in which we find ourselves.

Worse, we are losing sales to less ethical competitors who promise clean, simple solutions to murky, complex problems. These solutions don't work, of course, but by the time this is well-established, the situation is the new normal and everybody loses.

So strive for all the simplicity you can muster, but shun the simplistic with all your might.

The Decline and Fall of Some IT Groups

We are information systems consultants. As part of our typical engagement we work with our clients' internal IT groups. For this reason, our clients' end users consider us to be experts on their organization's IT groups. Recently a depressingly large number of our clients' end users have come to us to ask why their internal IT groups are doing such a bad job. They ask us this question because it is not an easy question to ask of a colleague; it is easier to ask an outsider.

This barrage of questions has forced us to consider this question, although we usually try to stay out of internal IT politics as much as possible.

When we took a step back to consider the different IT groups in different organizations, a pattern began to emerge. We are seeing an alarming number of them in the following situation:

1. They are being asked to "do more with less"
2. Positions are being filled only after long delays if at all
3. Functions are being taken in-house to save money

The first point is a function of the recession: every one is trimming their budgets. Pity the poor IT group: unless it has technically savvy senior management, the right things are not being cut. We see this directly all the depressing time, budget cutting by size of line item and not by return on investment or the marginal utility of the line item.

The second point is a function of the recession as well: there are so many applications for any given job that managers are tempted to wait for the PERFECT candidate. Here is an excellent column from James Surowiecki's reliably interesting and useful column in the New Yorker.

The third point is a function of muddled managing: if you have cut your IT group's budget, and failed to hire new people, then adding work to their plate is very unlikely to produce good results. The most common outcome we see is a decline in service to the users and a decline in morale within the IT group.

So if your formerly-good IT group is in steep decline, you might want to lobby for their budget to be increased or their headcount to go up. But if that is not an option, then perhaps you can temper your irritation at the decline in service with pity for the IT group itself: it can't be any fun to be on an ever-sinking ship.

Managing Consultants

I am currently paid on a consulting basis.

I put it this way, rather than "I am a consultant" because I do not have any great attachment to the business rules under which I get paid. I want to do good work for reasonable people and in return I want to be paid a reasonable amount.

For many reasons, many of them both tedious and beyond my control, consultants are being demonized; we are the new lawyers: a professional group we all love to hate and can agree to mock. For example, consider this poster from the often-hilarious http://www.despair.com:

While I agree that lawyers are often used ineffectively, I do not agree that lawyers are useless busybodies, just as I don't think that HR exists to making hiring difficult or facilities management exists to stop you from having nice office furniture.

(I blush to admit that I was going to use "first thing we do, let's kill all the lawyers" until I looked into it and found that I had that backwards: http://www.spectacle.org/797/finkel.html. Sigh. Another blow to my feelings of omniscience.)

Leaving aside the inherent problems with demonizing any group, I have a problem with demonizing consultants. To me, the big difference between generalizing about lawyers and generalizing about consultants is that lawyers are a fairly well-defined group providing a fairly well-defined service, while consultants are a ridiculously varied group with only a compensation methodology in common.

Mercifully, to make my point today I can leave aside the question of whether or not consultants are evil, because it does not really matter: at this point, almost no medium-sized or large companies can do without them.

(In my experience, small firms benefit most from consultants, but small firms seem better adjusted about this issue: happier and more accepting.)

So pick a side: consultants are a useful tool because you can rent expertise that you need for only as long as you need it, or consultants are parasites whose only goal is to addict you to their services. Either way, there they are.

I assert that in this day and age, managing consultants is a key skill, just as managing women was forty years ago, when women started showing up outside the steno pool.

I accept that consultants are not a panacea: institutional memory is good and employee loyalty is good.

I agree that consulting engagements need to be reviewed periodically to ensure that renting expertise is still the best way to accomplish whatever goal toward which your consultants are working.

What I cannot abide is the current mindless swing toward "No Consultants!" as a policy. This is so rarely practical in today's business environment. I say this not because such policies cut into my paycheck: hilariously, they don't cut into my paycheck, at least not so far. What they do is drive up my paycheck as I stand aside and then wait for the frantic emergency calls that always come--unless I hear, instead, the sickly silence of despair as organizations caught in the death spiral of "doing more with less" until they go out of business or fail.

Love them, hate them, but make sensible use of consultants. Don't ban them and don't cling to them forever. Instead, assess your needs, consider both cost and benefit and do what makes sense. That is more work than issuing clear, simple edicts such as "no consultants!" but that is why managers get paid the big bucks, isn't it?

Broke, Broker, Brokest

In response to my last post an old friend who is an organizational psychologist pointed out something that has been on my mind for a while: the tendency of many people to live the dictum "if it ain't broke, don't fix it."

(By the way, the history of this phrase really surprised me: check it out on Wikipedia if you are so inclined.)

I can see why this dictum is useful in large organizations: endless thrashing and reorganization is often the bane of their existence and they need some way to make sure that "fixes" are actually needed. (They also need to follow up those fixes to make sure that they worked and continue to work: see an earlier post.)

But I am seeing many instances of throwing out the baby with the bathwater in this regard. I see simple-minded adherence to this principle which eliminates all of the constant small decisions and leaves only the rare big decisions.

In a nutshell, there is more to business decision evaluation than "broken" vs "not broken". Very few processes either work unacceptably badly or perfectly. Surely we can do better than just these two gradations?

The determination of whether or not an issue or set of issues is worth addressing involves some possibly subtle cost versus benefit analyses. I assert that the deciding test of "Is this process hopelessly broken?" is simply not good enough to produce an effective or efficient organization.

I hear from managers that there is an additional complication: their organizations have tightened the purse strings so much that the level of effort required to push through a small investment seems too great: better to save your strength and only push through large projects. Since the projects are large, the scrutiny is great, so the projects have to be "no-brainers" so better to wait until your department is on the verge of collapse and then propose a giant, sweeping solution.  I do not have a good answer to this logic: if the premises are true, if the organization is that badly out of balance with respect to its decision-making, then perhaps only doing the big, dire things is the way to go.


And yet.... Especially in these recessionary times, with staffing levels so far below idea, don't we need to be as efficient as possible? Mindlessly trying to avoid expenditure while assuming that fixed labor costs will produce greater output under the magic incantation "do more with less!" will produce nothing but burnout and turnover.

So the next time someone suggests incremental refinement and modest investment, try to imagine a future that gets slowly better and better with modest cost and reasonable return.

IT Ecosystems or Many Small Benefits

I find my smart phone to be a terrific tool and a significant advance over what went before. I have been contemplating this recently because I see a similarity between trying to explain *why* I am happy with my smart phone to my father-in-law and trying to explain why systemic IT improvements make sense to some of my clients.

In both cases, the audience does not share my basic premises. In both cases, there is no "killer app" and no clear analogy from what they know to what they do not know. In both cases, I am asking for a leap of faith.

For my aged father-in-law, who does not particularly want people to be able to interrupt his day with their inane chatter, the mobile phone concept never made sense. Now, when I try to explain that telephony is the minority of what I want my smart phone to do, he chortles with delighted smugness: an expensive phone that isn't even mostly a phone! Imagine the kind of technocratic, free-spending fathead one would have to be to want that! He is becoming a little uncomfortable with the ever-growing percentage of the general population who have joined me in his native city, but he is still happy to soldier on.

However, my point is not his out-of-stepness; rather my point is that my inability to explain to him what he is rejecting. He is rejecting it and then filling in the reasons later. I have failed utterly to convey that the added value is holistic and lies in the totality of the ecosystem. His eyes glaze over when I list the rather large number of functions the "phone" routinely performs for me: check bank balance? Get driving directions? Surf the web to check the hours of a museum? Share links with friends in text messages from which they can go to the same web page? Track my billable hours and expenses? Take photos of friends and family? Take photos of computer parts I need to purchase? Take notes on the train for later access on my desktop? Listen to music? Buy music? Shop for apps for work? Download apps to help me enjoy the museum in which I am standing? It is all too much and too bizarre for him.

So I feel stuck: what benefits he will allow me to present are not enough to explain the actual value of the device+apps+network yet he will not sit through the (admittedly very boring and somewhat peculiar-to-me) actual list. So we have resolved it this way: he is certain that smart phones are expensive toys and that I own one because I like expensive toys, while I shake my head in wonder at what he is missing.

He is not alone in this situation, even in my experience. I have the exact same feeling when I try to explain to IT managers why a given subsystem or middleware is worth building or improving: an endless list of small benefits just does not seem to move people who are not already intimately involved with the given business process. Like a giant jar of small change, which actually has significant value, proposals of myriad incremental improvements sit in the corner, gathering dust.

What You Want To Hear

When I entered the field of computer programming, I had some illusions about knowability, about absolute truth, about right and wrong. I thought that there were real answers to vague questions such as "which of these approaches is better?" "Who is the better coder?" "What should we do to solve this problem?"

Instead of fact-based, rational discussion I find myself often mired in religious and political muck. Specifically, I am watching a client move a data repository from a lovingly handcrafted environment to a standard environment, which standard environment blows up every time.

How did we get here? The data repository is several years old and has been a spectacular success. It was custom built for its purpose and its environment. No radical changes were indicated; so why are we watching them try to move to a dramatically different implementation?

I do not object to the idea of change; I have some sympathy for the desire to update this implementation:

• The custom environment is getting long in the tooth--almost 6 years old as of this writing--so I find it reasonable to consider replacing it.
  
• The custom environment runs Linux which is not the Unix of choice in this particular shop, so I understand their desire to port to their standard, AIX.
  
• The custom environment is based on ReiserFS which is  bit out of the mainstream, so I can see reevaluating this implementation.

What strikes me as sad is how the next implementation was chosen: the proposed implementation is what they have lying around. While the current implementation was carefully researched, painstakingly prototyped and carefully constructed for just this purpose, the proposed implementation was none of these things.

So far as I can tell, this is how the technical design decisions were made:

• The current system is old, so it must be outdated.
• The proposed system is from IBM, so it must be reliable.
• The proposed system is expensive, so it must be capable.

Conclusion: what they already bought is just the ticket. With this conclusion, everyone's judgement is validated: the techs picked the right things, the execs paid for the right things, everyone is right. Moving from the custom environment to the standard environment becomes a no-brainer: cheaper, faster, better. As I mentioned before, everyone loves an easy decision, right?

Well, there is a problem: ReiserFS is great at what it does. AIX's standard offerings have not caught up. Neither has whatever their Network Attached Storage (NAS) is running. So they can't get to step 1: copy *most* of the data to the new home. We are working on our third attempt here.

Is it a surprise that the result-oriented process yielded better results than the blind assertion of capability? No; the surprise is that anyone thought that simply declaring what was in hand to be the best solution was a good idea.

When will they admit that this proposed implementation is unworkable? I am guessing that they will never admit that and nothing we say seems to make any difference: it just isn't what they want to hear.

Often In Error, Never In Doubt

This week I have a personal rant, pure and simple: I am sick of people acting as though all of their choices are perfect, without alternative and obvious.

Long ago and far away, I was introduced to rigorous decision making, a concept invented by the ancient Greeks. As part of this tradition, I was taught to carefully define my premises, apply logic to determine my alternatives and then make value judgments as to which of those alternatives I would support.

In such a methodology, there are facts, which are special because they are objective (universal), there are observations, suggestions and preferences, all of which are subjective (personal).

In attempting to make a decision, especially a decision involving a group of peers or near-peers, it is therefore important to establish the premises and agree upon the facts. Then alternatives can be generated which are both realistic and which tend to achieve the goal. Every member of the group can consider the alternatives, weigh them as they see fit, and discuss the relative merits of the alternatives and perhaps generate hybrid alternatives which have broad appeal.

Instead, I am constantly finding myself on the wrong side of this cycle:

1. There is an issue, usually not a critical one from my perspective
2. Someone senior announces a solution
3. Many someones junior voice objections
4. Objections are countered with simple denial
1. "You are wrong, that is a not a problem"
2. "Your problem exists, but it will handled by the solution"
5. The solution fails along the expected dimensions
6. Juniors are trained to pretend that the seniors were right
7. I am asked to join in the pretense while providing a non-pretend remedy

I am unclear as to why anyone would want to pretend that all problems have a clear, simple, perfect solution. This pretense is obviously absurd and continually contradicted by everyone's life experience. One of the primary skills of adulthood is the acceptance of imperfection and the pursuit of the as-good-as-possible. Why pretend that when at work, suddenly perfections are possible and always achieved by the right process?

This is particularly frustrating for someone in my position, the position of actually having to make systems work in the real world. Unlike senior management, I do not have the option of simply insisting that my work is perfect and that all criticism of it is invalid.

Now that I write that out, I can see that appeal: how awesome that would be! Except for the constantly failing part, which I would hate, even if I could browbeat people into never mentioning it.

My current working hypothesis is this:

1. We live in an age of astounding possibilities and baffling large numbers of choices
2. Our culture prizes confidence, at least in men, and rewards the confident whether or not they are correct (Often in error, never in doubt)
3. In order to be determined confident in complex, uncertain situations, one must be simplistic; keeping it simple is not enough
4. In order to be simplistic, one has to ignore complicating realities
5. Voila! we end up where I find myself: listening to falsehood asserted with authority

Of course, I could be wrong: that is part of my point, one can just about always be wrong which is why one should just about always be willing to engage dissenters and answer their objects. Alas, "Are no!" "Are too" does not constitute discussion, however often or loudly this interaction is repeated.

Painless Tech Transition

I just noticed my trusty Palm Tunsten E gathering dust on my desk. For years, it was my trusty and trusted PDA, worthy companion to my various non-smart phones.

I have been trying to tidy up my workspace and overcome my tendency to hang on to technology long past its expiration date--which reminds me, anyone need serial cables or an i-Opener?

The fact of the matter is that I haven't used my Palm in ages. Our consultancy made an utterly painless transition off of the Palm OS platform and on the iOS platform. In other words, we swapped out our various Palms and swapping in iPhones and even an iPod Touch.

I am struck by how well that tech transition went, especially given that so many of our clients experience tech transition as slow, expensive and painful. So why did ours go so well?

I argue that our transition was painless because it was also patient and precise. We made the list of what we needed, what we had in the Palm + phone era and therefore what we expected to have in the iPhone era. Then we waited until we could get everything that we used to have. Simply put, we wanted to stop carrying three devices and starting carrying one.

(Some of us had PDA + phone + pager; others had PDA + phone + MP3 player. Either way, it was two devices too many.)

Our requirements were not very exotic:

• ability to make and receive phone calls
• ability to send and receive text messages
• support for time-and-materials billing
• contact support
• ability to send and receive emails
• ability to make and share notes
• access to a shared calendar as well as a personal calendar

We wanted to cut-over without leaping into the void, so we didn't wait until our Palms died, although they were clearly wearing out. We started by having one of us get an iPod Touch and use it for a while. It took over a year before all the pieces we wanted were there and validated.

Once the iPod Touch user was up and running, moving to the iPhone was a no-brainer: the cost was significant, but well within reason. Our transition was painless in part because we had had a high degree of uniformity to start with. Her experience is moving from the Palm to the iPhone paved the way.

Originally, I had wanted to make sure that we had at least one Android user, one Blackberry user and one iPhone user. But the lengthy discovery process (largely me striking up conversations with fellow business travelers on trains and air planes) indicated that supporting multiple platforms internally is more resource intensive than I expected or could tolerate. To my surprise, the consensus was that the iPhone ecosystem was more mature and more business-oriented.

While the recent Windows phones are well-reviewed, at the time the Windows phones were so bad that I did long consider them.

I gather that Android is catching up and might be a more of contender now but I don't really know much about it.

What I do know is that the iOS platform has met and exceeded our expectations. The range of applications available is stunning and many of them are actually useful, instead of being merely impressive or amusing. I am shocked that I use the web browsing, mapping, on-line banking, built-in camera, music player and e-book reader as much as I do. I am not much of a gamer, but I even play games on it occasionally, which beats staring into space when I am too tired to read or work.

So a large part of the painless transition is the superior technology to which we transitioned, although I am constantly reminded of a warning given to me by an Android-oriented colleague: the iPhone is a lifestyle choice and one is likely happier if one embraces that. Although I am a Linux guy, I run a VirtualBox virtual machine with Windows in it mostly to support iTunes and to interface with my phone. So far, this has been a small price to pay.

But more than the quality of the target technology, I attribute our success to the lack of artificial deadlines and to existence of clear, precise definitions of success. The only drawback I see so far is the assumption by many of our clients that we got iPhone so we could be part of techno-cool kid herd. That way lies madness. Instead, shoot for making transitions that land you in a place at least as good as where you started. You would be surprised by how many people settle for less.

Rigid Engineers, Obsessive MBAs

I feel a good, old-fashioned rant coming on. Just for balance, I will combine two rants: Rigid engineers and obsessive MBAs.

RIGID ENGINEERS

Engineering is a great discipline. I like it. I try to adhere to its basic tenets. I feel that engineering has the following strengths: it is rigorous, it is regular and it focuses on the relevant.

Bridges tend to stay up. Planes tend to fly. Ships tend to float. If you have a house or a car or a smart phone, you owe the engineering discipline thanks.

Software engineering is a little bit different: our building materials are abstract and our rules are a bit fuzzy and our adherence to our rules is not guaranteed.

Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.

However, my rant today is not about loosey-goosey software engineering, but rather its opposite: inflexible software engineering. The discipline of engineering is not good when there is no good rule and some software usage requirements create a very sloppy execution path. Instead of a measured, graduated response to the complex set of requirement, instead of a long chain of events from messy reality to complex model of the messy reality, the rigid engineer gives us a clean abstraction. Even if the clean abstraction does not, in fact, support the actual business process to be automated.

I have had to deal with engineers who actually raised the following objections to proposed designs or changes to their beautiful cathedrals of structured code:

• "That would break my parser"
• "That would require a goto"
• "That would importing a large, messy library"
• "My structure can't do that"

The subtext is always the same: "I want reality to be changed to suit my model." This often expressed as "the user can do {long complex sequence of instructions} to get the same result. There is no need to change my code."

To me, this is the tail wagging the dog: software should do what it needs to do, first and foremost. It is the job of the engineer to make that happen. It is not the job of the engineer to change the scope of the task to make the software simpler, unless the scope is beyond the ability of the implementor. In that case, the implementation folks should say that and try to work with the users to find a compromise. Decreeing that some functions are a bad idea is not what I mean by "compromise."

This belief that if your implementation cannot support it, then the request is bad and the implementation is good is related to the "great code is not great software" idea I touched on before, here.

OBSESSIVE MBAs

A colleague drew my attention to this great article:

http://www.forbes.com/sites/adamhartung/2012/04/20/sayonara-sony-how-industrial-mba-style-leadership-killed-once-great-company/

This article is a study in how classical MBA management has hurt Sony. The article describes a situation that I find increasing common not just in consumer electronics, but also in IT. (I have ranted about other manifestations of this, notable here.)

This article describes the typical MBA focus on operating efficiency, a la Demming. This focus is great, short-term: assuming that what you are doing today is what you need to be doing tomorrow, then finding better ways to do that is a great idea.

But in technology, figuring out what you should be doing tomorrow is often very important. Operating efficiencies gained with obsolete technology are often smaller than operating efficiencies to be gained by using current or even leading-edge technology.

Taking chances and taking risks as you figure out what it all means to you pays real rewards.

But classical MBA management teaches managers to be wary of the new and to concentrate on the current. In an industry where 18 months is a typical development cycle, a three year plan to increase efficiency will be two cycles out of date by the time it is finished.

I understand that operating efficiency makes sense. I realize that most organizations cannot handle constant flux. I appreciate the need to plan and to budget. But I am horrified at the ever-increasing tendency of the managers we meet to assume that all technology is pretty much the same and that three-to-five year roll-out plans make sense for software, with "freezes" while the implementation is done and evaluated. No wonder so many companies are out of date and out of touch with IT.

Programming by Any Other Name

These days it is hard to find even moderately sized systems which do not claim to offer rule-based options for configuration. These systems claim to provide the flexibility of encoding business rules without the hassles or requirements of actual programming.

Ha! say I. Call it what you like, computer programming is computer programming.

Juliet:
"What's in a name? That which we call a rose
By any other name would smell as sweet."
Romeo and Juliet (II, ii, 1-2)

Inspired by the great Shakespeare I say that programming, by any other name, still smells of sweat.

Note that I am not saying that rule-based systems are not useful, or powerful, or neat-o. I am saying that translating real-world polices into absolute statements in some rule-definition language is not a common skill and is so similar to most kinds of computer programming as to be indistinguishable.

Dress it up however you like: if you are not good at programming you likely will not be good at writing rules, let alone debugging rule implementations or grasping the cumulative effect of a large rule set on a large and complex application.

I suppose one could divide up computer programming into "predicate logic" and "computer stuff" (RAM, file I/O, databases, communications, etc). Using this division, I can imagine a person who excels at predicate logic but has no feel for the computer stuff, making this person uninterested in computer programming.

If such a person ended up as a business executive instead of a formal logic academic, (or APL programmer) such a person might be quite good at writing rules. That person might even be able to debug her rules, to cope with the often-wide chasm between the abstraction presented by such systems and the concrete reality presented by the working environment. But how many such people can there be in the world? Certainly not several in every moderately sized company in the world.

So when sales people tell you that, at last! you free of the tedium of dealing with IT professionals, finally free of the tyranny of computer programmers, stop and ask yourself this: are you really ready to eat your own cooking?

As every even junior programmer knows, predicate logic in action is rife with  unintended consequences which are the inevitable-but-unforeseen results of rules as applied to the real world.

My personal favorite example of this lateness and snow storms. Most or all of our clients above moderate size use Kronos Time & Attendance products. Most or all of our clients, in the frenzied enthusiasm of the original roll-out, go overboard with the rules, often with amusing unintended consequences. One client decided to implement a "better never than late" policy: if its employees were more than one hour late for work, they could not swipe in at all. This was intended to enforce punctuality. I do not know how it worked in that regard, but I do know that when I arrived mid-morning after a large snowstorm, I was greeted with a stream of employees leaving work: the snow had delayed them and they could not swipe in; unsure that they would be paid for the day, they were going home. Not much got done that day, but what did get done was mostly done by upper management, who were exempt from that rule.

In this case in particular, I like the "silver bullet" trope because even though there is no magic solution to the problem of translating human policy into machine-readable form, the problem can, werewolf-like, sudden get very ugly very quickly under the right circumstances.

So see a doctor for medical problems, go to an accountant for accounting and get your rules from a machine-readable logic expert. Or don't, but at least go into your decision with your eyes open.

The Utility of Bad Examples

Today I am reflecting on lessons I have learned in my long career and how I learned them. The obvious approach is to remember the exceptional individuals who were great at their jobs while being pleasant and consistent and generally a pleasure to know. But who needs to be reminded that we can learn from paragons of excellence? Instead, I am contemplating what I learned from the deeply incompetent, the arrogant and the people I've met who cannot seem to learn from their negative experiences.

Many of us really love a good story about some fathead and his or her hideous blunders but that is what I present here. Like the poor, fatheads will always be with us. Avoiding them is the best course of action, if you have that luxury. Failing that, there are strategies for mitigating the ill effects of fatheadedness, but that is entirely different story. Besides, we often have no choice but let fatheadedness run its course. In those all-too-frequent cases, what is the benefit?

The benefit is experience, precious useful experience and the attendant opportunity to learn from someone else's mistakes. In post, I contemplate the utility of bad examples: what do the actions of fatheads teach us, to make us better at what we do?

In my experience, the truly spectacular fathead (SFH) is arrogant: so arrogant that he or she (usually a he, so I will use that pronoun for convenience) is above mere conventional wisdom. Sometimes "thinking outside the box" is valuable, but most of the time thinking outside the box is a waste of time or worse: for conventional situations, conventional wisdom usually suffices. In order to be worth it, thinking outside the box must present some clear additional benefit to compensate for the greater effort. I try to keep in mind that in today's IT environment, human attention is the most precious resource. Use it wisely.

(So why the endless praise of out-of-the-box thinking? Because it is the employment situation that gives kudos for doing the obvious, even if the obvious is the right choice. And, every once in a great while, thinking outside of the box saves the day. The trick is to pay attention so that you notice when your situation is abnormal, but that is another post.)

Since the SFH is arrogant, he is often an object lesson in understanding conventional wisdom: why do we never do that? Oh, THAT's why. Rather than recount amusing examples of stupid people doing stupid things, I offer this suggestion: keep track of instances of bold, innovative thinking. Follow up months or years later: did the bold innovation do better or worse that conventional wisdom expects? Why? I have learned much about why certain rules of thumb exist this way--mostly as I sat amid the smoking wreckage of some IT disaster, but at least I had something useful to do while I was sitting there.

Often the SFH makes the same kinds of mistake over and over again; after all, it is almost a requirement that the SFH be unable to learn from his mistakes if he is to remain the SFH. Once you see that patterns, the SFH is useful as the embodiment of a certain kind of habitual error: when faces with certain kinds of decision, ask yourself what the SFH would do and then DO SOMETHING ELSE. Ideally, do something that makes sense in light of what your SFH has inadvertently taught you.

Now we come to the most painful methodology: seeking useful feedback. I find that a simple description of an SFH strategy, preferably to someone outside the SFH's organization to avoid accidental embarrassment, often nets useful results. A very good outcome of bouncing your observations off of someone you respect is that they may explain levels to the problem you had not seen. The most useful, but least pleasant outcome of this exercise is the casual observation from your respected sounding board that YOU are guilty of the same bad judgment. If true, this feedback is invaluable in improving yourself.

I say "if true" because some people feel that it is either appropriate or polite to accuse the speaker of any fault the speaker finds in others. But this is not a get-out-of-jail card: not all unpleasant feedback is some kind of tit-for-tat reflex. This means that really have to think about that you said, what they responded and how their insight improves your understanding of your situation. If your understanding is not improved, then discard the feedback--but discard it discreetly and politely. You might need feedback in the future and remember: most people won't be willing to give it at all, so sometimes reflexly negative feedback is better than no feedback at all.

Ignorance is bliss--until it isn't. Think about why bad decisions turned out to be bad and you are on the path to making better decisions in the future.

Reasoning From Worst Principles

I have nothing against reasoning from first principles; in fact, I really enjoy it. Reasoning from first principles is an excellent alternative when experience and expertise fail us. I particularly like reasoning from first principles when helping people debug large, complex systems: I find that going back to basics very helpful in avoiding the prejudices and preconceptions that are usually at the heart of really thorny debugging sagas.

However, there is a situation in which I find reasoning from first principles to be very frustrating: when management values their reasoning from first principles over all other input, including experience and expertise.

I understand that modern IT decisions are complex and cover such a vast array of possibilities that making a truly informed decision is often impossible. But we should not let the fact that we can't be perfect provide us with an excuse to be terrible.

I find that in the face of uncertainty only flexibility offers a high probability of success.

To a recent example from my own consultancy, we are in the midst of adding disk space to our core server. This is a very common situation, although there are a few complicating factors:

• Our server is a High Availability Linux cluster with a shared RAID
• Our server is a bit long in the tooth
• We really prize our server uptime

 In consultation with our sys admin experts, I was presented with the following options:

1. Get a new server, which would be more current and have more capacity
2. In-place hard disk upgrades
3. External disk upgrades

I evaluated the options thusly:

1.  New server
1. Pros: effective, easy fall-back, modernized environment
2. Cons: expensive, housing old & new, possible environment issues, not indicated except for disk space
2. In-place upgrade
1. Pros: leaves most of server intact
2. Cons: always a risk to crack open a case, requires lots of human attention, process is complex
3.  External disk upgrades
1. Pros: cheap in hardware cost and human time, minimum disruption to servers
2. Cons: probably mediocre performance, another point of failure, chance to trip over cable, etc

I chose option 3 as being the cheapest, fastest, safest option.

Given what happened next, I was probably wrong:

• the first external SATA enclosures did not play well with the kernel
• the second external SATA + disk combination takes too long to wake up
• in rearranging the enclosures showed up an issue with the outdated internal disk on the primary node

So now it looks like we will have to do option 3 for now and look into option 2 as well. Oh, well: that's life when you don't know everything. At least the server has stayed up and useful, we haven't more than we can afford and I see a path to my goal. I had to change course with the first set of enclosures and now I will have to change course again. Responding to real-world consequences of actions is how adults deal with reality.

Sadly, in our information systems consulting, we are running into magical thinking with ever-greater frequency. Specifically, we are running into the following template:

We had a problem, P, which we needed to solve. To solve P, we had to decide on a solution. That decision is decision D. In order to make decision D, we over-simplified P and declared that system S would completely solve P. We paid so much for S that we reason, from first principles, that problem P is solved. Since this is the real world and therefore complicated, there are aspects of system S which are imperfect, but we declare that since system S cost so much, it must do so much and it must do enough.

This is not simply a case of my being nit-picky and snarky about the usual inability to think clearly. In this case, the managers have defined the problem as not existing. Fixing a problem that does not exist is either impossible or unnecessary  or both.

When this magical thinking is in place, the tension that our customers live under is huge and is caused by the fact that they live with problems but are declared to be without problems.

There is usually the added tension of the implication that not being able to use the expensive solution to solve their problems makes them incompetent; there is often the complication that someone in their chain of command had to consciously over-promise in order to get the money for the expensive solution, so trying to get help is viewed as disloyal.

As an outsider trying to solve the problem, it is very uncomfortable to be told that whatever else we do, we may not name the actual problem since the actual problem is officially nonexistent.

This gives us, the outside consultants, two options: either look for related solutions which might help the actual problem, or be heretical and name the actual problem. Sadly for us, many professionals view consultants as a kind of Foreign Legion: expendable troops. If we poison our relationship with their bosses by saying what they cannot, so what?

We understand that only we care about how uncomfortable or unpleasant our working conditions are and there is good money to be made as IT Foreign Legionnaires, so we cannot really complain about that. But what I can do is point out that once you enter the realm of magical thinking, of reasoning from worst principles, you make success almost impossible and you make your own working environment tense and unproductive. Who needs that?

Benign Neglect & Tech Support

In general, I am not a fan of the "benign neglect" concept, but I make an exception for some tech support questions.

(Psst, I'll tell you a secret if you promise to keep it to yourself: I am not the only one. But more on that below.)

By this, I mean that when I am providing tech support for products, which I do on occasion, I sometimes deliberately take my time responding. I sometimes take a long time because I am really busy, but that is an entirely different kettle of fish.

I should point out that my software company, as opposed to my consulting company, is a no-frills operation: we offer a high-quality product at a rock-bottom price. We do this by skimping on the human interaction: we provide a web site, a tech blog and a free on-line manual. What we don't provide is a call center filled with bodies to talk to you on the phone; we also provide email-based tech support and a cheerful, prompt, full refund if you would rather pay literally ten times as much to one of our competitors, who will happily talk to you on the phone.

So we have a severe market-segmentation device; them that like us love us and them that don't like us REALLY don't like us. Those that love us love the fact that when we get back to you, you get a thorough, thoughtful answer. But before you get that thorough, thoughtful answer you may get a thorough, thoughtful silence.

And not all tech support requests trigger the benign neglect policy: in fact, most do not. Most are reasonable questions from reasonable people which are a pleasure to answer. Some are valid criticisms that I cringe to acknowledge, but look forward to fixing. Then there are...the others.

There are two query categories to which I apply this policy: the breathless and the clueless.

By "breathless" I mean the really excited highly caffeinated poorly punctuated query that runs on and on and interrupts itself because the writer is so darn excited he-or-she cannot contain his-or-herself and just goes on and on with super-detailed but indiscriminatingly detailed accounts of some tale of horror involving my technology behaving in ways that it does not cannot never does behave.

By clueless, I mean the complaint or question reveals such a profound ignorance of the general topic and specific issue that it is hard to know where to begin to answer.

I give the breathless a chance to catch their breath, which they often do: the initial babbling request for tech support is often followed by the equivalent of "oops, sorry, I figured it out, I was doing something stupid." In other cases, I at least get a follow up that says "I figured out X and Y, but I still don't get Z." That is a query to which I can respond: I feel that we have a shared basic understanding and a clear context for a reply to issue Z, instead of a headache and desire to forget.

I give the clueless a chance to get a clue: to reconsider what they are trying to do or how they are trying to do it. Or to decide that they want one of those cheerful, prompt and full refunds. Which we are delighted to provide: spending time arguing with people about how a $250 product can't come with hours of hand-holding is a great way to lose money.

PS It is my experience that every tech support organization does something similar: they may have a polite ignoramus contact you to acknowledge your request for support, but the actual tech firepower is probably waiting for you to catch your breath or get a clue.

Experience is Wasted on the Young

Ageism is legendary in the computer technology business: we all decry the fact that managers are pressured to hire "kids right out of college who grew up with this technology." But the pressure continues and the outcome remains the same: otherwise inexplicably bad software.

For the sake of brevity, let us choose an example of a hot new development technology: Ruby on Rails (http://rubyonrails.org/). This assumes, of course, that some thing can still be considered "hot" if I have heard of it.

If you hire a group of neophytes who "grew up with" Ruby on Rails, what do you get? You get a group of programmers who can hit the ground running in your chosen development technology, which is good. What do you lack? You lack domain knowledge in your target domain, you lack real-world project experience, you lack experience in all that a working, commercial system must do other than meet a technical spec; in fact, you lack just about all relevant experience outside the narrow domain of Ruby on Rails development.

The only scenario I can imagine being well-served by a neophyte team of Ruby on Rails programmers is a contest of some sort where the goal is to create a Ruby on Rails app in a short time that will impress a panel of college students.

Organizations keep going down this path and keep being disappointed that the software they get isn't very good. In fact, it is often worse than the software it replaces. And there isn't usually a progression within a domain, of software getting better and better, and feature sets evolving to keep the best features and lose the apparently-cool-but-actually useless ones.


This is because evolution is a cumulative process: if you keep starting from scratch, you keep making the same mistakes. This is the unavoidable truth even if you are using ever-more-hip environments to make those mistakes: perhaps you will be able to make those mistakes faster and cheaper and on more platforms than you could before, but you will still make those mistakes.

I know that there are software Mozarts out there, able to write great code from an early age, but "great code" does not mean "great software." "Great code" usually means "code that impresses other programmers." All  too often users of the software built with that code are left wondering how the author of an irritatingly hard to use program is considered so highly by his technical peers.


As a practical example from my own experience, I offer this common task: loading data into a database. I am an expert in "moving data around" so I am often called in to assist with this particular job. I often find a young, inexperienced team of programmers who are expert in a particular technology, but not computer system experts per se. Increasingly, the home team are all experts in some interactive-oriented technology, such as Ruby on Rails or Visual BASIC, and not at all comfortable with batch-oriented technology such as SQL transaction sets.

The common mistake for the young and inexperienced is to try to force a batch operation into an interactive paradigm, to write code to take the input data and force it through the system as if a superhuman typist had entered it at the keyboard. This mistake has two possible failure modes; sometimes it fails in both ways at the same time:

• The loading does not work reliably
• The loading takes an unusably long time

When the loading is unreliable, the issues are almost always exception-handling and/or data processing.

Experience tells me that usually the right way to handle a batch update is to apply the transaction one-by-one, putting aside the failures for later human review, correction and reapplication. Only rarely does the user want an entire data set rejected because of a typo in one of the fields of one of the records.

Users generally are far more comfortable with a UI to tell them how many record were in the batch, how many went applied to the database and how many were not applied. They really like being able to inspect and correct the bad records and re-process them until the batch is closed. Does this fit the textbook SQL COMMIT & ROLLBACK paradigm? No, it does not. Sadly for our team of neophytes who will have to learn the hard way why the users are so unhappy with their work. Maybe as our team gets more experience, they will get better, if they can find a job after they hit 28.

When the loading is slow, the issue is almost always trying to put a large number of transactions through a pipe designed to service human input.

The overhead in servicing human input is vast, but the human data rate is so low that this overhead is quite acceptable. Turn that equation on its head--fast data rate, no user to react to the various edit checks--and you have a very poor computing paradigm.

Instead of trying to put data through the human channel, experience tells me that it is often quick and easy and reliable to process the incoming data directly into SQL INSERT, DELETE or UPDATE statements and let the database manager do its thing. If you need to make multiple passes over the data to establish relationships, then do that. If you need to sort through transactions to weed out duplicates or just take the most recent change, do that too.

When you are done fine-tuning your batch process, you can schedule the job to run in the background and voila! yesterday's data is loaded before the start of work today. The computer doesn't mind the extra work, so make the process redundant and have it try over and over again so that delays are not a problem and so that accidental resubmission of data is rejected.

Experience tells me that these operational aspects are what makes a data loading process robust and useful. I don't see how being expert with a given development environment, but ignorant of real-world issues, would make me better at my job.

Now if I could only convince the ever-growing cadre of number-crunching, budget-watching managers who came up through the management track that this delightful fallacy is rarely a good strategy. Maybe I need to find a Ruby on Rails book and add some more buzzwords to my CV.

Little Things Mean Alot, Eventually

Being a science-oriented male, I have a perhaps under nuanced view of the world. At both work and home, I tend to prioritize my tasks, which is usually good.

But when there is not enough time to do everything, this method has a huge drawback: because there is always a high priority task to do, low priority tasks never get done. This causes problems both at work and home: eventually the owners and stakeholders of even low priority tasks tire of waiting for attention.

As is so often the case, I drew on Linux kernel development for inspiration in my life. [This is a huge exaggeration for comic effect. Really. I hope.]

In the early days, the Linux kernel's scheduling was relatively simple: the more important tasks, such as disk I/O, had high priority and relatively un-time-critical tasks such as servicing the keyboard had relatively low priority. This meant that early Linux systems were amazingly good at getting performance out of mediocre hardware BUT performance degradation was not graceful. In other words, when things went bad they went very bad very quickly.

I remember hammering on the keyboard in frustration, having provoked a database server into taking on more work than it could handle, my typing apparently having no effect, until the system decided to see what was happening in keyboard land, whereupon all of my frustrated input, all of it, was faithfully processed to amusingly ill effect. Well, amusing with the distance of 19 years. At the time, I was beside myself.

Soon thereafter, the raise-the-priority-with-age algorithms were fine-tuned, giving me a much better experience even when I do something really stupid. Or when other people do something stupid. When stupidity raises its ugly head.

Human beings seem to have an innate sense that age should raise the priority of a task. Intellectually, we all want to do the most important tasks (or easiest, or most praise-garnering tasks) but viscerally we also want the small things which have been bugging us forever to get done.

As IT people, we often forget that. I know that I am inclined to avoid what I consider "trivia" because I feel that I am so important that such things are a waste of my time. Alas, I cannot seem to get much buy-in to this view of the importance of my time, so I  had to institute "laundry day" in our consultancy: a day every couple of weeks dedicated to doing the laundry: fixing or adding all the small, silly, useless things that users value and that they will eventually rage about unless we get to them.

We see many IT organizations which do not make time for trivia, to very ill effect. We sympathize with IT folks who are not given this opportunity: if your boss does not let you do the laundry, it isn't your fault that you stink (figuratively speaking: the tendency of programmers toward low standards of hygene is another matter entirely).

Which reminds me: currently, I am WAY overdue to replace one of the outside lights, which lights are important to my wife but not to me. I will try to get to that today even if I feel that I have more important things to do. Sigh.

Uneven Distribution of IT Talent or Expectation?

Once upon a time, I wrote a book about IT management. This was something of a vanity project: I got some stuff off my chest and then I felt better. (This was before blogs, which are much better suited for venting.)

Since I have a side business publishing books, this vanity project at least had some practical value: we debugged the process with my vanity project before moving on to other projects.

However it came to be, I was struck by the reaction to my book: the few people who read it fall neatly into one of two camps:

1. This book is useless: it describes solutions to management problems that no one has because no IT environment is this bad.
   
2. This book is a much-needed guide to the horror that is current American corporate IT.

This bifurcated reaction raises a question, regardless of my book per se: is IT talent massively unevenly distributed, or does expectation or understanding of IT performance vary massively from organization to organization?

I have not been able to make much headway answering this question: many of the people I have surveyed are MBAs who seem both confident in their ability to assess IT and greatly lacking in actual ability to assess IT at a fundamental level.

On the other hand, since IT management is increasingly about pleasing MBAs,  perhaps I just out of date about what constitutes IT competence.

In my consulting practice, we see a wide range of IT competence with a strong bias toward low competence. So I am inclined to believe that talent is vastly unevenly distributed. But why would that be? Is financial and accounting talent similarly unevenly distributed?

Many colleagues have pointed out to me that this bias in my observation is to be expected: we are consulting firm, after all, and only incompetent organizations need our services.

I am not a big fan of this idea: I maintain that rational, competent IT organizations can choose to outsource specific jobs for any or all of the following reasons:

1. There is not enough time to acquire the expertise in-house.
2. This is a pilot project with an uncertain future.
3. You are not a development shop and cannot find an off-the-shelf solution.

In the past, we have done project with companies internationally recognized for their excellence, so I dispute the notion that only weak organizations consume IT consulting, even though there is some of that.

My quest continues: someday I may know if the talent or the expectation is what varies so greatly from organization to organization.