When You Can't Read The Code

As the large enterprise solution becomes more and more common, I find that the keepers of such systems seem less and less interested in putting in the time to become domain experts.

As IT systems consultants, we are often interacting with these enterprise solutions. Some percentage of the time, we encounter behaviors in these systems which we find unexpected or undesirable. These behaviors may or may not be bugs, but we feel that we should understand them before we ignore them or program around them.

Back in the day, when more apps were homegrown and more systems were provided with source code licenses, the people we asked about unexpected or undesired behavior were the client's own technical people. If they did not know the answer, then they could find out by reading the code.

Increasingly, we find that our client contacts are either non-technical people or out-of-date technical people. Only a small minority of contacts are technically competent, helpful providers of deep background and technical information. We often have to bypass the client and go straight to their large system vendors because there is no local technical expertise.


To our dismay, it is our technical bretheren, our peeps, who are the hardest to deal with: now that they cannot read the code, they tend to either shrug off our questions or give us shallow guesses off the top of their heads. After all, if you can't be a hard master then why not just give up?

To our delighted surprise, the non-technical people are generally easier to deal with because they know that they don't know and they look for the answer. They will actually contact the vendor for information, or find documentation, or give us the benefit of the user's experience.

While we don't like treating systems as black boxes or as puzzles into which we pour various inputs and monitor the outputs, at least these methods actually turn up results. Getting half-baked guesses from formerly expert techies just wastes our time as we discover just how half-baked the guesses are.

Just recently I spent an afternoon running a series of experiments as we tried various inputs into a large system to see if we could figure out how to accomplish a particular upgrade for the user. We started with a half-baked guess from a supposed expert techie, proved that this techie was wrong about how the large system worked and got down to the business of figuring out to do what must be done.

It seems to me that technical people have to be able to operate in the absence of hard data. My checklist for answering technical questions goes like this:

1. Query experts if you can
2. Read the code if possible
3. Read the technical documentation if there is any
4. Query super-users if you can
5. Read the user documentation if you can
6. Query experienced normal users if you can
7. Write code to grok databases, logs, etc
8. Try to reason from first principles
9. Guess because there is absolutely no other option

And remember that when talking about technical matters, which are not generally matters of taste or subjective opinion, trust but verify.