Horse Trading, Deadlines & Budgets

Here is another take on my philosophy that the classical software engineering model is a poor fit for business process automation. This week's topic: horse trading, ie "you give me something and I'll give you something else."

As I understand it, this is the classical software engineering model:

1. Encounter a problem which can be solved or ameliorated with IT
2. Do extensive requirements gathering
3. Define a clear and fixed scope-of-work
4. Write a detailed design of an implementation which meets the requirements
5. Create an implementation plan with dates and milestones
6. Develop, debug, deploy

This is fine for many situations, especially situations which require the creation of something new that is complex and a team effort. I have participated in projects using this model in nearly every role and at nearly every level.

But I found this that model is poorly suited to business process automation, partly because this model is good at building something from scratch but business process automation is almost always a renovation, not a new construction.

A key way in which this model does not suit business process automation is this model's lack of accounting for horse trading, for the give-and-take required to deploy a solution in a working business.

(Horse trading also makes the apparent scope merely the tip of the iceberg, but we will cover that indirectly, below.)

Rather than define horse trading in the abstract, I will take the easy way out and give an example.

My firm specializes in medical information systems, particularly interfaces between medical information systems. Our client had an unusual situation: in addition to their Main Lab, they have a collection of loosely affiliated labs. To the outside world, this distinction is confusing and uninteresting, so this situation created a PR problem: why were some orders slower to process and harder to get results from?

The answer was the invisible divisions between the Main Lab and the Other Labs, but no one really wanted to know why, they just wanted better service. Once the customer grumpiness reached critical levels, the project became a crisis. Since we specialize in the difficult and the critical we were called in because decades of previous attempts had failed.

We were able to connect the three most important Other Labs, one at a time, quickly and cheaply. The secret? Horse trading and the flexibility horse trading requires.

In theory, the job is easy:

1. encode Other Lab results in an industry-standard format (HL7)
2. create a simple TCP/IP connection between us and Main Lab
3. send message in HL7 over TCP/IP connection

Our competitors bid about $50K per connection. We could do this part for $17K for all connections and still have made money, but we knew that the job was not this simple, or it would have already been done. We bid a base of $17K and about the same for each connection. We made money and got the job done, but here is how the project unfolded:

Lab A only processed orders from the Main Lab's parent organization and they had no internal computer system, so they wanted a simple work flow support app in exchange for going along with the computerization. So in addition to the straight interface, we created app A, which is driven entirely from Main Lab order for this lab and which lets Lab A see upcoming orders, confirm receipt of specimens, enter results and create local reports in case their customers want paper.

Lab B processed mostly orders from the Main Lab's parent organization and they also had no internal computer system, but they had automated analysers from which they wanted results automatically entered. So in addition to the straight interface, we created app B which is driven off of the analyser output, has different reports and pulls patient demographics from the Main Lab to flesh out the reports.

Lab C processed mostly orders from outside the Main Lab's parent organization. As a research-oriented organization, they had their own database of results kept in a homegrown app. They wanted to subcontract for a better internal app and database, which we did. We also created app C, a much simpler app mostly about importing results from their new internal app, allowing for verification and electronic signing of the results which were then passed along to the straight interface.

The project ended up giving everyone what they needed and being cheaper to boot. But  the way in which the project played out does not fit the classical model at all.

Would a classical requirements-gathering process have given us the true scope beforehand? I doubt it, since the outside players have no incentive to reveal their agenda until they are sure that they will get what they want. Often players in business processes do not know what they want until the new situation becomes clearer, nor do they know what is actually possible until they seem some progress.

So remember: not all project scope changes are caused unethical consultants or incompetent engineers. Bear in mind that business process automation is almost always renovation and that renovation is almost always a compromise. Renovation often means working around outdated decisions and respecting the need for on-going operations.

Some jobs are bigger than you can see and some situations need a little good faith horse trading in order to arrive at mutually beneficial solution.