The unintended consequences of automation

An important study from the University of Pennsylvania evaluates the unintended consequences of automating health care records. The full study is in the Journal of the American Medical Informatics Association, or JAMIA.

All that fudging doctors use to get past the bureaucracy suddenly bites. A doctor may need to input a patient's weight to get a certain prescription. He gives an estimate, then that estimate may be used by another doctor who actually needs a precise measurement.

Or, as the Penn motto holds, "laws without character are worthless."

And so it goes. What you could call ambiguous when it was handwritten becomes writ when it's in the computer. What you did last year, in detail, becomes the basis for what we do this year, even if last year's work was a guess that turned out wrong.

Electronic health records give insurers a "paper trail" with which to second guess your every diagnosis, every test you order, every move you make. They might run it through one of those "decision support systems," get a different answer, then refuse your bill.

That's the fear, anyway.

The point the study's authors are making is that you can't wait until after you implement a system before examining and seeking to mitigate these risks. Evaluation has to be ongoing, real-time, constant.

As the study's co-author, Michael Harrison, said, "Real time evaluations can reveal unintended consequences as they emerge, allowing remedial action to be taken."