"A Study of an Enterprise Health information System" - Finally, an Informatics Scientist Does A Rigorous Review of a Commercial EHR System, by Cerner

Australian professor Jon Patrick, who I wrote about at this blog in the past such as at my Nov. 2009 post "Academic Freedom and ED EHR's Down Under: An Update", has written what I consider the first truly serious, rigorous evaluation of a commercial EHR system.

He has evaluated the Cerner FirstNet ED EHR, being 'forced' on hospitals in the Australian state of New South Wales by their government.

His evaluation is entitled "A study of an Enterprise Health information System" and was released on March 4, 2011.

In my view, all current EHR's should undergo this level of scrutiny and critique.

His multi-part analysis, down the the level of the data schema, speaks for itself.

The report is at: http://sydney.edu.au/engineering/it/~hitru/index.php?option=com_content&task=view&id=91&Itemid=146

or download directly from: http://www.it.usyd.edu.au/~hitru/essays/Study%20of%20Enterprise%20Health%20IS%20-%20Parts%200-9%20PDF-ARCHIVE.zip [Note: this is a 12 Mb .zip file].

Summary:

This is a study into the roll-out of Cerner FirstNet into EDs in NSW. The original study was issued in Dec 2009 (Part 3.1). This has been added to with a new study in 2010 consisting of discussions with 7 ED Directors (Part 3.2), discussions with software experts who do performance evaluations on Cerner sites (Part 3.3), and reviews of Entity-Relationship Diagrams (Part 3.4), Schema descriptions and data tables from customer installations (Part 3.5 & 3.6). All this information is coalesced to establish a much more detailed picture of a Cerner installation (Part 3.7). A number of weaknesses are identified in the design and implementation and risk assessments are recommended for organisations using this software or intending to use it. Regulations that might minimise the risks to users of health software are recommended (Part 3.8). An alternative architecture and method for constructing clinical information systems is presented (Part 3.9).

The .zip file contains the following files:

3.0 Part 0 - Executive Summary
3.1 Part 1 - A Critical Essay on the Deployment of an ED Clinical Information System ‐ Systemic Failure or Bad Luck? First published here in Oct 2009, revised Dec 2009.
3.2 Part 2 - Discussions with ED Directors: Are we on the right track?
3.3 Part 3 - Discussions with Software Performance Experts.
3.4 Part 4 - Conceptual Data Modelling.
3.5 Part 5 - Database Relational Schema and Data Tables.
3.6 Part 6 - Coalescing the Analyses of the ER Diagrams, Relational Schemata and Data Tables.
3.7 Part 7 - The Integrated Assessment.
3.8 Part 8 - Future HIT Regulation Proposals.
3.9 Part 9 - Ockham's Razor of Design. Published at the IHI conference, Nov 2010 Washington.

A non-technical but revealing summary from Part 2:

Discussions were held with the Directors of 7 Emergency Departments in New South Wales (NSW) public hospitals assessing the impact of the introduction of the FirstNet information system into their Departments. All but one of the Directors has found that the system has had a deleterious impact on their department’s clinical work. The range of problems reported indicate that whilst the software is not fit-for-purpose, many of the problems are created or exacerbated by attitudes of the NSW Health IT support, Health Support Services (HSS). In most departments it was reported that staff have developed significant strategies for minimising and circumventing the use of the system. The Directors are frustrated by the lack of a reporting functionality that disables their ability to monitor their own department’s performance. Most Directors report an increase in the time required to deal with patients and therefore a deterioration in access block times. This has been masked by changes in the way this time has been redefined by NSW Health. Overall, most perceive that in moving from their previous information system EDIS to FirstNet they and their patients have suffered. Most Directors are resigned to the fact that no improvements will be made to the current performance of the system due to its inherent inadequacies and the attitude of HSS. A consequence of the ED Directors critique leads inevitably to the debate on the merits of enterprise wide systems versus best-of-breed systems. Emerging from these issues are criteria for a new technology for creating clinical information systems.

I will have more to say on this study later. I have to take an injured relative to the doctor. (Note - the culpable system was not by this vendor).

-- SS