How Academic and Government "Anecdotes Are Not Data" Ideologues Kill People

I'm already receiving comments that, regarding Prof. Jon Patrick's detailed exposé of the dangers of ill-suited-for-purpose ED EHR's, Patrick's observations are:

... not really valid because they're not peer reviewed; they're just anecdotal.

Only an egghead could pen such words.

I always get hives immediately after eating strawberries. But without a scientifically controlled experiment with all the right peer review, it's not reliable data. So I continue to eat strawberries every day, since I can't tell if they cause hives.

I'd already written about anecdotalist refrains at my Mar. 7, 2011 post "Australian ED EHR Study: Putting the Lie to the Line "Your Evidence Is Anecdotal, Thus Worthless" Used by Eggheads, Fools and Gonifs." In that essay I cite Dr. Patrick himself on "anecdotal evidence", regarding which he hit the ball out of the Southern Hemisphere in an editorial in "Applied Clinical Informatics" entitled "The Validity of Personal Experiences in Evaluating HIT."

Aside from the fact that eggheads also don't seem to care about the issues of faulty peer review, especially in profitable biomedical sectors, such as at "The Lancet Emphasizes the Threats to the Academic Medical Mission" with its embedded links, and "Has Ghostwriting Infected The Experts With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?", there's this simple fact:

Public health catastrophe warnings from responsible sources don't need peer review, they need investigation.

Yes, there were those pesky, off-narrative journalistic reports that the Japanese nuclear reactors were not entirely safe, that Bernie Madoff was a fraud, that mortgages for everyone was not a good idea, that the O-rings couldn't stand sub-freezing temperatures, that the the foam that broke off the Columbia launch tank caused a danger, and that the Titanic didn't have enough lifeboats, but they weren't peer reviewed...so we ignored them. Saved us a lot of money, too.

-- SS

Addendum:

At my post "Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2", I opined:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

... researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

I stand by this assertion.

Finally, I ask: at what point does ignoring work such as Prof. Patrick's, if patient harm is caused by the system he reviewed, constitute reckless endangerment and perhaps criminal negligence by hospital and government officials?

-- SS

Addendum Mar. 23:

As if on cue, this story appeared in the WSJ:

March 23, 2011

Japan Ignored Warning of Nuclear Vulnerability

TOKYO—Japanese regulators discussed in recent months the use of new cooling technologies at nuclear plants that could have lessened or prevented the disaster that struck this month when a tsunami wiped out the electricity at the stricken Fukushima Daiichi power facility.

However, they chose to ignore the vulnerability at existing reactors and instead focused on fixing the issue in future ones, government and corporate documents show. There was no serious discussion of retrofitting older plants with the alternative technology

I guess the "vulnerability reports" just weren't peer reviewed, therefore meaningless - or not reviewed by the "right" peers.

This sounds like our own FDA, ONC office and Institute of Medicine (via the Committee on Patient Safety and Health Information Technology), "choosing to ignore" health IT "vulnerabilities" (such as the aforementioned) and focusing on future issues such as comparative effectiveness research, "the common good", etc. instead.

I call this attitude "reckless endangerment" and hope plaintiff attorneys are paying close attention.

-- SS

How Academic and Government "Anecdotes Are Not Data" Ideologues Kill People

I'm already receiving comments that, regarding Prof. Jon Patrick's detailed exposé of the dangers of ill-suited-for-purpose ED EHR's, Patrick's observations are:

... not really valid because they're not peer reviewed; they're just anecdotal.

Only an egghead could pen such words.

I always get hives immediately after eating strawberries. But without a scientifically controlled experiment with all the right peer review, it's not reliable data. So I continue to eat strawberries every day, since I can't tell if they cause hives.

I'd already written about anecdotalist refrains at my Mar. 7, 2011 post "Australian ED EHR Study: Putting the Lie to the Line "Your Evidence Is Anecdotal, Thus Worthless" Used by Eggheads, Fools and Gonifs." In that essay I cite Dr. Patrick himself on "anecdotal evidence", regarding which he hit the ball out of the Southern Hemisphere in an editorial in "Applied Clinical Informatics" entitled "The Validity of Personal Experiences in Evaluating HIT."

Aside from the fact that eggheads also don't seem to care about the issues of faulty peer review, especially in profitable biomedical sectors, such as at "The Lancet Emphasizes the Threats to the Academic Medical Mission" with its embedded links, and "Has Ghostwriting Infected The Experts With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?", there's this simple fact:

Public health catastrophe warnings from responsible sources don't need peer review, they need investigation.

Yes, there were those pesky, off-narrative journalistic reports that the Japanese nuclear reactors were not entirely safe, that Bernie Madoff was a fraud, that mortgages for everyone was not a good idea, that the O-rings couldn't stand sub-freezing temperatures, that the the foam that broke off the Columbia launch tank caused a danger, and that the Titanic didn't have enough lifeboats, but they weren't peer reviewed...so we ignored them. Saved us a lot of money, too.

-- SS

Addendum:

At my post "Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2", I opined:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

... researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

I stand by this assertion.

Finally, I ask: at what point does ignoring work such as Prof. Patrick's, if patient harm is caused by the system he reviewed, constitute reckless endangerment and perhaps criminal negligence by hospital and government officials?

-- SS

Addendum Mar. 23:

As if on cue, this story appeared in the WSJ:

March 23, 2011

Japan Ignored Warning of Nuclear Vulnerability

TOKYO—Japanese regulators discussed in recent months the use of new cooling technologies at nuclear plants that could have lessened or prevented the disaster that struck this month when a tsunami wiped out the electricity at the stricken Fukushima Daiichi power facility.

However, they chose to ignore the vulnerability at existing reactors and instead focused on fixing the issue in future ones, government and corporate documents show. There was no serious discussion of retrofitting older plants with the alternative technology

I guess the "vulnerability reports" just weren't peer reviewed, therefore meaningless - or not reviewed by the "right" peers.

This sounds like our own FDA, ONC office and Institute of Medicine (via the Committee on Patient Safety and Health Information Technology), "choosing to ignore" health IT "vulnerabilities" (such as the aforementioned) and focusing on future issues such as comparative effectiveness research, "the common good", etc. instead.

I call this attitude "reckless endangerment" and hope plaintiff attorneys are paying close attention.

-- SS

"Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2

At my Mar. 15, 2011 post "What does a "problem list" of typical health IT look like?" I displayed a chart by Jon Patrick at U. Sydney on the ill-suited nature of an American EHR system for use in Emergency Dept. settings. That system had been mandated for rollout in public ED's in the entire Australian state of New South Wales.

As it turns out, that chart was just preliminary.

A new chart is up entitled "Analysis of Problems Defined by ED Directors", divided into four sections:

1. Workarounds and Abandonments (27 elements)
2. Functions Lost from the Pre-FirstNet System or Desirable Functions (31 elements)
3. Processes with Added Risk to the Integrity of the EMR (11 elements)
4. General Problem List - What is the Potential for Resolution? (60 elements)

The entire chart can be viewed at this link (best with browsers other than MS Internet Explorer): http://sydney.edu.au/engineering/it/~hitru/index.php?option=com_content&task=view&id=120&Itemid=116 .

It is lengthy, detailed and - stunning.

Prof. Patrick concludes (emphases mine):

Reviewing this compendium of difficulties and obstacles created for staff makes it entirely unsurprising that the patient throughput of most EDs dropped by 50% on the day FirstNet was introduced and now some years later throughputs are only just beginning to recover as staff have been able to instigate work practices to minimise the worst aspects of the system.

[A spectacular waste of clinician energy - ed.]

The workarounds and abandonments give an expression of the frustration of staff and their strategy for retaining equilibrium in their work practices despite FirstNet's presence. In a number of cases we have seen the practice guidelines of NSWHealth surrendered by the imperatives of the technology with the imposition of the HSS. It is astounding that practices defined from years of clinical experience can be discarded so whimsically.

[As I have written, the IT industry has invaded the healthcare sector, and this is the absolutely non-whimsical result - ed.]

Fortunately, in the case of one pathology laboratory, patient safety was put ahead of the technological imperative lest it jeopardise the registration of the laboratory. The described function-losses with FirstNet compared to the pre-FirstNet systems, and the functionality needs expressed by the staff indicate that they are acutely aware of the value of good technology and have a strong desire to be equipped with something that works properly without creating unacceptable risks to patients and a draconian reduction in their efficiency. The risks posed by the system to maintaining the integrity of the medical record is something that staff are acutely concerned about as they feel it fails to fulfil their legal obligations.

Emergency Departments are too important to have to endure these stressful and unproductive conditions.

[This is a first principle, as as such is not open to debate - ed.]

It is time that the knowledge and experience of the Directors and their staff were listened to and taken seriously [actually, six decades or so into the "computer revolution", I'd say that time was long ago - ed.] for the sake of improving our hospitals's use of technology. After all we have to ask: What business would commit to an interloping "integrated" system whose services are being necessarily dismembered piecemeal as a matter of survival by the users? This is a system whose pieces are not used by the staff, but rather are shadow mirrored by them, not for redundancy but primacy.

Who would want a system that is progressively de-activated by the staff to overcome the hazards and operational inefficiencies it has introduced?

[My answer: those who profit handsomely, and at no liability to themselves, from this arrangement. I leave it to the reader to decide who might fit in that category - ed.]

As a physician and medical informatics specialist myself, I would not want my ED care or that of my family interfered with by such IT.

The interference in care of such systems already nearly killed my relative in 2010, and may yet succeed in doing so. She is hospitalized and in extremis once more as of this day.

Several questions:

• How did a government for an entire state of a major country come to allow themselves to believe an EHR system such as this would improve conditions in the most mission critical section of their hospitals, the ED's?

• What testing and validation of the software was done by officials and representatives of said government, and who were they, exactly?

• What experience and background did the validators possess?

• How were clinician complaints during implementation, which has apparently been underway for several years now, handled?

• What other countries are going down the same path?

• Why is not all health IT subject to the same type of government regulator-led validation as this system was put under by a private academic researcher? (Note that the U.S., pharma IT validation is led by the FDA, but that same agency has essentially shied away from healthcare IT validation.)
  

• Would a country buy software as ill suited to purpose for, say, mitigating disaster risk in their nuclear power facilities?
  

Finally, I ask:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

I opine that researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

-- SS

"Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2

At my Mar. 15, 2011 post "What does a "problem list" of typical health IT look like?" I displayed a chart by Jon Patrick at U. Sydney on the ill-suited nature of an American EHR system for use in Emergency Dept. settings. That system had been mandated for rollout in public ED's in the entire Australian state of New South Wales.

As it turns out, that chart was just preliminary.

A new chart is up entitled "Analysis of Problems Defined by ED Directors", divided into four sections:

1. Workarounds and Abandonments (27 elements)
2. Functions Lost from the Pre-FirstNet System or Desirable Functions (31 elements)
3. Processes with Added Risk to the Integrity of the EMR (11 elements)
4. General Problem List - What is the Potential for Resolution? (60 elements)

The entire chart can be viewed at this link (best with browsers other than MS Internet Explorer): http://sydney.edu.au/engineering/it/~hitru/index.php?option=com_content&task=view&id=120&Itemid=116 .

It is lengthy, detailed and - stunning.

Prof. Patrick concludes (emphases mine):

Reviewing this compendium of difficulties and obstacles created for staff makes it entirely unsurprising that the patient throughput of most EDs dropped by 50% on the day FirstNet was introduced and now some years later throughputs are only just beginning to recover as staff have been able to instigate work practices to minimise the worst aspects of the system.

[A spectacular waste of clinician energy - ed.]

The workarounds and abandonments give an expression of the frustration of staff and their strategy for retaining equilibrium in their work practices despite FirstNet's presence. In a number of cases we have seen the practice guidelines of NSWHealth surrendered by the imperatives of the technology with the imposition of the HSS. It is astounding that practices defined from years of clinical experience can be discarded so whimsically.

[As I have written, the IT industry has invaded the healthcare sector, and this is the absolutely non-whimsical result - ed.]

Fortunately, in the case of one pathology laboratory, patient safety was put ahead of the technological imperative lest it jeopardise the registration of the laboratory. The described function-losses with FirstNet compared to the pre-FirstNet systems, and the functionality needs expressed by the staff indicate that they are acutely aware of the value of good technology and have a strong desire to be equipped with something that works properly without creating unacceptable risks to patients and a draconian reduction in their efficiency. The risks posed by the system to maintaining the integrity of the medical record is something that staff are acutely concerned about as they feel it fails to fulfil their legal obligations.

Emergency Departments are too important to have to endure these stressful and unproductive conditions.

[This is a first principle, as as such is not open to debate - ed.]

It is time that the knowledge and experience of the Directors and their staff were listened to and taken seriously [actually, six decades or so into the "computer revolution", I'd say that time was long ago - ed.] for the sake of improving our hospitals's use of technology. After all we have to ask: What business would commit to an interloping "integrated" system whose services are being necessarily dismembered piecemeal as a matter of survival by the users? This is a system whose pieces are not used by the staff, but rather are shadow mirrored by them, not for redundancy but primacy.

Who would want a system that is progressively de-activated by the staff to overcome the hazards and operational inefficiencies it has introduced?

[My answer: those who profit handsomely, and at no liability to themselves, from this arrangement. I leave it to the reader to decide who might fit in that category - ed.]

As a physician and medical informatics specialist myself, I would not want my ED care or that of my family interfered with by such IT.

The interference in care of such systems already nearly killed my relative in 2010, and may yet succeed in doing so. She is hospitalized and in extremis once more as of this day.

Several questions:

• How did a government for an entire state of a major country come to allow themselves to believe an EHR system such as this would improve conditions in the most mission critical section of their hospitals, the ED's?

• What testing and validation of the software was done by officials and representatives of said government, and who were they, exactly?

• What experience and background did the validators possess?

• How were clinician complaints during implementation, which has apparently been underway for several years now, handled?

• What other countries are going down the same path?

• Why is not all health IT subject to the same type of government regulator-led validation as this system was put under by a private academic researcher? (Note that the U.S., pharma IT validation is led by the FDA, but that same agency has essentially shied away from healthcare IT validation.)
  

• Would a country buy software as ill suited to purpose for, say, mitigating disaster risk in their nuclear power facilities?
  

Finally, I ask:

If the purpose of Medical Informatics is the improvement of healthcare (as opposed to career advancement of a small number of academics through publishing obscure articles about HIT benefits while ignoring downsides in rarified, echo-chamber peer reviewed journals), then:

• Who are the "real" medical informatics specialists, and;
  

• Who are the poseurs?

I opine that researchers like Jon Patrick who address real-world issues of great import to patients on HIT risks, and further go public on the web with their work without the full blessings of some dusty journal (and those like Ross Koppel who also directly address the downsides, and others who make available to the public material such as on blogs like this and this, papers like this and sites like this) are the former.

Those who deem only "peer reviewed" articles worthy of daylight, and everything else - especially and particularly reports of downsides - "anecdotal" (the anecdotalists) are the latter.

-- SS

What does a "problem list" of typical health IT look like?

Mar. 20, 2011 update: see the much more extensive charts via the link at my followup post "Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2.

What does a high level "problem list" of typical health IT look like?

Like this, from Prof. Jon Patrick at U. Sydney:

This Technology Problem List has been drawn up to collate all the technical problems in the report that have been identified in the Cerner Millenium version used for the FirstNet installations in Emergency Departments in the NSW, Australia.

[Link, as I wrote about in my Mar. 2011 post "On an EMR Forensic Evaluation by Professor Jon Patrick from Down Under: More Thoughts" and others - ed.]

This is a list of the errors and bugs identified by ED directors in the study published on the 4th March 2011 that seem to have a technical origin, that is we would expect that some change in the underlying technology would solve the problem. The likelihood of resolving the problem is a function of what part of the software the problem is created at. The explanations and dates presented here are postulated based on an interpretation of the likely cause for the fault drawn from the report. I'm happy to be informed whenever these problems are corrected or if new problems arise.

	Problem Description	Possible Explanation	Publication Date	Possible Correction Date
1	Problems arise when the documents are saved but not signed. Documents may become difficult to find even for the experts. The First Net trainers state “Never use the save button”. Cerner was asked to remove the save button. However, the answer was that it could not be done.	Button is built into the underlying software	4-03-2011	Never
2	If the record is unsigned and the doctor walks away to an emergency leaving the data on the screen. The unsigned data can be cancelled by the next person who wants to use the computer.
3	The templates are not symptom generated and try to cover every topic and every specialty. Each template is like an index to a comprehensive textbook on socio-economics combined with medicine. You scroll through a list of demographic information, social information, signs and symptoms. As with other templates, you circle or backslash to confirm or deny. However, when this information is converted to prose, it is a series of disconnected phrases that do not provide a grammatically correct easy to read document.	Built into underlying software	4-03-2011	Never
4	The FirstNet templates are a list of words. There are no diagrams although it is possible to find diagrams in another part of FirstNet. The templates do not flow according to usual work practice. They do not act as an aid to memory. They are not user friendly. If a junior doctor tries to use the template, it can take hours to document the record of a single patient.	Design not matched to user requirements. Built into software	4-03-2011	Never
5	Documentation Screen. To add a note click on the note icon and a window opens into which the note can be written. The note is then shown on the screen. Subsequently if you click on the text of the note it then disappears entirely. To add to the note you have to do a right click on the notes icon and then select Open to open the note. The delete process can only be reversed by moving to the top menu bar and selecting Edit and then Undo and without this knowledge new users are lead to the belief that their work has just disappeared and is irrecoverable.	Built into software	4-03-2011	Never
6	1. Triage box allows about 2.5 lines of text or ~50 words. Choosing from the triage problem list of 51 items is not compulsory nor even close to being comprehensive so staff workaround by writing their own entries. The entries are free text and thus not part of a searchable “reason for presentation” database field. 2. Most staff write in a text description of the problem as the problem list is defective with poor wording, "bleed haematuria" or unfamiliar descriptions "orbital disorder" and lacks common usage of terminology.	Configuration of problems list	4-03-2011	4-03-2013
7	The database search for diagnosis or problem requires an exact match and does not assist with finding synonyms or approximate matches not found because of spelling mistakes.	Built into the software	4-03-2011	Never
8	A referral letter needed to be re-sent to a different provider after the weekend. Staff went to the Discharge letter, ticked the box "Correct", and it was then noted that they could make changes to the original letter and save them to the system without needing to make any identification of self as editing the letter. The letter was saved under the name of the original authoring doctor and no record is made that it was the work of a different author or the extent to which it was changed. This means that the EMR does not constitute a valid legal record.	Built into the software	4-03-2011	Never
9	All notes on a patient cannot be seen in one page. In 'Clinical notes', when nurses fill out 'continuation notes', they should read old notes from 'clinical notes' and then see 'pathway' but they have to move to other pages to find this material so it is inefficient.	Built into software	4-03-2011	Never
10	Clinical staff have to sign in to confirm arranging a test for patients, at which time the user name cannot be changed. If one doctor signed on to one workstation and then went away to take care of patient, and during that time, another doctor has come to the workstation and signed the previous doctor off all his work will be lost.	Built into software	4-03-2011	Never
11	There is no button to add new clinical notes. When reviewing 'continuation notes', there is no button to add a new one. It needs to be done in another place, so it is inefficient.	Built into the software	4-03-2011	Never
12	Functionalities for clinical staff are of limited use. For example: From the list of medical pre-designed documents generated for the patient's electronic record, the index is not clear. Staff have to scroll through an index which has every document in the AHS from across many disciplines. For example, ED staff can choose Botox clinic documents. Staff are required to scroll through hundreds of documents one by one to identify what document they want to use.	Built into the software	4-03-2011	Never
13	Once they have identified the document they want to use such as 'ED discharge summary', there are too many clicks to complete the form, and the process is unnecessarily complicated	Built into the software	4-03-2011	Never
14	Before seeing a patient, a staff member goes to the ‘To be seen’ tab, allocates themselves to the patient. The KPI package pops up and staff insert the ‘seen by time’. Ideally it would be good if there was an arrival time default inserted into this time as a prompt for staff.	Configuration requirement	4-03-2011	4-03-2013
15	The Dr sees the patient and then returns to the computer and documents the interview. If they document in FirstNet when they record the patient’s chart they have multiple clicks to open the chart before they can start documenting.	Built into the software	4-03-2011	Never
16	The Dr can enter a diagnosis via the documentation / diagnosis (chart) or depart process. A diagnosis is nine clicks to enter.	Built into the software	4-03-2011	Never
17	A diagnosis is not flagged or seen on the tracking list so other staff need to search through the entire documentation to find it. This is time consuming as it is not possible to identify externally which note from a set of documents contains the diagnosis.	Configuration requirement	4-03-2011	4-03-2013
18	Dr “seen-time” needs to be on the tracking list (or LOS) since Dr “seen-time” is used to manage dept case load, and ensure triage categories get a balanced throughput.	Configuration requirement	4-03-2011	4-03-2013
19	FirstNet does not work well on differentiating practitioners and MOs, because both of them are assigned patients in the same place.	Configuration requirement	4-03-2011	4-03-2013
20	Unable to see previous observations - Lose ‘feel’ for patient as everything in silos.	Configuration requirement	4-03-2011	4-03-2013
21	1. Poor graphical representation - graphing not finite enough to be of use. 2. FirstNet has 'vital signs' functionality but the trends are not finite enough here the important thing for staff in ED is to see the trend. Successive results cannot be displayed together making it far more difficult to get a trend or sequence of processes.	Built into software	4-03-2011	Never
22	Staff will not search for trends because the system is too slow, making staff reluctant to open charts.	Engineering, Configuration and/or installation problem	4-03-2011	Uncertain
23	No prompts for unusual observations, e.g. diabetic BSL or any form of useful alerts.	Built into the software	4-03-2011	Never
24	Unnecessarily Difficult to add allergies: Staff think there are too many steps to record allergies and a complex and non-intuitive process. Adding an allergy is a 7 step process. Once the screen is opened it is difficult to work out 'how' to add the allergy. There is no rational flow to the data entry, nor are there prompts to guide the user. Staff are required to be trained or they would take a long time to work out the process.	Built into the software	4-03-2011	Never

This is an EHR slated for Emergency Departments, of all places, not the hemorrhoid clinic.

Do you want your emergency care mediated by such a device?

I don't.

As I wrote here regarding HIT vendor "hold harmless" clauses, I am aware of major healthcare organizations with "portfolios" of hundreds or thousands of issues and defects (often more "granular" than the above) awaiting remediation.

Worse, the CMIO's are struggling with cavalier bureaucracies who want the doctors even in critical care areas to live with the problems (I've faced that scenario personally), and with vendors who are not in a hurry to fix their products.

Some percentage of those "issues" and defects are deemed by their organizations to rise to the level of "may cause patient harm if uncorrected."

-- SS

What does a "problem list" of typical health IT look like?

Mar. 20, 2011 update: see the much more extensive charts via the link at my followup post "Real" Medical Informatics: What Does a Problem List of Typical Health IT Look Like, Part 2.

What does a high level "problem list" of typical health IT look like?

Like this, from Prof. Jon Patrick at U. Sydney:

This Technology Problem List has been drawn up to collate all the technical problems in the report that have been identified in the Cerner Millenium version used for the FirstNet installations in Emergency Departments in the NSW, Australia.

[Link, as I wrote about in my Mar. 2011 post "On an EMR Forensic Evaluation by Professor Jon Patrick from Down Under: More Thoughts" and others - ed.]

This is a list of the errors and bugs identified by ED directors in the study published on the 4th March 2011 that seem to have a technical origin, that is we would expect that some change in the underlying technology would solve the problem. The likelihood of resolving the problem is a function of what part of the software the problem is created at. The explanations and dates presented here are postulated based on an interpretation of the likely cause for the fault drawn from the report. I'm happy to be informed whenever these problems are corrected or if new problems arise.

	Problem Description	Possible Explanation	Publication Date	Possible Correction Date
1	Problems arise when the documents are saved but not signed. Documents may become difficult to find even for the experts. The First Net trainers state “Never use the save button”. Cerner was asked to remove the save button. However, the answer was that it could not be done.	Button is built into the underlying software	4-03-2011	Never
2	If the record is unsigned and the doctor walks away to an emergency leaving the data on the screen. The unsigned data can be cancelled by the next person who wants to use the computer.
3	The templates are not symptom generated and try to cover every topic and every specialty. Each template is like an index to a comprehensive textbook on socio-economics combined with medicine. You scroll through a list of demographic information, social information, signs and symptoms. As with other templates, you circle or backslash to confirm or deny. However, when this information is converted to prose, it is a series of disconnected phrases that do not provide a grammatically correct easy to read document.	Built into underlying software	4-03-2011	Never
4	The FirstNet templates are a list of words. There are no diagrams although it is possible to find diagrams in another part of FirstNet. The templates do not flow according to usual work practice. They do not act as an aid to memory. They are not user friendly. If a junior doctor tries to use the template, it can take hours to document the record of a single patient.	Design not matched to user requirements. Built into software	4-03-2011	Never
5	Documentation Screen. To add a note click on the note icon and a window opens into which the note can be written. The note is then shown on the screen. Subsequently if you click on the text of the note it then disappears entirely. To add to the note you have to do a right click on the notes icon and then select Open to open the note. The delete process can only be reversed by moving to the top menu bar and selecting Edit and then Undo and without this knowledge new users are lead to the belief that their work has just disappeared and is irrecoverable.	Built into software	4-03-2011	Never
6	1. Triage box allows about 2.5 lines of text or ~50 words. Choosing from the triage problem list of 51 items is not compulsory nor even close to being comprehensive so staff workaround by writing their own entries. The entries are free text and thus not part of a searchable “reason for presentation” database field. 2. Most staff write in a text description of the problem as the problem list is defective with poor wording, "bleed haematuria" or unfamiliar descriptions "orbital disorder" and lacks common usage of terminology.	Configuration of problems list	4-03-2011	4-03-2013
7	The database search for diagnosis or problem requires an exact match and does not assist with finding synonyms or approximate matches not found because of spelling mistakes.	Built into the software	4-03-2011	Never
8	A referral letter needed to be re-sent to a different provider after the weekend. Staff went to the Discharge letter, ticked the box "Correct", and it was then noted that they could make changes to the original letter and save them to the system without needing to make any identification of self as editing the letter. The letter was saved under the name of the original authoring doctor and no record is made that it was the work of a different author or the extent to which it was changed. This means that the EMR does not constitute a valid legal record.	Built into the software	4-03-2011	Never
9	All notes on a patient cannot be seen in one page. In 'Clinical notes', when nurses fill out 'continuation notes', they should read old notes from 'clinical notes' and then see 'pathway' but they have to move to other pages to find this material so it is inefficient.	Built into software	4-03-2011	Never
10	Clinical staff have to sign in to confirm arranging a test for patients, at which time the user name cannot be changed. If one doctor signed on to one workstation and then went away to take care of patient, and during that time, another doctor has come to the workstation and signed the previous doctor off all his work will be lost.	Built into software	4-03-2011	Never
11	There is no button to add new clinical notes. When reviewing 'continuation notes', there is no button to add a new one. It needs to be done in another place, so it is inefficient.	Built into the software	4-03-2011	Never
12	Functionalities for clinical staff are of limited use. For example: From the list of medical pre-designed documents generated for the patient's electronic record, the index is not clear. Staff have to scroll through an index which has every document in the AHS from across many disciplines. For example, ED staff can choose Botox clinic documents. Staff are required to scroll through hundreds of documents one by one to identify what document they want to use.	Built into the software	4-03-2011	Never
13	Once they have identified the document they want to use such as 'ED discharge summary', there are too many clicks to complete the form, and the process is unnecessarily complicated	Built into the software	4-03-2011	Never
14	Before seeing a patient, a staff member goes to the ‘To be seen’ tab, allocates themselves to the patient. The KPI package pops up and staff insert the ‘seen by time’. Ideally it would be good if there was an arrival time default inserted into this time as a prompt for staff.	Configuration requirement	4-03-2011	4-03-2013
15	The Dr sees the patient and then returns to the computer and documents the interview. If they document in FirstNet when they record the patient’s chart they have multiple clicks to open the chart before they can start documenting.	Built into the software	4-03-2011	Never
16	The Dr can enter a diagnosis via the documentation / diagnosis (chart) or depart process. A diagnosis is nine clicks to enter.	Built into the software	4-03-2011	Never
17	A diagnosis is not flagged or seen on the tracking list so other staff need to search through the entire documentation to find it. This is time consuming as it is not possible to identify externally which note from a set of documents contains the diagnosis.	Configuration requirement	4-03-2011	4-03-2013
18	Dr “seen-time” needs to be on the tracking list (or LOS) since Dr “seen-time” is used to manage dept case load, and ensure triage categories get a balanced throughput.	Configuration requirement	4-03-2011	4-03-2013
19	FirstNet does not work well on differentiating practitioners and MOs, because both of them are assigned patients in the same place.	Configuration requirement	4-03-2011	4-03-2013
20	Unable to see previous observations - Lose ‘feel’ for patient as everything in silos.	Configuration requirement	4-03-2011	4-03-2013
21	1. Poor graphical representation - graphing not finite enough to be of use. 2. FirstNet has 'vital signs' functionality but the trends are not finite enough here the important thing for staff in ED is to see the trend. Successive results cannot be displayed together making it far more difficult to get a trend or sequence of processes.	Built into software	4-03-2011	Never
22	Staff will not search for trends because the system is too slow, making staff reluctant to open charts.	Engineering, Configuration and/or installation problem	4-03-2011	Uncertain
23	No prompts for unusual observations, e.g. diabetic BSL or any form of useful alerts.	Built into the software	4-03-2011	Never
24	Unnecessarily Difficult to add allergies: Staff think there are too many steps to record allergies and a complex and non-intuitive process. Adding an allergy is a 7 step process. Once the screen is opened it is difficult to work out 'how' to add the allergy. There is no rational flow to the data entry, nor are there prompts to guide the user. Staff are required to be trained or they would take a long time to work out the process.	Built into the software	4-03-2011	Never

This is an EHR slated for Emergency Departments, of all places, not the hemorrhoid clinic.

Do you want your emergency care mediated by such a device?

I don't.

As I wrote here regarding HIT vendor "hold harmless" clauses, I am aware of major healthcare organizations with "portfolios" of hundreds or thousands of issues and defects (often more "granular" than the above) awaiting remediation.

Worse, the CMIO's are struggling with cavalier bureaucracies who want the doctors even in critical care areas to live with the problems (I've faced that scenario personally), and with vendors who are not in a hurry to fix their products.

Some percentage of those "issues" and defects are deemed by their organizations to rise to the level of "may cause patient harm if uncorrected."

-- SS