Investigation into 999 computer system outage on New Year’s Day 2017

27 June 2017

The London Ambulance Service has today (27 June) published the findings of an investigation into the computer outage that affected the handling of 999 calls on New Year’s Day.

On 1 January 2017, the Service moved to using its manual back-up system for recording 999 calls and sending ambulance responses when a technical fault occurred in the early hours of New Year’s Day, the busiest time of the year for ambulance services across England.

Today’s report summarises the findings of a number of separate reviews into the causes of the outage itself, the impact on patient care, the adequacy of the Service’s back-up systems and the overall resilience of its IT function.

The cause of the incident was identified as a historic upgrade that, over time, led the system to become overloaded, slowing it down to the extent that it became unusable. The incident exposed gaps in the expertise and capacity within the Service’s IT function.

The investigation found that, due to the high volume of calls and the fact they were being handled manually, ambulance staff were delayed in reaching one patient who sadly died on New Year’s Day. Other patients, particularly those with non-life threatening conditions, waited longer for a response. Clinicians in the control room were able to identify patients whose condition started to deteriorate and upgrade these calls so they received a faster response.

Chief Executive Garrett Emmerson said: “This review exposed some significant short-comings in our IT processes and governance, which we have taken action to address. However, it also found that the computer system that we use to respond to 999 calls is fit for purpose.

“The system fault that occurred on New Year’s Day has been fixed and cannot now happen again. However, I would like to apologise to patients who had to wait longer than they otherwise would have for medical help on that morning, and pay tribute to our staff who responded incredibly well, ensuring patients continued to receive care during our busiest time of the year.”

Anne Rainsberry, NHS England’s Regional Director for London, said: “We are satisfied with the immediate actions that the London Ambulance Service took to rectify the cause of the outage, as well as the organisation’s assessment of the impact that this had on patients. The Service has made substantial progress against the investigation’s recommendations and over coming months we will seek assurance that the outstanding recommendations have been actioned.”

Steve Russell, NHS Improvement’s Executive Regional Managing Director for London, said: “London Ambulance Service has worked openly and constructively with its regulators, suppliers and staff to fully understand and fix the problems that caused its CAD system to fail on New Year’s Day. The Service and its staff have made significant headway in addressing the recommendations stemming from the comprehensive investigations into the failure. The fact it has made such progress against the challenging backdrop of responding to four major incidents in the past three months is a sign of the team's commitment to keeping the capital safe and healthy. We will continue to oversee and support their continuous improvement.”

Ends

Notes to editor:

Background:

  • At 12.30 am on New Year’s Day 2017, the London Ambulance Service’s call taking and dispatch system slowed down to a point it was no longer working effectively.
  • The Service moved to using a manual system of recording calls on paper and passing information to ambulance crews over the radio.
  • The Service continued to assess and prioritise the most ill and injured patients, while the problem was identified. Other ambulance services helped to take incoming 999 calls, and referred back the ones where patients needed an ambulance crew to be sent to patients.
  • During the early hours of New Year’s Day, 2,810 emergency calls were received. This compares with 2,517 in 2016.
  • The computer system was brought back up within 90 minutes but, due to the need to transfer paper records, it wasn’t fully operational until 5.15 am.
  • An investigation into what happened has now been completed.
  • It found that the reason the computer system slowed down and did not work effectively was because the recycle bin for the system’s database had not automatically emptied when it reached capacity. This fault has been fixed. It was identified that the recycle bin was not needed to operate the call handling system; it was therefore emptied and then disabled.
  • Since January the Service has strengthened its senior level IT expertise, appointing a new Chief Information Officer, Ross Fullerton, and Non-Executive Director, Sheila Doyle to provide better oversight and management of the call handling system and wider IT issues
  • Changes have been made to processes in the control room so that calls can be handled more effectively when using the manual system; for example, simplifying the process for recording call details, increasing numbers of fax machines for transferring call information, and identifying where people need to be in the room to speed up the information flow.

Serious incident relating to a patient:

  • The Service carried out a separate investigation into one patient who died on New Year’s Day.
  •  In the early hours of New Year’s Day, the Service categorised a 999 call as a Red 2, but due to the volume of high priority calls, an ambulance crew could not be dispatched straightaway.
  • When the initial information about the patient was received, they were conscious and breathing.
  • The call handler remained on the line with the caller to monitor the patient’s condition while an ambulance was allocated to attend.
  • At the point it was clear the patient’s condition had deteriorated, the call was upgraded and an ambulance crew was sent immediately. The crew arrived with the patient within 12 minutes of this decision being made. This was 48 minutes after the initial information about the patient was received. Sadly, the patient died.
  • Without knowledge of the cause of death, which is still to be determined by the coroner, it is difficult to determine what effect the delayed response had on the patient’s outcome.
  • The Service has shared its investigation report with the patient’s family and have met with them to discuss it in preparation for submitting the final report to the coroner.
  • The supporting review documents for this investigation are available on request.

A summary report is available here.

For further information about this news release or London Ambulance Service please contact the communications department on 020 7783 2286.