Monthly Archives: July 2016

Root Cause Analysis - Incident Investigation

911 Outage in Baltimore

By Kim Smiley

Nobody ever wants to find themselves in the position of dialing 911.  But imagine how quickly a bad situation could get even worse if nobody answered your call for emergency help.  That is exactly what happened on July 16, 2016 to people in Baltimore, Maryland.  For about two hours, people dialing 911 in Baltimore got a busy signal.

This incident can be investigated by building a Cause Map, a visual root cause analysis.  A Cause Map intuitively lays out the many causes that contributed to an issue to show all the cause-and-effect relationships.  By focusing on the multiple causes, rather than a single root cause analysis, the range of solutions considered is naturally widened.

The first step in the Cause Mapping process is to fill in an Outline with the basic background information for the incident.  Additionally, the Outline is used to capture how the incident impacts the overall goals.  This incident, like most incidents, impacted more than one goal.  For example, the safety goal is impacted because of the delay in emergency help and the customer service goal is impacted because people were unable to reach 911 operators.

The bottom line on the Outline is used to note the frequency of similar incidents.  This is important because an incident that has occurred 12 times before may warrant a different level of investigation than an isolated incident.  For this example, newspapers reported a previous 911 outage in June in the Baltimore area. The outages appear to have been caused by different issues, but do raise questions about the overall stability of the 911 system in Baltimore. Investigators should determine if the multiple outages are related and indicative of bigger issues than just this one incident.

Once the Outline is completed, the Cause Map itself is built by asking “why” questions.  So why was there a 911 outage for about 2 hours? Newspapers have reported that the outage occurred because of electrical power failures after both the main and back-up power systems shut down.  The power systems shut down because of a malfunctioning air conditioning unit.  No details have been released about exactly why the air conditioning units malfunctioned, but additional information could quickly be added to the Cause Map as it becomes known.

The final step in the Cause Mapping process is to develop and implement solutions to reduce the risk of the problem reoccurring. The investigation into this incident is still ongoing and no information about potential long-term solutions has been announced. In the short term, callers were asked to dial 311 or call their closest fire station or police district station if they heard a busy signal or were otherwise unable to get through to 911.  It is probably not a bad idea for all of us to have the numbers of our local fire and police stations on hand, just in case.

Root Cause Analysis - Incident Investigation

Train Derails on Track Just Inspected

By ThinkReliability Staff

A train derailment in the Columbia River Gorge near Mosier, Oregon resulted in a fire that burned for 14 hours. The Federal Railroad Administration (FRA) preliminary investigation says the June 3rd derailment was caused by a broken lag bolt which allowed the track to spread, resulting in the 16-car derailment. Although there is only one other known instance of a broken lag bolt causing a train derailment, the FRA determined that the bolt had been damaged for some time, and had been inspected within days of the incident, raising questions about the effectiveness of these inspections.

Determining all the causes of a complex issue such as a train derailment can be difficult, but doing so will provide the widest selection of possible solutions. A Cause Map, or visual root cause analysis, addresses all aspects of the issue by developing cause-and-effect relationships for all the causes based on the impacts to an organization’s goals. We can create a Cause Map based on the preliminary investigation. Additional causes and evidence can be added to the map as more detail is known.

The first step in the Cause Mapping process is to determine the impacts to the organization’s goals. While there were no injuries in this case, the massive fire resulting from the derailment posed a significant risk to responders and nearby citizens, an impact to the safety goal. The release of 42,000 gallons of oil (although much of it was burned off in the fire) is an impact to the environmental goal. The customer service goal is impacted by the evacuation of at least 50 homes and the regulatory goal is impacted by the potential for penalties, although the National Transportation Safety Board (NTSB) has said it will not investigate the incident. The state of Oregon has requested a halt on oil traffic, which would be an impact to the schedule goal. The property goal is impacted by the damage to the train cars, and the labor/ time goal is impacted by the response and investigation.

The analysis, which is the second step in the Cause Mapping process, begins with one of the impacted goals and develops cause-and-effect relationships by asking ‘Why’ questions. In this case, the safety goal is impacted by the high potential for injuries. This is caused by the massive fire, which burned for 14 hours. There may be more than one cause resulting in an effect, such as a fire, which is caused by heat, fuel, and oxygen. The oxygen in this case is from the atmosphere. The heat source is unknown but could have been a spark caused by the train derailment. The fire was fueled by the 42,000 gallons of crude released due to damage to train cars, which were transporting crude from the Bakken oil fields, caused by the derailment.

The derailment of 16 cars of the train was caused by the broken lag bolt. Any mechanical failure, such as a break, results from the stress on that object exceeding the strength of the object. In this case, the stress was caused by the weight of the 94-car train. The length of a train carrying crude oil is not limited by federal regulations. The strength of the bolts was reduced due to previous damage, which was not identified prior to the failure. While the track strength is evaluated every 18 months by the Gauge Restraint Measurement System (GRMS), it did not identify the damage. It’s unclear the last time it was performed.

Additionally, although the track is visually inspected twice a week by the railroad, it is done by vehicle, which would have made the damage harder to spot. The FRA does not require walking inspections. Nor does the FRA inspect or review the railroad’s inspections very often – there are less than 100 inspectors for the 140,000 miles of track across the country. There are only 3 in Oregon.

As a result of the derailment, the railroad has committed to replacing the existing bolts with heavy-duty ones, performing GRMS four times a year, enhanced hyrail inspections and visual track inspections three times a week, and performing walking inspections on lag curves monthly.

The FRA is still evaluating actions against the railroad and is again calling for the installation of advanced electronic brakes, or positive train control (PTC). It has also recommended PTC after other incidents, such as the deaths of two railroad workers on April 3 (see our previous blog) and the derailment in Philadelphia last year that killed 8 (see our previous blog).

To view a one-page PDF of the Cause Mapping investigation, click on “Download PDF” above. Or, click here to read the FRA’s preliminary investigation.

Root Cause Analysis - Incident Investigation

FAA Proposes Amazon Fine for Hazardous Shipment

By Kim Smiley

The Federal Aviation Administration (FAA) recently proposed fining Amazon $350,000 for shipping a product that allegedly violated hazardous materials regulations. The package in question was shipped by Amazon from Louisville, Kentucky, to Boulder, Colorado and contained a one-gallon container of corrosive drain cleaner with the colorful name Amazing! LIQUID FIRE. During transit, the package leaked and 9 UPS workers were exposed to the drain cleaner and reported a burning sensation. The workers were treated with a chemical wash and experienced no further issues, but this incident highlights issues with improper shipment of hazardous materials.

A Cause Map, a visual root cause analysis, can be built to analyze this issue by visually laying out the cause-and-effect relationships that contributed to the issue.  The first step in the Cause Mapping method is to fill in an Outline.  The top part of the Outline lists the basic background information for the issue, such as the date and time.  The bottom portion of the Outline has a section to list how the problem impacts the overall goals of the organization.  Most problems have more than one impact and this incident is no exception.  For example, the safety goal is impacted because workers were exposed to hazardous chemicals and the regulatory goal is impacted because of the FAA investigation and the proposed fine.

The frequency of the issue is listed on the last line of the Outline.  Identifying the frequency is important because an issue that has occurred a dozen times may likely warrant a more detailed investigation than an issue that has been reported only once.  For this example, Amazon has had at least 24 hazardous materials violations between February 2013 and September 2015 so the concerns about improperly handling hazardous materials goes beyond the issues with this one package.

Once the Outline is completed, the Cause Map is built by starting at one of the impacted goals and asking “why” questions. Starting at the safety goal for this example, the first question would be “why were workers exposed to hazardous chemicals?”. This happened because the workers were handling a package containing hazardous chemicals, a package containing hazardous chemicals leaked, and inadequate precautions were taken to prevent the workers being exposed to the chemicals. When there is more than one cause that contributes to an effect, the causes are listed vertically and separated by an “and”.

To continue building the Cause Map, ask “why” questions for each of the causes already listed. The workers were handling the package because it shipped by air via UPS. Inadequate precautions were taken to prevent exposure to the chemical because workers were unaware that package contained hazardous chemicals. Chemicals leaked because they were not properly packaged.  Why questions should continue to be asked until no more information is known or no useful detail can be added to the Cause Map. To view an intermediate level Cause Map of this issue with more information, click on “Download PDF” above.

The final step in the Cause Mapping process is to use the Cause Map to develop and implement solutions to reduce the risk of the problem reoccurring. More information about what exactly led to improperly packaged and labeled hazardous materials being shipped would be needed to develop useful solutions in this example, but hopefully a fine of this size and the negative publicity it generated will help spark efforts to make improvements.