NTSB Safety Recommendation P-05-004

NTSB safety recommendation P-05-004.

TO THE PIPELINE AND HAZARDOUS MATERIALS SAFETY ADMINISTRATION: Change the liquid accident reporting form (PHMSA F 7000-1) and require operators to provide data related to controller fatigue.

Priority: CLASS II

Overall Status: Closed - Acceptable Action

Issued Date: 2005-12-23

Adopted Date: 2005-11-29

Overall Date Closed: 2010-04-06

Synopsis: In the pipeline industry, Supervisory Control and Data Acquisition (SCADA) systems are used to collect data from pipeline sensors in real time and display these data to humans who monitor the data from remote sites and remotely operate pipeline control equipment. This study was designed to examine how pipeline companies use SCADA systems to monitor and record operating data and to evaluate the role of SCADA systems in leak detection. The number of hazardous liquid accidents investigated by the National Transportation Safety Board in which leaks went undetected after indications of a leak on the SCADA interface was the impetus for this study. The Safety Board developed a survey to obtain data about the liquid pipeline industry’s use of SCADA systems with input from industry. In addition to obtaining survey data, the Safety Board visited 12 pipeline companies that had operating SCADA systems. Based on information from previous accidents investigated by the Board, survey results, and site visit results, the Safety Board’s review of SCADA systems in the hazardous liquid pipeline industry uncovered five areas for potential improvement: display graphics, alarm management, controller training, controller fatigue, and leak detection systems.

Keywords: Fatigue

Ntsbnumber: 80595

Report Number: SS-05-02

Addressee Name: PHMSA

Addressee Status: Closed - Acceptable Action

Addressee Date Closed: 2010-04-06

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2008-12-23

Communication Type: NPRM Response

Communication Contents: PHMSA states in the NPRh4 that it plans to address Safety Recommendation P-05-4 in a separate action. PHMSA published a notice at 73 Federal Register 5 1697 on September 4, 2008, requesting comments on its intention to revise the incident report forms for gas transmission and gathering systems, gas distribution systems, and hazardous liquid pipeline systems. Additionally, in a letter addressed to the Safety Board dated September 17, 2008, regarding Safety Recommendation P-05-5, PHMSA summarized its policies and requirements for leak detection systems. With the letter, PHMSA enclosed a copy of its report to congress3 addressing leak detection systems and the development of more effective technologies. The report, which was mandated under the PIPES Act, was forwarded to Congress on June 23, 2008. The Board will address PHMSAYs actions on Safety Recommendations P-05-4 and -5 in separate correspondence and encourages PHMSA to complete action on these recommendations promptly. Notation 8070: The National Transportation Safety Board has reviewed the Pipeline and Hazardous Materials Safety Administration’s (PHMSA’s) notice of proposed rulemaking (NPRM), “Control Room Management/Human Factors,” that was published at 73 Federal Register 53076 on September 12, 2008. PHMSA is proposing to revise the Federal pipeline safety regulations, 49 Code of Federal Regulations (CFR) Parts 192, 193, and 195, to address human factors and other components of control room management. Overview The Safety Board notes that the NPRM would implement new requirements for control room management at 49 CFR Parts 192, 193, and 195 for natural gas pipelines, liquefied natural gas (LNG) facilities, and hazardous liquid pipelines, respectively. Under the proposed requirements, each operator of a natural gas pipeline, LNG facility, or hazardous liquid pipeline with a controller and control room would have to establish and follow written control room management procedures. The procedures also would have to be integrated into the operator’s operations and maintenance manual, qualification program, and emergency plan, all of which are currently required under 49 CFR Parts 192, 193, and 195. The Safety Board further notes that in order to implement the proposed control room management procedures an operator would be required to do the following: define the roles and responsibilities of the controllers; provide controllers with accurate and timely system data; implement methods to prevent controller fatigue; ensure appropriate controller response to alarms and notifications when Supervisory Control and Data Acquisition (SCADA) systems are used; establish thorough and frequent communications among controllers, management, and field personnel when planning and implementing physical changes to pipeline/facility equipment and configuration; review control room procedures following any event that must be reported under existing regulations; establish and implement a training program that includes an annual review to identify potential improvements; have a qualification program for controllers; and conduct an annual validation by a senior executive for the operator verifying that adequate control room management procedures have been implemented and are being followed. The Safety Board supports the overall direction of the NPRM and believes that the NPRM is comprehensive and focuses on the major elements of effective control room management. The Board also commends PHMSA for proposing to apply the standards to both natural gas and hazardous liquid pipelines and LNG facilities. Still, the Board has more detailed comments to provide concerning the specific safety recommendations discussed in the NPRM and the proposed requirements pertaining to mitigation of controller fatigue. Safety Board SCADA Safety Recommendations The Safety Board notes that the NPRM references the five safety recommendations, Safety Recommendations P-05-1 through -5, issued to PHMSA as a result of the Board’s 2005 safety study titled Supervisory Control and Data Acquisition (SCADA) in Liquid Pipelines. The NPRM specifically addresses Safety Recommendations P-05-1, -2, and -3, which, respectively, pertain to the use of graphics for SCADA systems, alarm management, and controller training. The three recommendations also were directly incorporated into the Pipeline Inspection, Protection, Enforcement, and Safety Act of 2006 (PIPES Act). These recommendations are listed below; a discussion of the related provisions in the NPRM follows. Require operators of hazardous liquid pipelines to follow the American Petroleum Institute’s Recommended Practice 1165 [API RP-1165] for the use of graphics on the SCADA screens. (P-05-1) The NPRM states that whenever a SCADA system is used, the operator must implement API RP-1165 in its entirety, unless the operator can adequately demonstrate that a provision of API RP-1165 is not applicable or is impracticable in the SCADA system used. Require pipeline companies to have a policy for the review/audit of alarms. (P-05-2) The NPRM, if implemented, would require that each operator using a SCADA system ensure appropriate controller response to alarms and notifications. Operators would be required to review SCADA operations at least once a week and review SCADA configuration and alarm management operations at least once each calendar year but at intervals not to exceed 15 months. Require controller training to include simulator or non-computerized simulations for controller recognition of abnormal operating conditions, in particular, leak events. (P-05-3) The NPRM proposed training provisions would require that training programs for controllers include the use of a simulator or noncomputerized (tabletop) method to enable controllers to recognize abnormal operating conditions, in particular leak and failure events. The Safety Board notes that under the NPRM, such simulations and tabletop exercises would include representative communications between controllers and individuals that operators expect to be involved during actual events. Further, controllers would also be required to participate in improving and developing tabletop or simulation training scenarios. The Safety Board believes that implementation of these proposed provisions will satisfy Safety Recommendations P-05-1, -2, and -3. Safety Recommendations P-05-4 and -5, the two remaining safety recommendations issued to PHMSA as a result of the SCADA study, are listed below. Change the liquid accident reporting form (PHMSA F 7000-1) and require operators to provide data related to controller fatigue. (P-05-4) Require operators to install computer-based leak detection systems on all lines unless engineering analysis determines that such a system is not necessary. (P-05-5) PHMSA states in the NPRM that it plans to address Safety Recommendation P-05-4 in a separate action. PHMSA published a notice at 73 Federal Register 51697 on September 4, 2008, requesting comments on its intention to revise the incident report forms for gas transmission and gathering systems, gas distribution systems, and hazardous liquid pipeline systems. Additionally, in a letter addressed to the Safety Board dated September 17, 2008, regarding Safety Recommendation P-05-5, PHMSA summarized its policies and requirements for leak detection systems. With the letter, PHMSA enclosed a copy of its report to Congress addressing leak detection systems and the development of more effective technologies. The report, which was mandated under the PIPES Act, was forwarded to Congress on June 23, 2008. The Board will address PHMSA’s actions on Safety Recommendations P-05-4 and -5 in separate correspondence and encourages PHMSA to complete action on these recommendations promptly. Controller Fatigue Mitigation Despite the many positive provisions included in the NPRM, the Safety Board does not believe the NPRM satisfactorily addresses m

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2010-04-02

Communication Type: Official Correspondence

Communication Contents: The NTSB is currently reviewing the action taken by PHMSA on the above recommendations, identified in your letter, and expects to have a more substantive response back to you shortly. We apologize for the delay in responding regarding some of the recommendations; however, additional documentation was needed for these before their evaluation for closure could be completed. Action on Safety Recommendation R-89-53 (see enclosure) was completed on March 19, 2010.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2010-04-06

Communication Type: Official Correspondence

Communication Contents: Change the liquid accident reporting form (PHMSA F 7000-1) and require operators to provide data related to controller fatigue. Because PHMSA developed and published a revised accident form (PHMSA F 7000-1), as requested and mandated its use for reporting accidents that occur after January 1, 2010, Safety Recommendation P-05-4 is classified CLOSED -- ACCEPTABLE ACTION.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2009-09-22

Communication Type: Official Correspondence

Communication Contents: The NTSB notes that PHMSA developed a draft accident form as requested, had it reviewed by industry and state pipeline partners, published the draft form for comment in the September 4, 2008, Federal Register, and anticipates having the new form in place in early 2010. Accordingly, pending publication of the revised form F-7000-1 and its required use by operators in providing data related to controller fatigue, Safety Recommendation P-05-4 is classified OPEN -- ACCEPTABLE RESPONSE.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2008-03-18

Communication Type: Official Correspondence

Communication Contents: The Safety Board appreciates PHMSA’s comprehensive overview of action either underway or planned for implementation of these recommendations. The Board has reviewed the requirements of the PIPES Act, which, when implemented, will satisfy Safety Recommendations P-05-1 through -4. Section 19 of the PIPES Act requires implementation of Safety Recommendations P-05-1 through -3 by June 1, 2008, and implementation of Section 20 of the Act will satisfy Safety Recommendation P-05-4, as it requires PHMSA to amend the accident report form by December 31, 2007. Section 21 of the PIPES Act, as stated below, requires PHMSA to submit a report on leak detection systems with discussion about what can be done to foster development of more effective technologies. The Safety Board reminds PHMSA that to satisfy Safety Recommendation P-05-5, it needs to require operators to install computer-based leak detection systems. Not later than December 31, 2007, the Secretary of Transportation shall submit to Congress a report on leak detection systems utilized by operators of hazardous liquid pipelines. The report shall include a discussion of the inadequacies of current leak detection systems, including their ability to detect ruptures and small leaks that are ongoing or intermittent, and what can be done to foster development of better technologies as well as address existing technology inadequacies. The Safety Board is aware that the public comment period for PHMSA’s report was extended through January 18, 2008, and that PHMSA is currently compiling these comments before submitting the report to Congress. Because PHMSA is working to address the recommendations, Safety Recommendations P-05-1 through -5 are classified OPEN -- ACCEPTABLE RESPONSE, pending the completion of these efforts.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2006-08-08

Communication Type: Official Correspondence

Communication Contents: The Safety Board understands that PHMSA is monitoring the American Petroleum Institute's (API's) completion of the Recommended Practice and that API will consider the Board's recommendation P-05-1 while evaluating the draft Recommended Practice. Further, PHMSA's Controller Certification (CCERT) Project Team is examining the impact of SCADA graphics on controllers, the impact of alarms on controllers, controller training issues, and the need for data to evaluate controller fatigue. PHMSA announced in the May 1, 2006, Federal Register a public workshop to discuss opportunities to enhance the safety of pipeline control and findings from the CCERT Project. The Safety Board understood that the workshop would also assist PHMSA in completing the CCERT Project's report and support API's continued work on API RP 1165. The Safety Board would appreciate an update on the outcome of the workshop and how PHMSA now plans to address these recommendations. Pending a further response from PHMSA on these initiatives, Safety Recommendations P-05-1 through -4 are classified OPEN -- ACCEPTABLE RESPONSE.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2007-07-31

Communication Type: Official Correspondence

Communication Contents: Letter Mail Controlled 8/8/2007 2:24:08 PM MC# 2070395 - From Stacey L. Gerard, Assistant Administrator/ Chief Safety Officer: PHMSA has completed our CCERT project, discussed above in response to P-99-12, which covered various human factor control room issues. In January 2007, PHMSA submitted a report to Congress on the project that identified several areas for enhancing safety including improved graphics on SCADA screens, alarms, and training. The Pipeline Inspection, Protection, Enforcement, and Safety Act of 2006 (Act) requires PHMSA to issue regulations by June 1, 2008 that require operators to use the American Petroleum Institute’s Recommended Practice 1165, to review and audit alarm systems, and to develop training standards that include the recognition of abnormal operating conditions. The Act also requires PHMSA to submit a report to Congress on leak detection systems used by operators of hazardous liquid pipelines by December 3 1,2007. We are actively working on this report and intend to meet this deadline. PHMSA plans to include control room issues in a "Prevention Through People" regulatory effort that will incorporate the concepts of integrity management programs into risk-based regulations addressing human factors. PHMSA has begun work on this regulatory effort and expects to complete it this year. It will address both the Congressional direction and NTSB recommendations on use of graphics, review of alarms, controlling training, and fatigue. On May 23,2007 PHMSA held a public workshop that addressed best practices in addressing fatigue, man-machine interface, and qualifications and training and we are currently reviewing the workshop data. PHMSA has also begun work on a project to add data elements to accident reporting forms to capture information regarding the impact of fatigue on safety. PHMSA has ccnsistent!y pushed the hazardous liquid operators through integrity management to shore up the adequacy of their leak detection capabilities.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2010-02-25

Communication Type: Official Correspondence

Communication Contents: Letter Mail Controlled 3/2/2010 1:54:51 PM MC# 2100074 - From Cynthia L. Quarterman, Administrator: To address this recommendation, PHMSA conducted extensive outreach with joint data team meetings consisting of Federal, State, and industry representatives in 2007 and continuing into 2008. We issued a Federal Register notice (73 FR 5 1697) on September 4, 2008 seeking comments and the Office of Management and Budget (OMB) approval for the paperwork collection burden. The proposed forms and corresponding instructions were posted in the docket for a 60-day comment period. Additional meetings were conducted with stakeholders in 2009 and PHMSA responded to concerns identified during the comment period. A final Federal Register notice (74 FR 41496) was published August 17, 2009 and OMB approved the new forms on January 13, 2010. PHMSA has implemented the new forms for all incidents and accidents occurring after January 1, 2010. Copies of the forms are enclosed with this letter and the corresponding Advisory Bulletin published in the Federal Register can be found at the following link http://edocket.access.gpo.gov/2100pdf/2010-2210.pdf. This recommendation was also a mandate contained in Section 20 of the Pipeline Inspection, Protection, Enforcement and Safety (PIPES) Act of 2006. PHMSA addressed this recommendation as well as other needed improvements to the natural gas incident and hazardous liquid accident forms. In addition to requiring operators to provide data related to controller fatigue, we expanded the information collection to provide detailed information about the material, location, circumstances, and cause of the incident, improved consistency across the forms, enhanced the form instructions, and the online navigation for ease of filing. PHMSA takes the safety recommendations from the NTSB seriously and are focused on fulfilling the requirements of these recommendations. We have implemented the changes from NTSB safety recommendation P-05-4 and therefore request the classification of recommendation P-05-4 be changed to Closed-Acceptable Action. If you have questions, concerns, or comments, please feel free to contact me at 202-366-4433. Regards, Cynthia L. Quarterman, Enclosure.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2006-04-26

Communication Type: Official Correspondence

Communication Contents: Letter Mail Controlled 4/27/2006 9:59:08 AM MC# 2060218 4-26-06: - From Stacey L. Gerard, Assistant Administrator/ Chief Safety Officer: PFMSA is monitoring the American Petroleum Institute's (API) completion of the Recommended Practice and API has informed PHMSA of its intention to evaluate the draft Recommended Practice in light of the NTSB recommendations. PHMSA's Controller Certification (CCERT) Project Team is examining the impact of SCADA graphics on controllers, the impact of alarms on controllers, controller training issues, and the need for data to evaluate controller fatigue. This week PHMSA intends to announce in the Federal Register a public workshop in June to discuss opportunities to enhance the safety of pipeline control and findings from the CCERT Project. The workshop is a forum to discuss the adequacy of current regulations, some new concepts for improvement of control room management and operations, and for industry to provide feedback on any enhancements PHMSA offers for discussion on the topic. PHMSA would like to hear from the public and industry its experiences related to pipeline management processes, human fatigue issues, controller qualification training and other programs to assure the effective control of pipelines. Among the topics on the agenda will be the following: "Alarm and event management; "Shift scheduling and hand-off; "Current qualification programs; and "Need to assess knowledge, skills, and abilities such as eyesight and hearing. The discussion at this workshop will also aid PHMSA in completing the CCERT Project's report and support API's continued work on API RP 1 165. This discussion will enable us to provide a more informed response to these recommendations at a later date. We request the classification of this response to recommendations P-05-1 through P-05-4 as "Open-Acceptable Action." - Stacey L. Gerard, Acting Associate Administrator/Chief Safety Officer; (202) 366-4433.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2008-12-08

Communication Type: Official Correspondence

Communication Contents: Letter Mail Controlled 12/10/2008 3:05:08 PM MC# 2080723 - From Rick Kowalewski, Acting Assistant Administrator/ Chief Safety Officer: Safety recommendation P-05-04 asked that PHMSA change the liquid accident reporting form (PHMSA F 7000-1) and require operators to provide data related to controller fatigue. PHMSA agrees with this recommendation and has developed a draft accident form that was reviewed by industry and State pipeline partners. PHMSA published the draft form for comment on September 4,2008 and anticipates having the new form in place in early 2009.

Addressee Acronym: PHMSA

Addressee Organization Type: G-Federal Government

Communication Date: 2010-03-15

Communication Type: Official Correspondence

Communication Contents: Letter Mail Controlled 3/16/2010 1:16:29 PM MC# 2100096 - From Cynthia L. Quarterman, Administrator: I greatly appreciated meeting with you last month concerning the National Transportation Safety Board's (NTSB) recommendations to the Pipeline and Hazardous Materials Safety Administration (PHMSA). I look forward to meeting with you again, along with staff, on Tuesday, March 16th. Your recommendations, based on lessons learned from accident investigations, provide valuable safety information to our programs. We are committed toconsidering each of the recommendations and implementing those that are positive improvements in safety. I have been meeting with PHMSA's hazmat and pipeline safety programs to assess our actions on the NTSB recommendations. We are addressing these safety issues by taking actions to assure that the "unacceptable actions" are moved into the "open-acceptable" category and to achieve a "closed-acceptable" in a timely manner on as many recommendations as possible. I recognize that a number of "open-acceptable" recommendations are works in progress and maytake a year or more to complete. You indicated your interest in getting these issues resolved as well. As you requested, I asked our pipeline and hazmat staff to compile a copy of outstanding letters to the NTSB that request a change in the classification of a recommendation. I have attached a copy of those letters and am hopeful you and I can successfully resolve a number of these issues. In addition, I have askedour Chief Safety Officer, Cindy Douglass, and our Associate Administrators, Jeff Wiese for Pipeline Safety and Magdy El-Sibaie for Hazardous Materials Safety, to meet with your staff to help us better understand each recommendation and to clarify the actions the NTSB considers necessary for closure. Again, I look forward to meeting with you on these safety concerns and believe that, together, we will make a positive difference in the safe transportation of hazardous materials, including those transported by pipelines.