NTSB Safety Recommendation Letter P-11-024 through P-11-031

Page 1Official PDFNational Transportation Safety Board Washington, D.C. 20594 Safety Recommendation Date: September 26, 2011 In reply refer to: P-11-24 through -31 Mr. Christopher P. Johns President Pacific Gas and Electric Company 77 Beale Street San Francisco, California 94105 The National Transportation Safety Board (NTSB) is an independent Federal agency charged by Congress with investigating transportation accidents, determining their probable cause, and making recommendations to prevent similar accidents from occurring. We are providing the following information to urge your organization to take action on the safety recommendations in this letter. The NTSB is vitally interested in these recommendations because they are designed to prevent accidents and save lives. The recommendations are derived from the NTSB’s investigation of the September 9, 2010, San Bruno, California, pipeline accident and are consistent with the evidence we found and the analysis we performed. As a result of this investigation, the NTSB has issued 29 safety recommendations, 8 of which are addressed to the Pacific Gas and Electric Company (PG&E). Information supporting these recommendations is discussed below. The NTSB would appreciate a response from you within 90 days addressing the actions you have taken or intend to take to implement our recommendations. On September 9, 2010, about 6:11 p.m. Pacific daylight time, a 30-inch-diameter segment of an intrastate natural gas transmission pipeline known as Line 132, owned and operated by the PG&E, ruptured in a residential area in San Bruno, California. The rupture occurred at mile point 39.28 of Line 132, at the intersection of Earl Avenue and Glenview Drive. The rupture produced a crater about 72 feet long by 26 feet wide. The section of pipe that ruptured, which was about 28 feet long and weighed about 3,000 pounds, was found 100 feet south of the crater. PG&E estimated that 47.6 million standard cubic feet of natural gas was released. The released natural gas ignited, resulting in a fire that destroyed 38 homes and damaged 70. Eight people were killed, many were injured, and many more were evacuated from the area.1 < http://www.ntsb.gov/>. 1 For additional information, see Pacific Gas and Electric Company Natural Gas Transmission Pipeline Rupture and Fire, San Bruno, California, September 9, 2010, Pipeline Accident Report NTSB/PAR-11/01 (Washington, DC: National Transportation Safety Board, 2011), which is available on the NTSB website at 8275C#

Page 22 The NTSB determined that the probable cause of the accident was PG&E’s (1) inadequate quality assurance and quality control in 1956 during its Line 132 relocation project, which allowed the installation of a substandard and poorly welded pipe section with a visible seam weld flaw that, over time grew to a critical size, causing the pipeline to rupture during a pressure increase stemming from poorly planned electrical work at the Milpitas Terminal; and (2) inadequate pipeline integrity management program, which failed to detect and repair or remove the defective pipe section. Contributing to the accident were the California Public Utilities Commission’s (CPUC) and the U.S. Department of Transportation’s exemptions of existing pipelines from the regulatory requirement for pressure testing, which likely would have detected the installation defects. Also contributing to the accident was the CPUC’s failure to detect the inadequacies of PG&E’s pipeline integrity management program. Contributing to the severity of the accident were the lack of either automatic shutoff valves or remote control valves on the line and PG&E’s flawed emergency response procedures and delay in isolating the rupture to stop the flow of gas. Work Clearance Procedures for Electrical Work at Milpitas Terminal Because electrical work at the Milpitas Terminal resulted in the inadvertent pressure increases that preceded the rupture, the NTSB examined the procedures relating to this work. The investigation identified several deficiencies in the work clearance process used for the Milpitas Terminal electrical work. First, the system clearance form did not adequately detail the work to be performed. It did not discuss the equipment being worked on or the equipment that would be affected. The form indicated that normal function at the terminal would not be maintained, but there was no explanation, although the form called for such an explanation. The work clearance form also lacked any entries under “Sequence of Operations.” By contrast, NTSB investigators noted that the clearance form for the 2008 pressure increase to 400 pounds per square inch, gauge (psig) was highly detailed, with a step-by-step sequence of operations. Due to the lack of detail on the work clearance form for the September 9, 2010, work, the supervisory control and data acquisition (SCADA) operators would not have been aware of the scope and magnitude of the work being performed at the Milpitas Terminal. If the form had included the necessary information, the SCADA operators would have at least been aware that power interruptions were planned to specific instrumentation at the Milpitas Terminal and might have taken steps to mitigate the risk. This assumption was also illustrated when, after the rupture (at 7:05 p.m.), a SCADA operator incorrectly stated, “it was a regular scheduled clearance, it wasn’t supposed to affect anything.” In fact, the clearance form indicated that the work was expected to affect the normal functioning of equipment at the Milpitas Terminal. However, the form lacked clarity regarding how, and the extent to which, the normal functioning of equipment would be affected. Second, the preparation and contingency planning for the September 9, 2010, electrical work was inadequate as illustrated by the Milpitas technician’s queries to SCADA operators about whether the valves on the incoming lines would fail closed and whether the station could#

Page 33 be placed in bypass during the work. The fact that he needed to ask what impact certain steps in the work process would have on the equipment (as he did during phone calls at 3:36 p.m. and 4:03 p.m.) indicates that these issues were not evaluated in advance. Because of the lack of contingency planning, employees at both facilities were unprepared, as evidenced by the fact that workers at the Milpitas Terminal did not even realize that the regulator valves had opened at 5:22 p.m. until a SCADA operator informed them at 5:25 p.m. When the first unexpected power losses occurred at the Milpitas Terminal, the workers there decided to deviate from the assigned work and begin troubleshooting without stopping to notify the SCADA center2 or to assess the potential risk. By doing so, the workers at the Milpitas Terminal put themselves and the SCADA center in a reactive mode. Had a formal risk assessment been performed in advance, the SCADA staff might have taken precautionary measures to reduce the upstream pressures or have locked the regulating valves in a set position in advance, and retained those settings for the duration of the work, thus avoiding the accidental pressure increase. When work could potentially affect critical pipeline components, the involved personnel need to consider during the planning phase what failures might occur and be prepared to respond appropriately. The NTSB is concerned that the SCADA center approved an incomplete and inadequate system clearance form that did not address any such considerations. The NTSB concludes that had a properly prepared contingency plan for the Milpitas Terminal electrical work been in place and been executed, the loss of pressure control could have been anticipated and planned for, thereby minimizing or avoiding the pressure deviations. Accordingly, the NTSB recommends that PG&E revise its work clearance procedures to include requirements for identifying the likelihood and consequence of failure associated with the planned work and for developing contingency plans. Internal Communication and Coordination The SCADA operators’ initial response to the high pressure alarms at 5:25 p.m. was appropriate, and the required analysis took place within the first 10 minutes. After analyzing the alarms, the SCADA operator and the Milpitas technician agreed on a corrective action to reduce the monitor valve pressure set points back to within the maximum operating pressure, which was initiated within about 20 minutes of the alarms. When high pressure persisted and control could not be restored at the Milpitas Terminal, the SCADA center took further action and reduced the upstream pressure set points for the gas lines coming into the Milpitas Terminal. However, after the rupture, about 6:15 p.m., as the staff was confronted with both the Milpitas Terminal anomalies, as well as the low pressure alarms at the Martin Station coupled with the reports of a fire in San Bruno, it was evident from the communications between the SCADA center staff, the dispatch center, and various other PG&E employees that the roles and responsibilities for dealing with such emergencies were poorly defined. The SCADA operators and coordinators sat near each other in the SCADA center and communicated frequently throughout the evening, exchanging operational information among 2 In this letter, SCADA center refers to PG&E’s gas control center.#

Page 44 themselves about the ongoing electrical work at the Milpitas Terminal and its impact on the rest of the system. However, the NTSB notes that the lack of assigned roles and responsibilities resulted in SCADA staff not allocating their time and attention in the most effective manner. They did not initially notice the dropping pressures at the Martin Station after the rupture, but rather were alerted by staff at the Brentwood SCADA facility. Also, there was unnecessary overlap and duplication of their efforts. Several SCADA operators contacted the same SCADA transmission and regulation supervisor, but seemed unaware that the senior SCADA coordinator had already made contact with the supervisor. Further, the low pressure alarms at Martin Station were initially acknowledged by two SCADA coordinators. SCADA center staff provided many telephone briefings and updates to various PG&E employees and officials, occupying a significant portion of staff time during the first 90 minutes after the rupture. In addition, the SCADA staff received multiple calls from other PG&E employees and officials regarding the opening of various emergency response centers. These incoming and outgoing calls were handled by whichever SCADA staff member was available, without any command structure. It would have been beneficial to have had one SCADA operator designated as the sole point of contact for the workers at the Milpitas Terminal so that others could handle and monitor the remainder of the system. Such staff allocation would have permitted a direct exchange of information aimed at resolving the issue, while permitting other SCADA personnel to continue monitoring the entire system, maintain situational awareness, and communicate with internal and external entities, as needed. The lack of a centralized command structure was also evident in that key information was not disseminated in a reliable manner. Each SCADA staff member was left to form his or her own impression as to the nature and severity of the rupture based on the information they had, resulting in some conflicting and erroneous assessments. The lack of a centralized command structure was also reflected in the conflicting instructions regarding whether to remotely close valves at the Martin Station. At 7:25 p.m., an on-call supervisor called the SCADA center to ask whether mechanics should manually close the valves or whether the SCADA center could close them remotely. SCADA operator D3 responded that they would prefer to lose as few customers as possible and would rather have mechanics manually close the line valves. However, at 7:27 p.m., an on-scene SCADA transmission and regulation supervisor called the SCADA center and requested that they close the remote valves at Martin Station, which SCADA operator D did. Finally, the supervising engineer for the SCADA controls group seemed slow to get involved, despite the fact that he is responsible for all SCADA and control systems throughout the PG&E gas transmission pipeline system. His first contact with the SCADA center was at 6:51 p.m., requesting information. (By that time, he had already left the office for the day.) At 7:19 p.m., he called again to inform the senior SCADA coordinator that the workers at the Milpitas Terminal had said they did not need his help; he asked the senior SCADA coordinator to call him in a half hour if SCADA was still not working. The senior SCADA coordinator responded by suggesting to the supervising engineer that he go to the Milpitas Terminal to help with restoring SCADA functionality, noting that technicians had already been working on the 3 SCADA operators B, C, and D referenced in this letter were all working at the SCADA center in San Francisco. Operator D became the primary point of contact for workers at the Milpitas Terminal on the evening of the accident.#

Page 55 problem for 2 hours without success, but the supervising engineer said he would wait and asked that the SCADA center call him with updates. He arrived at the Milpitas Terminal about 9:00 p.m. The PG&E dispatch center learned of the explosion at 6:18 p.m. The dispatch center initially dispatched only a single service representative (at 6:23 p.m.) to assess the scene and did not immediately dispatch a qualified crew to shut off valves. At 6:27 p.m., the dispatch center informed SCADA operator C of the fire in San Bruno. At 6:31 p.m., SCADA operator B informed the dispatch center that there was a possibility the fire was being fed by a PG&E gas line. At 6:35 p.m., a mechanic informed the dispatch center that there was a PG&E transmission line in the area and that the flame seemed consistent with a transmission line fire. At 6:48 p.m., a PG&E supervisor called the dispatch center to request “gas crews to cut off distribution.” At 6:54 p.m., the San Bruno Police Department informed the dispatch center that they needed the gas shut off. At 6:55 p.m., SCADA operator B informed the dispatch center that the fire was being fed by Line 132. However, no one had yet been officially dispatched to shut off the valves and isolate the rupture. Further, no one within PG&E was compiling and assessing the information generated from the SCADA system and telephone communications from outside sources or between the SCADA center and dispatch. In summary, PG&E’s response to the Line 132 break lacked a command structure with defined leadership and support responsibilities within the SCADA center. Execution of the PG&E emergency plan resulted in delays that could have been avoided by better utilizing the SCADA center’s capability. The NTSB concludes that PG&E lacked detailed and comprehensive procedures for responding to a large-scale emergency such as a transmission line break, including a defined command structure that clearly assigns a single point of leadership and allocates specific duties to SCADA staff and other involved employees. Therefore, the NTSB recommends that PG&E establish a comprehensive emergency response procedure for responding to large-scale emergencies on transmission lines; the procedure should (1) identify a single person to assume command and designate specific duties for SCADA staff and all other potentially involved company employees; (2) include the development and use of trouble-shooting protocols and checklists; and (3) include a requirement for periodic tests and/or drills to demonstrate the procedure can be effectively implemented. Line Break Recognition Although SCADA staff quickly realized that there had been a gas line break in San Bruno, they were slow to recognize the connection between the line break and the overpressure at the Milpitas Terminal, and some staff were initially unsure of whether the break was in a transmission or a distribution line. In a postaccident interview, SCADA operator B stated that within 7 minutes of the rupture, he knew there had been a break in Line 132, and that by 6:30 p.m., he knew it was within a 12-mile corridor in the vicinity of San Bruno. At 6:53 p.m., SCADA operator D indicated that he knew the break was in Line 132, telling the on-scene SCADA transmission and regulation supervisor, “Yeah, absolutely we believe it’s a break on Line 132.” However, at about that time, there was still confusion among other employees as indicated by comments made at 6:51 p.m. by SCADA operator C to a PG&E pipeline engineer, indicating that although the#

Page 66 engineer said he thought there was a PG&E transmission line close to the area of the fire, SCADA operator C did not think the break was in a transmission line. At 6:55 p.m., in a telephone discussion between SCADA operator C and the on-scene PG&E gas maintenance and construction superintendent, both indicated that they believed a distribution line and not a transmission line had been breached. SCADA staff also had difficulties determining the exact location of the rupture. At 6:49 p.m., the SCADA center was still uncertain of the rupture point, as illustrated by the comment of the senior SCADA coordinator to a dispatch employee, “We are going to feed the line break at this pressure but I would take the pressure down if I knew more about what was feeding it… .” The PG&E SCADA system lacked several tools that could have assisted the staff in recognizing and pinpointing the location of the rupture, such as real-time leak or line break detection models, and closely spaced flow and pressure transmitters. A real-time leak detection application is a computer-based model of the transmission system that runs simultaneously with SCADA and provides greater feedback to SCADA operators when a large scale leak, line break, or system anomaly is present. Such models use actual SCADA pressures and flows to calculate actual and expected hydraulic performance; when the values do not match, an alarm is generated. Appropriate spacing of pressure transmitters at regular intervals4 allows SCADA operators to quickly identify pressure decreases that point toward a leak or line break. The NTSB concludes that PG&E’s SCADA system limitations contributed to the delay in recognizing that there had been a transmission line break and quickly pinpointing its location. Accordingly, the NTSB recommends that PG&E equip its SCADA system with tools to assist in recognizing and pinpointing the location of leaks, including line breaks; such tools could include a real-time leak detection system and appropriately spaced flow and pressure transmitters along covered transmission lines. Rapid Shutdown, Automatic Shutoff Valves, and Remote Control Valves Two mechanics had self-reported to the Colma yard at 6:35 p.m., and they decided to depart the yard at 7:06 p.m. to shut off the valves. Because gas was being supplied to the break from both the north and the south, shutdown and isolation of the rupture required closure of manual shutoff valves closest to the break, which were located about 1.5 miles apart, on either end of the break. The mechanics identified and manually closed those valves at 7:30 p.m. (south valve) and 7:46 p.m. (north valve). Also, about 7:29 p.m., the SCADA center remotely closed valves at the Martin Station in response to a request from a SCADA transmission and regulation supervisor who had joined the mechanics. The NTSB is concerned that the mechanics were unnecessarily held at the Colma yard and that the response could have been delayed even longer if the two mechanics had waited for official orders from PG&E. Further, the SCADA center staff could have reduced the flow sooner by shutting the remote valves at the Martin Station sooner, but they did not. These delays 4 SCADA data on Line 132 are currently received from only a few transmitters at randomly spaced intervals.#

Page 77 needlessly prolonged the release of gas and prevented emergency responders from accessing the area. The total heat and radiant energy released by the burning gas was directly proportional to the time gas flowed freely from the ruptured pipeline. Therefore, as vegetation and homes ignited, the fire would have spread and led to a significant increase in property damage. The pressurized flow from the south resulted in an intense flame front similar to a blowtorch, and emergency responders were unable to gain access to the area. If the gas had been shut off earlier, removing fuel flow, the fire would likely have been smaller and resulted in less damage. Also, buildings that would have provided protection to residents in a shorter duration fire were compromised because of the elevated heat. In addition to exposing residents and their property to increased risk, the prolonged fire also negatively affected emergency responders, who were put at increased risk by having to be in close proximity to fire for a longer time and were not available to respond to other potential emergencies while they were waiting for the fire to subside. The NTSB concludes that the 95 minutes that PG&E took to stop the flow of gas by isolating the rupture site was excessive. This delay, which contributed to the severity and extent of property damage and increased risk to the residents and emergency responders, in combination with the failure of the SCADA center to expedite shutdown of the remote valves at the Martin Station, contributed to the severity of the accident. The NTSB has long been concerned about the lack of standards for rapid shutdown and the lack of requirements for automatic shutoff valves (ASV) or remote control valves (RCV) in high consequence areas (HCA). As far back as 1971, the NTSB recommended, in Safety Recommendation P-71-1, the development of standards for rapid shutdown of failed natural gas pipelines. In 1995, the NTSB recommended, in Safety Recommendation P-95-1, that the Research and Special Programs Administration (RSPA), the predecessor agency of the Pipeline and Hazardous Materials Safety Administration (PHMSA), expedite requirements for installing automatic- or remote-operated mainline valves on high-pressure pipelines in urban and environmentally sensitive areas to provide for rapid shutdown of failed pipeline segments. The NTSB classified Safety Recommendation P-95-1 “Closed—Acceptable Action,” believing that the RSPA 2004 integrity management rulemaking (requiring that each gas transmission operator determine whether installing ASVs or RCVs would be an efficient means of adding protection to an HCA) would lead to a more widespread use of ASVs and RCVs. However, it did not. Federal regulations prescribe, at Title 49 Code of Federal Regulations (CFR) 192.179, the spacing of valves on a transmission line based on class location. However, other than for pipelines with alternative maximum allowable operating pressures (MAOP), 5 the regulations do not require a response time to isolate a ruptured gas line, nor do they explicitly require the use of ASVs or RCVs. The regulations give the pipeline operator discretion to decide whether ASVs or 5 Under 49 CFR 192.620, “Alternative Maximum Allowable Operating Pressure for Certain Steel Pipelines,” issued in 2008, an operator is allowed to operate a pipeline at up to 80 percent specified minimum yield strength in class 2 locations as long as it meets a very specific and stringent set of criteria. Section 192.620(c)(3) states that an RCV or ASV is required for such pipelines if the response time to mainline valves exceeds 1 hour under normal driving conditions and speed limits.#

Page 88 RCVs are needed in HCAs as long as they consider the factors listed under 49 CFR 192.935(c).6 Therefore, there is little incentive for an operator to perform an objective risk analysis, as illustrated by PG&E’s June 14, 2006, memorandum—which was issued after the CPUC 2005 audit identified PG&E’s failure to consider the issue but does not directly discuss any of the factors listed in section 192.935(c). Rather, it cites industry references to support the conclusion that most of the damage from a pipeline rupture occurs within the first 30 seconds, and that the duration of the resulting fire “has (little or) nothing to do with human safety and property damage.” The memorandum concludes that the use of an ASV or an RCV as a prevention and mitigation measure in an HCA would have “little or no effect on increasing human safety or protecting properties.” In the case of the San Bruno transmission line break, nearby RCVs could have significantly reduced the amount of time the fire burned, and thus the severity of the accident. Had the two isolation valves, located 1.5 miles apart, been outfitted with remote closure capability, prompt closure of those valves would have reduced the amount of fuel burned by the fire and allowed firefighters to enter the affected area sooner. The PG&E manager of gas system operations acknowledged at the NTSB’s investigative hearing held on March 1–3, 2011, that the use of RCVs could have reduced the time it took to isolate the rupture by about 1 hour. Damage from the pipeline rupture could have been reduced significantly if the valves on either end of the rupture point had been equipped with ASVs. Analysis of pressure differentials indicated that the San Bruno rupture would have resulted in the closure of an ASV at the downstream location7 and would likely also have resulted in the closure of an ASV at the upstream location.8 Even the closing of a downstream ASV alone would have been beneficial in that it would have immediately alerted SCADA to a more precise location of the break. Concerns about ASVs have focused on the cost of installation and their susceptibility to inadvertently trip based on pressure transients in the system. However, vendors have developed newer models that address these shortcomings by combining the features of traditional ASVs with RCVs. These “smart” valves include sensors that can trend the pressure transients on a line to identify what constitutes normal operation, thereby lessening the chances of an inappropriate shutdown. Also, the newer models can alert a SCADA center when the valve hits a trip point, allowing SCADA operators the option of overriding the valve closure and precluding an undesired shutdown. 6Those factors are (1) the swiftness of leak detection and pipe shutdown capabilities; (2) the type of gas being transported; (3) the operating pressure; (4) the rate of potential release; (5) the pipeline profile; (6) the potential for ignition; and (7) the location of nearest response personnel. 7 The pressure decay at the Martin Station showed a decrease from 386 to 200 psig in the course of 3 minutes (62 psig per minute), beginning at 6:11 p.m. This drop would have been more than sufficient to trip an ASV located at the downstream valve near the rupture point. 8 The pressure decay in Line 132 was not captured because the transmitter at that location was not installed directly on the main line but on a smaller transmission line (at Half Moon Bay) that branched off from Lines 132 and 109. Although the Half Moon Bay pressure readings cannot be used past 6:11 p.m. to approximate the Line 132 pressures upstream of the rupture, because the differential pressure was great enough to trip an ASV on the smaller line branching off Line 132 at Half Moon Bay, an ASV located on Line 132 likely would have tripped as well. (The smaller line crossed the San Andreas fault and, therefore, was equipped with an ASV to address seismic risk.)#

Page 99 The NTSB concludes that the use of ASVs or RCVs along the entire length of Line 132 would have significantly reduced the amount of time taken to stop the flow of gas and to isolate the rupture. The NTSB is aware that PG&E is in the process of expanding its use of ASVs and RCVs and has added this capability to some valve locations since the accident. However, the NTSB recommends that PG&E expedite the installation of ASVs and RCVs on transmission lines in HCAs and class 3 and 4 locations, and space them at intervals that consider the factors listed in 49 CFR 192.935(c). Deficiencies in Postaccident Drug and Alcohol Testing After the accident, PG&E identified four employees at the Milpitas Terminal for postaccident toxicological testing pursuant to 49 CFR 199.105 and 49 CFR 199.225. Test results were negative for the presence of specified drugs. Testing for drugs was accomplished successfully within the time constraints defined in 49 CFR 199.105; that is, within 32 hours of the accident. However, alcohol testing was not conducted properly in accordance with 49 CFR 199.225, which requires that testing be administered within 8 hours of an accident, and, if it is not, the operator shall cease attempts to do so. Results for the alcohol tests were invalid and therefore, the use of alcohol cannot be excluded. Alcohol testing of the four Milpitas Terminal employees commenced at 3:10 a.m. and concluded at 5:02 a.m. on September 10, 2010. The accident occurred at about 6:11 p.m. on the previous evening. Therefore, alcohol testing should have been completed by 2:11 a.m. on September 10, at the latest. PG&E officials explained that toxicological testing was delayed because the decision to perform testing was not made until approximately midnight and that the request for testing was made at 12:30 a.m. The NTSB is concerned by PG&E’s delay in contacting the toxicological testing contractor until 12:30 a.m., more than 6 hours after the rupture. Further, upon arrival at the Milpitas Terminal about 2:00 a.m., the contractor should have determined the time of the rupture and attempted to expedite alcohol testing, given that only minutes remained before the regulations prohibited testing. The NTSB is concerned that the alcohol testing was conducted after the prescribed 8 hours following an accident. Further, the NTSB is concerned that PG&E did not perform any drug or alcohol testing of its SCADA staff. The regulations in 49 CFR 199.105 and 49 CFR 199.225 require testing of any employee whose performance cannot be discounted completely as a contributing factor to the accident and that a decision not to administer a test must be based on a determination that the employee’s performance “could not have contributed to the accident.” The SCADA personnel were directly involved in monitoring and controlling the events that unfolded during the accident scenario. Therefore, the SCADA personnel should have been tested. The NTSB concludes that the 6-hour delay before ordering drug and alcohol testing, the commencement of alcohol testing at the Milpitas Terminal 1 hour after it was no longer permitted, the failure to properly record an explanation for the delay, and the failure to conduct drug or alcohol testing on the SCADA center staff all demonstrate that the PG&E postaccident#

Page 1010 toxicological program was ineffective. Therefore, the NTSB recommends that PG&E revise its postaccident toxicological testing program to ensure that testing is timely and complete. Summary of PG&E Integrity Management Practices In summary, the PG&E gas transmission integrity management program (1) was based on a geographic information system (GIS) that did not contain, and PG&E did not require it to contain, complete and accurate pipeline information; (2) significantly understated the threats due to external corrosion and design and manufacturing, and overstated the threats due to third-party damage and ground movement; (3) did not consider known longitudinal seam cracks in Line 132 dating to the 1948 construction and at least one longitudinal seam leak in a double submerged arc welded weld in its identification and assessment procedures; (4) allowed PG&E to chose an inappropriate assessment method for Line 132 that was inadequate to detect seam defects, such as the weld defect in the accident segment that led to the rupture; (5) considered known manufacturing- and construction-related defects on Line 132 to be stable even though the pipeline had not been pressure tested to at least 1.25 times its MAOP; and (6) included self-assessments that were superficial and resulted in no improvements to the integrity management program. Therefore, the NTSB concludes that the PG&E gas transmission integrity management program was deficient and ineffective. PG&E may already be addressing its GIS data deficiencies as part of its response to the NTSB’s January 3, 2011, safety recommendations, by beginning to verify the underlying records for the more than 1,800 miles of pipeline covered by the recommendations. Nevertheless, the NTSB is concerned that many unaddressed deficiencies still remain. Accordingly, the NTSB recommends that PG&E assess every aspect of its integrity management program, paying particular attention to the areas identified in this investigation, and implement a revised program that includes, at a minimum, (1) a revised risk model to reflect PG&E’s actual recent experience data on leaks, failures, and incidents; (2) consideration of all defect and leak data for the life of each pipeline, including its construction, in risk analysis for similar or related segments to ensure that all applicable threats are adequately addressed; (3) a revised risk analysis methodology to ensure that assessment methods are selected for each pipeline segment that address all applicable integrity threats, with particular emphasis on design/material and construction threats; and (4) an improved self-assessment that adequately measures whether the program is effectively assessing and evaluating the integrity of each covered pipeline segment. The NTSB further recommends that PG&E conduct threat assessments using the revised risk analysis methodology incorporated in its integrity management program, as recommended in Safety Recommendation P-11-29, and report the results of those assessments to the CPUC and PHMSA. Adequacy of PG&E’s Public Awareness Program Evaluation The NTSB also examined how PG&E conducts its public awareness program evaluations. The NTSB reviewed PG&E’s annual self-evaluations for the two review periods before the accident, June 2008–June 2009 and June 2009–June 2010. The NTSB found that the reviews simply documented that all required mailings were issued. Further, the action items in#

Page 1111 both reviews were nearly identical. The reviews did not further evaluate the public awareness program, such as reviewing the content, quality, or effectiveness of the information delivered. In the program evaluation and effectiveness section of its program plan, PG&E states that the annual review will be conducted using the guidelines of the plan. However, no specific guidelines for annual evaluations are defined. Examples of measurement components might include documenting website traffic on the company’s awareness pages and documenting the numbers of actual attendees in classes and meetings for emergency officials. If the data from these two example measures were gathered and documented, the effectiveness of the program could be tracked through the years. However, the NTSB notes that capturing such numbers is only the first step toward gauging whether attending the classes or ordering the materials PG&E offers translates into understanding, retention, and changed behavior. In addition to its annual self-assessments, PG&E had its contractor Paradigm Alliance, Inc., conduct a separate program documentation review (dated 2010) of its public awareness program for noncustomers along the pipeline right-of-way. As a program effectiveness measure, Paradigm sent out brochures with detachable business reply mail survey postcards to 15,302 addresses. Paradigm received only 20 survey postcards in response. The answers received indicated that the majority of respondents did not recall seeing any recent pipeline safety information, and nearly half had low awareness about pipeline proximity. The lack of responses, the poor comprehension reflected in the responses received, and PG&E’s failure to make changes based on such dismal effectiveness data raise concerns about PG&E’s commitment to the quality of its public awareness program and to continuously improving it. American Petroleum Institute (API) Recommended Practice 1162 specifies that operators continuously improve their public awareness programs. In 2007, PG&E participated in the API’s Public Awareness Program Effectiveness Research Survey (PAPERS) program. This particular PAPERS surveyed only the affected public that lived along the right-of-way; PG&E’s customers were left out. The survey results showed that the affected public respondent pool had the lowest awareness about pipeline safety, whereas the emergency responder respondents had the highest awareness but also had indicated they needed more information and appropriate training. These 2007 results were still keenly reflective of PG&E’s public awareness program effectiveness on the day of the accident in 2010, showing that residents were unaware and emergency responders were in need of more information. Had PG&E interpreted the 2007 PAPERS effectiveness results and acted upon them 3 years prior, PG&E could have made meaningful changes to its program and increased awareness in the San Bruno community. As of the date of this report, no program revisions have been noted as a result of the PAPERS. The NTSB concludes that PG&E’s public awareness program self-evaluation was ineffective at identifying and correcting deficiencies. Therefore, the NTSB recommends that PG&E develop, and incorporate into its public awareness program, written performance measurements and guidelines for evaluating the plan and for continuous program improvement. Therefore, the National Transportation Safety Board makes the following recommendations to the Pacific Gas and Electric Company:#

Page 1212 Revise your work clearance procedures to include requirements for identifying the likelihood and consequence of failure associated with the planned work and for developing contingency plans. (P-11-24) Establish a comprehensive emergency response procedure for responding to large-scale emergencies on transmission lines; the procedure should (1) identify a single person to assume command and designate specific duties for supervisory control and data acquisition staff and all other potentially involved company employees; (2) include the development and use of trouble-shooting protocols and checklists; and (3) include a requirement for periodic tests and/or drills to demonstrate the procedure can be effectively implemented. (P-11-25) Equip your supervisory control and data acquisition system with tools to assist in recognizing and pinpointing the location of leaks, including line breaks; such tools could include a real-time leak detection system and appropriately spaced flow and pressure transmitters along covered transmission lines. (P-11-26) Expedite the installation of automatic shutoff valves and remote control valves on transmission lines in high consequence areas and in class 3 and 4 locations, and space them at intervals that consider the factors listed in Title 49 Code of Federal Regulations 192.935(c). (P-11-27) Revise your postaccident toxicological testing program to ensure that testing is timely and complete. (P-11-28) Assess every aspect of your integrity management program, paying particular attention to the areas identified in this investigation, and implement a revised program that includes, at a minimum, (1) a revised risk model to reflect the Pacific Gas and Electric Company’s actual recent experience data on leaks, failures, and incidents; (2) consideration of all defect and leak data for the life of each pipeline, including its construction, in risk analysis for similar or related segments to ensure that all applicable threats are adequately addressed; (3) a revised risk analysis methodology to ensure that assessment methods are selected for each pipeline segment that address all applicable integrity threats, with particular emphasis on design/material and construction threats; and (4) an improved self-assessment that adequately measures whether the program is effectively assessing and evaluating the integrity of each covered pipeline segment. (P-11-29) Conduct threat assessments using the revised risk analysis methodology incorporated in your integrity management program, as recommended in Safety Recommendation P-11-29, and report the results of those assessments to the California Public Utilities Commission and the Pipeline and Hazardous Materials Safety Administration. (P-11-30)#

Page 1313 Develop, and incorporate into your public awareness program, written performance measurements and guidelines for evaluating the plan and for continuous program improvement. (P-11-31) The NTSB also issued safety recommendations to the U.S. Secretary of Transportation, the Pipeline and Hazardous Materials Safety Administration, the governor of the state of California, the California Public Utilities Commission, the American Gas Association, and the Interstate Natural Gas Association of America. Additionally, the report reclassifies two previously issued recommendations to the Pipeline and Hazardous Materials Safety Administration. In response to the recommendations in this letter, please refer to Safety Recommendations P-11-24 through -31. If you would like to submit your response electronically rather than in hard copy, you may send it to the following e-mail address: correspondence@ntsb.gov. If your response includes attachments that exceed 5 megabytes, please e-mail us asking for instructions on how to use our secure mailbox. To avoid confusion, please use only one method of submission (that is, do not submit both an electronic copy and a hard copy of the same response letter). Chairman HERSMAN, Vice Chairman HART, and Members SUMWALT, ROSEKIND, and WEENER concurred in these recommendations. Chairman HERSMAN filed a concurring statement and Vice Chairman HART filed a concurring and dissenting statement, both of which are attached to the pipeline accident report for this accident. [Original Signed] By: Deborah A.P. Hersman Chairman#