BOARDWALK PETROCHEMICAL PIPELINE, LLC — Notice of Amendment

party submissionOfficial PDF32023013NOA_Operator Response to Notice_08172023_(22-250434).pdf#

32023013NOA_Closure Letter_12072023_(22-250434)_text.pdf, page 1Official PDFVIA ELECTRONIC MAIL TO: stan.horton@bwpipelines.com; tina.baker@bwpipelines.com December 7, 2023 Mr. Stanley Horton President/CEO Boardwalk Petrochemical Pipeline, LLC 9 Greenway Plaza, Suite 2800 Houston, TX 77036 RE: CPF 3-2022-013-NOA Dear Mr. Horton: From September 19 through September 23, 2022, a representative from the Pipeline and Hazardous Materials Safety Administration (PHMSA), pursuant to chapter 601 of 49 United States Code, conducted an on-site pipeline safety inspection of Boaredwalk Petrochemical Pipeline, LLC (Boardwalk) procedures in Sulphur, Louisiana. As a result of the inspection, Boardwalk was issued a Notice of Amendment on July 2023, which proposed amendment of your procedures. Boardwalk submitted its amended procedures on August 17, 2023. My staff reviewed the amended procedures, and it appears that the inadequacies outlined in this Notice of Amendment have been corrected. This letter is to inform you no further action is necessary and this case is now closed. Thank you for your cooperation. Sincerely, Gregory A. Ochs Director, Central Region, Office of Pipeline Safety Pipeline and Hazardous Materials Safety Administration cc: Tina Baker, Manager Compliance Service, tina.baker@bwpipelines.com#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 1Official PDFNOTICE OF AMENDMENT VIA ELECTRONIC MAIL TO: stan.horton@bwpipelines.com; tina.baker@bwpipelines.com July 20, 2023 Mr. Stanley Horton President/CEO Boardwalk Petrochemical Pipeline, LLC 9 Greenway Plaza, Suite 2800 Houston, TX 77036 CPF 3-2023-013-NOA Dear Mr. Horton: From September 19 to September 23, 2022, a representative of the Pipeline and Hazardous Materials Safety Administration (PHMSA), pursuant to Chapter 601 of 49 United States Code (U.S.C.), inspected, by means of video conference, Boardwalk Petrochemical Pipeline, LLC’s (Boardwalk) procedures for Control Room Management (CRM) in Sulphur Springs, Louisiana. As a result of the inspection, PHMSA has identified the apparent inadequacies found within Boardwalk’s plans or procedures. The items inspected and the inadequacies are described below: 1. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by § 195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h)#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 2must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) Roles and Responsibilities. Each operator must define the roles and responsibilities of a controller during normal, abnormal and emergency operation conditions. To provide for a controller’s prompt and appropriate response to operating conditions, and operator must define each of the following: (1) …. (3) A controllers role during an emergency, even if the controller is not the first to detect the emergency, including the controllers responsibility to take specific actions and to communicate with others; Boardwalk's procedures CRM Plan 7/27/2022 Rev 5.30 and BWP 6618-HL were not adequate to support control room evacuation. Missing from the procedure were steps relating to muster point locations, and who decides to return to the primary control room or go to the backup control room. Also, the plan included utilizing the qualified manager of the control room to monitor the system remotely on a laptop computer. This required quick notification of the manager to allow for monitoring during the controller absence. Missing from the procedure was a step related to calling the manager to remotely monitor the system. The procedure needs to be amended to include additional information related to muster points, who makes the decision to return to the primary control room or go to back up control room, and a step to call the manager to remotely monitor the system. Following the inspection, Boardwalk followed up with amended procedures. Upon review, the amended procedures were determined adequate. No further action is required. 2. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) ….#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 3(c) Provide adequate information. Each operator must provide its controllers with the information tools, processes and procedures necessary for the controllers to carry out the roles and responsibilities the operator has defined by performing each of the following: (1) Implement API RP 1165 (incorporated by reference, see §195.3) whenever a SCADA system is added, expanded or replaced, unless the operator demonstrates that certain provisions of API RP 1165 are not practical for the SCADA system used; Boardwalk’s CRM Plan Rev 5.30 7/27/2022 was not adequate to define addition, expansion or replacement to demonstrate compliance when implementing API 1165. In addition to definitions, there were no examples to provide support of the definitions and future control room / SCADA activities. The procedure needs to be amended to provide definitions and examples of addition, expansion, and replacement of a SCADA system to implement API 1165. Following inspection, Boardwalk followed up with amended procedures. Upon review, the amended procedures were determined adequate. No further action is required. 3. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (c) Provide adequate information. Each operator must provide its controllers with the information tools, processes and procedures necessary for the controllers to carry out the roles and responsibilities the operator has defined by performing each of the following: (1) …. (2) Conduct point-to-point verification between SCADA displays and related field equipment when field equipment is added or removed and when other changes that affect pipeline safety are made to field equipment or SCADA displays;#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 4Boardwalk’s CRM Plan Rev 5.30 7/27/2022 was not adequate to define and prioritize leak alarms as safety-related. A review of the SCADA alarm database for both analogue and status points revealed there were a few additional alarms identified as safety-related that were not included in the procedure. Boardwalk also provided Rate of Change (ROC) alarms for pressures and flows. The benchmark value was not designed with any priority or set point parameters for alarming. Because Appendix 8, of the CRM Plan, identified that pressure and flow ROC can be an indication for a leak, some alarming level should be considered as safety-related; none was provided. The procedure needs to be amended to provide some consideration for leak detection points/alarms and ROC as safety-related, as well as include the additional tags identified in the SCADA database noted as safety-related (if the operator determines them to be safety-related). 4. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (c) Provide adequate information. Each operator must provide its controllers with the information, tools, processes and procedures necessary for the controllers to carry out the roles and responsibilities the operator has defined by performing each of the following: (1) …. (3) Test and verify an internal communication plan to provide adequate means for manual operation of the pipeline safely, at least once each calendar year, but at intervals not to exceed 15 months; Boardwalk’s CRM Plan Rev 5.30 7/27/2022 Section 4.9 and Task 6650 was not adequate to test and verify their internal communication plan for the manual operation of the pipeline. While the procedure was well thought out, the procedure was missing: a consistent method to record the information related to shut down to ensure everything was in the desired appropriate state for shutdown, a process to document information over a long outage timeframe over multiple#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 5controllers shifts, and a process for monitoring and reporting leaks, abnormal operations and emergencies. The procedure needs to be amended to provide a consistent method to record the information related to shut down to ensure everything was in the desired appropriate state for shutdown. The procedure must include steps to document information over a long outage timeframe over multiple controllers shifts and a process for monitoring and reporting leaks, abnormal operations and emergencies. 5. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (d) Fatigue mitigation. Each operator must implement the following methods to reduce the risk associated with controller fatigue that could inhibit a controllers’ ability to carry out the roles and responsibilities the operator has defined: Boardwalk’s CRM Plan Rev 5.30 7/27/2022 was not adequate to identify the risks associated with controller fatigue to reduce the risk through mitigation strategies. Boardwalk did not specifically state and define fatigue risks in Section 5 of the CRM Plan, other than noting commute time as a possible risk. Section 9.5 of the CRM Plan identified fatigue risks but addresses them as Fatigue Mitigation Training. Boardwalk also had Task List 6605 Fatigue Mitigation Guidelines. This task list covered fatigue mitigation countermeasures. The procedure needs to be amended to identify fatigue risks for the controllers that can then support fatigue mitigation and training. Following inspection, Boardwalk followed up with amended procedures. Upon review, the amended procedures were determined adequate. No further action is required.#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 66. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (d) Fatigue mitigation. Each operator must implement the following methods to reduce the risk associated with controller fatigue that could inhibit a controllers’ ability to carry out the roles and responsibilities the operator has defined: Boardwalk’s CRM Plan Section 5 was inadequate because it did not require the manager of the control room, who is OQ qualified and can be called into service to monitor the pipeline, to track his hours of work in SchedulePro, a time tracking application. Even though the manager does not work a shift, he is someone identified in procedure who can be called upon to monitor the system in the event the control room must be evacuated and or controllers need to relocate to the backup control center. Tracking hours worked for all qualified controllers who can be pulled into service is important. The manager is bound by the hours-of-service standards that apply to all controllers. If the manager is required to monitor and control the system, for any reason, his hours of service must be considered for fatigue and or deviation from the hours of service. The procedure must be amended to include tracking time and hours of service for all personnel who are qualified and could be pulled into service to monitor and control the pipeline system. 7. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 7must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (1) Review SCADA safety-related alarm operations using a process that ensures alarms are accurate and support safe pipeline operations; Boardwalk’s CRM Plan was not adequate because it did not provide a specific process for managing stale or unreliable data to ensure alarms are accurate and support safe pipeline operations. Boardwalk's Display Standards and Style Guide: Liquids Control 7/27/2022 provided a definition of stale data icons. This offered a very good definition of stale data and how the indicator was generated and presented to the controller. Task List 6619 mentioned malfunctioning or loss of communication, but it did not include stale data, which may be a subset of both or something totally different. Therefore, it warranted independent consideration. Section 2.0 of the CRM Plan did not offer definitions for malfunctioning, inaccurate, stale or unreliable alarms. The procedure offered no guidance to the controller on how to recognize stale or unreliable data or what to do if identified through alarms or other means. The procedure must be amended to include guideance for the controller on alarm response and handling of stale and unreliable data. Additionally, it should provide more definition around malfunctioning, loss of communication, stale and unreliable alarms as they are different in nature and require different responses. Following inspection, Boardwalk followed up with amended procedures. Upon review, these procedures were determined adequate. No further action is required. 8. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 8by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (1) …. (5) Monitor content and volume of general activity being directed to and required of each controller at least once each calendar year, but at intervals not exceeding 15 months, that will assure controllers have sufficient time to analyze and react to incoming alarms; Boardwalk's CRM Plan was not adequate because it did not define the criteria to determine if the controller has sufficient time to analyze and react to incoming alarms. Boardwalk’s plan states in Task List 6611 that if the controller activity review concludes that any aspect of the controller workload is "excessive," adjustments will be made. However, the procedure did not define criteria for determining “excessive” Controller workload. From all the information gathered and analyzed, there should be some criteria that is the driver for determining "excessive" or "sufficient." The procedure needs to be amended to define the criteria that will be used to identify the controller has sufficient time to analyze and react to incoming alarms. 9. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms.#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 9An operator’s plan must include provisions to: (1) …. (6) Address deficiencies identified through the implementation of paragraphs (e)(1) through (e)(5) of this section. Boardwalk’s CRM procedures for addressing deficiencies identified through the implementation of paragraphs (e)(1) - (5) was not adequate. Section 6.9 of the CRM Plan addressed deficiencies found implementing §§ 195.446(e)(1) through 195.446(e)(5) and identified Form BWP-HL 6640 Alarm Management Deficiency Review. The procedure stated that Boardwalk “[w]ill promptly correct specific issues commensurate with their importance to safety.” It also stated that documentation should also record the basis for the selection and scheduling of corrective action. During the inspection, a review of the form was completed. The review demonstrated that documenting a deficiency was very similar to documenting a deviation. These are not the same. The operator used GMS tickets, a subset of MAXIMO, to assign alarm or equipment repairs to the field, such as malfunctioning or inaccurate alarms. This practice was not defined in the CRM procedure or their Task 6619 for handling malfunctioning and inaccurate alarms. MOC's were also used for alarm changes, which could be required due to identified deficiencies. The procedure did not include who is responsible for reviewing these work orders to resolution. The CRM procedure and Task needs to be amended to identify types of deficiencies that could arise while implementing §§ 195.446(e)(1) through (5), and how the deficiencies, when identified, will be recorded and tracked to resolution. The procedure must include a periodic review process to support tracking to resolution. 10. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated as appropriate, with the operator’s written procedures required by §195.402. An operator must develop the procedures no later than August 1, 2011, and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4), and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than August 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. (b) …. (f) Change Management. Each operator must assure that changes that could affect control room operations are coordinated with the control room personnel by#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 10performing each of the following: (1) Implement section 7 of API RP 1168 (incorporated by reference, see §195.3) for control room management change and require coordination between control room representatives, operator’s management, and associated field personnel when planning and implementing physical changes to pipeline equipment or configuration; Boardwalk's CRM procedure was not adequate to substantiate that the OMS MAXIMO system is the overarching management of change used by the company. While it appears in many Task Lists as a way to initiate and track different types of changes or work requests, it appeared to be more of a practice used by the company rather than a policy or procedure. It was clear from Task List 6602 that information for changes that involved SCADA came through the OMS system. But changes that were not SCADA related, yet could still affect control room operations, were initiated through OMS – they were not defined in procedure. Section 7 of the CRM Plan required pipeline operations, engineering, and construction to contact the control center when making field changes that affect the operations of the control center. It provided options of communications as telephone, text, email, memorandum, mail, or another method found acceptable. It did not include the OMS MAXIMO system. Yet, it was referenced as the communication change method in Task List 6602 as well as in other procedures that required controllers to review the OMS system, as in BWP 6638-HL and BWP-6649-HL. Boardwalk's procedure needs to be amended to include how it is getting information from the OMS system related to changes initiated outside the control room and internal changes initiated within the control room for both SCADA and non-SCADA changes. Response to this Notice This Notice is provided pursuant to 49 U.S.C. § 60108(a) and 49 C.F.R. § 190.206. Enclosed as part of this Notice is a document entitled Response Options for Pipeline Operators in Enforcement Proceedings. Please refer to this document and note the response options. Be advised that all material you submit in response to this enforcement action is subject to being made publicly available. If you believe that any portion of your responsive material qualifies for confidential treatment under 5 U.S.C. § 552(b), along with the complete original document you must provide a second copy of the document with the portions you believe qualify for confidential treatment redacted and an explanation of why you believe the redacted information qualifies for confidential treatment under 5 U.S.C. § 552(b). Following the receipt of this Notice, you have 30 days to submit written comments, revised procedures, or a request for a hearing under § 190.211. If you do not respond within 30 days of receipt of this Notice, this constitutes a waiver of your right to contest the allegations in this Notice and authorizes the Associate Administrator for Pipeline Safety to find facts as alleged in this Notice without further notice to you and to issue an Order Directing Amendment. If your plans or procedures are found inadequate as alleged in this Notice, you may be ordered to amend your plans or procedures to correct the inadequacies (49 C.F.R. § 190.206). If you are not contesting this Notice, we propose that you submit your amended procedures to my office within#

32023013NOA_Notice of Amendment_07202023_(22-250434)_text.pdf, page 11thirty days of receipt of this Notice. This period may be extended by written request for good cause. Once the inadequacies identified herein have been addressed in your amended procedures, this enforcement action will be closed. It is requested that Boardwalk maintain documentation of the safety improvement costs associated with fulfilling this Notice of Amendment (preparation/revision of plans, procedures) and submit the total to Gregory Ochs, Director, Central Region, Pipeline and Hazardous Materials Safety Administration. In correspondence concerning this matter, please refer to CPF 3-2023-013-NOA and, for each document you submit, please provide a copy in electronic format whenever possible. Sincerely, Gregory A. Ochs Director, Central Region, Office of Pipeline Safety Pipeline and Hazardous Materials Safety Administration Enclosure: Response Options for Pipeline Operators in Enforcement Proceedings cc: Tina Baker, Manager Compliance Service, tina.baker@bwpipelines.com#