CPF 32022028NOA
CPF 32022028NOA
case documentOfficial PDF32022028NOA_Operator Post-Hearing Submission_01132023_(20-173062).pdf#
party submissionOfficial PDF32022028NOA_Operator Response to Notice_06272022_(20-173062).pdf#
party submissionOfficial PDF32022028NOA_Operator Withdrawal of Hearing Request_09012023_(20-173062).pdf#
32022028NOA_PHC Hearing Rescheduled_10172022_(20-173062)_text.pdf, page 1Official PDFOctober 17, 2022 VIA ELECTRONIC MAIL TO: catherine.little@bracewell.com and gregory.ochs@dot.gov Ms. Catherine D. Little, Esq. Counsel for Colonial Pipeline Company Bracewell LLP 600 Peachtree Street, NE, Suite 3000 Atlanta, GA 30308 Mr. Gregory Ochs Director, Central Region Pipeline and Hazardous Materials Safety Administration 901 Locust Street, Suite 480 Kansas City, MO 64106-2641 Re: Notice of Rescheduled Hearing, Colonial Pipeline Company CPF Nos. 3-2022-026-NOPV and 3-2022-028-NOA Dear Ms. Little and Mr. Ochs: In accordance with 49 C.F.R. § 190.211, an informal hearing will be held regarding the Notice of Probable Violation, Proposed Civil Penalty, and Proposed Compliance Order and the Notice of Amendment issued to Colonial Pipeline Company by the Office of Pipeline Safety in the above- referenced matters. The hearing, which was originally scheduled for October 11, 2022, has been rescheduled to take place on November 30, 2022, beginning at 8:30 a.m. Central Time. The hearing will take place at the PHMSA Central Region office, 901 Locust Street, Suite 480, Kansas City, Missouri 64106. Upon arrival at the building, attendees will be required to present photo identification to security personnel. A contact phone number for the day of the hearing is (816) 329-3800. If you have any questions, do not hesitate to contact me. Sincerely, Larry White Presiding Official cc: Mr. Joseph A. Blount, President and Chief Executive Officer, Colonial Pipeline Company, jblount@colpipe.com#
32022028NOA_PHC Hearing Rescheduled_10172022_(20-173062)_text.pdf, page 2Mr. Quintin Frazier, Director, Pipeline Compliance, Colonial Pipeline Company, qfrazier@colpipe.com Mr. Joseph Hainline, Counsel, Central Region, Office of Pipeline Safety, PHMSA, joseph.hainline@dot.gov#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 1Official PDFSeptember 1, 2023 VIA ELECTRONIC MAIL TO: MLittle@colpipe.com Melanie Little President and Chief Executive Officer Colonial Pipeline Company 1000 Lake Street Alpharetta, Georgia 30009 CPF No. 3-2022-026-NOPV CPF No. 3-2022-028-NOA Dear Ms. Little: Enclosed please find a Consent Order incorporating the terms of the Consent Agreement between the Pipeline and Hazardous Materials Safety Administration (PHMSA) and Colonial Pipeline Company, which was executed on August 30, 2023. Service of the Consent Order and Consent Agreement by electronic mail is deemed effective upon the date of transmission and acknowledgement of receipt, or as otherwise provided under 49 C.F.R. § 190.5. Thank you for your cooperation in this matter. Sincerely, Alan K. Mayberry Associate Administrator for Pipeline Safety Enclosure: Consent Order and Consent Agreement cc: Mr. Gregory Ochs, Director, Central Region, Office of Pipeline Safety, PHMSA Ms. Angela D. Kolar, Senior Vice President, Operations, EHS, & Compliance, Colonial Pipeline Company, akolar@colpipe.com Ms. Catherine Little, Counsel for Colonial Pipeline Company, Bracewell LLP, catherine.little@bracewell.com CONFIRMATION OF RECEIPT REQUESTED#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 2U.S. DEPARTMENT OF TRANSPORTATION PIPELINE AND HAZARDOUS MATERIALS SAFETY ADMINISTRATION OFFICE OF PIPELINE SAFETY WASHINGTON, D.C. 20590 ____________________________________ In the Matter of ) ) ) Colonial Pipeline Company, ) CPF No. 3-2022-026-NOPV ) CPF No. 3-2022-028-NOA Respondent. ) ____________________________________) CONSENT ORDER By letter dated May 5, 2022, the Pipeline and Hazardous Materials Safety Administration (PHMSA), Office of Pipeline Safety (OPS), issued a Notice of Probable Violation, Proposed Civil Penalty and Proposed Compliance Order (Notice) to Colonial Pipeline Company (Colonial or Respondent). PHMSA issued a separate Notice of Amendment (NOA) to Colonial by letter on May 27, 2022. In response to the Notice, Respondent requested a hearing on Items 1, 2, 4, 5, and 7 of the Notice, the associated proposed civil penalty, and the proposed compliance order obligations (Response). Respondent did not contest Items 3 and 6 of the Notice. In its Response, Colonial also requested a hearing on Items 5 and 7 of the NOA. Respondent did not contest Items 1, 2, 3, 4, 6, and 8 of the NOA. Colonial also asked for the opportunity to meet informally with PHMSA to discuss the issues raised by the contested Notice and NOA Items. Respondent and PHMSA (The Parties) subsequently met on several occasions to discuss the issues raised in the Response. Through these meetings, the Parties agreed on a resolution of all Items except Item 5 of the Notice and Item 5 of the NOA. A hearing was held on November 30, 2022, regarding Notice Item 5 and NOA Item 5. In July 2023, the Parties resumed informal settlement discussions for Item 5 of the Notice and Item 5 of the NOA. As a result of those resumed discussions, as explained in more detail below, the Parties have agreed to a Consent Agreement which resolves the allegations set forth in the Notice and NOA, to include a reduced civil of $948,400. Accordingly, the Consent Agreement is hereby approved and incorporated by reference into this Consent Order. Colonial is hereby ordered to comply with the terms of the Consent Agreement pursuant to its terms. Pursuant to 49 U.S.C. § 60101, et seq., failure to comply with this Consent Order may result in the assessment of civil penalties as set forth in 49 U.S.C. § 60122 and 49 C.F.R. § 190.223, or in referral to the Attorney General for appropriate relief in a district court of the United States. The terms and conditions of this Consent Order are effective upon service in accordance with 49 C.F.R. § 190.5. September 1, 2023 _________________________________ ________________________ Alan K. Mayberry Date Issued Associate Administrator for Pipeline Safety#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 3U.S. DEPARTMENT OF TRANSPORTATION PIPELINE AND HAZARDOUS MATERIALS SAFETY ADMINISTRATION OFFICE OF PIPELINE SAFETY WASHINGTON, D.C. 20590 In the Matter of ) ) ) Colonial Pipeline Company, ) CPF No. 3-2022-026-NOPV ) CPF No. 3-2022-028-NOA Respondent. ) ____________________________________) CONSENT AGREEMENT From January 27 to November 12, 2020, and from October 29 to November 4, 2021, a representative of the Pipeline and Hazardous Materials Safety Administration (PHMSA or the Agency), Office of Pipeline Safety (OPS), pursuant to Chapter 601 of 49 United States Code (U.S.C.) inspected Colonial Pipeline Company’s (Colonial or Respondent) procedures and records for Control Room Management (CRM) in Linden, New Jersey; Hebert, Texas; Greensboro, North Carolina; and Alpharetta, Georgia. As a result of the inspection, the Director, Central Region, OPS (Director), issued to Colonial, by letter dated May 5, 2022, a Notice of Probable Violation, Proposed Civil Penalty and Proposed Compliance Order (Notice), and a separate Notice of Amendment (NOA) and Warning Letter issued by letters on May 27, 2022. In accordance with 49 C.F.R. § 190.207, the Notice proposed finding that Respondent had violated seven provisions of 49 C.F.R. Part 195 (Items 1 through 7), proposed ordering Respondent to take certain measures to correct the alleged violations, and proposed a civil penalty in the amount of $986,400. In accordance with 49 C.F.R. § 190.206, the NOA alleged eight procedures were inadequate to assure the safe operation of a pipeline facility and proposed that Respondent submit revised procedures to address the inadequacies identified. In accordance with 49 C.F.R. § 190.205, the Warning Letter advised Colonial to address two areas to avoid potential future enforcement action, but otherwise did not require further response by Respondent. During and following PHMSA’s inspections in 2020 and 2021, Respondent worked with the Agency to better understand and proactively address the comments and concerns raised by PHMSA, including those raised by PHMSA shortly before issuance of the Notice. With that as background and in response to the Notice, Respondent requested a hearing to address the issues presented by Items 1, 2, 4, 5, and 7 of the Notice, the associated proposed civil penalty, and the associated proposed compliance order (PCO) obligations. Respondent did not contest Items 3#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 4and 6 of the Notice. Item 6 of the Notice was brought as a warning requiring no further action by Respondent. In response to the NOA, Respondent requested a hearing on Items 5 and 7 of the NOA. Respondent did not contest Items 1, 2, 3, 4, 6, and 8 of the NOA and timely submitted revised procedures for review by the Director, Central Region, to address those Items. In responding to the Notice and the NOA, Respondent asked for the opportunity to meet informally with PHMSA to discuss the issues raised by the contested Notice and NOA Items. Respondent and PHMSA (the Parties) subsequently met on several occasions to discuss the issues. Through those meetings, the Parties agreed on a resolution of all items except Item 5 of the Notice and Item 5 of the NOA. Additionally, Respondent addressed all PCO obligations with subsequent approval by PHMSA. Joint stipulations were provided to the Presiding Official on November 29, 2022 regarding Notice Items 1-4, 6-7 and NOA Items 1-4, 6-8 in advance of a Hearing limited to Item 5 of the Notice and Item 5 of the NOA. A Hearing was held on November 30, 2022 in Kansas City, Missouri regarding Notice Item 5 and NOA Item 5. After the Hearing, post-hearing briefings were timely filed by the Parties. The Parties resumed informal settlement meetings in July 2023. As a result of those discussions, and subject to the clarifications set forth below, the Parties have agreed to resolve the allegations set forth in the Notice and NOA and Respondent has agreed to pay a reduced civil penalty in the amount of $948,400. Having agreed that settlement of the Notice and the NOA will avoid further administrative proceedings or litigation of those Items and will serve the public interest by promoting safety and protection of the environment, pursuant to 49 C.F.R. Part 190, and upon consent and agreement of Respondent and PHMSA, the Parties hereby agree as follows: I. General Provisions 1. Respondent acknowledges that as the operator of the pipeline facilities subject to the Notice and NOA, Respondent and its referenced hazardous liquid pipeline facilities are subject to the jurisdiction of the Federal pipeline safety laws, 49 U.S.C. § 60101, et seq., and the regulations and administrative orders issued thereunder. For purposes of this Consent Agreement (Agreement), Respondent acknowledges that it received proper notice of PHMSA's actions in these proceedings and that the Notice and NOA state claims upon which relief may be granted pursuant to 49 U.S.C. § 60101, et seq., and the regulations and orders issued thereunder. 2. After Respondent returns this signed Agreement to PHMSA, the Agency’s representative will present it to the Associate Administrator for Pipeline Safety, recommending that the Associate Administrator adopt the terms of this Agreement by issuing an administrative order (Consent Order) incorporating the terms of this Agreement. The terms of this Agreement constitute an offer of settlement until accepted by the Associate Administrator. Once accepted, the Associate Administrator will issue a Consent Order incorporating the terms of this Agreement. 3. Respondent consents to the issuance of the Consent Order, and hereby waives any further procedural requirements with respect to its issuance as long as it is consistent with the terms of this Agreement. Respondent waives all rights to contest the adequacy of the Notice or#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 5the NOA, or the validity of the Consent Order or this Agreement, including all rights to administrative or judicial hearings or appeals, except as set forth herein. 4. This Agreement shall apply to and be binding upon PHMSA and Respondent, its officers, directors, and employees, and its successors, assigns, or other entities or persons otherwise bound by law. Respondent agrees to provide a copy of this Agreement and any incorporated work plans and schedules to all of Respondent’s officers, employees, and agents whose duties might reasonably include compliance with this Agreement. 5. This Agreement constitutes the final, complete, and exclusive agreement and understanding between the Parties with respect to the settlement embodied in this Agreement. The Parties acknowledge that there are no representations, agreements or understandings relating to settlement other than those expressly contained in this Agreement, except that the terms of this Agreement may be construed by reference to the Notice and the NOA. 6. Nothing in this Agreement affects or relieves Respondent of its responsibility to comply with all applicable requirements of the Federal pipeline safety laws, 49 U.S.C. § 60101, et seq., and the regulations and orders issued thereunder. Nothing in this Agreement alters PHMSA's right of access, entry, inspection, and information gathering or PHMSA's authority to bring enforcement actions against Respondent pursuant to the Federal pipeline safety laws, the regulations and orders issued thereunder, or any other provision of Federal or State law. 7. For all transfers of ownership or operating responsibility of Respondent’s pipeline system referenced herein, Respondent will provide a copy of this Agreement to the prospective transferee at least 30 days prior to such transfer. Respondent will provide written notice of the transfer to the PHMSA Central Regional Director no later than 60 days after the transfer occurs. 8. This Agreement does not waive or modify any Federal, State, or local laws or regulations that are applicable to Respondent’s pipeline system. This Agreement is not a permit, or a modification of any permit, under any Federal, State, or local laws or regulations. Respondent remains responsible for achieving and maintaining compliance with all applicable Federal, State, and local laws, regulations and permits. 9. This Agreement does not create rights in, or grant any cause of action to, any third party not party to this Agreement. The U.S. Department of Transportation is not liable for any injuries or damages to persons or property arising from acts or omissions of Respondent or its officers, employees, or agents carrying out the work required by this Agreement. Respondent agrees to hold harmless the U.S. Department of Transportation, its officers, employees, agents, and representatives from any and all causes of action arising from any acts or omissions of Respondent or its contractors in carrying out any work required by this Agreement. 10. Except as noted below, Respondent neither admits nor denies any allegation or conclusion in the Notice, the NOA, or this Agreement. Respondent agrees for purposes of this Agreement to accept the findings of violation and to comply with the terms of this Agreement. 11. During informal discussions, Respondent presented evidence showing that the number of prior offenses during the 5-year period preceding the issuance of the Notice was incorrectly calculated in the Pipeline Safety Violation Report (Violation Report). As such,#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 6PHMSA agrees to reduce the number of prior offenses during the 5-year period preceding the Notice date from 4 to 3 for Items 1, 2, 3, 5, and 7. 12. Except as set forth herein, this Agreement does not constitute a finding of violation of any other federal law or regulation and may not be used in any civil proceeding of any kind as evidence or proof of any fact, fault or liability, or as evidence of a violation of any law, rule, regulation, or requirement, except in a proceeding to enforce the provisions of this Agreement or in future PHMSA enforcement actions. 13. Upon issuance of the Consent Order, the Parties agree to the following terms. II. Notice - Warning Item 14. Item 6: 49 C.F.R. § 195.446(c)(4): The Notice alleged that Respondent failed to test the Supervisory Control and Data Acquisition (SCADA) backup servers at the Linden, Hebert, and Greensboro field operations control rooms at least once each calendar year, but at intervals not to exceed 15 months, for the years 2017, 2018, and 2019. This Item was brought as a warning and does not constitute a finding of violation for any purpose. Respondent was advised to promptly correct this issue going forward, and that a failure to do so may result in future enforcement action. Respondent did not contest this Warning Item. Following PHMSA’s inspection and without admission, Respondent developed procedures and performed tests of its servers to meet the requirements of this section. The Director found that the procedures and tests for Linden, Hebert and Greensboro are adequate. III. Notice - Findings of Violation 15. Item 1: 49 C.F.R. § 195.446(a): The Notice alleged that Respondent failed to follow its procedure, ADM-CPC-008 Rev.2 7/1/2019 Point-To-Point Verification, when documenting a point-to-point verification between SCADA displays and related field equipment at Linden Station in calendar year 2019. Respondent neither admits nor denies the allegation of violation for this Item, but, for purposes of settlement, agrees to accept the finding of violation. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(a). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent. During informal discussions, Respondent, without admission, provided updated procedures to address this Item and presented evidence showing that the failure to document a point-to-point verification between SCADA displays and related field equipment at Linden Station was a records violation, not an activities violation. As such, PHMSA agrees to amend the nature of the offense to a records violation. 16. Item 2: 49 C.F.R. § 195.446(a): The Notice alleged that Respondent failed to follow its procedures when conducting and documenting point-to-point verifications in Safety Life Cycle Management (SLM) system for Safety Related Alarms to ensure alarms are accurate and support safe pipeline operations. Respondent neither admits nor denies the allegation of violation for this Item, but has without admission updated its procedures to address this Item. Further, for purposes of settlement, Respondent agrees to accept the finding of violation. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(a). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent.#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 717. Item 3: 49 C.F.R. § 195.446(a): The Notice alleged that Respondent failed to complete and document verifications of alarm set-point and alarm descriptions in compliance with its procedures when associated field instruments were calibrated or changed for safety related points at the Greensboro facility. The Notice also alleged that for the years 2017, 2018, and 2019, Respondent was not able to verify that all safety-related alarm set-point values and alarm descriptions were correct. Respondent did not contest this Item. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(a). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent. 18. Item 4: 49 C.F.R. § 195.446(a): The Notice alleged that Respondent failed to provide a procedure to satisfy the requirements of 49 C.F.R. § 195.446(e)(3), which requires verification of the correct safety-related alarm set-point values and alarm descriptions when associated field instruments are calibrated or changed and at least once each calendar year not to exceed 15 months. Respondent neither admits nor denies the allegation of violation for this Item but has without admission updated its procedures to address this Item. Further, for purposes of settlement, Respondent agrees to accept the finding of violation. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(a). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent. 19. Item 5: 49 C.F.R. § 195.446(c)(3): The Notice alleged that Respondent failed to “test and verify its internal communication plan to provide adequate means for manual operation of the pipeline safely at least once each calendar year, not to exceed 15 months,” as required by 49 C.F.R. § 195.446(c)(3), at Linden and Hebert in 2017, 2018, and 2019, Greensboro in 2018 and 2019, Alpharetta in 2017, and Baton Rouge, Collins, and Charlotte in 2018, 2019, 2020. Specifically, the Notice alleged that Respondent’s plan is required to include communications for monitoring and manual operation of the pipeline which may include manual shut down and/or start up in the event of SCADA loss as well as a method for leak detection. 20. Respondent maintains that, in compliance with 49 C.F.R. § 195.446(c)(3), it tested and verified its internal communication plan annually, not to exceed 15 months, through actual events at other locations on its system. While testing based on actual events may be used to satisfy the requirements of § 195.446(c)(3), PHMSA maintains that Respondent’s failure to test and verify an internal communication plan at all of its control room locations is a violation. Respondent expressly denies that its alleged failure to comply with the regulation was a deliberate decision as set forth in the Violation Report. Further, PHMSA is not alleging that Respondent did anything to cause the May 7, 2021, criminal cyberattack on Respondent’s pipeline system and PHMSA acknowledges that the criminal cyberattack has no bearing on the underlying facts giving rise to the allegation; i.e., the criminal cyberattack was not a factor causing or contributing to the underlying alleged non-compliance. 21. Prior to the Notice, Respondent believed that testing and verification of the internal communication plan was required in a single control room operating under the same control room management plan. Based on discussions with PHMSA since issuance of the Notice, Respondent acknowledges the importance of having an internal communication plan which has been tested and verified at all control rooms to facilitate manual operations. Upon further discussion between the Parties, Respondent (1) updated its plans and procedures to address this Item, (2) tested and verified its internal communication plan in each of its control#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 8rooms in 2022, and (3) has and will continue to test and verify its internal communication plan in each applicable control room on an annual basis, not to exceed 15 months. Respondent provided its updated plans and procedures and testing and verification records to Central Region for review. Respondent neither admits nor denies the allegation of violation for this Item, but, for purposes of settlement, agrees to accept the finding of violation. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(c)(3). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent. 22. Item 7: 49 C.F.R. § 195.446(e)(2): The Notice alleged that Respondent, for the control rooms in Greensboro, Hebert, and Linden, failed to identify and record, at least monthly, all points affecting safety that had been taken off scan in the SCADA host; all points that have had alarms inhibited; or that have had forced or manual values for periods of time exceeding that required for associated maintenance or operating activities for the years 2017, 2018 and 2019. Respondent neither admits nor denies the allegation of violation for this Item, but, for purposes of settlement, agrees to accept the finding of violation. As such, PHMSA finds a violation of 49 C.F.R. § 195.446(e)(2). This violation may be considered by PHMSA as a prior offense in any future PHMSA enforcement action taken against Respondent. During informal discussions, Respondent without admission provided updated procedures to address PHMSA’s concern and presented evidence showing that this offense was a records violation, not an activities violation. Respondent also presented evidence showing that its failure to comply with the regulation was not deliberate. As such, PHMSA agrees to amend the nature of the offense to a records violation and reduce the culpability assigned to it. IV. Civil Penalty 23. Item 1: The Notice proposed assessing a civil penalty in the amount of $31,100 for Item 1. As noted in paragraphs 11 and 15, during the informal meetings between the Parties, Respondent provided additional information showing that the number of prior offenses was incorrectly calculated in the Violation Report and that the nature of the offense was a records violation. Based on the information provided, PHMSA agrees to reduce the proposed civil penalty. As such, Respondent shall pay a reduced civil penalty in the amount of $24,100, to be paid in full no later than 30 days from the Effective Date of this Agreement. 24. Item 2: The Notice proposed assessing a civil penalty in the amount of $31,700 for Item 2. As noted in paragraph 11, during the informal meetings between the Parties, Respondent provided additional information showing that the number of prior offenses was incorrectly calculated in the Violation Report. Based on the information provided, PHMSA agrees to reduce the proposed civil penalty. As such, Respondent shall pay a reduced civil penalty in the amount of $28,300, to be paid in full no later than 30 days from the Effective Date of this Agreement. 25. Item 3: The Notice proposed assessing a civil penalty in the amount of $31,700 for Item 3. As noted in paragraph 11, during the informal meetings between the Parties, Respondent provided additional information showing that the number of prior offenses was incorrectly calculated in the Violation Report. Based on the information provided, PHMSA agrees to reduce the proposed civil penalty. As such, Respondent shall pay a reduced civil#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 9penalty in the amount of $28,300, to be paid in full no later than 30 days from the Effective Date of this Agreement. 26. Item 5: The Notice proposed assessing a civil penalty in the amount of $846,300 for Item 5. Prior to the Notice, Respondent believed that testing and verification of the internal communication plan was required in a single control room operating under the same control room management plan and for this reason expressly denies that its alleged failure to comply with the regulation was deliberate. As noted in paragraph 11, during the informal meetings between the Parties, Respondent provided additional information showing that the number of prior offenses was incorrectly calculated in the Violation Report. Based on the information provided, PHMSA agrees to reduce the proposed civil penalty. As such, Respondent shall pay a reduced civil penalty in the amount of $842,900, to be paid in full no later than 30 days from the Effective Date of this Agreement. 27. Item 7: The Notice proposed assessing a civil penalty in the amount of $45,600 for Item 7. As noted in paragraphs 11 and 22, during the informal meetings between the Parties, Respondent provided additional information showing that the number of prior offenses was incorrectly calculated in the Violation Report, that the offense was a records violation, and that its failure to comply with the regulation was not deliberate. Based on the information provided, PHMSA agrees to reduce the proposed civil penalty. As such, Respondent shall pay a reduced civil penalty in the amount of $24,800, to be paid in full no later than 30 days from the Effective Date of this Agreement. V. Compliance Order 28. Items 1, 2, 3, 4, and 7: The Notice proposed certain compliance order actions to address the allegations of non-compliance. Respondent, in response to the Notice, requested an informal discussion with PHMSA to discuss the terms of proposed compliance order. During these discussions, Respondent, without admitting or denying the allegations of violation, presented evidence of the remedial actions, including updated and revised procedures, it undertook in response to Notice Items 1, 2, 3, 4, and 7 and associated PCO actions. Based upon these remedial actions and revised procedures, PHMSA finds the terms of the proposed compliance order for these Items have been satisfied. 29. Item 5: The Notice proposed certain compliance order actions to address the allegation of non-compliance. Respondent, without admitting or denying the allegation of violation, completed and provided documentation, which has been approved by the Director, Central Region, of Respondent’s tests and verifications of its internal communication plan to provide adequate means for manual operation for all eleven (11) CRM control rooms (including Alpharetta and field) in 2022, and for five (5) of the eleven (11) control rooms thus far in 2023. Based upon these remedial actions, PHMSA finds the terms of the proposed compliance order for Item 5 have been satisfied.#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 10VI. NOA – Findings of Inadequacy 30. Item 1: 49 C.F.R. § 195.446(a): The NOA alleged that Respondent failed to define the roles and responsibilities of controllers related to time absent from the console when conducting a shift turnover during normal operations, and the procedures were therefore inadequate to comply with 49 C.F.R. § 195.446(b)(1). Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. 31. Item 2: 49 C.F.R. § 195.446(a): The NOA alleged that Respondent failed to provide specific actions for the controller, or control room staff, to take including communicating with others, and were therefore inadequate to comply with 49 C.F.R. § 195.446(b)(3). Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. 32. Item 3: 49 C.F.R. § 195.446(a): The NOA alleged that Respondent failed to have a shift hand-over process for instances when an outgoing controller was not/unable to be present and was therefore inadequate to comply with 49 C.F.R. § 195.446(b)(4). Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. 33. Item 4: 49 C.F.R. § 195.446(b)(4): The NOA alleged that Respondent’s procedure at the Greensboro Control room for periodically changing physical responsibility failed to cover the local desk operation where the pipeline is shut down, and when the console logged out and controllers leave the console unattended on weeknights and weekends, and was therefore inadequate to comply with 49 C.F.R. § 195.446(b)(4). The NOA also alleged there was no alternative in the procedure for monitoring alarms normally assigned to the local desk during the unattended console times. Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. 34. Item 5: 49 C.F.R. § 195.446(a): The NOA alleged that Respondent failed to address the testing and verification of the internal communications plan to facilitate manual operation and that the procedures were therefore inadequate to comply with the requirements of 49 C.F.R. § 195.446(c)(3). Respondent neither admits nor denies the NOA allegation, but for purposes of settlement, has amended its CRM plans and certain site-specific procedures, which have been approved by the Director, Central Region, to require testing and verification of an internal communication plan to facilitate manual operations of the pipeline safely once each calendar year, not to exceed fifteen (15) months. Specifically, the Director, Central Region, reviewed and approved Respondent’s amended CRM manual and related attachments regarding the internal communication plan for manual operation. The operator provided certain amended site-specific operating procedures for each of Respondent’s eleven (11) control rooms and one (1) representative records log, which were reviewed and approved by PHMSA. For the amended site-specific operating procedures for the five (5) control rooms tested in 2023 using Respondent’s amended 2022 procedures (i.e., Collins, Charlotte, Greensboro, Linden, and#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 11Woodbury), the Director found that these plans and procedures reviewed, as revised, are adequate to provide adequate means for manual operations and no further action is necessary as to those specific procedures. Respondent will provide the remaining revised site-specific operating procedures associated with six (6) of its control rooms (i.e., Alpharetta, Baton Rouge, Hebert, Houston, Lake Charles, and Port Arthur) to the Director, Central Region, for review and approval within forty-five (45) calendar days of the Effective Date. The Central Region will review and notify Respondent whether the revised procedures are adequate within thirty (30) business days of their receipt. 35. Item 6: 49 C.F.R. § 195.446(a): The NOA alleged that Respondent’s procedure for testing the backup SCADA systems failed to require testing the field servers and was therefore inadequate to comply with 49 C.F.R. § 195.446(c)(4). Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. 36. Item 7: 49 C.F.R. § 195.446(e)(2): The NOA alleged that Respondent’s procedure for the monthly review of safety related alarms failed to define the required process of review. Respondent neither admits nor denies the NOA allegation, but, for purposes of settlement, agreed to amend its procedures. During the Parties’ informal discussions, Respondent without admission provided PHMSA an updated version of its procedures that address the alleged inadequacy. As such, PHMSA agrees to withdraw the allegation of inadequacy for this Item. 37. Item 8: 49 C.F.R. § 195.446(h): The NOA alleged that Respondent’s controller training program (CRM Plan Revision 13.0 7/2/2020) failed to require a review at least once each calendar year, but at intervals not to exceed fifteen (15) months, and was therefore inadequate to comply with 49 C.F.R. § 195.446(h). Respondent did not contest this allegation of inadequacy and provided revised procedures for review by the Director, Central Region. The Director found that the procedures, as revised, are adequate and no further action is necessary. VII. Enforcement 38. This Agreement is subject to all enforcement authorities available to PHMSA under 49 U.S.C. § 60101, et seq., and 49 C.F.R. Part 190, including administrative civil penalties under 49 U.S.C. § 60122, of up to $239,142 per violation for each day the violation continues (as may be adjusted for inflation) and referral of the case to the Attorney General for judicial enforcement, if PHMSA determines that Respondent is not complying with the terms of this Agreement in accordance with determinations made by the Director, or if appealed, in accordance with decisions of the Associate Administrator. The maximum civil penalty amounts are adjusted annually for inflation. See 49 C.F.R. § 190.223. VIII. Force Majeure 39. Respondent agrees to perform all the terms of this Agreement within the timeframes established within this Agreement, including pursuant to modifications under Section XI, unless performance is delayed by a force majeure. For purposes of this Agreement, a force#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 12majeure is defined by an event arising from causes beyond the control of the Respondent, or any entity controlled by Respondent or Respondent’s contractors, which delays or prevents performance of any obligation under this Agreement despite Respondent’s commercially reasonable efforts to fulfill the obligation. 40. If a force majeure event occurs or has occurred that may delay the performance of any term of this Agreement beyond the approved timeframe, Respondent shall notify the Director, in writing, within five (5) business days of when Respondent knew that the event might cause a delay. Such notice shall identify the cause of the delay or anticipated delay and the anticipated duration of the delay, state the measures taken or to be taken to prevent or minimize the delay, and estimate the timetable for implementation of those measures. Failure to comply with the notice provision of this paragraph and to undertake reasonable efforts to avoid and minimize the delay shall waive any claim of force majeure by Respondent. 41. If the Director determines, upon notification by Respondent, that a delay or anticipated delay in performance is or was attributable to a force majeure, then the Director will extend the time period for the performance of that term for a reasonable period. The Director will notify Respondent, in writing, of the length of any extension of performance of such terms affected by the force majeure. Any such extensions shall not alter Respondent’s obligation to perform or complete other terms of this Agreement which are not affected by the force majeure. IX. Dispute Resolution 42. The Director and Respondent will informally attempt to resolve any disputes arising under this Agreement, including but not limited to any decision of the Director. If Respondent and the Director are unable to informally resolve the dispute within fifteen (15) calendar days after the dispute is first raised, in writing, to the Director, Respondent may submit a written request for a determination resolving the dispute from the Associate Administrator for Pipeline Safety, PHMSA. Such request must be made in writing and provided to the Director, counsel for the Central Region, and to the Associate Administrator for Pipeline Safety, no later than ten (10) calendar days from the fifteen (15) day deadline for informal resolution referenced in this paragraph. Along with its request, Respondent must provide the Associate Administrator with all information Respondent believes is relevant to the dispute. Decisions of the Associate Administrator under this paragraph will constitute final agency action. The existence of a dispute and PHMSA’s consideration of matters placed in dispute will not excuse, toll, or suspend any term or timeframe for completion of any work to be performed under this Agreement during the pendency of the dispute resolution process, except as agreed by the Director or the Associate Administrator in writing, or ordered by a court of competent jurisdiction. X. Effective Date 43. The term “Effective Date,” as used herein, is the date on which the Consent Order is issued by the Associate Administrator, PHMSA, incorporating the terms of this Agreement. XI. Modification 44. The terms of this Agreement may be modified by mutual agreement of the Parties. Such modifications must be in writing and signed by both parties.#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 13XII. Termination 45. The Agreement shall not terminate until the Director confirms, in writing, that the Agreement is terminated in accordance with this paragraph. XIII. Recordkeeping and Information Disclosure 46. Respondent agrees to maintain records demonstrating compliance with all requirements of this Agreement for a period of at least five (5) years from the date of termination of the Agreement. XIV. Review and Approval Process 47. With respect to any submission under Section V (Compliance Order) or Section VI (NOA – Findings of Inadequacy) of this Agreement that requires the approval of the Director, the Director may: (a) approve, in whole or in part, the submission; (b) approve the submission on specified, reasonable conditions; (c) disapprove, in whole or in part, the submission; or (d) any combination of the foregoing. If the Director approves, approves in part, or approves with conditions, Respondent will take all actions as approved by the Director, subject to Respondent’s right to invoke the dispute resolution procedures with respect to any conditions the Director identifies. If the Director disapproves all or any portion of the submission, the Director will provide Respondent a written notice of the deficiencies. Respondent will correct all deficiencies within the time specified by the Director and resubmit it for approval. XV. Ratification 48. The Parties’ undersigned representatives certify that they are fully authorized to enter into the terms and conditions of this Agreement and to execute and legally bind such party to this document. 49. The Parties hereby agree to all conditions and terms of this Agreement. [Signature Lines on Following Page]#
32022028NOA_Consent Agreement and Order_09012023_(20-173062)_text.pdf, page 14For Colonial Pipeline Company: Date For PHMSA: Gregory A. Ochs, Director, Central Region, OPS Date#
32022028NOA_Closure Letter_04092024_(20-173062)_text.pdf, page 1Official PDFVIA ELECTRONIC MAIL TO: MLittle@colpipe.com; QFrazier@colpipe.com; akolar@colpipe.com; catherine.little@bracewell.com April 9, 2024 Melanie Little President and Chief Executive Officer Colonial Pipeline Company 1000 Lake Street Alpharetta, Georgia 30009 RE: CPF 3-2022-028-NOA Dear Ms. Little: From January 27 through November 12, 2020, a representative of the Pipeline and Hazardous Materials Safety Administration (PHMSA) pursuant to Chapter 601 of 49 United States Code inspected Colonial Pipeline Company’s (Colonial) Control Room Management (CRM) procedures in Linden, New Jersey; Hebert, Louisiana; Greensboro, North Carolina; and Alpharetta, Georgia. As a result of the inspection, Colonial was issued a Notice of Amendment on May 27, 2022, which proposed amendment of its procedures. Colonial submitted its amended procedures on March 4, 2024. My staff has reviewed the amended procedures and it appears that the inadequacies outlined in the Notice of Amendment have been corrected. This letter is to inform you no further action is necessary and this case is now closed. Thank you for your cooperation. Sincerely, Gregory A. Ochs Director, Central Region, Office of Pipeline Safety Pipeline and Hazardous Materials Safety Administration cc: Quintin Frazier, Director, Pipeline Compliance, Colonial Pipeline Co. QFrazier@colpipe.com Angela D. Kolar, Senior Vice President, Operations, EHS, & Compliance, Colonial Pipeline Co. , akolar@colpipe.com Catherine Little, Esq., Counsel for Colonial Pipeline Co., Bracewell LLP, catherine.little@bracewell.com#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 1Official PDFNOTICE OF AMENDMENT VIA ELECTRONIC MAIL TO: jblount@colpipe.com; QFrazier@colpipe.com May 27, 2022 Joseph A. Blount President & CEO Colonial Pipeline Company 1185 Sanctuary Parkway, Suite 100 Alpharetta, GA 30009 CPF 3-2022-028-NOA Dear Mr. Blount: From January 27, through November 12, 2020, a representative of the Pipeline and Hazardous Materials Safety Administration (PHMSA) pursuant to Chapter 601 of 49 United States Code inspected Colonial Pipeline Company’s (Colonial’s) Control Room Management (CRM) procedures in Linden, NJ, Hebert, LA, Greensboro, NC, and Alpharetta, GA. This Notice is in response to PHMSA’s CRM Initiative, a national level program that includes inspectors from every region. As a result, you may have received this Notice from a different Regional Director than typical because the CRM Initiative inspections are currently separate from the standard inspection program. Notices and correspondence from other types of inspections will remain unchanged. On the basis of the inspection, PHMSA has identified the apparent inadequacies found within Colonial’s plans or procedures, as described below: 1. § 195.446 Control Room (a)General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedure required by §195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4) and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than august 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph.#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 2Colonial procedure ADM-CPC-006 Revision 13.0: 7/2/2020 (CRM Plan) fails to define the roles and responsibilities of controllers related to time absent from the console when conducting a shift turnover during normal operations, and is therefore inadequate to comply with the requirements of § 195.446(b)(1). In the CRM Plan, two sections cover the unplanned changes in controller responsibilities: Control Room Area Activities and Controller Shift Turnover and Handover of Responsibility Method. • Control Room Area Activities states: “If the controlling individual must leave the control area, then another qualified individual must assume responsibility for any operations that occurs during this period and shall document this turnover of responsivity in the narrative log. This may also include responding to Abnormal or Emergency Operating Conditions (AOC’s or EOC’s). An appropriate exchange of information concerning system status along with any current or upcoming operations that may occur during the normal controlling individual’s absence.” (CRM Plan, at 6) • Controller Shift Turnover and Handover of Responsibility Method states: “A shift turnover/handover of responsivity shall not include when a Controller steps away for a brief, reasonable amount of time (i.e., restroom break, personal call, etc.) where the Controller, or another qualified individual may monitor the console, line or system and respond to audible or visible alarm indication.” (CRM Plan, at 12). The procedures did address covering a console during a "reasonable" amount of time to cover breaks and to provide an “appropriate exchange”. The words "reasonable" and “appropriate” did not impose a time limit or require specific information to be exchanged. These ambiguities left the interpretation to the individual, where the meaning can vary from one controller to the next. Colonial must amend its procedures to adequately define time limits during short absences before a shift change is required and the information to be shared during these short breaks to clarify the roles and responsibilities of the controller. 2. § 195.446 Control room management. (a)General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedure required by § 195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures require€ by paragraphs (c)(1) through (4), (d)(1), (d)(4) and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than august 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. Colonial’s procedures for the evacuation of the control room at both field and Alpharetta Main locations failed to provide specific actions for the controller, or control room staff, to take including communicating with others, and were therefore inadequate to comply with the requirements of § 2#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 3195.446(b)(3). The procedures leave out steps that are critical in the flow of the evacuation. Additionally, there are undefined job responsibilities for the lead controller for this emergency condition. The combination of undefined roles, responsibilities and procedural steps is inadequate to comply with PHMSA’s regulation. During the inspection, PHMSA identified that EOP-CC-K20 provided for the controller to leave, shut down the pipes, if they can, and call security. There was a step that required the controller to call facilities VG4 group, but did not prescribe a specific message when making the notification for evacuation (a pre- determined and programmed notification system). Then the controller called the Colonial Supervision to notify of the evacuation. Procedure COP-CPC-005 Relocation of Pipeline Control identified that the supervisor determined the need to evacuate to the backup center. This call was the responsibility of the lead controller, but it was not defined in this procedure, in the COP-CPC-005 Relocation of Pipeline Control procedure, or in the roles and responsibilities sections. This call was crucial as the supervisor determined the need to evacuate to the backup center under Colonial’s procedure. The two procedures were co-dependent but did not fully support each other. Lastly, Colonial did mention evacuation in the introduction of the CRM Plan, but the CRM Plan did not have a comprehensive evacuation plan for field locations. The field control rooms did not have backup control centers, but instead had muster sites for each location. This was not the same as managing absence from the console. Evacuation may be short term or long term and these considerations were not available through a procedure. Colonial must amend its procedures to provide clear roles for controllers and control room staff during an emergency, even if the controller and staff were not the first to detect the emergency, including the controller’s and control room staff’s responsibilities to take specific actions and communicate with others when needing to evacuate the control room. 3. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedure required by §195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures require€ by paragraphs (c)(1) through (4), (d)(1), (d)(4) and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than august 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. Colonial’s procedures failed to have a shift hand-over process for instances when an outgoing controller was not/unable to be present and was therefore inadequate to comply with the requirements of § 195.446(b)(4). While the operator stated they would shut down the console pipelines until they get a replacement for shift coverage or just keep the line shut down for the night, Colonial did not have a procedure to follow in making that decision. 3#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 4Colonial must amend its procedures to include a method of recording controller shift-changes and any hand-over of responsibility between controllers in the event a controller is unable to complete her shift and a replacement is not readily available. 4. § 195.446 Control room management. (a) . . . . (b) Roles and Responsibilities. Each operator must define the roles and responsibilities of a controller during normal, abnormal, and emergency operating conditions. To provide for a controller’s prompt and appropriate response to operating conditions, and operator must define each of the following: (1) . . . . (4) A method of recording controller shift-changes and any hand-over of responsibility between controllers; Colonial’s procedure for periodically changing physical responsibility failed to cover the local desk operation, for the Greensboro Control room, where the pipeline is shut down, and when the console logged out and controllers leave the console unattended on weeknights and weekends, and is therefore inadequate to comply with the requirements of § 195.446(b)(4). The procedure required shift change documents and conversations for the normal 24/7/365 consoles, but the local desk is a different operation involving two daytime shifts: approximately 6 a.m. to 2 p.m., and 2 p.m. to 10 p.m. In practice, there is a shift change document and conversation completed for the 2 p.m. weekday shift change, but no document created at the end of the 2 to 10 p.m. shift. A shift change document is required to provide the next oncoming controller a summary of the operations. Additionally, there was no alternative, in the procedure, for monitoring alarms, normally assigned to the local desk during the unattended console times. Even if the pipeline is shut down, it was still considered in operation and therefore must have been monitored for leaks at a minimum. Colonial must amend its procedure to adequately specify the method of recording shift changes and responsibility changes for controllers as required by § 195.446(b)(4). 5. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedure required by §195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures required by paragraphs (c)(1) through (4), (d)(1), (d)(4) and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than august 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. Colonial’s procedures failed to address the testing and verification of the internal communications plan for manual operation and was therefore inadequate to comply with the requirements of § 195.446(c)(3). 4#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 5Colonial’s CRM plan only requires “[T]esting and Verification of the internal communication and manual operations procedures for a controlled shutdown of the pipeline(s)” Colonial indicated that because its system is so large, having a procedure for all contingencies would be too great a burden. So, instead, the operator had opted to shut down the pipeline and write specific procedures for the manual operation of the system. Colonial had performed such operations in the past. While actual events are acceptable methods to count as a test of a plan or process, an operator must conduct a test within the required intervals. Therefore, simply not having an actual event would not relieve the operator from the testing requirement. Additionally, Colonial had two procedures, EOP-COC-K30 Complete Loss of Communication (SCADA and Voice) and AOP-CPC-J32 (SCADA only failure). The second procedure did not require a complete shut down, but accommodated sending personnel to field locations to monitor or control and report back to the controller or send monitor/control to Alpharetta. AOP-CPC-J32, as written, required a plan and testing requirements, which were not provided in the procedure. The CRM plan did not reference the second procedure, AOP-CPC-J32. Colonial must amend its procedures to require testing of manual operations of the pipeline once each calendar year not to exceed 15 months. 6. § 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedure required by §195.402. An operator must develop the procedures no later than August 1, 2011 and must implement the procedures according to the following schedule. The procedures required by paragraphs (b), (c)(5), (d)(2) and (d)(3), (f) and (g) of this section must be implemented no later than October 1, 2011. The procedures require€ by paragraphs (c)(1) through (4), (d)(1), (d)(4) and (e) must be implemented no later than August 1, 2012. The training procedures required by paragraph (h) must be implemented no later than august 1, 2012, except that any training required by another paragraph of this section must be implemented no later than the deadline for that paragraph. Colonial's procedure for testing the backup SCADA systems failed to require testing the field servers and was therefore inadequate to comply with the requirements of § 195.446(c)(4). Colonial interpreted the field servers (Hebert, Linden and Greensboro) as "redundant" and therefore outside the testing requirements for backup SCADA systems. The two servers were “redundant” in that they had identical hardware, software, programming, etc. Theoretically, if one server failed, the other server should, by design, automatically and instantaneously take control of the operation. However, classifying a pair of servers as “redundant” does not exempt them from the testing requirement. The regulation requires each operator to “[t]est any backup SCADA systems." “Backup SCADA systems” is a broad term that considers all facets and factors of a control room operation. Without these assets, the system must be operated in manual mode. The two servers, as a pair, were a backup SCADA system and Colonial must test them and verify and document that, if one failed, the other successfully switched during normal operations. Colonial must amend its procedure to provide a test of the field SCADA servers once calendar year not to exceed 15 months. 5#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 6PHMSA notes that after the field inspections, Colonial addressed this inadequacy by developing a procedure for testing the SCADA field servers. No further action is required. 7. § 195.446 Control room management. (a) . . . . (e) Alarm management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (1) . . . . (2) Identify at least once each calendar month points affecting safety that have been taken off scan in the SCADA host, have had alarms inhibited, generated false alarms, or that have had forced or manual values for periods of time exceeding that for required associated maintenance or operating activities. Colonial's procedure for the monthly review of safety related alarms failed to define the process of review as required by § 195.446(e)(2). Specifically, Colonial did not include the process of review in its CRM Plan Revision 13.0 7/2/2020 or its Alarm Management Plan Revision 19.0 7/2/2020. Colonial had included roles and responsibilities of various positions for the review, but did not provide details on how the personnel would conduct the reviews. The Alarm Management Plan described how these events were to be logged, but Colonial did not include any language related to the form, format, review purposes, or type of analysis to be performed. Also, the Alpharetta control room SCADA system and field control room SCADA (iFix) systems were different. Colonial only applied the monthly review to Alpharetta because Colonial indicated the iFix system did not have the capability to manage points (off scan, inhibit, force or manual), and collect and review. However, the iFix systems may, in the RTU or PLC, take points off-scan, and inhibit or place a point in a forced or manual condition. Therefore, these devices were part of the SCADA system and the iFix systems’ events needed to be included in the scope of the monthly review. Colonial did not mention this in their procedure, and they stated that since their SCADA did not have these capabilities, they had not developed a procedure to address this requirement. Colonial must amend its procedures to define the process for capturing the events, gathering for review, analyzing and documenting points affecting safety that have been taken off scan in the SCADA host, had alarms inhibited, generated false alarms, or that had forced or manual values for periods of time exceeding that for required associated maintenance or operating activities. Additionally, Colonial must amend its procedures to identify any deviations from this requirement, especially if it relates to system capabilities. 8. § 195.446 Control room management. (a) . . . . (h) Training. Each operator must establish a controller training program and review the training program content to identify potential improvements at least once each calendar year, but at intervals not to exceed 15 months. An operator’s program must provide for training each controller to carry out the roles and responsibilities defined by the operator. In addition, the training program must include the following elements: Colonial's controller training program (CRM Plan Revision 13.0 7/2/2020) failed to require a review at least once each calendar year, but at intervals not to exceed 15 months and was therefore inadequate to comply with the requirements of § 195.446(h). Colonial had separate training paths. One path was for 6#
32022028NOA_Notice of Amendment_05272022_(20-173062)_text.pdf, page 7the Alpharetta Main Control Room group, developed and managed by Control Room staff, and the other was for Field personnel, developed and managed by the corporate training group. The corporate training group did not have a specific procedure to review any field training content, under their responsibility. A representative from the corporate training group indicated they utilized a “continual effectiveness review” process, but the operator could not provide any written procedure that included the annual review requirement. Colonial had also decided its compliance group would audit the training programs, but there was no written procedure in place for this group to perform such an audit. While accountability and responsibility for the training programs and content fell under separate organizations, both must have adhered to the review requirements under § 195.446(h). Colonial must amend its procedures to require a controller training program review at least once each calendar year, at intervals not to exceed 15 months. The procedure must include sufficient detail and documentation to identify what was reviewed, including all findings and improvements. Response to this Notice This Notice is provided pursuant to 49 U.S.C. § 60108(a) and 49 C.F.R. § 190.206. Enclosed as part of this Notice is a document entitled Response Options for Pipeline Operators in Enforcement Proceedings. Please refer to this document and note the response options. Be advised that all material you submit in response to this enforcement action is subject to being made publicly available. If you believe that any portion of your responsive material qualifies for confidential treatment under 5 U.S.C. § 552(b), along with the complete original document you must provide a second copy of the document with the portions you believe qualify for confidential treatment redacted and an explanation of why you believe the redacted information qualifies for confidential treatment under 5 U.S.C. § 552(b). Following the receipt of this Notice, you have 30 days to submit written comments, revised procedures, or a request for a hearing under § 190.211. If you do not respond within 30 days of receipt of this Notice, this constitutes a waiver of your right to contest the allegations in this Notice and authorizes the Associate Administrator for Pipeline Safety to find facts as alleged in this Notice without further notice to you and to issue an Order Directing Amendment. If your plans or procedures are found inadequate as alleged in this Notice, you may be ordered to amend your plans or procedures to correct the inadequacies (49 C.F.R. § 190.206). If you are not contesting this Notice, we propose that you submit your amended procedures to my office within 30 days of receipt of this Notice. This period may be extended by written request for good cause. Once the inadequacies identified herein have been addressed in your amended procedures, this enforcement action will be closed. It is requested that Colonial Pipeline maintain documentation of the safety improvement costs associated with fulfilling this Notice of Amendment (preparation/revision of plans, procedures) and submit the total to Gregor A. Ochs, Director, Central Region, Pipeline and Hazardous Materials Safety Administration. In correspondence concerning this matter, please refer to CPF 3-2022-028-NOA and, for each document you submit, please provide a copy in electronic format whenever possible. Sincerely, Gregory A. Ochs Director, Central Region, OPS Pipeline and Hazardous Materials Safety Administration cc: Quintin Frazier, Director, Pipeline Compliance QFrazier@colpipe.com Enclosure: Response Options for Pipeline Operators in Enforcement Proceedings 7#
This material provides agency context. It does not replace binding regulatory text, and its legal effect depends on the underlying authority and facts.