CPF 32022035NOA
CPF 32022035NOA
party submissionOfficial PDF32022035NOA_Operator Response to Notice_05242022_(21-201004).pdf#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 1Official PDFNOTICE OF AMENDMENT VIA ELECTRONIC MAIL TO: jmcclure@nnogc.com and jrobertson@nnogc.com March 1, 2022 Mr. James McClure, CEO Navajo Nation Oil and Gas PO Box 4439 Window Rock, AZ 86515 CPF 3-2022-035-NOA Dear Mr. McClure: From May 24 to 28, 2021, a representative of the Pipeline and Hazardous Materials Safety Administration (PHMSA), pursuant to Chapter 601 of 49 United States Code, inspected Navajo Nation Oil and Gas (NNOG Running Horse Pipeline or NNOG-RHPT) procedures for Control Room Management in Montezuma Creek, Utah. On the basis of the inspection, PHMSA has identified the apparent inadequacies found within NNOG-RHPT plans or procedures, as described below: 1. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (b) Roles and responsibilities. Each operator must define the roles and responsibiliites of a controller during normal, abnormal, and emergency operating conditions. To provide for a controller’s prompt and appropriate response to operating conditions, an operator must define each of the following: (5) The roles, responsibiliites and qualifications of other who have the authority to direct or supersede the specific technical actions of controllers.#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 2NNOG Running Horse Pipelines procedure RHPT-500: Control Room Management Plan Version 3 April 2021 (CRM Plan) was inadequate because it did not provide guidance for the controller or Control Room Supervisor, who was authorized to direct or supersede the technical actions of a controller, when the control room supervisor implemented their authority, as required by Section §195.446(b)(5). Section 500-2.2.3, of the CRM Plan, states “[T]he Control Room Supervisor has sole authority to direct or supersede specific technical actions of a Controller when necessary to protect people, property or environment.” It did not provide conditions where directing or superseding would be considered or warranted, how is it communicated that the action would be taken and how is the action would be documented. The procedure needs to be amended to provide more structure around the conditions, communication, and documentation for directing and superseding a controller. 2. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (c) Provide adequate information. Each operator must provide its controllers with the information, tools, processes and procedures necessary for the controllers to carry out the roles and responsibilities the operator has defined by performing each of the following: (2) Condtuct a point-to-point verification between SCADA displays and related field equipment when field equipment is added or moved and when other changes that affect pipeline safety are made to field equipment or SCADA displays; NNOG Running Horse Pipeline's CRM Plan was inadequate because it did not provide directions to conduct and document a thorough point-to-point verification. Procedure 500- 3.2 in the CRM plan was high level and was supported by procedure 507-1. However, procedure 507-1 was not referenced and different forms were referenced to document the point-to-point verification. Section 500-3.2 references Calibration Check Form IC-002 used for annual calibration set point and alarm descriptor verification. The operator presented during inspection Form 500.4, which is referenced in procedure 507-1. Form IC-002, while adequate for documenting annual calibrations and changes described in §195.446(e)(3), does not meet the requirements for new or relocated field equipment verifications described in §195.446(c)(2). Form RHPT-500.4 did not provide elements to capture information required for a thorough point-to-point verification. Form 500-4 does not require verification of screens, alarm set point presentations at the right set point and with the correct priority, color, and description. The form also did not include who performed the verification, if the test failed or encountered problems, and if there was resolution to a successful verification. The procedure needs to be amended to reference the appropriate supporting procedures, the correct form(s) for the correct process and all information required to be documented for a successful test and the compliance record. The amendment also needs to include the forms to match the requirements of the procedure.#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 33. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (d) Fatigue mitigation. Each operator must implement the following methods to reduce the risk associated with controller fatigue that could inhibit a controller’s ability to carry out the roles and responsibilities the operator has defined: (1) Establish shift lengths and schedule rotations that provide controllers off-duty time sufficient to achieve eight hours of continuous sleep; NNOG Running Horse Pipeline’s CRM Plan was inadequate because it did not include a process to track and monitor controllers’ time worked on shift and other to to show that shift lengths and schedule rotations are adequate to provide controllers off-duty time sufficient to achieve 8 hours of continuous sleep as required by §195.446(d)(1). The operator was only reviewing payroll cards to evaluate and approve overtime. They were not evaluating hours spent responding to alarm calls or monitoring the system after hours or on weekends. They indicated the only work hours considered, after normal hours, were those where the controller went to the office to respond to a pipeline condition, which was considered overtime pay. There was no process to look forward or review history to ensure compliance with the regulation. Therefore, the procedure needs to be amended to document and review all hours worked, on shift and other, to show that shift lengths and schedule rotations are adequate to provide controllers, and those who can control, off-duty time sufficient to achieve 8 hours of continuous sleep. 4. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (d)Each operator must implement the following methods to reduce the risk associated with controller fatigue that could inhibit a controller’s ability to carry out the roles ad responsibilities of the operator… NNOG Running Horse Pipeline's CRM Plan, procedure 500-4.5.1 was inadequate to review the fatigue program identified in §195.446(d) for effectiveness as required by §195.402(c)(13) because processes had not been developed to collect, analyze and review the criterion defined to determine effectiveness. Also, Form RHPT-500.2, referenced in the procedure, was a sleep log and not designed to document the findings to determine#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 4effectiveness. The procedure needs to be amended to support the review and analysis of the established criteria to determine effectiveness of the fatigue program. The procedure also needs to require that deficiencies identified from the review must be documented and implemented. 5. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (1) Review SCADA safety-related alarm operations using a process that ensures alarms are accurate and support safe pipeline operations; NNOG Running Horse Pipeline's procedure RHPT-501: Alarm Management Plan, Version 3 April 2021(AMP), was inadequate in defining safety related points as required in §195.446(e)(1) because the operator did not have a process for alarm rationalization which would support identification of safety-related points and associated alarms. Section 501-3.1 also stated, “[C]ontrollers and Control Room Supervisor will review all the data points collected by the SCADA system to determine if they are safety-related based on the potential consequence for inaction.” There was no procedure to define how this was to be done. It was determined that the SCADA database and system was installed and developed around 2008 and not much work to rationalize new alarms has been done post commissioning. Fail safe points/alarms were identified in section 501-3.1 and safety related points were identified in Table 1 of Section 501-3.1.2 of the AMP. It appeared the points in Table 1 was a sub-set of the points in Section 501-3.1. Safety-related points are separately listed in procedure 507-1 Point to Point Procedure Version 2 November 2020. There was not a clear tie between the points identified in the AMP and procedure 507-1. They appeared to be independently considered. The AMP section 501-3.5 identified priority levels as low, medium, high and urgent, but the UCOS SCADA database identified priority levels as 1, 2, 3 with no identified 4. The procedure did not correlate the priority naming conventions. The procedure did not reflect the database and there is no definition of which is more urgent; a 1 or a 3. The procedure also failed to provide how the safety related alarms were to be distinguished in the SCADA master database to make them identifiable from other alarms or points to the controllers. The procedure needs to be amended to include a process for alarm rationalization and how the points and alarms will be identified in the master SCADA database. An outcome of alarm rationalization should be criteria to determine safety related points the associated safety related alarms. Since RHPT operates a failsafe system, fail safe alarms should also be#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 5a consideration of the alarm rationalization process. Also, because rationalization should define priority levels for alarm response, the operator also needs to amend the procedure to define alarm priority levels and then the SCADA database must match those defined alarm levels. Alarm levels need to include the associated color and messaging. Additionally, procedures need to address alarm handling requirements to train controllers on their roles and responsibilities when alarms present. 6. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (2) Identify at least once each calendar month points affecting safety that have been taken off scan in the SCADA host, have had alarms inhibited, generated false alarms, or that have had forced or manual valves for periods of time exceeding that required for associated maintenance or operating activities; NNOG Running Horse Pipeline's CRM Plan procedure was inadequate to identify, record, review and analyze, monthly, points affecting safety that have been taken off scan, have had alarms inhibited, generated false alarms, or that have had forced or manual values for periods of time exceeding that required for associated maintenance or operating activities as required by §195.446(e)(2). Section 501-3.2 of the AMP roughly paraphrases the regulation. The procedure did not describe how they would perform the review. It describes documenting the review on form 500.9, which was also identified in 501-3.1.2 Handling Stale Data and 501-3.7 Off-Scan Points 501-3.1.1 Process for Handling/Reviewing Alarms, all which were to be used for the monthly review. While Roles and Responsibilities in Section 500-2.2.1 (Controllers) and 500-2.2.1 (Supervisor) states, "[i]dentifying points that have been taken off-scan, have alarms inhibited, have generated false alarms, or have had manual values," there was no further instructions on how to document false alarms, inhibited alarms or manual/forced points identified for monthly review or analysis. The procedure needs to be amended to provide details on how the monthly review analysis and documentation of points affecting safety that have been taken off scan, have had alarms inhibited, generated false alarms, or that have had forced or manual values for periods of time exceeding that required for associated maintenance or operating activities will be completed. The procedure also needs to include how the data for review will be collected along with the identification and documentation of false alarms.#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 67. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (e) Alarm Management. Each operator using a SCADA system must have a written alarm management plan to provide for effective controller response to alarms. An operator’s plan must include provisions to: (6) Address deficiencies identified through the implementation of paragraphs (e)(1) through (e)(5) of this section. NNOG Running Horse Pipeline’s CRM Plan was inadequate because they did not have a durable tracking method to address deficiencies identified in implementing § 195.446(e)(1) through § 195.446(e)(5) have been resolved, as required by §195.446(e)(6). The operator’s practice is to document everything in either the logbook or emails; this makes it difficult to track issues and the follow-up to verify whether issues are resolved. Their SCADA system is maintained by a third party (UCOS) and changes to SCADA points or alarms require an MOC, which was easier to track. The procedure needs to be amended to establish a tracking method to address deficiencies identified through the implementation of § 195.446(e)(1) through § 195.446(e)(5) to verify they have been resolved and provide an audit trail for analysis and compliance. 8. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (g) Operating experience. Each operator must assure that lessons learned from its operating experience are incorporated, as appropriate, into its control room management procedures by performing each of the following: (1) Review accidents that must be reported pursuant to § 195.50 and 195.52 to determine if control room actions contirbtued to the event… NNOG Running Horse Pipeline’s CRM Plan was inadequate because it failed to identify how it evaluates if control room actions contributed to a reportable event as required by §195.446(g)(1). The Form RHPT-ADM 6 Accident Investigation Form failed to address controller fatigue, SCADA configurations. SCADA performance, and control room procedures. The procedures need to be amended to include guidelines to review the required items and include them in form RHPT-ADM6.#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 79. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (h) Training. Each operator must establish a controller training program and review the training program content to identify potential improvements at least once each calendar year, but at intervals not to exceed 15 months... NNOG Running Horse Pipeline's CRM Plan was inadequate in defining a process to review the training content to identify potential improvements at least once each calendar year, but at intervals not to exceed 15 months, as required by §195.446(h). Rather than providing requirements for the review, procedure 500-8.5 roughly restates the regulation. The procedure needs to be amended to provide a fully developed process to review the training program content once each calendar year, at intervals not to exceed 15 months. 10. 195.446 Control room management. (a) General. This section applies to each operator of a pipeline facility with a controller working in a control room who monitors and controls all or part of a pipeline facility through a SCADA system. Each operator must have and follow written control room management procedures that implement the requirements of this section. The procedures required by this section must be integrated, as appropriate with the operator’s written procedures required by § 195.402… (h) Training. Each operator must establish a controller training program and review the training program content to identify potential improvements at least once each calendar year, but at intervals not to exceed 15 months. An operator’s program must provide for training each controller to carry out the roles and responsibilities defined by the operator. In addition, the training program must include… (6) Control room team training and exercises that include both controllers and other individuals, defined by the operator, who would reasonably be expected to operationally collaborate with controllers (control room personnel) during normal, abnormal, or emergency situations… NNOG Running Horse Pipeline’s CRM Plan in procedure 500-8.4 was inadequate because it did not address when initial Team Training occurred for new controllers and those who would be reasonably expected to operationally collaborate with controllers, as required by §195.446(h)(6). The procedure indicated Team Training was conducted every three years. Additionally, the procedures did not include requirements to include content to be covered during team training sessions. There were no requirements for lessons learned from internal and industry events, soft skills, abnormal operations, etc.#
32022035NOA_Notice of Amendment_03012022_(21-201004)_text.pdf, page 8The procedure needs to be amended to provide when initial Team Training needs to be provided as well as training content. Response to this Notice This Notice is provided pursuant to 49 U.S.C. § 60108(a) and 49 C.F.R. § 190.206. Enclosed as part of this Notice is a document entitled Response Options for Pipeline Operators in Compliance Proceedings. Please refer to this document and note the response options. Be advised that all material you submit in response to this enforcement action is subject to being made publicly available. If you believe that any portion of your responsive material qualifies for confidential treatment under 5 U.S.C. 552(b), along with the complete original document you must provide a second copy of the document with the portions you believe qualify for confidential treatment redacted and an explanation of why you believe the redacted information qualifies for confidential treatment under 5 U.S.C. 552(b). Following the receipt of this Notice, you have 30 days to submit written comments, revised procedures, or a request for a hearing under §190.211. If you do not respond within 30 days of receipt of this Notice, this constitutes a waiver of your right to contest the allegations in this Notice and authorizes the Associate Administrator for Pipeline Safety to find facts as alleged in this Notice without further notice to you and to issue an Order Directing Amendment. If your plans or procedures are found inadequate as alleged in this Notice, you may be ordered to amend your plans or procedures to correct the inadequacies (49 C.F.R. § 190.206). If you are not contesting this Notice, we propose that you submit your amended procedures to my office within 30 days of receipt of this Notice. This period may be extended by written request for good cause. Once the inadequacies identified herein have been addressed in your amended procedures, this enforcement action will be closed. It is requested (not mandated) that NNOG – NNOG Running Horse Pipeline maintain documentation of the safety improvement costs associated with fulfilling this Notice of Amendment (preparation/revision of plans, procedures) and submit the total to Gregory A. Ochs, Director, Central Region, Pipeline and Hazardous Materials Safety Administration. In correspondence concerning this matter, please refer to CPF 3-2022-035-NOA and, for each document you submit, please provide a copy in electronic format whenever possible. Sincerely, Gregory A. Ochs Director, Central Region, Office of Pipeline Safety Pipeline and Hazardous Materials Safety Administration cc: Joseph Robertson, Director of Pipeline Operations jrobertson@nnogc.com Enclosure: Response Options for Pipeline Operators in Enforcement Proceedings#
32022035NOA_Closure Letter_09082023_(21-201004)_text.pdf, page 1Official PDFVIA ELECTRONIC MAIL TO: drubenking@nnogc.com; jrobertson@nnogc.com September 8, 2023 Mr. David Rubenking, Acting CEO Navajo Nation Oil and Gas PO Box 4439 Window Rock, AZ 8651 RE: CPF 3-2022-035-NOA Dear Mr. Rubenking: From May 24 to 28, 2021, a representative from the Pipeline and Hazardous Materials Safety Administration (PHMSA), pursuant to chapter 601 of 49 United States Code, conducted an on- site pipeline safety inspection of Navajo Nation Oil and Gas’ (NNOG) procedures in Farmington, New Mexico. As a result of the inspection, NNOG was issued a Notice of Amendment on March 1, 2022, which proposed amendment of your procedures. NNOG submitted its amended procedures on September 7, 2023. My staff reviewed the amended procedures, and it appears that the inadequacies outlined in this Notice of Amendment have been corrected. This letter is to inform you no further action is necessary and this case is now closed. Thank you for your cooperation. Sincerely, Gregory A. Ochs Director, Central Region, OPS Pipeline and Hazardous Materials Safety Administration cc: Joseph Robertson, Director of Pipeline Operations jrobertson@nnogc.com#
This material provides agency context. It does not replace binding regulatory text, and its legal effect depends on the underlying authority and facts.