{"operation":"document","citation":"CPF 32025016NOA","title":"HOLLY ENERGY PARTNERS - OPERATING, L.P. — Notice of Amendment","source_type":"enforcement","agency":"Pipeline and Hazardous Materials Safety Administration","status":"historical","official":true,"published_on":"2025-07-14","effective_on":null,"summary":"CLOSED notice of amendment citing 195.446(a), 195.446(b), 195.446(b)(5), 195.446(c)(2), 195.446(c)(3), 195.446(c)(4), 195.446(e)(1), 195.446(e)(2), 195.446(e)(3), 195.446(e)(5), 195.446(h), 195.446(h)(6).","machine_formats":{"json":"https://regulus.evalyn.ai/document/phmsa-enforcement-32025016noa.json","markdown":"https://regulus.evalyn.ai/document/phmsa-enforcement-32025016noa.md"},"app_url":"https://regulus.evalyn.ai/document/phmsa-enforcement-32025016noa","source_url":"https://primis.phmsa.dot.gov/enforcement-data/case/32025016NOA","body":"Notice of Amendment involving HOLLY ENERGY PARTNERS - OPERATING, L.P.. PHMSA's enforcement data identifies the cited regulations as 195.446(a),  195.446(b),  195.446(b)(5),  195.446(c)(2),  195.446(c)(3),  195.446(c)(4),  195.446(e)(1),  195.446(e)(2),  195.446(e)(3),  195.446(e)(5),  195.446(h),  195.446(h)(6). The case was opened on 2025-07-14 and is reported as closed as of 2026-02-02. Open the official case record for notices, responses, orders, and the latest status.\n\nOfficial case documents:\n\n32025016NOA_Closure Letter_02022026_(24-297199).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32025016NOA/32025016NOA_Closure%20Letter_02022026_(24-297199).pdf\n\n32025016NOA_Closure Letter_02022026_(24-297199)_text.pdf: https://primis.phmsa.dot.gov/enforcement-documents/32025016NOA/32025016NOA_Closure%20Letter_02022026_(24-297199)_text.pdf\n\n32025016NOA_Notice of Amendment_07142025_(24-297199).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32025016NOA/32025016NOA_Notice%20of%20Amendment_07142025_(24-297199).pdf\n\n32025016NOA_Notice of Amendment_07142025_(24-297199)_text.pdf: https://primis.phmsa.dot.gov/enforcement-documents/32025016NOA/32025016NOA_Notice%20of%20Amendment_07142025_(24-297199)_text.pdf\n\n32025016NOA_Operator Response to Notice_11122025_(24-297199).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32025016NOA/32025016NOA_Operator%20Response%20to%20Notice_11122025_(24-297199).pdf\n\n32025016NOA_Notice of Amendment_07142025_(24-297199)_text.pdf\n\nNOTICE OF AMENDMENT\nVIA ELECTRONIC MAIL TO: Steve.Ledbetter@HFSinclair.com;\nTane.Hartman@HFSinclair.com; Charles.Curl@HFSinclair.com\nJuly 14, 2025\nMr. Steven Ledbetter\nChief Executive Officer and President\nHF Sinclair Corporation\n2323 Victory Avenue, Suite #1400\nDallas, TX 75219\nCPF 3-2025-016-NOA\nDear Mr. Ledbetter:\nFrom May 6 through May 23, 2024, representatives of the Pipeline and Hazardous Materials\nSafety Administration (PHMSA), pursuant to Chapter 601 of 49 United States Code (U.S.C.),\ninspected onsite Holly Energy Partners, L.P.’s (Holly Energy)1 plans and procedures for Control\nRoom Management (CRM) in Artesia, New Mexico.\nAs a result of the inspection, PHMSA has identified the apparent inadequacies found within\nHolly Energy’s plans or procedures. The items inspected and the inadequacies are described\nbelow:\n1. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section….\n(b) Roles and responsibilities. Each operator must define the roles and\nresponsibilities of a controller during normal, abnormal, and emergency operating\nconditions. To provide for a controller's prompt and appropriate response to\noperating conditions, an operator must define each of the following:\n1 Holly Energy Partners, L.P., is a wholly owned subsidiary of HF Sinclair Corporation. See\nhttps://www.hollyenergy.com/Home/default.aspx.\n\n\n\nHolly Energy’s CRM Plan, (rev. 13, effective December 15, 2023) (CRM Plan) was inadequate\nbecause it failed to define the roles and responsibilities of a controller during normal, abnormal,\nand emergency operating conditions, per the requirements of § 195.446(b). Specifically, the\nCRM Plan failed to define how a controller is expected to respond to different leak detection\nalarms. Holly Energy used three leak detection systems in the control room: Pipeline Manager\n(PLM), Sim Suite and ATMOS. Each leak detection system had a different alarm system. The\ndifferent leak detection alarms that were presented to controllers was not described in the CRM\nPlan procedure or the Alarm Management Plan, (ver. 1.0, effective July 28, 2011) (ALM Plan).\nThe CRM Plan, section 17.9, only provided a high-level overview of how each system functions\nto detect leaks. While the ATMOS system was functioning on one two-mile stretch of pipeline\non a single console, this was a different alarm than the alarm detailed in the CRM Plan.\nHolly Energy’s procedures must be amended to describe the different leak alarms that can be\npresented to a controller, define what they represent, and define the controller’s expected\nresponse depending on the alarm. For example, whether a different controller response is\nexpected between a warning alarm and a leak alarm.\n2. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section….\n(b) Roles and responsibilities. Each operator must define the roles and\nresponsibilities of a controller during normal, abnormal, and emergency operating\nconditions. To provide for a controller's prompt and appropriate response to\noperating conditions, an operator must define each of the following:\n(1) . . . .\n(5) The roles, responsibilities and qualifications of others who have the authority to\ndirect or supersede the specific technical actions of controllers.\nHolly Energy’s CRM Plan was not adequate because it did not define the roles, responsibilities\nand qualifications of others who have the authority to direct or supersede the specific technical\nactions of controllers, per the requirements of § 195.446(b)(5). CRM Plan, section 2.8, described\nwho can direct or supersede the specific technical actions of controllers, and that those\nindividuals were “DOT OQ qualified” and “have the expertise to know when conditions or\nsituations warrant the necessity to use their authority to direct or supersede the technical actions\nof a Controller” as well as how to implement and document the action. DOT OQ Qualifications\nrelates to many covered tasks that have no bearing on the covered tasks required to be a trained\nand qualified controller. Additionally, what type of “expertise” is required to know when\n“conditions or situations warrant the necessity to use their authority to direct or supersede the\ntechnical actions of a Controller.” This was not adequate because it was not specific enough to\ndefine what qualifications are required for an individual to have authority to direct or supersede a\ncontroller.\n\n\n\nCRM Plan, section 9.6 Team Training, identified other individuals who would be reasonably\nexpected to collaborate with controllers. Section 9.6 did not describe how it would be\ncommunicated who the individuals are who have authority to direct or supersede the specific\ntechnical actions of controllers to those identified as others and controllers. Section 9.6 also did\nnot describe the process for documenting the communication of these roles and responsibilities\nto individuals with such authority.\nThe CRM Plan must be amended to provide a requirement for the qualification of those with\nauthority to direct or supersede the specific technical actions of controllers, the method of\ncommunication with these individuals, and the timing of when the initial communication will be\nmade to new controllers and others new to their roles.\n3. § 195.446 Control Room Management.\n(a)General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(c) Provide adequate information. Each operator must provide its controllers with\nthe information, tools, processes and procedures necessary for the controllers to\ncarry out the roles and responsibilities the operator has defined by performing each\nof the following:\n(1) . . . .\n(2) Conduct a point-to-point verification between SCADA displays and related field\nequipment when field equipment is added or moved and when other changes that\naffect pipeline safety are made to field equipment or SCADA displays.\nHolly Energy’s CRM Plan and ALM Plan were not adequate to define safety related points nor\nhow they are determined, per the requirements of § 195.446(c)(2). The ALM Plan, section 2,\ndefined “Safety Related Alarm” as “[a]ny alarm that affects safe operation of the pipeline. Any\noperational factor that is necessary to maintain pipeline integrity or that could lead to the\nrecognition of a condition that could impact the integrity of the pipeline, or a developing\nabnormal or emergency situation. This includes but is not limited to; deviation of rate, high\npressure, high level, and gas detector alarms.” Table 1 of the CRM Plan provided a definition\nfor “Safety Related Operations” as, “any operational factor that is necessary to maintain pipeline\nintegrity or that could lead to the recognition of a condition that could impact the integrity of the\npipeline, or a developing abnormal or emergency situation.” What was not defined was a point\naffecting safety.\nALM Plan section 1.4 stated, “HEP has a process in place to ensure that SCADA safety related\nalarms are accurate, and they support safe operations of the pipeline.” However, neither the\nALM Plan nor the CRM Plan offered what this process is. CRM Plan, section 31, contains a\n“Safety Related Point Flow Chart” that provides flow charts equipment and operational\nconditions that support determination of whether a point is safety related. There is no reference\n\n\n\nto these flow charts in either the CRM Plan or the ALM Plan. The procedures did not contain\nany instructions on (1) when to use the flow charts, (2) which employee is responsible to\ndetermine if a point is safety related, and (3) how the determination will be documented. In\npractice, Holly Energy controllers would use SCADA tags that are designated safety related at\nthe front of the alarm description. This was not identified in the procedure.\nBecause Holly Energy has established an alarming system that can be generated from both the\nfield Programmable Logic Controllers (PLC) and the SCADA system, the system of tagging\nsafety related alarms should be standardized and clarified in procedure. It is unclear if the flow\ncharts contained in CRM Plan, section 31, are utilized by field personnel when designing and\nconstructing the field PLCs. In practice, all alarming occurs at the PLC for safety related alarms.\nThis practice has not been defined in procedure.\nThe procedure must be amended to provide more instruction on who, when, and how the “Safety\nRelated Point Flow Charts” will be employed to designate a point and alarm safety related.\nThere needs to be clarification on whether the field is using this process or if it is strictly for the\nSCADA controllers. In addition, the method of identifying safety related alarms to the controller\nusing the word “SAFETY” in front of the alarm description must be in the procedure, as well as\nan explanation of the process used for alarming at the PLC and matching it to SCADA tags that\nare also receiving an analogue value.\n4. § 195.446 Control Room Management.\n(a)General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(c) Provide adequate information. Each operator must provide its controllers with\nthe information, tools, processes and procedures necessary for the controllers to\ncarry out the roles and responsibilities the operator has defined by performing each\nof the following:\n(1) . . . .\n(2) Conduct a point-to-point verification between SCADA displays and related field\nequipment when field equipment is added or moved and when other changes that\naffect pipeline safety are made to field equipment or SCADA displays.\nHolly Energy’s CRM Plan was not adequate because its procedure for conducting a thorough\npoint-to-point (P2P) verification was insufficient to demonstrate compliance with §\n195.446(c)(2). Specifically, CRM Plan, section 32, provided an inadequate procedure to verify\npoints between SCADA and the field. CRM Plan, section 33, provided the P2P verification\ncheck-out sheet. The check-out sheet was not referenced in the procedure to document the\nverification, nor did it match the verification requirements established in CRM Plan sections 32\nand 33. The check-out sheet did not include: the date, the name of the field tech, the name of the\nControl Center representative, documentation of analog values following verification, and\n\n\n\nrecording verification of displays. Additionally, there was no space to document point value or\nstatus conditions, and the form did not capture whether the point was tested in live or simulated\nmode.\nHolly Energy presented for inspection a P2P verification form for Pathfinder/Sanford and Station\n10. However, this was a different form than the check-out sheet. It was the form used for the\nSinclair Transportation control room prior to the Holly Energy-Sinclair merger.\nIf this is the form Holly Energy plans to use for P2P verification, this should be detailed in its\nCRM Plan. The CRM Plan also did not include a process to retest the point and document the\ntest and the result in the event the point failed the P2P verification. Furthermore, the procedure\nfailed to include verification of the SCADA tag to the end device and verification of the\nsequence of the device on the pipeline and its match to the SCADA display placement.\nThe CRM Plan and its associated form need to be amended to include recording of the analog\nvalues and status comparison confirmations, recording the displays that were verifying the\nSCADA tag to the end device, the sequence of the device on the pipeline, its match to reviewed\nand verified SCADA display placement, and documentation of whether the point was verified\nlive or simulated. The CRM Plan also must include a process to retest the point and document\nthe test and test result in the event the point failed the initial P2P verification. These items\nidentified in the amended procedure and form will provide a thorough P2P verification in\ncompliance with § 195.446(c)(2).\n5. § 195.446 Control Room Management\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(c) Provide adequate information. Each operator must provide its controllers with\nthe information, tools, processes and procedures necessary for the controllers to\ncarry out the roles and responsibilities the operator has defined by performing each\nof the following:\n(1) . . . .\n(3) Test and verify an internal communication plan to provide adequate means for\nmanual operation of the pipeline safely, at least once each calendar year, but at\nintervals not to exceed 15 months.\nHolly Energy’s CRM Plan, section 4, was not adequate to provide means for manual operation of\nthe pipeline safely, as required by § 195.446(c)(3). Specifically, the CRM Plan lacked detail to\nsupport the controller’s step-by-step response to loss of communication or SCADA, how\ncomplete shutdown of field facilities will be verified, how field personnel will be dispatched to\nkey locations, and how the manual readings will be communicated and documented. It also\nfailed to include how abnormal and emergency conditions will be reported and how monitoring\nleaks will be conducted and reported.\n\n\n\nCRM Plan, section 4.1.2, only restated the regulation, section 4.5 provided roles and responsibilities\nto notify field personnel to manually operate, and section 4.6 referenced Operations and\nMaintenance Manual (effective June 10, 2024) (O&M Manual), section 4.1.2.3,\n2 which provided\nvery high-level duties of the controller to notify field personal and to document names of field\npersonnel at each location and all changes that were made, as well as the time communication was\nrestored. The O&M Manual did not provide any specific plans related to an internal\ncommunication plan for the manual operation of the pipeline safely.\nCRM Plan, section 17.1.7, “Manual Operation,” did not directly relate to manual operations.\nRather, it addressed the controllers’ responsibilities during abnormal operations. CRM Plan,\nsection 17.9.2, “Procedure for Monitoring of Idle or Shut in Pipeline Segments and Factors to\nConsider Investigating Pressure Variances,” provided guidance on how to manually monitor and\nevaluate the pipeline to identify leaks. These procedures did not include considerations for leak\nmonitoring during manual operations in the event of loss of communication to a large area or\ntotal loss of SCADA.\nHolly Energy utilized a “Job Plan Form” to prepare for its annual backup control room failover\ntest. Holly Energy also utilized this form to document and facilitate a combined test for the\nInternal Communication Plan requirement, as defined in CRM Plan, section 4.8. CRM Plan,\nsection 26(A)(iv), “Control Center Failover Plan,” required the test be developed through the\n“Job Plan Form” process. However, there was no requirement for the Internal Communication\nPlan test to be developed in the Job Plan Form. A review of the “Job Plan Form” records,\ndocumenting the tests for the years 2021, 2022 and 2023, identified that the language used, to\nframe the test, is almost identical for each test. Holly Energy used the “Job Plan Form” to\ndevelop a plan, each year, for the Internal Communication Plan for manual operation of the\npipeline. Holly Energy had no formal Internal Communication Plan that provided the controller\ndetailed steps to follow for: initiating the manual shutdown of the hazardous liquid pipelines,\ndispatching of field personnel to the gas transmission pipeline key location, the documents used\nto record manual field readings, the frequency for field personnel to report field conditions, how\nabnormal operations and emergencies will be reported to the control room, or how leak detection\nwill be supported.\nHolly Energy must amend its procedures to create a formalized Internal Communication Plan for\nthe manual operation of the pipeline safely that includes: detailed steps to follow to initiate the\nmanual shutdown of the hazardous liquid pipelines, the dispatching of field personnel to the gas\ntransmission pipeline key locations, the documents used to record manual field readings, the\nfrequency for field personnel to report field conditions, how abnormal operations and\nemergencies will be reported to the control room, and leak detection will be supported. The\nprocedures must also be amended to define the test of the internal communication plan that\nincludes all the different elements of the plan, including a minimum time duration to gather\nenough field readings to evaluate effectiveness off the plan. Additionally, the test procedure also\nneeds to conduct an evaluation of the test to determine effectiveness and modify as needed.\n2 This was an incorrect reference; O&M Manual section 6.10 should have been referenced.\n\n\n\n6. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(c) Provide adequate information. Each operator must provide its controllers with\nthe information, tools, processes and procedures necessary for the controllers to\ncarry out the roles and responsibilities the operator has defined by performing each\nof the following:\n(1) . . . .\n(4) Test any backup SCADA systems at least once each calendar year, but at\nintervals not to exceed 15 months.\nHolly Energy’s CRM Plan was not adequate to test any backup SCADA systems at least once\neach calendar year, but at intervals not to exceed 15 months, per the requirements of\n§ 195.446(c)(4). Specifically, the CRM Plan lacked a detailed process for transferring operations\nfrom the primary control room to the back-up control room. CRM Plan, section 26, “Control\nCenter Failover Plan,” directed controllers to “shut down the appropriate assets under his control\nusing normal operating procedures” and stated:\nEach Controller shall take the Console Procedure Book with schedules and phone\nlists for their respective console to the Secondary Control Center. Notifications\nwill need to be made to all HEP Terminals, associated Refineries, and third-party\ncustomers as soon as practical that Operations have evacuated the Primary\nControl Center and are relocating to the Secondary Control Center.\nThe CRM Plan included a map and directions to the backup control room facility. It also\nincluded instructions on how to switch from the primary to secondary SCADA system. CRM\nPlan section 26 (A)(iv) stated, “[t]here are many reasons for switching from the Primary Control\nCenter to the Secondary Control Center. This makes it difficult to write a specific procedure to\nmove from the Secondary Control Center to the Primary Control Center prior to knowing why\nyou switched to the Secondary Control Center.” While there may be many reasons to switch\ncontrol centers, the process to switch should be standardized to manage a successful transition.\nTransferring servers, phones, accessing building space, powering up consoles, in that order or\nreversed, should not require much deviation. Per CRM Plan, section 26 (A)(iv), Holly Energy\nrequired a Job Plan to be written to relocate to the backup control center based on the specific\nreason for the relocation. The CRM Plan was missing a process to return personnel and transfer\ncontrol back to the primary control room.\nCRM Plan, section 26(A)(iv), provided considerations of what to include in the Job Plan, but\nprovided no details of how to test the backup SCADA systems. No form or checklist was\nprovided. Holly Energy documented SCADA testing on a “Lessons Learned” form. This\ndocumentation was a practice that was not directed through procedure to be used to document\nthe test. This form provided a summary of what went well, what failed, and what, if any,\ncorrective actions needed to be taken. The CRM Plan did not have a standardized form to\n\n\n\nprovide a consistent test protocol and test documentation of the back-up SCADA system. The\nCRM Plan also failed to include a required duration for the test.\nThe CRM Plan needs to be amended to provide the full process for the controllers to move the\ncontrol room, transfer control, and return back to the primary control room. The CRM Plan also\nneeds to define all elements that are to be tested and documented during the test. This also needs\nto include a form to provide consistency for all consoles.\n7. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(e) Alarm management. Each operator using a SCADA system must have a written\nalarm management plan to provide for effective controller response to alarms. An\noperator's plan must include provisions to:\n(1) Review SCADA safety-related alarm operations using a process that ensures\nalarms are accurate and support safe pipeline operations.\nHolly Energy’s CRM Plan was not adequate to ensure alarms are accurate and support safe\npipeline operations, per the requirements of § 195.446(e)(1). CRM Plan, section 2.3.1, stated\nthat Holly Energy’s controllers must, “[g]enerate work requests for equipment failures,\nmechanical problems and SCADA alarms.” Section 6.8.1 stated, “[s]afety related alarms found\nout of calibration, in fault or otherwise not operating as designed will have a work order assigned\nby the controller responding to the alarm.” In practice, as described by Holly Energy during the\ninspection, when a controller discovered an alarm was malfunctioning or inaccurate, the\ncontroller created a ticket in the SAP application. The process to create a ticket in SAP was not\ndefined in the procedure. Absent a defined process, consistency of work orders cannot be\nensured; thereby undermining the process used for reviewing SCADA safety-related alarm\noperations. While the CRM Plan did require the safety-related alarm list to be reviewed monthly\nby the Alarm Management Group and for the group to develop an action plan to address the\nprocess for correcting critical safety related alarms, there was no detail or expectation for how\nlong alarms could be in repair mode or escalation if the workorder exceeded the return to service\nexpectation.\nThe CRM Plan needs to be amended to provide more detail on the current practice of creating\nworkorders for malfunctioning or inaccurate alarms and the monthly review process, including\nexpectations for return to service and escalation if workorder expectations are exceeded.\n8. § 195.446 Control room management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\n\n\n\nwritten control room management procedures that implement the requirements\nof this section. . . .\n(b) . . . .\n(e) Alarm management. Each operator using a SCADA system must have a written\nalarm management plan to provide for effective controller response to alarms.\nAn operator's plan must include provisions to:\n(1) . . . .\n(2) Identify at least once each calendar month points affecting safety that have\nbeen taken off scan in the SCADA host, have had alarms inhibited,\ngenerated false alarms, or that have had forced or manual values for\nperiods of time exceeding that required for associated maintenance or\noperating activities.\nHolly Energy’s CRM Plan Section 6.18.1 was not adequate to identify at least once each calendar\nmonth points affecting safety that have been taken off scan in the SCADA host, have had alarms\ninhibited, generated false alarms, or that have had forced or manual values for periods of time\nexceeding that required for associated maintenance or operating activities as required by\n§ 195.446(e)(2).\nHolly Energy’s CRM Plan, Section 6.18.1, only paraphrased the regulation. It did not provide\ndetails on (1) how Holly Energy’s employees would collect the prescribed information, (2) how\nHolly Energy’s employees would evaluate the information against the associated maintenance or\noperating activities, and (3) how deficiencies identified through the process would be addressed.\nThe CRM Plan also did not provide a process that described Holly Energy’s practice for identifying\nfalse alarms and including these in the monthly alarm review.3\nThe procedure must be amended to provide details on how to identify and correct at least once\neach month points affecting safety that have been taken off scan in the SCADA host, have had\nalarms inhibited, generated false alarms, or that have had forced or manual values for periods of\ntime exceeding that required for associated maintenance or operating activities as required by\n§ 195.446(e)(2). The procedure must include who is accountable for gathering, reviewing and\nreporting the data, the source of the data, the date taken out of service (off-scan, inhibited,\nforced/manual) and returned to service, document the reason the point or alarm was taken out of\nservice, a process to compare the duration of out of service to the maintenance activity,\nidentification of the deficient points and return to service as soon as possible. It must also include\nhow false alarms will be documented and reviewed.\n9. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\n3 PHMSA’s Control Room Management Frequently Asked Questions (FAQs) (effective Jan. 16, 2018) address false\nalarms. FAQ E.02 states “[f]or the purpose of the CRM regulations, any alarm that is presented to the controller that\ndid not accurately reflect the actual operational parameter or condition, or an alarm that can mislead a controller to\nbelieve a condition exists, but that does not exist, is considered a false alarm.”\n\n\n\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(e) Alarm management. Each operator using a SCADA system must have a written\nalarm management plan to provide for effective controller response to alarms. An\noperator's plan must include provisions to:\n(1) . . . .\n(3) Verify the correct safety-related alarm set-point values and alarm descriptions\nwhen associated field instruments are calibrated or changed and at least once each\ncalendar year, but at intervals not to exceed 15 months.\nHolly Energy’s CRM Plan was not adequate to verify the correct safety-related alarm set-point\nvalues and alarm descriptions when associated field instruments are calibrated or changed at\nleast once each calendar year, but at intervals not to exceed 15 months, as required by\n§ 195.446(e)(3). The CRM Plan, section 6.1.1, simply restated the regulation requirement with\nno instruction on how to conduct this review. Section 6.17.1 stated, “[t]he HEP Alarm Analyst\nSenior will annually review the SCADA safety related alarms.” A method to identify and list the\nsafety related alarms and descriptions, and document the verification findings, was not included\nin the procedure. The CRM Plan did include procedures used in the field for calibrations, and\nthese included notification of the control room and verification of alarm set points. But they did\nnot include alarm descriptions. Also, there was no reference to how the control room would\ndocument and verify this activity had been completed to demonstrate compliance. Holly Energy\nwas relying on the field to complete the calibration and documentation.\nThe CRM Plan must be amended to provide details to conduct the verification of the correct\nsafety-related alarm set-point values and alarm descriptions when associated field instruments\nare calibrated or changed, at least once each calendar year, but at intervals not to exceed 15\nmonths, to demonstrate compliance with § 195.446(e)(3). The CRM Plan must detail who is\nresponsible for conducting the annual review; documentation that includes the listing of all\nsafety related alarms, set points, and descriptions; the standard those set points will be verified\nto; and documentation of deficiencies and associated corrections. For field instrument changes\nand calibrations, the CRM Plan must include a process to document that all verifications were\nmade with the field personnel in SCADA for the set points and descriptions.\n10. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(e) Alarm management. Each operator using a SCADA system must have a written\nalarm management plan to provide for effective controller response to alarms. An\noperator's plan must include provisions to:\n(1) . . . .\n\n\n\n(5) Review the alarm management plan required by this paragraph at least once\neach calendar year, but at intervals not exceeding 15 months, to determine the\neffectiveness of the plan.\nHolly Energy’s CRM Plan, section 6.17.2.1, did not have adequate details to ensure that the\neffectiveness of the alarm management plan would be reviewed at least once each calendar year,\nbut at intervals not exceeding 15 months to determine the effectiveness of the plan, as required\nby § 195.446(e)(5). Specifically, CRM Plan, section 6.17.2.1 only paraphrased the regulation\nand identified the personnel responsible to conduct the review. Missing from the CRM Plan was\nwhat would be evaluated as part of the review, metrics for determining effectiveness, and how\ndeficiencies would be addressed and resulting plan modifications.\nThe CRM Plan must be amended to include the details of what will be reviewed to determine\neffectiveness of the alarm management plan. For example, the CRM Plan could include a review\nof KPIs to determine whether the alarm management plan meets the expected performance\noutcomes, a review to determine whether the work order ticket process facilitates quick repair, an\nevaluation to verify set points are not being changed without knowledge through the MOC, etc.\nAs appropriate, the CRM Plan must include some criteria that determines the effectiveness of the\nplan.\n11. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(h) Training. Each operator must establish a controller training program and\nreview the training program content to identify potential improvements at least\nonce each calendar year, but at intervals not to exceed 15 months. An operator's\nprogram must provide for training each controller to carry out the roles and\nresponsibilities defined by the operator. In addition, the training program must\ninclude the following elements:\nHolly Energy’s CRM Plan did not adequately describe the training program and process to\nreview the training program content once each calendar year not to exceed 15 months, as\nrequired by § 195.446(h). Holly Energy developed and implemented a compliant training\nprogram; however, they did not define any of it in the CRM Plan. CRM Plan, section 9,\nprovided a very high level of training components and in many subsections paraphrased the\nregulation. CRM Plan section 9.8.1 stated:\nThe HEP Training Program contains a process for managing changes to the\ncourse materials due to technological advances, regulatory requirements,\nprocedural revisions, Lessons Learned captured during Incident Investigation,\nand through Controller feedback, etc. During the annual review of the Control\nRoom Management Plan, the Review Board will discuss the Controller training\n\n\n\nprocess and make updates, if necessary, to the process and this section. This\nwill be documented in the summary overview document referenced in CRMP\nSection 1.5.\nBecause the training program is a stand-alone program and is currently a practice rather than a\nprocedure, the content review process was not documented. The CRM Plan must be modified to\ndescribe the training program, how it provides for training each controller to carry out the roles\nand responsibilities defined by the operator, and detail how the annual review of the training\nprogram content will be conducted and documented.\n12. § 195.446 Control Room Management.\n(a) General. This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section.…\n(b) . . . .\n(h) Training. Each operator must establish a controller training program and\nreview the training program content to identify potential improvements at least\nonce each calendar year, but at intervals not to exceed 15 months. An operator's\nprogram must provide for training each controller to carry out the roles and\nresponsibilities defined by the operator. In addition, the training program must\ninclude the following elements:\n(1) . . . .\n(6) Control room team training and exercises that include both controllers and other\nindividuals, defined by the operator, who would reasonably be expected to\noperationally collaborate with controllers (control room personnel) during normal,\nabnormal or emergency situations. Operators must comply with the team training\nrequirements under this paragraph no later than January 23, 2018.\nHolly Energy’s CRM Plan did not adequately define who was required to participate in the team\ntraining meeting, per the requirements of § 195.446(h)(6). CRM Plan, section 9.6.1, identified\nother individuals, defined by Holly Energy, who would reasonably be expected to operationally\ncollaborate with controllers (control room personnel) during normal, abnormal or emergency\nsituations. However, the CRM Plan was vague in how the team training will be conducted.\nHolly Energy used the list contained in CRM Plan section 9.6.1, to act as a speakers list. These\nindividuals were invited to make presentations to the controllers. This process offered the\ncontrollers opportunities to ask questions and interact and network with some of the identified\nindividuals. However, these identified individuals were not invited to participate in the team\ntraining meeting. The CRM Plan should be modified to allow for controllers and other\nindividuals to participate as equal members in the team training meeting. The meetings must\ninclude at least one controller.\n\n\n\nResponse to this Notice\nThis Notice is provided pursuant to 49 U.S.C. § 60108(a) and 49 CFR § 190.206. Enclosed as\npart of this Notice is a document entitled Response Options for Pipeline Operators in\nEnforcement Proceedings.\nPlease refer to this document and note the response options. Be advised that all material you\nsubmit in response to this enforcement action is subject to being made publicly available. If you\nbelieve that any portion of your responsive material qualifies for confidential treatment under\n5 U.S.C. § 552(b), along with the complete original document you must provide a second copy\nof the document with the portions you believe qualify for confidential treatment redacted and an\nexplanation of why you believe the redacted information qualifies for confidential treatment\nunder 5 U.S.C. § 552(b).\nFollowing the receipt of this Notice, you have 30 days to submit written comments, revised\nprocedures, or a request for a hearing under § 190.211. If you do not respond within 30 days of\nreceipt of this Notice, this constitutes a waiver of your right to contest the allegations in this\nNotice and authorizes the Associate Administrator for Pipeline Safety to find facts as alleged in\nthis Notice without further notice to you and to issue an Order Directing Amendment. If your\nplans or procedures are found inadequate as alleged in this Notice, you may be ordered to amend\nyour plans or procedures to correct the inadequacies (49 CFR § 190.206). If you are not\ncontesting this Notice, we propose that you submit your amended procedures to my office within\n30 days of receipt of this Notice. This period may be extended by written request for good\ncause. Once the inadequacies identified herein have been addressed in your amended procedures,\nthis enforcement action will be closed.\nIt is requested (not mandated) that Holly Energy LP maintain documentation of the safety\nimprovement costs associated with fulfilling this Notice of Amendment (preparation/revision of\nplans, procedures) and submit the total to the","truncated":true,"body_characters":42341}