{"operation":"document","citation":"CPF 32021055NOA","title":"MISSISSIPPI HUB, LLC — Notice of Amendment","source_type":"enforcement","agency":"Pipeline and Hazardous Materials Safety Administration","status":"historical","official":true,"published_on":"2021-09-10","effective_on":null,"summary":"CLOSED notice of amendment citing 192.631(a)(1), 192.631(c)(3), 192.631(e)(2).","machine_formats":{"json":"https://regulus.evalyn.ai/document/phmsa-enforcement-32021055noa.json","markdown":"https://regulus.evalyn.ai/document/phmsa-enforcement-32021055noa.md"},"app_url":"https://regulus.evalyn.ai/document/phmsa-enforcement-32021055noa","source_url":"https://primis.phmsa.dot.gov/enforcement-data/case/32021055NOA","body":"Notice of Amendment involving MISSISSIPPI HUB, LLC. PHMSA's enforcement data identifies the cited regulations as 192.631(a)(1),  192.631(c)(3),  192.631(e)(2). The case was opened on 2021-09-10 and is reported as closed as of 2021-11-16. Open the official case record for notices, responses, orders, and the latest status.\n\nOfficial case documents:\n\n32021055NOA_Closure Letter_11162021_(20-179020).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32021055NOA/32021055NOA_Closure%20Letter_11162021_(20-179020).pdf\n\n32021055NOA_Closure Letter_11162021_(20-179020)_text.pdf: https://primis.phmsa.dot.gov/enforcement-documents/32021055NOA/32021055NOA_Closure%20Letter_11162021_(20-179020)_text.pdf\n\n32021055NOA_Notice of Amendment_09102021_(20-179020).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32021055NOA/32021055NOA_Notice%20of%20Amendment_09102021_(20-179020).pdf\n\n32021055NOA_Notice of Amendment_09102021_(20-179020)_text.pdf: https://primis.phmsa.dot.gov/enforcement-documents/32021055NOA/32021055NOA_Notice%20of%20Amendment_09102021_(20-179020)_text.pdf\n\n32021055NOA_Operator Response to Notice and Request for Time Extension_10072021_(20-179020).pdf: https://primis.phmsa.dot.gov/enforcement-documents/32021055NOA/32021055NOA_Operator%20Response%20to%20Notice%20and%20Request%20for%20Time%20Extension_10072021_(20-179020).pdf\n\n32021055NOA_Closure Letter_11162021_(20-179020)_text.pdf\n\nVIA ELECTRONIC MAIL TO: paul.bieniawski@enstorinc.com & todd.cash@enstorinc.com\nNovember 16, 2021\nPaul Bieniawski\nChief Executive Officer\nEnstor Mississippi Hub\n10375 Richmond Avenue, Suite 1900\nHouston, TX 77042\nRe: CPF 3-2021-055-NOA\nDear Mr. Bienaiwski:\nFrom November 2 through 6, 2021, a representative from the Pipeline and Hazardous Materials\nSafety Administration (PHMSA), pursuant to chapter 601 of 49 United States Code, conducted a\nremote pipeline safety inspection of Enstor Mississippi Hub procedures in Control Room\nManagement. As a result of the inspection, Enstor Mississippi Hub was issued a Notice of\nAmendment on September 10, 2020, which proposed amendment of your procedures.\nEnstor Mississippi Hub submitted its amended procedures on November 10, 2021. My staff\nreviewed the amended procedures, and it appears that the inadequacies outlined in this Notice of\nAmendment have been corrected.\nThis letter is to inform you no further action is necessary and this case is now closed. Thank you\nfor your cooperation.\nSincerely,\nGregory A. Ochs\nDirector, Central Region, Office of Pipeline Safety\nPipeline and Hazardous Materials Safety Administration\ncc: Todd Cash, Director Compliance todd.cash@enstorinc.com\n\n32021055NOA_Notice of Amendment_09102021_(20-179020)_text.pdf\n\nNOTICE OF AMENDMENT\nVIA ELECTRONIC MAIL TO: paul.bieniawski@enstorinc.com & todd.cash@enstorinc.com\nSeptember 10, 2021\nPaul Bieniawski\nChief Executive Officer\nEnstor Mississippi Hub\n10375 Richmond Avenue, Suite 1900\nHouston, TX 77042\nCPF 3-2021-055-NOA\nDear Mr. Bienaiwski:\nFrom November 2 through6, 2020, a representative of the Pipeline and Hazardous Materials\nSafety Administration (PHMSA) pursuant to Chapter 601 of 49 United States Code inspected\nEnstor Mississippi Hub (Enstor) procedures for Control Room Management in Katy, Texas.\nThis Notice is in response to PHMSA’s Control Room Management (CRM) Initiative, which is a\nnational level program that includes inspectors from every region. As a result, you may have\nreceived this Notice from a different Regional Director than typical because the CRM Initiative\ninspections are currently separate from the standard inspection program. Notices and\ncorrespondence from other types of inspections will remain unchanged.\nOn the basis of the inspection, PHMSA has identified the apparent inadequacies found within\nMississippi Hub’s plans or procedures, as described below:\n1. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\n\n\n\nEnstor’s O&M Procedure 1120 CRM Plan was inadequate because it failed to provide\ninformation to define what constitutes additions, expansions, or replacements to the SCADA\nsystem required by § 192.631 (c)(1). Section 7.3 of O&M Procedure 1120 simply restated the\nregulation. While Enstor had adopted the ROC CRM plan, it did not employ the ROC SCADA\nsystem for its pipeline remote operations. Enstor used its own SCADA system and the ROC\ncontrollers remotely logged in to Enstor’s SCADA system. Therefore, the SCADA system was\nmanaged by Enstor and its procedures are required to be compliant with the regulation.\nThe procedure must be amended to provide definitions of additions, expansion, or replacements\nof the SCADA system, as well as examples of these conditions to provide adequate information\nof when to implement the required sections of API 1165.\n2. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\nEnstor’s O&M Procedure 1120 was inadequate because it failed to provide an adequate process\nfor when a point-to-point must be completed and instruction on how to complete a point-to-point\nverification as required by § 192.631(c)(2). Section 7.4 of O&M Procedure 1120, and form\n1100-27 provided the compliance procedure and documentation for point-to-point verifications.\nThe procedure did not include instructions on how to complete the form, and the form did not\nprovide enough detail to demonstrate thoroughness of the point-to-point verification. The point-\nto-point needs to include verification of field to control room HMI data and status, alarm set\npoints, alarm messaging, alarm attributes (color, blinking, audio), tag name, who conducted the\npoint to point, correct end device and location, date of completion and screen verification for all\nscreens where the point presents.\nThe procedure also did not define when and under what conditions a point-to-point needs to be\nconducted.\n3. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\nEnstor’s O&M procedure 1109 was inadequate because it failed to provide a clear process to test\nthe SCADA backup system in coordination with the ROC SCADA back up test. Enstor’s\n2\n\n\n\nSCADA system was separate from ROC’s SCADA system and needed to be tested per Section §\n192.631(c)(4). During ROC’s back up test, Enstor utilized a laptop to connect with their main\nSCADA server. While the test was successful, the documentation of the ROC SCADA back up\ntest did not include any consideration for the Enstor laptop, and Enstor did not have\ndocumentation of the test. The ROC procedure was silent in the coordination of the testing of\nthe two systems.\nThe procedure needs to be amended to consider either coordination with ROC during their test or\na separate test of the Mississippi Hub back up SCADA. The procedure needs to include any\nlogistical considerations, backup server testing, assessments of monitoring as well as verification\nof alarming and command functions. It must also include documentation of the test including\nany findings, corrective actions and lessons learned. The tests must be completed once each\ncalendar year not to exceed 15 months.\nThe procedure must be amended to include situations and conditions under which a point-to-\npoint must be conducted as well as instructions and forms with detail to provide a thorough\npoint-to-point verification.\n4. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\nThe ROC CRM Plan dated July 7, 2020 was not adequate because it relied heavily on self-\nreporting fatigue. Section 6.12, Countermeasures for 9-12-hour days, relied on the controller\nself-reporting fatigue. It did not provide a countermeasure for supervisors or others working in\nthe control room to monitor and report a person suspected or observed to be fatigued. In sections\n3.5 and 3.6, Roles and Responsibilities of the Control Room Manager, it stated in items 4 and 6,\nrespectively, \"[m]anage fatigue risk and fatigue countermeasures\". While these are important\nand appropriate roles and responsibilities for managers, manage is different than monitor,\nrecognize and respond. The section expressly states in Section § 192.631(d)(3) to \"train\ncontrollers and supervisors to recognize the effects of fatigue\". The procedure did not offer a\nrequirement for managers, supervisors, or other controllers to monitor, identify and\nrespond/report others. Fatigue management and identification, while it is a responsibility of the\ncontrollers for themselves, is also a responsibility all members of the control room.\nThe procedure needs to be amended to include roles and responsibilities for all control room\nemployees to monitor, identify and respond to fatigue of the individual and others.\n3\n\n\n\n5. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\nEnstor’s procedure was not adequate because it did not include a list of abnormal operating\nconditions (AOC's) that are likely to occur simultaneously or in sequence, as required by section\n§ 192.631(h)(1). The operator’s third party (ROC) control room manager indicated these\nconsiderations were included in the training documents. Upon review of the training\ndocumentation, it was identified that the training material does cover AOCs that happen in\nsequence, but nothing in the training materials discussed how to address ones that occur\nsimultaneously. Additionally, while ROC may have a standard list, it is also necessary to\nidentify asset specific conditions (Enstor) that may occur outside of the generic examples\nprovided by the third-party contractor.\nThe procedure needs to be amended to include a list or examples of abnormal operating\nconditions likely to occur simultaneously or in sequence. The list needs to consider conditions\nspecific to Mississippi Hub operations.\n6. § 192.631 Control room management.\n(a) General. (1) This section applies to each operator of a pipeline facility with a\ncontroller working in a control room who monitors and controls all or part of a\npipeline facility through a SCADA system. Each operator must have and follow\nwritten control room management procedures that implement the requirements of\nthis section, except that for each control room where an operator’s activities are\nlimited to either or both of:\nEnstor’s procedures were inadequate to provide requirements of what information the controllers\nneeded to record and document in their log that demonstrates compliance with the requirements\nof this section per § 192.631(j)(1). When attempting to determine if field personnel and others\nwere contacting the control room prior to implementing physical changes to pipeline equipment\nor configuration, inspectors were not able to make determinations through the controller logs. In\nreviewing the procedures, it was identified in sections 5.5, 5.5.1, 5.6 and 7.6 Enstor’s O&M\nManual 1120, dated 1/14/2020, that field personnel were required to contact the control room\nprior to making changes to the field. Procedures for annual valve maintenance, annual relief\nvalve calibration/maintenance, and testing ESDs all require contacting the control center prior to\ncommencing work. Section 5.7.1 of O&M Manual 1120 required documenting work activities\nand/or other pertinent information that transpired during the shift, including but not limited to\n\"operational activities (e.g. operations, maintenance, communications, notifications)\".\nDocumenting this type of maintenance, as well as other activities on the pipeline, is an important\nrecord for activities on the pipeline that can support compliance with other sections of the\n4\n\n\n\nregulations, supports trouble shooting issues as well as responding to abnormal conditions or\nemergencies if they develop.\nEnstor’s procedure needs to be amended to include information that must be recorded and\ndocumented in the controller log to demonstrate compliance with the requirements of section §\n192.631.\n7. § 192.631 Control room management.\n(a) . . . .\n(c) Provide adequate information. Each operator must provide its controllers with\nthe information, tools, processes, and procedures necessary for the controllers to\ncarry out the roles and responsibilities the operator has defined by performing each\nof the following:\n(3) Test and verify an internal communication plan to provide adequate means for\nmanual operation of the pipeline safely, at least once each calendar year, but at\nintervals not to exceed 15 months;\nEnstor’s CRM Plan was inadequate to provide guidance to the controller when performing\nmanual operations through actual or test condition as required. Remote Operation Control\n(ROC) began providing Enstor with control room services on August 6, 2019. Enstor’s Control\nRoom Management (CRM) procedure adopted ROC’s CRM Plan dated July 31, 2020. Section\n4.6 of the plan set forth times when manual operation of the pipeline must be initiated. While the\nprocedure addressed the requirement for field personnel to call the control room and provide\nfield operating information, it did not provide guidance on what the controller should do with the\ninformation. There was no language in the procedure related to how to document the field\ninformation, how to incorporate the information into safely operating the system, or how to\nmaintain the information as a record. Form 11.13 was the document of record for the required\ntest and included questions related to the manual operation conditions and requirements. This\nform did not have space to document the field information called in to the controller\nThe procedure must be amended to provide direction and tools for the controller to document\nfield information, and to incorporate that information into the safe manual operation of the\npipeline in the event of SCADA failure or if a shutdown of the SCADA system becomes\nnecessary.\n8. § 192.631 Control room management.\n(a) . . . .\n(e) Alarm management. Each operator using a SCADA system must have a written\nalarm management plan to provide for effective controller response to alarms. An\noperator’s plan must include provisions to:\n(1) . . . .\n(2) Identify at least once each calendar month points affecting safety that have been\ntaken off scan in the SCADA host, have had alarms inhibited, generated false\n5\n\n\n\nalarms, or that have had forced or manual values for periods of time exceeding that\nrequired for associated intervals not to exceed 15 months;\nEnstor’s Alarm Management procedure was inadequate because it failed to define all points\naffecting safety. The procedure does identify safety related alarms. Safety related points often\nsupport the definition of safety related alarms, but a safety related point, for operational reasons,\ncan stand on its own without alarming. Several regulatory sections depend upon the operator\ndefining points affecting safety and safety related alarms to meet the compliance requirement;\nthese included §§ 192.631(c)(2), 192.631(e)(1), 192.631(e)(2), and 192.631(e)(3). Because\nEnstor utilized their own SCADA system, they did not share the ROC Alarm Management Plan,\nor the identified safety related points and alarms contained therein.\nAlso noted in the procedure was that Enstor did not identify a designation for identifying safety\nrelated alarms in their SCADA database.\nThe procedure needs to be amended to identify the safety related points, safety related alarms\nand how to distinguish whether a point or alarm is safety related in the master SCADA database.\nResponse to this Notice\nThis Notice is provided pursuant to 49 U.S.C. § 60108(a) and 49 C.F.R. § 190.206. Enclosed as\npart of this Notice is a document entitled Response Options for Pipeline Operators in\nCompliance Proceedings.\nPlease refer to this document and note the response options. Be advised that all material you\nsubmit in response to this enforcement action is subject to being made publicly available. If you\nbelieve that any portion of your responsive material qualifies for confidential treatment under 5\nU.S.C. 552(b), along with the complete original document you must provide a second copy of\nthe document with the portions you believe qualify for confidential treatment redacted and an\nexplanation of why you believe the redacted information qualifies for confidential treatment\nunder 5 U.S.C. 552(b).\nFollowing the receipt of this Notice, you have 30 days to submit written comments, revised\nprocedures, or a request for a hearing under §190.211. If you do not respond within 30 days of\nreceipt of this Notice, this constitutes a waiver of your right to contest the allegations in this\nNotice and authorizes the Associate Administrator for Pipeline Safety to find facts as alleged in\nthis Notice without further notice to you and to issue an Order Directing Amendment. If your\nplans or procedures are found inadequate as alleged in this Notice, you may be ordered to amend\nyour plans or procedures to correct the inadequacies (49 C.F.R. § 190.206). If you are not\ncontesting this Notice, we propose that you submit your amended procedures to my office within\n30 days of receipt of this Notice. This period may be extended by written request for good\ncause. Once the inadequacies identified herein have been addressed in your amended\nprocedures, this enforcement action will be closed.\nIt is requested (not mandated) that Enstor Mississippi Hub maintain documentation of the safety\nimprovement costs associated with fulfilling this Notice of Amendment (preparation/revision of\nplans, procedures) and submit the total to Gregory A. Ochs, Director, Central, Pipeline and\n6\n\n\n\nHazardous Materials Safety Administration. In correspondence concerning this matter, please\nrefer to CPF 3-2021-055-NOA and, for each document you submit, please provide a copy in\nelectronic format whenever possible.\nSincerely,\nGregory A. Ochs\nDirector, Central Region, Office of Pipeline Safety\nPipeline and Hazardous Materials Safety Administration\ncc: Todd Cash, Director Compliance todd.cash@enstorinc.com\nEnclosure: Response Options for Pipeline Operators in Enforcement Proceedings\n7","truncated":false,"body_characters":19891}