Gray Oak Project Overview

Champion Technology Services nears completion of largest project in company history


Courtesy grayoakpipeline.com

In 2018, Phillips 66 awarded Champion Technology Services the Gray Oak Pipeline project. This project is Champion’s largest to date – not only in dollar value, but in the effort that the project required from our company as a whole. It has taken approximately 18 months, 45,500 hours, and effort from over 60 Champions to make this project happen.  

The Gray Oak Pipeline project encompasses 22 facilities and 32 pipeline block valve sites throughout 848 miles of pipeline that stretch throughout western Texas. The sites house a combination of pumps, tanks, and truck loading/unloading instrumentation. Champion was responsible for the turn-key delivery of the control system, including the high-quality design and fabrication of the control panels and controllers that are responsible for keeping the pipeline up and running. 

The project added complexity by splitting design and configuration into two phases: an accelerated phase and a full service phase

Project Management

Several key personnel were responsible for the massive coordination and execution of the project as a whole. Specifically, Brad Jordan, Gulf Coast Regional Manager, worked as the Project Sponsor and Dan Ward, Project Manager, served as the Program Manager. As Program Manager, Dan acted as the key point of contact for the whole project by coordinating between the customer and the other Project Managers and Technical Specialists on the team. He spent much of his time with Phillips 66 stakeholders, attending meetings, gathering information, capturing scope, fielding customer requests, and providing feedback on the entire project. 

The Gray Oak project showed how Champion is continuing to break down its silos and operating as a unit. It is evident in the collaboration between the Gulf Coast and the Rockies teams – from Project Administrators who have had to also adapt to the demands of the project, to the Engineers who used the right technology for the right application, and to the Leadership team who made it possible to scale the project as the demands grew. Dan says, “[The Gray Oak Project] would not have happened without the right kind of people. It’s an amazing group of people. If we didn’t have the people that we have, this would have been a real nightmare.” 

One of biggest challenges during this project was adjusting to and accommodating the customer’s continuous edits to the scope of work as they came through. “The project kept growing, and there’s never been a design freeze,” Dan said. Despite its true size, “Our team’s ability to function like a small integration group, being flexible, dynamic, and being able to accommodate changes was a huge factor in our success.” 

Design on the Fly

The project did, in fact, keep growing. In total, our design team developed drawings for 185 panels: 126 enclosures that are housed in Power Distribution Centers (PDCs) and 59 racks. On the programming side, our Engineering team was responsible for programming over 45 ControlLogix® and 33 CompactLogix® control systems, with 10,100 total I/O.   

The accelerated phase of the project included the design and fabrication of 75 enclosures and 28 racks. Champion’s designers completed many of the initial designs, and Champion subcontracted MMR to complete the fabrication portion of the work. Khaled Matarneh, Project Manager, was responsible for overseeing the design and fabrication portion of the project. Khaled reiterates Dan’s sentiments that Champion’s team made the real difference in the project’s success. “Everyone stepped up. We made it work despite not having a perfect plan or being perfectly prepared. We have a lot of lessons learned going into the future,” he says. 

While the team experienced challenges with procurement and fabrication, from design changes that affected Bill of Material specifications to renegotiating priorities with the subcontractor, the team was able to deliver the high-quality product that the customer expected. Khaled says, “The team was the reason we succeeded; a lot of people jumped in, stepped up, and did what they needed to do to make this work. Even though some members didn’t have direct experience in the role they were playing, they stepped up and made it happen. Everyone learned something new on this project.” 

Configuration

On the configuration side, Asa Ange, Engineering Supervisor, lead the crew as its Technical Director. Programming 88 controllers and 10,100 I/O is no small feat. During the accelerated portion of the work, the team had to configure 15 controllers and ~1,900 I/O for 10 sites.  Normallythat amount of work for unique systems would take a year of preparation and programming to complete, but by taking advantage of the similarities in each system the Gray Oak engineering team completed it in five (5) months.  

To expedite the configuration during the accelerated portion of the process, the team used Rockwell Automation’s Studio 5000 Application Code Manager to build templates for each of the somewhat similar meter sites to produce FAT-ready logic. Specifically, the team built logic templates using the  Rockwell Automation PlantPAx(R) DCS Application Library. While the initial templates took several months to create and refine, each controller was completed within about three days of commencing its programming. Design began in February 2019, and the logic for all 15 controllers was ready for testing by June 2019. The rest of the controllers and I/O were programmed during the Full Service portion of the project using the same strategy.  

The team developed 150 PlantPAx graphics during the accelerated portion of the work, or up to 10-20 graphics per site. In total, the project required 750 graphics for Operator monitoring and control of the pipeline. 

“Even though we didn’t face any ground-breaking technical issues, it was the quantity of controllers and the scale of the project that was the biggest challenge,” Asa recounts. “Our team configured the controllers and graphics on parallel tracks, so when anyone found a bug or something that could potentially affect all of the controller programs, we had to make sure to confirm the change with the customer. If the customer confirmed a global change, we had to modify everything across the board.”  

Making sure that modifications were consistent across the controllers proved to be challenging. Asa says that high levels of document control and intense communication and coordination were key to success. “Being systematic about code updates was crucial, especially since site-specific details would add complexity to any requested change.” 

Keys to Success

It’s clear to see that constant, consistent, and timely communication was the key to the project’s overall success. Sharing information in a timely manner was also a key factor.  

“This project showed the project team a new way of thinking about the way projects can be executed, from taking advantage of Collaboration technology through SharePoint and live document editing,” Dan Ward mentioned.  

“Coordinating 40-50 people within various Champion organizations, making sure that information is flowing, and getting information from the customer to the people that need it was a big challenge,” said Asa. “We had to set up the structure and methodology to disseminate information. The struggle was that we needed to build the structure from ground-up.” 

While it proved to be an exceptional learning experience, the process saved the team time and kept them nimble enough to keep up with the customer’s requests. 

Finishing Up

The project is currently in the commissioning phase. Commissioning started on the accelerated portion in August and ended in November. Full Service commissioning began in January with an expected completion in May 2020. We currently have 10 commissioning teams – each team includes Champion and Phillips 66 resources deployed at different sites with one team per site. 

Our Champion family shows the commitment that it takes to complete a project like Gray Oak 

The effort required to get the Gray Oak project started and finished took intense coordination among 60 plus Champion personnel who contributed to the project. Everyone added their effort and expertise at various stages in the process, and no contribution went unnoticed. On behalf of everyone at Champion, we’d like to thank all the project contributors for being exceptional individuals and for their hard work and dedication that makes our company truly special.  

Champion appreciates the extraordinary commitment and extraordinary sacrifices that the project team and their families have made to make this project a success. Thank you. 



Allen-Bradley Solution For Spice And Herb Sterilizer Process

Project Highlights

PROJECT OVERVIEW

Houston, TX – Champion’s Houston Automation group completed a Sterilizer Automation Upgrade Project for Olam Spices and Vegetables Ingredients Inc., a food preparation and packaging company, in Las Cruces, New Mexico. The Houston team designed a single new control system for two sterilizer lines after jointly observing existing operations of the sterilizer process and developing a Functional Requirement Specification (FRS) with the customer based on their desired operation.

Two of the customer’s major considerations for upgrading the system were: the existing control logic could only be adjusted by the manufacturer’s representatives (due to the PC based control system having compiled logic), and the system only provided approximately 40% reliability (up-time).

The customer’s goal was to give their operators full access to the controller for adjusting all variables in each product’s recipe, as well as tuning loops, and changing setpoints. In addition, they wanted the operators to be able to replace hardware if necessary.  Lastly, OlamSVI had a new fluid bed provided by a third-party vendor that they wanted to integrate into the system as well.

In short, the main purpose of the automation upgrade was to provide OlamSVI personnel complete control of operating and maintaining the system, to integrate a new fluid bed for cooling and drying herbs and spices, and to improve overall reliability of the system.

THE CHAMPION SOLUTION

Champion’s solution included installing an Allen-Bradley CompactLogix control system with new I/O cards for about 250 I/O points and programming the controller with an improved sequence of operations based on the newly-developed FRS. As the primary contractor, Champion provided the following services:

  • Design and fabrication of the PLC cabinet;
  • Development of all technical documentation;
  • Development of electrical take-off drawings for use by the electrical installer;
  • On-site supervision of PLC cabinet installation;
  • On-site supervision of fluid bed installation; and
  • Commissioning and initial start-up services.

The Control System combines several technologies, including:

Champion also provided OlamSVI with a Logix5000 developer license and a remote HMI license for their Engineering station, which now allows operators to make changes that may be needed during operation.

The customer refined the graphics standards using a review-and-approval process. Then Champion developed 15 graphics for the Allen-Bradley Panel View HMI that streamlined and standardized operator inputs using FT View Studio for Machine Edition (v9.0) software.

Figure 1. The Sterilizer System Overview graphic allows operators to view and control the system

Figure 2. The Recipes A/B graphic allows operators to modify recipes as necessary in real time.

DESIGN & BUILD

Champion provided design, procurement, programming & configuration, documentation, assembly and testing services. The existing enclosures housed the original automation control system and could not be repurposed, so Champion developed the design package for a UL-Listed NEMA 4X enclosure which housed the PLC and primary Panel View HMI, and two other junction box enclosures with remote terminal strips to tie in the I/O for each line, all of which Steeline Enclosures fabricated.

PROJECT RESULTS 

The project was a success in every respect. Champion’s system enables OlamSVI to better control product quality, as well as to create and adjust recipes. The new system also offers better control in drying processes for food particles smaller than 2mm in diameter.

By adding the ability for OlamSVI to adjust system parameters without outside assistance, the customer can make changes based on operational needs, which makes them more nimble and efficient as well. The solution improved system reliability through reduced downtime. Also, with new complete and accurate reference documentation Champion has improved their troubleshooting efficiency and reduced their troubleshooting costs.

OlamSVI was pleased with the results, which we are confident will lead to Champion being considered as a preferred integrator for future upgrade projects at other facilities.

For more information about this project, reach out to Rama Sinha in the Houston Automation office.

 Kudos to the project team for a job well done!

  • Allen Hobbs, Project Manager
  • Wilson Santamaria, Technical Lead/Engineer
  • Suraj Shetty, Engineer
  • Alexis Adams, Design/Drafting
  • Ryan Partin, Assembly
  • Shawna Grieve and Alana Skipski, Admin Support

 

 


Nevada Environmental Response Trust AP 5 Controls

Project Highlights

PROJECT OVERVIEW

Salt Lake City, UT – Champion’s Salt Lake office commissioned a small wastewater purification project on behalf of Tetra Tech in conjunction with Bodec Electric for installation services.

The project was state-mandated by the Nevada Environmental Response Trust to clean up the fuel waste in the AP5 collection pond that occurred during the 1988 PEPCON plant explosion. The customer installed new storage tanks, pumps, and piping to remove the liquid from the AP5 Pond, remove the solids, dilute the concentration of AP5, and integrate the remaining liquid into the existing treatment facility. New controls and instrumentation were needed for the new storage tanks, pumps, and other equipment.

THE CHAMPION SOLUTION

Work on this project lasted eight months from bid to completion. Champion’s role included providing an automation system to control the new cleanup equipment at the AP5 Pond and integrating the new controls into the existing SCADA system. Champion provided standard services of control system design, programming, panel fabrication, instrumentation specs, and commissioning.

The control system that Champion implemented monitors the storage tank equipment and transfers diluted AP5 solutions to the existing treatment facility in custom batches. Specifically, it monitors the levels of the three (3) main storage tanks, controls the dilution of the AP5 concentration per operator setpoints, and transfers the diluted liquid to the existing treatment facility.

CONFIGURATION DETAILS

The technology associated with this system included Siemens-based hardware and software.

The system required approximately 100 I/O points, Siemens S7-300 logic, WinCC graphics, and TIA Portal-based graphics. Champion developed several new WinCC graphics and updated other existing graphics using the customer’s standards for their existing SCADA system to ensure consistency and to decrease the learning curve for the new controls. Lastly, engineers developed 15 graphics for the local HMI in the AP5 area using TIA Portal for a Siemens Comfort Panel HMI.

Figure 1. HMI Overview graphic.

 

Figure 2. Main Control Screen added to the existing SCADA system.

The new equipment can be both monitored and controlled remotely. To aid with remote monitoring, Champion implemented a new Phoenix Contact AutoDialer system, which will send text messages to operators’ phones and other remote devices any time alarm conditions occur.

DESIGN & BUILD

Champion provided design and fabrication services for two NEMA 4X stainless steel panels: one main PLC panel and one remote I/O panel. As typical, our designers developed unique drawing packages for each panel and loop drawings for all new I/O. All designs and selected equipment were both approved by and met the customer’s previously supplied specifications.

PROJECT RESULTS

The project was completed successfully, on time, and within budget. The system was seamlessly integrated into the existing treatment facility without issue. After installation and commissioning, Champion provided informal training of the system to the operators.
The customer’s representatives provided positive feedback of their satisfaction with the project’s results and Champion’s work on this project. They have also entrusted Champion’s Salt Lake team with additional work, including support work and remote programming services for the controls on the rest of the treatment plant.

For more information about this project, reach out to John Hargroder or Jeff Volpe in the Salt Lake office.
Kudos to the project team for a job well done!

• Jeff Volpe, Project Manager
• Avram Neal, Technical Lead/Engineer
• Sorin Blaga, Engineer
• Dustin Trapier, Design
• David Swenson, Assembly
• Dona Wallgren, Admin Support


Champion Presents at Rockwell Automation Process Solutions User Group (PSUG)

Several Automation Engineers prepared and gave presentations at this year’s Rockwell Automation Process Solution User’s Group (PSUG) to showcase two of Champion’s recently-completed projects.

William Sulzer and Joseph Driver from the Baton Rouge office presented “Migrating from Bailey® INFI90 to PlantPAx® using a Multi-Phase Approach,” while Asa Ange from Lake Charles presented “Phillips 66 Upgrades Automation System and Improves Uptime and Change Management with PlantPAx® Platform” for both he and Daniel Kopsco (on a project startup at the time).  Both presentations were well-attended and well-received by audience members.

Abstracts for each presentation are shown below, and please see this more detailed spotlight article on the Control Global website highlighting the presentation that Asa gave.  For questions on either of the topics or more detailed information on the presentations, please reach out to the Joe, William, Asa, or Daniel.


Joe Driver of Champion (left) and William Sulzer of Champion (right)

Baton Rouge Presentation Team

Joseph Driver, Corporate Application Engineer

William Sulzer, Senior Automation Engineer

Migrating from Bailey® INFI90 to PlantPAx® using a Multi-Phase Approach

Looking to capitalize on extensive in-house Allen-Bradley® knowledge and reduce training time, a specialty chemical producer selected Champion Technology Services, Inc. to oversee the migration of their Bailey INFI90 to the PlantPAx® DCS. Phase 1 of the multi-phase migration laid the foundation by introducing FactoryTalk® View SE graphics and redundant HMI servers, including a live cutover with no downtime and no shutdowns, resulting in improved product quality and operator experience. The planned Phase 2 will complete the migration and is scheduled to occur during the customer’s scheduled downtime before fully migrating the system to PlantPAx®.


Asa Ange of Champion (center) with Scott Thompson of Champion (left) and Justin Ryan of Rockwell (right)

Lake Charles Presentation Team

Asa Ange, Senior Automation Engineer

Daniel Kopcso, Automation Engineer

Phillips 66 Upgrades Automation System and Improves Uptime and Change Management with PlantPAx® Platform

The Phillips 66 Pasadena Terminal is capable of loading 65,000 barrels per day via truck through a tank farm with approximately 3.2 million barrels of storage capacity. The existing pipeline/terminal and truck loading control system consisted of 24 SLC-500s, one PLC-5®, and seven Wonderware standalone HMIs. Phillips 66 worked with Champion Technology Services, Inc. to deliver a solution that would drive consistency across the facility and reduce the number of controllers. The implemented PlantPAx® solution consolidated all logic execution into two ControlLogix®, the backward compatibly of the platform minimized capital investment as existing I/O remained intact, and substantial messaging was eliminated. The result is a single application that increased uptime and improved system maintainability.


 

Champion’s Migration of KMCO Honeywell HC900 to Rockwell PlantPAx

This past November, Rama Sinha, Houston Automation’s Business Manager, presented at Rockwell Automation’s Process Solution Process Users Group. The topic of the presentation was the Houston Automation group’s recent Honeywell HC900 to Rockwell PlantPAx migration for KMCO at their 200-acre petrochemical and manufacturing facility in Crosby, Texas.

KMCO develops various specialty products, including glycol based products, antifreezes, brake fluids, cement grinding aids, and a wide range of oilfield intermediates for formulation into products for the petroleum exploration, production, refining, chemical process, and hydrocarbon additives industries. They provide custom processing for raw materials in various batch sizes, and the facility operates reactor systems, distillation columns, flashers, evaporators, filtrations and a variety of other systems. Prior to the migration, all operations were controlled with redundant Honeywell HC900 controllers that were installed in 2013. Sadly, the system suffered from numerous hardware failures and software issues which occurred two or three times a month, costing the facility approximately $10,000 per month on replacements components and callouts. Some of these were:

  • Communications failures
  • Fiber connection failures
  • I/O card failures
  • Lack of a reliable PC sound alarm on the HMI
  • Slow HMI graphics
  • Easy confusion with many Thin Clients using Remote Desktop

It was ultimately determined that the root cause of the issue was system overload and stress, since the HC900 system designed for small to medium automation projects was not robust enough for the level of KMCO’s use.

An evaluation was done on whether to continue using Honeywell by upgrading to an Experion PKS system or to migrate to Rockwell PlantPAx – a modular and cost-effective automation platform with DCS capabilities. Factors such as the current plant infrastructure using mostly Rockwell-based wiring, the need for robust batch management controls, and the capacity to handle controls for the entire plant as well as future expansions aided in choosing PlantPAx, but the primary factors were significantly shorter downtime for cutover and lower overall system cost.

Figure 1. Totalizer configuration in Honeywell HC900.

Technical Feature

The network architecture for the migration included redundant ControlLogix PACs for centralized logic operations, communicating with four remote I/O racks and various Modbus and RTA modules, as well as a FactoryTalk Server running in a VMWare virtual environment on Stratus hardware serving up data to 17 thin clients for multiple operator stations. The bulk of the migration involved migrating Honeywell function blocks to PlantPAx function blocks. This had to be done manually, but was a reasonable smooth process due to the similarity of function block diagrams in both systems.

Figure 2. Totalizer configuration in Rockwell PlantPAx.

Additionally, PlantPAx’s tools allowed quick manipulation and configuration for signal and calculation blocks. The most relevant advantage of the new system is the ability of the operator to apply parameter changes using the graphic faceplate from the HMI. This allows the facility to troubleshoot, provide maintenance, and optimize the process more efficiently.

It should be noted that the best strategy for configuring routines in ControlLogix controllers (and most systems) is to break up logic into small routines, rather than large routines encompassing each process individually, which allows the controller to process logic quickly, and serve data to update graphics in real time.

Benefits

Figure 4. PlantPAx Configuration Pop-up for Product Totalizer

Figure 3. PlantPAx Configuration Pop-up for PID Controller

PlantPAx increased the flexibility of upgrading hardware and software as well as provided room for future process changes and additions in the entire plant. It also facilitated access to plant information in real-time for increased control. In addition, the facility will get significant use out of the FactoryTalk ProductionCentre components of Inventory Manager, PI Historian, Batch Management, and Vantage Point.

The choice to migrate to PlantPAx aligned with the customer’s goals for future standardization, and PlantPAx required a relatively low number of programming changes. By migrating to PlantPAx, the customer realized the following benefits:

  • Increased Uptime and Production
  • Reduced Maintenance and Troubleshooting Time
  • Improved Safety
  • Improved Operator Readability
  • Increased Operation Efficiency
  • Improved HMI Security
  • Reduced Configuration Time

The Houston Automation group was selected for this project in a competitive bid with three other companies. We believe that several positive experiences for KMCO with two of Champion’s Louisiana offices in the past encouraged the customer to select Champion to complete this project as well.


Freeport-McMoRan Oil and Gas PLC Upgrade

Champion’s Lake Charles, Baton Rouge, and Lafayette offices recently collaborated to complete a topside PLC-5 to ControlLogix upgrade for the Freeport-McMoRan Marlin platform. The project team faced a limited timeline for startup, dynamic existing system logic/features, and a challenging work environment but utilized detailed planning and extensive testing to complete the project within budget and ahead of schedule.

THE CHALLENGE
Marlin is a two-level, 200 foot by 200 foot floating tension-leg production platform located in the Gulf of Mexico. Marlin produces up to 30,000 barrels per day of oil and up to 34 million cubic feet per day of gas.

The Marlin platform generates tens of thousands of dollars in revenue per hour so uptime is important. However, the platform’s obsolete PLC-5s (controlling topside equipment such as the safety system and processes including sending product to and from the separators), unreliable redundancy gateways, and less-than-dependable communications resulted in expensive planned and unplanned downtime. Faced with these challenges, Freeport-McMoRan contacted Champion to submit a competitive bid to replace the PLC-5s with Allen-Bradley’s latest technology offering. Champion won the project as a result of our detailed proposed solution and our reputation as an experienced systems integrator.

Freeport-McMoRan required Marlin’s control system be in operation as much as possible during the system upgrade to prevent loss of platform habitability and loss of revenue. To accomplish this goal, onsite commissioning and testing needed to occur within Freeport-McMoRan’s 45-day planned window.

Additionally, as safety is always a top priority for both Champion and Freeport-McMoRan, Marlin needed to stay completely in compliance with the Bureau of Safety and Environmental Enforcement (BSEE) regulations during the upgrade. BSEE regulations help ensure/promote platform safety, environment protection, and resource conservation. The BSEE also requires the platform to be habitable (with potable water, sewer, air conditioning, etc.) anytime personnel are onboard.

THE SOLUTION
The existing Marlin system consisted of five redundant PLC-5s with approximately 5,000 I/O points, three large ControlNet networks, and 69 remote I/O racks throughout the platform. Prior to the PLC project, Champion upgraded the redundant Honeywell Experion operator interface which serves as the HMI for the platform.

Champion’s upgrade solution replaced the aging five PLC-5s with four redundant ControlLogix processors (version 20.55) and moved the critical ControlLogix communications to a fault tolerant DLR Ethernet network. Champion also provided drawings, detailed review of the system SAFE chart and Cause and Effect charts, extensive factory acceptance testing, and onsite commissioning.

Champion developed a Habitability Plan to address the requirements necessary for certain processes to keep running so the platform could remain habitable with personnel onboard during the upgrade. Our engineers carefully planned the system implementation, researched which electrical points could remain connected, and determined how to bypass essential systems to ensure vital processes (generators, potable water, etc.) would not be shut down.

Throughout almost a year of project execution, Champion received change requests from Freeport-McMoRan and made required modifications to the PLC-5 system (including software and hardware updates, process changes, feature additions, etc.). Our team created new logic that was in sync with the evolving system so the changes were not lost during the conversion.

RESULTS
The project team finished startup of the system in only 21 days versus the initial estimated commissioning time of 45 days. Extensive planning, detailed research, thorough factory acceptance testing, and verification of the Cause and Effect charts streamlined the commissioning process. Champion and the knowledgeable technical liaisons at Freeport-McMoRan were able to upgrade five major systems in less than half the anticipated downtime, even with many commissioning interruptions such as underwater welding, mechanical tasks, etc.

Champion simplified the hardware with the control system replacement which resulted in increased reliability and conservation of valuable real estate on the offshore platform. Additionally, the upgrade included new HMI functionality such as color additions and information for pumps and valves that increased readability for operators. PID tuning was brought into the HMI stations, and hardware failures were made visible at the HMI level which improved diagnostics and ease of troubleshooting.

The Marlin upgrade required a diverse, high level of technical skills, and Champion was able to deliver an impressive solution by utilizing our dynamic resource base from three of the Gulf Coast offices. With efficient management, collaboration, and strong technical lead efforts, the three offices worked as one unit to provide a successful system upgrade.

“Thanks and job well done to everyone involved in the PLC upgrade project and associated PLC hardware and logic changes for Marlin! Enough cannot be said for the appreciation of the efforts put into making this project a success.”