Upcoming Events:

Click here to check out the brand new ISPE Boston Area Chapter Blog!

Value Proposition - I/O Simulation & Testing

by Eric Bird

Innovation in automated systems can often be a challenge in pharmaceutical manufacturing. While other industries have been implementing plant-floor and information-control systems for many years to realize gains in quality, efficiency and productivity, cGMP regulatory requirements have often been perceived to stifle the use of innovative practices or technologies. Consequently, the gains other industries have been realizing through upgrades to process control and information management systems have been to a large extent missed by pharma in relation to:

  • new facility initiatives and
  • implementation of process improvements on existing validated production systems.

Quite simply, the stringent regulations, while serving a vital quality and safety purpose, have created a situation in which risks associated with change are prohibitively high. So high that once a validated system is in place and product is moving out the door, the overriding question becomes, “Why change a functioning process and open up to unnecessary risk?”
Against this backdrop of high risk associated with change, innovation becomes a secondary issue. And understandably so. Consider that many pharmaceutical companies that have forged ahead with process upgrades have done so only to become painfully aware of the “10 times” rule: It takes 10 times as long to make a system change during each subsequent phase of project delivery - design, engineering, commissioning, and finally qualification.

Typically, pharmaceutical manufacturing environments consist of a wide variety of types of equipment made by multiple vendors. The equipment often runs on proprietary software, with each piece of equipment presenting a different look and feel. Network communications, controls and data integration can be extremely complex and difficult to troubleshoot. When testing is executed, software issues are often not discovered until late in the project cycle (i.e., during system installation and commissioning or, worse yet, during qualification) and troubleshooting becomes a significant challenge that negatively impacts operations and the bottom line. Companies that have experienced testing and associated troubleshooting under these conditions necessarily hesitate when talks of future upgrade initiatives percolate. The question “Why change?” magnifies, and any persuasion to change has to be nothing short of compelling.

The good news for the pharmaceutical industry is that there has been a shift of thinking within the FDA. The agency now recognizes that:

  • innovation is vital for the industry to remain globally competitive and
  • regulatory control and innovation must be harmonized.

Accordingly, the FDA is promoting innovation in manufacturing. Pharmaceutical manufacturers are challenged to think more critically about production processes and consider embracing new technologies. Many pharmaceutical companies will be proactive and pursue change. Those that do must still do all they can to help mitigate risk by applying the risk-mitigation tools and practices that are available in the marketplace.

I/O Simulation is a practice used by select controls integrators to manage the implementation of new automated systems. I/O Simulation involves the use of PC-based software to simulate physical field devices such as instruments, sensors, and unit operations for the purpose of testing control system code. A graphic illustration showing a comparison between traditional testing in the production environment and I/O Simulation testing of controls systems is displayed as follows:


The top half of the graphic (entitled Production Environment) represents the traditional approach in which testing of the control system source code is conducted after all equipment and devices are fully installed in the manufacturing facility. When traditional testing is applied, the root causes of problems are difficult to isolate; installation and commissioning times are prolonged and testing is constrained by equipment availability in addition to physical and time limitations.

The bottom half of the graphic (Simulation Environment) represents the I/O Simulation model in which control system software is tested in an offline environment. The software is programmed to make the system that is being tested believe it is connected to physical field devices - those inputs and outputs that it is programmed to control.

Simulation allows engineers to create a lab model that mimics the real-time, dynamic behavior of the physical field units and provides the controls systems with feedback. To these systems, there is no difference between controlling the simulated I/O and the actual process. In this way, the software can be tested, debugged, and challenged before it goes near the floor. This means that the majority of software issues are identified and eliminated early in the project cycle, essentially taking software off the critical path. Owners are assured that the field commissioning stage is not used for debugging software but, instead, is used for mechanical and system integration and final system testing.

Factor in a risk-based validation approach to the overall effort and the testing performed in a simulated environment can be leveraged to meet validation requirements and thus reduce the overall validation testing in the field using the actual production equipment. The primary focus of the validation effort is then less on retesting every feature and function of the software (as it has already been tested using simulation) and instead is targeted on higher-level system testing to ensure that equipment is fit for use to make product.

In addition to debugging software and streamlining validation, the advantages of this testing strategy are numerous and well documented. Consider the following:

  • Thoroughly tested emergency and fault-handling procedures assure users that the first true test of sections of PLC logic is not during an actual emergency.
  • Code that has been “stressed” in simulation challenges hard-to-test scenarios, verifies security parameters, tests alarms, and uncovers potential for mistakes without endangering production, equipment or personnel. This is especially effective for complex systems or systems with a high risk of endangering operators or equipment.
  • Operators can be trained well in advance of the equipment arriving on site. Involving operators early in the process helps them better understand the sequence of “live” operations and the situations they will face on a daily basis.
  • Simulation allows for early development of Standard Operating Procedures (SOPs) and operator instructions, which can be applied to the training of new operators or to reviews of the impact of potential upgrades or changes to the system.
  • Lastly, simulation is a very effective way to conduct design reviews with the many stakeholders involved in a new system install.

On a recent project involving the delivery of a new powder fill facility, the ROI for applying simulation was very high - over 400 percent before even factoring in the key performance measure of faster time to market. Other highlights of this project included resolving over 500 software discrepancies prior to commissioning, completing automation work under budget and reducing the overall schedule by 36 days.

Understanding how to apply a well-executed simulation and testing program to mitigate the risk inherent in any new systems or upgrades involving technology helps pharmaceutical manufacturers innovate with confidence. Experience has shown that a simulation program can significantly increase the probability of success on capital projects involving automation, process control systems, manufacturing IT and manufacturing execution systems, and essentially takes software off the critical path. And being able to count on the success of a project is vitally important to regulated manufacturers, whose projects are subject to risk factors from many angles - schedule, cost, quality, regulatory compliance, and time to market.

Eric Bird is a practice leader for the Life Sciences Industry Division of Brock Solutions, an engineering solutions company that specializes in projects requiring automation and drives engineering, information management/manufacturing execution systems (MES), manufacturing IT, and project and installation management. Eric has 10 years experience in the implementation and qualification of plant-floor controls, automation, data management, and manufacturing execution systems. He can be reached at 519 572 0382 or ebird@brocksolutions.com.

Page last updated: 5 March 2009