Industrial cybersecurity has always been and remains a moving target. Awareness of the challenge that cyber-attacks represent for industry, infrastructure, and the public sector has increased significantly over the last several years, along with associated investments. However, both the sophistication of the attacks and the number of possible attack vectors have also increased.
Certainly, the large number of legacy systems designed and installed before the term, “secure by design,” entered the popular lexicon remain particularly vulnerable. However, ongoing vigilance is needed to keep even much of today’s current operational technology secure from disruptive, costly, and potentially life-threatening incursions.
Considering the damage that a serious cyber incursion could cause to a company’s operations, reputation, and bottom line; it’s no wonder that the topic has become top of mind for everyone from shift supervisors on the plant or factory floor to engineering and risk management executives, CSIOs, CIOs, and even CEOs, who can - and have been - named in associated lawsuits.
Most professionals reading this report will already be aware that the ongoing convergence of information technology (IT) and operational technology (OT), utilization of essentially commercial off-the-shelf (COTS) technologies in industrial and other critical environments, and the increasing adoption of both IoT-connected devices and Industrial IoT-enabled solutions further compound the already complex cybersecurity challenge.
For over a decade now, ARC Advisory Group analysts and consultants have been tracking technology trends in cybersecurity for our industrial automation clients. In recent years, this coverage has broadened out to also encompass cybersecurity for our growing base of Industrial IoT, infrastructure, and public sector clients. This includes those responsible for developing, managing, and operating our supply chains, smart transportation networks, and smart cities.
This ARC Strategy Report, a compendium of selected ARC cybersecurity-related reports published since 2013, shares many of our key industrial and other cybersecurity-related learnings.
One of the common themes that emerge from these reports is the clear and pressing need for technology and service suppliers, integrators, end users, industry groups, and government agencies to collaborate; share relevant information; and agree on appropriate standards, frameworks, and best practices. What’s not so clear, however, is how to accomplish this without exposing that information to bad actors who could then use it to reveal potential new attack vectors and specific vulnerabilities. The difficulty in being able to tell the “good guys” from the “bad guys” is another vexing problem, particularly since many of to-day’s good guys were yesterday’s bad guys, and vice versa.
Another common theme is the need for close collaboration on cybersecurity between IT, OT, engineering, and senior management within the same company, public utility, or municipality. This is needed to establish common goals and targets, identify potential vulnerabilities, and implement countermeasures that work for everyone. This type of internal cooperation and buy in by all parties also helps organizations to develop appropriate work processes and practices, cultivate a cybersecurity-aware company culture, and “keep it real.”
Since resources with the requisite expertise in cybersecurity in general and OT cybersecurity in particular are in short supply these days; many end users will need to look outside the organization for assistance to help identify cyber-related vulnerabilities and associated risks and establish the appropriate technical and other countermeasures. This is not necessarily a bad thing, since it can provide a relatively cost-effective way to supplement an organization’s internal resources while bringing in some fresh thinking and/or cross-industry expertise.
The ARC Advisory Group cybersecurity team hopes you find the insights and recommendations provided in this compendium report helpful. As always, we welcome your questions, suggestions, and feedback.
By Sid Snitkin
(Editor’s note: While “cyber-apathy” is less of a problem today than when this was first published on Sept. 19, 2013, much of the content remains relevant.)
Events like Stuxnet and Shamoon have increased concern about the cybersecurity of industrial control systems. Few systems are sold today without the firewalls and software needed to provide multi-layered defense against cyber intrusions. But ARC Advisory Group has observed that efforts to protect the enormous base of legacy systems appear sluggish at best.
Pundits offer a variety of explanations and solutions for this apparent apathy. Some argue that industrial managers are still unaware of the risks, so they prescribe more education and encouragement, rather than tangible action. Others recommend more standards and guidelines because they see the problem as an inability to formulate an action plan. Still others think that this behavior is just due to corporate irresponsibility or irrationality, and they call for more stringent regulations and compliance. [Former] President Obama's Improving Critical Infrastructure Cybersecurity Executive Order seems to imply that all these gaps exist, and all the solutions are needed.
Will any of these suggestions really address the apparent apathy of industrial organizations in dealing with their cyber problems? Or, is apathy appropriate when the facts are filtered from all the Y2K-style hype? In that case, do we need to accept our vulnerabilities and prepare for periodic cyber events, or should we be considering other preemptive actions? Understanding the true causes and solutions for cyber apathy is critical to ensuring the safety and security of our industrial infrastructure.
Is the Cybersecurity Awareness Gap Real?
With cybersecurity reports plastered across industry publications, websites, and newsletters, it’s hard to believe that managers of industrial plants and facilities are still unaware of cyber threats. But it’s not all that surprising that much of this information is being ignored. Most managers simply don’t understand cybersecurity jargon or how a vulnerability in a social site or smartphone app could become the foothold for a major plant attack. Instead of prompting action, such reports probably evoke memories of similar coverage that misled them regarding Y2K. And, like the periodic orange alerts in airport security, over-exuberance in warning people of threats causes concern to wane, not increase.
More information is not the answer; but improving information relevance might have an impact. Industrial managers are trained problem solvers who respond decisively to issues they understand and can address. News of an attack on another plant in their industry would trigger most industrial managers to launch a review of their own defenses and an action plan to plug any holes. But this only occurs when the information is relevant, timely, and trustworthy. Like people facing a hotel fire alarm, industrial managers can be reluctant to act until they know that others, in similar positions, are taking the threat seriously. Efforts to improve collaboration and cyber information sharing among industrial peers could help on both fronts.
Does Industry Need More Cybersecurity Guidance?
Recent reports on the threat of cyber warfare by nation states or terrorist organizations are raising public concern. But this is hardly a new issue for industrial organizations, many of whom have been actively addressing cybersecurity for over a decade. Over this time, groups like ISA and NERC have developed various guidelines and standards. ISA began its work on cybersecurity in 2002 and released the first ISA99 standard in 2007 (ISA-99.00.01-2007). Efforts in the power industry have a similar history with NERC CIP version 1 being released in 2008. Both organizations continue this important work with the new ISA/IEC-62443 series of standards, and NERC CIP version 5. Government groups like NIST and ICS-CERT have also produced various cybersecurity guidelines, tools, and standards for industrial control systems.
Cybersecurity threats are dynamic, so these standards require ongoing updates. Education about existing guidelines could also be improved. But it is hard to believe that industrial organizations need more standards or additional guidance to better understand what they need to do about industrial cybersecurity.
ARC research suggests that apathy lies more in C-level executives and business managers than the technical people responsible for designing and implementing cybersecurity programs. If gaps in standards and guidelines exist, they lie in areas that enable transparency and business understanding of the organization’s real security risks and status. Homogenization of existing standards and development of meaningful cybersecurity metrics are two steps that could be taken to improve management’s understanding.
Is Industry Behavior Irrational or Irresponsible?
Lack of action by organizations that are aware of threats and understand the solution can appear irrational or irresponsible to outsiders. Surely, there are cases where this assessment is correct, leaving plants inadequately protected, but it would be a mistake to assume that this is true for every organization. Cybersecurity is a risk management challenge and organizations can rationally decide that the risks do not justify immediate action. Forcing action through regulatory mandates may be warranted in some cases, but wasteful and counter to free market principles in others.
Organizations naturally evaluate investments from their own perspective. Most care about their communities, but no one should expect impact on the public to play a major role in their analyses unless these affect the organizations’ profitability, reputation, or service commitments. Individuals, as well as organizations, tend to be self-serving.
While parochial perspectives are understandable, there are clearly situations where the government should be proactive in forcing industry to act in addressing cybersecurity. For example, a major disruption of airline service could seriously impact businesses throughout the economy. In today’s internet-connected world, loss of telecommunication services could have even greater impact. And, of course, every business depends on reliable electric supply. Most people would agree that some form of government encouragement, like regulations or cyber investment tax credits, are called for in these kinds of situations.
While we discount the value of more cyber-awareness training, educating industrial companies in cyber risk analysis could have merit. Organizations can easily underestimate the probability of rare events like cyber-attacks because they have never experienced them (see research on “black swan” and “long tail” events). Many organizations also underestimate the benefits of cybersecurity investments in blocking malicious and unintentional “internal threats” to control systems. Education on the broad impact of cyber-attacks is another fruitful area to consider. Impacts are hard to identify and evaluate. Providing organizations with checklists of the kinds of things that have occurred in similar organizations could help them overcome some cyber security investment hurdles.
Regardless of what the government does to influence industrial risk management decisions, it is important to prepare the public for less-than-perfect cyber protection. Cybersecurity will never be foolproof, and we all need to expect that some attacks will get around even the best defenses. Efforts to educate the public will encourage ongoing vigilance in personal behavior and temper the hysteria that may arise from the never-ending barrage of Y2K-style doomsday reports.
Based on ARC research and analysis, we recommend the following actions for those involved with industrial cybersecurity:
- Focus educational efforts on improving the relevance, timeliness, and trustworthiness of threat and vulnerability advisories; helping the public understand cybersecurity risks and the need for ongoing vigilance.
- Focus standardization and guideline efforts on continuously updating and unifying existing standards; developing metrics that enable business leaders to overcome technological jargon and understand the true security status of their facilities; and developing information, like impact checklists and industry-specific cyber-threat lists to guide organizations in their cyber-risk analysis efforts.
- Focus regulatory efforts on industries and events that can have broad impact on the economy and public safety
Fundamental Concepts for Industrial Cybersecurity
By Eric C. Cosman (originally published June 6, 2019)
For nearly two decades, we’ve seen considerable emphasis placed on the need to protect the integrity of the various elements of the critical infrastructure. In response to this imperative, industrial cybersecurity has become a well-defined discipline, complementing the more mature discipline of information security. While many of the practices employed in this newer discipline are like those used for information protection, others have evolved to address the specific characteristics and constraints of the industrial systems environment.
While many asset owners understand and have accepted the imperative to improve the security of their industrial systems, they still face significant challenges. Industrial cybersecurity is a complex and specialized subject that can be intimidating for those not deeply involved. The diversity of available sources of guidance further worsens the situation.
It is often difficult to select the specific guidelines and standards most appropriate for a particular situation and, once selected, it may be difficult to understand some of the complex associated concepts.
Just as with other technical disciplines, professionals working in industrial cybersecurity must do everything possible to clarify their message and explain their technology and practices to those requiring services in this area. It’s important to avoid unnecessary jargon and it helps to use a well-defined set of fundamental concepts.
Industrial Cybersecurity as a Discipline
The imperative to improve the security of automated systems employed in critical infrastructure has led to the emergence of industrial cybersecurity as a professional discipline. This complements the more established discipline of traditional information security. To secure complex industrial processes it is necessary to combine the skills and experience from traditional information security with those from instrumentation, automation, and process engineering to be able to identify the most effective response.
Although protecting critical infrastructure has been a primary driver, the practices associated with this developing discipline are also suitable to apply in less-critical sectors where ensuring the reliability and integrity of their processes is key to both safety and profitability. Sectors as diverse as petrochemicals, pharmaceuticals, and discrete manufacturing employ common automation systems and technology so it is reasonable to assume that security practices would also commonly apply.
While practices such as least privilege, defense-in-depth, and separation of duties have long been applied to secure business and enterprise systems, others have evolved or been adapted to address characteristics and constraints that are specific for industrial applications. To be most effective and appropriate in this context, these practices must be based on and supported by an appropriate set of policies, principles, models, and requirements. Collectively, these elements form a body of knowledge that provides the foundation for training and certifying experts in the discipline.
Challenges for Asset Owners
Although many available standards and guidelines provide much of this information, there are overlaps and possible inconsistencies. Several of the available resources remain industry- or sector-specific. Asset owners are unlikely to have the time or specialized resources required to become industrial cybersecurity experts. Those with operations that span multiple sectors may be confused as to what guidance is most effective for their situation.
Asset owners need simple and practical guidance to help take the necessary steps to improve the availability and resilience of their systems and associated processes. They must be free to spend less time trying to interpret and understand what is available to them, and more time to applying this information. While a considerable amount of information is available in the form of standards, practices and guidelines, there are often inconsistencies in some concepts and terminology. This leads to confusion and uncertainty when attempting to apply this information, which can then lead to an ineffective response.
In conversations with asset owners and end users, it is common to hear requests for checklists or similar documents that provide step-by-step direction as to the measures that are required to secure systems. Unfortunately, the complexity of the subject combined with the wide variety of possible configurations makes such an approach impractical for anything beyond the simplest of measures. Collectively, these simple measures are often characterized as “cybersecurity hygiene.”
Although these simple measures are important, they are not sufficient. Just as with other more traditional technical disciplines, assembling a comprehensive response must start with understanding and interpreting a set of basic concepts.
Fundamental Concepts Emerge
As the industrial cybersecurity discipline has matured, several fundamental concepts have emerged in widely accepted and adopted standards and practices. The terms used may be different, but the basics are essentially the same.
In all but the simplest of cases, industrial automation systems can be quite complex, consisting of a combination of hardware and software products and components, focused solutions (i.e., addressing a specific function), and smaller self-contained systems or subsystems. This can lead to confusion as terms like “system,” “solution,” and “product” are often used inconsistently or even interchangeably, resulting in confusion and lack of clarity.
Systems engineering provides methods to describe how such systems are constructed. It begins with defining the taxonomy that classifies the elements used to construct an industrial automation system and how they are related. In the case of industrial automation, the overall systems consist of one or more focused solutions (e.g., batch, optimization, etc.). Each of these solutions is constructed using one or more products that are in turn constructed using components.
Just as a taxonomy helps characterize the technology used to create automation systems, principal role descriptions can help describe the people dimension. Although people may serve in many roles in the operation of cybersecurity management systems, a small number of principal roles must be clearly defined. Each may be described in terms of the activities it performs and its duties (responsibilities or accountabilities).
- The asset owner is accountable for all decisions related to operation of the automation system and responsible for providing the necessary policies and procedures.
- The product supplier is accountable for the inherent security-related feature of their products and responsible for developing these products using proven and accepted processes.
- The integration service provider is responsible for assembling the automation system using the necessary subsystems, solutions, and components.
The maintenance service provider is responsible for activities required to keep the automation system operating in a safe and secure manner.
Lifecycles and Processes
The cybersecurity response must address all phases of the product, system, and solution lifecycles, from conception, development and delivery of products and systems through their implementation, operation, and support.
Product and technology suppliers use a product lifecycle that requires them to take the necessary steps to ensure that, to the extent possible, their products are “secure by design.” Although such products can operate securely in an integrated system, it is still possible to deploy and operate them in a less-than-secure manner. The asset owners and service providers must address security during specification, integration, operation, and support.
Understanding lifecycles and how they are connected is essential to secure automation systems. Suppliers must share the capabilities and limitations found during development with customers, and asset owners and service providers must share their experiences with suppliers to allow them to improve the security of their products.
Most industrial cybersecurity standards and guidelines identify the need to segment complex systems based on assessed risk. This is essential as all but the simplest of industrial automation systems include many subsystems and components employed to control various parts of the underlying process. Just as with safety protection, the risks associated with some segments are higher than with others. Although the threat and vulnerability components of risks may be common, the consequences are not.
Methods and tools such as process hazard analysis (PHA) used in safety systems design are often adapted to assess security risk. With the results of these analyses it is possible to segment a complex system and apply compensating countermeasures that complement the intrinsic security of individual elements. The use of powerful firewalls or unidirectional gateways in high-risk portions of the network is a common example.
Just as with safety, there must be a simple way to describe a security level for specific parts of a complex automation system. The ISA/IEC 62443 standards define a qualitative set of levels based on the perceived nature of the threat. Similar scales could be developed based on potential consequence. Regardless of the details, it is important to identify the target and achieved security levels, as these help to choose appropriate compensating countermeasures and assess the effectiveness of the response. For example, if the achieved target security level of a portion of the system falls short of the target level, further countermeasures may be required.
The ISA/IEC 62443-3-2 standard describes a detailed approach to risk assessment and system segmentation. Approved by both IEC and ISA, this standard is expected to be available later this year. (Editor’s note: Or, more likely, early in 2020).
Cybersecurity management programs evolve and improve over time as risks change and new and improved capabilities are added. Ways to assess the state of the program at any given point in time as well as to identify areas for improvement are needed.
Maturity models provide a method for doing this. The ISA/IEC 62443 standards have adapted the capability maturity model concept used in areas such as software development and quality management. These define four maturity levels: initial, managed, defined, and improving. The standards apply this concept to product or systems development as well as operation and support.
ARC’s maturity model provides a framework for non-technical managers to understand the risk reduction benefits of cybersecurity technologies and the coverage of supplier products. Each step in this model addresses a specific, easily understandable security issue like securing individual devices, defending plants from external attacks, containing malware that finds a way into the control system, monitoring systems for signs of cyber-compromise, and managing active attacks and cyber incidents.
Proceeding sequentially through the steps ensures that the integrity of each layer is sustained. Each step has an associated set of people, processes, and technologies required to accomplish its goals. The associated security technologies indicate the kinds of solutions that companies should consider in building defenses that achieve the steps’ respective security goals. Security management technologies are needed to select, implement, and sustain the effectiveness of the spanned security technologies.
Security Program Rating
Security program rating is a more recently developed concept related to industrial automation cybersecurity. It extends the concept of security level by considering how the underlying automation solution is operated and maintained. It indicates the level of confidence for reaching a certain level of protection against cyberattacks that an organization can place in a security program. Recognizing that security results from a combination of people, process, and technology; this level of confidence is based on both technical and organizational measures. Weaknesses may come from each of these legs, so all are necessary contributors to a comprehensive defense-in-depth strategy. Each must be evaluated when rating the security program of an IACS in operation.
The people dimension relates to the skills and competence of personnel who perform these activities. The process dimension is composed of security-aware operational and administrative activities needed to install, configure, operate, and maintain the automation system. The technology dimension is composed of security measures that are both electronic and physical in nature. This concept is described in more detail in the ISA/IEC 62443-2-2 standard.
Taken collectively, the above fundamental concepts provide a solid foundation for an effective cybersecurity program based on established practices.
Based on ARC research and analysis, we recommend the following actions:
Apply and comment on concepts – Asset owners should use these concepts as the basis for their cybersecurity response. Comments or suggested improvements can be sent to the author for consideration in future research.
Use common terminology – Consultants, advisors, and other experts who are members of the industrial cybersecurity discipline must challenge themselves to use clear and consistent terminology when working with asset owners and other clients. This is particularly important when addressing challenges across sectors as there is often a temptation to use industry-specific jargon.
Engage stakeholders – Asset owners and others faced with defining their cybersecurity programs must take great care to engage all the necessary stakeholders, from operations staff to members of the various engineering disciplines. Industrial cybersecurity is inherently cross-functional, and success depends on including all perspectives.
Highlight what works – Perhaps the most important recommendation for the asset owner is to take the opportunity to share their experiences with what has been most effective in their facilities. Asset owners are more likely to communicate in terms that their peers will understand, without resorting to confusing and complex jargon.
ARC’s Industrial Cybersecurity Maturity Model Evolves
By Sid Snitkin (originally published May 9, 2019)
ARC Advisory Group released its first Industrial Cybersecurity Maturity Model in 2016 to help clients manage industrial cybersecurity investments. The model provided a framework for non-technical managers to understand the risk reduction benefits of cybersecurity technologies and the coverage of supplier products. The underlying structure showed the need to align people, processes, and technology investments to help ensure that security benefits are achieved.
Industrial cybersecurity has changed significantly since ARC released that first model. High profile, sophisticated attacks have increased demands for better visibility of control system cyber risks. Integration of IT and OT cybersecurity programs drive the need for increased system access by remote support teams. Digital transformation programs require new approaches to ensure secure deployment of multitudes of new, potentially insecure devices within plant boundaries. ARC’s new industrial cybersecurity maturity model provides the additional information managers need to manage industrial cybersecurity strategies in this new reality.
ARC Industrial/OT Cybersecurity Maturity Model Version 2.0
The figure below shows ARC’s new industrial/OT cybersecurity maturity model. It structures industrial/OT cybersecurity as a sequence of steps that organizations should take to build a cybersecurity program that meets their risk management goals. The model’s incremental nature enables managers to balance program costs with their company’s respective tolerance for risk.
Each step in ARC’s model addresses a specific, easily understandable, security issue like securing individual devices, defending plants from external attacks, containing malware that finds a way into the control system, monitoring systems for signs of cyber compromise, and managing active attacks and cyber incidents. Each step adds a layer of protection that prepares the facility for more sophisticated cyber-attacks. Proceeding sequentially through the steps ensures that the integrity of each layer is sustained.
Each step has an associated set of people, processes, and technologies that are required to accomplish its goals. The associated security technologies indicate the kinds of solutions that companies should consider in building defenses that achieve the steps’ respective security goals. Security management technologies (shown separately) are needed to select, implement and sustain the effectiveness of the spanned security technologies. Companies should strive to have the associated people, processes, and security management technologies in place before implementing the associated security technologies.
Color is used to distinguish reactive and proactive cybersecurity maturity levels. The blue, reactive levels reduce the likelihood of a system compromise by blocking unauthorized access and detectable malware. The orange, proactive levels, add capabilities that minimize the impact of an actual compromise. These steps address the people, processes, and technologies needed to rapidly detect and respond effectively to these events.
Relationship to Other Industrial/OT Cybersecurity Models
A key goal of ARC’s model is to help managers evaluate cybersecurity technology investments in light of their unique cybersecurity concerns and tolerance for risk. This necessitated development of a sequential implementation model. This structure is different than popular references like the NIST Cybersecurity Framework. For example, the three reactive steps of ARC’s model cover the NIST Identify and Protect categories; while the two proactive steps in ARC’s model cover the NIST Detect, Respond and Recover categories.
The structure of ARC’s model helps communicate the specific security benefits of different technologies and the need for coordination between the implementation of technology and the organization’s investments in security management technology, processes, and people. These additional investments represent the organization’s cybersecurity maturity, which is a measure of the organization’s ability to derive value and manage the effectiveness of its security technologies.
While structure and terminology vary, ARC’s model still incorporates the technology guidance provided in the NIST Cybersecurity Framework and industry standards like ISA/IEC-62443 and NERC-CIP.
Security for Evolving Technologies and Architectures
ARC’s model goes beyond the recommendations in recognized standards and guidelines by including emerging technologies, like edge gateways, new practices, like PKI, and new strategies, like integration of IT and OT cybersecurity programs.
Business imperatives are driving companies to adopt these developments despite the lack of industry guidance. We’ve included these developments in the model to help cybersecurity teams understand, anticipate, and prepare for these developments.
ARC’s cybersecurity analysts periodically review and update the industrial/OT cybersecurity maturity model based on information from colleagues across the globe. This includes developments in control system technologies, management practices used by leading industrial companies, and new business strategies. We’ve also incorporated new technologies and practices being used within the security community.
ARC’s model provides a useful tool for planning and tracking cybersecurity investments. It can also help companies evaluate technology solutions and suppliers based on how well they cover critical issues in the model.
ARC’s initial model proved to be a powerful and relevant planning tool for many of our industrial clients. ARC reports on the current state of industrial cybersecurity have used the model to describe the critical resource and technology gaps that industrial companies must address. The model has also been used to communicate unique capabilities of cybersecurity technology providers and suppliers of industrial cybersecurity services. The new version of this model extends its explanatory power without compromising its inherent simplicity and compactness.
End users can apply ARC’s industrial/OT cybersecurity maturity model in comparable ways to assess their cybersecurity programs, identify and describe gaps, and justify needed investments to top management. Since the model incorporates recognized guidance, users can leverage assessments based on the details in these sources and use ARC’s model to aggregate and present the findings. This will highlight the need to address mismatches between technology investments and the organization’s ability to fully-leverage these capabilities in reducing cyber risks. It will also help cyber professionals address the false sense of security that often emerges after managers authorize technology investments.
Make Industrial Cyber Resilience Your Goal
By Sid Snitkin (originally published Oct. 12, 2017)
Operational continuity is critical for industrial and infrastructure organizations. Disruptions to normal operations are costly and jeopardize safety, compliance, and company reputation. Reliable equipment and control systems are necessary, but not sufficient. Organizations also need to address cyber resilience to limit the impact of unanticipated failures on key performance indicators for operations.
Reliability has long been a key goal in the design of industrial plants and infrastructure systems. These efforts eliminate many potential problems but are not foolproof. Some failure scenarios will be overlooked, and protective measures limited, particularly when facilities include legacy equipment. Cyber attackers will likewise find ways to overcome even the best defensive measures.
Smart organizations recognize these residual risks and invest equally in measures to ensure that operations are resilient. This includes management practices and technology that enables early detection of unexpected failures and rapid recovery.
Industrial organizations employ a variety of practices that minimize the impact of unexpected mechanical equipment failures. Periodic inspections are used to detect failing components. Vibration sensors are used for early detection of problems in rotating equipment, like turbines and generators. Early detection enables staff to coordinate repair outages with customers and prepare parts and resources to minimize the downtime.
Such techniques do not work for digital systems. Digital systems’ failures are random and lack advance indicators. This situation frustrates engineers and plant managers, as digital technology is increasingly being employed in control and other plant systems. As a result, it’s becoming increasingly important to have visibility into the health of digital assets. Companies also need a way to promptly detect latent malware and sophisticated cyberattacks that evade detection by firewalls and anti-virus software.
Cybersecurity technology for anomalous network message detection can provide an answer to this dilemma and thus enhance industrial cyber resilience. These solutions monitor network traffic within control systems to quickly detect and identify abnormal activity, whether a cyber-attack or digital system failure, to speed remediation.
What Is Industrial Cyber Resilience?
Resilience is the capability to quickly identify and recover from problems. Techniques like network segmentation, functional isolation, and redundancy are used to build basic resilience into control systems. But these measures only protect networks and operations from certain failure scenarios. Other problems, like misconfigurations, programming mistakes, and poor operations and management practices, can still undermine the operation and security of control systems.
Anomalous network message detection solutions help companies address these gaps. They alert operators and maintenance personnel to erroneous and undesired system behavior, thereby increasing the control system cyber resilience. In addition, they provide needed context to help operators diagnose and resolve problems.
ARC research indicates that anomalous network message detection solutions with enhanced capabilities for industrial cyber resilience are already helping plant staffs detect a wide range of control system problems before they can impact operational performance. These include:
- Failures of non-operating, backup processors
- Failures of intermittently used controllers
- Error states and malfunctions of key controllers
- Inadvertent configuration and programming errors
- Inappropriate operational requests (invalid setpoints, etc.)
- Latent malware communicating with command and control sites and other control system elements
Anomalous message detection solutions are also valuable in situations where control system problems immediately disrupt operations. They provide information to minimize the time needed to identify the cause of the failure, whether it is a mistake or malicious cyberattack, and quickly restore operations.
Industrial Cyber Resilience Benefits Many Stakeholders
The benefits of industrial cyber resilience are broad-based and address key concerns of stakeholders across industrial organizations, including plant managers, plant engineers, chief information officers (CIOs) and information technology (IT) managers, and chief information security officers (CISOs).
Plant managers are directly responsible for the safety, costs, and revenues of their facilities. They understand how deviations in normal operations increase the likelihood of safety incidents. They are also keenly aware of the costs and customer issues that arise when operations are not immediately restored. While they appreciate that control system failures will occur, they are rightfully frustrated when restoration is delayed by complications in understanding the source of problems. The benefits they receive from reduced outages and outage durations are clear and significant. Industrial cyber resilience provides this through its ability to anticipate cyber failures, reduce downtime risks, enable predictive maintenance, increase productivity, and reduce the costs for problem mitigation.
Plant engineers are responsible for control system reliability. They work with vendors to ensure that as many system failures as possible are anticipated, appropriate protections are incorporated, and adequate spares and support are readily on hand to keep systems operating. They have a vested interest in ensuring that unanticipated failures are promptly addressed before they cause accidents or disrupt operations, and that repair time and effort are minimized. Industrial cyber resilience directly supports these goals through early indicators of problems and threats, minimized troubleshooting effort and resolution time, enhanced reliability and availability of control systems, etc.
CIOs/IT managers are often responsible for control system servers, workstations, and networking equipment. Their interests in system reliability align with the plant engineers. Likewise, they are concerned about operational disruptions being caused by networking failures and misconfiguration. These managers receive significant benefit through identification of misconfigured equipment and network services, validation of network changes and maintenance operations, quick identification of net-work failures, etc.
CISOs are responsible for cybersecurity across the organization. They also have primary responsibility for managing the organization’s risks and compliance. However, their typically limited understanding of control systems frustrates efforts to assess overall security posture. Constraints on control system defenses and updates is also a major concern. The additional layer of defense provided by industrial cyber resilience helps reduce these risks and alleviate many associated concerns. Industrial cyber resilience provides CISOs with visibility into what is happening behind OT firewalls, support for compliance, reduced exposure to cyber threats, detection of cyber-compromises, etc.
What Makes an Effective Industrial Cyber Resilience Solution?
Effective anomalous network message detection is fundamental for an effective industrial cyber resilience program. But many solutions in this category lack the essential features that companies need to rapidly detect, identify, and recover from unanticipated control system failures.
Advanced, next-generation firewalls may have the capability for deep packet inspection of messages, but they operate in-line and only look for malware based upon signatures and other indicators. Also, their goal is to block malware, not advise users of anomalous messages. These solutions are valuable for use at facility perimeters, but rarely used within control systems to monitor internal messages. The risks of disrupting control system timing or blocking critical control messages far outweigh the potential benefits of detecting malware that may originate within control system devices.
The features of anomalous network message detection solutions for industrial facilities are distinctly different than those of next-generation firewalls. These solutions are specifically designed for use within industrial control systems. While they can detect malicious software that evades perimeter firewalls, they more importantly monitor all messages that flow between internal control system devices and alert on any anomalous behavior. They connect passively to control networks through span or mirror ports and collect information without active device pinging. These industrial solutions are also different than solutions that look for anomalous messages in conventional IT systems. In fact, they are mostly built from the ground up with industrial control system threats and requirements in mind.
While industrial anomalous message detection solutions share a common focus, they vary in features and capabilities. To maximize the industrial cyber resilience benefits from an investment in this kind of technology, users should look for a solution that provides the following kinds of support:
- Ability to parse the specific industrial control system protocols used within the organization’s facilities
- Automatic development of control system cyber asset inventories and network maps
- Automatic learning of baselines of “normal” communication patterns and message content
- Libraries of known ICS-cyber threats and anomalous system activities and behaviors for a responsive, reliable detection with a low rate of false positives
- Context-rich alerts that enable people to quickly identify the source of problems and appropriate remediation actions
- A user-friendly dashboard with visual analysis of network flows and commands, both real-time and historical, and capability for users to track and monitor communications
- Ability to proactively search the network for emerging threats and to prevent the spread of existing ones (threat hunting).
- Capability to continuously record and store network traffic and support efficient analysis of this data in threat hunting and problem analysis
- Ability to specify custom controls and company policies regarding device interactions, user actions, etc.
The financial, safety, and compliance risks of disruptions in industrial operations are too large to ignore. While control systems are designed to be reliable, unanticipated failures will still occur. Given today’s challenging cyber environment the likelihood of such events is increasing. Every industrial organization needs to ensure that they are doing all they can to minimize the potential impact of these kinds of events. Technologies and practices that facilitate rapid detection and repair are essential to ensure that systems are resilient to problems.
ARC research shows that anomalous message detection solutions that support industrial cyber resilience are already helping many companies mitigate their risks of operational disruptions. The benefits that these solutions provide generally outweigh the cost, especially when all affected stakeholders are considered. Implementing such a solution should be on the radar of every industrial organization.
Meeting the Need for Industrial Cybersecurity Expertise
By Eric C. Cosman (originally published Nov. 9, 2017)
Much has been written on the current imbalance between supply and demand for skilled and experienced cybersecurity professionals. While many related reports often focus on general-purpose IT cybersecurity, the situation is even more acute for industrial cybersecurity. Proficiency in that field requires more than expertise in IT security and network design. It also requires detailed knowledge and experience with various aspects of process design and control strategy development.
Public and private sector initiatives to better understand the specific needs and make plans for expanding the available pool of cybersecurity expertise have produced competency models and similar tools, leading to more detailed curricula and certification programs. Although these tools are necessary, skills development cannot be achieved through education alone. Practical experience is required, particularly in industrial cybersecurity, where the most effective response requires a thorough understanding of potential consequences for the physical equipment and processes.
We Live in a Risky World
Several widely reported attacks were either directed at or had indirect impact on industrial automation systems. Direct attacks have ranged from the Stuxnet attack of several years ago to more recent attacks on steel mills and other facilities.
Directed attacks are not the only threat. Most industrial systems use common, commercial-off-the-shelf (COTS) computers, networks, and software. This puts them at risk for collateral damage from malicious software designed to exploit known or recently discovered vulnerabilities in a wide variety of COTS technology.
The number of vulnerabilities identified continues to increase as more researchers focus on industrial control systems and applications. According to SCADAhacker, “Data obtained from the former Open-Source Vulnerability Database shows that through the end of 2014, more than 85% of all ICS vulnerabilities have been disclosed since 2011…”
The increased number of vulnerabilities has in turn led to an increased number of disclosures of ICS-related security incidents.
Perhaps the most important component of risk is potential consequence, which can only be fully understood with knowledge and experience in the specific area or domain in question. Just examining and assessing the computer and network elements of the automation system is not sufficient to understand these consequences. It is also essential to have a detailed understanding of the process and equipment under control. Process engineers use hazard identification methods to study consequences such as physical damage, release of hazardous material, or other business or safety risks to be able to avoid or mitigate these when designing the process.
How Asset Owners Are Responding
Asset owners are responding to changing risk in a variety of ways. These range from evaluating new tools and solutions to developing sophisticated cybersecurity management processes. Typically, the imperative originates in governance and management functions and is then channeled through the IT security function. However, the response must take the form of a comprehensive program that involves IT, OT, and other stakeholders. Regardless of the specifics, it is generally accepted that these programs can only be successful over the long term if they address each of the three essential elements of people, process, and technology. The people and process elements are commonly combined in proposed organizational changes.
Technology is often the first element addressed. There is typically a desire to achieve quick results by applying new products and solutions, but no amount of technology will fully address the problem. On the contrary, complex or incompatible products and technology can complicate the problem.
People and Processes
Addressing the process element almost always involves a critical review of the organizations responsible for managing cybersecurity and how they work together. Those responsible for IT infrastructure security often challenge engineering and operations to demonstrate that their systems have been adequately protected. This can exacerbate the friction that may already exist between the IT and operations functions in a company, often due to a lack of understanding of their respective drivers and constraints. Much more must be done to identify common concerns, imperatives, and objectives, which are essential prerequisites for effective partnerships.
People and Expertise Are Key
The development of appropriate skills and experience is a key element in addressing this challenge. These skills must reflect the content of available standards and practices that have been developed for both general information cybersecurity and industrial cybersecurity.
There have been many discussions and debates about the best approach for achieving such expertise. Is it more effective for security experts to develop knowledge of and appreciation for the nature of the manufacturing operations environment, or should operations engineers strive to develop expertise in security? While an argument could be made for either approach, the unfortunate truth is that neither is guaranteed to work in all situations.
Success in an ICS cybersecurity role is determined less by previous background than by the individual acknowledging the gaps in their skills and experience and willingness to learn. There are several examples of recognized experts who have come from both the IT and OT worlds.
Understanding the Process Under Control
To fully understand and appreciate what is required to secure an industrial control system and associated networks, it is first necessary to understand the physical process and system under control, as well as the logic developed to automate it.
The process and equipment may be described in documents with names like “process description” or “process overview.” Although these may take the form of narrative descriptions, it is more common for them to include some combination of diagrams and tables giving design conditions. The logic used to control the equipment may be available in a variety of forms, ranging from narrative documents to logic diagrams or even computer source code. Such documents may have names such as “control system design” or “automation strategy.” The information in these documents helps in the design of more resilient networks and segmentation of the controllers. It also defines what normal network traffic should look like.
Even with access to such documents it may not be possible to fully understand the physical process without assistance from a production or control engineer or operations staff responsible for its operation. It is quite common – especially with older facilities – for the above documents to be out of date, or simply not available.
Although gaining the necessary understanding of the production processes may take considerable time and effort, it is critical for developing an effective cybersecurity response.
While defining and developing roles with the necessary expertise can be challenging, some helpful tools are available.
A competency model describes the knowledge, skills, and abilities a person needs to perform well in a particular occupation. The US National Institute of Standards and Technology (NIST) provides a general framework that can be used to develop competency models for use in specific areas.
The Automation Federation worked with industry experts and representatives from the US Department of Labor to develop a competency model for automation that serves as a pathway for building the next generation of automation and engineering professionals. The content of the model is based on a variety of sources, including the Guide to the Automation Body of Knowledge from ISA, licensing requirements for professional engineers, and various professional certificates.
While the automation competency model contains some elements that pertain to this subject, there is also a separate competency model devoted entirely to cybersecurity. The Automation Federation has also contributed enhancements to this model to help differentiate between the needs for operations and business systems.
Unfortunately, competency models are insufficient in themselves to completely define the roles and expertise required.
NIST NICE Framework
Since the practice of industrial cybersecurity crosses the disciplines of security, engineering, and operations, it is necessary to define a common set of concepts and terminology that bridges these disciplines.
The National Initiative for Cybersecurity Education (NICE) at NIST has produced a Framework that serves as a reference resource for describing and sharing information about cybersecurity work and the knowledge, skills, and abilities needed to complete tasks that can strengthen an organization’s cybersecurity posture.
This framework provides a taxonomy and common lexicon that addresses all cybersecurity work and workers, irrespective of where or for whom the work is performed. The intent of the framework is to allow employers to use focused, consistent language in professional development programs when using industry certifications and academic credential and selecting relevant training opportunities for their workforce.
Much of its content is in the form of a series of appendices that describe the following elements:
- categories of common cybersecurity functions
- specialty areas of cybersecurity work
- roles, comprised of specific knowledge, skills, and abilities required to perform tasks in a work role
- knowledge, skills, and abilities required to perform tasks, generally demonstrated through relevant experience or performance-based education and training
- tasks or specific work activities that could be assigned to a professional working in one of the work roles.
Having defined the competencies and responsibilities required to address industrial cybersecurity, organizations need an effective way to assess the level of achievement of individuals. This is commonly addressed using exams and associated certifications. Industrial cybersecurity certifications are available from several sources and typically complement those used for information security.
Certifications do not obviate the need for individual interviews and skills assessment, but they do provide a valuable starting point and establish a common baseline.
Success in industrial cybersecurity requires a combination of aptitude, skills, and experience in both information security and industrial applications. Asset owners and others who seek to develop or acquire this expertise should employ available tools in the context of a comprehensive cybersecurity program that describes the organizational models and roles required to meet well-defined requirements.
Based on ARC research and analysis, we recommend the following actions for asset owners and others wishing to develop industrial cybersecurity expertise:
- Establish the "Shared Vision" – Take the time to clearly describe the characteristics of a future state that can be turned into goals that are shared across organizations.
- Risk Assessment – Understand the nature of the physical process, associated equipment and the supporting network. Perform a risk assessment to define, identify, and classify the security vulnerabilities in your industrial control system.
- People and Processes before Organization – In planning for the cybersecurity response, common practice is to focus on organizational details. A better approach is to identify the processes and procedures required, and the skills necessary to execute them. These skills are then described in the form of roles.
- RACI Analysis – Conduct a detailed analysis of the specific roles and individuals that must be responsible, accountable, consulted or informed (RACI) with respect to the various tasks required.
- Role and Skill Definition – Consider the use of formal competency models as tools in the development of careers in automation and cybersecurity.
- Share Case Studies – If you have examples of successful responses to the challenges in obtaining or developing the necessary expertise, consider sharing them in the form of case studies. The ARC Industry Forum provides an excellent opportunity for industry participants to both present and learn from these.
New Versions of NIST Framework Addresses Risk Management
By Larry O’Brien (originally published July 19, 2018)
Managing risk and adopting a risk-based approach to cybersecurity are increasingly necessary in the age of convergence. We’re already seeing a proliferation of risk-based services and approaches to cyber insurance, engineering, and design throughout the industrial and critical infrastructure segments. While many companies have their own methodologies for assessing risk, very few seem to focus specifically on manufacturing, infrastructure, or smart cities. So, how do we use risk assessments to craft cybersecurity policy for the operational technology (OT) domain?
The National Institute of Standards and Technology (NIST) has received considerable recognition over the past few years for developing the Cybersecurity Framework (CSF), which is now widely used as the basis for establishing effective security management systems. NIST recently released version 1.1 of its Framework for Improving Critical Infrastructure Cybersecurity. While this falls short of being a fully constructed risk management model for cybersecurity, the new framework does contain much expanded guidance on the element of risk in cybersecurity.
Version 1.1 of NIST Cybersecurity Framework
The US Commerce Department’s National Institute of Standards and Technology (NIST) recently released version 1.1 of its Framework for Improving Critical Infrastructure Cybersecurity, widely known as the Cybersecurity Framework. US Secretary of Commerce, Wilbur Ross, made an appeal to C-level management at all companies in the US to use the framework as the first line in their overall cyber-defense strategy. The framework was originally developed to address industries deemed vital to US national and economic security, including energy, banking, communications and the defense industrial base. It has since proven flexible enough to be adopted voluntarily by large and small companies and organizations across all industry sectors, as well as by federal, state, and local governments.
Increased Focus on Risk Management
Risk management is at the forefront of this new release. NIST hinted at this at the recent ARC Industry Forum in Orlando, where NIST Project Manager of Cybersecurity for Smart Manufacturing Systems, Keith Stouffer, hinted at methods, metrics, and tools to enable manufacturers to assess the cyber risk to their systems quantitatively. ARC is already seeing increased use of risk-based approaches to cybersecurity that borrow heavily from the HAZOP and risk matrix concepts in process safety. This NIST Framework stops short of being an actual model for cybersecurity risk management, but other available resources do that well, including the IEC 62443-3-2 cybersecurity standard.
Version 1.1 of the NIST Framework includes updates on authentication and identity, self-assessing cybersecurity risk, managing cybersecurity within the supply chain, and vulnerability disclosure. NIST based the changes to the framework on feedback collected through public calls for comments, questions received by team members, and workshops held in 2016 and 2017. Two drafts of Version 1.1 were circulated for public comment to help NIST address stakeholder inputs comprehensively. A new section 4.0, called Self-Assessing Cybersecurity Risk, explains how the framework can be used by organizations to understand and assess their cybersecurity risk, including the use of measurements.
An expanded Section 3.3, Communicating Cybersecurity Requirements with Stakeholders, helps users better understand Cyber Supply Chain Risk Management (SCRM). A new Section 3.4, Buying Decisions, highlights use of the framework to understand risk associated with commercial off-the-shelf products and services. Additional SCRM criteria were added to the implementation tiers. Finally, a Supply Chain Risk Management Category, including multiple subcategories, has been added to the framework core.
Later in 2018, NIST plans to release an updated companion document, the Roadmap for Improving Critical Infrastructure Cybersecurity, which describes key areas of development, alignment, and collaboration.
NIST provides a lot of good resources beyond the framework document. A NIST guide to managing risk in information security, for example, outlines some of the fundamentals and NIST’s overall approach to risk management. NIST’s Guide For Conducting Risk Assessments provides guidance on various types of risk models and approaches to doing risk assessments. However, none of these documents provide a risk model that encompasses both IT and OT for critical infrastructure or manufacturing industries.
Managing risk and adopting a risk-based approach to cybersecurity is increasingly necessary in the age of convergence. There is already a proliferation of risk-based services and risk-based approaches to cyber insurance, engineering, and design through the industrial and critical infrastructure segments. While many companies have their own methodologies for assessing risk, very few seem to focus specifically on manufacturing, infrastructure, or smart cities.
A wide range of companies and organizations are converging on this space. These include engineering service providers, process safety lifecycle management suppliers, cyber insurance companies, consulting companies, Big Data and analytics software suppliers, standards bodies, and government regulators. No single standard or model exists for measuring risk as it relates to cybersecurity in industrial organizations. Cyber insurance companies are either developing their own methodologies or using partners that have their own models, scoring systems, and evaluation capabilities.
ARC will continue to monitor and analyze the many developments in the cybersecurity risk-related space.
Industrial IoT Cybersecurity Trends and Developments
By Sid Snitkin (originally published August 1, 2019)
It’s reasonable for companies to be concerned about the security of IoT device deployments in industrial environments. Each device has an associated risk to data and operational integrity. A compromised internet-connected device could create a pathway for attacks on connected systems, including critical control systems.
While essential, secure-by-design IoT devices are not enough to manage all the risks. Threats to device security exist throughout the complex industrial IoT device supply chains. This includes compromised software from technology developers, poor security practices by OEMs, weaknesses in distributor security practices, poor security enablement by implementers, and insecure connections of devices to IoT platforms and cloud systems. Maintaining security throughout a device’s lifetime requires careful coordination with external suppliers and support groups. Companies need to understand these challenges and develop appropriate risk management strategies before widespread deployment of industrial IoT devices.
ARC Advisory Group held a workshop on this critical issue at the 2019 ARC Industry Forum in Orlando, Florida. Experts in various aspects of industrial IoT cybersecurity shared their insights and addressed attendee questions.
The Industrial IoT Cybersecurity Challenge
Demand for information about industrial operations has never been higher. Companies want data to understand and improve costs and productivity. Unlimited cloud storage, high-speed connectivity, and powerful analysis tools have created an explosion in improvement projects. Leaders of these initiatives want new sensors installed to monitor people, processes, and products and increased access to plant data stores.
Industrial product suppliers recognize the growing demand for information. Connectivity and ease of data sharing have become mandatory requirements for new applications and controllers. Networks are being upgraded with powerful new edge devices that support local data collection, analysis, and control. Smart assets, like robots and autonomous vehicles, are being designed for remote monitoring and support. All system elements are becoming industrial IoT devices that must be protected.
While the benefits are clear, the explosion in connected devices under-mines the effectiveness of conventional industrial cybersecurity strategies. Perimeter defenses and isolation techniques are ineffective when every device can communicate with external services. Constraining open connectivity may be prudent, but it frustrates business leaders who need to reduce costs and address competitive challenges. Companies need strategies (other than avoidance) to deal with the cyber risks of industrial IoT devices.
Secure-by-Design IoT Devices
IoT device technologies and capabilities vary widely. Some are simple sensors, while others are complex control systems. Security concerns differ, but users expect reliable and predictable performance from all IoT devices. This requires confidence in the initial and ongoing integrity of the device’s hardware, software, and critical information it may contain.
Ensuring that devices are secure-by-design is essential. Designers need to follow secure development lifecycle practices (SLDC) and devices need capabilities that protect against local tampering and compromising communications with external systems. Organizations like ISASecure and UL offer device certification services that help industrial IoT device suppliers demonstrate design security. Unfortunately, most devices still don’t have such certifications.
Secure Industrial IoT Supply Chains
Industrial IoT cybersecurity strategy needs to cover more than secure-by-design devices. SDLC practices and designs alone can’t guarantee that a specific unit is free of malware, security features properly enabled during installation, certificates have been protected, connections with external systems are secure, and security is sustained during operation.
Industrial IoT supply chains are complex, with innumerable opportunities for security compromises. Enforcing security across every participant and handoff is often impractical, but managers can be made aware of the risks they are accepting before they deploy devices in an industrial environment. This can increase appreciation of security team concerns and garner support for basic cybersecurity controls like vendor qualification and pre-deployment testing of industrial IoT devices.
Supply chain participants are well aware of the importance of addressing end user security concerns. Without appropriate security, suppliers of industrial IoT devices and manufacturers of smart industrial equipment lose revenues and opportunities to sell new services, like remote support. Many are supporting industry efforts, like the Industrial Internet Consortium (IIC) and The Open Group, to develop common security frameworks. The US government shares this concern and supports resolution through NIST programs and funding of special projects, like FACT, to reduce the risks of compromised software throughout the supply chain.
Efforts to Address Industrial IoT Challenges
A diverse group of five industrial IoT experts participated in the Industrial IoT Cybersecurity workshop at this year’s ARC Industry Forum in Orlando. They shared their observations on the state of industrial IoT cybersecurity and various efforts to address this challenging issue.
Security of IoT Devices
Marcellus Bucheit, President and CEO of Wibu-Systems is a member of the IIC trustworthiness task group. This group explores aspects of trustworthiness relevant to the IIoT and IIC's vision of an IIoT ecosystem.
Mr. Bucheit explained why trustworthiness should be the goal for users of industrial IoT devices. Trustworthiness is the degree of confidence one has that a device will perform as expected with respect to safety, security, privacy, reliability and resilience while facing environmental disruptions, human errors, system faults and attacks. In his experiences, IT and OT groups have restricted efforts to addressing only subsets of these requirements. Industrial IoT expands their perspectives and drives a shared understanding of the need for complete trustworthiness.
Security for Open Process Automation Systems
Camilo Gomez is Global Cybersecurity Strategist at Yokogawa and a member of the Open Process Automation Forum (a forum of The Open Group), which developed the standards-based, open, secure, interoperable process control architecture O-PAS. This platform reflects the growing demand for control systems with openness, connectivity, and upgradability.
Mr. Gomez is co-chair of the security working group charged with developing requirements for security across the O-PAS Framework. Goals of openness, interoperability, and agnosticism mean that systems require security controls similar to industrial IoT devices.
Use of standards, a fundamental tenet of O-PAS, is reflected in how the security framework leverages various parts of the ISA/IEC 62443 standards set. This provides Design, Build, and Maintain security guidelines for technology providers and IoT device OEMs. The framework’s standards for secure connectivity have also been validated against the capabilities of OPC-UA and Redfish.
Managing the Risks of Compromised Software
Eric Byres, CEO of aDolus, is well known in the ICS cybersecurity community for developing the Tofino firewall. He focused his comments on his current efforts to address the problems occurring in IoT software and firmware supply chains. Beyond malicious code and counterfeit software, companies don’t know what subcomponents are hidden in their products. Likewise, they have no easy way to determine if firmware upgrades are valid, free of vulnerabilities, and current.
Mr. Byres noted that information to solve this problem is available, but technicians need a quick way to access and use it to assess the trustworthiness of update files. Using a grant from the US Department of Homeland Security (DHS), aDolus created FACT, a community platform that aggregates information and provides a measure of trustworthiness similar to a FICO credit score. This platform supports the needs of vendors, asset owners, integrators, consultants, and security partners. He encouraged attendees to download complimentary versions of the solution to test its effectiveness and engage their organizations in the effort.
Richard Kerslake is Intel’s IoT Program Director. Improving the efficiency and security of IoT device provisioning is a key focus for the company. Intel’s research shows that the costs and complexity of manual cybersecurity provisioning constrain IoT deployments and result in security gaps. A collaborative effort with ARM has produced a solution to this problem that Intel terms Secure Device Onboard.
Mr. Kerslake likened the new device onboarding process to familiar plug-and-play methods used to connect a new printer to a personal PC. According to Mr. Kerslake, device onboarding to any cloud platform normally takes less than one minute, without exposing critical device security information. Intel’s goal is to share this intellectual property with standards bodies and open source groups in 2019.
Jon Taylor, Senior Manager and Principal Consultant, OT/IIoT Services at Revolutionary Security, has fifteen years of experience in security of smart remote systems spanning various companies like Caterpillar. His roles have included communications engineer, telematics engineer, CISO, product line manager and consultant.
Mr. Taylor’s experience gives him a unique perspective on the cybersecurity challenges across the industrial IoT supply chain. While he believes that individual products are improving, the smart systems industry is still slow in advancing security. Core to this problem is where they are focusing educational efforts. Engineers are trained to achieve product feature requirements, not security. Today they are being asked to enable connectivity with no real training in what this entails. His advice to smart asset manufacturers is to recognize this serious gap and solve the problems at the source, rather than waiting for flaws to show up in finished designs. It’s less costly and will help companies avoid embarrassing situations and product recalls.
Industrial IoT cybersecurity is a serious concern. Delays in rolling out digital transformation programs hurt everyone involved in the industrial IoT supply chain. Companies proceeding without appropriate security programs may be planting the seeds for costly cyber incidents.
While the challenges are large, panelist comments in the 2019 ARC Forum industrial IoT cybersecurity workshop show that there are things that companies can do to reduce the risks. Based on this, we recommend the following actions for end users and suppliers of industrial IoT devices:
- Recognize all IoT device risks – Make a list of all the steps involved in your IoT device supply chain and develop security requirements for each product/service. Educate procurement and operations groups about the importance of requiring compliance from all parties.
- Leverage standards to ensure IoT lifecycle security – Learn about available standards and include compliance to relevant ones in all product and service procurement specifications.
- Support industrial IoT cybersecurity initiatives – Encourage suppliers to learn about and participate in the various security efforts discussed in this report. This will simplify user efforts to manage security.
- Address deployment and update risks – Establish procedures to ensure that all devices are properly configured and provisioned during installation. Ensure that safe, secure procedures and pathways are provided to sustain the security of all industrial IoT devices and systems.
- Train key personnel in security – Ensure that all design engineers understand security requirements and embrace security as an important design goal.
IT/OT Cybersecurity Convergence – Part 1
Many industrial companies are considering converging their IT and OT cybersecurity programs to address security gaps, optimize use of limited cybersecurity resources, and enable secure deployment of digital transformation programs. At the same time, they recognize that convergence can be challenging given the differences in culture, goals, and environments.
ARC Advisory Group devoted two sessions to this important issue at the 2019 ARC Industry Forum in Orlando, Florida. Part 1 of this report discusses the findings of the workshop conducted on the first day of the Forum. Part 2 will discuss the presentations given on the second day of the event by three CISOs who lead converged programs.
The IT/OT Cybersecurity Convergence workshop began with a brief ARC presentation about the goals, challenges, and strategies underlying many convergence programs. This provided a framework for the subsequent panel discussions. Panelists included three IT leaders and three OT leaders. All are actively involved in successful convergence efforts. They discussed the challenges encountered and the methods used to build integrated IT/OT cybersecurity programs.
IT/OT Cybersecurity Convergence Challenges and Strategies
IT/OT cybersecurity convergence has the potential to solve many industrial cybersecurity challenges. Shared responsibility for the security of IT/OT interfaces can help companies eliminate malware propagation across systems. Cross-trained, collaborative teams can fill critical expertise gaps and improve incident response efforts. Common processes and metrics can increase visibility of risks and help companies focus efforts and investments on the most critical issues.
To reap these benefits, organizations need to make changes in every aspect of their cybersecurity strategies.
An integrated technology strategy is essential to ensure full visibility of vulnerabilities and threats. This also amplifies the effectiveness of resources and minimizes licensing, training, and support costs. While conditions may require different solutions for IT and OT, effort must be made to ensure that these tools are compatible and fully integrated.
From a process perspective, organizations need to rationalize cybersecurity activities and ensure that every aspect of the company’s operations has the same level of security and visibility. Companies cannot afford to have any weak links in their security chain. Key areas to consider include managing vulnerabilities and detecting anomalous events.
Integrating cybersecurity teams is probably the biggest challenge for companies considering IT/OT cybersecurity convergence. Significant differences in IT and OT cultures must be overcome to drive effective collaboration and cross-domain support. Establishing the right organizational structure is essential.
The most common strategy seems to be creating a single cybersecurity organization charged with three major objectives:
- Shared, end-to-end responsibility for securing all business processes
- Global corporate governance of all cybersecurity policies, procedures, technology, guidelines, etc.
- Continuous management of all cyber-assets, vulnerabilities, and threats regardless of where they appear
This may be implemented through formal organizational changes or through virtual teams of people who work in all related areas like IT operations, OT operations, and security operations centers (SOCs). Most organizational plans also include third parties with specific expertise. These are often used for peripheral tasks like program audits, system assessments, and pen testing.
The CISO, CIO, or someone else in top management is generally given responsibility for overall coordination and reporting to the board of directors.
End User Concerns and Suggestions
Six end users participated as workshop panelists. These included Dawn Cappelli, CISO at Rockwell Automation; Mandy Huth, VP Cybersecurity at Kohler Corporation; Tammy Klotz, Information Security Director for Versum Materials; Rob Cox, Manager Operations Technology at Georgia-Pacific; Chris Da Costa, Operations Technology–Cyber Security Manager for Air Products and Chemicals; and, Jim LaBonty, Director, Global Technology & Engineering for Pfizer.
This mix of IT and OT expertise was intentional to help ensure a good mix of perspectives. While their comments reflected these different backgrounds, it was heartening to see similar views regarding convergence benefits and challenges. Recommendations for overcoming the roadblocks also indicated a shared view of cybersecurity as a separate discipline. Ideally, these professionals can support cyber-assets in any environment once they are trained in the unique needs and constraints. Some key observations panelists noted during the discussions follow.
There was general consensus that people represent the biggest challenge in IT/OT cybersecurity convergence. Building trust between IT and OT personnel is essential and should be fostered through collaborative involvement of both groups to develop common metrics, standards, policies, and processes. Common terminology, shared understanding of risks, and recognition of individual strengths facilitate effective teamwork and help focus everyone’s effort on the issues that represent the most risk to the entire organization.
Gaining support from plant operations and engineering is equally important. This requires successful interactions that demonstrate respect for plant performance goals. Plant engineers are often happy to offload responsibility for cybersecurity tasks like patching, but only if they are confident that cybersecurity professionals will not create problems for normal system operation.
All panelists expressed the importance of developing and retaining cybersecurity resources. The general shortage of cybersecurity experts frustrates efforts to hire people and all face the challenges of aging workforces. They stressed the importance of giving employees rewarding assignments and establishing clear career paths for people who join the cybersecurity team. Framing cybersecurity as a separate profession with cross-training opportunities helps in this regard.
All the panelists indicated that they used third parties to support certain parts of their programs. This included security assessments, penetration tests, audits, workshops, and table-top exercises. One company outsources most of its security; only three out of 60 people involved with the program are full-time employees.
The panelists highlighted the importance of establishing common cybersecurity metrics, policies, and processes. These people are tasked with developing and managing global IT/OT cybersecurity strategies. Common metrics and processes provide a means to assess and improve security across all of a company’s systems.
Most of the panelists built programs around the NIST Cybersecurity Framework. The comprehensiveness of this framework and its general acceptance ensured that all issues were addressed. This helped identify areas where common practices and technologies could be deployed and recognize when unique approaches were justified. The framework also provided metrics that were helpful in creating the critical measures needed to monitor and manage security across many different facilities.
While technology was not a focus of this workshop, panelists noted that responsibilities, processes, and policies must anticipate the introduction of new technologies like the Industrial Internet of Things (IIoT). All are dealing with the security impact of digital transformation programs as well as the rapid changes occurring in IT and OT technologies. Some panelists also noted that they have added cybersecurity as a category in their existing change management processes to help ensure that the security team is kept abreast of every plant change request.
Most of the panelists indicated that their convergence programs were still a work-in-progress, but already generating benefits. This included improved teamwork and expertise sharing, better recognition of key security risks, closer coordination between digital transformation and security teams, and broader acceptance of cybersecurity investment decisions.
Panelist comments in the 2019 ARC Forum IT/OT cybersecurity convergence workshop reflect the findings of other ARC research on this topic. IT/OT cybersecurity convergence offers significant potential benefits, but building trust between IT and OT is the biggest challenge. Collaborative development of common policies, processes, and metrics is essential to build effective teams.
Based on this and other ARC research, we recommend the following actions for end users:
- Begin with a Good Framework – Tools like the broad-based, generally-accepted NIST Cybersecurity Framework facilitate communication, understanding, and acceptance across groups with different perspectives.
- Conduct Collaborative Workshops – Engaging IT and OT people in all discussions regarding the program’s people, processes, and technology strategy facilitates team building and development of trust across people with different backgrounds.
- Develop a Common Set of Metrics – Choosing metrics applicable to both IT and OT cybersecurity enables visibility of security issues across the organization. Common acceptance of these metrics also helps everyone understand and accept decisions regarding the company’s deployment of limited human and financial resources.
- Embrace Continuous Improvement – Cybersecurity is a journey, not a goal. Sustaining and improving programs is essential to keep ahead of attackers. Learning from the experiences of peers at events like the ARC Industry Forums is the best way to stay abreast of emerging threats and best practices for avoiding incidents.
IT/OT Cybersecurity Convergence – Part 2
By Sid Snitkin (originally published May 23, 2019)
Many industrial companies are considering converging their IT and OT cybersecurity programs to address security gaps, optimize use of limited cybersecurity resources, and enable secure deployment of digital transformation programs. At the same time, they recognize that convergence can be challenging given the differences in culture, goals, and environments.
ARC Advisory Group devoted two sessions to this important issue at the 2019 ARC Industry Forum in Orlando, Florida. In this, the second part of this two-part report, we discuss the presentations given on the second day of the Forum by three CISOs who lead convergence programs.
These three presentations illustrate the universality of industrial cybersecurity challenges and the general applicability of IT/OT cybersecurity convergence. Whether a company produces specialty chemicals, electrical equipment, or bathroom fixtures it must deal with diverse security program requirements, resource constraints, and cultural differences between IT, OT, and operations groups. The way these companies are overcoming these hurdles can provide guidance for others that want to build effective integrated IT/OT cybersecurity programs.
The Evolution of Rockwell Automation’s IT/OT Cybersecurity Strategy
Dawn Cappelli, VP Global Security and CISO of Rockwell Automation, is responsible for protecting Rockwell Automation and its ecosystem of customers, suppliers, distributors, and partners from the ever-changing global threat landscape. Her team, Global and Information Security (G&IS), serves as thought leaders and a center of excellence for global, integrated cyber and physical security. The team develops and executes a series of strategic security frameworks in partnership with Information Technology, the Product Security Office, and a network of business and function liaisons from across the company. Her presentation described the company’s journey from various disparate IT and OT cybersecurity programs to its “Connected Enterprise Security Strategy.”
Ms. Cappelli’s journey began in 2016. The prior CISO had already established an extended security team with business and functional liaisons and a partnership with IT. They also had three foundational programs in place that included secure development environment, insider risk program, and third-party risk program. The G&IS team built on this foundation to establish risk-based strategies for information and manufacturing security using the NIST Cybersecurity Framework as the basis for both programs. This enabled common terminology and metrics for managing security across the organization. Collaborative workshops were held with plant managers and OT cybersecurity teams to develop a manufacturing security strategy based on the framework’s principles.
While these strategies supported IT/OT convergence, major cyber incidents in 2017–2018, like NotPetya, Wannacry, and Trisis made it clear that this was not enough. These attacks did not involve Rockwell Automation products, but they exploited automation company systems and services as attack vectors. Protecting customers from such attacks is vitally important to Rockwell Automation and led to the development of the company’s expanded Connected Enterprise Security Strategy that spans all of aspects of its business.
While the company still manages internal security with the NIST Cybersecurity Framework, it recognized the need for different standards for products and interactions with customers and third parties. The additional standards include ISA/IEC 62443 and ISO 27001.
Ms. Cappelli closed her presentation by highlighting the importance of involving operations, engineering, and business leaders when evaluating cyber risks and prioritizing security investments. Top management can’t ignore recommendations that are mutually endorsed across the company. She also offered advice to others involved in IT/OT convergence: A holistic strategy is essential for securing your IT/OT ecosystem; a standards/risk-based approach is best; and, cross-functional teams need to build the strategy together.
Spinning Securely at Versum Materials
Tammy Klotz, Information Security Director for Versum Materials, is responsible for the company’s entire cybersecurity program including Information Risk Management, Plant Cyber Security, and IT Security, Risk & Compliance. Versum Materials, which produces special materials for the electronics and semiconductor industries, is a spinoff of Air Products & Chemicals. Conditions of the spinoff meant that Ms. Klotz’s team only had twelve months to set up a complete IT/OT cybersecurity program or be forced into a costly extension of its transition services agreement with Air Products & Chemicals. Her presentation highlighted the key factors underlying their success.
Build Strong IT/OT Partnerships
C-Level Executive Sponsorship is critical. At Versum the program was co-sponsored by the CIO and COO. This ensured shared goals and objectives, shared responsibility for outcomes, and constant awareness of the criticality of plant operations and the business impact of an outage. This partnership philosophy was also reflected in the project team which included resources from corporate IT, corporate OT, and site process control engineering.
Extensive face time between corporate and plant personnel reinforced partnership perspectives. As building trust was crucial, IT people made visits to every facility in the corporation. Listening and learning was encouraged to ensure that everyone had a good understanding of each product, process, and system.
Know What You Have and Want
A key step in the transition was to ensure that every cyber system was identified and equipped with appropriate security measures. A physical inventory was made at each site and a standard tool kit used to drive development of specific, detailed migration task lists. Decisions were based on design principles and standards defined by the team. This included common OT cybersecurity recommendations like segregation of OT networks from corporate networks; least privilege access controls at a site and process level; no unnecessary cross-zone, cross-site communications; and no access to the internet from OT networks.
Conduct ICS Risk Assessment
Engineering, in cooperation with Security and Safety organizations performed risk assessments at every site. Risk elements included: product hazard; location and community; lost revenue; PC operating systems and hardware; and, control system software and hardware. This resulted in product and location risk scores.
Enable Informed Decision-making and Prioritization
Risk assessment results were used to prioritize remediation activities. Prioritization was essential since everyone accepted that “We can’t do it all.” Independent risk assessments allowed business leaders to determine the level of risk they were willing to accept. This also helped ensure that new capital projects included the cyber-resiliency requirements.
Make Security Synonymous with Safety
Like at most other companies, safety is the #1 priority at Versum Materials and the team worked hard to raise security to the same level. The team focused on the fact that safety and security controls need to work together to mitigate risk and minimize the business impact of any incident. Security has now become a part of the corporate safety program and is recognized in training curriculum.
Keep It Real
Sustaining the security program is considered equally important. The company is addressing this through bi-weekly IT-OT-ICS team meetings that review current issues, new demands, and changes to applications. Versum also instituted a cyber management of change program that forces consideration of security in all changes to ICS systems. These requests may be initiated by IT, OT or plant personnel, but approval is required by all three parties. Annual refresh of risk assessments, access control privileges, ongoing communications with plant operators and site visits are other parts of their strategy to maintain relationships.
Establishing IT/OT Cybersecurity at Kohler Corporation
Mandy Huth, Vice-President Cybersecurity at Kohler Corporation, has responsibility for ensuring the security of the company’s Hospitality, Power, and Kitchen & Bath business units. Protecting these drastically different businesses demands a broad-based cybersecurity strategy. Hospitality requires reliable IT systems and secure customer data to ensure the success of the company’s global portfolio of golf and resort destinations. Power needs assurance that the OT systems in its global manufacturing plants are safe from cyber incidents. Kitchen & Bath must be confident that customers can trust the reliability and privacy of the company’s new, smart products. As Ms. Huth is new to the company, her presentation focused on what she is doing to build a cybersecurity program that meets these diverse needs.
Not surprisingly, Kohler’s program must address all facets of the cybersecurity C-I-A (confidentiality-integrity-availability) triad. Confidentiality is essential to protect the IP developed through 150 years of operation and ensure new smart home product designs preserve consumer privacy. Availability is key to success in the company’s competitive product markets and data integrity in transit is important for credit card transactions and for tracking devices used throughout the company’s complex supply chains.
Kohler has several internal challenges that must be overcome to implement an effective IT/OT cybersecurity program. The idea of secure-by-design principles must be embedded in everything it does. Cross-functional training and support from other company leaders, like the engineering VP, is needed to build a team that can support cyber assets in all environments. Ways to share costs must be established that are acceptable across this cost-conscious organization.
Use of a consistent framework across all assets is essential to ensure they focus on the key issues. They are using a variety of recognized standards and guidelines as the basis for this, including the NIST Cybersecurity Framework, CIS 20 Critical Controls, Risk Heat Maps, etc. Approval for all decisions and actions require people to answer three key questions: What is the problem? What is the solution? And what are the benefits?
Ms. Huth believes that people are the most important ingredient of a successful cybersecurity strategy. The company’s use of business solution managers reflects this. These act as brokers between the security team and functional departments. Next in importance is a centralized configuration management database supported with physical tagging and labeling of all cyber assets. Security, analysis, and reporting technology can then be rationally and efficiently selected, deployed, and sustained. The company measures the effectiveness of these efforts based on how well they reduce costs, increase efficiency, eliminate redundancy, and improve its visibility into and understanding of cyber risks.
She closed her presentation with the following recommendations: Use a standard framework and risk alignment methodology; be realistic in your expectations – change takes time; engage everyone in secure-by-design principles to minimize future problems; identify a broker for business and IT; and, drive shared responsibility for costs, risks, and implementation.
The three speakers in this ARC Industry Forum session faced significantly different challenges, yet, their strategies and approaches reflect a common set of key principles:
- Building effective, cross-trained teams is the key success factor and biggest challenge for IT/OT cybersecurity convergence programs
- Use of accepted frameworks eases acceptance across technology domains and facilitates shared views of risks and priorities
- It’s essential to recognize cybersecurity as a journey that demands ongoing effort to sustain trusting relationships across the organization
ARC believes that these principles can help every industrial company build an effective integrated IT/OT cybersecurity program.
Table of Contents
- Executive Overview
- Overcoming Cyber-apathy
- Fundamental Concepts for Industrial Cybersecurity
- ARC’s Industrial Cybersecurity Maturity Model Evolves
- Make Industrial Cyber Resilience Your Goal
- Meeting the Need for Industrial Cybersecurity Expertise
- New Version of NIST Framework Addresses Risk Management
- Industrial IoT Cybersecurity Trends and Developments
- IT/OT Cybersecurity Convergence – Part 1
- IT/OT Cybersecurity Convergence – Part 2
- Standards Address Need for Secure-by-Design ICS Products
- Experts Debate Use of Cybersecurity Profiles
- New Malware Targets Process Safety Systems: Now What?
- Who Is Accountable for Safety Systems’ Cybersecurity?
- Smart Cities Have a Cybersecurity Problem
- Building Good Cybersecurity Programs for Smart Cities and Infrastructure
- ISA Forms ISAGCA to Promote ISA 62443 Cybersecurity Standard
- Cybersecurity Across Sectors: More Unites Us Than Separates Us
- Appendix 1: Government Guidance on Cybersecurity
ARC Advisory Group clients can view the complete report at ARC Client Portal
If you would like to buy this report or obtain information about how to become a client, please Contact Us