How to improve operational resilience in a multi-vendor and multi-site world

Introduction to OT/ICS Security

Reviewing the OT (Operational Technology) cybersecurity threat landscape from 2021 to 2022, we observe that since the outbreak of the COVID-19 crisis, the world's reliance on networks and information systems has surged to unprecedented levels, with industries and services becoming increasingly interconnected. The COVID-19 crisis has demonstrated the necessity for the digital transformation of key global industries to be adequately prepared, particularly in enhancing the cyber resilience of critical infrastructure services (such as energy, transportation, chemicals, and critical manufacturing). The increased scope and variety of cyberattacks in recent years highlight the world's need for a higher level of cyber resilience to protect crucial industries. We summarize the threat landscape into the following five points:

Emerging threat

In 2022, numerous RaaS with full ecosystems (such as Black Basta, Pandora, LockBit 3.0) emerged, adopting various extortion tactics like destroying data, holding data for ransom, selling data on the dark web, threatening customers or suppliers, and targeting industries like Smart Manufacturing, Energy, Food & Agriculture, and Healthcare & Public Health. Some ransomware used advanced techniques to prevent analysis, for example, requiring a pass parameter to parse the main program (such as Egregor, LockBit 3.0). This made it difficult for researchers to analyze and deepen the attack's impact on organizations. They even employed fast encryption methods and hardening tactics to evade detection and prevent attacks. Moreover, hackers exploited vulnerabilities (like Log4j) and used legitimate Windows/Microsoft Defender tools to download malicious DLL files and encrypted Cobalt Strike payloads.

Additionally, supply chain threats were another focal point. In 2022, cyberattacks on suppliers in key industries were recorded, and we found that the Energy and Critical Manufacturing industries were among the most affected. Critical manufacturing accounted for 24% of the total. The direct impact of supply chain attacks caused business operation interruptions. For instance, in the first half of 2022, Toyota had to halt production at 14 auto factories due to cyberattacks on its suppliers of plastic parts and electronic components. Shell suffered a loss in oil production because of a cyberattack on its logistics and storage supplier.

 

Geopolitical tensions

According to a report by the European Union Agency for Cybersecurity (ENISA), the Russia-Ukraine conflict has led to a surge in radical hacking activities, with 128 government agencies in 42 supporting countries being targeted by state-sponsored hacker groups. Countries such as China, Iran, and North Korea have also increased their espionage efforts, with national hacker organizations setting their sights on nations like Southeast Asia, Japan, and Australia. As geopolitical tensions continue to escalate in Asia, these hacker groups have targeted countries with close ties to Taiwan, including EU member states like the Czech Republic and Poland. These attacks often exploit zero-day vulnerabilities or target OT networks, with a primary focus on critical infrastructure. Social engineering, disinformation, and data threats are also common attack methods used by national hackers.

 

Collateral Damage

In terms of attack surfaces, most OT attacks originate from IT incidents, which we refer to as collateral damage. This leads to operational impacts and data being held hostage, ultimately resulting in financial losses. The inherent vulnerabilities of OT cybersecurity, both internal and external, stem from factors such as supply chain attacks from new assets or insufficient protection within the environment. For instance, ransomware attacks on organizations like Colonial Pipeline and JBS Foods have drawn attention to the dangers IT attacks pose to OT systems. Though these attacks may not initially target OT systems, a compromise on IT systems can force OT teams to manually shut down operations for security reasons, leading to indirect impact on OT. According to our survey, 94% of the enterprises interviewed acknowledged the likelihood of IT security incidents impacting the OT environment. With ransomware being capable of lateral movement, relying solely on either OT or IT systems is insufficient. These incidents underscore the urgency of incorporating IT-OT fusion defenses in security strategies.

 

Incomplete cybersecurity architecture

In the past, the most common defense for OT/ICS was relying on "complete isolation" (Air-gapped). As a result, the network architecture of OT/ICS was designed with little consideration for cybersecurity defense capabilities. This assumption leads to inadequate planning and deployment of cybersecurity countermeasures, such as the absence of security considerations for regional management in the OT/ICS network architecture or even a lack of detailed level segregation. Another concern is internal/supply chain threats. Mobile devices in air-gapped environments, along with lenient management policies, can potentially allow malicious programs to compromise OT/ICS environments or steal sensitive data. For instance, USB flash drives used for data transfer, laptops used for maintenance, and any equipment brought in by suppliers can all serve as perfect carriers for malware propagation.

 

Additionally, legacy operating systems are common security vulnerabilities. Typically, OT/ICS endpoints are the weakest links in OT/ICS cybersecurity, as many older OT/ICS endpoints perform critical operations or function as decision points in production lines. Key assets running on outdated systems do not receive software and firmware updates, leaving newly discovered vulnerabilities unpatched. For example, every Windows XP or Windows 7 system is an easy target for attacks.

 

Shortage of cybersecurity professionals

The next challenge lies in the difficulty of hiring qualified cybersecurity professionals from a limited talent pool. In the IT sector, talent shortage is already a significant issue, and it is even worse in the OT sector. Given the recent trend of digital transformation, coupled with the convergence of IT and OT, cybersecurity is not only a highly complex field but also constantly evolving due to leaps in technological innovation, creating a perpetually changing landscape. For instance, Germany's manufacturing industry is grappling with the impact of cybersecurity talent shortages. According to our annual survey, 37% of German manufacturers face this issue. This indicates that establishing a dedicated OT/ICS cybersecurity training program or course within the company is a wise choice to better prepare for potential future security risks.

 

Understanding the potential impact of cyber attacks on OT systems

In 2021, many industries fell victim to ransomware attacks, with the most notable case being the Colonial Pipeline ransomware attack. This incident became the largest scale cyber assault against oil infrastructure in US history, leading to the shutdown of fuel transportation pipelines. As a result, many airlines experienced jet fuel shortages, and numerous locations faced gas station fuel scarcity and sTXOne Networksrocketing prices, causing a frantic rush to purchase gasoline among the public.

However, in 2022, industries faced a more diverse range of threat attacks, with software attacks, supply chain assaults, and strikes targeting critical infrastructure coming into focus. Coupled with geopolitical issues introducing more state-sponsored APT attacks, some regions and governments are intensifying efforts to implement cybersecurity regulations for critical infrastructure.

 

1.      Future Risks: IT/OT Convergence and Industry 4.0 Associated Risks

The concept of IT/OT convergence aims to integrate physical (OT) equipment and devices into the digital (IT) realm. Although this idea has existed for many years, it didn't truly take center stage in the industry until after 2020. According to a recent report by IoT Analytics [1], starting in 2020, approximately 50% of industrial assets in factories were connected to some form of local or remote data collection system. The COVID-19 pandemic, in particular, highlighted how the Industrial Internet of Things (IIoT) could enhance organizational resilience even in the face of catastrophic events.

In manufacturing, for example, the Industrial Internet of Things, also known as Industry 4.0, is considered key to significantly reducing downtime, enabling new business models, and providing better customer experiences. With the rise of new IIoT architectures, traditional centralized SCADA and MES system communication methods have started to change. For instance, many sensors now employ IoT communication protocols like LoRaWAN, SigFox, or NB-IoT, connecting industrial sensors directly to the cloud. Moreover, industrial computer manufacturers have started to develop edge servers supporting software application platforms that link devices to the cloud, such as Advantech's ADAM-3600 RTU, which supports Azure cloud connectivity. Some small and medium-sized enterprise factories even prefer using open-source devices and communication protocols, like Linux-based HMI and gateways, or gradually adopting OPC-UA protocol-supporting servers.

However, these trends may increase the attack surface for hackers. For example, according to ICS-CERT data from 2010 to 2021 [3], the number of reported vulnerabilities has been increasing annually since the launch of the ICS-CERT reporting program, with a cumulative total of 4,436 vulnerabilities. Furthermore, the number of reported vulnerabilities in 2021 marked the highest annual increase ever, highlighting the growing arsenal of hidden weapons hackers can utilize in ICS environments.

 

2.      OT threat vectors may be different from IT

In the realm of IT, the most common attacks involve social engineering or internal personnel oversights, which grant hackers network access privileges. Hackers then exploit these opportunities for privilege escalation and lateral movement, remaining undetected until they locate critical OT control systems. For example, DoppelPaymer infiltrated Foxconn's Mexico factory using spear-phishing emails containing malicious links, or by employing attachments disguised as legitimate files to deceive unsuspecting victims into executing malicious code. This code, in turn, downloads even more potent malware (such as Emotet) onto the victim's computer.

However, the most dangerous scenario in OT occurs when hackers penetrate a device supplier's network and infect devices with malicious code before they are delivered to customers. Often, OT attacks transpire when hackers infiltrate the software development process of suppliers before the software is compiled, turning the supplier's products into malware-laden software. In these cases, the story of cybersecurity takes a more sinister turn, as unsuspecting organizations unwittingly introduce infected devices into their critical systems, leaving them vulnerable to devastating consequences.

 

3.      OT security incidents come from new assets

New assets are where the majority of OT security incidents stem from, as they contain vulnerabilities, and they can be carrying malicious files by default. In the US, this is an issue for 54% of the sample size, whereas in Japan this happens 44% of the time. Germany is the outlier here, with 51% of their OT security incidents stemming from IT activities instead of from new assets. These challenges result in both financial losses and badly compromised productivity.

 

How to improve operational resilience in a multi-vendor and multi-site world

 

Implementing Zero Trust Cybersecurity Defense in OT through Asset Lifecycle Management

OT managers need to adopt approaches distinct from IT cybersecurity and adhere to the zero trust principle: never trust, always verify. Embracing a zero trust architecture for OT ensures that network defenses never assume trust by default, and continuously assess trustworthiness across the network. By employing automated methods, the zero trust framework can be realized in OT, spanning applications, device controls, and networks to prioritize productivity. It is recommended that OT managers utilize an asset lifecycle management approach to deploy and implement a zero trust cybersecurity framework, which encompasses four critical stages of the asset lifecycle: device arrival, configuration, production, and maintenance:

 

1. Onboarding stage:

Before assets are transported to your factory facility, suppliers should scan each asset and establish an OT/ICS health record, proving that the equipment is free of malicious software. This process is akin to international flight customs checks, where both parties involved in the transaction must independently verify the security of the equipment. Upon arrival at the factory facility, each device must be considered "hostile" until it undergoes a threat scan and any potential exploitable vulnerabilities are documented, similar to customs inspections during international flight arrivals, ensuring that the equipment does not contain malicious software or severe vulnerabilities.

 

TXOne Networks recommends using Portable Inspector solution, which allows for endpoint security checks without the need for software installation. Through the use of portable scanning tools, automated scanning and system configuration checks can be performed without a network connection. The cybersecurity inspection tool can be used to ensure supply chain security before devices enter the facility:

1. Minimize the impact of antivirus software on machinery or avoid violating machine warranty terms.
2. Suitable for network-free environments, allowing offline virus and configuration checks even in air-gapped environments.
3. Quickly verify whether personnel and supplier electronic devices carry malicious software, then execute cleanup or isolation.
4. Record asset information collected during each scan and send it to a central management console for viewing and archiving.
5. In addition to scanning for malicious software during data transfers, AES-256 hardware encryption is employed to protect files, ensuring data integrity during transfers.

 

2. Staging stage:

The Staging stage is the process of hardening assets to eliminate avenues of attack, which includes addressing cybersecurity vulnerabilities and shutting down non-essential services, such as applications, user privileges, user accounts, network ports, and other unneeded system functions. By hardening assets, technicians can minimize the chances of attackers accessing computers responsible for critical tasks and prevent the execution of malicious software. However, traditional antivirus software is not designed for industrial control environments. It requires constant internet connectivity for updates to its scanning engine and virus signatures. Moreover, file scanning demands significant computational and memory resources, often leading to excessive endpoint load and frequent false positives.

 

TXOne Networks suggested deploying the following measures for OT endpoint protection:

1. Application Trust List Technology: For older machines, built-in whitelist mechanisms can be used to protect endpoints from malicious software infections or unauthorized changes. A four-in-one lockdown mechanism can be implemented, including USB lockdown, data lockdown, operation lockdown, and configuration lockdown.
2. Next-Generation Antivirus Technology: For modern machines, a dual-engine approach can be adopted. The Advanced Threat Scan Engine technology scans for known attacks that can be addressed. In addition, when offline, a next-generation machine learning engine detects unknown threats, achieving defense against both known and unknown malicious software attacks. With a built-in ICS software database capable of identifying software from over a hundred industrial control vendors, threat scans avoid interfering with recognized ICS software, ensuring zero disruption and minimal false positives for the lightest-weight operation.
3. Centralized Management Platform: A client-server architecture central control platform can manage endpoint protection software. Under the same central control, different main functions can be formulated, such as management rules, reporting mechanisms, user management, and scan settings.

 

3. Production stage:

In the production phase, any issue that arises can immediately lead to economic losses. At this stage, network security becomes a new variable, requiring meticulous protection to ensure operational integrity and resilience. This means balancing the needs of both modern and legacy assets. Factory owners must be prepared to defend against a variety of cyber threats that hackers are eager to exploit through the network, ensuring network resilience. Zero trust networking can be employed using network segmentation, optimized network access control, and enhanced intrusion detection and analysis to prevent or mitigate the impact of compromised assets from escalating into large-scale disasters. Simultaneously, it simplifies monitoring and makes it more difficult for hackers to gather information or move within the OT network.

 

TXOne Networks suggested deploying the following measures for OT network defense:

1. Network Isolation Techniques: Network isolation can be further divided into Internal Segmentation and Micro-Segmentation. Internal Segmentation is suitable for large-scale scopes or regions, defined based on available technology, bandwidth, and communication protocols in use. Similarly, Micro-Segmentation refers to technical solutions that allow users to narrow down the protected scope or region to a smaller scale, even down to a single asset, achieving OT network visibility, ICS protocol filtering of devices on the network, without altering the existing network architecture or disrupting current configurations.
2. Virtual Patching Technology: This can be implemented through Host-based Intrusion Prevention Systems (IPS) or network IPS. Such devices feature specially designed network policies for packet filtering, specifically designed to defend against attacks exploiting known vulnerabilities, without forcing endpoints to undergo system updates. This means no need to restart the system or halt the production line.
3. Network Trust Whitelist: Supporting a variety of industrial control network communication protocols and in-depth analysis of L2-L7 network traffic, it allows for communication protocol instruction editing and endpoint connection allowlist operations, creating network rule whitelists. Moreover, all hardware protection devices can be visually managed through a central control platform, paired with network segmentation, to reduce risk based on the principle of least privilege.

 

4. Maintenance stage:

Maintenance encompasses not only the repair of hardware components but also software configuration changes, system upgrades, and security updates. Typically, factories carry out equipment maintenance operations through scheduled periodic routines. Currently, technicians need to synchronize assets with state-of-the-art cybersecurity protection to ensure that replaced hardware components, such as computing and storage components, are free from potential malware. Additionally, software changes must comply with the asset owner's security configuration rules and minimize new software vulnerabilities. Consequently, during maintenance, production equipment managers inevitably need to perform repeated malware and vulnerability scans. It is recommended to conduct additional malware scans when replacing a component of the equipment or when making software or configuration changes. Furthermore, suppose a device in the production facility requires a software update. In that case, vulnerability scans should be performed on the network-facing components of the production equipment to confirm compliance with the asset owner's security configuration rules.

Since security inspection tasks are usually located on-site at factories, the environment is typically offline. Therefore, maintenance personnel must execute vulnerability scans in a non-disruptive, software-free, and network-independent manner, ensuring that the scanned original equipment software and configurations remain entirely undisturbed while verifying the cleanliness of the equipment.

TXOne Networks suggested deploying the following measures for OT maintenance protection:

1. Utilizing Portable Inspector tool can verify the cleanliness of components and software within production equipment, providing centralized logs on a single management platform for archiving and reference. By swiftly scanning and recording results, production equipment can be returned to operation.
2. Inspection of all equipment brought into the semiconductor factory by visitors, as well as the common offline asset checks within the production environment.
3. Integration into industrial control SOPs for routine network security on external maintenance personnel's computers.

5. Situational Awareness:

In order to achieve robust OT cybersecurity, a solid understanding of operations is essential. Consequently, factory security teams require a clear and visible platform to manage the information security of numerous devices in real-time, enabling administrators to promptly detect and address attacks as they occur. Additionally, maintaining situational awareness of all assets, software configuration changes, system upgrades, and security updates is crucial.

By continuously monitoring routine schedules and incorporating TXOne's AI, the complexity of each asset is learned, establishing protection baselines for factory units and deploying appropriate factory defense methods. This ensures smooth operation and ongoing operational continuity within the workplace. TXOne centralizes OT/ICS device-related cybersecurity logs into a single window for comprehensive situational awareness, or archives asset configuration information for managerial analysis and reference. This includes:

1. StellarOne allows management from a single pane of glass with support for Syslog forwarding, indicators of compromise (IoC) integration, and centralized monitoring
2. EdgeOne gives comprehensive visibility of the connected, air-gapped, and standalone OT environment using Edge series products. EdgeOne is a centralized management platform for multiple sites for each deployed Edge series product.
3. ElementOne collected asset information can be to the CSV format through the centralized management program as an asset inventory or sent to a SIEM or Rsyslog server for further asset management such as maintaining OT asset inventory or identifying impact levels, known vulnerabilities, and cyber risks.

Incorporating AI/ML technologies into the OT zero trust framework allows for the study of network packets and processes, facilitating the recognition of potential threats and anomalies. This innovative approach enhances the ability to proactively defend against cyberattacks, as well as adapt and respond to evolving threat landscapes. By embracing AI-driven zero trust architectures, organizations can significantly bolster their cybersecurity posture and better protect their critical OT environments.

 

How to Maximize OT/ICS cybersecurity with Valmet and TXOne

Valmet and TXOne Networks are joining forces to better provide the protection needed by their joint industrial automation clients in today's vulnerable OT (Operational Technology) environment. Combining TXOne Networks’ security inspection, endpoint protection, and network defense solutions with the leading Valmet DNA Distributed Control System (DCS), Valmet IQ Quality Control System (QCS), machine vision, analyzers and measurements, valves, and Industrial Internet of Things offers various advantages. The collaboration enhances industrial clients' abilities to protect, respond to, and recover from cybersecurity incidents using Valmet's cybersecurity services.

Leveraging TXOne Networks’ cutting-edge technology, Valmet's cybersecurity services add significant value by reducing cybersecurity risks and ensuring business continuity. From a range of OT security solutions, we identified TXOne Networks’ offerings to meet the demand for providing network and endpoint protection and services. Together, Valmet and TXOne Networks deliver highly secure solutions for their clients.

Applying a "zero trust cybersecurity" approach to OT network protection, clients' critical IT infrastructure and industrial requirements are safeguarded from the ever-present threat landscape. TXOne Networks’ "OT zero trust" method of protecting operational environments consists of three stages: segmenting networks, scanning inbound and mobile assets using portable rapid scanning devices, and protecting endpoints with defense solutions tailored to endpoint types (traditional or modernized).

Conclusion

Ensuring the cybersecurity of critical infrastructure and strategic manufacturing industries' OT/ICS is becoming increasingly vital, and industry leaders need to understand the necessary steps to protect their OT/ICS systems. TXOne Networks proposes an asset lifecycle protection approach to assist OT managers in meeting the diverse and unique defense requirements of OT/ICS workshops. The OT zero trust cybersecurity defense architecture simplifies cybersecurity compliance, ensuring highly resilient network and endpoint protection while minimizing the impact on operations. Meanwhile, Valmet's engineering team is dedicated to helping industry OT/ICS security leaders streamline the regulatory process for critical infrastructure. Through Valmet's cybersecurity services, they support industry leaders in ensuring the cybersecurity and resilience of their OT/ICS and strengthening secure supply chains, collectively contributing to the collaborative defense of the global digital ecosystem.

This article was originally published at TXOne Networks website.