Tuesday, November 9, 2021

How to Avoid an Attack like Industroyer

How to Avoid an Attack like Industroyer (Also known as CrashOverride)


Industroyer, or "CrashOverride" as it is alternatively called, was an attempt to cause widespread and lasting power outages in Ukraine.  It was initiated on December 17, 2016, roughly 1 year after a more successful cyber attack on December 23, 2015.  It may have been a follow-on effort utilizing some of the reconnaissance gained in the earlier event.  Industroyer was a ‘failure’ as far as the desired results of the attackers were concerned, causing loss to only about 20% of the city of Kiev for an hour, but may have been an incremental step in building an attack framework. 


The most striking feature of this attack was the depth of knowledge of the underlying OT protocols and the attempt to assemble a modular, scalable attack platform. The level of sophistication required to auto-discover vulnerable endpoint devices and specifically target them utilizing known OT system functions moves beyond the ‘rock breaker’ mindset into an area of finesse that will represent the reality of future attacks.


This blog intends to address some ‘common sense’ steps that can be implemented to prevent an attack like Industroyer.  We will look at the structure of the attack from a sequence of events standpoint as well as by utilizing the ATT&CK for ICS framework.

The attack was designed to accomplish three goals:  shut down energy flow, damage physical energy infrastructure, and prevent rapid recovery.  The initial stages of the attack involved common techniques to access credentials, use Living Off The Land (LOTL) tools to learn the network, determine access points to the ICS, steal privileged credentials to gain access to the ICS network. 


Why did this not cause the widespread outages and damage intended?  Subsequent analysis of logs and data show that there were some minor errors in the endianness of messages and some message format errors.  Probably this was a validation test in preparation for future use.  Considering the sophistication of this attack, the writer expects that these errors will be corrected before their next use.  This portion of the process is shown diagrammatically below.





[T0865 Spearfishing Attachment] and [T0890 Exploitation for Privilege Escalation] In the Enterprise network, there are many tools and practices that can limit exposure and this document is not a treatise on the IT methods to achieve least privilege access, but these efforts are important and typically are ongoing. Thinking in a broader sense, however, with the progression from compromise in the Enterprise network to ultimately compromise of OT and core energy management functions, a ‘birds-eye view’ of anomalies across the IT/OT boundary would be a benefit. This kind of visibility is a profound weakness in many industries today.


[T0886 Remote Services] While it may be inconvenient, there is a very simple method to prevent remote access to the ICS network - disconnect it.  RDP, VNC, ssh, or other remote access methods all rely on up-to-date patching of servers in a DMZ, limited access to information, sometimes NATting, and access privileges.  Because of the complex sequence of steps required to remotely access ICS networks and the relative infrequency of use, privileged users often store instructions in text files or spreadsheets on their machines to facilitate making the connections when required.  This information is of course very valuable to an attacker and is available once elevated privilege access is achieved on the enterprise network.  The simplest and most effective security method is to install a basic on/off switch in line with the jump box in the DMZ.  When a remote user needs to connect, he contacts an authorized individual (possibly an operator) in the facility and requests the switch to be turned on for a specific time period.  When the individual is done with remote work, they contact the authorized person to open the switch.  These time periods should be logged to create a permanent record with the name of the person closing the switch and the name of the person requesting remote access.  This simple method provides a record of and accountability for any changes during this period.  Because of the coordination required, it limits capricious remote access for non-critical reasons.  This simple control would have completely broken this attack string.

If this method is not chosen, an agentless solution like Microsoft Defender for IoT could monitor network switches in the DMZ and can provide alerts on remote access.  While this is effective, it may not be monitored 24/7 and so remote access can go on for significant periods unsupervised.  If the network is monitored only on the OT side of the DMZ firewall, it may not be possible to identify who initiated the remote access session.  This adds overhead to the evaluation of each remote access session consuming the analyst's limited time.


[T0842 Network Sniffing] This is difficult to detect without an agent on the sniffing machine since it is a passive activity.  However, this is usually accomplished by modifying or replacing a program that is monitoring a specific port of interest. Capturing all traffic on a port requires it to be changed to promiscuous mode.   This would be readily discoverable using agent-based tools since such changes would be recorded in the system logs. 

Network sniffing usually requires the capability to move files to the target machine.  Microsoft Defender for IoT can identify SMB, FTP, or other file transfer protocols and alert on this kind of traffic.


[T0840 Network Connection Enumeration] This tactic uses LOTL methods and is very difficult to detect even with agents.  Since netstat is a valid system command, its use can be part of normal business and system firmware provides basic network data normally.


[T0884 Connection Proxy] Once a proxy is established, is easily detectable by a Network Traffic Analysis solution like Microsoft Defender for IoT.  Several indicators will point to this tactic.  First, a new port/protocol will appear in use.  Second, devices will attempt to connect to Command & Control (C2) servers with internet addresses.  Both of these are ‘noisy’ events. An attacker understands that the timing of these kinds of activities is important.  This adversary was well aware of normal work hours and made these kinds of changes during off-hours. 

Once this connection was established, the Windows Notepad application was trojanized with embedded code so that it would reconnect to the C2 server if the connection was broken.  Any application could have served this purpose, but Notepad, due to its frequent use was chosen by the attackers.

A simple solution to this tactic would be to disallow any internet connectivity to OT equipment.  Once again, this introduces some inconvenience.  Offsite vendor support will be more difficult and downloading updates will require a transfer from an IT-managed machine to the OT network. This transfer, however, provides an opportunity to scan all incoming files for malware. These obstacles can be challenging but are not insurmountable.  Consider that this simple strategy would have stopped this attack dead in its tracks.


[T0869 Standard Application Layer Protocol] The next part of this process is where the situation rapidly deteriorated.  The attackers ran a shell script to execute the main components of the malware.  The structure of the shell script looked like this:


Each environment variable identified a filename or path to the specific files used.  The files were replacements of seldom used Windows components or executables. The CONFIGURATION.ini file detailed the specific parameters, timing values, etc. associated with each protocol.  See the continuation of the diagram below:




How could these changes have been prevented?  The simplest and most effective way is to use whitelisting.  Antivirus might have picked up some of this activity if the file signatures happened to match some malware variants, but properly configured whitelisting (based on file hashes) would prevent these kinds of file substitutions and new, unapproved executables would not be allowed to run. Whitelisting entails some significant work to initially configure and needs to be disabled for system upgrades, but in OT environments where patching is infrequent, it can be a very strong defense.

The modules were installed on the target machines using, in this case, the HTTP POST method.  This method would not be in common use in most ICS installations.  Custom alerts could be configured in Microsoft Defender for IoT if this method is in use.  Specific alerts like this are useful, but they must cover every conceivable method to achieve the attacker’s goal. The effort to achieve this is major and may not be complete.

Finally, environment variables should be quite static in an OT environment.  System logs would identify changes to these.  While this would have identified something happening, it would have been possible for the attacker to accomplish these changes without using environment variables.  As a result, this may have been an indicator but not a stoplog.


[T0861 Point & Tag Identification] The OPC portion of the attack attempted enumeration of I/O to determine which were breakers.  This may have occurred concurrently with the exercise of the IEC 101, 104, and 61850 devices (which were apparently already known).  In any case, this service enumerates the available OPC servers for available devices matching breaker attributes.  Since this is not a normal activity, Microsoft Defender for IoT could have alerted on this activity.  Also, [T0815 Denial of View] would have declared many points in “out of limits” state.  This also would be highly abnormal and could be alerted.  However, both of these alerts, if they were concurrent with the brute force (below) could not have been responded to in time.


[T0806 Brute Force I/O] In the final phase of this attack, once the scripted actions began, it is doubtful whether any evasive actions could be achieved by human actions.  Operations research shows that in infrequent and unfamiliar situations, humans take minutes to assess and begin to respond to events.  Certainly, the majority of the damage would have been done by this point. 

It is at this point that automated response systems could be effective.  In real life electrical transmission and distribution operations, breaker status changes are very infrequent and multiple operator-initiated changes are almost never commanded simultaneously.  If the actual external connections were monitored by firewalls and rules were instituted in a Network Traffic Analysis device, sudden and simultaneous commands to multiple breakers could trigger a command block in the firewall.  This could have had the effect of slowing the progression of events such that human response was possible.


[T0837 Loss of Protection] In this case, a specially crafted UDP packet sent to port 50000 can put the protective relay in the ‘firmware update’ mode.  When in this mode, the relay is inert, waiting on new firmware.  The intent behind this was to operate breakers without normal protective functions active thereby causing physical distribution system damage.  Protection against this vulnerability involves installing vendor-approved patches.  As an alternative prior to patching, firewalls can be installed with filtering to block UDP port 50000.  Additionally, alerts could be configured in Microsoft Defender for IoT to identify the specific port and payload.


[T0826 Loss of Availability] The final step in this program was to render operator workstations and engineering tools unavailable.  This was accomplished by a program called haslo.dat which cleared all Windows Registry keys, overwrote portions of all generic Windows files, and overwrote ICS-specific configuration files. Again, whitelisting would have prevented execution of this kind of program and this appears to be the best overall solution to this issue.

Below the progression of the attack is mapped onto the ATT&CK for ICS Matrix.




In summary, four intentional changes to the OT environment could have completely eviscerated this campaign. 

  1. A remote-access disconnect switch with associated procedures
  2. Blocking internet access completely from the OT network
  3. Whitelisting applications on the OT computers
  4. Thoughtful configuration of Microsoft Defender for IoT monitoring the OT Network and DMZ


Posted at https://sl.advdat.com/3qqWp1M