By Folker Schmidt and Daniel Jedecke
Months after the appearance of the critical vulnerability in Citrix Application Delivery Controller (ADC) and NetScaler Gateway (CVE-2019-19781, also known as „Shitrix“), more and more cases are now becoming known where the vulnerability was exploited very early on, but was not used for extortion until much later, and ongoing.
Our incident responders found that in a critical period in early 2020, backdoors were installed in some cases, which are actively exploited now, 8 months later, for ransomware attacks. In many cases, the attackers have not even bothered to adapt the known proofs-of-concept of the „Project Zero India“ group. In the cases known to us, the user „pwnpzi1337“ was created, which was used in the original PoC.
The subsequent steps of the attack are basically the same as they were before, network discovery, lateral movement within the network (for example using Dridex or Cobalt-Strike), data exfiltration, and later encryption of the data e.g. via DoppelPaymer.
However, there is a multitude of other conceivable malware variants on the market. Due to the current wave of attacks, every administrator and system administrator is strongly recommended to check their Citrix NetScaler systems in particular for possible new users or conspicuous network activity. A check of the folder /var/vpn/bookmark is definitely recommended, since the XML files smuggled in by the attacker can be found there. As a quick check for possible compromise, the instructions under http://deyda.net/index.php/en/2020/01/15/checklist-for-citrix-adc-cve-2019-19781 have proved to be helpful.
If there is any doubt about the integrity of the system, our forensic experts recommend rebuilding the system, which is the standard procedure for ransomware attacks. We are happy to refer to the recommendations of the German BSI (Federal Cyber Security Agency). Back in January 2020, the BSI issued a Citrix vulnerability warning (see CSW No. 2020-172597-1531, Version 1.5, 30.01.2020 and https://www.bsi.bund.de/DE/Service-Navi/Presse/Pressemitteilungen/Presse2020/Citrix_Schwachstelle_160120.html), which contains, among other helpful bits, the following measures:
- Disconnection from network of the compromised Citrix instance
- Backup of the old Citrix instance (entire system, or at least the log files under /var/log/*)
- Restart of Citrix instance with the latest build of the respective version branch and implementing the workaround (workaround recommendation when no patches were available yet)
- Creation of new SSL/TLS certificates, recall of old SSL/TLS certificates
- Resetting of all Windows Active Directory passwords if the Citrix user group cannot be restricted
- Depending on the network connection, check the Windows domain for further compromises
- If wildcard SSL certificates (*.example.com) were used on a compromised Citrix system, all other systems using the wildcard certificate must be taken into account in the certificate exchange mentioned above.
Citrix administrators should always subscribe to Citrix security bulletins with notices of new firmware versions to receive information about new firmware updates
The German Alliance for Cyber Security has recommended further action. These can be found at https://www.allianz-fuer-cybersicherheit.de/SharedDocs/Downloads/Webs/ACS/DE/Ransomware/Ransomware_Massnahmenkatalog.html.
It is also important to note in this case that the attackers could potentially move around the network unnoticed for months, which could jeopardize the integrity of backups that have been made long ago. Special care must be taken here when restoring affected systems.
At the beginning of August 2020, still around 200 vulnerable Citrix systems in Germany were accessible from the Internet (https://twitter.com/certbund/status/1291017699351580675). The number of systems that are secured but have already implemented a backdoor cannot be quantified.
