Basehunter488

On April 25, 2020, Sophos published a knowledge basearticle (KBA) 135412 which warned about apre-authenticated SQL injection (SQLi) vulnerability, affecting the XG Firewallproduct line. According to Sophos this issue had been actively exploited atleast since April 22, 2020. Shortly after the knowledge base article, a detailed analysis of the so called Asnarök operationwas published. Whilst the KBA focused solely on the SQLi, this write up clearly indicatedthat the attackers had somehow extended this initial vector to achieve remote code execution (RCE).

Apr 28, 2020 Applies to the following Sophos products and versions Sophos Firewall What to do As a general best security practice to reduce attack surface wherever possible, Sophos recommends disabling HTTPS admin services on the WAN interface. If the User Portal is not being used, Sophos also recommends deactivating this service on the WAN as well. Sophos has contacted customers today to disclose this information to them and advise them of next steps. In addition to the hotfix, impacted customers need to take additional steps to fully remediate this issue. A full background of this incident can be found in KBA 135412.

The criticality of the vulnerability prompted us to immediately warn our clients of the issue.As usual we provided lists of exposed and affected systems. Of course we also started an investigation into the technical details of the vulnerability. Due to the nature of the affected devices and the prospect of RCE, this vulnerability sounded like a perfect candidate for a perimeter breach in upcoming red team assessments. However, as we will explain later, this vulnerability will most likely not be as useful for this task as we first assumed.

Our analysis not only resulted in a working RCEexploit for the disclosed vulnerability (CVE-2020-12271) but also led to the discovery ofanother SQLi, which could have been used to gain code execution (CVE-2020-15504). Thecriticality of this new vulnerability is similar to the one used in theAsnarök campaign: exploitable pre-authentication either via an exposeduser or admin portal. Sophos quickly reacted to our bug report, issuedhotfixes for the supported firmware versions and released new firmwareversions for v17.5 and v18.0 (see also the Sophos Community Advisory).

I am Groot

The lab environment setup will not be coveredin full detail since it is pretty straight forward to deploy a virtual XGfirewall. Appropriate firmware ISOs can be obtained from the official downloadportal. What is notable is the fact that the firmware allows administrators direct root shell access via the serial interface, theTelnetConsole.jsp in the web interface or the SSH server. Thus there wasno need to escape from any restricted shells or to evade otherprotection measures in order to start the analysis.

Sophos stops everything malicious and provides us with alerts, so we can respond quickly—and that’s worth its weight in gold.” Cliff Hogan, CIO, D4C Dental Brands Switching to Sophos Central was a simple transition and 80% of the work was carried out within just one week.”.

After getting familiar with the filesystem layout,exposed ports and running processes we suddenly noticed a message in the XGcontrol center informing us that a hotfix for the n-day vulnerability, we wereinvestigating, had automatically been applied.

We leveraged this behavior to create a file-systemsnapshot before and after the hotfix. Unfortunately diffing the web rootfolders in both snapshots (aiming for a quick win) resulted in only one changedfile with no direct indication of a fixed SQL operation.

Architecture

In order to understand the hotfix, it wasnecessary to delve deep into the underlying software architecture. As thepublished information indicated that the issue could be triggered via the webinterface we were especially interested in how incoming HTTP requestswere processed by the appliance.

Both web interfaces (user and admin) are basedon the same Java code served by a Jetty server behind an Apache server.

Most interface interactions (like a loginattempt) resulted in a HTTP POST request to the endpoint/webconsole/Controller. Such a request contained at least twoparameters: mode and json. The former specified a number which wasmapped internally to a function that should be invoked. The latter specified thearguments for this function call.

The corresponding Servlet checked if therequested function required authentication, performed some basic parametervalidation (code was dependent on the called function) and transmitted a messageto another component - CSC.

This message followed a custom format and wassent via either UDP or TCP to port 299 on the local machine (the firewall). Themessage contained a JSON object which was similar but not identical to thejson parameter provided in the initial HTTP request.

The CSC component (/usr/bin/csc) appeared to bewritten in C and consisted of multiple sub modules (similar to a busyboxbinary). To our understanding this binary is a service manager for the firewallas it contained, started and controlled several other jobs. We encountered asimilar architecture during our Fortinet research.

Sophos Kba 135412 Error

CSC parsed the incoming JSON object and calledthe requested function with the provided parameters. These functions however,were implemented in Perl and were invoked via the Perl C language interface.In order to do so, the binary loaded and decrypted an XOR encrypted file(cscconf.bin) which contained various config files and Perl packages.

Another essential part of the architecture werethe different PostgreSQL database instances which were used by the web interface,the CSC and the Perl logic, simultaneously.

Locating the Perl logic

As mentioned earlier, the Java componentforwarded a modified version of the JSON parameter (found in the HTTPrequest) to the CSC binary. Therefore we started by having a closer look atthis file. A disassembler helped us to detect the different sub modules whichwere distributed across several internal functions, but did not reveal any logicrelated to the login request. We did however find plenty of imports related tothe Perl C language interface. This led us to the assumption that the relevantlogic was stored in external Perl files, even though an intensive search on thefilesystem had not returned anything useful. It turned out, that the missing Perlcode and various configuration files were stored in the encrypted tar.gzfile (/_conf/cscconf.bin) which was decrypted and extracted during theinitialization of CSC. The reason why we previously could not locate the decrypted fileswas that these could only be found in a separate linux namespace.

As can be seen in the screenshot below thebinary created a mount point and called the unsharesyscall with the flag parameter set to 0x20000. This constant translatesto the CLONE_NEWNS flag, which disassociates the process from theinitial mount namespace.

For those unfamiliar with Linux namespaces: ingeneral each process is associated with a namespace and can only see, and thususe, the resources associated with that namespace. By detaching itself from theinitial namespace the binary ensures that all files created after theunshare syscall are not propagated to other processes. Namespacesare a feature of the Linux kernel and container solutions like docker heavily rely on them.

Therefore even within a root shell we were notable to access the extracted archive. Whilst multiple approaches exist toovercome this, the most appealing at that point was to simply patch thebinary. This way, it was possible to copy the extracted config to aworld-writable path. In hindsight, it would probably have been easier to just scpnsenter to the appliance.

From a handful of information to the N-Day(CVE-2020-12271)

The rolled out hotfix boiled down to themodification of one existing function (_send) and the introduction oftwo new functions (getPreAuthOperationList andaddEventAndEntityInPayload) in the file/usr/share/webconsole/WEB-INF/classes/cyberoam/corporate/CSCClient.class.

The function getPreAuthOperationListdefined all modes which can be called unauthenticated. The functionaddEventAndEntityInPayload checks if the mode specified in the requestis contained in the preAuthOperationsList and removes the Entityand Event keys from the JSON object if that is the case.

Analysis

Based on the hotfix we assumed that thevulnerability must reside within one of the functions specified in thegetPreAuthOperationList. However, after browsing through the relevantPerl code in order to find blocks that made use of the Entity orEvent key, we were pretty confident that this was not the case.

What we did notice though is that regardless ofwhich mode we specify, every request was processed by the apiInterfacefunction. Sophos denoted the functions mapped to the mode parameter internallyas opcodes.

The apiInterface function was also theplace where we finally found the SQLi vulnerability aka execution ofarbitrary SQL statements. As is depicted in the source excerpt below, thisopcode called the executeDeleteQuery function (line 27) which took a SQLstatement from the query parameter and ran it against the database.

Unfortunately, in order to reach the vulnerablecode, our payload needed to pass every preceding CALL statement whichenforced various conditions and properties on our JSON object.

The first call (validateRequestType)required that Entity was not set to securitypolicy and that therequest type was ORM after the call.

The preceding call(variableInitialization) initialized the Perl environment and shouldalways succeed. In order to keep our request simple and not to introduceadditional requirements, the Entity value in our payload should not be one ofthe following: securityprofile, mtadataprotectionpolicy,dataprotectionpolicy, firewallgroup, securitypolicy, formtemplate orauthprofile. This allowed us to skip the checks performed in the functionopcodePreProcess.

The checkUserPermission function doeswhat its name suggests. Whereas, the function body that can be seen below isonly executed if the JSON object passed to Perl included a __usernameparameter. This parameter was added by the Java component before the request wasforwarded to the CSC binary, if the HTTP request was associated with a validuser session. Since we used an unauthenticated mode in our payload, the__username parameter was not set and we could ignore the respectivecode.

To skip over the preMigration call wejust had to choose a mode which was unequal to 35 (cancel_firmware_upload),36 (multicast_sroutes_disable) or 1101 (unknown). On top of that allthree modes required authentication making them unusable for our purposes,anyway.

Depending on the request type, the functioncreateModeJSON employed a different logic to load the Perl moduleconnected to the specified entity. Whereas each POST request initially startedas ORM request, we needed to be careful that the request type was notchanged to something else. This was required to satisfy the last if statementbefore the vulnerable function was called inside the apiInterfacefunction. Therefore the condition on line 15 had to be not satisfied. Therespective code checked if the request type specified in the loaded Perl moduleequaled ORM. We leave the identification of such an Entity as an exerciseto the interested reader.

We skipped the call to themigrateToCurrVersion function since it was not important for our chain.The next call to createJson verified if the previously loaded Perlpackage could actually be initialized and would always work as long as it referred to anexisting Entity.

The function handleDeleteRequest onceagain verified that the request type was ORM. After removing duplicatekeys from our JSON, it ensured that our JSON payload contained a namekey. The code then looped through all values which were specified in ourname property and searched for foreign references in other databasetables in order to delete these. Since we did not want to delete any existingdata we simply set the name to a non-existing value.

We skipped the last two function calls toreplyIfErrorAtValidation and getOldObject because they were notrelevant to our chain and we had already walked through enough Perl code.

What did we learn so far?

• We need a mode which can be calledfrom an unauthenticated perspective.
• We should not use certain Entities.
• Ourrequest needed to be of type $REQUEST_TYPE{ORMREQUEST}.
• Therequest had to contain a name property which held some garbage value.
• The EventProperties of the loaded Entity, and in particular theDELETE property, had to set the ORM value to true.
• OurJSON object had to contain a query key which held the actual SQL statement wewanted to execute.

When we satisfied all of the above conditions wewere able to execute arbitrary SQL statements. There was only one caveat: we couldnot use any quotes in our SQL statements since the csc binary properlyescaped those (see the escapeRequest sub 0-day chapter for details). As a workaround we definedstrings with the help of the concat and chr SQL functions.

From SQLi to RCE

Once we had gained the ability to modify thedatabase to our needs, there were quite a few places where the SQLi could beexpanded into an RCE. This was the case because parameters contained within the database were passedto exec calls without sanitation in multiple instances. Here we will onlyfocus on the attack path which was, based on our understanding and the detailsreleased in Sophos' analysis, used during the Asnarök campaign.

According to the published information, theattackers injected their payloads in the hostname field of the SophosFirewall Manager (SFM) to achive code execution. SFM is a separateappliance to centrally manage multiple appliances. This raised the question:what happens in the back end if you enable the central administration?

To locate the database values related to theSFM functionality we dumped the database, enabled SFM in the front end,and created another dump. A diff of the dumps was then used to identify thechanged values. This approach revealed the modification of multiple databaserows. The attribute CCCAdminIP in the tabletblclientservices was the one used by the attackers to inject theirpayload. A simple grep for CCCAdminIP directed us to the functionget_SOA in the Perl code.

As can be seen on line 15, the code retrieves thevalue of the CCCAdminIP from the database and passes itunfiltered into the EXECSH call on line 22. Due to some kind of cronjobthe get_SOA opcode is executed regularly leading to the automaticexecution of our payload.

What made this particular attack chain veryunfortunate was the if condition on line 11, as it allowed us to reach the EXECSHcall only if the automatic installation of hotfixes is active (which is thedefault setting) and if the appliance is configured to use SFM for central management (which is not the default setting). This resulted in a situation in which the attackers most likelyonly gained code execution on devices with activated auto-updates - leading toa race condition between the hotfix installation and the moment of exploitation.

Installations that do not have automatic hotfixes enabled or have not moved to the latest supported maintenance releases could still be vulnerable.

From N to Zero (CVE-2020-15504)

Another promising approach for discovery of then-day, instead of starting at a patch diff, seemed to be an analysis of allback end functions (callable via the /webconsole/Controllerendpoint) which did not require authentication. The respective function numberscould, for example, be extracted from the Java functiongetPreAuthOperationList.

SQL-Injection countermeasures inside the Perllogic

Despite of the fact that the back end performedall its SQL operations without prepared-statements, those were not automaticallysusceptible to injection.

The reason for this was, that all functionparameters coming in via port 299 were automatically escaped via theescapeRequest function before being processed.

So everything is safe?

One function which caught our attention wasRELEASEQUARANTINEMAILFROMMAIL (NR 2531) as the corresponding logicsilently bypassed the automatic escaping. This happened because the functiontreated one of the user-controllable parameters as a Base64 string and used thisparameter, decoded, inside a SQL statement. As the global escaping took placebefore the function was actually called, it only ever saw the encoded string andthus missed any included special characters such as single quotes.

After the parameter was decoded, it was split intodifferent variables. This was done by parsing the string based on thekey=value syntax used in HTTP requests. We were concentrating on thehdnFilePath variable, as its value did not need to satisfy anycomplicated conditions and ended up in the SQL statement later on.

The only constraint for$requestData{hdnFilePath} was, that it did not contain the sequence../ (which was irrelevant for our purposes anyway). After crafting a releaseparameter in the appropriate format we were now able to trigger a SQLi inthe above SELECT statement. We had to be careful to notbreak the syntax by taking into account that the manipulated parameter wasinserted six times into the query.

Upgrading the boring Select statement

The ability to trigger a sleep enables anattacker to use well known blind SQLi techniques to read out arbitrary databasevalues. The underlying Postgres instance (iviewdb) differed fromthe one targeted in the n-day. As this database did not seem to store anyvalues useful for further attacks, another approach was chosen.

With the code-execution technique usedby Asnarök in mind, we aimed for the execution of an INSERT operation alongside a SELECT.In theory, this should be easily achievable by using stacked queries.After some experimentation, we were able to confirm that stacked queries weresupported by the deployed Postgres version and the used database API. Yet, itwas impossible to get it to work through the SQLi. After some frustration,we found out that the function iviewdb_query (/lib/libcscaid.so)called the escape_string (/usr/bin/csc) function before submitting thequery. As this function escaped all semicolons in the SQL statement, the use ofstacked queries was made impossible.

Giving up yet?

At this point, we were able totrigger an unauthenticated SQL Injection in a SELECT statement in theiviewdb database, which did not provide us with any meaningful starting points for an escalation to RCE.Not wanting to abandon the goal of achieving code execution we brainstormed for other approaches.Eventually we came up with the following idea - what if we modified our payload in such away that the SQL statement returned values in the expected form? Could thisallow us to trigger the subsequent Perl logic and eventually reach a point wherea code execution took place? Constructing a payload which enabled us to return arbitraryvalues in the queried columns took some attempts but succeeded in the end.

After we had managed to construct such a payload weconcentrated on the subsequent Perl logic. Looking at the source we found apromising EXEC call just after the database query. And one of theparameters for that call was derived from a variable under user control.

Unfortunately, the variable $g_ha_mode(most likely related to the high availability feature) was set to falsein the default configuration. This prompted us to look for a better way.The function mergequarantine_manage did not contain any furtherexec calls but triggered two other Perl functions in the same file, under theright conditions. Those functions were triggered via theapiInterface opcode which generated a new CSC request on port 299.

In our case $request->{action} wasalways set to release restricting us to a call tomanage_quarantine. This function used its submitted parameters(result-set from the query in mergequarantine_manage) to trigger anotherSELECT statement. When this statement returned matching values an EXECcall was triggered, which got one of the returned values as a parameter.

The question now was how the result-set ofthe second SELECT statement could be manipulated through the result-set ofthe first statement? How about returning values in the first query which would triggera SQLi in the second statement? Because string concatenation was used to construct the statement this should have been possible in theory.Unfortunately, we were unable to obtain the desired results. This was after having invested quite a bit of work to craft such a payload. A brief analysis of how our payload was processed, revealed that it was somehow escaped before reaching thesecond query. As it turned out, the reason for this was actually pretty obvious. As the functionwas triggered via a new CSC request, it automatically passed through thepreviously described escape logic.

Time to accept our defeat and be happy withthe boring SQLi? Not quite...

Desperately looking for other ways to weaponizethe injection we dug deeper into the involved components. At an earlier stagewe already created a full dump of the iviewdb database but did not paytoo much attention to it after having realized that it did not include any usefulinformation. On revisiting the database, one of its features - so called user-definedfunctions - heavily used by the appliance, stood out.

User-defined functions enable the extension of thepredefined database operations by defining your own SQL functions. Those can bewritten in Postgres' own language: PL/pgSQL. What made such functionsinteresting for our attack was, that previously defined functions could be calledin-line in SELECT statements. The call-syntax is the same as for any other SQLfunction, i.e. SELECT my_function(param1, param2) FROM table;.

The idea at this point was, that one of theexisting user-defined functions might allow the execution of stacked queries.This would be the case as soon as a parameter was used for a SQL statement withoutproper filtering inside a function. Walking over the database dump revealedmultiple code blocks matching this characteristic and to our surprise an even simpler way to execute arbitrary statements - the function execute.The respective code expected only one parameter which was directly executed as SQLstatement without any further checks.

This function would, in theory, allow us toexecute an INSERT statement inside the SELECT query ofmergequarantine_manage. This could be then used to add database rows tothe table tblquarantinespammailmerge which should later end up in theexec call in manage_quarantine.

After fiddling around for quite some time wewere finally able to construct an appropriate payload (see below).

Explanation:

• Line 1-2: Defining the two HTTP parameters needed for mode2531.
• Line 3-6: Defining the three Base64 encoded parameters, that areneeded to pass the initial checks in mergequarantine_manage.
• Line 7: Triggering the SQLi by injecting a single quote.
• Line8-11: Utilizing the user-defined function execute in order to triggerdifferent SQL operations than the predefined SELECT.
• Line 10: Addinga new row to the table tblquarantinespammailmerge that contains ourcode-execution payload in the field quarantinearea and sets messageidto 'a'. Note the .eml portion inside the payload, which isrequired to reach the exec call.
  
• Line 9: Delete all rowsfrom tblquarantinespammailmerge where the messageid equals 'a'.This ensures that the mentioned table contains our payload only once (rememberthat the vector is injected 6x in the initial statement). Whereas this is notabsolutely necessary it simplifies the path taken after the SELECT statement inmanage_quarantine and prevents our payload to be executed multipletimes.
• Line 12-14: Needed to comply with the syntax of the predefinedstatement.

Using the above payload resulted in theexecution of the following Perl command:

So finally our job was done... but somehow there seemed to be no time delay, which would indicate that oursleep has not actually triggered. But why? Did we not use exactly the same executionmechanism as in the n-day? Turns out - not quite. Asnarök usedEXECSH we have EXEC. Unfortunately EXEC is treatingspaces in arguments correctly by passing them in single values to the script.

I assume we better bury our heads in thesand

We had come to far to give up now, so we carried on. Finally we were able to execute code through the SQLi and it was good ol' Perl which allowed us to do so.

Adding this last piece to the attack chain andfixing a minor issue in the posted payload is left up to the reader.

Timeline

• 04.05.2020 - 22:48 UTC: Vulnerabilityreported to Sophos via BugCrowd.
• 04.05.2020 - 23:56 UTC: First reactionfrom Sophos confirming the report receipt.
  
• 05.05.2020 - 12:23UTC: Message from Sophos that they were able to reproduce the issue and areworking on a fix.
• 05.05.2020: Roll out of a first automatic hotfix bySophos.
• 16.05.2020 - 23:55 UTC: Reported a possible bypass for theadded security measurements in the hotfix.
• 21.05.2020: Second hotfixreleased by Sophos which disables the pre-auth email quarantine releasefeature.
• June 2020: Release of firmware 18.0 MR1-1 which contains abuilt-in fix.
• July 2020: Release of firmware 17.5 MR13 which contains a built-in fix.
• 13.07.2020: Release of the blog post in accordance with the vendor after ensuring that the majority of devices either received the hotfix or the new firmware version.
  

We highly appreciate the quick response times,very friendly communication as well as the hotfix feature.

Sophos Kba 135412 Security

On April 25, 2020, Sophos published a knowledge basearticle (KBA) 135412 which warned about apre-authenticated SQL injection (SQLi) vulnerability, affecting the XG Firewallproduct line. According to Sophos this issue had been actively exploited atleast since April 22, 2020. Shortly after the knowledge base article, a detailed analysis of the so called Asnarök operationwas published. Whilst the KBA focused solely on the SQLi, this write up clearly indicatedthat the attackers had somehow extended this initial vector to achieve remote code execution (RCE).

The criticality of the vulnerability prompted us to immediately warn our clients of the issue.As usual we provided lists of exposed and affected systems. Of course we also started an investigation into the technical details of the vulnerability. Due to the nature of the affected devices and the prospect of RCE, this vulnerability sounded like a perfect candidate for a perimeter breach in upcoming red team assessments. However, as we will explain later, this vulnerability will most likely not be as useful for this task as we first assumed.

Sophos Kba 135412 Update

Our analysis not only resulted in a working RCEexploit for the disclosed vulnerability (CVE-2020-12271) but also led to the discovery ofanother SQLi, which could have been used to gain code execution (CVE-2020-15504). Thecriticality of this new vulnerability is similar to the one used in theAsnarök campaign: exploitable pre-authentication either via an exposeduser or admin portal. Sophos quickly reacted to our bug report, issuedhotfixes for the supported firmware versions and released new firmwareversions for v17.5 and v18.0 (see also the Sophos Community Advisory).