(or How We Accidentally DoS’ed a Buggy FTP Server on an Under-powered Host with a TCP Connect Scan)
In this post, I’m going to share an interesting bug in some not-widely used but freely available FTP server software when experimenting with different parameters to Nmap scans. The particular server, which I’ll just call Server X, doesn’t appear much in the wild as far as we know, but we’ll take a look at why it responded the way it did, and under what particular hardware configurations.
When a scan becomes an unintentional Denial of Service (DoS)
A few years ago, using earlier prototypes of what has now become Siege’s R4NG3R technology, we were provisioning various network configurations with different representative services installed on various nodes, and comparing the output from Nmap when run with different parameters.
Most types of scans ran fine against most targets, but there were several test configurations that consistently failed to return results. It seemed that the virtual machines being scanned had completely hung. While automated testing and data collection is our usual modus operandi, manual investigation is always an important fallback (and knowing what we can do by hand informs tomorrow’s automated processes).
We fired up a few test environments, recreating some where results had been obtained, and some where the virtual machines had hung. Looking at the configurations, it seemed that the hangs were related to:
- An Nmap scan with a particular type of scanning technique
- Server X (all failures included this server) running on a particular version of Windows.
(It turns out that the operating system wasn’t important, but we’ll see why it looked that way initially.)
We fired up the Nmap scans on the targets. In the configurations known to be problematic, we found:
This was consistent with our expectations. We had hypothesized that the failure to retrieve logs from the scanned target was the result of the machine being unresponsive. In the configurations known to not be problematic, which used the same Nmap scan, but a different operating system version, we were somewhat surprised to find a similar, but not fatal, CPU load.
But why would a different operating system version cause a difference like a difference like that in the way that Server X uses compute resources? It turns out that we had unwittingly introduced a hidden variable that correlated with operating system version. We had created the test virtual machines by cloning some preconfigured base images, and had not included hardware parameters, such as the number of CPU sockets and virtual processors, as variables in the test. As you have probably guessed, the base image for one version of the operating system had only one virtual processor, while the other had two. That led to a quickly testable hypothesis:
The Nmap scan causes Server X to completely utilize one available processor.
We created new target virtual machine with five virtual processors and launched the scan. Sure enough, 20% utilization.
Another reasonable question is how Server X responds after it’s in this state. For instance, if it’s a very simple single-threaded application, it might not respond at all after it enters this state. Alternatively, if it responds to an incoming connection by spawning a new thread or process, and it’s only the new thread or process that’s spinning, the server might still be semi-operational. We re-ran the Nmap scan against the target with five virtual processors.
Very interesting. We’d unintentionally found a way to DoS a system running Server X. But Nmap isn’t really aimed at being an attack tool; it’s a network scanner. What’s going on here?
What Nmap scan option was used?
In the course of testing, we’d been using various combinations of Nmap flags, watching for different behaviors in target systems, or in the information discovered. We aimed to examine lots of different combinations, not just “the best choice” for a given environment. The behavior described in the article occurred while using -sT. Nmap’s --help output tells us that this makes Nmap use a TCP Connect() scan:
$ nmap --help
Nmap 7.92 ( https://nmap.org )
...
SCAN TECHNIQUES:
-sS/sT/sA/sW/sM: TCP SYN/Connect()/ACK/Window/Maimon scans
The Nmap book has an entry on TCP Connect Scan (-sT), which begins:
TCP connect scan is the default TCP scan type when SYN scan is not an option.
SYN Scan is described in another chapter of the Nmap book, TCP SYN (Stealth) Scan (-sS), which begins:
SYN scan is the default and most popular scan option for good reason. It can be performed quickly, scanning thousands of ports per second on a fast network not hampered by intrusive firewalls. SYN scan is relatively unobtrusive and stealthy, since it never completes TCP connections.
When a TCP connection is established there is a three-way handshake:
- Client sends a SYN packet to the server.
- Server responds to the SYN with a SYN/ACK packet.
- Client responds to the SYN/ACK with an ACK packet.
It’s only after the three-way handshake is complete that applications become aware of the new connection. However, a client and (potential) server could also exchange this sequence of packets:
- Client sends a SYN packet to the server.
- Server responds to the SYN with a SYN/ACK packet.
- Client response to the SYN/ACT with an RST packet, resetting (aborting) the connection.
In this modified exchange, the full connection is never actually established, and the application never becomes aware of a new communication channel. This is what Nmap’s SYN scan does, and is why it’s stealthy.
To perform that exchange, however, requires higher privileges for constructing custom packets, so isn’t an option if Nmap is being run higher privileges. The TCP Connect scan uses the three-way handshake to establish a connection, and then immediately terminates that connection. The exchange is:
- Client sends a SYN packet to the server.
- Server responds to the SYN with a SYN/ACK packet.
- Client responds to the SYN/ACK with an ACK packet.
- Server may send content…
- Client sends RST to abort the connection.
Because this establishes a full connection, it is less stealthy (firewalls and applications could log the connection, etc.) and takes longer.
Why did Server X respond so poorly?
Scanning with -sT uses complete TCP connections when scanning, and this is less stealthy since applications, firewalls, and so on, are aware of the connection. But servers are supposed to accept incoming connection, so why would opening a connection and then immediately closing it cause a CPU hang? The TCP Connect Scan (-sT) section also has this to say about Connect scans (emphasis added):
Many services on your average Unix system will add a note to syslog, and sometimes a cryptic error message, when Nmap connects and then closes the connection without sending data. Truly pathetic services crash when this happens, though that is uncommon.
The wording is a bit harsh, but the FTP server is some what old and has known vulnerabilities, so it’s not surprising that its connection handling logic might not be perfect either.
Playing with Server X
So, we know that Server X pegs a CPU to total utilization when hit by Nmap’s Connect scan, but we don’t know exactly yet why.
I tried reproducing the behavior with nc instead of nmap. It’s easy enough to open a connection, write nothing, and then disconnect:
# nc 10.0.2.4 21
220- ... Server X's banner here
220- ...
222
^CThe ^C at the end is the console’s way of showing that I pressed Control-C to kill nc. Somewhat surprisingly, this didn’t cause Server X to hang. I don’t believe that nc sent any input to the server, so I think the behavior is pretty similar to what nmap did. I fired up tcpdump to capture the traffic during the nmap and nc exchanges. After the initial three-way handshake, the differences are more pronounced. nmap immediately sends the RST packet, terminating the connection. nc, however, reads a message from Server X, sends an ACKnowledgement, and then (when I pressed Control-C) initiates the four-way connection termination handshake.
There’s some subtle behavior going on here that merits more investigation.
Conclusion
We experimented with an old, evidently buggy, FTP server on a variety of operating systems and treated it to a range of Nmap scans. When we observed some installations hanging in response to what should be an innocuous scan, we dug deeper. It initially appeared that operating system version may have played a role, but it turned out that there was a hidden variable (number of CPUs) that strongly correlated to operating system (because the template virtual machines for different operating systems had different numbers of CPUs). With tcpdump and Wireshark, we observed the different packet sequences that occur with the Nmap TCP Connect scan (-sT) (which elicits the hang) and with a manual nc session (that didn’t).
This raises some more questions, that I hope to address in a followup post, such as:
- If Nmap can do an immediate disconnect (sending RST) before reading any content from the server and can do it with application level code (no special packet crafting), how does it do it? Can we do the same?
- What is the server actually doing that responds so poorly to this interaction? Is it in an endless loop waiting for the client to read the banner, or waiting for input from the client? (But if it’s just waiting for input from the client, why didn’t it hang when we used
nc?)- How can we figure that out? From a distance, we might be able to tease out some information with some custom client code. Close-up options include looking at disassembly of the server, attaching a debugger to (threads in) the running server.
Until then, enjoy your winter holidays, and get some RST