Introduction
Lately, we can spot quite a lot of reports, claiming that Google (8.8.8.8) and Cloudflare (1.1.1.1) DNS servers are under DoS attacks. For instance, @GossiTheDog has provided the info that he noticed attack destined for 1.1.1.1 in Qihoo 360 feeds. Then, @yiminggong from 360 Netlab provided info that 8.8.8.8 is also frequently attacked.
Let us look at what DDoS Mon tells us about the attacks on Google DNS with IP 8.8.8.8:
As we can see, attackers used many different vectors. They were mainly using amplifications vectors (NTP, CLDAP, Chargen and Portmap). However, raw SYN floods and DNS`floods also appeared.
Now, let's look at the Cloudflare DNS with IP 1.1.1.1. We definitely have more events detected here. Just to provide a few:
udp@attack@small_package_flood_target
tcp@attack@syn_flood_target
udp@attack@amp_flood_target-CLDAP
udp@attack@amp_flood_target-SSDP
tcp@attack@syn_flood_target-payload
udp@attack@amp_flood_target-MEMCACHE
- and others.
According to this data, something is definitely going on, so we have decided to investigate this DoS attacks. In earlier blogposts, we were mainly showing how we can fingerprint botnets using our network telescope (darknet). In this blog post, we are going to show how we can use it for the DoS observations.
We have started to observe some attacks in the beginning of the May. For the presentation purposes, we will analyze data from 26 May 2018 to 12 Jun 2018.
8.8.8.8
Let us start with the Google DNS. For the beginning, let's list all the attacks that we were able to distinguish:
TCP SYN
floodUDP
floodICMP PING
floodNTP
floodDNS
floodDNS NXDOMAIN
flood
All the packets signalizing the attack are presented on the following chart with the logarithmic scale. As it can be seen, Google is mainly dealing with the DNS floods. However, we can see that three different kind of attacks were visible at the same time: DNS normal and NXDOMAIN flood + PING flood. It can be seen perfectly on 30.05.2018 or 01.06.2018.
Now, let's dig deeper into some details.
8.8.8.8 - SYN flood
Nothing really interesting in there, as we have not seen many packets. Port 53 was flooded and no signatures have been detected.
8.8.8.8 - NTP flood
Again, we have not received many NTP packets, but it is interesting that we have received any NTP responses from the Google DNS server. There is a probability that this packets were spoofed, because we are unaware of NTP being hosted on 8.8.8.8. However, TTL values included in the NTP packets were the same as in packets with different protocol/payload, thus signalizing that we were in fact receiving valid NTP responses from 8.8.8.8.
8.8.8.8 - UDP flood
By the UDP flood, we mean UDP packets without any payload. You can ask us, how have we noticed these in darknet, as the flood is targeting some ports which are not opened and should provide no answers. So, we have received them with the ICMP packets (actually, original packets are stored in the payload of ICMP packets), which were unable to reach its destination. We actually love these ICMP packets, because we can see original spoofed packets. It allows us to dig even deeper when we are looking for the signatures in the protocols headers.
We have been receiving following ICMP control messages:
- 11:0 - TTL expired
- 3:3 - Destination port unreachable
We have not observed many of these packets. However, on their basis we can claim that:
- source ports during floods were randomized,
- destination ports were narrowed to some small group or they were just incremented during flood.
8.8.8.8 - PING flood
It's getting more interesting in here, as we have definitely seen more packets for PING floods. Let's have a look at the timeline of ICMP echo replies seen in the network telescope.
Now, let's analyze payloads for every day.
- 26.05 - many different payloads appeared, without any visible pattern
- 30.05 - every packet had the same payload, where next bytes were incremented
6162636465666768696a6b6c6d6e6f70...
- 01.06 - again, the same payload as on 30.05, so
6162636465666768696a6b6c6d6e6f70...
- 04.06 - all packets had the same payload, filled up with
0x00
values:00000000000000000000000000000000...
- 07.06 and further - many different payloads, however after 10.06, payloads from 30.05 and 01.06 appeared again.
We can clearly see that some payloads were used more frequently than the others. Let's focus on 30.05 and look how these ping flood backscatter packets were distributed in time.
Interesting is the fact that only our 1 IP address was hit by these packets. It can mean at least one of the following:
- attack was not so big,
- we were unlucky and only one of our IP addresses was drawn,
- attack was well distributed among whole IPv4 address space.
8.8.8.8 - DNS NXDOMAIN flood
DNS NXDOMAIN
flood is a type of the attack, when the attacker queries for non existing DNS records. Because of that, DNS server is trying to look for non existing records and wastes its resources.
Next chart presents timeline of the NXDOMAIN flood backscatter observed in our network telescope.
As we were analyzing day 30.05 earlier, next chart presents traffic during that day.
There are two interesting facts about this flood from 30.05:
- timeline matches perfectly the timeline of
PING
flood - again, only 1 IP from our darknet was hit by these packets.
Volume of this flood was definitely the highest between 10.06 and 12.06, but we will talk about it later. Now, let's have a look on the top 4 domains queried during NXDOMAIN flood.
This domains will also appear in the further part of the blog post, so keep them in the memory.
8.8.8.8 - DNS flood
We are reaching the final stage of the 8.8.8.8 analysis. However, things are the most interesting in here. Next chart presents once again logarithmic chart with the events timeline, but with the DNS floods only. As you can see, NXDOMAIN floods and typical DNS floods were usually correlated. The only anomaly in here is the very big DNS flood on 30.05. According to that, we have decided to split the analysis of these floods into two parts:
- 30.05 flood, with about 4 millions of backscatter packets,
- everything else.
We will start with the "everything else" stuff. For example, let's analyze the 10.06 - 12.06 period. Next chart shows timeline for the DNS NXDOMAIN floods and the typical DNS floods during that period.
As it can be noticed, timelines for these floods correlate nicely. Are these performed by the same actor? Well, we can check if information stored in collected network packets can help. First of all, DNS_ID and Source Port values (seen in backscatter) are randomized in both cases, so we can learn nothing from them. However, let us look at the top queried domains (there were a lot of different domains used during attacks, but for the presentation purposes we used the most common ones only).
We can clearly see that domains used during typical DNS flood were often reused and combined during NXDOMAIN flood, thus signalizing the same actor being responsible for these floods.
Okay, now we can start the dessert: DNS flood on 30.05. For the beginning: timeline of the attack.
Once per 5 minutes, we were observing about 250 000 packets and the whole attack lasted for about 80 minutes. Once per 5 minutes, about 12 000 IP addresses from our darknet were being hit. Additionally, the timeline doesn't match the PING floods and NXDOMAIN floods from that day. What is more, we have observed many different domains during the attack, but mainly only two unique domains were used for the DNS queries:
- www.baidu.com - 2 104 624 DNS backscatter packets,
- www.qq.com - 1 878 501 DNS backscatter packets.
Now the best part. Our signature analyzer has detected the following signature in the backscatter packets:
DNS RESPONSE -> **DNS_ID == DPORT**
It means that attackers were using the same value in DNS_ID field and Source Port field during packet generation (in responses we analyze destination port, but from the attacker point of view, signature concerns source port).
This is a pretty big deal, because such a signature may be used for the fingerprinting of botnet/actor responsible for the attack. However, it is not an end. During such a big floods, we usually receive ICMP packets with original spoofed packet stored in the payload. It was exactly the same this time, as we have received about 40 ICMP packets with either type:code equal to 11:0 or 3:3.
In this case, our signature analyzer has found the following packet generation signature in the original spoofed packets:
DNS QUERY -> **DNS_ID == SPORT == IP_ID**
So, not only DNS_ID was equal to Source Port during the packet generation phase, but also to IP_ID. As IP_ID of query is lost in the response, we have not seen this part of the signature during the observation of backscatter packets. Currently, we are unaware of botnet or tool using this signature.
8.8.8.8 - summary
To sum up this part, we can state that:
- We have observed several different vectors used during attack (and we cannot observe amplified DRDoS attacks in the darknet - DDoS Mon signalizes that there were a lot of these attacks too).
- Some of the attacks can be correlated with each other and can be distinguished by the analysis of payload (ICMP PING flood), domains (DNS floods) or any other input provided to the protocol headers.
- We have found very clear PGA (Packet Generation Algorithm) signature during huge DNS flood - which signalizes that some botnet or customized tool was used for the attack.
Right now, we can analyze the DoS activity for the Cloudflare DNS - 1.1.1.1.
1.1.1.1
In order to have something to start with, let's have a look at the type of attacks that we have detected for the 1.1.1.1 DNS:
TCP SYN
flood,TCP ACK
flood,UDP
flood,ICMP PING
flood,DNS
flood,DNS NXDOMAIN
flood.
Some of these look familiar, right? Despite the TCP ACK flood, all these attacks types were also visible for the 8.8.8.8. The next chart presents timeline of these attacks (logarithmic scale).
From this chart, we can right away learn two interesting things:
- TCP floods were more frequent than DNS floods.
- typical DNS floods and DNS NXDOMAIN flood are perfectly correlated.
1.1.1.1 - UDP flood
Nothing really interesting in here. We have received several ICMP packets, where payload contained UDP packets with and without payload. These packets targeted random ports and were having random source ports too.
1.1.1.1 - PING flood
In case of this IP, we did not see many PING flood backscatter packets (about 70 packets only). All of the packets had the same payload filled up with 0x00 values: 00000000000000000000000000000000...
1.1.1.1 - TCP ACK flood
In this case, we were observing RST packets from the victim, which signalize that ACK flood was performed (or combination of ACK with other TCP flags). We can see that these packets were sent from time to time and we did not see any short but aggressive flood (maybe despite the flood on 29.05, where we have observed thousands of packets).
Interesting fact: port 80, not 53 was attacked. During the attack, random source ports were generated and we did not found any useful signatures in this traffic.
1.1.1.1 - TCP SYN flood
This flood still targets port 80. However, things are different here. First of all, let's have a look at the timeline.
Again, we did not see a lot of backscatter packets (14 200 and these were distributed among many days). However, it is still clear that Cloudflare DNS was under attack. Moreover, we have observed very interesting signature during this attack.
It has occurred that almost every SYN-ACK packet received from 1.1.1.1 had the Acknowledgment number equal to 0x00000001. It means that during the attack, all packets were generated with Sequence number equal to 0.
Moreover, let us look at the top destination ports observed in the backscatter SYN-ACK packets (so, source ports from SYN packet).
As you can notice, port 1234 was used more frequently than the others. What is more, packets with Destination Port equal to 1234 had also the Acknowledgment number equal to 0x00000001. It means that during the attacks, we have fingerprinted two different PGA signatures (written from the attacker point of view):
SYN -> **SEQ == 0**
_and_
SYN -> **SEQ == 0 _and_ SPORT == 1234**
So, we have been able to find another signature, which can be pinned to the proper botnet, tool or/and actor.
1.1.1.1 - DNS flood and DNS NXDOMAIN flood
Alright, we are heading to the end with the analysis of DNS floods. As we could see on the chart with the timeline of the attacks, the real activity of these floods started on 10.06 and finished on 12.06 (the same as in case of DNS floods targeting 8.8.8.8). Also, both type of floods correlate nicely on the chart.
Yes, you have probably guessed already - domains used for both attacks (8.8.8.8 and 1.1.1.1) were identical.
Top 4 queries during DNS flood (domain:type syntax):
- collector-185.newrelic.com:1,
- collector-185.newrelic.com:28,
- poczta.radiolodz.pl:1,
- poczta.radiolodz.pl:28.
Top 4 queries during DNS NXDOMAIN flood:
- collector-185.newrelic.com.radiolodz.pl:1,
- collector-185.newrelic.com.radiolodz.pl:28,
- poczta.radiolodz.pl.radiolodz.pl:1,
- poczta.radiolodz.pl.radiolodz.pl:28.
According to this data, we can safely state that the DNS floods between 10.06 and 12.06 were performed by the same actor.
Summary
In this blog post, we have shown that the network telescope (darknet) can be very useful for DoS mitigation and for analysis of this kind of attacks. On the example of Google DNS and Cloudflare DNS, we have been able to show you that:
- Many different attack vectors were used for the attacks. In darknet, we are only able to see backscatter from spoofed attacks. However, other sources show that amplified DRDoS attacks were also performed.
- Weird signatures in protocol headers (Packet Generation Algorithm) can be visible not only during scanning activity but also during DoS activity. We have been able to determine several signatures during spotted floods. Having this signatures pinned to particular malware, tool or actor, we are able to track their malicious activity and connect their targets. PGA signatures found during discussed DoS attacks:
DNS QUERY -> **DNS_ID == SPORT == IP_ID**
SYN -> **SEQ == 0**
SYN -> **SEQ == 0 _and_ SPORT == 1234**
- We can connect attacks together not only by the PGA signatures. We have been able to show that DoS floods between 10.06.2018 and 12.06.2018 targeting both 8.8.8.8 and 1.1.1.1 were probably performed by the same actor. In order to state that, we have analyzed timeline of packets and domains used for the attack.