DoS Attack on mail.rollernet.us (Sep. 3, 2008)

[RollerNetSupport]

We are currently investigating a possible denial of service attack against our network. Details will be posted as they become available.

http://status.rollernet.us/

[RollerNetSupport]

Our initial netflow searching reveals that traffic from Postini may be the culprit. We are going to null-route 207.126.150.0/24 and see if that makes a difference.

[RollerNetSupport]

Null routing Postini resulted in an immediate drop in traffic. All traffic was directed to mail.rollernet.us.

[RollerNetSupport]

Mail server "mail.rollernet.us" is busy catching up on the incoming queue backlog.

[RollerNetSupport]

Refining out netflow search a bit, the top talkers during the DoS period were all Postini mail servers:

Code: Select all

TCP    207.126.150.132:0     ->     208.79.240.2:25
TCP    207.126.150.130:0     ->     208.79.240.2:25
TCP    207.126.150.133:0     ->     208.79.240.2:25
TCP    207.126.150.125:0     ->     208.79.240.2:25
TCP    207.126.150.129:0     ->     208.79.240.2:25
TCP    207.126.150.128:0     ->     208.79.240.2:25
TCP    207.126.150.126:0     ->     208.79.240.2:25
TCP    207.126.150.127:0     ->     208.79.240.2:25
TCP    207.126.150.124:0     ->     208.79.240.2:25


We have reported this to Postini and will continue to null route 207.126.150.0/24 until we hear back from them.

[RollerNetSupport]

We are continuing to investigate this attack since it resulted in loss of service of "mail.rollernet.us" and network congestion for all of our other services.

[RollerNetSupport]

We have finally discovered the source (a Postini customer) and notified them.

We're going to try just blocking the source domain at the mail server level and remove the null route in our network core. This is preferred since we don't wish to block legitimate mail from Postini.

[RollerNetSupport]

Traffic patterns continue to appear normal. Here's what our traffic looked like today:



The big green abnormal spike after Wed 12:00 is mostly a SYN flood. The impact was amplified by one of our BGP peers bouncing due to hold timer expiration. Following countermeasures, ingress traffic returned to normal, and the two blue spikes are the mail server catching up with pending mail.