Other issues with this approach:
- Proxies would route lots of traffic through few IP addresses
- Large campus networks (e.g. universities) sometimes use single outbound IP addresses
- If you export APIs, these may be being polled too frequently accidentally
- External services which fetch from/aggregate your site
- Extremely large DDoS with low per-client connections would not be detected
- It might not be clear why your server is responding to some people but not others 2 years down the line
One thing to mention with IPv6 is that the namespace is /FAR/ larger than IPv4 (10^29 times as big, roughly), so internet wide scans will no longer be feasible based solely on incrementing IP addresses (they could filter it down by only the assigned IP addresses though, but that's still a pretty large namespace). [At 1 American trillion (10^12) addresses per second it would take ~10^19 years to scan the entire namespace, vs just 72 minutes for IPv4.]
One of the risks of DDoS is that it can tie down your servers/max out your disk space/break replication/overload network disk bandwidth/cause other issues which may take hours or days to resolve afterwards. One way to deal with this is to have a 'panic button' script which you can hit and it forwards all requests to a static copy of your site hosted on a large CDN such as Amazon S3. Purchases wouldn't go through this way, though, it just helps your servers to not become overwhelmed and break.
Cheers,
Benjie.
On 5 Sep 2011, at 16:58, James Bensley wrote:
> Hi Damian,
>
> I like your script for pulling out IPs and counting their entries,
> works just fine on my dev machine, but I don't see how it could be
> practically used. Looking at the number of times alone one IP has
> accesses your site is not a good measurement of being DDoS'ed. It just
> means someone loves your site.
>
> I guess it would be obvious if you have say 1000 hits a day total
> aggregate on average, and you see one single IP access your site
> 10,000 in the last ten minutes. Before that can happen though you need
> to add some date functionality in there otherwise the data is
> meaningless; you have nothing to reference it against, presumably it
> needs to be at least 'hits per IP over X time period'.
>
> Also, I think if your script passed those IPs to iptables as rules
> that would be awesome! In fact now that I think about it, maybe you
> could just do this all in IP tables without a script?
>
> /sbin/iptables -N HTTPHITS
> /sbin/iptables -A HTTPHITS -p tcp -m state --state NEW -m tcp --dport
> 80 -m recent --set --name HTTP --rsource
> /sbin/iptables -A HTTPHITS -p tcp -m state --state NEW -m tcp --dport
> 80 -m recent --update --seconds 180 --hitcount 1000 --name HTTP
> --rsource -j DROP
>
> So these rules will essentials drop traffic from an IP that has
> already made 1000 requests to your server within the last 3 minutes,
> something like that perhaps? Like your script though, just going on
> numbers of hits is a dangerous method.
>
> If you run high traffic sites though (or low for that matter) the
> first give away for the (D)DoS is (IMO) going to be (as you would
> expect from a DoS) the sudden peak in number of open connections, CPU
> utilisation, memory utilisation, increased network throughput,
> increased drive I/O. A sudden spike in your load anywhere in fact,
> such as on your DB servers, or front end servers, load balancers, edge
> routers blah blah blah. So, graph everything all the time, set up
> alerts and you should be OK (YMMV!).
>
> Regarding your other points. My first port of call would be my
> upstream connectivity provider, I would get them to black whole the
> traffic, if you aren't dropping it with automatic IP tables rules that
> is :) Most providers support communities when you directly peer with
> them through BGP and have black-wholing communities etc so there is
> scope to work with your upstream provider.
>
>> What general growing problems do systems engineers face in the future?
> Regarding what exactly? Security, or infrastructure, or both?
>
>> Will IPv6reduce DDoS attack success or enhance the attacker's tool kits?
> I don't think IPv6 gives any extra powers to those wishing to DoS, it
> just puts a different spin on it. For example, IP blacklists and
> firewall tables are going to become massive! Manufacturers are going
> to be under pressure to forward at 100Mbps, 1Gbps, 10Gbps and 100Gbps
> with giant caching tables/routing tables. For your firewall devices,
> comparing a list of millions of blocked IPv6 addresses against every
> flow passing through in a few milliseconds to maintain a high
> throughout rate is going to require some seriously fast technology!
>
>> Can we reassure customers that they will not lose business to DDoS
>> without investing large amounts capital in security technologies?
> Yes and no :) IMO, very careful and meticulous planning and mitigation
> can prevent a high percentage of (D)DoS attacks or stamp them out very
> quickly, but if the opposition has lots of capital or infrastructure
> at their disposal there can be only one winner!
>
>> What do you think? - is DDoS a global or local problem; or both?
> Do you mean local as internal to your network? So attacks sourced from
> your local network vs globally, then globally IMO.
>
>
> Sorry about the length!
>
> --
> James.
> http://www.jamesbensley.co.cc/
>
> --
> Please post to: Hampshire@???
> Web Interface: https://mailman.lug.org.uk/mailman/listinfo/hampshire
> LUG URL: http://www.hantslug.org.uk
> --------------------------------------------------------------
