Re: [Hampshire] [OT] Big network outage

Rob Malpass wrote:
> Hi all
>
> I hope this doesn't turn into a thread that spirals off in all directions but if someone could shed some light on this for me (a self taught enthusiastic amateur) I'd be grateful.
>
> Today at work, there was no web access. To the best of my knowledge, internet email was ok. Calling our IT guys, a recorded message said there was a national problem and n3 (whoever they are) and BT were working on it.
>
> Being the enthusiastic amateur that I am, I did a few tests from the W2k command line...
>
> ping www.bbc.co.uk - no packet loss
> ping www.abc.net.au - 100% packet loss
> ping www.itn.co.uk - 100% packet loss
>
> [and on all 3 occasions, it reported the IP address it was trying to ping so I conclude (probably incorrectly) no DNS issue]

That's often a good test - but I bet that your local network has a DNS
cache, so it might have been getting it from there. A good tool to use
for testing DNS issues is either dig (part of BIND on *nix), or nslookup
(on most Linux systems, and Windows). This will let you directly query a
DNS server. As you're an enthusiastic amateur, I'll let you google to
find out how to use them ;)

> I also tried tracert www.bbc.co.uk which sometimes worked, sometimes timed out at hops that didn't identify themselves.

That would be consistent with a "major outage" in BT speak. Obviously, I
don't know the extent of the problem, but from that, I would probably
conclude that BT probably lost a connection to one of their Internet
transit points, or lost a core router.

Or an engineer just decided to break the router... it happens more often
than you might think, it just often goes unnoticed ;)

> Webpages looked as if they would work, but after the first few bytes e.g. just the google logo, they all just hung.

Could potentially be a proxy issue - without further testing I couldn't
tell you. If you're being transparently proxied (as NTL do) then if
their proxy network died, you'd lose access to some/all websites, but
other connections and services would continue to work.

Additonally, as some of your packets would be trying to traverse over
the 'dead' link, you'd lose connection half way through loading a page,
and this would explain why it sometimes worked, sometimes only gave you
half the data, and sometimes didn't work at all.

I'd blame a BT engineer - he probably just broke the BGP routing tables
and blackhole'd half the int0rw3b ;)

(As a test - if there's anything you don't understand above - Google it
and find out - it'll increase your knowledge of networking and how the
internet "works" no end! :)

Cheers,

/j
- --
Jon Fautley RHCE, RHCX <jfautley@???>   direct: +44 1483 739615
 Technical Account Manager                     office: +44 1483 300169
 Red Hat UK                                    mobile: +44 7841 558683
 10 Alan Turing Road, Surrey Research Park, Guildford, Surrey, GU2 7YF