print

Tip of the Month

March 2004

Troubleshooting a Severe Broadcast Storm!

Excessive broadcast packets continue to be an important troubleshooting issue, even in today's sophisticated, switched networks, because switches forward broadcast packets out every port on the VLAN. Therefore, broadcast packets have a much larger effect on overall network performance than an equivalent amount of unicast packets would have. EtherPeek NX has a "Broadcast Storm" diagnosis that can help you detect excessive broadcast traffic, but what should you do when you see this diagnosis?

I'd say that first, confirm that there's really a problem. Depending on the expert's thresholds, you may be getting a false alarm. Capture using the "broadcast/multicast" filter, then count the total kbps used by broadcasts/multicasts on the link. If it's less than about 3% of the available bandwidth on the link (3 Mbps or 3,000 kbps for a 100 Mbps Ethernet link) then don't worry. If it's between 3 and 10 percent, that's questionable. If it's more than 10%, it's definitely worth investigation.

If you decide to investigate, the next step will be to determine who is sending the multicasts and broadcasts, and why. Look at the source address for the packets: that's the station sending. Look at the destination: if it's a multicast address, that can point to the application that's sending them, since some applications use a custom multicast address just for themselves. Look at the protocol, if EtherPeek decodes it: that can also point to the application.

If you've got a lot of broadcasts/multicasts, then one possible cause is a virus. This will usually result in a lot of ARPs as the station tries to locate other stations to infect. The destination address for the ARPs will either be sequential IP addresses (if the virus is dumber) or pseudo-random addresses (if the virus is smarter), but the key will be that there are a lot of ARPs for non-existent addresses, and the destination address of the ARPs will be fairly randomly distributed. In addition, probably more than one station will be ARP'ing, since the infection will likely have spread.

If one station is sending all of the broadcasts, then, obviously, quarantine it. Put it on a hub in a lab and start killing processes and shutting down applications until the ARPs stop. That won't always work, but it might identify the application or process that is causing the problem.

If many stations are sending the ARPs, it could be an authorized application, but it would have to be one that all the stations share. Possibly, the application could be reconfigured not to send as much broadcast and multicast traffic. If your network has an application that is supposed to send a lot of multicast and broadcast traffic, such as an IP television multicasting app, then you should ignore the "broadcast storm" diagnosis and focus on making sure that the broadcasts and multicasts are as efficient as possible (e.g. use IGMP and multicast-aware switches to prevent the broadcasts and multicasts from going everywhere on the network), but recognize that you're always going to have a lot of them.