print

Tip of the Month

April 2004

Network Slowdowns!

Often, network problems manifest as a decrease in performance: Web pages take much longer to load, file downloads get lower throughput, and so on. Here are some tips for troubleshooting "network slowdowns."

1. Check the lower layers first. You will want to evaluate the amount of corrupted packets (those with CRC errors). These stats can be obtained by watching incremental frame error counts from router or switch ports via an SNMP tool, or by direct capture from Etherpeek. For direct capture, ensure that you have the proper NIC and drivers to capture error frames, and that you are directly (as in hub attached--switches do not forward error frames, even to mirrored ports) on the suspect error-prone segment on the path between the client and server.

If there are excessive corrupted frames and TCP is in use, the expert system in EPNX will also indicate excessive TCP time-outs and retries (TCP retransmissions). Retransmissions can be found at any point on the path prior to the corrupted frames. Even as little as one frame error per second on a given client-server conversation can dramatically reduce throughput due to TCP recovery.

If a large number of clients are experiencing TCP retransmissions, then focus on points of concentration, such as a server port, firewall, or hub. Analyzing from the "core out" or most-to-least concentrated traffic areas, will help you to more quickly eliminate certain devices and segments.

2. A TCP ACK that takes too long to return from a server can also cause retransmissions. If the server is slow in processing a request to return data, an ACK is usually returned first, to inform the sender that the packet has safely arrived. If the expected time to receive an ACK is delayed due to a slow network (either getting the original packet to the server or expeditiously returning the ACK), then the sending TCP will resend the packet.

This is a problem typical with "implied-retransmit" protocols like TCP that assume that packets are going to be lost and use the lack of an ACK as a signal for a retransmission.

The alternative is an "explicit-retransmit" protocol, in which all packets are assumed to be received, and the receiver must explicitly request retransmission of those packets that it detects were lost. In an "explicit-retransmit" protocol, the receiver can basically wait as long as it wants to ACK because the sender is not going to retransmit unless the receiver explicitly requests one.

3. If all else fails and you can locate the point of slowdown, evaluate what else is happening on the network at that time. It's possible that some other operation is consuming a large part of your bandwidth.