Traffic graphics and statistics using MRTG

Displaying statistical graphics to assess the use of the Internet access band is considered an optional feature of a router. Yet, it is important to know this information to understand whether in Internet access there are inefficiencies due to poor band distribution among the traffic types (VoIP, WWW, P2P, FTP, …) competing to use the Internet connection.
Lots of routers use SNMP (Simple Network Management Protocol) to export the value of incoming and outgoing traffic counters for each of the network interfaces. Using software such as MRTG (Multi Router Traffic Grapher) it is possible to repeatedly, and at regular time intervals, run SNMP queries towards these routers and save the traffic counters. Once this is done, MRTG enables the graphic analysis, via a browser, of incoming and outgoing traffic progression from the router interfaces.

Example of MRTG graphic relating to WWW classified traffic
Example of MRTG graphic relating to WWW classified traffic

Zeroshell does not follow this export strategy using SNMP (see note *), but integrates MRTG directly within to enable the analysis of parameters which go well beyond those obtained using SNMP. In virtue of this, the following parameters can be analysed directly from the Zeroshell web interface:

  • System load
  • Number of active connections (TCP/UDP) from and to Internet;
  • Incoming and outgoing interface traffic, whether an Ethernet card, a VLAN 802.1q, a VPN, a bridge, a bond, a PPPoE connection (e.g. ADSL) or a 3G mobile connection (e.g. UMTS/HSDPA);
  • Traffic classified by traffic shaping in a determined QoS class (VoIP, HTTP, peer to peer, …) in relation to the overall interface outgoing traffic;
  • Balancing of Internet traffic on various WAN Gateways (Load Balancing and Failover) compared to the total traffic from and to Internet.

The remainder of the document is sub-divided into the following sections:

System load average

The statistical information on Load Average does not directly cover network traffic, but is however useful to understand whether the router hardware resources (the processor in particular) are a bottleneck for the LAN and slows down connections independent of the band available on the access links to the Internet. For a system load graphic click on the [Graphics] link in the frame on the top right. A window appears like the one displayed below.

 

Graphic relating to System Load
Graphic relating to System Load

The average load calculated every 5 minutes multiplied by 100 is taken into consideration. The percentage of system use (reported in round brackets) takes into account the number of router CPU. In other words, let’s assume a load of 100 on a system with 2 processors, the percentage of use indicated is 50%. Therefore the critical threshold for which the router can be suspected of being a bottleneck is 200 equal to 100% use.

The factors mainly contributing to CPU use in increasing order are:

  • Firewall Rules, QoS classification and manual Load Balancing
  • Firewall Rules and QoS that use the Layer 7 filters to run the DPI when a lot of connections are present. Note that the L7 filters inspect the content of the packets only as soon as a connection is established, while the remainder are identified using Connection Tracking. This highlights that the application level filters do not load the system based on the band used, but on the basis of the number of new TCP/UDP connections opened.
  • Writing the result of Connection Tracking in the logs. Keeping track of the TCP/UDP connections is not a very wasteful functionality in terms of CPU. Yet, it can be if the system is configured to register connections (source IP, source port, destination IP, destination port) in the logs.
  • Captive Portal active on a LAN with plenty of active clients, but not yet authenticated. Often, the presence of WORMs or other software that use the TCP 80 and 443 ports for requests other than classic HTTP/HTTPS requests can make the situation worse.
  • Use of the transparent proxy http with antivirus (ClamAV) or a filter on web content (DansGuardian). In fact, having to examine the content of web pages will inevitably heavily occupy the CPU. In such cases, it is necessary to also ensure an adequate RAM quantity to avoid disk swapping.

Active TCP/UDP connections

The progression of the number of active connections is a good index to monitor network activity. For example, a high number of connections could mean file exchanges using P2P techniques.

 

Graphic relating to the number of active connections
Graphic relating to the number of active connections

Remember that Zeroshell is different from certain routers that forget TCP connections over a short timeout period, because it is configured to keep track of connections which do not exchange traffic even for long periods of time (e.g. interactive SSH sessions in IDLE for days). If on the one hand this is an advantage, on the other, where connections are not correctly closed, it can cause connections to be saved that haven’t been active for some time. If you wish to set a timeout for TCP connections, set the parameter /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established to the number of seconds after which a connection is considered expired, after inactivity, and therefore cancelled by the Connection Tracking tables.

Traffic incoming and outgoing from a network interface

The traditional use of MRTG is to enable traffic monitoring of the network interfaces of a router both upstream and downstream. The same graphic tracks the incoming traffic in GREEN, while outgoing traffic is in BLUE.

 

Graphic relating to incoming and outgoing network interface traffics
Graphic relating to incoming and outgoing network interface traffics

The percentages refer, where possible, to the maximum band the interface can support. Zeroshell enables the traffic graphic to be obtained in download/upload of the following interface types: Ethernet, VPN, PPPoE and 3G. The same can be said for interface combinations such as bonds and bridges and for VLAN 802.1q. Furthermore, if Zeroshell is used as a Wi-Fi Access Point with multiple SSID, it is possible to obtain the traffic graphic for each SSID.

Traffic graphics sub-divided by QoS classes

If traffic shaping is active on a network interface, it is possible to display the graphic relating to the outgoing traffic classified by traffic type. The diagram of the total traffic outgoing from the interface is tracked in BLUE, while the traffic classified in the chosen QoS class is in GREEN.

 

Graphic relating to traffic per QoS class
Graphic relating to traffic per QoS class

The colour AMBER represents the QoS percentage of use compared to the total interface traffic. Therefore, the figure displayed above easily shows that the VoIP outgoing, ETH03 interface traffic is on average 4% of the total traffic, with peaks reaching 33%.

Traffic distribution on Internet Gateways in load balancing

Thanks to Net Balancer, Zeroshell can distribute Internet access traffic over multiple WAN connections which can be xDSL, 3G or another. Balancing can be automatic with weighted Round-Robin or manual with rules (similar to those of Firewall and QoS classifier) that force determined types of traffic to use a determined gateway. For automatic load balancing, it is useful to consult the traffic distribution graphic to understand whether the gateways are used in proportion to the maximum band available to them. If on the contrary the weight of the gateway can be modified. This parameter is in fact directly proportional to the probability the connection is routed on that link.

 

Graphic relating to traffic distribution on an Internet Gateway
Graphic relating to traffic distribution on an Internet Gateway

GREEN indicates the incoming and outgoing traffic for the chosen gateway, while BLUE indicates the total Internet traffic.
The percentage ratio between the traffic on the chosen link and the overall traffic is in AMBER.

 


Note:
(*) If instead of using the integrated MRTG package you prefer to export the traffic counters via SNMP and use an external monitoring packet, install the net-snmp packageĀ  from the Zeroshell’s Package Manager.