Monitoring network bandwidth usage

During the development of my monitoring tool, I've faced a little challenge - find a crossplatform (Linux/FreeBSD) way to monitor network bandwidth usage. Surprisingly, there is no simple way to do it. In this post I'll reveal my research and the solution I've found and implemented.


Linux

One of the ways to do it is using ifconfig output:

$ ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:23:54:15:54:96
          inet addr:192.168.1.4  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::223:54ff:fe15:5496/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23242 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:24061454 (24.0 MB)  TX bytes:3265008 (3.2 MB)
          Interrupt:30 Base address:0xe000

We can ask for RX (received) and TX (transmitted) bytes each second, and the differences give us the required speeds. This tool use /proc/net/dev data for showing the statistic.


FreeBSD

This OS have the standard tool - systat.

% systat -if 1
                    /0   /1   /2   /3   /4   /5   /6   /7   /8   /9   /10
     Load Average                                                       
                     
      Interface           Traffic               Peak                Total
           age0  in      0,000 KB/s        107,444 KB/s            2,230 MB
                 out     0,000 KB/s         10,668 KB/s          372,108 KB



FreeBSD doesn't provide any analogue of /proc/net/dev data, and uses internal structures for gathering the statistic.


Universal way

But there is a way which works on all the platforms: libpcap. The Packet Capture library provides a high level interface to packet capture systems. All packets on the network, even those destined for other hosts, are accessible through this mechanism.

Unfortunately, direct use of libpcap requires root privileges, so some daemon/service have to be used for gathering the statistic. I've stopped on pmacct project, and will explain further how to set up and use it.

pmacct is a small set of passive network monitoring tools to measure, account, classify, aggregate and export IPv4 and IPv6 traffic; its main features are:
  • Runs on Linux, BSDs, Solaris and embedded systems
  • Support for both IPv4 and IPv6
  • Collects data through libpcap, Netlink/ULOG, NetFlow v1/v5/v7/v8/v9 and sFlow v2/v4/v5
  • Saves data to a number of backends including memory tables, MySQL, PostgreSQL and SQLite
  • Exports data to remote collectors through NetFlow v5/v9 and sFlow v5
  • Replicates incoming NetFlow and sFlow packets to remote collectors
  • Flexible architecture to tag, filter, redirect, aggregate and split captured data
  • Pluggable architecture for easy integration of new capturing environments and data backends
  • Careful SQL support: data pre-processing, triggers, dynamic table naming
  • It's free, open-source, developed and supported with passion and open mind

    Pmacct installation

    Ubuntu:

    $ sudo apt-get install pmacct

    FreeBSD:

    $ cd /usr/ports/net-mgmt/pmacct
    $ sudo make install clean

    After the installation pmacctd daemon and pmacct client tool will be available.


    Pmacctd daemon configuring

    We have to configure two files - the daemon configuration file and networks file describing the networks which should be monitored.


    Daemon configuration file (/etc/pmacct/pmacctd.conf in Ubuntu, /usr/local/etc/pmacct/pmacctd.conf in FreeBSD):

    ! pmacctd configuration
    daemonize: true
    pidfile: /var/run/pmacctd.pid
    syslog: daemon
    ! on this interface (age0 or similar for FreeBSD)
    interface: eth0
    ! interested in in and outbound traffic
    networks_file: /usr/local/etc/networks.def
    aggregate[in]: dst_net
    aggregate[out]: src_net
    ! storage methods
    plugin_buffer_size: 2048
    plugin_pipe_size: 2048000
    plugins: memory[in], memory[out]
    imt_path[in]: /tmp/pmacct_in.pipe
    imt_path[out]: /tmp/pmacct_out.pipe

    Here we set up to listen eth0 interface, aggregate all the network information, and store the information in the memory database. Pmacct client will connect the daemon via pipes declared by imt_path parameters.


    Networks file (/usr/local/etc/networks.def):

    192.168.1.0/24

    My IP-address (192.168.1.X) is in this network, so I listen all the network.


    Now you can test the configuration by starting the daemon with it:

    $ pmacctd -f pmacctd.conf

    If everything is fine, it will start without any complains. For debugging purposes, you can start it without daemonizing and with debug: true option.


    Using the pmacct

    For getting statistic use the pmacct client:

    $ pmacct -s -p /tmp/pmacct_in.pipe
    DST_IP PACKETS BYTES
    :: 1 56
    192.168.1.0 26707 24994698
    0.0.0.0 22672 3691429

    For a total of: 3 entries

    Use the different pipes for getting inbound/outbound statistic. To get the bytes only use clarified requests like:

    $ pmacct -N 192.168.1.0 -c dst_net -p /tmp/pmacct_in.pipe
    25308311
    $ pmacct -N 192.168.1.0 -c src_net -p /tmp/pmacct_out.pipe
    4368614

    pmacct has a useful option "-r" (reset counters), so if you use it with this option every second you'll have network speed (bytes per second), as I do in my monitoring tool:



    FreeBSD daemon

    FreeBSD requires one more step - make pmacctd starting during the loading process. For this I've put the pmacctd shell script in the /etc/rc.d:

    #!/bin/sh

    # PROVIDE: pmacctd
    # REQUIRE: DAEMON
    # BEFORE:  LOGIN
    # KEYWORD: nojail shutdown

    . /etc/rc.subr

    name="pmacctd"
    rcvar=`set_rcvar`
    command="/usr/local/sbin/${name}"
    command_args="-f /usr/local/etc/pmacctd.conf"
    pidfile="/var/run/${name}.pid"

    load_rc_config $name
    run_rc_command "$1"

    And set up /etc/rc.conf to start the daemon during the system loading:

    pmacctd_enable="YES"


    That's all. Hope this help you in monitoring and auditing your systems.

    Comments

    Popular posts from this blog

    DIY: Business cards in LaTeX

    Python vs JS vs PHP for embedded systems

    Web application framework comparison by memory consumption