Server Performance Data Collection – Collectl
Server Performance Data Collection
1. Disk
2. CPU
3. Network … Etc
There are a number of times in which you find yourself needing performance data. These can include benchmarking, monitoring a system’s general heath or trying to determine what your system was doing at some time in the past. Sometimes you just want to know what the system is doing right now. Depending on what you’re doing, you often end up using different tools, each designed to for that specific situation.
Unlike most monitoring tools that either focus on a small set of statistics, format their output in only one way, run either interatively or as a daemon but not both, collectl tries to do it all. You can choose to monitor any of a broad set of subsystems which currently include buddyinfo, cpu, disk, inodes, infiniband, lustre, memory, network, nfs, processes, quadrics, slabs, sockets and tcp.
By default it shows cpu, network and disk stats in brief format. The key point of this format is all output appears on a single line making it much easier to spot spikes or other anomalies in the output:
[oracle@Mudhalvan] collectl
while writing to an NFS mounted filesystem, collectl displays interrupts, memory usage and nfs activity with timestamps. Keep in mind that you can mix and match any data and in the case of brief format you simply need to have a window wide enough to accommodate your output.
[oracle@Mudhalvan] collectl -sjmf -oT
You can also display the same information in verbose format, in which case you get a single line for each type of data at the expense of more screen real estate, as can be seen in this example of network data during NFS writes. Note how you can actually see the network traffic stall while waiting for the server to physically write the data.
[oracle@Mudhalvan] collectl -sn –verbose -oT
[oracle@Mudhalvan] collectl -sJ -oTm
Output can also be saved in a rolling set of logs for later playback or displayed interactively in a variety of formats. If all that isn’t enough there are additional mechanisms for supplying data to external tools by generating output as s-expressions, a format of choice for some tools such as supermon or in another format called list-expressions. This output can be written to a file or sent over a socket. You can even create files in space-separated format for plotting with external packages like gnuplot or colplot, part of the collectl utilities project, which provides a web-based interface to gnuplot.
Collectl runs on all linux distros (it’s included as part of Fedora) and only requires perl. If the perl Time::Hires module is installed, you will be able to use fractional intervals and display timestamps in msecs. If the Compress::Zlib module is installed the recorded data will be compressed and therefore use on average 90% less storage when recording to a file. Also note that the above links are not for RPMs. If you’d rather work with RPMs there are far too many versions out there to link to and so I’m sorry to say you’re on your own.
Did you know there was an inconsistency in the way Linux reported disk metrics that wasn’t even noticed/fixed until the 2.6-14 kernel was released? Collectl did. Or how about the fact that network stats may not accurately reported by most network monitoring tools at one second intervals? See this page for a description of the problem and how you can get more accurate stats by simply running collectl at a sub-second interval.
