Welcome to the new Linux Foundation Forum!

Help with colletcl tool

peter79peter79 Posts: 6
in System Management

Hey Guys,

Im fairly new to the world of linux so I need a bit of help. Im currently using a tool called collectl (http://collectl.sourceforge.net/) to gather sone I/O information about a Red Hat server. Ive figured how to capture the data but my problem is I really need to get the results into a CSV file for Excel.

Has anyone on this forum any experience using collectl and if so can the data be exported to a CSV file?

Thanks.

Comments

  • marksegermarkseger Posts: 3
    I'm actually the author of collectl! Personally I don't like csv formatted data because I find it a pain to parse and most utilities that read CSV also read space separated data so that's the default when you with with -P. If if you include -f it ill write to a file in that directory. BUT if you really want csv data ;), you can use "--sep ','" to tell it to use a comma as the separator instead. Does that answer your question or are you looking for more details?
    -mark
  • peter79peter79 Posts: 6
    Mark,

    Thanks for your reply. Where in the command would you use --sep? For example if I ws running

    colletl -sd -c10 --P -p/temp

    Also there is a chance I might have to gather the data for up to a week, how big do you think the csv file would get?

    Thanks in advance.

    By the way its a great tool!
  • marksegermarkseger Posts: 3
    If you really only care about disk data and nothing more - it's not that expensive to grab more - the question is do you just want summaries of totals across ALL disks or would you prefer data on individual ones? -sd will get you the summary, -sD the details and -sdD both. Anyhow assuming you really want summary as you've indicated above AND if you want it for 5 days, run collectl as a daemon. just edit the line DaemonCommands in /etc/collectl.conf from:

    DaemonCommands = -f /var/log/collectl -r00:00,7 -m -F60 -s+YZ

    to

    DaemonCommands = -f /var/log/collectl -r00:00,7 -sd -P --sep ','

    that will write the csv data to a file in /var/log/collectl, creating a new one each midnight. To start it just do /etc/init.d/collectl start and after 5 days /etc/init.d/collectl stop. The default collection interval is 10 seconds.

    The output, which you can generate interactively to look at would be from the command:

    "collectl -sd -P --sep ','" and look like the following, noting the interactive default interval is 1 sec:

    [[email protected] collectl]# collectl -sd -P --sep ','
    waiting for 1 second sample...
    #Date,Time,[DSK]ReadTot,[DSK]WriteTot,[DSK]OpsTot,[DSK]ReadKBTot,[DSK]WriteKBTot,[DSK]KbTot,[DSK]ReadMrgTot,[DSK]WriteMrgTot,[DSK]MrgTot
    20110111,08:04:36,0,0,0,0,0,0,0,0,0
    20110111,08:04:37,0,2,2,0,264,264,0,64,64
    20110111,08:04:38,0,0,0,0,0,0,0,0,0

    this should be enough to get you going...
    -mark
  • peter79peter79 Posts: 6
    Mark,

    Thanks again for yout response just to give some background info. The data I need to captutre is wtites per second on the disk subsystem. So I will be using -sd. I would like to capture the at least 3 days worth of data mayne up to 5.

    I would like to have the data captured as a CSV file so it can be viewed in excel.

    I tried running the command below because I want to see what the results look like in Excel. However it did not run is the syntax correct?

    collectl -sd -P -p/temp --sep','
  • marksegermarkseger Posts: 3
    why didn't you say excel? ;)
    excel can read collectl's space-separated format with no problems, you just need to open as you would any file and tell excel the fields are space-separated when it asks.

    Anyhow if you want to capture disk writes/sec, -sd will show the total writes on all your disks. If all but one are idle, this should work fine, but if you have multiple disks active, -sD will get you exact numbers for each.

    I don't know where -p in your command above is coming from. You need -f/temp if you want to write the output to the directory /temp. By default, if you have the compression library installed, collectl will write its output in a compressed 'gz' file, which you can always manually uncompress. Alternatively you can add the switch -oz which will tell it not to compress.

    So, the command I'd consider using is:

    collectl -sD -P -f/temp -oZ

    then try importing the /temp/*dsk into excel without around with --sep. It's really not hard...
    of you use -sd you'll have a file named *.tab instead. if you use -sdD you'll get both!

    -mark
  • peter79peter79 Posts: 6
    Mark,

    I really appreciate all yout help on this. I have tried opening the tab output files in Excel and while it does open the file it just was a bit of a mess (I have attached the excel file tp this post).

    Is it possbile to to format the output so that only the writes pe second is recorded? Also I tried configuring collectl to run as daemon but when I attempted to start the service it failed. Please note that I'm currently testing this out on a ESX server (ESX is built on a red hat kernel so it shouldnt be a problem.
Sign In or Register to comment.