Welcome to the Linux Foundation Forum!
Help with colletcl tool
peter79
Posts: 6
Hey Guys,
Im fairly new to the world of linux so I need a bit of help. Im currently using a tool called collectl (http://collectl.sourceforge.net/) to gather sone I/O information about a Red Hat server. Ive figured how to capture the data but my problem is I really need to get the results into a CSV file for Excel.
Has anyone on this forum any experience using collectl and if so can the data be exported to a CSV file?
Thanks.
0
Comments
-
I'm actually the author of collectl! Personally I don't like csv formatted data because I find it a pain to parse and most utilities that read CSV also read space separated data so that's the default when you with with -P. If if you include -f it ill write to a file in that directory. BUT if you really want csv data
, you can use "--sep ','" to tell it to use a comma as the separator instead. Does that answer your question or are you looking for more details?
-mark0 -
Mark,
Thanks for your reply. Where in the command would you use --sep? For example if I ws running
colletl -sd -c10 --P -p/temp
Also there is a chance I might have to gather the data for up to a week, how big do you think the csv file would get?
Thanks in advance.
By the way its a great tool!0 -
If you really only care about disk data and nothing more - it's not that expensive to grab more - the question is do you just want summaries of totals across ALL disks or would you prefer data on individual ones? -sd will get you the summary, -sD the details and -sdD both. Anyhow assuming you really want summary as you've indicated above AND if you want it for 5 days, run collectl as a daemon. just edit the line DaemonCommands in /etc/collectl.conf from:
DaemonCommands = -f /var/log/collectl -r00:00,7 -m -F60 -s+YZ
to
DaemonCommands = -f /var/log/collectl -r00:00,7 -sd -P --sep ','
that will write the csv data to a file in /var/log/collectl, creating a new one each midnight. To start it just do /etc/init.d/collectl start and after 5 days /etc/init.d/collectl stop. The default collection interval is 10 seconds.
The output, which you can generate interactively to look at would be from the command:
"collectl -sd -P --sep ','" and look like the following, noting the interactive default interval is 1 sec:
[root@poker collectl]# collectl -sd -P --sep ','
waiting for 1 second sample...
#Date,Time,[DSK]ReadTot,[DSK]WriteTot,[DSK]OpsTot,[DSK]ReadKBTot,[DSK]WriteKBTot,[DSK]KbTot,[DSK]ReadMrgTot,[DSK]WriteMrgTot,[DSK]MrgTot
20110111,08:04:36,0,0,0,0,0,0,0,0,0
20110111,08:04:37,0,2,2,0,264,264,0,64,64
20110111,08:04:38,0,0,0,0,0,0,0,0,0
this should be enough to get you going...
-mark0 -
Mark,
Thanks again for yout response just to give some background info. The data I need to captutre is wtites per second on the disk subsystem. So I will be using -sd. I would like to capture the at least 3 days worth of data mayne up to 5.
I would like to have the data captured as a CSV file so it can be viewed in excel.
I tried running the command below because I want to see what the results look like in Excel. However it did not run is the syntax correct?
collectl -sd -P -p/temp --sep','0 -
why didn't you say excel?

excel can read collectl's space-separated format with no problems, you just need to open as you would any file and tell excel the fields are space-separated when it asks.
Anyhow if you want to capture disk writes/sec, -sd will show the total writes on all your disks. If all but one are idle, this should work fine, but if you have multiple disks active, -sD will get you exact numbers for each.
I don't know where -p in your command above is coming from. You need -f/temp if you want to write the output to the directory /temp. By default, if you have the compression library installed, collectl will write its output in a compressed 'gz' file, which you can always manually uncompress. Alternatively you can add the switch -oz which will tell it not to compress.
So, the command I'd consider using is:
collectl -sD -P -f/temp -oZ
then try importing the /temp/*dsk into excel without around with --sep. It's really not hard...
of you use -sd you'll have a file named *.tab instead. if you use -sdD you'll get both!
-mark0 -
Mark,
I really appreciate all yout help on this. I have tried opening the tab output files in Excel and while it does open the file it just was a bit of a mess (I have attached the excel file tp this post).
Is it possbile to to format the output so that only the writes pe second is recorded? Also I tried configuring collectl to run as daemon but when I attempted to start the service it failed. Please note that I'm currently testing this out on a ESX server (ESX is built on a red hat kernel so it shouldnt be a problem.0
Categories
- All Categories
- 164 LFX Mentorship
- 164 LFX Mentorship: Linux Kernel
- 724 Linux Foundation IT Professional Programs
- 368 Cloud Engineer IT Professional Program
- 161 Advanced Cloud Engineer IT Professional Program
- 69 DevOps IT Professional Program - Discontinued
- 1 DevOps & GitOps IT Professional Program
- 94 Cloud Native Developer IT Professional Program
- 33 Express Training Courses & Microlearning
- 31 Express Courses - Discussion Forum
- 2 Microlearning - Discussion Forum
- 7.4K Training Courses
- 25 LFC110 Class Forum - Discontinued
- 15 LFC131 Class Forum - DISCONTINUED
- 54 LFD102 Class Forum
- 254 LFD103 Class Forum
- 1 LFD103-JP クラス フォーラム
- 17 LFD110 Class Forum
- LFD114 Class Forum
- 54 LFD121 Class Forum
- 3 LFD123 Class Forum
- 2 LFD125 Class Forum
- 3 LFD133 Class Forum
- 4 LFD134 Class Forum
- 4 LFD137 Class Forum
- 1 LFD140 Class Forum
- 66 LFD201 Class Forum
- 7 LFD210 Class Forum
- 3 LFD210-CN Class Forum
- 1 LFD213 Class Forum - Discontinued
- 1 LFD221 Class Forum
- 127 LFD232 Class Forum - Discontinued
- 2 LFD233 Class Forum - Discontinued
- 4 LFD237 Class Forum
- 24 LFD254 Class Forum
- 759 LFD259 Class Forum
- 110 LFD272 Class Forum - Discontinued
- 2 LFD272-JP クラス フォーラム - Discontinued
- 22 LFD273 Class Forum
- 657 LFS101 Class Forum
- 4 LFS111 Class Forum - Discontinued
- 2 LFS112 Class Forum
- LFS114 Class Forum
- 4 LFS116 Class Forum
- 6 LFS118 Class Forum
- 2 LFS120 Class Forum
- 1 LFS140 Class Forum
- 11 LFS142 Class Forum
- 9 LFS144 Class Forum
- 5 LFS145 Class Forum
- 6 LFS146 Class Forum
- 7 LFS147 Class Forum
- 26 LFS148 Class Forum
- 22 LFS151 Class Forum - Discontinued
- 4 LFS157 Class Forum
- 167 LFS158 Class Forum
- 1 LFS158-JP クラス フォーラム
- 17 LFS162 Class Forum
- 1 LFS166 Class Forum - Discontinued
- 8 LFS167 Class Forum
- 4 LFS170 Class Forum
- 1 LFS171 Class Forum - Discontinued
- 3 LFS178 Class Forum - Discontinued
- 3 LFS180 Class Forum
- 2 LFS182 Class Forum
- 6 LFS183 Class Forum
- 2 LFS184 Class Forum
- 42 LFS200 Class Forum
- 736 LFS201 Class Forum - Discontinued
- 2 LFS201-JP クラス フォーラム - Discontinued
- 23 LFS203 Class Forum
- 151 LFS207 Class Forum
- 2 LFS207-DE-Klassenforum
- 3 LFS207-JP クラス フォーラム
- 301 LFS211 Class Forum - Discontinued
- 55 LFS216 Class Forum - Discontinued
- 60 LFS241 Class Forum
- 51 LFS242 Class Forum
- 41 LFS243 Class Forum
- 18 LFS244 Class Forum
- 8 LFS245 Class Forum
- 1 LFS246 Class Forum
- 1 LFS248 Class Forum
- 164 LFS250 Class Forum
- 3 LFS250-JP クラス フォーラム
- 2 LFS251 Class Forum - Discontinued
- 164 LFS253 Class Forum
- 1 LFS254 Class Forum - Discontinued
- 3 LFS255 Class Forum
- 18 LFS256 Class Forum
- 2 LFS257 Class Forum
- 1.4K LFS258 Class Forum
- 12 LFS258-JP クラス フォーラム
- 149 LFS260 Class Forum
- 164 LFS261 Class Forum
- 45 LFS262 Class Forum
- 82 LFS263 Class Forum - Discontinued
- 15 LFS264 Class Forum - Discontinued
- 11 LFS266 Class Forum - Discontinued
- 25 LFS267 Class Forum
- 27 LFS268 Class Forum
- 38 LFS269 Class Forum
- 10 LFS270 Class Forum
- 202 LFS272 Class Forum - Discontinued
- 2 LFS272-JP クラス フォーラム - Discontinued
- 1 LFS274 Class Forum - Discontinued
- 4 LFS281 Class Forum - Discontinued
- 32 LFW111 Class Forum
- 265 LFW211 Class Forum - Discontinued
- 190 LFW212 Class Forum - Discontinued
- 18 SKF100 Class Forum
- 2 SKF200 Class Forum
- 3 SKF201 Class Forum
- 789 Hardware
- 202 Drivers
- 68 I/O Devices
- 37 Monitors
- 95 Multimedia
- 173 Networking
- 89 Printers & Scanners
- 86 Storage
- 764 Linux Distributions
- 81 Debian
- 67 Fedora
- 20 Linux Mint
- 13 Mageia
- 23 openSUSE
- 150 Red Hat Enterprise
- 31 Slackware
- 13 SUSE Enterprise
- 355 Ubuntu
- 459 Linux System Administration
- 31 Cloud Computing
- 72 Command Line/Scripting
- Github systems admin projects
- 94 Linux Security
- 78 Network Management
- 100 System Management
- 46 Web Management
- 67 Mobile Computing
- 18 Android
- 38 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 381 Off Topic
- 117 Introductions
- 174 Small Talk
- 29 Study Material
- 731 Programming and Development
- 309 Kernel Development
- 404 Software Development
- 893 Software
- 286 Applications
- 182 Command Line
- 5 Compiling/Installing
- 68 Games
- 316 Installation
- 62 All In Program
- 62 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)