Quick reference of this noble and old school command.
Very useful for troubleshooting and server performance.
Checking CPU Activity
By default, sar command will report the CPU activity of the server. The option -u
can be used to get the CPU utilization report.
To get the CPU activity report in every 2 seconds for 3 times:
$ sar -u 2 3
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 09/06/2020 _x86_64_ (2 CPU)
22:26:54 CPU %user %nice %system %iowait %steal %idle
22:26:56 all 4,03 0,00 0,50 0,00 0,00 95,47
22:26:58 all 8,08 0,00 0,51 0,00 0,00 91,41
22:27:00 all 12,50 0,00 1,00 0,00 0,00 86,50
Average: all 8,21 0,00 0,67 0,00 0,00 91,11
CPU usage for each processor
To find the CPU activity on all processors separately, you need to use the -P option.
$ sar -P ALL 1 1
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
05:38:18 CPU %user %nice %system %iowait %steal %idle
05:38:19 all 3,03 0,00 0,00 0,00 0,00 96,97
05:38:19 0 3,96 0,00 0,99 0,00 0,00 95,05
05:38:19 1 3,00 0,00 0,00 0,00 0,00 97,00
Average: CPU %user %nice %system %iowait %steal %idle
Average: all 3,03 0,00 0,00 0,00 0,00 96,97
Average: 0 3,96 0,00 0,99 0,00 0,00 95,05
Average: 1 3,00 0,00 0,00 0,00 0,00 97,00
Check Memory usage
To find the memory usage (used and free memory of the server) over time using the -r
switch.
$ sar -r 1 3
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
05:41:04 kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty
05:41:05 855300 3025220 77,96 0 1495276 2426864 30,55 1265008 1158172 100
05:41:06 854800 3025720 77,97 0 1495276 2426864 30,55 1265132 1158172 100
05:41:07 855032 3025488 77,97 0 1495276 2426864 30,55 1265012 1158172 100
Average: 855044 3025476 77,97 0 1495276 2426864 30,55 1265051 1158172 100
Check Swap Activity
Check the swap usage of the machine using the -W
option:
$ sar -W 1 3
Linux 2.6.18-274.18.1.el5 (myserver) 09/06/2012
03:31:12 PM pswpin/s pswpout/s
03:31:13 PM 16.16 0.00
03:31:14 PM 16.00 0.00
03:31:15 PM 15.84 0.00
Average: 16.00 0.00
Find load averages over time
You can find the load averages overtime using the -q
option:
$ sar -q 1 3
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
06:16:13 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked
06:16:14 0 329 0,00 0,04 0,09 0
06:16:15 0 329 0,00 0,04 0,09 0
06:16:16 0 329 0,00 0,04 0,09 0
Average: 0 329 0,00 0,04 0,09 0
Statistics for currently mounted filesystems
$ sar -F 2 3
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
06:30:18 MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
06:30:20 30410 5532 15,39 15,39 18245113 166407 0,90 /dev/mapper/centos-root
06:30:20 796 218 21,52 21,52 523947 341 0,07 /dev/sda1
06:30:20 MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
06:30:22 30410 5532 15,39 15,39 18245113 166407 0,90 /dev/mapper/centos-root
06:30:22 796 218 21,52 21,52 523947 341 0,07 /dev/sda1
06:30:22 MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
06:30:24 30410 5532 15,39 15,39 18245113 166407 0,90 /dev/mapper/centos-root
06:30:24 796 218 21,52 21,52 523947 341 0,07 /dev/sda1
Summary: MBfsfree MBfsused %fsused %ufsused Ifree Iused %Iused FILESYSTEM
Summary 30410 5532 15,39 15,39 18245113 166407 0,90 /dev/mapper/centos-root
Summary 796 218 21,52 21,52 523947 341 0,07 /dev/sda1
Details of inode, kernel tables and file tables
$ sar -v 2 3
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
06:57:23 dentunusd file-nr inode-nr pty-nr
06:57:25 160833 2400 132081 3
06:57:27 160833 2400 132081 3
06:57:29 160833 2400 132081 3
Average: 160833 2400 132081 3
Network statistics
$ sar -n DEV
Linux 3.10.0-1127.10.1.el7.x86_64 (localhost.localdomain) 10/06/2020 _x86_64_ (2 CPU)
19:20:23 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
19:20:24 ens192 11,00 4,00 1,80 2,75 0,00 0,00 0,00
19:20:24 lo 79,00 79,00 43,62 43,62 0,00 0,00 0,00
Average: IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s
Average: ens192 11,00 4,00 1,80 2,75 0,00 0,00 0,00
Average: lo 79,00 79,00 43,62 43,62 0,00 0,00
Which interrupt number could be causing a CPU bottleneck?
$ sar -I XALL 2 10
02:07:10 AM INTR intr/s
02:07:12 AM 0 992.57
02:07:12 AM 1 0.00
02:07:12 AM 2 0.00
02:07:12 AM 3 0.00
02:07:12 AM 4 0.00
02:07:12 AM 5 0.00
02:07:12 AM 6 0.00
02:07:12 AM 7 0.00
02:07:12 AM 8 0.00
02:07:12 AM 9 350.50
‘/proc/interrupts’ file will also provide helpful information. Interrupt halts CPU processing so that I/O or other operations can occur. Processing resumes after the specific operation takes place. It is very important that each device installed in machine is provided with an interrupt setting that does not conflict with the settings used by the hardware and other devices.
$ sudo cat /proc/interrupts
CPU0 CPU1
0: 48 0 IO-APIC-edge timer
1: 54 0 IO-APIC-edge i8042
8: 1 0 IO-APIC-edge rtc0
9: 0 0 IO-APIC-fasteoi acpi
12: 35 116 IO-APIC-edge i8042
14: 0 0 IO-APIC-edge ata_piix
15: 0 0 IO-APIC-edge ata_piix
16: 118 2513 IO-APIC-fasteoi vmwgfx
24: 0 0 PCI-MSI-edge PCIe PME, pciehp
25: 0 0 PCI-MSI-edge PCIe PME, pciehp
26: 0 0 PCI-MSI-edge PCIe PME, pciehp
27: 0 0 PCI-MSI-edge PCIe PME, pciehp
NMI: 0 0 Non-maskable interrupts
LOC: 35392807 14792833 Local timer interrupts
SPU: 0 0 Spurious interrupts
PMI: 0 0 Performance monitoring interrupts
IWI: 2677624 215297 IRQ work interrupts