The iostat, netstat, and vmstat commands have functional characteristics that make them useful for continuous monitoring of system performance:
The following example shows samples of the periodic reports produced by these programs.
$ iostat 5 2
tty: tin tout cpu: % user % sys % idle % iowait
0.0 0.0 0.0 0.2 99.6 0.1
Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk0 0.1 0.3 0.0 18129 56842
cd0 0.0 0.0 0.0 0 0
tty: tin tout cpu: % user % sys % idle % iowait
0.0 0.0 23.1 9.0 65.9 2.0
Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk0 2.4 6.4 1.6 0 32
cd0 0.0 0.0 0.0 0 0
$ vmstat 5 2
procs memory page faults cpu
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
0 0 2610 1128 0 0 0 0 0 0 112 1 19 0 0 99 0
0 0 2505 1247 0 0 0 0 0 0 125 1056 37 22 9 67 2
$ netstat -I tr0 5
input (tr0) output input (Total) output
packets errs packets errs colls packets errs packets errs colls
532099 1664 985 0 0 532111 1664 997 0 0
45 0 6 0 0 45 0 6 0 0
44 1 5 0 0 44 1 5 0 0
Remember that the first report from each of these programs is for cumulative activity since the last system boot. The second report shows activity for the first 5-second interval. Even this small sample shows that the activity in the first interval was significantly higher than the average.
These commands are the basic foundation on which a performance-monitoring mechanism can be constructed. Shell scripts can be written to perform data reduction on *stat command output and warn of performance problems or record data on the status of the system when a problem is occurring. For example, a shell script could test the CPU idle percentage for zero and execute another shell script when that CPU-saturated condition occurred. A script such as:
$ ps -ef | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
would record the 15 active processes that had consumed the most CPU time recently (other than the processes owned by the user of the script).
Depending on the required level of sophistication, creating such a family of shell scripts can be a substantial project. Fortunately, there are packages available that require less development and setup and have considerably more function than the typical installation would want to implement locally.