Archiving and Storing Performance Data

Let's continue with the example we started in the previous section. We collected data on 25 servers using a one-minute time interval. Most server bottlenecks can be found at this resolution. After all, if the server is slow only for a minute or less, it's not that inconvenient for the server's users.

Once the day is over, or perhaps the week, we no longer need that much detail. Now it's time to relog the data to a Performance Monitor archive. An archive is just a log file with data from multiple days that we'd like to keep around for a while. Internally it has the same format as the original log file.

Suppose we use Performance Monitor to open the first daily log file. The sensible thing to do here is to set the time window on a couple of busy periods of the day, such as mid-morning and mid-afternoon. (Alternatively, you might want to chart a key value such as System: Total Interrupts/sec to find where to set the time window.) If you are collecting from multiple servers, it is usually better to collect the data from them all at the same time of day.

After you've set the time window, open the log settings file with which you created the original log (here's where you're glad you saved that settings file). The settings file selects all the computers and objects you logged the first time. An alternative is to have a separate settings file for archiving the logged data, in which only a subset of the original objects are logged. For example, you probably don't need the Cache object in the archive, and you might not need all levels of the TCP/IP protocol if you logged them originally.

Having selected the objects to archive, use the Log Options dialog box to name the archive file. In this file you are building up a history of network activity during peak hours. Each day's peak activity is appended to the end of this file when you relog it, as shown in Figure 8.2.

Figure 8.2 Creating an archive log file from daily logs

Before you append today's data to the archive, change the time interval to something like 600 seconds or whatever suits you. This reduces the data to one-tenth its original size (assuming the original log was made with a one-minute time interval). If you also archive only half of your workday (such as two hours of peak activity in the morning and two more in the afternoon), the size of the data is reduced to 4.2 MB from your original 84 MB, assuming we keep all those original objects in the archive. This is not an outrageous amount of data for a daily record of 25 systems. This is about 20 MB per work week, or one gigabyte for the year.

One gigabyte is not to be sneezed at, and Performance Monitor would be slow (to say the least) to process such a large file. So once a month we should engage in some further data reduction. You will want to browse through the counters to determine the ones you think best indicate system usage growth. System: % Total Processor Time, System: Total Interrupts/sec, Total Bytes/sec of the protocol, % Disk Time, and % Disk Free Space suggest themselves immediately. Number of connections and files open might also be interesting. To monitor system memory, you'll want to watch Pages/sec, but also keep an eye on Page Faults/sec and Cache Faults/sec so you can determine whether your paging is due to disk file activity or too many large processes.

You then chart the counters you selected over the month's time. At this point, we have 4 hours per day times 6 observations per hour, or 24 data points per day. With 22 working days in a month, this gives 528 data points for the month for each chart line. Of course, on a Performance Monitor chart you will see only 100 points, but as they say Down Under, no worries.