550 PERFORMANCE

There are three ways to estimate the instruction utilization U and, hence, obtain T - each providing increasingly better answers. The first defines either a typical or average instruction. The second uses standard benchmarks to characterize a machine's performance precisely. In this way, machines can be compared with an absolute measure. Finally, since the actual use has not been characterized in terms of the standard benchmark (and may even be difficult to characterize in terms of it), a specific unique benchmark may be necessary. Such a characterization is quite possibly needed for real-time and transaction processing where computer selection and installation is predicated on the job.

The simplest, single parameter of performance is the instruction time for some simple operation (e.g., add). These were used in the first two computer generations when high level languages were less used. Such a metric is an approximation to the average instruction time and assumes that all machines have about the same ISP and thus there is little difference among instructions, or that a specific data-type is used more heavily than another, or that a typical add time will be given (e.g., the operand is in a random location in primary memory call rather than being cached or in a fast register).

Although it is possible to take the average instruction time by executing one of every possible instruction, since the instruction use depends so much on the data interpreted, this average is relatively meaningless. A better measure is to keep statistics about the use of all programs and to give the average instruction time based on use on all programs. Again, such a measure, while useful for comparing two machines' implementations of models of the same architecture, is relatively useless for particular practices.

Many years ago, there were attempts to make better characterizations by weighting instruction use (i.e., forming a typical U) as to what each one did (e.g., floating point versus indexing and character handling) to give a better performance measure. Instruction mixes were developed that began to better evaluate performance. These mixes, from Bell and Newell [1971:50], are given in Table 5.

The Gibson mix, best known, is still used even today. It has a decidedly commercial flavor and quite possibly reflects the proportion of machines executing commercial, as opposed to scientific, mixes with character operations, switching, and control, where proportionally more integer and floating-point data-types are used. Such mixes are still better approximations than a single instruction average, because use enters in. Note that if the data-type operation is not present in the machine, the programmed subroutine time must be given - typically a factor of 10-20 times greater than for built-in operations.

The best estimate of real use comes from carefully designed standard benchmarks that are understood and that are used by other machines. Several organiza-