Benchmarking Terminology


Rating is the primary metric used to report performance results of the benchmarks. It is defined as the number of benchmarks that can be run on a given machine (in sequence) in a 24 hour period. It is computed by dividing the number of seconds in a day (86400 seconds) by the number of seconds required to run the benchmark. A higher rating means faster performance.


Speedup is the ratio of wall-clock time required to complete a given calculation using a single processor to that of the equivalent calculation performed on a concurrent machine. Its value ranges from 1 to the number of processors used for the parallel run. When the speedup is equal to the number of processors used, the speedup is called perfect or linear. Sometimes the speedup will exceed the number of processors. This is referred to as super-linear speedup, and is often caused by the availability and use of larger amounts of "fast" memory (e.g. cache or local memory) compared to the single processor run.


Efficiency is the speedup normalized by the number of processors used, presented as a percentage. It indicates the overall utilization of the CPUs during a parallel calculation. An efficiency of 100% indicates that each CPU is completely occupied with computation during the course of a run, and corresponds to a linear speedup. An efficiency of 60% indicates that each CPU is performing useful computation only 60% of the time. The remaining time is spent waiting for other functions, such as parallel communication or work on other processors, to complete.