14.arch/AJS.worley
.ls 2
.na
.LP
Computer System Benchmarking
.sp
Steven Von Worley

(Professor A. J. Smith)

(NASA) NCC-2-550

Within the past decade, supercomputers have evolved into diverse
configurations of multiple processors. Measuring the performance of
these computers with standard benchmarks is difficult; the results are
often difficult to use for comparisons between architectural classes,
and concern arises as to how well the benchmarks mirror the workload in
question.

Recently, researchers have begun to study the prediction of the
performance of programs running on parallel computers. Performance
prediction is interesting because it is decoupled from the machine and
workload; with a good system, we may accurately predict the runtime of
any desired program solely with previous measurements of the machine
and/or estimates of system parameters.

The initial goal of this research is to create a portable test suite
that will measure the characteristics of a given supercomputer and
compiler. We have picked \s-2FORTRAN\s+2 90 as the base language, since it is
relatively stable, a subset of newer parallel \s-2FORTRAN\s+2 variants, and
supported by most vendors. The test suite will 1) measure and
parameterize the runtimes of \s-2FORTRAN\s+2 90 sequential, array, reduction,
and communication operations, and 2) measure the ability of the
compiler to produce code that efficiently uses the underlying
architecture. We will test for a wide range of optimizations, from
scalar expansion to automatic data layout. Several limited test suites
currently exist, and we will attempt to incorporate these wherever
possible.

After the test suite is completed, we will attempt to learn the
predictive abilities of our measurements by attempting to estimate the
runtime of several interesting kernels.

We will produce this research in the context of a completely automatic
performance prediction system, which will use our measurements, in
combination with trace data from an instrumented copy of the program in
question, to predict its runtime on a given supercomputer.