FAQ
What is the objective
of TopCrunch?
The objective of TopCrunch is to track the aggregate performance
trends of high performance computer systems and engineering
software. Instead of using a synthetic benchmark, actual engineering
software applications are used with real data and are run
on high performance computer systems.
Who runs TopCrunch?
Prof. David Benson at UCSD runs TopCrunch, with support from
his students, post-docs, research engineers, and web site
support from the Jacobs School of Engineering.
How do I contact
TopCrunch?
For web site related problems, click on “webmaster”
at the bottom of the page. For issues associated with the
benchmarks themselves, contact dbenson@ucsd.edu.
Who funds TopCrunch?
TopCrunch is supported by DARPA
HPCS through a subcontract from the USC Information
Sciences Institute.
Why do you use production
codes for the benchmarks?
The objective is to track the aggregate computing performance
available to scientists and engineers. Synthetic benchmarks
are usually more highly optimized than production codes because
they are much smaller and simpler codes, and therefore don’t
reflect real world performance. Research codes are generally
not available to the broad range of people we would like to
see perform the benchmarks, lack the breadth we are seeking,
and the limited resources of the developers means that they
can’t provide the support required for non-specialists
to use them.
How are the benchmark
codes chosen?
The benchmarks are chosen to reflect the types of calculations
performed in the mechanical and aerospace communities. Therefore,
codes associated with structural dynamics (LS-DYNA), fluid
flow (CTH), and materials science (SPaSM) have been chosen.
These codes have different challenges to address in terms
of domain decomposition, message passing, load balancing,
and dynamic memory allocation that makes the comparison of
their relative scaling interesting. Additional codes may be
added in the future.
How are the benchmarks
problems chosen?
The benchmark problems are chosen to reflect current engineering
practice in the real world, and to have a structure that allows
them to be scaled up as computer performance grows. The problems
are not intended to be optimal analyses, i.e., the fastest
possible choice of options to achieve a particular solution,
because engineers rarely have time to optimize their analyses
in real life. For example, the accuracy of the stress distribution
in a structural element increases with the number of Gaussian
quadrature points, but at the expense of speed. For a given
level of accuracy, there is, therefore, a choice that maximizes
speed. It is, however, common engineering practice to use
more points than absolutely necessary because an inaccurate
solution will require rerunning the analysis, which effectively
doubles its cost and more than doubles the wall clock time
to get an acceptable answer. The same general observation
holds true for many other analysis choices to be made.
How do I obtain
the benchmark codes?
The codes are available directly from their authors, and
are not available on this site. The contact information is
given on the code description pages.
How do I run the
benchmark problems?
The general command lines required for executing a program
are given on the code description pages, along with pointers
to additional documentation for the codes.
Can I change the
problem data?
No! Aside from the number of processors, no changes are permitted.
As discussed, the problems are chosen to represent current
industrial practice, and are not optimized for performance.
Our goal is to track software and hardware performance, not
analyst performance!
How do I obtain
technical support for running the problems?
Technical support for running the codes is available through
the software providers. If a particular problem downloaded
from this site doesn’t run, contact David
Benson.
How do I submit
the benchmark results?
Mail the ASCII output files from the analyses, which contain
the timings, to Administrator.
The entire file must be submitted, not just the timings.
How does TopCrunch
prevent people from cheating?
We can’t. We try to avoid obvious cheating by requiring
the ASCII output files, but we recognize that a determined
cheater can readily edit the output. We reserve the right
to withdraw or not post results that appear suspicious, and
to require benchmarkers to provide additional proof such as
the binary restart and plot files. But we explicitly don’t
warranty the accuracy of the benchmark results. If you feel
that results posted here are incorrect, please contact the
Administrator.
Can I run the benchmark
problems with my own code?
As one objective of the site is to track software advances,
competition between codes is welcomed. However, in the interests
of fairness, if you wish to have your results posted, you
must
- Run a model that is identical in all respects to the
benchmark model. This includes, but is not limited to, element
formulation, the number of integration points, time step
scaling, material models, and domain decomposition (including
the specification of contact surfaces).
- Be willing to supply us with access to the results for
verification and provide a version of the code to a platform
of our choice under our control to run your data file and
program independently at no cost to us.
Should I buy my
computer based on the posted results?
We recognize that these results may be useful to some users
in making basic decisions about their computing requirements.
However, be aware of the following caveats:
- The performance of a computer is the result of many aspects
of the system, including the chip, the available memory,
the connection between the processors (e.g., Gigabit Ethernet
vs. Myrinet), the version of the operating system, the version
of MPI, and the particular version of the benchmark code.
- The performance is also subject to the particular circumstances
under which it was performed, e.g., a benchmark run in a
single user mode will usually be faster than one that was
run with other users on the system.
- We explicitly don’t warranty the accuracy of the
numbers posted on this site.
|
