Implementing Parallel I/O: MPI-IO and HPSS
John May
Lawrence Livermore National Laboratory
Abstract
Moving data from memory to long-term storage has often been an
afterthought in the design of large scientific applications. However,
compute power is growing faster I/O rates. More and more, I/O can
dominate the running time of a large scientific application, so it has
become important for developers to focus on I/O issues. The
Department of Energy's Accelerated Strategic Computing Initiative
(ASCI) has serveral projects to address high performance I/O problems
for its scientific simulation codes. This talk will briefly outline
ASCI's I/O strategies, and then it will examine in detail the one
component in this strategy, an implementation of the MPI-IO parallel
I/O standard for the HPSS archival storage system. Both MPI-IO and
HPSS are complex systems, and combining them presented a number of
interesting problems in parallel software design. We will look at how
some of these problems were solved and present I/O performance data
for the current system.