ExaHDF5 - Presentation - March 2017 CS PI meeting

Paper Presentation

Modern computational science faces a number of challenges on the path to exascale. Simulation codes are producing massive amounts of data that need to be stored, visualized, and analyzed on increasingly complex hardware in order to gain scientific insights. Our work in this project consists of three thrust areas that address these contemporary challenges.

  • Developing high performance I/O middleware that makes effective use of computational platforms, researching a number of optimization strategies and deploying them through the HDF5 software
  • Improving the productivity of application developers by hiding the complexity of parallel I/O via new auto-tuning and transparent data re-organization techniques, and by extending our existing work in easy-to-use, high-level APIs that expose scientific data models
  • Facilitating scientific analysis for users by extending query-based techniques, developing novel in situ analysis capabilities, and making sure that visualization tools use best practices when reading HDF5 data

Our research is driven by close collaborations with a broad range of DOE science codes; we will ensure that new capabilities are responsive to scientists’ emerging needs and are deployed in production HPC environments. Our approach includes a clear path for release and maintenance of software, enabling the broader science community to benefit from our project.