Dr Lorna Smith, group manager at EPCC at the University of Edinburgh

It hardly seems any time since the European Commission announced a multi-million euro investment in the development of European exascale technologies. At that time the HPC community was only just beginning to consider and understand the challenges of developing these technologies, and the IESP published its software roadmap. Against this backdrop the EC announced that three flagship projects had been funded: CRESTA, DEEP and Mont-Blanc.

Fast-forward three years and the CRESTA project is now complete, having successfully delivered all of its objectives and outputs. As one of the first real projects funded to prepare and develop software, tools and applications for future exascale technologies, the experiences of the project are particularly important for the community. But as we continue to ensure CRESTA’s software is fully exploited, how has the European exascale landscape changed and how did CRESTA contribute?

Of the three flagship projects, CRESTA was the only purely software focused project, with DEEP and Mont-Blanc developing prototype hardware. The project was also unusual due to the scale of investment in a software project – 11.3M euros in total. This allowed the 13 project partners from around Europe to explore and deliver a range of innovative and disruptive products. If we have learnt anything from exascale software development in CRESTA, it is that it is rarely an incremental development process. Significant investment is required to allow developers to explore truly disruptive solutions. Many developers struggle to find direct funding for software development, with funding often directed at the science enabled by the software. Hence the EC’s investment in software has been critical and is perhaps a factor behind Europe’s recognition as an expert in HPC software.

Another unusual aspect to CRESTA has been the use of a co-design process. While co-design is widely used in the development of embedded systems, its use on HPC projects is less common. However the significant change anticipated in exascale hardware argues strongly that the traditional disjoint between application scientists and hardware and software developers is no longer viable. As one of the first projects to use a co-design process in practice, CRESTA has acted as an important demonstrator. From implementing an appropriate management methodology, through motivating and engaging partners, to measuring and assessing the impact of co-design on outcomes, the process has touched on all aspects of the project. The end result has clearly been a success, and some outcomes simply could not have been achieved without this approach. The use of co-design in HPC development is growing and a number of the European Exascale Projects are also now pursuing this approach.

As many of you involved in HPC software projects will be aware, the EC is looking to ensure projects such as these deliver economic growth, European research leadership and tackle major societal challenges, and rightly so. However articulating and quantifying the impact of software development projects can be challenging. Across the community many developers are looking for ways to achieve this and CRESTA was no exception. Ensuring the project outputs were available and utilised beyond the end of the project became key – in addition to the obvious open source, on-line repository, CRESTA established a series of pilot projects which were successful in ensuring the exploitation of the software. The applications all had important societal outputs, and a series of case studies showcased these.

So as ISC 2015 approaches, how has the European exascale landscape changed? Since the original three projects were funded in 2011, the EC has invested further funding in exascale technologies, resulting in three additional projects: EPiGRAM, NUMEXAS and EXA2CT. These six projects have collaborated closely, holding joint workshops, Birds-of-a-feather and exhibition booths. More recently, the EC has announced 21 new projects in this area. Of these, EPiGRAM, NextGenIO, INTERTWinE and ExaFLOW all incorporate outcomes from CRESTA. This investment is allowing European exascale technology investment to go from strength to strength.

ISC EVENT SCHEDULE The European Exascale Projects are holding a full day Workshop at ISC 2015 on Thursday July 16 titled “Is Europe Ready For Exascale? A Summary of Four Years of European Exascale Research”. Please come along and learn more – or drop by the EPCC booth (no 1203) or the European Exascale Projects Booth (no 634).

About the author :
Dr Lorna Smith is a group manager at EPCC at the University of Edinburgh. She is an experienced project manager, having managed both European, international and UK projects. She was involved in the establishment and management of the Exascale Technology Centre at EPCC, was the project manager for the FP7 funded Exascale Project CRESTA and is the local coordinator for the Exascale EPiGRAM project. She is the deputy directory of the ARCHER CSE service, providing training, user support and programme management for the UK’s National HPC service. She has a wide range of interests in supercomputing including the investigation of new languages and the challenges of utilising future Exascale platforms.