Neurological Simulation Milestone Reached After UCL Embraces Allinea’s Tools On UK’s Largest Supercomputer

UCL breaks through scaling roadblock with development tools at scale and achieves first simulations of complex regions of the human brain.

Researchers at University College London (UCL) modelling intracranial blood flow have been able to reach a major milestone by using development tools from Allinea Software.

A massive simulation using 50,000 processor cores on the UK’s largest supercomputer, ARCHER, became possible after solving an application crash that only occurred at this high scale.

UCL’s HemeLB software applies computational fluid dynamics to model blood flow around cerebral vessels and simulate pressure at points of weakness such as aneurysms.  Using data from MRI scans of a patient’s blood vessels, the group anticipates that simulations will one day help to decide the best clinical option for individual patients.

The complexity of the simulations requires the UCL team to use ARCHER, the UK’s flagship Cray XC30 system, which is managed by Edinburgh Parallel Computing Centre (EPCC) on behalf of EPSRC and other UK research councils.

The UCL team used Allinea Software’s performance profiling tool, Allinea MAP, to increase the performance of HemeLB on the system. Having already improved performance on some test cases by over 25% they wanted to run a larger simulation.

“We’d never been able to look at this many cores – and get a clear view of how the time was being used – we were keen to see it in Allinea MAP,” UCL Post-doctoral Researcher Derek Groen said.

However, the application crashed when using 50,000 processor cores, which stopped them in their tracks.

“The crash was totally unexpected. I didn’t know how I would diagnose or fix it at that scale – it was beyond anything I had tried to do before.” says Groen. “Allinea Software helped us straight away – they knew that if we could run the simulation with their debugger, we would find the problem.”

The UCL team and Allinea Software were brought together by the EU CRESTA project – which is preparing applications for future extreme-scale computing.  Running applications at extreme scales poses challenges, which Allinea Software’s tools help to solve.

Allinea DDT is the only parallel debugger that handles that scale of the problem – which was resolved quickly by debugging all 50,000 application processes simultaneously.

“Getting HemeLB to scale to 50,000 ARCHER cores is a real achievement”, says Professor Peter Coveney, Director, Centre for Computational Science, UCL. “We are thankful for the productive collaborations we enjoy with Allinea Software that have allowed us to reach these intoxicating heights,  which are enabling us to study hemodynamics within the Circle of Willis for the first time.”

“Working alongside the HemeLB team has been a great experience for us,” says David Lecomber, CEO of Allinea Software. “It helps us to understand the issues they face better. It is inspiring to see the impact HPC could make to lives in the future – and we’ve seen at first-hand the impact on progress that the challenges make. Tools designed to handle scale are changing the way developers think about those hard challenges.”

“Seeing HemeLB used at this scale on ARCHER is very impressive, and highlights the success of the co-design approach used by UCL and Allinea Software on the CRESTA project”, says Dr Lorna Smith, CRESTA project manager and Deputy Director of the ARCHER CSE team. “This achievement with Allinea Software is a great example of the importance of providing a rich tooling environment for our users on ARCHER”.

Source: Allinea