Dr Pekka Manninen is a senior HPC specialist at CSC. He works with support and optimization of end-user applications on CSC supercomputers.

Increased application performance via burst I/O buffering

In HPC systems of today, there is a horrendous performance gap between DRAM-based main memory and the storage system, typically based on Lustre or similar parallel file system. On Lustre, the I/O bandwidth is very high because of its highly parallel design, but with a downside of pronounced I/O latency. This is due to the access over an interconnect plus the separate metadata server. In practice, this means that small and frequent I/O accesses are poisonous for application performance – and there are several algorithms and codes in scientific computing that require the small but frequent non-contiguous reads and writes.

These bottlenecks have been alleviated in the current generation of HPC systems by introducing some kind of local storage on a compute node; earlier by spinning disks and more recently by SSD (solid state disk) based technologies. Typically these options have been available only on mid-range cluster systems, since designs aimed towards the Top-100 systems cannot host these due to packaging density considerations. Furthermore, the local disk solutions are separate mount points i.e. separate directories. It has been up to the user or the application code to use the proper storage location.

Next year will bring some improvements to the situation by introducing more levels to the storage hierarchy, between the main memory and the Lustre-type file system. For instance, Cray is introducing to the current XC family of supercomputers (for which addition of compute node disks of any kind has not been possible) their DataWarp technology, which is a flash-based burst I/O buffering and a cache level for Lustre. The final version of the technology will be fully transparent to the application, DataWarp absorbing all the bursty I/O and taking it to the Lustre in the background, and being also capable of prefetching data. DDN will hit the market with their Infinite Memory Engine solution, aiming at providing similar outcome but with a different philosophy.

Early versions of these technologies have shown very promising performance benefits for real-world applications. CSC’s Cray XC “Sisu” will get two DataWarp blades in mid-January in order to experiment with the burst buffer technology. The prediction? These technologies will become more or less an auto-include in systems aimed for hosting real-world, heterogeneous workloads deployed in 2016!

First steps towards practical quantum computing
Rather than storing information as 0’s and 1’s as conventional computers (from tablets to supercomputers) do, a quantum computer uses qubits, which can be 0 and 1 at the same time. This quantum superposition, along with the quantum mechanical phenomena of entanglement and tunneling, enable quantum computers to consider and manipulate all combinations of bits simultaneously.

The first commercial realization of a quantum computer by Canadian startup company D-Wave Systems has gained interest during this year, announcing installations by e.g. Lockheed Martin, a joint procurement by Google and NASA and a joint procurement of Los Alamos and Sandia National Laboratories. Their solution is a quantum annealing computer, consisting of 1024 (in fact a bit more) qubits based on superconducting loops. It is capable of finding an energy minimum of more or less arbitrarily complex optimization problem in a constant time (exactly one operation). Early benchmarks by Google show 108-fold speedup in quantum system simulation as compared to the Quantum Monte Carlo method calculation on a conventional computer.

My related prediction for the year 2016 is that the D-Wave Systems will have their order book full, announcing more and more installations by national laboratories and leading computing centres across the globe. In addition, we are going to see first computational challenges (intractable by conventional computers) solved by quantum computers published in scientific journals. Even if it is unlikely that CSC will be installing a quantum computer in the near future, some introductory activity on this topic may be coming up. Stay tuned!