How cost efficient is HPC in the Cloud?
An IDC study from 2007 [2] found that only about 7% of the total cost of acquiring and operating an HPC system is coming from the hardware (additional analysis shows that this situation hasn’t changed since then). The much bigger portion of the pie is coming from the high cost of expertise (staffing), equipment, maintenance, software, training, i.e. the Total Cost of Ownership (TCO). A recent cost analysis for an in-house HPC system versus an HPC Cloud service [13] shows that overall cost depends on the average utilization of the in-house sever. The following table shows the total cost of one CPU core hour for a 16-node (256-core) in-house HPC server depending on average percentage of utilization (or ‘number of busy nodes’) of this server.

For this example, with an average utilization of say 25%, the real cost for one core hour is $0.60, i.e. 4 times higher than for a 100% utilized cluster. For 50% utilization, which is not uncommon in HPC, the cost of one core hour is still $0.30. For comparison, the actual cost of one core hour of Microsoft’s Azure HPC Cloud instances D and G vary between $0.09 – $0.30 [14].

According to Table 1, the cost of the cloud (say $0.20 per core hour) will exceed the one for the in-house solution only at about 75% (and higher) average utilization of the in-house HPC cluster, a situation which we only heard of from academic or big-industry Supercomputing Centers serving hundreds or even thousands of users. If we extrapolate these numbers to one year and include a hybrid (in-house / in-cloud) scenario, we’ll get the results shown in Figure 4 (see details in [13]).

UberCloud Experiments Accelerating HPC in the Cloud
The voluntary free UberCloud Experiment provides a crowd-sourcing platform for researchers and engineers to explore, learn, and understand the end-to-end process of accessing and using HPC Clouds, to identify the concerns, and resolve the roadblocks [10]. End-user, software provider, resource provider, and an HPC expert are collaborating in a team and are guided through a 23-step process, jointly solving the end-user’s application problem.

Since July 2012, HPC Experiment has attracted 3000+ organizations from 72 countries (status July 2015). The organizers were able to build 175 of these teams, in CFD, FEM, and Computational Biology, and to publish many case studies in two Compendiums (sponsored by Intel) [15]. In an experiment, the team participants are the industrial end-user, a suitable resource provider, the software provider, and an HPC expert.

Then, with modest guidance from the UberCloud Experiment, the user, resource provider, and HPC expert will implement and run the task and deliver the results back to the end-user. The hardware and software providers will measure resource usage; the HPC expert will summarize the steps of analysis and implementation; the end-user will evaluate the quality of the process and of the results and the degree of user-friendliness this process provides. Finally, the team will get together, extract lessons learned, and present further recommendations as input to their case study.

As a glimpse into the wealth of practical use case results so far, here is a selection of Cloud experiments (selected from 175 so far) demonstrating the wide spectrum of CAE applications in the Cloud.

Team 94: Gas-liquid Two-phase Flow Application
End-User in this team was Kyoji Ishikawa from Chiyoda Corporation in Japan, together with Wim Slagter from ANSYS and Hiroyuki Kanazawa from resource provider Fujitsu Ltd. which provides HPC services including the virtual desktop function under the name of Technical Computing (TC) Cloud. The use case was the evaluation of the rate of gas entrainment due to liquid flow in the liquid storage facility for an energy plant (see Figure 5). Gas-liquid two-phase flow was simulated using ANSYS Fluent software.

Because the end-user sometimes experiences a computational resource shortage for urgent, unexpected or unplanned projects, he decided to explore the use of remote cloud computing. Two simulation cases were carried out with total mesh numbers 300,000 and 2 million. Computation time on the in-house system for case 1 was 20 hours and for case 2 required 5 days. Cloud computing resources were 32 parallel cores (16 cores × 2 nodes) and Gigabit Ethernet or InfiniBand (IB). Pre- and post-processing has been done with remote visualization. That way the end-user was able to reduce the internal run-time from up to 5 days down to less than a day.

Team 99: North Sea Asset Life Extension – Assessing Impacts on Helicopter Operations in the Cloud
This team consisted of the end-user Dan Hamilton from Atkins Energy, software provider James Britton, CD-adapco, resource provider Jerry Dixon, OCF, and team mentor Dennis Nagy from BeyondCAE. The team tested the feasibility of using cloud services for the simulation of airflow over an offshore platform using STAR-CCM+ from CD-adapco, to determine the change in conditions within the helideck landing area as a result of geometrical changes stemming from a life extension project on an existing North Sea asset.

For OCF, the availability of the STAR-CCM+ “Power-on-Demand” licensing is a perfect fit for their enCORE service. The user experience of using STAR-CCM+ in batch on enCORE was identical to that on in-house hardware. As the simulation files can be large, enCORE’s policy of not charging for bandwidth usage is appealing. Having a resource like enCORE available for use allows to bid for and propose work requiring computational resources, which exceeds what is available in-house.