Home / Blogs

Advantage of Commercial Clouds vs. HPC for Scientific Research

Bill St. Arnaud

The Department of Energy (DoE) recently came out with an excellent report, called the Magellan report, on the advantages and disadvantages of using commercial clouds versus in house High Performance Computers (HPC) for leading edge scientific research.

The DoE probably supports the largest concentration of HPC facilities in the world. I agree with the report that for traditional applications such as computational chemistry, astrophysics, etc. will still need large HPC facilities. But traditional computational intensive applications are becoming a niche market and increasingly many of these applications can now run on specialized commercial "HPC" clouds as for example Nimbix.

The biggest growth in demand for computing is not in computational intensive modeling but data intensive processing. New disciplines such as Astroinformatics, Matinformatics (real-time chemical analysis), Systems biology, Meta-genomics, Computational history, computational linguistics, etc. are the driving force for research computing. Most of these data intensive applications are loosely coupled and are ideally suited for using clouds.

While the growth of data intensive science and use of clouds is well recognized, it is still ongoing debate whether researchers should use in-house clouds or commercial facilities. The DoE report did an extensive analysis on the cost of commercial clouds versus in-house facilities. They compared the cost per compute core of an in-house facility versus that of a commercial provider. While I may argue with some of the assumptions in their analysis: for example they did not include cost of money or real estate in their analysis, nor did they use much lower spot market for commercial cloud pricing, I still agree that, in the near term, commercial clouds will be marginally more expensive than in-house facilities.

From a funding agency perspective, however, there is huge advantage of promoting commercial clouds over an in-house facility. Despite the higher per core costs, the elimination of up-front capital costs of using a commercial cloud is incredibly significant, especially in this time period of fiscal constraint. Any capital expenses that can be delayed or eliminated, and yet not impact the quality of the research, has a huge cost benefit to funding agencies. This is also advantageous to the researcher as well. Usually it takes several years to make a proposal, get approval, acquire and install a large HPC facility. With commercial clouds a researcher can start immediately to undertake their computational research. The upfront cost is very small and their time to market (i.e. publishing the results) can be much faster with a commercial facility. In fact some commercial clouds like Amazon and Azure offer a free pilot service to allow researchers and businesses to migrate their software to the cloud and shake out any possible kinks in their software.

With a commercial facility researchers can scale their application as warranted without incurring any additional capital costs. There is no need for peer review to determine the resources that may be made available to the researcher. More importantly, because the incremental per core costs are very small, many other venues for funding for the computation facility are available, as opposed to the limited funding channels available for the purchase of an in-house facility. For example, some commercial organizations will broker their cloud infrastructure for little or no cost to university researchers, as opposed to commercial users. Many R&E networks are also negotiating significant bulk discounts for commercial cloud services on behalf of the R&E community.

By Bill St. Arnaud , Green IT Networking Consultant

Related topics: Cloud Computing, Data Center, Networks

 
   

Don't miss a thing – get the Weekly Wrap delivered to your inbox.

Comments

To post comments, please login or create an account.

Related Blogs

Related News

Explore Topics

Dig Deeper

Verisign

Cybersecurity

Sponsored by Verisign
Afilias Mobile & Web Services

Mobile Internet

Sponsored by Afilias Mobile & Web Services
Afilias

DNS Security

Sponsored by Afilias

Promoted Posts

Now Is the Time for .eco

.eco launches globally at 16:00 UTC on April 25, 2017, when domains will be available on a first-come, first-serve basis. .eco is for businesses, non-profits and people committed to positive change for the planet. See list of registrars offering .eco more»

Industry Updates – Sponsored Posts

Attacks Decrease by 23 Precent in 1st Quarter While Peak Attack Sizes Increase: DDoS Trends Report

Verisign Releases Q2 2016 DDoS Trends Report - Layer 7 DDoS Attacks a Growing Trend

Verisign Q1 2016 DDoS Trends: Attack Activity Increases 111 Percent Year Over Year

Mobile Web Intelligence Report: Bots and Crawlers May Represent up to 50% of Web Traffic

i2Coalition to Host First Ever Smarter Internet Forum

What Holds Firms Back from Choosing Cloud-Based External DNS?

Data Volumes and Network Stress to Be Top IoT Concerns

Verisign Mitigates More Attack Activity in Q3 2015 Than Any Other Quarter During Last Two Years

Verisign & Forrester Webinar: Defending Against Cyber Threats in Complex Hybrid-Cloud Environments

Dyn Evolves Internet Performance Space with Launch of Internet Intelligence

Verisign's Q2'15 DDoS Trends: DDoS for Bitcoin Increasingly Targets Financial Industry

Protect Your Network From BYOD Malware Threats With The Verisign DNS Firewall

Hybrid Cloud Proves Clouds Are Worthy of Email Infrastructure

Verisign OpenHybrid for Corero and Amazon Web Services Now Available

Verisign iDefense 2015 Cyber-Threats and Trends

3 Questions to Ask Your DNS Host About DDoS

Afilias Partners With Internet Society to Sponsor Deploy360 ION Conference Series Through 2016

Neustar to Build Multiple Tbps DDoS Mitigation Platform

3 Questions to Ask Your DNS Host about Lowering DDoS Risks

Tips to Address New FFIEC DDoS Requirements