It is great to see US and European governments undertake initiatives to promote the development of research into Big Data utilizing commercial clouds.

Many cloud providers are offering free resources to support these initiatives. R&E networks will play a critical role in linking researchers to the commercial clouds and developing collaboration platforms and portals. The recent Apache-Rave announcement in partnership with XSEDE and COmanage in the US and SURFconext in Netherlands is a great example of developing “Research as a Service” using commercial clouds. See Ian Foster presentation. Peering with commercial cloud providers will also be critical.

I have long argued that development of commercial clouds to support research will fundamentally change cyber-infrastructure at universities. As Dr Ed Lazowska commented in a New York Times article: “The need to analyze vast amounts of data from a broad array of sensors is going to be far more pervasive than the use of numerical simulation—even though the use of numerical simulation continues to increase. Even in fields such as national security and scientific discovery, for decades the flagships for HPC, large-scale data analysis is growing to equal importance. And this requires entirely different hardware and software architectures than does traditional HPC.” HPC will remain an important niche, but analyzing large volumes of data is ideally suited for commercial clouds.

Many people have argued for public funded academic clouds. The big disadvantage of an academic cloud is that it requires new infrastructure updates every few years in order to meet ongoing demand for additional computation resources. So the situation, from a funding council perspective is an ongoing requirement to continuously upgrade computer resources whether they stand alone systems or are lumped together within an academic cloud. But with commercial clouds funding agencies do not have to purchase infrastructure to enable researchers to use these facilities. Commercial clouds make the necessary investment to upgrade their infrastructure over time as demand warrants. Many commercial cloud providers spend hundreds of millions per year on computer upgrades—which dwarfs the annual expenditure most funding councils spend on HPC facilities.

Many R&E networks are providing brokered commercial cloud services which will further reduce cost of using clouds (for those that are not free).