Enabling big geoscience data analytics with a cloud-based, MapReduce-enabled and service-oriented workflow framework

PLoS One. 2015 Mar 5;10(3):e0116781. doi: 10.1371/journal.pone.0116781. eCollection 2015.

Abstract

Geoscience observations and model simulations are generating vast amounts of multi-dimensional data. Effectively analyzing these data are essential for geoscience studies. However, the tasks are challenging for geoscientists because processing the massive amount of data is both computing and data intensive in that data analytics requires complex procedures and multiple tools. To tackle these challenges, a scientific workflow framework is proposed for big geoscience data analytics. In this framework techniques are proposed by leveraging cloud computing, MapReduce, and Service Oriented Architecture (SOA). Specifically, HBase is adopted for storing and managing big geoscience data across distributed computers. MapReduce-based algorithm framework is developed to support parallel processing of geoscience data. And service-oriented workflow architecture is built for supporting on-demand complex data analytics in the cloud environment. A proof-of-concept prototype tests the performance of the framework. Results show that this innovative framework significantly improves the efficiency of big geoscience data analytics by reducing the data processing time as well as simplifying data analytical procedures for geoscientists.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Algorithms
  • Cloud Computing*
  • Computational Biology / methods*
  • Earth Sciences*
  • Internet
  • Workflow*

Grants and funding

This research is supported by NSF (PLR-1349259, IIP-1338925, CNS-1117300) and NASA (NNG12PP37I). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.