Performance and scaling behavior of bioinformatic applications in virtualization environments to create awareness for the efficient use of compute resources

PLoS Comput Biol. 2021 Jul 20;17(7):e1009244. doi: 10.1371/journal.pcbi.1009244. eCollection 2021 Jul.

Abstract

The large amount of biological data available in the current times, makes it necessary to use tools and applications based on sophisticated and efficient algorithms, developed in the area of bioinformatics. Further, access to high performance computing resources is necessary, to achieve results in reasonable time. To speed up applications and utilize available compute resources as efficient as possible, software developers make use of parallelization mechanisms, like multithreading. Many of the available tools in bioinformatics offer multithreading capabilities, but more compute power is not always helpful. In this study we investigated the behavior of well-known applications in bioinformatics, regarding their performance in the terms of scaling, different virtual environments and different datasets with our benchmarking tool suite BOOTABLE. The tool suite includes the tools BBMap, Bowtie2, BWA, Velvet, IDBA, SPAdes, Clustal Omega, MAFFT, SINA and GROMACS. In addition we added an application using the machine learning framework TensorFlow. Machine learning is not directly part of bioinformatics but applied to many biological problems, especially in the context of medical images (X-ray photographs). The mentioned tools have been analyzed in two different virtual environments, a virtual machine environment based on the OpenStack cloud software and in a Docker environment. The gained performance values were compared to a bare-metal setup and among each other. The study reveals, that the used virtual environments produce an overhead in the range of seven to twenty-five percent compared to the bare-metal environment. The scaling measurements showed, that some of the analyzed tools do not benefit from using larger amounts of computing resources, whereas others showed an almost linear scaling behavior. The findings of this study have been generalized as far as possible and should help users to find the best amount of resources for their analysis. Further, the results provide valuable information for resource providers to handle their resources as efficiently as possible and raise the user community's awareness of the efficient usage of computing resources.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Algorithms
  • Benchmarking
  • Cloud Computing
  • Computational Biology / methods*
  • Computational Biology / standards
  • Computational Biology / statistics & numerical data
  • Computers
  • Computing Methodologies
  • Data Interpretation, Statistical
  • Databases, Factual / statistics & numerical data
  • High-Throughput Nucleotide Sequencing
  • Humans
  • Image Interpretation, Computer-Assisted
  • Machine Learning
  • Sequence Alignment
  • Software
  • User-Computer Interface

Grants and funding

This work has been supported by the High Performance and Cloud Computing Group at the Zentrum für Datenverarbeitung of the University of Tübingen providing the overall environment including the position of JK, the state of Baden-Württemberg through bwHPC funding the position of FB and the German Research Foundation (DFG)[https://www.dfg.de] through grant no INST 37/935-1 FUGG funding compute resources. Part of the work presented here was also supported through the BMBF funded project de.NBI (031 A 534A)[https://www.bmbf.de] supplying the cloud computing hardware and the position of MH. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.