Compi: a framework for portable and reproducible pipelines

Hugo López-Fernández; Osvaldo Graña-Castro; Alba Nogueira-Rodríguez; Miguel Reboiro-Jato; Daniel Glez-Peña

doi:10.7717/peerj-cs.593

Compi: a framework for portable and reproducible pipelines

PeerJ Comput Sci. 2021 Jun 18:7:e593. doi: 10.7717/peerj-cs.593. eCollection 2021.

Authors

Hugo López-Fernández^{1

2

3}, Osvaldo Graña-Castro⁴, Alba Nogueira-Rodríguez^{1

2

3}, Miguel Reboiro-Jato^{1

2

3}, Daniel Glez-Peña^{1

2

3}

Affiliations

¹ ESEI: Escuela Superior de Ingeniería Informática, University of Vigo, Ourense, Galicia, Spain.
² CINBIO - Centro de Investigaciones Biomédicas, University of Vigo, Vigo, Galicia, Spain.
³ SING Research Group, Galicia Sur Health Research Institute (IIS Galicia Sur). SERGAS-UVIGO, Vigo, Galicia, Spain.
⁴ Bioinformatics Unit, Structural Biology Programme, Spanish National Cancer Research Centre (CNIO), Madrid, Madrid, Spain.

Abstract

Compi is an application framework to develop end-user, pipeline-based applications with a primary emphasis on: (i) user interface generation, by automatically generating a command-line interface based on the pipeline specific parameter definitions; (ii) application packaging, with compi-dk, which is a version-control-friendly tool to package the pipeline application and its dependencies into a Docker image; and (iii) application distribution provided through a public repository of Compi pipelines, named Compi Hub, which allows users to discover, browse and reuse them easily. By addressing these three aspects, Compi goes beyond traditional workflow engines, having been specially designed for researchers who want to take advantage of common workflow engine features (such as automatic job scheduling or logging, among others) while keeping the simplicity and readability of shell scripts without the need to learn a new programming language. Here we discuss the design of various pipelines developed with Compi to describe its main functionalities, as well as to highlight the similarities and differences with similar tools that are available. An open-source distribution under the Apache 2.0 License is available from GitHub (available at https://github.com/sing-group/compi). Documentation and installers are available from https://www.sing-group.org/compi. A specific repository for Compi pipelines is available from Compi Hub (available at https://www.sing-group.org/compihub.

Keywords: Application development framework; Computational pipelines; Workflow management systems.

Grants and funding

This work was supported by the Consellería de Educación, Universidades e Formación Profesional (Xunta de Galicia) under the scope of the strategic funding ED431C2018/55-GRC Competitive Reference Group and Ministerio de Economía, Industria y Competitividad, Gobierno de España under the scope of the PolyDeep project (DPI2017-87494-R). Alba Nogueira-Rodríguez is supported by a pre-doctoral fellowship from Xunta de Galicia (ED481A-2019/299). There was no additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.