A Scalable Data Access Layer to Manage Structured Heterogeneous Biomedical Data

PLoS One. 2016 Dec 9;11(12):e0168004. doi: 10.1371/journal.pone.0168004. eCollection 2016.

Abstract

This work presents a scalable data access layer, called PyEHR, designed to support the implementation of data management systems for secondary use of structured heterogeneous biomedical and clinical data. PyEHR adopts the openEHR's formalisms to guarantee the decoupling of data descriptions from implementation details and exploits structure indexing to accelerate searches. Data persistence is guaranteed by a driver layer with a common driver interface. Interfaces for two NoSQL Database Management Systems are already implemented: MongoDB and Elasticsearch. We evaluated the scalability of PyEHR experimentally through two types of tests, called "Constant Load" and "Constant Number of Records", with queries of increasing complexity on synthetic datasets of ten million records each, containing very complex openEHR archetype structures, distributed on up to ten computing nodes.

MeSH terms

  • Computational Biology*
  • Database Management Systems*
  • Information Storage and Retrieval*

Grants and funding

The authors received no specific funding for this work.