Empirical evaluation of an architectural technical debt index in the context of the Apache and ONAP ecosystems

Roberto Verdecchia; Ivano Malavolta; Patricia Lago; Ipek Ozkaya

doi:10.7717/peerj-cs.833

Empirical evaluation of an architectural technical debt index in the context of the Apache and ONAP ecosystems

PeerJ Comput Sci. 2022 Feb 7:8:e833. doi: 10.7717/peerj-cs.833. eCollection 2022.

Authors

Roberto Verdecchia¹, Ivano Malavolta¹, Patricia Lago^{1

2}, Ipek Ozkaya³

Affiliations

¹ Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
² Chalmers University of Technology, Gothenburg, Sweden.
³ Carnegie Mellon Software Engineering Institute, Pittsburgh, USA.

Abstract

Background: Architectural Technical Debt (ATD) in a software-intensive system denotes architectural design choices which, while being suitable or even optimal when adopted, lower the maintainability and evolvability of the system in the long term, hindering future development activities. Despite the growing research interest in ATD, how to gain an informative and encompassing viewpoint of the ATD present in a software-intensive system is still an open problem.

Objective: In this study, we evaluate ATDx, a data-driven approach providing an overview of the ATD present in a software-intensive system. The approach, based on the analysis of a software portfolio, calculates severity levels of architectural rule violations via a clustering algorithm, and aggregates results into different ATD dimensions.

Method: To evaluate ATDx, we implement an instance of the approach based on SonarQube, and run the analysis on the Apache and ONAP ecosystems. The analysis results are then shared with the portfolio contributors, who are invited to participate in an online survey designed to evaluate the representativeness and actionability of the approach.

Results: The survey results confirm the representativeness of the ATDx, in terms of both the ATDx analysis results and the used architectural technical debt dimensions. Results also showed the actionability of the approach, although to a lower extent when compared to the ATDx representativeness, with usage scenarios including refactoring, code review, communication, and ATD evolution analysis.

Conclusions: With ATDx, we strive for the establishment of a sound, comprehensive, and intuitive architectural view of the ATD identifiable via source code analysis. The collected results are promising, and display both the representativeness and actionability of the approach. As future work, we plan to consolidate the approach via further empirical experimentation, by considering other development contexts (e.g., proprietary portfolios and other source code analysis tools), and enhancing the ATDx report capabilities.

Keywords: Empirical evaluation; Index; Software architecture; Software metrics; Software portfolio analysis; Technical debt.

Grants and funding

This article is based upon work funded and supported by the Department of Defense under Contract No. FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. There was no direct funding.