Privacy-enhanced multi-party deep learning

Abstract

In multi-party deep learning, multiple participants jointly train a deep learning model through a central server to achieve common objectives without sharing their private data. Recently, a significant amount of progress has been made toward the privacy issue of this emerging multi-party deep learning paradigm. In this paper, we mainly focus on two problems in multi-party deep learning. The first problem is that most of the existing works are incapable of defending simultaneously against the attacks of honest-but-curious participants and an honest-but-curious server without a manager trusted by all participants. To tackle this problem, we design a privacy-enhanced multi-party deep learning framework, which integrates differential privacy and homomorphic encryption to prevent potential privacy leakage to other participants and a central server without requiring a manager that all participants trust. The other problem is that existing frameworks consume high total privacy budget when applying differential privacy for preserving privacy, which leads to a high risk of privacy leakage. In order to alleviate this problem, we propose three strategies for dynamically allocating privacy budget at each epoch to further enhance privacy guarantees without compromising the model utility. Moreover, it provides participants with an intuitive handle to strike a balance between the privacy level and the training efficiency by choosing different strategies. Both analytical and experimental evaluations demonstrate the promising performance of our proposed framework.

Keywords: Differential privacy; Homomorphic encryption; Multi-party deep learning; Privacy; Privacy budget.