Non-uniqueness and instability are characteristic features of image reconstruction methods. As a result, it is necessary to develop regularization methods that can be used to compute reliable approximate solutions. A regularization method provides a family of stable reconstructions that converge to a specific solution of the noise-free problem as the noise level tends to zero. The standard regularization technique is defined by a variational image reconstruction that minimizes a data discrepancy augmented by a regularizer. The actual numerical implementation makes use of iterative methods, often involving proximal mappings of the regularizer. In recent years, Plug-and-Play (PnP) image reconstruction has been developed as a new powerful generalization of variational methods based on replacing proximal mappings by more general image denoisers. While PnP iterations yield excellent results, neither stability nor convergence in the sense of regularization have been studied so far. In this work, we extend the idea of PnP by considering families of PnP iterations, each accompanied by its own denoiser. As our main theoretical result, we show that such PnP reconstructions lead to stable and convergent regularization methods. This shows for the first time that PnP is as mathematically justified for robust image reconstruction as variational methods.