Current state-of-the-art denoising methods, such as non-local low-rank approaches, have shown to give impressive results. They are, however, mainly tuned to work with uniform Gaussian noise corruption and known variance, which is far from the real noise scenario. In fact, noise level estimation is already a challenging problem and denoising methods are quite sensitive to this parameter. Moreover, these methods are based on shrinkage models that are too simple to reflect reality, which results in over-smoothing of important structures, such as small-scale text and textures. We propose in this paper a new approach for more realistic image restoration based on the concept of low-rankness transfer. Given a training clean/noisy image pair, our method learns a mapping between the non-local noisy singular values and the optimal values for denoising to be transfered to a new noisy input. One single image is enough for training the model and can be adapted to the noisy input by taking a correlated image. Experiments conducted on synthetic and real camera noise show that the proposed method leads to an important improvement both visually and in terms of PSNR/SSIM.