Recursive density-matrix perturbation theory [A.M.N. Niklasson and M. Challacombe, Phys. Rev. Lett. 92, 193001 (2004)] provides an efficient framework for the linear scaling computation of materials response properties [V. Weber, A.M.N. Niklasson, and M. Challacombe, Phys. Rev. Lett. 92, 193002 (2004)]. In this article, we generalize the density-matrix perturbation theory to include properties computed with a perturbation-dependent nonorthogonal basis. Such properties include analytic derivatives of the energy with respect to nuclear displacement, as well as magnetic response computed with a field-dependent basis. The theory is developed in the context of linear scaling purification methods, which are briefly reviewed.