We propose a general framework to study the stability of the subspace spanned by P consecutive eigenvectors of a generic symmetric matrix H0 when a small perturbation is added. This problem is relevant in various contexts, including quantum dissipation (H0 is then the Hamiltonian) and financial risk control (in which case H0 is the assets' return covariance matrix). We argue that the problem can be formulated in terms of the singular values of an overlap matrix, which allows one to define an overlap distance. We specialize our results for the case of a Gaussian orthogonal H0, for which the full spectrum of singular values can be explicitly computed. We also consider the case when H0 is a covariance matrix and illustrate the usefulness of our results using financial data. The special case where the top eigenvalue is much larger than all the other ones can be investigated in full detail. In particular, the dynamics of the angle made by the top eigenvector and its true direction defines an interesting class of random processes.