We probe the onset and effect of contact changes in soft harmonic particle packings which are sheared quasistatically. We find that the first contact changes are the creation or breaking of contacts on a single particle. We characterize the critical strain, statistics of breaking versus making a contact, and ratio of shear modulus before and after such events, and explain their finite size scaling relations. For large systems at finite pressure, the critical strain vanishes but the ratio of shear modulus before and after a contact change approaches one: linear response remains relevant in large systems. For finite systems close to jamming the critical strain also vanishes, but here linear response already breaks down after a single contact change.