Silver nanoparticles (AgNPs) have wide-ranging applications, including as additives in consumer products and in medical diagnostics and therapy. Therefore, understanding how AgNPs interact with biological systems is important for ascertaining any potential health risks due to the likelihood of high levels of human exposure. Besides any severe, acute effects, it is desirable to understand more subtle interactions that could lead to milder, chronic health impacts. Nanoparticles are small enough to be able to enter biological cells and interfere with their internal biochemistry. The initial contact between the nanoparticle and cell is at the plasma membrane. To gain fundamental mechanistic insight into AgNP-membrane interactions, we investigate these phenomena in minimal model systems using a wide range of biophysical approaches applied to lipid vesicles. We find a strong dependence on the medium composition, where colloidally stable AgNPs in a glucose buffer have a negligible effect on the membrane. However, at physiological salt concentrations, the AgNPs start to weakly aggregate and sporadic but significant membrane perturbation events are observed. Under these latter conditions, transient poration and structural remodeling of some vesicle membranes are observed. We observe that the fluidity of giant vesicle membranes universally decreases by an average of 16% across all vesicles. However, we observe a small population of vesicles that display a significant change in their mechanical properties with lower bending rigidity and higher membrane tension. Therefore, we argue that the isolated occurrences of membrane perturbation by AgNPs are due to low-probability mechanomodulation by AgNP aggregation at the membrane.