Vesicle structures assembled from short peptides are excellent carriers for drug delivery. Modifying peptides with photo-responsive azobenzene (azo) moieties is expected to generate smart vesicles that could release cargos under stimulation of ultraviolet (UV) or visible (vis) light irradiation. The modified azo groups could dramatically affect delicate intermolecular interactions, thereby perturbing the self-assembly pathways of peptides. However, through well molecular design and screening, it should be possible to manipulate the self-assembly to obtain such smart vesicle structures. Coarse-grained (CG) molecular dynamics (MD) simulations were employed to complete the screening of azo-containing peptide derivatives and to clarify molecular mechanisms underlying the self-assembly of vesicles and the photo-response performance. Our simulations demonstrate that grafting an azo moiety to the side chain of phenylalanyl-alanine (FA) generates the F(azo)A molecule that can self-assemble into vesicles, and the addition of diphenylalanine (FF) improves the self-assembly efficiency. The formation of vesicles undergoes three stages: nucleation, fusion, and curling. On the one hand, FF molecules promote the fusion and curling stages, facilitating the co-assembly process. On the other hand, the trans-cis isomerization of F(azo)A side chains perturbs the packing of F(azo)A-FF membranes, inducing photo-responsive morphology transition and permeability change. These results are expected to promote the future regulation of self-assembly behaviors and design of smart self-assembly materials.