We performed a series of molecular dynamics (MD) simulations of phospholipid bilayers under shear flow to estimate the effect of the system size on Kelvin-Helmholtz (KH)-like instability of the bilayer at the molecular scale. To extend the estimation by the MD simulations to the microscale, we introduced linear stability analysis for the fluid-fluid interface consisting of a thin membrane. For both the MD simulations and theoretical model, the critical velocity difference across the bilayer, where instability occurs, decreased with increasing wavelength of the bilayer undulation λ, which corresponds to the system size. When λ was more than about ten times larger than the bilayer thickness, the critical velocity difference in the MD simulations was in quantitative agreement with that obtained by the theoretical model. This means that the theoretical model is applicable for the shear-induced KH-like instability of the bilayer for large λ. The theoretical model showed that the critical velocity difference for the KH-like instability was proportional to λ^{-3/2}. Based on these results, we discuss the implications of the shear-induced bilayer instability in the shear-induced cell damage observed in experiments.