We have applied replica exchange with solute tempering (REST) molecular dynamics to study a short fragment of the Aβ peptide, Aβ25-35, in water and a much larger system incorporating two Aβ10-40 peptides binding to the zwitterionic dimyristoylphosphatidylcholine (DMPC) bilayer. As a control, we used traditional replica exchange molecular dynamics (REMD) applied to the same systems. Our objective was to assess the practical utility of REST simulations. Taken together, our results suggest four conclusions. First, compared to REMD, the number of replicas in REST simulations can be reduced four to five times without affecting the temperature range or compromising an efficient random walk of REST replicas over temperatures. Second, although overall REST produces much fewer conformational states than REMD, there is no substantial difference in the collection of unique states for the wild-type replica in REST and REMD, especially for the system featuring Aβ peptides binding to the lipid bilayer. Third, we performed a thorough comparison of REST and REMD equilibrium conformational ensembles, including thermal averages and probability distributions. REST reproduces REMD data extremely well for the system of Aβ peptides binding to the DMPC lipid bilayer. The agreement between REST and REMD equilibrium sampling of Aβ25-35 in water is less perfect, but it improves with addition of new REST simulations. Surprisingly, REST demonstrates much better convergence for the system featuring ordered peptides binding to lipid bilayer rather than for a small unstructured peptide solvated in water. Fourth, REST delivers its full computational advantage over REMD when applied to peptides interacting with lipid bilayers. For peptides solvated in water, REST does not appear to offer computational gain but may make replica simulations practically feasible due to a lower requirement for parallel computing environments. Our study is expected to facilitate wider application of REST in biomolecular simulations.