We develop a nonlinear optimization model to identify minimum cost designs for osmotically assisted reverse osmosis (OARO), a multistaged membrane-based process for desalinating high-salinity brines. The optimization model enables comprehensive evaluation of a complex process configuration and operational decision space that includes nonlinear process performance and implicit relationships among membrane stages, saline sweep cycles, and makeup, purge, and recycle streams. The objective function minimizes cost, rather than energy or capital expenditures, to accurately account for the trade-offs in capital and operational expenses inherent in multistaged membrane processes. Generally, we find that cost-optimal OARO processes minimize the number of stages, eliminate the use of saline makeup streams, purge from the first sweep cycle, and successively decrease stage membrane area and sweep flow rates. The optimal OARO configuration for treating feed salinities of 50-125 g/L total dissolved solids with water recoveries between 30-70% results in costs less than or equal to $6 per m3 of product water. Sensitivity analysis suggests that future research to minimize OARO costs should focus on minimizing the membrane structural parameter while maximizing the membrane burst pressure and reducing the membrane unit cost.