Embedded mean-field theory (EMFT) provides a simple, flexible framework for describing subsystems at different levels of mean-field theory. Subsystems are defined by partitioning a one-particle basis set, with a natural choice being the atomic orbital (AO) basis. Although generally well behaved, EMFT with AO partitioning can exhibit unphysical collapse of the self-consistent solution. To avoid this issue, we introduce subsystem partitioning of a block-orthogonalized (BO) basis set; this eliminates the unphysical collapse without significantly increasing computational cost. We also investigate a non-self-consistent implementation of EMFT, in which the density matrix is obtained using BO partitioning and the final energy evaluated using AO partitioning; this density-corrected EMFT approach is found to yield more accurate energies than BO partitioning while also avoiding issues of the unphysical collapse. Using these refined implementations of EMFT, previously proposed descriptions of the exact-exchange coupling between subsystems are compared: although the EX1 coupling scheme is slightly more accurate than EX0, the small improvement does not merit its substantially greater computational cost.