We investigate whether the many-body ground states of bosons in a generalized two-mode model with localized inhomogeneous single-particle orbitals and anisotropic long-range interactions (e.g., dipole-dipole interactions) are coherent or fragmented. It is demonstrated that fragmentation can take place in a single trap for positive values of the interaction couplings, implying that the system is potentially stable. Furthermore, the degree of fragmentation is shown to be insensitive to small perturbations on the single-particle level.