The magnetic and electrical properties of high-quality single crystals of A-site disordered (solid solution) Ln0.5Ba0.5MnO3 are investigated near the phase boundary between the spin-glass insulator and colossal-magnetoresistive ferromagnetic metal, locating near Ln=Sm. The temperature dependence of the ac susceptibility and the x-ray diffuse scattering of Eu0.5Ba0.5MnO3 are analyzed in detail. The uniformity of the random potential perturbation in Ln0.5Ba0.5MnO3 crystals with a small bandwidth yields, rather than the phase separation, an homogeneous short ranged charge or orbital order which gives rise to a nearly atomic spin-glass state. Remarkably, this microscopically disordered "charge-exchange-glass" state alone is able to bring forth the colossal magnetoresistance.