The conditions for the formation of local bumps in the activity of binary attractor neural networks with spatially dependent connectivity are investigated. We show that these formations are observed when asymmetry between the activity during the retrieval and learning is imposed. An analytical approximation for the order parameters is derived. The corresponding phase diagram shows a relatively large and stable region where this effect is observed, although critical storage and information capacities drastically decrease inside that region. We demonstrate that the stability of the network, when starting from the bump formation, is larger than the stability when starting even from the whole pattern. Finally, we show a very good agreement between the analytical results and the simulations performed for different topologies of the network.