Understanding swarm pattern formation is of great interest because it occurs naturally in many physical and biological systems, and has artificial applications in robotics. In both natural and engineered swarms, agent communication is typically local and sparse. This is because, over a limited sensing or communication range, the number of interactions an agent has is much smaller than the total possible number. A central question for self-organizing swarms interacting through sparse networks is whether or not collective motion states can emerge where all agents have coherent and stable dynamics. In this work we introduce the phenomenon of swarm shedding in which weakly-connected agents are ejected from stable milling patterns in self-propelled swarming networks with finite-range interactions. We show that swarm shedding can be localized around a few agents, or delocalized, and entail a simultaneous ejection of all agents in a network. Despite the complexity of milling motion in complex networks, we successfully build mean-field theory that accurately predicts both milling state dynamics and shedding transitions. The latter are described in terms of saddle-node bifurcations that depend on the range of communication, the inter-agent interaction strength, and the network topology.