The switching of electric polarization induced by electric fields, a fundamental functionality of ferroelectrics, is closely associated with the motions of the domain walls that separate regions with distinct polarization directions. Therefore, understanding domain-walls dynamics is of essential importance for advancing ferroelectric applications. In this Letter, we show that the topology of the multidomain structure can have an intrinsic impact on the degree of switchable polarization. Using a combination of polarization hysteresis measurements and piezoresponse force microscopy on a uniaxial organic ferroelectric, α-6,6'-dimethyl-2,2'-bipyridinium chloranilate, we found that the head-to-head (or tail-to-tail) charged domain walls are strongly pinned and thus impede the switching process; in contrast, if the charged domain walls are replaced with electrically neutral antiparallel domain walls, bulk polarization switching is achieved. Our findings suggest that manipulation of the multidomain topology can potentially control the switchable polarization.