Fully convolutional neural network (FCN) has achieved great success in semantic segmentation. However, the performance of the FCN is generally compromised for multi-object segmentation. Multi-organ segmentation is very common while challenging in the field of medical image analysis, where organs largely vary with scales. Different organs are often treated equally in most segmentation networks, which is not quite optimal. In this work, we propose to divide a multi-organ segmentation task into multiple binary segmentation tasks by constructing a multi-to-binary network (MTBNet). The proposed MTBNet is based on the FCN for pixel-wise prediction. Moreover, we build a plug-and-play multi-to-binary block (MTB block) to adjust the influence of the feature maps from the backbone. This is achieved by parallelizing multiple branches with different convolutional layers and a probability gate (ProbGate). The ProbGate is set up for predicting whether the class exists, which is supervised clearly via an auxiliary loss without using any other inputs. More reasonable features are achieved by summing branches' features multiplied by the probability from the accompanying ProbGate and fed into a decoder module for prediction. The proposed method is validated on a challenging task dataset of multi-organ segmentation in abdominal MRI. Compared to classic medical and other multi-scale segmentation methods, the proposed MTBNet improves the segmentation accuracy of small organs by adjusting features from different organs and reducing the chance of missing or misidentifying these organs.