Image segmentation for human organs is an important task for the diagnosis and treatment of diseases. Current deep learning-based methods are fully supervised and need pixel-level labels. Since the medical images are highly specialized and complex, the work of delineating pixel-level segmentation masks is very time-consuming. Weakly supervised methods are then chosen to lighten the workload, which only needs physicians to determine whether an image contains the organ regions of interest. These weakly supervised methods have a common drawback, in that they do not incorporate prior knowledge that alleviates the lack of pixel-level information for segmentation. In this work, we propose a weakly supervised method based on prior knowledge for the segmentation of human organs. The proposed method was validated on three data sets of human organ segmentation. Experimental results show that the proposed image-level supervised segmentation method outperforms several state-of-the-art methods.