We systematically evaluate a Deep Learning model in a 3D medical image segmentation task. With our model, we address the flaws of manual segmentation: high inter-rater contouring variability and time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the model reduces the number of detection disagreements by [Formula: see text] [Formula: see text]. Secondly, we show that the model improves the inter-rater contouring agreement from [Formula: see text] to [Formula: see text] surface Dice Score [Formula: see text]. Thirdly, we show that the model accelerates the delineation process between [Formula: see text] and [Formula: see text] times [Formula: see text]. Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases; thus, preserving the significance of the results. Besides the clinical evaluation, we also share intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation.