The aim of the study is to present a new Convolutional Neural Network (CNN) based system for the automatic segmentation of the colorectal cancer. The algorithm implemented consists of several steps: a pre-processing to normalize and highlights the tumoral area, the classification based on CNNs, and a post-processing aimed at reducing false positive elements. The classification is performed using three CNNs: each of them classifies the same regions of interest acquired from three different MR sequences. The final segmentation mask is obtained by a majority voting. Performances were evaluated using a semi-automatic segmentation revised by an experienced radiologist as reference standard. The system obtained Dice Similarity Coefficient (DSC) of 0.60, Precision (Pr) of 0.76 and Recall (Re) of 0.55 on the testing set. After applying the leave-one-out validation, we obtained a median DSC=0.58, Pr=0.74, Re=0.54. The promising results obtained by this system, if validated on a larger dataset, could strongly improve personalized medicine.