Accurate clinical target volume (CTV) delineation is essential to ensure proper tumor coverage in radiation therapy. This is a particularly difficult task for head-and-neck cancer patients where detailed knowledge of the pathways of microscopic tumor spread is necessary. This paper proposes a solution to auto-segment these volumes in oropharyngeal cancer patients using a two-channel 3D U-Net architecture. The first channel feeds the network with the patient's CT image providing anatomical context, whereas the second channel provides the network with tumor location and morphological information. Radiation therapy simulation computer tomography scans and their corresponding manually delineated CTV and gross tumor volume (GTV) delineations from 285 oropharyngeal patients previously treated at MD Anderson Cancer Center were used in this study. CTV and GTV delineations underwent rigorous group peer-review prior to the start of treatment delivery. The convolutional network's parameters were fine-tuned using a training set of 210 patients using 3-fold cross-validation. During hyper-parameter selection, we use a score based on the overlap (dice similarity coefficient (DSC)) and missed volumes (false negative dice (FND)) to minimize any possible under-treatment. Three auto-delineated models were created to estimate tight, moderate, and wide CTV margin delineations. Predictions on our test set (75 patients) resulted in auto-delineations with high overlap and close surface distance agreement (DSC > 0.75 on 96% of cases for tight and moderate auto-delineation models and 97% of cases having mean surface distance ⩽ 5.0 mm) to the ground-truth. We found that applying a 5 mm uniform margin expansion to the auto-delineated CTVs would cover at least 90% of the physician CTV volumes for a large majority of patients; however, determination of appropriate margin expansions for auto-delineated CTVs merits further investigation.