X-ray in-line phase contrast imaging relies on the measurement of Fresnel diffraction intensity patterns due to the phase shift and the attenuation induced by the object. The recovery of phase and attenuation from one or several diffraction patterns is a nonlinear ill-posed inverse problem. In this work, we propose supervised learning approaches using mixed scale dense (MS-D) convolutional neural networks to simultaneously retrieve the phase and the attenuation from x-ray phase contrast images. This network architecture uses dilated convolutions to capture features at different image scales and densely connects all feature maps. The long range information in images becomes quickly available, and greater receptive field size can be obtained without losing resolution. This network architecture seems to account for the effect of the Fresnel operator very efficiently. We train the networks using simulated data of objects consisting of either homogeneous components, characterized by a fixed ratio of the induced refractive phase shifts and attenuation, or heterogeneous components, consisting of various materials. We also train the networks in the image domain by applying a simple initial reconstruction using the adjoint of the Fréchet derivative. We compare the results obtained with the MS-D network to reconstructions using U-Net, another popular network architecture, as well as to reconstructions using the contrast transfer function method, a direct phase and attenuation retrieval method based on linearization of the direct problem. The networks are evaluated using simulated noisy data as well as images acquired at NanoMAX (MAX IV, Lund, Sweden). In all cases, large improvements of the reconstruction errors are obtained on simulated data compared to the linearized method. Moreover, on experimental data, the networks improve the reconstruction quantitatively, improving the low-frequency behavior and the resolution.