Present architecture of convolution neural network for diabetic retinopathy (DR-Net) is based on normal convolution (NC). It incurs high computational cost as NC uses a multiplicative weight that measures a combined correlation in both cross-channel and spatial dimension of layer's inputs. This might cause the overall DR-Net architecture to be over-parameterised and computationally inefficient. This paper proposes EDR-Net - a new end-to-end, DR-Net architecture with depth-wise separable convolution module. The EDR-Net architecture was trained with DRKaggle-train dataset (35,126 images), and tested on two datasets, i.e. DRKaggle-test (53,576 images) and Messidor-2 (1,748 images). Results showed that the proposed EDR-Net achieved predictive performance comparable with current state-of-the-arts in detecting referable diabetic retinopathy (rDR) from fundus images and outperformed other light weight architectures, with at least two times less computation cost. This makes it more amenable for mobile device based computer-assisted rDR screening applications.