Diabetic Retinopathy (DR) is a disease which affect the vision ability. The observation by an ophthalmologist usually conducted by analyzing the retinal images of the patient which are marked by some DR features. However some misdiagnosis are usually found due to human error. Here, a deep learning-based low-cost embedded system is established to assist the doctor for grading the severity of the DR from the retinal images. A compact deep learning algorithm named Deep-DR-Net which fits on a small embedded board is afterwards proposed for such purposes. In the heart of Deep-DR-Net, a cascaded encoder-classifier network is arranged using residual style for ensuring the small model size. The usage of different types of convolutional layers subsequently guarantees the features richness of the network for differentiating the grade of the DR. Experimental results show the capability of the proposed system for detecting the existence as well as grading the severity of the DR symptomps.