With the rapid development of the world economy and increasing improvement of people's living standards, the number of diabetic patients has been growing quickly. Meanwhile, the complications of diabetes especially retinopathy have been affecting their daily life seriously. The only way to prevent it from getting worse and even leading to blindness is to make corresponding diagnosis as early as possible. However, it's extremely impossible for professionals to diagnose all the patients through their fundus images. It couldn't be better to solve the problem by automatic systems, so we present a novel network to learn the features of diabetic retinopathy (DR) and its complication diabetic macular edema (DME) and the relationship between them, focus on some vital areas in the pictures and eventually obtain the grades of the two diseases at the same time. Experimental results further prove the effectiveness of our proposed module comparing to the only joint grading network before.