Aphids, brown spots, mosaics, rusts, powdery mildew and Alternaria blotches are common types of early apple leaf pests and diseases that severely affect the yield and quality of apples. Recently, deep learning has been regarded as the best classification model for apple leaf pests and diseases. However, these models with large parameters have difficulty providing an accurate and fast diagnosis of apple leaf pests and diseases on mobile terminals. This paper proposes a novel and real-time early apple leaf disease recognition model. AD Convolution is firstly utilized to replace standard convolution to make smaller number of parameters and calculations. Meanwhile, a LAD-Inception is built to enhance the ability of extracting multiscale features of different sizes of disease spots. Finally, the LAD-Net model is built by the LR-CBAM and the LAD-Inception modules, replacing a full connection with global average pooling to further reduce parameters. The results show that the LAD-Net, with a size of only 1.25MB, can achieve a recognition performance of 98.58%. Additionally, it is only delayed by 15.2ms on HUAWEI P40 and by 100.1ms on Jetson Nano, illustrating that the LAD-Net can accurately recognize early apple leaf pests and diseases on mobile devices in real-time, providing portable technical support.