Biomarkers are one of the primary medical signs to facilitate the early detection of Alzheimer's disease. The small beta-amyloid (Aβ) peptide is an important indicator for the disease. However, current methods to detect Aβ pathology are either invasive (lumbar puncture) or quite costly and not widely available (amyloid PET). Thus a less invasive and cheaper approach is demanded. MRI which has been used widely in preclinical AD has recently shown the capability to predict brain Aβ positivity. This motivates us to develop a method, SDF sparse convolution, taking MRI to predict Aβ positivity. We obtain subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and use our method to discriminate Aβ positivity. Theoretically, we provide analysis towards the understanding of what the network has learned. Empirically, it shows strong performance on par or even better than state of the art.