Brain regions in Alzheimer's (AD) exhibit distinct vulnerability to the disease's hallmark pathology, with the entorhinal cortex and hippocampus succumbing early to tau tangles while others like primary sensory cortices remain resilient. The quest to understand how local/regional genetic factors, pathogenesis, and network-mediated spread of pathology together govern this selective vulnerability (SV) or resilience (SR) is ongoing. Although many risk genes in AD are known from gene association and transgenic studies, it is still not known whether and how their baseline expression signatures confer SV or SR to brain structures. Prior analyses have yielded conflicting results, pointing to a disconnect between the location of genetic risk factors and downstream tau pathology. We hypothesize that a full accounting of genes' role in mediating SV/SR would require the modeling of network-based vulnerability, whereby tau misfolds, aggregates, and propagates along fiber projections. We therefore employed an extended network diffusion model (eNDM) and tested it on tau pathology PET data from 196 AD patients from the Alzheimer's Disease Neuroimaging Initiative (ADNI). Thus the fitted eNDM model becomes a reference process from which to assess the role of innate genetic factors. Using the residual (observed - model-predicted) tau as a novel target outcome, we obtained its association with 100 top AD risk-genes, whose baseline spatial transcriptional profiles were obtained from the Allen Human Brain Atlas (AHBA). We found that while many risk genes at baseline showed a strong association with regional tau, many more showed a stronger association with residual tau. This suggests that both direct vulnerability, related to the network, as well as network-independent vulnerability, are conferred by risk genes. We then classified risk genes into four classes: network-related SV (SV-NR), network-independent SV (SV-NI), network-related SR (SR-NR), and network-independent SR (SR-NI). Each class has a distinct spatial signature and associated vulnerability to tau. Remarkably, we found from gene-ontology analyses, that genes in these classes were enriched in distinct functional processes and encompassed different functional networks. These findings offer new insights into the factors governing innate vulnerability or resilience in AD pathophysiology and may prove helpful in identifying potential intervention targets.