Oil migration in chocolate and chocolate-based confections leads to undesirable visual and textural changes. Establishing ways to slow this unavoidable process would increase shelf life and reduce consumer rejection. Diffusion is most often credited as the main pathway by which oil migration occurs. Here, we use fluorescence recovery after photobleaching (FRAP) to explore the diffusion coefficients of vegetable and mineral oil through fat crystal networks at different solid fat contents (SFC). Differences in compatibility between the fat and oil lead to unique primary crystal clusters, yet those variations do not affect diffusion at low SFCs. Trends deviate at higher SFCs, which we ascribe to the influence of the differing crystal cluster structures. We relate our results to the strong and weak-link rheological regimes of fat crystal networks. Finally, we connect the results to relationships developed for polymer gel systems.