The dynamics of cardiac fibrillation can be described by the number, the trajectory, the stability, and the lifespan of phase singularities (PSs). Accurate PS tracking is straightforward in simple uniform tissues but becomes more challenging as fibrosis, structural heterogeneity, and strong anisotropy are combined. In this paper, we derive a mathematical formulation for PS tracking in two-dimensional reaction-diffusion models. The method simultaneously tracks wavefronts and PS based on activation maps at full spatiotemporal resolution. PS tracking is formulated as a linear assignment problem solved by the Hungarian algorithm. The cost matrix incorporates information about distances between PS, chirality, and wavefronts. A graph of PS trajectories is generated to represent the creations and annihilations of PS pairs. Structure-preserving graph transformations are applied to provide a simplified description at longer observation time scales. The approach is validated in 180 simulations of fibrillation in four different types of substrates featuring, respectively, wavebreaks, ionic heterogeneities, fibrosis, and breakthrough patterns. The time step of PS tracking is studied in the range from 0.1 to 10 ms. The results show the benefits of improving time resolution from 1 to 0.1 ms. The tracking error rate decreases by an order of magnitude because the occurrence of simultaneous events becomes less likely. As observed on PS survival curves, the graph-based analysis facilitates the identification of macroscopically stable rotors despite wavefront fragmentation by fibrosis.