We present three-dimensional simulations of HgCdTe-based focal plane arrays (FPAs) with two-color and dual-band sequential infrared pixels having realistic truncated-pyramid shape, taking into account also the presence of compositionally graded transition layers. After a validation against the spectral responsivity of two-color, mid-wavelength infrared detectors from the literature, the method is employed for a simulation campaign on dual-band, mid-, and long-wavelength infrared FPAs illuminated by a Gaussian beam. Simulation results underscore the importance of a full-wave approach to the electromagnetic problem, since multiple internal reflections due to metallizations and slanted sidewalls produce non-negligible features in the quantum efficiency spectra, especially in the long-wavelength band. Evaluations of the optical and diffusive contribution to inter-pixel crosstalk indicate the effectiveness of deep trenches to prevent diffusive crosstalk in both wavebands. In its present form, the detector seems to be subject to significant optical crosstalk in the long-wavelength infrared band, which could be addressed through pixel shape optimization.