The effects of the individual scattering process on the formations of both the particle hologram and its corresponding reconstructed three-dimensional particle image are investigated using the Debye series. A particle hologram model using the Debye series decomposes the object wave into different scattering modes and thus permits evaluating the effects of the individual scattering process [diffraction, reflection, transmission, refractions with (p-1) internal reflections] on the particle holography quantitatively. In the Gabor inline holography of a transparent droplet, the transmission light causes small discrepancies between the hologram fringes of an opaque particle (diffraction) and a transparent particle near the zero point of the Bessel-like modulation function, eventually giving rise to the glory spot in the center of the reconstructed dark particle image. For off-axis holography, this paper reveals the effects of reflection, particularly total reflection by bubbles, transmission, and refractions with (p-1) internal reflections of the scattered light on the formation and the reconstructed glory spot images of typical forward and backward off-axis holography.