We numerically study the droplet coalescence of an oil-in-water (O/W) emulsion permeating through a fibrous filter. Our numerical simulation method is based on the phase-field model for capturing a free interface, the immersed boundary method used to calculate fluid-solid interactions, and the wetting model that assigns an order parameter to the solid surface according to the wettability. To represent realistic flow inside the filter during simulation, the voxel data obtained from X-ray computed tomography (CT) images of the filter microstructure are used in the simulation. The effects of the filter microstructure, such as fiber arrangement and orientation of the droplet coalescence, are investigated by using several filter domains. Our simulations demonstrate that the arrangement of closely attached fibers placed at the permeate-side surface enhances droplet coalescence. In addition, the parallel orientation of the fiber to the main flow direction suppresses droplet enlargement due to the coalescence but reduces the number of droplet passages without coalescence in the filter.