Cohesive interaction free energies entail an entropic component related to fluctuations of the energy associated with the attractive portion of the solute-solvent potential. The corresponding "fluctuation entropy" is fundamental in the solvation thermodynamics of macromolecular solutes and is linked to interfacial solvent density fluctuations and hydrophobic effects. Since the direct calculation of fluctuation entropy in molecular simulations is hampered by the poor sampling of high-energy tails in the solute-solvent energy distribution, indirect, and often approximate, routes for the calculation of fluctuation entropy are usually required, involving the modeling of geometrically frozen repulsive solute cavities in thermodynamic integration approaches. Herein, we propose a method to directly compute the fluctuation entropy by employing indirect umbrella sampling (INDUS). To validate the method, we consider model systems consisting of subnanometer oil droplets in water for which the fluctuation entropy can be computed exactly using indirect methods. The fluctuation entropy calculated with the newly proposed direct method agrees with the indirect reference calculations. We also observe that the solvation free energy and the contribution of the fluctuation entropy to it are of comparable magnitudes, particularly for larger oil droplets (∼1 nm). The proposed method can readily be employed for flexible macromolecular solutes and systems with extended hydrophobic surfaces or in the vicinity of a dewetting transition.