Capillary suspensions are ternary solid-liquid-liquid systems produced via the addition of a small amount of secondary fluid to the bulk fluid that contained the dispersed solid particles. The secondary fluid could exert strong capillary forces between the particles and dramatically change the rheological properties of the suspension. So far, research has focused on capillary suspensions that consist of additive-free fluids, whereas capillary suspensions with additives, particularly those of large molecular weight that are highly relevant for industrial purposes, have been relatively less studied. In this study, we performed a systematic analysis of the properties of capillary suspensions that consist of paraffin oil (bulk phase), water (secondary phase), and α-Al2O3 microparticles (particle phase), in which the aqueous secondary phase contained an important eco-friendly polymeric binder, sodium alginate (SA). It was determined that the yield stress of the suspension increased significantly with the increase in the SA content in the aqueous secondary phase, which was attributed to the synergistic effect of the capillary force and hydrogen bonding force that may be related to the increase in the number of capillary bridges. The amounts of SA used to induce a significant change in the yield stress in this study were very small (<0.02% of the total sample volume). The addition of Ca2+ ions to the SA-containing secondary phase further increased the yield stress with possible gelation of the SA chains-in the presence of excess Ca2+ ions, however, the yield stress decreased because of the microscopic phase separation that occurred in the aqueous secondary phase. The microstructures of the sintered porous materials that were produced by using capillary suspensions as precursors were qualitatively well correlated to the rheological behavior of the precursor suspensions, suggesting a new method for the subtle control of the microstructures of porous materials using the addition of minute amounts of polymeric additives.