The surface free energy, or surface tension, of a liquid interface gives rise to a pressure jump when the interface is curved. Here we show that a similar capillary pressure arises at the interface of soft solids. We present experimental evidence that immersion of a thin elastomeric wire into a liquid induces a substantial elastic compression due to the solid capillary pressure at the bottom. We quantitatively determine the effective surface tension from the elastic displacement field and find a value comparable to the liquid-vapor surface tension. Most importantly, these results also reveal the way the liquid pulls on the solid close to the contact line: the capillary force is not oriented along the liquid-air interface, nor perpendicularly to the solid surface, as previously hypothesized, but towards the interior of the liquid.