Center-surround modulation in visual processing reflects a normalization process of contrast gain control in the responsive neurons. Prior adaptation to a clockwise (CW) tilted grating, for example, leads to the percept of counterclockwise tilt in a vertical grating, referred to as the tilt-aftereffect (TAE). We previously reported that the magnitude of the TAE is modulated by adding a same-orientation annular surround to an adapter, suggesting inhibitory lateral modulation. To further examine the property of this lateral modulation effect on the perception of a central target, we here used center-surround sinusoidal patterns as adapters and varied the adapter surround and center orientations independently. The target had the same spatial extent as the adapter center with no physical overlap with the adapter surround. Participants were asked to judge the target orientation as tilted either CW or counterclockwise from vertical after adaptation. Results showed that, when the surround orientation was held constant, the TAE magnitude was determined by the adapter center, peaking between 10° and 20° of tilt. More important, the adapter surround orientation modulated the adaptation effect such that the TAE magnitude first decreased and then increased as the surround orientation became increasingly more different from that of the center, suggesting that the surround modulation effect was indeed orientation specific. Our data can be accounted for by a divisive inhibition model, in which (1) the adaptation effect is represented by increasing the normalizing constant and (2) the surround modulation is captured by two multiplicative sensitivity parameters determined by the adapter surround orientation.