Contagion, a concept from epidemiology, has long been used to characterize social influence on people's behavior and affective (emotional) states. While it has revealed many useful insights, it is not clear whether the contagion metaphor is sufficient to fully characterize the complex dynamics of psychological states in a social context. Using wearable sensors that capture daily face-to-face interaction, combined with three daily experience sampling surveys, we collected the most comprehensive data set of personality and emotion dynamics of an entire community of work. From this high-resolution data about actual (rather than self-reported) face-to-face interaction, a complex picture emerges where contagion (that can be seen as adaptation of behavioral responses to the behavior of other people) cannot fully capture the dynamics of transitory states. We found that social influence has two opposing effects on states: adaptation effects that go beyond mere contagion, and complementarity effects whereby individuals' behaviors tend to complement the behaviors of others. Surprisingly, these effects can exhibit completely different directions depending on the stable personality or emotional dispositions (stable traits) of target individuals. Our findings provide a foundation for richer models of social dynamics, and have implications on organizational engineering and workplace well-being.