In the neurosciences of music, a consensus on the nature of affective states during music listening has not been reached. What is undeniable is that subjective affective states can be triggered by various and even opposite musical events. Here we review the few recent studies on the neural determinants of subjective affective processes of music, contrasted with early automatic neural processes linked to the objective universal properties of music. In particular, we focus on the evaluative judgments of music by subjects according to its aesthetic and structural values, on music-specific emotions felt by listeners, and on conscious liking. We then discuss and seek to stimulate further research on the interplay between the emotional attributes of music and the subjective cognitive, psychological, and biographic factors, such as personality traits and cognitive strategies of listening. We finally draw the neuroscientist's attention to the sociocultural context as a relevant variable to study when considering music as an aesthetic domain.