Many animal and human societies exhibit hierarchical structures with different degrees of steepness. Some of these societies also show cooperative behavior, where cooperation means working together for a common benefit. However, there is an increasing evidence that rigidly enforced hierarchies lead to a decrease of cooperation in both human and non-human primates. In this work, we address this issue by means of an evolutionary agent-based model that incorporates fights as social interactions governing a dynamic ranking, communal work to produce a public good, and norm internalization, i.e. a process where acting according to a norm becomes a goal in itself. Our model also includes the perception of how much the individual is going to retain from her cooperative behavior in future interactions. The predictions of the model resemble the principal characteristics of human societies. When ranking is unconstrained, we observe a high concentration of agents in low scores, while a few ones climb up the social hierarchy and exploit the rest, with no norm internalization. If ranking is constrained, thus leading to bounded score differences between agents, individual positions in the ranking change more, and the typical structure shows a division of the society in upper and lower classes. In this case, we observe that there is a significant degree of norm internalization, supporting large fractions of the population cooperating in spite of the rank differences. Our main results are robust with respect to the model parameters and to the type of rank constraint. We thus provide a mechanism that can explain how hierarchy arises in initially egalitarian societies while keeping a large degree of cooperation.