Groups of people or even robots often face problems they need to solve together. Examples include collectively searching for resources, choosing when and where to invest time and effort, and many more. Although a hierarchical ordering of the relevance of the group members' inputs during collective decision making is abundant, a quantitative demonstration of its origin and advantages using a generic approach has not been described yet. Here we introduce a family of models based on the most general features of group decision making, and show that the optimal distribution of competences is a highly skewed function with a structured fat tail. Our results are obtained by optimizing the groups' compositions through identifying the best-performing distributions for both the competences and for the members' flexibilities/pliancies. Potential applications include choosing the best composition for a group intended to solve a given task.