As of today, data analysis focuses primarily on the findings to be made inside the data and concentrates less on how those findings relate to the domain of investigation. Contemporary visualization as a field of research shows a strong tendency to adopt this data-centrism. Despite their decisive influence on the analysis result, qualitative aspects of the analysis process such as the structure, soundness, and complexity of the applied reasoning strategy are rarely discussed explicitly. We argue that if the purpose of visualization is the provision of domain insight rather than the depiction of data analysis results, a holistic perspective requires a qualitative component to to be added to the discussion of quantitative and human factors. To support this point, we demonstrate how considerations of qualitative factors in visual analysis can be applied to obtain explanations and possible solutions for a number of practical limitations inherent to the data-centric perspective on analysis. Based on this discussion of what we call qualitative visual analysis, we develop an inside-outside principle of nested levels of context that can serve as a conceptual basis for the development of visualization systems that optimally support the emergence of insight during analysis.