To answer major questions of cell biology, it is often essential to understand the complex phenotypic composition of cellular systems precisely. Modern automated microscopes produce vast amounts of images routinely, making manual analysis nearly impossible. Due to their efficiency, machine learning-based analysis software have become essential tools to perform single-cell-level phenotypic analysis of large imaging datasets. However, an important limitation of such methods is that they do not use the information gained from the cellular micro- and macroenvironment: the algorithmic decision is based solely on the local properties of the cell of interest. Here, we present how various features from the surrounding environment contribute to identifying a cell and how such additional information can improve single-cell-level phenotypic image analysis. The proposed methodology was tested for different sizes of Euclidean and nearest neighbour-based cellular environments both on tissue sections and cell cultures. Our experimental data verify that the surrounding area of a cell largely determines its entity. This effect was found to be especially strong for established tissues, while it was somewhat weaker in the case of cell cultures. Our analysis shows that combining local cellular features with the properties of the cell's neighbourhood significantly improves the accuracy of machine learning-based phenotyping.