Due to inappropriate agricultural management practices, soil erosion is becoming one of the most dangerous forms of soil degradation in many olive farming areas in the Mediterranean region, leading to significant decrease of soil fertility and yield. In order to prevent further soil degradation, proper measures are necessary to be locally implemented. In this perspective, an increase in the spatial accuracy of remote sensing datasets and advanced image analysis are significant tools necessary and efficient for mapping soil erosion risk on a fine scale. In this study, the Revised Universal Soil Loss Equation (RUSLE) was implemented in the spatial domain using GIS, while a very high resolution satellite image, namely a QuickBird image, was used for deriving cover management (C) and support practice (P) factors, in order to map the risk of soil erosion in Kolymvari, a typical olive farming area in the island of Crete, Greece. The results comprised a risk map of soil erosion when P factor was taken uniform (conventional approach) and a risk map when P factor was quantified site-specifically using object-oriented image analysis. The results showed that the QuickBird image was necessary in order to achieve site-specificity of the P factor and therefore to support fine scale mapping of soil erosion risk in an olive cultivation area, such as the one of Kolymvari in Crete. Increasing the accuracy of the QB image classification will further improve the resulted soil erosion mapping.