We introduce an image-based algorithm to find the probability density function (PDF) of particle displacements from a sequence of images. Conventionally methods based on cross correlation (CC) of image ensembles estimate the standard deviation of an assumed Gaussian PDF from the width of the CC peak. These methods are subject to limiting assumptions that the particle intensity profile and distribution of particle displacements are both Gaussian. Here, we introduce an approach to image-based probability estimation of displacement (iPED) without making any assumptions about the shape of particles' intensity profile or the PDF of the displacements. In addition, we provide a statistical convergence criterion for iPED to achieve an accurate estimate of the underlying PDF. We compare iPED's performance with the previous CC method for both Gaussian and non-Gaussian particle intensity profiles undergoing Gaussian or non-Gaussian processes. We validate iPED using synthetic images and show that it accurately resolves the PDF of particle displacements with no underlying assumptions. Finally, we demonstrate the application of iPED to real experimental data sets and evaluate its performance. In conclusion, this work presents a method for the estimation of the probability density function of random displacements from images. This method is generalized and independent of any assumptions about the underlying process and is applicable to any moving objects of any arbitrary shape.