Dynamic object perception is an important yet challenging direction in the field of robot navigation. Without any prior knowledge about motion and objects, a novel dynamic rigid bodies mining and motion estimation method based on monocular camera is proposed in this article. Different from the existing works based on sampling that associate feature points to motion hypotheses according to the reprojection errors, our work endeavors to find the intrinsic relevance among motion hypotheses. To represent this relevance, the concept of the probabilistic field on the Lie group Sim(3) manifold is introduced, which is established using random sampling. It provides a computable way for the regions on the manifold where rigid bodies possibly appear. The probability of a motion hypothesis falling on a region is expressed by its confidence. The regions with large confidences in the probabilistic field are selected as potential rigid bodies, whose corresponding feature points are further sampled for pose calculation. As a result, the randomness of sampling is reduced and the inliers for possible rigid bodies are enhanced, which guarantees the accuracy of motion estimation. On this basis, the tracking of rigid bodies is achieved. The proposed method distinguishes the feature points of dynamic objects with 3-D motion from those in the static background, thus enabling simultaneous localization and mapping (SLAM) to be initialized in dynamic environments. The experimental results on the KITTI, Hopkins 155, and MTPV62 datasets demonstrate the effectiveness. Comparison experiments indicate that our method outperforms the other methods in sensitivity of dynamic objects perception.