This paper addresses the problem of relative pose estimation for multiple cameras in the context of motion-based camera calibration. Relative pose is found from a set of camera-target relative poses. A least-square kind loss function is established using 3D relative Riemannian metrics of targets and cameras. Lie algebra-based solvers are proposed for orientation and pose estimation. Multiple cameras are calibrated without common targets or features in space using our proposed framework. The simulation and real data evaluation demonstrate that our proposed algorithms can achieve high accuracy with limited and precise camera-target poses.