We propose a quantitative phase imaging system with exact slightly-off-axis configuration and suitable for objects both smaller and larger than the size of the effective recording region of the image sensors. In this system, the object is illuminated by a convergent spherical beam and a specially designed aperture filter is placed on the spatial frequency plane of the object wave; at the same time, a point source emitting from the edge of the aperture is taken as the reference beam, so that the optimal frequency condition for reconstruction of slightly-off-axis digital holograms can be always guaranteed for both large and small objects as well as different magnification (or the field of view) configurations. At the same time, a 1x2 single-mode optical fiber splitter is used for generating the reference and the illumination beams. Benefited from such fiber-based slightly-off-axis design, the proposed system provides a low-cost way to convert a regular microscope into a slightly-off-axis holographic one for microbiological specimens with a high spatial resolution.