Optical scanning holography (OSH) involves the principles of optical scanning and heterodyning. The use of heterodyning leads to phase-preserving, which is the basic idea of holography. While heterodyning has numerous advantages, it requires complicated and expensive electronic processing. We investigate an off-axis approach to OSH, thereby eliminating the use of heterodyning for phase retrieval. We develop optical scanning theory for holographic imaging and show that by properly designing the scanning beam, we can perform coherent and incoherent holographic recording. Simulation results are provided to verify the proposed idea.