Single crystals of CeGe and its non-magnetic analog LaGe have been grown by the Czochralski method. The CeGe compound crystallizes in the orthorhombic FeB-type crystal structure with the space group Pnma (#62). The anisotropic magnetic properties have been investigated for well oriented single crystals by measuring the magnetic susceptibility, electrical resistivity and heat capacity. It has been found that CeGe orders antiferromagnetically at 10.5 K. Both transport and magnetic studies have revealed large anisotropy, reflecting the orthorhombic crystal structure. The magnetization data at 1.8 K reveal metamagnetic transitions along the [010] direction at 4.8 and 6.4 T and along the [100] direction at a critical field of 10.7 T, while the magnetization along the [001] direction increases linearly without any anomaly up to a field of 16 T. From the magnetic susceptibility and the magnetization measurements it has been found that the [010] direction is the easy axis of magnetization. The electrical resistivity along the three crystallographic directions exhibits an upturn at T(N), indicating superzone gap formation below T(N) in this compound. We have performed crystal electric field analysis on the magnetic susceptibility and the heat capacity data and found that the ground state is a doublet, and the energies of splitting from the ground state to the first and second excited doublet states were estimated to be 39 and 111 K, respectively.