Strong anisotropy of photoluminescence of a (100)-cut β-Ga2O3 and a Mg-doped β-Ga2O3 single crystals was found in UV and visible spectral range, the bands of which were attributed to different types of transitions in the samples. Green photoluminescence in the Mg-doped sample was enhanced approximately twice. A remarkable enhancement of two-photon absorption and self-focusing in β-Ga2O3 after doping was revealed by 340-fs laser Z-scanning at 515 nm. The absolute value of complex third order susceptibility χ(3) determined from the study increases by 19 times in [001] lattice direction. Saturable absorption and associated self-defocusing were found in the undoped crystal in the [010] direction, which was explained by the anisotropic excitation of F-centers on intrinsic oxygen defects. This effect falls out of resonance in the Mg-doped crystal. The χ(3) values which are provided by a decrease of bandgap in Mg-doped β-Ga2O3 are χ(3) [001] = 1.85·10-12 esu and χ(3) [010]=χ(3)yyyy = 0.92·10-12 esu. Our result is only one order of magnitude lower than the best characteristic in green demonstrated by a Mg-doped GaN, which encourages subsequent development of Mg-doped β-Ga2O3 as an effective nonlinear optical material in this region.