The combination of organic and inorganic nonlinear active units to obtain organic-inorganic hybrid materials has been proved to be a very effective method to obtain nonlinear optical (NLO) materials with excellent properties. Herein we reported two hybrid melamine-based compounds, namely, acentric (C3H7N6)6(H2PO4)4(HPO4)·4H2O (1) and centrosymmetric (C3H7N6)2SO4·2H2O (2), which were synthesized via facile solvent evaporation method. Compound 1 features a three-dimensional (3D) network structure composed of ∞[(H2PO4)4(HPO4)(H2O)4]6- layers which are further linked with ∞[(C3H7N6)6]6+ layers via hydrogen bonds. Compound 2 displays a 3D structure composed of [(C3H7N6)2(SO4)(H2O)2]∞ layers further linked with each other by hydrogen bonds. Compound 1 presents a second harmonic generation signal of about 0.1 × KDP. Furthermore, UV-vis and infrared spectra, thermal analyses, and theoretical calculation were also adopted to evaluate its NLO performance. The theoretical calculations showed that the SHG response and large birefringence of 1 were primarily caused by the (C3H7N6)+, (H2PO4)-, and (HPO4)2- groups.