Stimulated by the discovery of the iron oxypnictide superconductor with ZrCuSiAs-type structure in 2008, extensive exploration of its isostructural and isoelectronic compounds has started. These compounds, including oxides, fluorides, and hydrides, can all be simply recognized as valence compounds for which the octet rule is valid. We report herein the first example of a ZrCuSiAs-type hydride, CaNiGeH, which violates the octet rule. This hydride was synthesized by hydrogenation of the CeFeSi-type compound CaNiGe under pressurized hydrogen. Powder diffraction and theoretical simulation confirm that H enters into the interstitial position of the Ca(4) tetrahedron, leading to notable anisotropic expansion of the unit cell along the c axis. Density functional theory calculations indicate the modification of the chemical bonding and formation of ionic Ca-H bond as a result of hydrogen insertion. Furthermore, CaNiGeH shows Pauli paramagnetism and metallic conduction similar to that of CaNiGe, but its carrier type changes to hole and the carrier density is drastically reduced as compared to CaNiGe. Mn-doping at the Ni site introduces magnetism to both the parent compound and the hydride. The measurement demonstrates that hydrogenation of CaNi(1-x)Mn(x)Ge reduces ferromagnetic ordering of Mn ions and induces huge magnetic hysteresis, whereas the spin glass state observed for the parent compound is preserved in the hydride. The hydrogenation-induced changes in the electric and magnetic properties are interpreted in terms of development of two-dimensionality in crystal structure as well as electronic state.