We revealed the origin of the structural features of a series of tetrasilacyclobutadiene analogues based on a detailed study of their electronic structure and the pseudo Jahn-Teller effect (PJTE). Starting with the D4h symmetry of the Si4R4 system with a square four-membered silicon ring as a reference geometry, and employing ab initio calculations of energy profiles along lower-symmetry nuclear displacements in the ground and several excited states, we show that the ground-state boat-like and chair-like equilibrium configurations are produced by the PJT interaction with appropriate excited sates. For Si4F4 a full two-mode b1g-b2g adiabatic potential energy surface is calculated showing explicitly the way of transformation from the unstable D4h geometry to the two equilibrium C2h configurations via the D2h saddle point. The PJTE origin of these structural features is confirmed also by estimates of the vibronic coupling parameters. For Si4R4 with large substituents the origin of their structure is revealed by analyzing the PJT interaction between the frontier molecular orbitals. The preferred chair-like structures of Si4R4 analogues with amido substituents, and heavier germanium-containing systems Ge4R4 (potential precursors for semiconducting materials) are predicted.