Multilayered structures extensively studied as a novel type of substrates for surface acoustic wave (SAW) devices are characterized by an asymmetry of wave propagation: acoustic wave characteristics generally change with inversion of propagation direction or interchange of top/bottom surfaces in one of the layers, though separately each material is symmetric for such inversions. In this article, the matrix formalism known as an effective tool for theoretical and numerical investigation of acoustic wave propagation in multilayered structures is applied to explain the existence of asymmetry and analyze its relation to the symmetry and orientations of combined materials. This phenomenon is illustrated by the examples of layered structures combining LiTaO3 (LT) plate with quartz or Si, previously reported as potential substrates for SAW devices with improved performance. Asymmetry arises from anisotropy of combined materials and occurs even when one of these materials is non-piezoelectric. It was estimated numerically as a variation of SAW resonator characteristics with substrate or plate inversion and was analyzed as a function of plate or substrate orientation. In particular, it was shown that "polarity inverted" structure enabling alternative resonator performance for the same material layers can be obtained either by an interchange of top/bottom surfaces of a piezoelectric plate or by inversion of propagation direction in a substrate. Asymmetry decreases with the introduction of an isotropic layer at the plate-substrate interface.