Structural colors produced from nanostructures have attracted much attention due to their promising advantages of long-term stability and high resolution. Many nanostructures like metasurfaces have been demonstrated to generate color information in the transmission or reflection mode. Here, a strategy of combining polarization-insensitive and polarization-sensitive transmissive structural color is proposed to realize convenient and diverse encrypted pattern designs. A two-dimensional metasurface, whose polarization characteristics are determined by the size of a nanobrick unit, is embedded inside an optical cavity to produce transmissive structural color. The polarization-insensitive transmissive structural color exhibits a wide color gamut and high excitation purity in all polarization states, while the polarization-sensitive transmissive structural color maintains the similar color appearance in x-direction polarization but appears nearly black in y-direction polarization. Combining these two transmissive structural colors can achieve diverse images designed at different polarizations instead of simply hiding the image in a specific polarization state. An image of "flower and flowerpot" using the generated colors is visually illustrated, which shows that the proposed transmissive structural colors would have great potential in the areas of security information encryption.