Benefiting from advances in nanofabrication technology, emerging metasurfaces are promising for compact and wearable multicolor meta-holograms with large fields of view. However, due to the inherent electromagnetic properties of the structures that are used, current multicolor meta-holograms are often sensitive to the incident light polarization, which greatly restricts the application of meta-holography. Here, we took advantage of the amplitude properties of metasurfaces and the off-axis illumination method to carry out experiments involving polarization-insensitive colorful meta-holography with anisotropic nanostructures. With red, green and blue lasers illuminating the meta-hologram along different angles, a polarization-insensitive colorful holographic image was achieved and the disturbance from zero-order diffraction light was essentially eliminated. To the best of our knowledge, the current work was the first time that a polarization-insensitive colorful meta-hologram with anisotropic nanostructures was experimentally demonstrated. We expect our approach to provide promising prospects for the use of metasurfaces in applications such as flat meta-lenses, data storage and virtual/augmented reality.