Realization of distributed quantum systems requires fast generation and long-term storage of quantum states. Ground atomic states enable memories with storage times in the range of a minute, however their relatively weak interactions do not allow fast creation of non-classical collective states. Rydberg atomic systems feature fast preparation of singly excited collective states and their efficient mapping into light, but storage times in these approaches have not yet exceeded a few microseconds. Here we demonstrate a system that combines fast quantum state generation and long-term storage. An initially prepared coherent state of an atomic memory is transformed into a non-classical collective atomic state by Rydberg-level interactions in less than a microsecond. By sheltering the quantum state in the ground atomic levels, the storage time is increased by almost two orders of magnitude. This advance opens a door to a number of quantum protocols for scalable generation and distribution of entanglement.