Seed development starts at double fertilization when two sperms fuse with a female gamete, the egg and central cell, giving rise to the embryo and endosperm, respectively. Uniting two parental genomes into one, unique, zygotic genome is certainly the first event requiring large-scale chromatin modifications and remodeling. Although little is known about the molecular mechanisms, recent progress was made allowing live imaging of parental chromatin dynamics at fertilization. Further growth and patterning processes will shape the future plant seedling and its surrounding nurse tissue. Studies over the last decade identified key chromatin modifiers involved in these processes. However, the dynamics of these modifications mediated, in particular, by the Polycomb group complexes only start to be understood. The precise molecular mechanisms altering chromatin state in relation to early seed development remains difficult to address because of the relative inaccessibility of the fertilization products. Nonetheless, with the emergence of in vivo imaging techniques, laser-capture dissection, and genome-wide chromatin modification profiling, the future promises new, exciting discoveries.