Endogenous retroviruses (ERVs), which are abundant in mammalian genomes, can modulate the expression of nearby genes, and their expression is dynamic and stage-specific during early embryonic development in mice and humans. However, the functions and mechanisms of ERV elements in regulating embryonic development remain unclear. Here, we utilized several methods to determine the contribution of ERVs to the makeup and regulation of transcripts during embryonic genome activation (EGA). We constructed an ERV library and embryo RNA-seq library (IVF_2c and IVF_8c) of goat to serve as our research basis. The GO and KEGG analysis of nearby ERV genes revealed that some ERV elements may be associated with embryonic development. RNA-seq results were consistent with the features of EGA. To obtain the transcripts derived from the ERV sequences, we blasted the ERV sequences with embryonic transcripts and identified three lncRNAs and one mRNA that were highly expressed in IVF-8c rather than in IVF-2c (q-value <0.05). Then, we validated the expression patterns of nine ERV-related transcripts during early developmental stages and knocked down three high-expression transcripts in EGA. The knockdown of lncRNA TCONS_00460156 or mRNA HSD17B11 significantly decreased the developmental rate of IVF embryos. Our findings suggested that some transcripts from ERVs are essential for the early embryonic development of goat, and analyzing the ERV expression profile during goat EGA may help elucidate the molecular mechanisms of ERV in regulating embryonic development.