The fifth harmonic (162 nm, 11 fs), generated in a short argon cell from 12 fs Ti-sapphire laser pulses, was used to excite C2H4 and C2D4 in the maximum of the first pi pi* transition. Around 10% of the molecules were excited to the pi3s Rydberg state instead. The subsequent motion of the wave packet, moving over the potentials from the Franck-Condon region down to the ground state, was monitored by nonresonant ionization at 810 nm with mass-selective detection of the ion yield. Five time constants (from approximately 20 fs in excited states to 0.6-11 ps in the hot ground state) and four coherent oscillations (CC stretch and torsion vibrations or hindered free rotation) were determined for each isotopomer. The initial relaxation follows a superposition of CC twist and stretch coordinates; this explains a surprisingly small deuterium isotope effect of the initial time constant (21 versus 24 fs). Also the vibrations in the Franck-Condon region have such a mixed character and a correspondingly small isotope shift. From the perpendicular minimum the wave packet reaches (within 17 or 21 fs for the two isotopomers) a conical intersection via a direction that also involves partial hydrogen migration. This is concluded from the detection of ethylidene (CH3CH), formed simultaneously with ground-state ethylene. This carbene isomerizes in the ground state within 0.6 ps (1.6 ps for CD3CD) to ethylene. Two time constants for dissociation (4.5 and 11 ps) in the hot ground state were also identified. The small yields of bimolecular reactions (photodimerization, addition reactions involving a "suddenly polarized" excited state, carbene reactions) are interpreted in terms of the short lifetimes. It is pointed out that the relaxation path starting from the Rydberg state merges into that from the pi pi* state; nevertheless, there is a wavelength dependence in the photochemistry of olefins, because due to a momentum effect the wave packet remembers from which state it came.