Femtosecond laser pulses can be used to induce ultrafast changes of the magnetization in magnetic materials. However, one of the unsolved questions is that of conservation of the total angular momentum during the ultrafast demagnetization. Here we report the ultrafast transfer of angular momentum during the first hundred femtoseconds in ferrimagnetic Co0.8Gd0.2 and Co0.74Tb0.26 films. Using time-resolved X-ray magnetic circular dichroism allowed for time-resolved determination of spin and orbital momenta for each element. We report an ultrafast quenching of the magnetocrystalline anisotropy and show that at early times the demagnetization in ferrimagnetic alloys is driven by the local transfer of angular momenta between the two exchange-coupled sublattices while the total angular momentum stays constant. In Co0.74Tb0.26 we have observed a transfer of the total angular momentum to an external bath, which is delayed by ~150 fs.