Terahertz (THz) emissions from fast electron and ion currents driven in relativistic, femtosecond laser-foil interactions are examined theoretically. We first consider the radiation from the energetic electrons exiting the backside of the target. Our kinetic model takes account of the coherent transition radiation due to these electrons crossing the plasma-vacuum interface as well as of the synchrotron radiation due to their deflection and deceleration in the sheath field they set up in vacuum. After showing that both mechanisms tend to largely compensate each other when all the electrons are pulled back into the target, we investigate the scaling of the net radiation with the sheath field strength. We then demonstrate the sensitivity of this radiation to a percent-level fraction of escaping electrons. We also study the influence of the target thickness and laser focusing. The same sheath field that confines most of the fast electrons around the target rapidly sets into motion the surface ions. We describe the THz emission from these accelerated ions and their accompanying hot electrons by means of a plasma expansion model that allows for finite foil size and multidimensional effects. Again, we explore the dependencies of this radiation mechanism on the laser-target parameters. Under conditions typical of current ultrashort laser-solid experiments, we find that the THz radiation from the expanding plasma is much less energetic-by one to three orders of magnitude-than that due to the early-time motion of the fast electrons.