The thermoelectric properties of graphene and graphene nanostructures have recently attracted significant attention from the physics and engineering communities. In fundamental physics, the analysis of Seebeck and Nernst effects is very useful in elucidating some details of the electronic band structure of graphene that cannot be probed by conductance measurements alone, due in particular to the ambipolar nature of this gapless material. For applications in thermoelectric energy conversion, graphene has two major disadvantages. It is gapless, which leads to a small Seebeck coefficient due to the opposite contributions of electrons and holes, and it is an excellent thermal conductor. The thermoelectric figure of merit ZT of a two-dimensional (2D) graphene sheet is thus very limited. However, many works have demonstrated recently that appropriate nanostructuring and bandgap engineering of graphene can concomitantly strongly reduce the lattice thermal conductance and enhance the Seebeck coefficient without dramatically degrading the electronic conductance. Hence, in various graphene nanostructures, ZT has been predicted to be high enough to make them attractive for energy conversion. In this article, we review the main results obtained experimentally and theoretically on the thermoelectric properties of graphene and its nanostructures, emphasizing the physical effects that govern these properties. Beyond pure graphene structures, we discuss also the thermoelectric properties of some hybrid graphene structures, as graphane, layered carbon allotropes such as graphynes and graphdiynes, and graphene/hexagonal boron nitride heterostructures which offer new opportunities. Finally, we briefly review the recent activities on other atomically thin 2D semiconductors with finite bandgap, i.e. dichalcogenides and phosphorene, which have attracted great attention for various kinds of applications, including thermoelectrics.