A general thermodynamic framework is presented for open quantum systems in fixed contact with a thermal reservoir. The first and second law are obtained for arbitrary system-reservoir coupling strengths, and including both factorized and correlated initial conditions. The thermodynamic properties are adapted to the generally strong coupling regime, approaching the ones defined for equilibrium, and their standard weak-coupling counterparts as appropriate limits. Moreover, they can be inferred from measurements involving only system observables. Finally, a thermodynamic signature of non-Markovianity is formulated in the form of a negative entropy production rate.