We study the thermal expansion of chains formed by self-assembly of magnetic colloidal particles in a magnetic field. Using video microscopy, complete positional data of all the particles of the chains is obtained. By changing the ionic strength of the solution and the applied magnetic field, the interaction potential can be tuned. We analyze the thermal expansion of the chain using a simple model of a one-dimensional anharmonic crystal of finite size.