Poly(3,4-ethylenedioxythiophene) (PEDOT) semiconductor plays a relevant role in the development of organic thermoelectric (TE) devices for low-power generation. While dopant counterions are usually needed to provide electrical conductivity, their overall effects on the thermoelectric response of the systems are unknown and uncontrolled. Here, we present a first principles study of the electronic and thermal transport of PEDOT crystalline assemblies, specifically analysing the role played by tosylate dopants on the thermoelectric figure of merit of the doped system. Our results demonstrate that, beside the desired charging effect, the presence of dopants impacts the bulk configuration by inflating the packing structure and worsening the intrinsic transport properties of the PEDOT host. This provides a rationale for the necessity of controlling the optimal amount and the structural incorporation of dopant in order to maximize the thermoelectric response of organic materials.