The influence of the structural detail and defects on the thermal and electronic transport properties of graphene nanoribbons (GNRs) is explored by molecular dynamics and non-equilibrium Green's function methods. A variety of randomly oriented and distributed defects, single and double vacancies, Stone-Wales defects, as well as two types of edge form (armchair and zigzag) and different edge roughnesses are studied for model systems similar in sizes to experiments (>100 nm long and >15 nm wide). We observe substantial reduction in thermal conductivity due to all forms of defects, whereas electrical conductance reveals a peculiar defect-type-dependent response. We find that a 0.1% single vacancy concentration and a 0.23% double vacancy or Stone-Wales concentration lead to a drastic reduction in thermal conductivity of GNRs, namely, an 80% reduction from the pristine one of the same width. Edge roughness with an rms value of 7.28 Å leads to a similar reduction in thermal conductivity. Randomly distributed bulk vacancies are also found to strongly suppress the ballistic nature of electrons and reduce the conductance by 2 orders of magnitude. However, we have identified that defects close to the edges and relatively small values of edge roughness preserve the quasi-ballistic nature of electronic transport. This presents a route of independently controlling electrical and thermal transport by judicious engineering of the defect distribution; we discuss the implications of this for thermoelectric performance.