We study the formation and structure of stable electrostatic complexes between polyanions (DNA and poly(styrene-sulfonate)) and linear polyethylenimine. The charge ratio x of the mixture is tuned by varying the concentration of the polycation at constant concentration of polyanion. In agreement with recent theories, dynamic light scattering and electrophoretic mobility measurements show two distinct regimes of weak and strong complexation. At low polycation concentration, negatively charged small complexes involving a few polyanion chains are observed first. By further increasing x, these small complexes condense at a precise charge ratio x(c) < 1 to form large anionic aggregates. The inversion of the charge of the condensed complexes coincides with the maximum of complexation and precedes the dissolution of the aggregates which occurs at a well-defined decondensation threshold x(d) > 1. Above x(d), positively charged complexes containing again a few overcharged polyanion chains are observed. The macroscopic phase diagram is qualitatively well corroborated by AFM observation of the complexes. The influence of entropic effects is probed by varying parameters like concentration, polycation molecular weight and ionic strength. Structure of stable negatively charged complexes is investigated at higher concentration using Small Angle Neutron Scattering. In the condensed regime, we observe large soluble bundles with sharp interfaces where the local structure of the polyanions is preserved.