Mechanically robust and biomimicking scaffolds are needed for structural engineering of tissues such as the intervertebral disc, which are prone to failure and incapable of natural healing. Here, the formation of thick, randomly aligned polycaprolactone electrospun fibre structures infiltrated with alginate is reported. The composites are characterised using both indentation and tensile testing and demonstrate substantially different tensile and compressive moduli. The composites are mechanically robust and exhibit large strains-to-failure, exhibiting toughening mechanisms observed in other composite material systems. The method presented here provides a way to create large-scale biomimetic scaffolds that more closely mimic the composite structure of natural tissue, with tuneable tensile and compressive properties via the fibre and matrix phases, respectively.