The formation of a segregated network structure (wittingly uneven distribution of a filler) is one of the most promising strategies for the fabrication of electrically conductive polymer composites at present. However, the simultaneous achievement of high values of electrical conductivity with the retention of well mechanical properties within this approach remains a great challenge. Here, by means of X-ray photoelectron spectra (XPS), near-edge X-ray absorption fine structure (NEXAFS) spectra, scanning electron microscopy (SEM), dielectric spectroscopy, and compression engineering stress-strain curve analysis, we have studied the effect of a segregated network structure on the electrical conductivity and mechanical properties of a set of polymer composites. The composites were prepared by applying graphene oxide (GO) with ultralarge basal plane size (up to 150 μm) onto the surface of polymer powder particles, namely, poly(vinyl chloride) (PVC), poly(vinylidene fluoride-co-tetrafluoroethylene) (P(VDF-TFE)), and ultrahigh-molecular-weight poly(ethylene) (UHMWPE) with the subsequent GO reduction and composite hot pressing. A strong dependence of the segregated network polymer composites' physical properties on the polymer matrix was demonstrated. Particularly, 12 orders of magnitude rise of the polymers' electrical conductivity up to 0.7 S/m was found upon the incorporation of the reduced GO (rGO). A 17% increase in the P(VDF-TFE) elastic modulus filled by 1 wt % of rGO was observed. Fracture strength of PVC/rGO at 0.5 wt % content of the filler was demonstrated to decrease by fourfold. At the same time, the change in strength was not significant for P(VDF-TFE) and UHMWPE composites in comparison with pure polymers. Our results show a promise to accelerate the development of new composites for energy applications, such as metal-free supercapacitor plates and current collectors of lithium-ion batteries, bipolar plates of proton-exchange membrane fuel cells, antistatic elements of various electronic devices, etc.