Due to the outstanding physicochemical properties arising from its truly two-dimensional (2D) planar structure with a single-atom thickness, graphene exhibits great potential for use in sensors, catalysts, electrodes, and in biological applications, etc. With further developments in the theoretical understanding and assembly techniques, graphene should enable great changes both in scientific research and practical industrial applications. By the look of development, it is of fundamental and practical significance to translate the novel physical and chemical properties of individual graphene nanosheets into the macroscale by the assembly of graphene building blocks into macroscopic architectures with structural specialities and functional novelties. The combined features of a 2D planar structure and abundant functional groups of graphene oxide (GO) should provide great possibilities for the assembly of GO nanosheets into macroscopic architectures with different macroscaled shapes through various assembly techniques under different bonding interactions. Moreover, macroscopic graphene frameworks can be used as ideal scaffolds for the incorporation of functional materials to offset the shortage of pure graphene in the specific desired functionality. The advantages of light weight, supra-flexibility, large surface area, tough mechanical strength, and high electrical conductivity guarantee graphene-based architectures wide application fields. This critical review mainly addresses recent advances in the design and fabrication of graphene-based macroscopic assemblies and architectures and their potential applications. Herein, we first provide overviews of the functional macroscopic graphene materials from three aspects, i.e., 1D graphene fibers/ribbons, 2D graphene films/papers, 3D network-structured graphene monoliths, and their composite counterparts with either polymers or nano-objects. Then, we present the promising potential applications of graphene-based macroscopic assemblies in the fields of electronic and optoelectronic devices, sensors, electrochemical energy devices, and in water treatment. Last, the personal conclusions and perspectives for this intriguing field are given.