Inorganic nanoparticles (NPs) with diversified functionalities are promising candidates in future optoelectronic and biomedical applications, which greatly depend on the capability to arrange NPs into higher-order architectures in a controllable way. This issue is considered to be solved by means of self-assembly. NPs can participate in self-assembly in different manners, such as smart self-organization with blended molecules, as the carriers of host molecules for assembly and disassembly with guest molecules, as netpoints to endow the architectures specific functionalities, and so forth. To enhance the structural stability of the as-prepared assembly architectures, polymers have been utilized to create NP-polymer composites. Meanwhile, such a strategy also demonstrates the possibility of integrating the functionalities of NPs and/or polymers by forming regular architectures. The emerging interest in the current optoelectronic and biological areas strongly demands intelligent nanocomposites, which are produced by combination of the excellent functionalities of NPs and the responsiveness of polymers. On the basis of the recent progress in fabricating NP-polymer composites, this critical review summarizes the development of new methods for fabricating regular self-assembly architectures, highlights the reversible assembly and disassembly behavior, and indicates the potential applications.