Bone is a hard-connective tissue composed of matrix, cells and bioactive factors with a hierarchical structure, where the matrix is mainly composed of type I collagen and hydroxyapatite. Collagen fibers assembled by collagen are the template for mineralization and make an important contribution to bone formation and the bone remodeling process. Therefore, collagen has been widely clinically used for bone/cartilage defect regeneration. However, pure collagen implants, such as collagen scaffolds or sponges, have limitations in the bone/cartilage regeneration process due to their poor mechanical properties and osteoinductivity. Different forms of collagen-based composites prepared by incorporating natural/artificial polymers or bioactive inorganic substances are characterized by their interconnected porous structure and promoting cell adhesion, while they improve the mechanical strength, structural stability and osteogenic activities of the collagen matrix. In this review, various forms of collagen-based biocomposites, such as scaffolds, sponges, microspheres/nanoparticles, films and microfibers/nanofibers prepared by natural/synthetic polymers, bioactive ceramics and carbon-based materials compounded with collagen are reviewed. In addition, the application of collagen-based biocomposites as cytokine, cell or drug (genes, proteins, peptides and chemosynthetic) delivery platforms for proangiogenesis and bone/cartilage tissue regeneration is also discussed. Finally, the potential application, research and development direction of collagen-based biocomposites in future bone/cartilage tissue regeneration are discussed.