The mechanical and morphological cues of fibrillar extracellular matrices (ECMs) play vital roles in controlling the cellular behaviors. Understanding and regulating the correlation of the mechanics with morphologies, at the micro-/nanoscale are of great relevance to guide the growth and differentiation of stem or progenitor cells into the desired tissues. However, the investigations directed toward acquiring such a kind of correlation are very limited and far from satisfactory. Here, rheological and nanoindentation tests were employed to appraise the mechanical behaviors of biomimetic ECMs assembled from type I collagen solutions containing the equivalent content of alginate but with different molecular weights (MWs). An alginate-molecular-weight-dependent trend was found in the fibrillogenesis process and the fibril aggregation of these collagen-alginate (CA) matrices. The present study revealed that the viscoelasticity and nonlinear elasticity of the CA matrices relied upon their specific fibrillar architectures in which a heterogeneous structure formed with varying alginate MW, including the coexistence of small fibrils and larger fibrillar bundles. The correlation of the mechanical behaviors with the inhomogeneity in the fibrillar structures was further discussed in combination with those of Ca2+ ionically cross-linked CA matrices. This study not only presented the delicate mechanics of fibrillar ECM analogues but also showed that the introduction of affiliative matters such as polysaccharides (alginate with different MWs) is a simple and convenient strategy to achieve biomimetic hydrogels with tunable viscoelastic properties.