Biocompatible and functionalizable hydrogels have a wide range of (potential) medicinal applications. The hydrogelation process, particularly for systems with very low polymer weight percentages (<1 wt %), remains poorly understood, making it challenging to predict the self-assembly of a given molecular building block into a hydrogel. This severely hinders the rational design of self-assembled hydrogels. In this study, we demonstrate the impact of an N-terminal group on the self-assembly and rheology of the peptide hydrogel hFF03 (hydrogelating, fibril forming peptide 03) using molecular dynamics simulations, oscillatory shear rheology, and circular dichroism spectroscopy. We find that the chromophore and even its specific regioisomers have a significant influence on the microscopic structure and dynamics of the self-assembled fibril, and on the macroscopic mechanical properties. This is because the chromophore influences the possible salt bridges, which form and stabilize the fibril formation. Furthermore, we find that the solvation shell fibrils by itself cannot explain the viscoelasticity of hFF03 hydrogels. Our atomistic model of the hFF03 fibril formation enables a more rational design of these hydrogels. In particular, altering the N-terminal chromophore emerges as a design strategy to tune the mechanic properties of these self-assembled peptide hydrogels.