Angiogenesis is a complex cellular process involving highly orchestrated invasion and organization of endothelial cells (ECs) in a three-dimensional (3D) environment. Recent evidence indicates that Notch signaling is critically involved in regulating specialized functions and distinct fates of ECs in newly formed vasculatures during angiogenesis. Here, we demonstrated, for the first time, the application of a microengineered biomimetic system to quantitatively investigate the role of Notch signaling in regulating early angiogenic sprouting and vasculature formation of ECs in a 3D extracellular matrix. Morphological features of angiogenesis including invasion distance, invasion area, and tip cell number were quantified and compared under pharmacological perturbations of Notch signaling. In addition, influences of Notch signaling on EC proliferation in angiogenic vasculatures and directional invasion of tip cells were also investigated. Moreover, leveraging a novel nanobiosensor system, mRNA expression of Dll4, a Notch ligand, was monitored in invading tip cells using live cell imaging during the dynamic angiogenic process. Our data showed that inhibition of Notch signaling resulted in hyper-sprouting endothelial structures, while activation of Notch signaling led to opposite effects. Our results also supported the role of Notch signaling in regulating EC proliferation and dynamic invasion of tip cells during angiogenesis.