The development of ductal structures during branching morphogenesis relies on signals that specify ductal progenitors to set up a pattern for the ductal network. Here, we identify cellular asymmetries defined by the F-actin cytoskeleton and the cell adhesion protein ZO-1 as the earliest determinants of duct specification in the embryonic submandibular gland (SMG). Apical polarity protein aPKCζ is then recruited to the sites of asymmetry in a ZO-1-dependent manner and collaborates with ROCK signaling to set up apical-basal polarity of ductal progenitors and further define the path of duct specification. Moreover, the motor protein myosin IIB, a mediator of mechanical force transmission along actin filaments, becomes localized to vertices linking the apical domains of multiple ductal epithelial cells during the formation of ductal lumens and drives duct maturation. These studies identify cytoskeletal, junctional and polarity proteins as the early determinants of duct specification and the patterning of a ductal tree during branching morphogenesis of the SMG.