Recently, there has been significant progress in medical image segmentation utilizing deep learning techniques. However, these achievements largely rely on the supposition that the source and target domain data are identically distributed, and the direct application of related methods without addressing the distribution shift results in dramatic degradation in realistic clinical environments. Current approaches concerning the distribution shift either require the target domain data in advance for adaptation, or focus only on the distribution shift across domains while ignoring the intra-domain data variation. This paper proposes a domain-aware dual attention network for the generalized medical image segmentation task on unseen target domains. To alleviate the severe distribution shift between the source and target domains, an Extrinsic Attention (EA) module is designed to learn image features with knowledge originating from multi-source domains. Moreover, an Intrinsic Attention (IA) module is also proposed to handle the intra-domain variation by individually modeling the pixel-region relations derived from an image. The EA and IA modules complement each other well in terms of modeling the extrinsic and intrinsic domain relationships, respectively. To validate the model effectiveness, comprehensive experiments are conducted on various benchmark datasets, including the prostate segmentation in magnetic resonance imaging (MRI) scans and the optic cup/disc segmentation in fundus images. The experimental results demonstrate that our proposed model effectively generalizes to unseen domains and exceeds the existing advanced approaches.