The electroencephalogram (EEG) is extensively employed for detecting various brain electrical activities. Nonetheless, EEG recordings are susceptible to undesirable artifacts, resulting in misleading data analysis and even significantly impacting the interpretation of results. While previous efforts to mitigate or reduce the impact of artifacts have achieved commendable performance, several challenges in this domain still persist: 1) due to black-box skepticism, deep-learning-based automatic EEG artifact removal methods have been impeded from being applied in clinical environments. How to support reliable denoised EEG signals with high accuracy is important; and 2) effectively exploring valuable local and global information from contaminated contexts remains challenging. On the one hand, feature extraction and aggregation in prior works are often performed blindly and assumed to be accurate, which is not always the case. On the other hand, global contextual information is gradually modeled by local fixed single-scaled convolutional filters layer by layer, which is neither efficient nor effective. To address the above challenges, we propose an Uncertainty-aware Denoising Network (UDNet) with multi-scaled pooling attention for efficient context capturing. Specifically, we predict the aleatoric and epistemic uncertainty existing during the denoising process to assist in finding and reducing the uncertain feature representation. We further propose a simple yet effective architecture to capture local and global contexts at multiple scales. Our proposed method can serve as an effective metric for identifying low-confidence epochs that warrant deferral to human experts for further inspection and assessment. Experimental results on two public datasets show that the proposed model outperforms state-of-the-art baselines.