The high prevalence rate of Alzheimer's disease (AD) and mild cognitive impairment (MCI) has been a serious public health threat to the modern society. Recently, many studies have demonstrated the potential of using non-invasive electroencephalography (EEG) and machine learning to assist the diagnosis of AD/MCI. However, the majority of these research recorded EEG signals from a single center, leading to significant concerns regarding the generalizability of the findings in clinical settings. The current study aims to reevaluate the effectiveness of EEG-based machine learning model for the detection of AD/MCI in the case of a relatively large and diverse data set. We collected resting-state EEG data from 150 participants across six hospitals and examined the classification performances of Linear Discriminative Analysis (LDA) classifiers on the phase lag index (PLI) feature. We also compared the performance of PLI over the other commonly-used EEG features and other classifiers. The model was first tested on a training set to select the feature subset and then further validated with an independent test set. The results demonstrate that PLI performs the best compared to other features. The LDA classifier trained with the optimal PLI features can provide 82.50% leave-one-participant-out cross-validation (LOPO-CV) accuracy on the training set and maintain a good enough performance with 75.00% accuracy on the test set. Our results suggest that PLI-based functional connectivity could be considered as a reliable bio-maker to detect AD/MCI in the real-world clinical settings.