Electroencephalogram (EEG)-based emotion recognition has gradually become a research hotspot. However, the large distribution differences of EEG signals across subjects make the current research stuck in a dilemma. To resolve this problem, in this article, we propose a novel and effective method, Multi-Source Feature Representation and Alignment Network (MS-FRAN). The effectiveness of proposed method mainly comes from three new modules: Wide Feature Extractor (WFE) for feature learning, Random Matching Operation (RMO) for model training, and Top- h ranked domain classifier selection (TOP) for emotion classification. MS-FRAN is not only effective in aligning the distributions of each pair of source and target domains, but also capable of reducing the distributional differences among the multiple source domains. Experimental results on the public benchmark datasets SEED and DEAP have demonstrated the advantage of our method over the related competitive approaches for cross-subject EEG-based emotion recognition.