Accurately predicting the binding affinity of protein-ligand pairs is an essential part of drug discovery. Since wet laboratory experiments to determine the binding affinity are expensive and time-consuming, several computational methods for binding affinity prediction have been proposed. In the representation of compounds, most methods only focus on the structural properties such as SMILES and ignore the bioactive properties. In this study, we proposed a novel model named PLA-MoRe to predict protein-ligand binding affinity, which represents compounds based on both structural and bioactive properties and mainly contains three feature extractors. First, a structure feature extractor based on the graph isomorphism network was constructed to learn the representations of the molecular graphs. Second, we designed an Autoencoder-based bioactive feature extractor to integrate the multisource bioactive information including chemical, target, network, cellular, and clinical. The above two parts aimed to learn representations of compounds in terms of structures and bioactivities, respectively. Then, we constructed a sequence feature extractor to learn embeddings for protein sequences. The output of the three extractors was concatenated and fed into a fully connected network for affinity prediction. We compared PLA-MoRe with three state-of-the-art methods, and an ablation study was conducted to test the role of each part of the model. Further attention visualization showed that our model had the potential to locate the binding sites, which might help explain the mechanism of interaction. These results prove that PLA-MoRe is competitive and reliable. The resource codes are freely available at the GitHub repository https://github.com/QingyuLiaib/PLA-MoRe.