Drug combination therapies are well-established strategies for the treatment of cancer with low toxicity and fewer adverse effects. Computational drug synergy prediction approaches can accelerate the discovery of novel combination therapies, but the existing methods do not explicitly consider the key role of important substructures in producing synergistic effects. To this end, we propose a significant substructure-aware anticancer drug synergy prediction method, named SDDSynergy, to adaptively identify critical functional groups in drug synergy. SDDSynergy splits the task of predicting drug synergy into predicting the effect of individual substructures on cancer cell lines and highlights the impact of important substructures through a novel drug-cell line attention mechanism. And a substructure pair attention mechanism is incorporated to capture the information on internal substructure pairs interaction in drug combinations, which aids in predicting synergy. The substructures of different sizes and shapes are directly obtained from the molecular graph of the drugs by multilayer substructure information passing networks. Extensive experiments on three real-world data sets demonstrate that SDDSynergy outperforms other state-of-the-art methods. We also verify that many of the novel drug combinations predicted by SDDSynergy are supported by previous studies or clinical trials through an in-depth literature survey.