Integration of cognitive tasks into artificial general intelligence test for large models

Youzhi Qu; Chen Wei; Penghui Du; Wenxin Che; Chi Zhang; Wanli Ouyang; Yatao Bian; Feiyang Xu; Bin Hu; Kai Du; Haiyan Wu; Jia Liu; Quanying Liu

doi:10.1016/j.isci.2024.109550

Integration of cognitive tasks into artificial general intelligence test for large models

iScience. 2024 Mar 22;27(4):109550. doi: 10.1016/j.isci.2024.109550. eCollection 2024 Apr 19.

Authors

Youzhi Qu¹, Chen Wei¹, Penghui Du¹, Wenxin Che¹, Chi Zhang¹, Wanli Ouyang², Yatao Bian³, Feiyang Xu⁴, Bin Hu⁵, Kai Du⁶, Haiyan Wu⁷, Jia Liu⁸, Quanying Liu¹

Affiliations

¹ Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen 518055, China.
² Shanghai AI Laboratory, Shanghai 200232, China.
³ Tencent AI lab, Shenzhen 518057, China.
⁴ iFLYTEK AI Research, Hefei 230088, China.
⁵ School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China.
⁶ Institute for Artificial Intelligence, Peking University, Beijing 100871, China.
⁷ Centre for Cognitive and Brain Sciences and Department of Psychology, University of Macau, Macau 999078, China.
⁸ Department of Psychology, Tsinghua University, Beijing 100084, China.

Abstract

During the evolution of large models, performance evaluation is necessary for assessing their capabilities. However, current model evaluations mainly rely on specific tasks and datasets, lacking a united framework for assessing the multidimensional intelligence of large models. In this perspective, we advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests, including crystallized, fluid, social, and embodied intelligence. The AGI tests consist of well-designed cognitive tests adopted from human intelligence tests, and then naturally encapsulates into an immersive virtual community. We propose increasing the complexity of AGI testing tasks commensurate with advancements in large models and emphasizing the necessity for the interpretation of test results to avoid false negatives and false positives. We believe that cognitive science-inspired AGI tests will effectively guide the targeted improvement of large models in specific dimensions of intelligence and accelerate the integration of large models into human society.

Keywords: Cognitive neuroscience; Neuroscience; Research methodology social sciences; Social sciences.

Publication types

Review