Learning brain effective connectivity (EC) networks from functional magnetic resonance imaging (fMRI) data has become a new hot topic in the neuroinformatics field. However, how to accurately and efficiently learn brain EC networks is still a challenging problem. In this paper, we propose a new algorithm to learn the brain EC network structure using ant colony optimization (ACO) algorithm combining with voxel activation information, named as VACOEC. First, VACOEC uses the voxel activation information to measure the independence between each pair of brain regions and effectively restricts the space of candidate solutions, which makes many unnecessary searches of ants be avoided. Then, by combining the global score increase of a solution with the voxel activation information, a new heuristic function is designed to guide the process of ACO to search for the optimal solution. The experimental results on simulated datasets show that the proposed method can accurately and efficiently identify the directions of the brain EC networks. Moreover, the experimental results on real-world data show that patients with Alzheimers disease (AD) exhibit decreased effective connectivity not only in the intra-network within the default mode network (DMN) and salience network (SN), but also in the inter-network between DMN and SN, compared with normal control (NC) subjects. The experimental results demonstrate that VACOEC is promising for practical applications in the neuroimaging studies of geriatric subjects and neurological patients.