A novel classification algorithm of hyperspectral imagery based on ant colony compositely optimizing support vector machine in spatial and spectral features was proposed. Two types of virtual ants searched for the bands combination with the maximum class separation distance and heterogeneous samples in spatial and spectral features alternately. The optimal characteristic bands were extracted, and bands redundancy of hyperspectral imagery decreased. The heterogeneous samples were eliminated form the training samples, and the distribution of samples was optimized in feature space. The hyperspectral imagery and training samples which had been optimized were used in classification algorithm of support vector machine, so that the class separation distance was extended and the accuracy of classification was improved. Experimental results demonstrate that the proposed algorithm, which acquires an overall accuracy 95.45% and Kappa coefficient 0.925 2, can obtain greater accuracy than traditional hyperspectral image classification algorithms.