The image information of distant objects shows a diffuse speckle pattern due to diffraction limit, non-uniform scattering, etc., which is difficult to achieve object discrimination. In this study, we have developed a staring spectral video imaging system mounted on a ground-based telescope observation platform to detect the high orbit space objects and gain their spectral images for six groups of GEO targets. The speckle remains basically the same characteristic as the projection structure of the object due to "the balloon inflation phenomenon of near parallel light during long-distance atmospheric transmission" under the premise of considering the bi-directional reflection distribution function (BRDF), Rayleigh scattering theory, and the memory effect. Based on this phenomenon, a mathematical model of remote target scattering spectrum imaging is established where the speckle can be treated as both a global speckle and speckle combination of texture blocks caused by various components of the target. The radial basis function (RBF) neural network is separately used to invert the global speckle and the speckle combination of the texture blocks on account of the typical target material database. The results show that the target materials are of relatively fewer kinds in the global inversion with only including gallium arsenide panel (GaAs) and carbon fiber (CF), for which the highest goodness of curve fitting is only 77.97. An improved algorithm makes their goodness of fit reach 90.29 and 93.33, respectively, in view of one conjecture that the target surface contains unknown materials. The spectral inversion result of the texture blocks shows that the types of materials in each target texture block increase significantly, and that the area ratio of different materials inverted in the block is different from each other. It is further confirmed that the speckle image contains the overall projection structure of distant target and the spectral image projection of each component is relatively fixed, which is the result of the comprehensive action of various mechanisms of ultra-long-haul atmospheric transmission and optical system focusing imaging after BRDF spectral scattering. The spectral image fine inversion is expected to restore the clear structure of the target. This discovery provides important support for the remote imaging and identification of distant and ultra-diffractive targets.