Infrared image super-resolution technology aims to overcome the pixel size limitation of the infrared focal plane array for higher resolution images. Due to the real-world images with different resolutions having more complex degradation processes than mathematical calculation, most existing super-resolution methods using the synthetic data obtained by bicubic interpolation achieve unsatisfactory reconstruction performance in real-world scenes. To solve this, this paper innovatively proposes an infrared real-world dataset with different resolutions based on a refrigerated thermal detector and the infrared zoom lens, enabling the network to acquire more realistic details. We obtain images under different fields of view by adjusting the infrared zoom lens and then achieve the scale and luminance alignment of high and low-resolution (HR-LR) images. This dataset can be used for infrared image super-resolution, with an up-sampling scale of two. In order to learn complex features of infrared images efficiently, an asymmetric residual block structure is proposed to effectively reduce the number of parameters and improve the performance of the network. Finally, to solve the slight misalignment problem in the pre-processing stage, contextual loss and perceptual loss are introduced to improve the visual performance. Experiments show that our method achieves superior results both in reconstruction effect and practical value for single infrared image super-resolution in real scenarios.