Composite materials are commonly used in aircraft, and the integrity of these materials affects both flight and safety performance. Damage detection technology involving infrared nondestructive testing has played an important role in damage detection in aircraft composite materials. Traditional manual detection methods are inefficient, and the use of intelligent detection methods can effectively improve detection efficiency. Due to the diverse types of damage that can occur in composite materials, this damage is difficult to distinguish solely from infrared images. The introduction of infrared signals, which is temporal signals, provides the possibility of judging the type of damage. In this paper, a 1D-YOLOv4 network is established. The network is based on the YOLOv4 network and adds a changing neck and a 1D-CNN for improvement. Testing shows that the algorithm can identify infrared images and infrared signals in composite materials. Its recognition accuracy is 98.3%, with an AP of 91.9%, and a kappa of 0.997. Comparing the network in this paper with networks such as YOLOv3, YOLOv4 and YOLOv4+Neck, the results show that the proposed network is more effective. At the same time, the detection effects of the original data, the fitted data, the first derivative data and the second derivative data are studied, and the detection effect of the first derivative data has the best outcome.