Fungus is enormously notorious for food, human health, and archives. Fungus sign and symptoms in medical science are non-specific and asymmetrical for extremely large areas resulting into a challenging task of fungal detection. Various traditional and computer vision techniques were applied to meet the challenge of early fungus detection. On the other hand, features learned through the convolutional neural network (CNN) provided state-of-the-art results in many other applications of object detection and classification. However, the large amount of data is an essential prerequisite for its effective application. In pursuing this idea, we present a novel fungus dataset of its kind, with the goal of advancing the state of the art in fungus classification by placing the question of fungus detection. This is achieved by gathering various images of complex fungal spores by extracting samples from contaminated fruits, archives, and laboratory-incubated fungus colonies. These images primarily consisted of five different types of fungus spores and dirt. An optical sensor system was utilized to obtain these images, which were further annotated to mark fungal spores as a region of interest using specially designed graphical user interface. As a result, 40,800 labeled images were used to develop the fungus dataset to aid in precise fungus detection and classification. The other main objective of this research was to develop a CNN-based approach for the detection of fungus and distinguish different types of fungus. A CNN architecture was designed, and it showed the promising results with an accuracy of 94.8%. The obtained results proved the possibility of early detection of several types of fungus spores using CNN and could estimate all possible threats due to fungus.