Electroencephalography (EEG) is a noninvasive electrophysiological monitoring technique that records the electrical activities of the brain from the scalp using electrodes. EEG is not only an essential tool for diagnosing diseases and disorders affecting the brain, but also helps us to achieve a better understanding of brain's activities and structures. EEG recordings are weak, nonlinear, and nonstationary signals that contain various noise and artifacts. Therefore, for analyzing them, advanced signal processing techniques are required. Second order statistical features are usually sufficient for analyzing most basic signals. However, higher order statistical features possess characteristics that are missing in the second order; characteristics that can be highly beneficial for analysis of more complex signals, such as EEG. The primary goal of this article is to provide a comprehensive survey of the applications of higher order statistics or spectra (HOS) in EEG signal processing. Therefore, we start the survey with a summary of previous studies in EEG analysis followed by a brief mathematical description of HOS. Then, HOS related features and their applications in EEG analysis are presented. These applications are then grouped into three categories, each of which are further explored thoroughly with examples of prior studies. Finally, we provide some specific recommendations based on the literature survey and discuss possible future directions of this field.