In epilepsy, semiology refers to the study of patient behavior and movement, and their temporal evolution during epileptic seizures. Understanding semiology provides clues to the cerebral networks underpinning the epileptic episode and is a vital resource in the pre-surgical evaluation. Recent advances in video analytics have been helpful in capturing and quantifying epileptic seizures. Nevertheless, the automated representation of the evolution of semiology, as examined by neurologists, has not been appropriately investigated. From initial seizure symptoms until seizure termination, motion patterns of isolated clinical manifestations vary over time. Furthermore, epileptic seizures frequently evolve from one clinical manifestation to another, and their understanding cannot be overlooked during a presurgery evaluation. Here, we propose a system capable of computing motion signatures from videos of face and hand semiology to provide quantitative information on the motion, and the correlation between motions. Each signature is derived from a sparse saliency representation established by the magnitude of the optical flow field. The developed computer-aided tool provides a novel approach for physicians to analyze semiology as a flow of signals without interfering in the healthcare environment. We detect and quantify semiology using detectors based on deep learning and via a novel signature scheme, which is independent of the amount of data and seizure differences. The system reinforces the benefits of computer vision for non-obstructive clinical applications to quantify epileptic seizures recorded in real-life healthcare conditions.