Brain surgery is complex and has evolved as a separate surgical specialty. Surgical procedures on the brain are performed using dedicated micro-instruments which are designed specifically for the requirements of operating with finesse in a confined space. The usage of these microsurgical tools in an operating environment defines the surgical skill of a surgeon. Video recordings of micro-surgical procedures are a rich source of information to develop automated surgical assessment tools that can offer continuous feedback for surgeons to improve their skills, effectively increase the outcome of the surgery, and make a positive impact on their patients. This work presents a novel deep learning system based on the Yolov5 algorithm to automatically detect, localize and characterize microsurgical tools from recorded intra-operative neurosurgical videos. The tool detection achieves a high 93.2% mean average precision. The detected tools are then characterized by their on-off time, motion trajectory and usage time. Tool characterization from neurosurgical videos offers useful insight into the surgical methods employed by a surgeon and can aid in their improvement. Additionally, a new dataset of annotated neurosurgical videos is used to develop the robust model and is made available for the research community.Clinical relevance- Tool detection and characterization in neurosurgery has several online and offline applications including skill assessment and outcome of the surgery. The development of automated tool characterization systems for intra-operative neurosurgery is expected to not only improve the surgical skills of the surgeon, but also leverage in training the neurosurgical workforce. Additionally, dedicated neurosurgical video based datasets will, in general, aid the research community to explore more automation in this field.