Robot-assisted catheterization is routinely carried out for intervention of cardiovascular diseases. Meanwhile, the success of endovascular tool navigation depends on visualization and tracking cues available in the robotic platform. Currently, real-time motion analytics are lacking, while poor illumination during fluoroscopy affects existing physics- and learning-based methods used for tool segmentation. A multi-lateral branched network (MLB-Net) is herein proposed for tool segmentation in cardiovascular angiograms. The model has an encoder with multi-lateral separable convolutions and a pyramid decoder. Model training and validation are done on 1320 angiograms obtained during robot-assisted catheterization in rabbit. Model performance, explained with F1-score of 89.01% and mean intersection-over-union of 90.05% on 330 frames, indicates the model's robustness for guidewire segmentation in angiograms. The MLB-Net offers better performance than the state-of-the-art segmentation models such as U-Net, U-Net++ and DeepLabV3. Thus, it could provide basis for endovascular tool tracking and surgical scene analytics during cardiovascular interventions.