Cataract surgery remains the definitive treatment for cataracts, which are a major cause of preventable blindness worldwide. Adequate and stable dilation of the pupil are necessary for the successful performance of cataract surgery. Pupillary instability is a known risk factor for cataract surgery complications, and the accurate segmentation of the pupil from surgical video streams can enable the analysis of intraoperative pupil changes in cataract surgery. However, pupil segmentation performance can suffer due to variations in surgical illumination, obscuration of the pupil with surgical instruments, and hydration of the lens material intraoperatively. To overcome these challenges, we present a novel method called tensor-based pupil feature extraction (TPFE) to improve the accuracy of pupil recognition systems. We analyzed the efficacy of this approach with experiments performed on a dataset of 4,560 intraoperative annotated images from 190 cataract surgeries in human patients. Our results indicate that TPFE can identify features relevant to pupil segmentation and that pupil segmentation with state-of-the-art deep learning models can be significantly improved with the TPFE method.