The presence of non-coherent blood speckle patterns makes the assessment of lumen size in intravascular ultrasound (IVUS) images a challenging problem, especially for images acquired with recent high frequency transducers. In this paper, we present a robust three-dimensional (3D) feature extraction algorithm based on the expansion of IVUS cross-sectional images and pullback directions onto an orthonormal complex brushlet basis. Several features are selected from the projections of low-frequency 3D brushlet coefficients. These representations are used as inputs to a neural network that is trained to classify blood maps on IVUS images. We evaluated the algorithm performance using repeated randomized experiments on sub-samples to validate the quantification of the blood maps when compared to expert manual tracings of 258 frames collected from three patients. Our results demonstrate that the proposed features extracted in the brushlet domain capture well the non-coherent structures of blood speckle, enabling identification of blood pools and enhancement of the lumen area.