Ultrasound (US) is widely used as a low-cost alternative to computed tomography or magnetic resonance and primarily for preliminary imaging. Since speckle intensity in US images is inherently stochastic, readers are often challenged in their ability to identify the pathological regions in a volume of a large number of images. This paper introduces a generalized approach for volumetric segmentation of structures in US images and volumes. We employ an iterative random walks (IRW) solver, a random forest learning model, and a gradient vector flow (GVF) based interframe belief propagation technique for achieving cross-frame volumetric segmentation. At the start, a weak estimate of the tissue structure is obtained using estimates of parameters of a statistical mechanics model of US tissue interaction. Ensemble learning of these parameters further using a random forest is used to initialize the segmentation pipeline. IRW is used for correcting the contour in various steps of the algorithm. Subsequently, a GVF-based interframe belief propagation is applied to adjacent frames based on the initialization of contour using information in the current frame to segment the complete volume by frame-wise processing. We have experimentally evaluated our approach using two different datasets. Intravascular ultrasound (IVUS) segmentation was evaluated using 10 pullbacks acquired at 20 MHz and thyroid US segmentation is evaluated on 16 volumes acquired at [Formula: see text] MHz. Our approach obtains a Jaccard score of [Formula: see text] for IVUS segmentation and [Formula: see text] for thyroid segmentation while processing each frame in [Formula: see text] for the IVUS and in [Formula: see text] for thyroid segmentation without the need of any computing accelerators such as GPUs.