We study theoretically the stretching dynamics of a yield stress material that exhibits both elastic and viscoplastic behavior. The material is confined between two coaxial disks, forming initially a cylindrical liquid bridge and then a neck when the disks are pulled apart. The material follows the Saramito-Herschel-Bulkley constitutive model and yields according to the von Mises criterion. We find that an elongated thin neck is formed when elasticity prevails, connecting the upper and lower parts of the filamentous bridge. This neck has been observed in breakup experiments of yield stress bridges, but this is the first theoretical study that predicts it. Earlier numerical and theoretical studies of filament stretching of yield stress materials failed to do so, because they excluded elasticity from the constitutive model they used in the simulations. Our results indicate that increasing elasticity leads to shorter pinching times and filament length than the viscoplastic case. This is caused by the fact that larger areas of the filament remain unyielded, while they undergo small deformation even before yielding, and only the remaining smaller yielded areas carry the burden of visible deformation. Our findings suggest that the value of the yield strain, defined as the ratio of the yield stress to the elastic modulus, should be used with caution to determine whether elastic effects will affect the filament stretching procedure or not.