Shape and deformation measurement of diffusely reflecting 3D objects are very important in many application areas, including quality control, nondestructive testing, and design. When rough objects are exposed to coherent beams, the scattered light produces speckle fields. A method to measure the shape and deformation of 3D objects from the sequential intensity measurements of volume speckle field and phase retrieval based on angular-spectrum propagation technique is described here. The shape of a convex spherical surface was measured directly from the calculated phase map, and micrometer-sized deformation induced on a metal sheet was obtained upon subtraction of the phase, corresponding to unloaded and loaded states. Results from computer simulations confirm the experiments.