The morphogenesis of colonial stony corals is the result of the collective behaviour of many coral polyps depositing coral skeleton on top of the old skeleton on which they live. Yet, models of coral growth often consider the polyps as a single continuous surface. In the present work, the polyps are modelled individually. Each polyp takes up resources, deposits skeleton, buds off new polyps and dies. In this polyp oriented model, spontaneous branching occurs. We argue that branching is caused by a so called "polyp fanning effect" by which polyps on a convex surface have a competitive advantage relative to polyps on a flat or concave surface. The fanning effect generates a more potent branching mechanism than the Laplacian growth mechanism that we have studied previously. We discuss the application of the polyp oriented model to the study of environmentally driven morphological plasticity in stony corals. In a few examples we show how the properties of the individual polyps influence the whole colony morphology. In our model, the spacing of polyps influences the thickness of coral branches and the overall compactness of the colony. Density variations in the coral skeleton may also be important for the whole colony morphology, which we address by studying two variants of the model. Finally, we discuss the importance of small scale resource translocation in the coral colony and its effects on the morphology of the colony.