Reducing complexity of the pipeline of instance segmentation is crucial for real-world applications. This work addresses this problem by introducing an anchor-box free and single-shot instance segmentation framework, termed PolarMask++, which reformulates the instance segmentation problem as predicting the contours of objects in the polar coordinate, leading to several appealing benefits. (1) The polar representation unifies instance segmentation (masks) and object detection (bounding boxes) into a single framework, reducing the design and computational complexity. (2) We carefully design two modules (soft polar centerness and polar IoU loss) to sample high-quality center examples and optimize polar contour regression, making the performance of PolarMask++ does not depend on the bounding box prediction and thus more efficient in training. (3) PolarMask++ is fully convolutional and can be easily embedded into most off-the-shelf detectors. To further improve the accuracy of the framework, a Refined Feature Pyramid is introduced to improve the feature representation at different scales. Extensive experiments demonstrate the effectiveness of PolarMask++, which achieves competitive results on COCO dataset, and new state-of-the-art results on text detection and cell segmentation datasets. We hope polar representation can provide a new perspective for designing algorithms to solve single-shot instance segmentation. Code is released at: github.com/xieenze/PolarMask.