Voltage controlled 180° magnetization reversal has been achieved in BiFeO3-based multiferroic heterostructures, which is promising for the future development of low-power spintronic devices. However, all existing reports involve the use of an in-plane voltage that is unfavorable for practical device applications. Here, we investigate, using phase-field simulations, the out-of-plane (i.e., perpendicular to heterostructures) voltage controlled magnetism in heterostructures consisting of CoFe nanodots and (110) BiFeO3 thin film or island. It is predicted that the in-plane component of the canted magnetic moment at the CoFe/BiFeO3 interface can be reversed repeatedly by applying a perpendicular voltage across the bottom (110) BiFeO3 thin film, which further leads to an in-plane magnetization reversal in the overlaying CoFe nanodot. The non-volatility of such perpendicular voltage controlled magnetization reversal can be achieved by etching the continuous BiFeO3 film into isolated nanoislands with the same in-plane sizes as the CoFe nanodot. The findings would provide general guidelines for future experimental and engineering efforts on developing the electric-field controlled spintronic devices with BiFeO3-based multiferroic heterostructures.