Magic-sized clusters (MSCs) can be isolated as intermediates in quantum dot (QD) synthesis, and they provide pivotal clues in understanding QD growth mechanisms. We report syntheses for two families of heterogeneous-atom-incorporated InP MSCs that have chlorine or zinc atoms. All the MSCs could be directly synthesized from conventional molecular precursors. Alternatively, each series of MSCs could be prepared by sequential conversions. 386-InP MSCs could be converted to F360-InP:Cl MSCs, then to F399-InP:Cl MSCs. Similarly, F360-InP:Zn MSCs could be converted to F408-InP:Zn MSCs, then to F393-InP:Zn MSCs. As the conversion proceeded, evolution from uni-molecule-like to QD-like characters was observed. Early stage MSCs showed active inter-state conversions in the excited states, which is characteristics of small molecules. Later stage MSCs exhibited narrow photoinduced absorptions at lower-energy region like QDs. The crystal structure also gradually evolved from polytwistane to more zinc-blende.