In this article, the data-driven optimal formation control problem is addressed for a heterogeneous quadrotor team with a virtual leader. Each quadrotor is considered as a highly nonlinear system with six degrees of freedom and the accurate dynamic information of the quadrotor is difficult to obtain in practical applications. An optimal cascade formation controller, including a position controller and an attitude controller, is proposed to track a virtual leader and form a predesigned formation. By using the reinforcement learning (RL) approach, the optimal formation controller is learned from the quadrotor system data without any knowledge of dynamic information of the quadrotors. Simulation results of a heterogeneous multiquadrotor system in a formation flight are given to show the effectiveness of the proposed controllers.