This paper presents a new model, to the best of our knowledge, of a mixed underlay cognitive radio frequency (RF)/free space optical (FSO) system in which both RF and FSO links consider multiple-input multiple-output (MIMO) orthogonal space-time block coding (OSTBC). In a dual-hop decode-and-forward configuration, the underlay cognitive radio network RF and FSO links experience κ-μ and Γ-Γ fading, respectively. For the above system model, a closed expression for the outage probability of the mixed underlay cognitive RF/FSO system with MIMO-OSTBC is derived, and the simulation results are verified using the Monte Carlo method. The results show that considering MIMO-OSTBC in all links of the mixed underlay cognitive RF/FSO system can effectively improve the communication performance of the mixed system and alleviate the degradation in the communication quality caused by the atmospheric turbulence. The communication performance of the MIMO-OSTBC mixed underlay cognitive RF/FSO system is further improved by changing the key parameters, such as peak transmit power, fading parameters, the number of secondary user transmit antennas and relay receive antennas, and relative speed of the primary user.