Increases in seawater temperature can cause coral bleaching through loss of symbiotic algae (dinoflagellates of the family Symbiodiniaceae). Corals can recover from bleaching by recruiting algae into host cells from the residual symbiont population or from the external environment. However, the high coral mortality that often follows mass-bleaching events suggests that recovery is often limited in the wild. Here, we examine the effect of pre-exposure to heat stress on the capacity of symbiotic algae to infect cnidarian hosts using the Aiptasia (sea-anemone)-Symbiodiniaceae model system. We found that the symbiont strain Breviolum sp. CS-164 (ITS2 type B1), both free-living and in symbiosis, loses the capacity to infect the host following exposure to heat stress. This loss of infectivity is reversible, however, a longer exposure to heat stress increases the time taken for reversal. Under the same experimental conditions, the loss of infectivity was not observed in another strain Breviolum psygmophilum CCMP2459 (ITS2 type B2). Our results suggest that recovery from bleaching can be limited by the loss of symbiont infectivity following exposure to heat stress.