Many species of phytoplankton produce toxins that may provide protection from grazing. In that case one would expect toxin production to be costly; else all species would evolve toxicity. However, experiments have consistently failed to show any costs. Here, we show that costs of toxin production are environment dependent but can be high. We develop a fitness optimization model to estimate rate, costs, and benefits of toxin production, using PST (paralytic shellfish toxin) producing dinoflagellates as an example. Costs include energy and material (nitrogen) costs estimated from well-established biochemistry of PSTs, and benefits are estimated from relationship between toxin content and grazing mortality. The model reproduces all known features of PST production: inducibility in the presence of grazer cues, low toxicity of nitrogen-starved cells, but high toxicity of P-limited and light-limited cells. The model predicts negligible reduction in cell division rate in nitrogen replete cells, consistent with observations, but >20% reduction when nitrogen is limiting and abundance of grazers high. Such situation is characteristic of coastal and oceanic waters during summer when blooms of toxic algae typically develop. The investment in defense is warranted, since the net growth rate is always higher in defended than in undefended cells.