This brief investigates the infinite horizon optimal control problem for stochastic multivalued logical dynamical systems with discounted cost. Applying the equivalent descriptions of stochastic logical dynamics in term of Markov decision process, the discounted infinite horizon optimal control problem is presented in an algebraic form. Then, employing the method of semitensor product of matrices and the increasing-dimension technique, a succinct algebraic form of the policy iteration algorithm is derived to solve the optimal control problem. To show the effectiveness of the proposed policy iteration algorithm, an optimization problem of p53-Mdm2 gene network is investigated.