The chemotactic network of Escherichia coli has been studied extensively both biophysically and information theoretically. Nevertheless, connection between these two aspects is still elusive. In this work, we report such a connection. We derive an optimal filtering dynamics under the assumption that E. coli's sensory system optimally infers the binary information whether it is swimming up or down along an exponential ligand gradient from noisy sensory signals. Then we show that a standard biochemical model of the chemotactic network is mathematically equivalent to this information-theoretically optimal dynamics. Moreover, we demonstrate that an experimentally observed nonlinear response relation can be reproduced from the optimal dynamics. These results suggest that the biochemical network of E. coli chemotaxis is designed to optimally extract the binary information along an exponential gradient in a noisy condition.