To better understand how animals make ethologically relevant decisions, we studied egg-laying substrate choice in Drosophila. We found that flies dynamically increase or decrease their egg-laying rates while exploring substrates so as to target eggs to the best, recently visited option. Visiting the best option typically yielded inhibition of egg laying on other substrates for many minutes. Our data support a model in which flies compare the current substrate's value with an internally constructed expectation on the value of available options to regulate the likelihood of laying an egg. We show that dopamine neuron activity is critical for learning and/or expressing this expectation, similar to its role in certain tasks in vertebrates. Integrating sensory experiences over minutes to generate an estimate of the quality of available options allows flies to use a dynamic reference point for judging the current substrate and might be a general way in which decisions are made.