We assess the impact of lateral coastline-like boundaries on mixing and transport in a laboratory quasi-two-dimensional turbulent flow using a transfer-operator approach. We examine the most coherent sets in the flow, as defined by the singular vectors of the transfer operator, as a way to characterize its mixing properties. We study three model coastline shapes: a uniform boundary, a sharp embayment, and a sharp headland. Of these three, we show that the headland affects the mixing deep into the flow domain because it has a tendency to pin transport barriers to its tip. Our results may have implications for the siting of coastal facilities that discharge into the ocean.