Stochasticity in gene expression can give rise to fluctuations in protein levels and lead to phenotypic variation across a population of genetically identical cells. Recent experiments indicate that bursting and feedback mechanisms play important roles in controlling noise in gene expression and phenotypic variation. A quantitative understanding of the impact of these factors requires analysis of the corresponding stochastic models. However, for stochastic models of gene expression with feedback and bursting, exact analytical results for protein distributions have not been obtained so far. Here, we analyze a model of gene expression with bursting and feedback regulation and obtain exact results for the corresponding protein steady-state distribution. The results obtained provide new insights into the role of bursting and feedback in noise regulation and optimization. Furthermore, for a specific choice of parameters, the system studied maps on to a two-state biochemical switch driven by a bursty input noise source. The analytical results derived provide quantitative insights into diverse cellular processes involving noise in gene expression and biochemical switching.