A plasmonic cavity is shown to greatly reduce the inhomogeneity of dynamic optical properties such as quantum efficiency and radiative lifetime of InGaN quantum dots. By using an open-top plasmonic cavity structure, which exhibits a large Purcell factor and antenna quantum efficiency, the resulting quantum efficiency distribution for the quantum dots narrows and is no longer limited by the quantum dot inhomogeneity. The standard deviation of the quantum efficiency can be reduced to 2% while maintaining the overall quantum efficiency at 70%, making InGaN quantum dots a viable candidate for high-speed quantum cryptography and random number generation applications.