The burden of antimicrobial resistance (AMR) is rapidly growing across antibiotic classes, with increased detection of isolates resistant to carbapenems. Data on the prevalence of carbapenem resistance in developing countries is limited; therefore, in this study, we determined the prevalence of carbapenemase genes among multidrug resistant gram negative bacteria (MDR-GNB) isolated from clinical specimens in a tertiary hospital in Mwanza, Tanzania. A total of 227 MDR-GNB isolates were analyzed for carbapenem resistance genes. For each isolate, five different PCR assays were performed, allowing for the detection of the major carbapenemase genes, including those encoding the VIM-, IMP-, and NDM-type metallo-beta-lactamases, the class A KPC-type carbapenemases, and the class D OXA-48 enzyme. Of 227 isolates, 80 (35%) were positive for one or more carbapenemase gene. IMP-types were the most predominant gene followed by VIM, in 49 (21.59%) and 28 (12%) isolates, respectively. Carbapenemase genes were most detected in K. pneumoniae 24 (11%), followed by P. aeruginosa 23 (10%), and E. coli with 19 isolates (8%). We have demonstrated for the first time a high prevalence of MDR-GNB clinical isolates having carbapenem resistance genes in Tanzania. We recommend routine testing for carbapenem resistance among the MDR-GNB particularly in systemic infections.