Stigma has been identified as a major barrier to HIV response. While much is known about stigma directed towards people living with HIV (PLHIV), less is known about stigma experienced by health workers who treat PLHIV. This study aims to explore the perceptions and experiences of health workers regarding how stigma influences their work with HIV-positive patients. The study employed a qualitative design involving individual semi-structured in-depth interviews with 14 health workers, purposively selected from hospitals and detention centres for people who use drugs and sex workers in Hanoi, Vietnam. Findings showed that the stigma experienced by health workers may be organised around several themes: (1) lack of social prestige associated with HIV work; (2) fear of infection expressed by family members; (3) feelings of being devalued within the healthcare field; and (4) work-related stress and burnout, especially for staff working in detention centres for drug users and female sex workers. Efforts are needed to improve the public image of HIV work, scale up stigma reduction, enhance stress management and create a safe and supportive working environment for health workers.