This cross sectional survey was conducted to determine the support in making Penang UNESCO World Heritage Site (GTWHS) smoke free and to determine the influence of tolerance towards smoking on this support. This is the first phase in making Penang, Malaysia a smoke free state. A multistage sampling process was done to select a sample of respondents to represent the population of GTWHS. Attitude towards smoking was assessed using tolerance as a proxy. A total of 3,268 members of the community participated in the survey. A big majority (n = 2969; 90.9%) of the respondents supported the initiative. Support was lowest among the owners and residents/tenants, higher age groups, the Chinese, men, respondents who had poor knowledge of the places gazetted as smoke free, and respondents with poor knowledge of the health effects on smokers and on passive smokers. The odds (both adjusted and unadjusted) of not supporting the initiative was high among those tolerant to smoking in public areas. Tolerance towards smoking was associated with 80.3% risk of non-support in the respondents who were tolerant to smoking and a 57.2% risk in the population. Health promotion and education concerning the harm of tobacco smoke in Malaysia, which has mainly targeted smokers, must change. Health education concerning the risks of second hand smoke must also be given to non-smokers and efforts should be made to denormalize smoking.