Air-conditioning system plays a significant role in providing users a thermally comfortable indoor environment, which is a necessity in modern buildings. In order to save the vast energy consumed by air-conditioning system, the building envelopes in envelope-load dominated buildings should be well designed such that the unwanted heat gain and loss with environment can be minimized. In this paper, a new design of concrete wall panel that enhances thermal insulation of buildings by adding a gypsum layer inside concrete is presented. Experiments have been conducted for monitoring the temperature variation in both proposed sandwich wall panel and conventional concrete wall panel under a heat radiation source. For further understanding the thermal effect of such sandwich wall panel design from building scale, two three-story building models adopting different wall panel designs are constructed for evaluating the temperature distribution of entire buildings using finite element method. Both the experimental and simulation results have shown that the gypsum layer improves the thermal insulation performance by retarding the heat transfer across the building envelopes.