As described in this paper, a physical activity classification algorithm is proposed for energy expenditure estimation. The proposed algorithm can improve the classification accuracy using both the triaxial acceleration and heart rate. The optimal classification also contributes to improvement of the accuracy of the energy expenditures estimation. The proposed algorithm employs three indices: the heart rate reserve (%HRreserve), the filtered triaxial acceleration, and the ratio of filtered and unfiltered acceleration. The percentage HRreserve is calculated using the heart rate at rest condition and the maximum heart rate, which is calculated using Karvonen Formula. Using these three indices, a decision tree is constructed to classify physical activities into five classes: sedentary, household, moderate (excluding locomotive), locomotive, and vigorous. Evaluation results show that the average classification accuracy for 21 activities is 91%.