The objective of the present study was to investigate whether a previously reported apparent negative relationship between fat mass and daily physical activity in Japanese adult women would also be observed in Japanese adult men. The subjects were grouped into quartiles of BMI and body fat percentage (%BF). The number of steps walked each day and the duration of light- to vigorous-intensity physical activity were assessed by an accelerometer over the same period of time as for the doubly labelled water experiment. The results showed that BMI negatively correlated with the number of steps and time spent in moderate-intensity physical activity, whereas %BF showed a negative relationship with physical activity-related energy expenditure (PAEE)/body weight (BW) and physical activity level. The analysis of data using %BF quartiles revealed that PAEE/BW decreased from the second quartile in which the BMI was < 25 kg/m2. These observations are similar to those reported in our previous study in Japanese adult women. These cross-sectional studies cannot prove causality, and that obesity causes physical inactivity may be the case. However, the results of the present study provide information regarding which physical activity variables should be used in longitudinal studies.