Sedentary behavior (SB) expression and its underlying causal factors have been progressively studied, as it is a major determinant of decreased health quality. In the present study we applied Genotype x Age (GxAge) and Genotype x Sex (GxSex) interaction methods to determine if the phenotypic expression of different SB traits is influenced by an interaction between genetic architecture and both age and sex. A total of 1345 subjects, comprising 249 fathers, 327 mothers, 334 sons and 325 daughters, from 339 families of The Portuguese Healthy Family Study were included in the analysis. SB traits were assessed by means of a 3-d physical activity recall, the Baecke and IPAQ questionnaires. GxAge and GxSex interactions were analyzed using SOLAR 4.0 software. Sedentary behaviour heritability estimates were not always statistically significant (p>0.05) and ranged from 3% to 27%. The GxSex and GxAge interaction models were significantly better than the single polygenic models for TV (min/day), EEsed (kcal/day), personal computer (PC) usage and physical activty (PA) tertiles. The GxAge model is also significantly better than the polygenic model for Sed (min/day). For EEsed, PA tertiles, PC and Sed, the GxAge interaction was significant because the genetic correlation between SB environments was significantly different from 1. Further, PC and Sed variance heterogeneity among distinct ages were observed. The GxSex interaction was significant for EEsed due to genetic variance heterogeneity between genders and for PC due to a genetic correlation less than 1 across both sexes. Our results suggest that SB expression may be influenced by the interactions between genotype with both sex and age. Further, different sedentary behaviors seem to have distinct genetic architectures and are differentially affected by age and sex.