Activities of daily living is an important entity to monitor for promoting healthy lifestyle for chronic disease patients, children and the healthy population. This paper presents a smartwatch and earbuds inertial sensors based multi-modal power efficient end-to-end mobile system for continuous, passive and accurate detection of broad daily activity classes. We collected various posture, stationary and moving activity data from 40 diverse subjects using earbuds and smartwatch and develop the novel power optimized end-to-end operational system consisting of i) optimized device sampling rates and Bluetooth packet transfer rates, ii) data buffering mechanism, iii) background services, and iv) optimized model size, and demonstrating 93% macro recall score in detecting various activities. Our power optimized solution uses 80%, 40% and 33.33% less battery power for the smartphone, smartwatch, and earbuds respectively, compared to a power agnostic system with an estimated continuous no-charging run time of 50 hours, 16.67 hours, and 25 hours for the smartphone, smartwatch, and earbuds respectively.Clinical relevance- The end-to-end power optimized activity detection system presented in this paper will assist practicing clinicians toward treatment of various chronic disease patients (e.g. diabetes, hypertension, heart disease and obesity) by long-term, continuous monitoring of their lifestyle and sedentary behavior.