Pushrim-activated power-assisted wheels (PAPAWs) are assistive technologies that provide on-demand torque assistance to wheelchair users. Although the available power can reduce the physical load of wheelchair propulsion, it may also cause maneuverability and controllability issues. Commercially-available PAPAW controllers are insensitive to environmental changes, leading to inefficient and/or unsafe wheelchair movements. In this regard, adaptive velocity/torque control strategies could be employed to improve safety and stability. To investigate this objective, we propose a context-aware sensory framework to recognize terrain conditions. In this paper, we present a learning-based terrain classification framework for PAPAWs. Study participants performed various maneuvers consisting of common daily-life wheelchair propulsion routines on different indoor and outdoor terrains. Relevant features from wheelchair frame-mounted gyroscope and accelerometer measurements were extracted and used to train and test the proposed classifiers. Our findings revealed that a one-stage multi-label classification framework has a higher accuracy performance compared to a two-stage classification pipeline with an indoor-outdoor classification in the first stage. We also found that, on average, outdoor terrains can be classified with higher accuracy (90%) compared to indoor terrains (65%). This framework can be used for real-time terrain classification applications and provide the required information for an adaptive velocity/torque controller design.