Forecasting the stock market prices is complicated and challenging since the price movement is affected by many factors such as releasing market news about earnings and profits, international and domestic economic situation, political events, monetary policy, major abrupt affairs, etc. In this work, a novel framework: deep predictor for price movement (DPP) using candlestick charts in the stock historical data is proposed. This framework comprises three steps: 1. decomposing a given candlestick chart into sub-charts; 2. using CNN-autoencoder to acquire the best representation of sub-charts; 3. applying RNN to predict the price movements from a collection of sub-chart representations. An extensive study is operated to assess the performance of the DPP based models using the trading data of Taiwan Stock Exchange Capitalization Weighted Stock Index and a stock market index, Nikkei 225, for the Tokyo Stock Exchange. Three baseline models based on IEM, Prophet, and LSTM approaches are compared with the DPP based models.