Analysis of laboratory animal behavior allows assessment of animal wellbeing. We present a method for the classification of different activities of laboratory mice by analyzing video clips using three deep learning methods. Animals placed in observation cages are filmed and short video clips are labelled as belonging to one of five defined behaviors. Subsequently, three different methods based on convolutional neural networks (CNNS) are applied to classify the clips. The best performing method - a two-stream network that analyzes individual frames as well as the video's optical flow - achieves an accuracy of 86.4%, including detection of important behavioral patterns such as self-grooming. These results show that the presented analysis protocol allows automated assessment of animal behavior by algorithmic analysis of videos of mice on observation boxes.