The outbreak of COVID-19 threatens the lives and property safety of countless people and brings a tremendous pressure to health care systems worldwide. The principal challenge in the fight against this disease is the lack of efficient detection methods. AI-assisted diagnosis based on deep learning can detect COVID-19 cases for chest X-ray images automatically, and also improve the accuracy and efficiency of doctors' diagnosis. However, large scale annotation of chest X-ray images is difficult because of limited resources and heavy burden on the medical system. To meet the challenge, we propose a capsule network model with multi-head attention routing algorithm, called MHA-CoroCapsule, to provide fast and accurate diagnostics for COVID-19 diseases from chest X-ray images. The MHA-CoroCapsule consists of convolutional layers, two capsule layers, and a non-iterative, parameterized multi-head attention routing algorithm is used to quantify the relationship between the two capsule layers. The experiments are performed on a combined dataset constituted by two publicly available datasets including normal, non-COVID pneumonia and COVID-19 images. The model achieves the accuracy of 97.28%, recall of 97.36%, and precision of 97.38% even with a limited number of samples. The experimental results demonstrate that, contrary to the transfer learning and deep feature extraction approaches, the proposed MHA-CoroCapsule has an encouraging performance with fewer trainable parameters and does not require pretraining and plenty of training samples.