Hypertension is a common and chronic disease and causes severe damage to patients' health. Blood pressure of a human being is controlled by the autonomic nervous system. Heart rate variability (HRV) is an impact of the autonomic nervous system and an indicator of the balance of the cardiac sympathetic nerve and vagus nerve. HRV is a good method to recognize the severity of hypertension due to the specificity for prediction. In this paper, we proposed a novel fine-grained HRV analysis method to enhance the precision of recognition. In order to analyze the HRV of the patient, we segment the overnight electrocardiogram (ECG) into various scales. 18 HRV multidimensional features in the time, frequency, and nonlinear domain are extracted, and then the temporal pyramid pooling method is designed to reduce feature dimensions. Multifactor analysis of variance (MANOVA) is applied to filter the related features and establish the hypertension recognizing model with relevant features to efficiently recognize the patients' severity. In this paper, 139 hypertension patients' real clinical ECG data are applied, and the overall precision is 95.1%. The experimental results validate the effectiveness and reliability of the proposed recognition method in the work.