Automatic segmentation of liver tumors is crucial to assist radiologists in clinical diagnosis. While various deep learningbased algorithms have been proposed, such as U-Net and its variants, the inability to explicitly model long-range dependencies in CNN limits the extraction of complex tumor features. Some researchers have applied Transformer-based 3D networks to analyze medical images. However, the previous methods focus on modeling the local information (eg. edge) or global information (eg. morphology) with fixed network weights. To learn and extract complex tumor features of varied tumor size, location, and morphology for more accurate segmentation, we propose a Dynamic Hierarchical Transformer Network, named DHT-Net. The DHT-Net mainly contains a Dynamic Hierarchical Transformer (DHTrans) structure and an Edge Aggregation Block (EAB). The DHTrans first automatically senses the tumor location by Dynamic Adaptive Convolution, which employs hierarchical operations with the different receptive field sizes to learn the features of various tumors, thus enhancing the semantic representation ability of tumor features. Then, to adequately capture the irregular morphological features in the tumor region, DHTrans aggregates global and local texture information in a complementary manner. In addition, we introduce the EAB to extract detailed edge features in the shallow fine-grained details of the network, which provides sharp boundaries of liver and tumor regions. We evaluate DHT-Net on two challenging public datasets, LiTS and 3DIRCADb. The proposed method has shown superior liver and tumor segmentation performance compared to several state-of-the-art 2D, 3D, and 2.5D hybrid models.