Segmentation of skin lesions is a critical step in the process of skin lesion diagnosis. Such segmentation is challenging due to the irregular shape, fuzzy contours and severe noise interference in the skin lesion region. Existing deep learning-based skin lesion segmentation methods are usually computationally expensive, hindering their deployment in dermoscopic devices with poor computational power. To address these challenges, we propose an ultralightweight fully asymmetric convolutional network for skin lesion segmentation, called ULFAC-Net. we use a parallel asymmetric convolutional (PAC) module to extract features instead of the traditional square convolution, and innovatively propose a PAC module with dual attention (Att-PAC) to enhance the feature representation. Based on the PAC and Att-PAC modules, we further propose a lightweight textual information submodule. To balance the number of parameters and performance of the model, we also hand-design an asymmetric encoder-decoder architecture. In this paper, we validate the effectiveness and robustness of the proposed ULFAC-Net on four publicly available skin lesion segmentation datasets (ISIC2018, ISBI2017, ISIC2016 and PH2 datasets). The experimental results show that ULFAC-Net achieves competitive segmentation performance with only 0.842 million(0.842M) parameters and 3.71 gigabytes of floating point operations (GFLOPs) compared to other state-of-the-art methods.