MQTT-S and CoAP are two protocols able to use the publish/subscribe model in Wireless Sensor Networks (WSNs). The high scalability provided by the publish/subscribe model may incur a high packet loss and therefore requires an efficient reliability mechanism to cope with this situation. The reliability mechanism of MQTT-S and CoAP employs a method which defines a fixed value for the retransmission timeout (RTO). This article argues that this method is not efficient for deploying publish/subscribe in WSN, because it may be unable to recover a packet, therefore resulting in a lower packet delivery ratio (PDR) at the subscriber nodes. This article proposes and evaluates an adaptive RTO method, which consists in using a Smooth Round-trip Time and multiplying it by a constant parameter (K). Thanks to this method, the reliability mechanism of MQTT-S and CoAP would be able to react properly to packet loss and would also be lightweight in terms of energy, memory and computing for sensor nodes where these resources are critical. We present a detailed evaluation of the effects of the K value on the calculation of the adaptive RTO method. We also establish the setting for obtaining the highest PDR on the subscriber nodes for single-hop and multi-hop scenarios. The results for single-hop scenario show that use of the appropriate K value for the adaptive RTO method increases the PDR up to 76% for MQTT-S and up to 38% for CoAP when compared with the use of fixed RTO method for both protocols, respectively. Meanwhile the same comparison for multi-hop scenario, the adaptive RTO method increases the PDR up to 36% for MQTT-S and up to 14% for CoAP.