Smart card data are widely used in generating the origin and destination (O-D) matrix for public transit, which contains important information for transportation planning and operation. However, the generation of the O-D matrix is limited by the smart card data information that includes the boarding (origin) information without the alighting (destination) information. To solve this problem, trip chain methods have been proposed, thereby greatly contributing in estimating the destination using the smart card data. Nevertheless, unlinked trips, that is, trips with unknown destinations, are a persisting issue. The purpose of this study is to develop a method for estimating the destination of unlinked trips, in which trip chain methods cannot be applied, using temporal travel patterns and historical boarding records of the passengers based on long-term smart card data. The passengers were clustered by k-means clustering, and the time-of-day travel patterns were estimated for each cluster using a Gaussian mixture model. The travel patterns were formulated to estimate the destination of the passengers from the smart card data. The proposed method was verified using the 2018 smart card data collected in Sejong City, South Korea. The existing trip chain method matched the destinations of 60.0% of the total trips, whereas the proposed method improved the matching to 74.9% by additionally matching the destinations of 37.2% of the unlinked trips.