Hurricane Matthew was the deadliest Atlantic storm since Katrina in 2005 and prompted one of the largest recent hurricane evacuations along the Southeastern coast of the United States. The storm and its projected landfall triggered a massive social media reaction. Using Twitter data, this paper examines the spatiotemporal variability in social media response and develops a novel approach to leverage geotagged tweets to assess the evacuation responses of residents. The approach involves the retrieval of tweets from the Twitter Stream, the creation and filtering of different datasets, and the statistical and spatial processing and treatment to extract, plot and map the results. As expected, peak Twitter response was reached during the pre-impact and preparedness phase, and decreased abruptly after the passage of the storm. A comparison between two time periods-pre-evacuation (October 2th-4th) and post-evacuation (October 7th-9th)-indicates that 54% of Twitter users moved away from the coast to a safer location, with observed differences by state on the timing of the evacuation. A specific sub-state analysis of South Carolina illustrated overall compliance with evacuation orders and detailed information on the timing of departure from the coast as well as the destination location. These findings advance the use of big data and citizen-as-sensor approaches for public safety issues, providing an effective and near real-time alternative for measuring compliance with evacuation orders.