The COVID-19 pandemic has highlighted how the healthcare system could be overwhelmed. Telehealth stands out to be an effective solution, where patients can be monitored remotely without packing hospitals and exposing healthcare givers to the deadly virus. This article presents our Intel award winning solution for diagnosing COVID-19 related symptoms and similar contagious diseases. Our solution realizes an Internet of Things system with multimodal physiological sensing capabilities. The sensor nodes are integrated in a wearable shirt (vest) to enable continuous monitoring in a noninvasive manner; the data are collected and analyzed using advanced machine learning techniques at a gateway for remote access by a healthcare provider. Our system can be used by both patients and quarantined individuals. The article presents an overview of the system and briefly describes some novel techniques for increased resource efficiency and assessment fidelity. Preliminary results are provided and the roadmap for full clinical trials is discussed.