This study aimed to evaluate the performance of cuffless blood pressure (BP) measurement techniques in a large and diverse cohort of participants. We enrolled 3077 participants (aged 18-75, 65.16% women, 35.91% hypertensive participants) and conducted followed-up for approximately 1 month. Electrocardiogram, pulse pressure wave, and multiwavelength photoplethysmogram signals were simultaneously recorded using smartwatches; dual-observer auscultation systolic BP (SBP) and diastolic BP (DBP) reference measurements were also obtained. Pulse transit time, traditional machine learning (TML), and deep learning (DL) models were evaluated with calibration and calibration-free strategy. TML models were developed using ridge regression, support vector machine, adaptive boosting, and random forest; while DL models using convolutional and recurrent neural networks. The best-performing calibration-based model yielded estimation errors of 1.33 ± 6.43 mmHg for DBP and 2.31 ± 9.57 mmHg for SBP in the overall population, with reduced SBP estimation errors in normotensive (1.97 ± 7.85 mmHg) and young (0.24 ± 6.61 mmHg) subpopulations. The best-performing calibration-free model had estimation errors of -0.29 ± 8.78 mmHg for DBP and -0.71 ± 13.04 mmHg for SBP. We conclude that smartwatches are effective for measuring DBP for all participants and SBP for normotensive and younger participants with calibration; performance degrades significantly for heterogeneous populations including older and hypertensive participants. The availability of cuffless BP measurement without calibration is limited in routine settings. Our study provides a large-scale benchmark for emerging investigations on cuffless BP measurement, highlighting the need to explore additional signals or principles to enhance the accuracy in large-scale heterogeneous populations.