WHOOP Proven Most Accurate Wearable in Heart Rate & Heart Rate Variability Measurements

How does WHOOP stack up against other wearables compared to gold-standard assessment of heart rate and sleep? An overview of new third-party published studies and what they mean for WHOOP.

Wearable technology has the power to help people better understand their bodies and improve their health through personalized insights. However, recommendations across sleep, recovery, and training are only actionable if the underlying physiological data are accurate and reliable. Heart rate variability (HRV) and heart rate (HR) are two metrics for which accuracy and reliability are vital as they are at the heart of all of the WHOOP scores. The Australian Institute of Sport (AIS) funded an independent, third-party study conducted by Central Queensland University (CQUniversity) which was recently published in Sensors, a leading international, peer-reviewed journal, examining the accuracy of six different wearable devices for estimating HR, HRV, and sleep. The study found that WHOOP is 99.7% accurate in measuring heart rate and 99% accurate* in measuring heart rate variability – levels of accuracy that surpassed all other wearables in the study. By contrast, other wearables scored between 41% and 96% in HR accuracy and 24% and 69% in HRV accuracy. The study also found that WHOOP was excellent in identifying sleep when compared to the gold-standard polysomnography (PSG) and outperformed the other devices in calculating total time asleep.

AIS also funded a second study published in Sensors focused specifically on the reliability of WHOOP 3.0 for use in guiding training. This new research reviewed the day-to-day variability of HR and HRV, finding that WHOOP measurements also had a high level of reliability.

VALIDATION OF SIX WEARABLE DEVICES

AIS is a government-funded agency and global leader in high performance, supporting Australian athletes on their journey to achieve international success. CQUniversity independently examined the validity of six commonly used wearables (WHOOP 3.0, Oura Ring Generation 2, Apple Watch S6, Garmin Forerunner 245 Music, Polar Vantage V, and Somfit) for assessing sleep, heart rate, and heart rate variability during sleep. The study was conducted at The Sleep Lab at CQUniversity’s Appleton Institute for Behavioural Sciences in Wayville, South Australia and participants were fitted with all six wearable devices along with ECG and polysomnography (PSG), the gold-standard of sleep tracking. It’s often challenging to compare the accuracy of wearables across multiple studies as each one may have a different design and population. By having every participant wear all devices simultaneously, this new research provides a clean view of how each wearable performs under truly equivalent conditions.

VALIDATING ACCURACY: HEART RATE AND HEART RATE VARIABILITY

The study found that WHOOP 3.0 was the most accurate of the wearables evaluated in measuring HR compared to the gold-standard ECG-derived heart rate. WHOOP had the smallest errors in measurement with a standard deviation in data of 1 beat per minute (bpm) while the other wearables ranged from 2.1 to 12.8 bpms. A Bland-Altman plot is a simple way to visualize the differences between two quantitative measurements. In the below analysis of HR measurements from the wearables in the study, the X-axis shows the gold-standard measured data from each study participant. The Y-axis shows the difference between that gold-standard and what each device reported.

Any data clustering along the flat line at zero were perfect (i.e. the device calculated the same value as ECG) and errors are shown above and below that line. In comparison to other leading wearables, WHOOP 3.0 clearly delivered the most accurate HR measurements. WHOOP also had by far the smallest error in HRV with a standard deviation of 3.9 milliseconds (ms) while other wearables ranged from 28.1 to 46.9 ms.

The analysis of HRV clearly shows that WHOOP excelled in measuring HRV while other wearables were unreliable in identifying accurate measurements compared to gold-standard. Notably, several other wearables struggled to measure higher HRVs - with some even underestimating some values by 200 ms.

SLEEP STAGING: IDENTIFYING REM & SLOW WAVE SLEEP

In addition to validating WHOOP 3.0 as providing the most accurate HR and HRV measurements of any wearables in the study, researchers also tested sleep tracking capabilities compared to gold-standard PSG data. WHOOP and other devices like Apple Watch and Oura Ring calculate sleep via photoplethysmography (PPG), a technique that involves measuring blood flow by assessing superficial changes in blood volume. When you shine specific wavelengths of light onto the skin, blood volume can be measured by looking at the light reflected back from our skin since blood absorbs specific colors and reflects others. Once blood flow is measured, we can then derive heart rate, heart rate variability, and respiratory rate, all of which are used in the WHOOP sleep detection and staging algorithm. Among all wearables in the study with PPG sensors, WHOOP had the greatest agreement with gold-standard PSG-derived REM and slow wave sleep (SWS) measurements. WHOOP was also found to be excellent in identifying sleep overall.

In reviewing the four stages of sleep, REM sleep is when your brain is restored and SWS sleep is when your muscles repair and grow. Those two stages make up restorative sleep which is essential for recovery. WHOOP is able to combine its industry-leading accuracy in classifying restorative sleep with actionable, personalized insights to help people improve their health.

EVALUATING THE RELIABILITY OF WHOOP HR AND HRV

In a second recent, independent, third-party study, AIS sought to determine the reliability of HR and HRV measurements by WHOOP to gauge athlete's readiness to perform in everyday use. Researchers monitored athletes wearing WHOOP over a 16-week period, comparing their WHOOP automated readings to manual data collection. Readiness to perform is especially important for athletes and their coaches -- by better understanding the impact of stress, coaches can prioritize recovery and tailor training protocols to avoid overtraining. WHOOP performed well compared to manual readings in a much more convenient form factor. According to the study findings, “WHOOP allows for frequent and convenient measurement of HR and HRV” and it can be “confidently utilised by sport and exercise science practitioners to record HR and HRV in practical settings.”

WHAT THIS MEANS FOR WHOOP 4.0

The validation provided by both studies from the Australian Institute of Sport underscore the quality of WHOOP measurements that underpin the personalized recommendations we provide to our members. We’ve built upon the industry-leading accuracy in WHOOP 3.0 as a foundation for WHOOP 4.0. In addition to designing hardware that is 33% smaller than WHOOP 3.0, we aligned on an upgraded sensor configuration that contains 5 LEDs (three green, one red, and one infrared) and 4 photodiodes to further improve accuracy. Research and development was conducted at WHOOP Labs and in real-world environments to test WHOOP 4.0 against a variety of member use cases. Diversity in study participants was also essential to ensure optimal performance in all conditions, including people of varying ages, body types, fitness levels, skin tones, genders, and handedness. Thousands of people participated in data collections for WHOOP 4.0 prior to its launch, resulting in more than 20,000 data sets pairing WHOOP with measurements from chest straps and ECGs. The WHOOP 4.0 trial process included more than 50 million heart beats which enabled our signal processing team to refine and train our algorithms for even more accurate heart rate measurements. WHOOP continues to invest in ongoing studies - both in person and virtually - to develop new features and enhance proprietary algorithms for further validations. WHOOP is committed to providing the most accurate, industry-leading wearable technology in addition to a best-in-class membership experience. WHOOP Labs is always conducting research and looking for different people to participate in studies. Visit whoop.com/whoop-labs to learn more about how to take part in current data collections in Boston.   *When compared to the gold-standard electrocardiogram-derived (ECG) metrics