Podcast No. 62: WHOOP Sleep Validation Study

On this week's podcast, Kristen Holmes and Emily Capodilupo discuss the results of the study and what it means for WHOOP.

A recent study conducted by the University of Arizona found that WHOOP is outstanding at tracking sleep. Researchers tested the accuracy of WHOOP against the gold-standard in sleep evaluation, a polysomnography test, and concluded that WHOOP is excellent by comparison. Better yet, this study also discovered that WHOOP improves sleep habits. From our earliest days, we have always believed that accuracy and actionable insights will be what delivers a superior user experience for our members. This is a validating moment for all of us on our mission to optimize human performance. On this week's podcast, Kristen Holmes and Emily Capodilupo discuss the results of the study and what it means for WHOOP.

Sleep Validation Study Podcast Show Notes:

2:10 - Setting the Bar. “Sleep is the key to recovery and one of the best things people can do to improve their mental and physical health. It’s vital that wearable devices are as accurate as possible considering the immense value of sleep and the widespread adoption of tools to track sleep. The bar should be high.” 3:23 - The Anatomy of a Study. Emily details the 3-year validation process. “This paper, which might just seem like a couple of pages of validation, is three years of process in the making.” 6:03 - Expecting Strong Results. “We had already done hundreds of internal validation studies, so we knew exactly what they were going to say before they said it. The value that our members saw in getting a third party to validate [our findings] was if WHOOP says WHOOP is great that only carries so much weight.” 7:20 - Getting the Right Data. “One of the things that’s particularly challenging about any device that is trying to do sleep staging from the wrist is that these different sleep stages are defined by changes in brain waves. You can’t read brain waves from the wrist. So you’re measuring heart rate and heart rate variability and respiratory rate and movement which are all downstream correlates of what’s going on in your brain, but you’re actually not making a direct measurement of any of those things.” 8:29 - Creating an Accurate Device. “WHOOP was actually built from the ground up, all the way from the hardware, to be optimized to get the most accurate heart rate data, to get the most accurate sleep data that we possibly could. Certain optimizations that we’ve made from different studies that we’ve done over time have changed everything from the form factor of the wrist to even the material of the band to make it easier to get a clean heart rate signal, because from heart rate, from the PPG signal on WHOOP, we get the heart rate, we get heart rate variability, we get respiratory rate, all of those things come from heart rate. So if the signal that the physical device is able to read is cleaner, all of those things become more accurate, so then the inputs to the sleep staging algorithm are cleaner so that we’re better able to stage sleep.” 9:27 - Not Just Sleep. “While this paper in so many ways looks like it’s a validation of sleep, it’s really a validation of the whole platform and everything that goes into it.” 11:48 - Making Positive Change. “[The study showed] just by wearing WHOOP, self-reported measures of sleep quality improved. This is something we talk about a ton… There’s something really powerful about telling someone they’re getting a B- in sleep. They hate it and they improve. You have to quantify something in order to know what you’re trying to work on.” 13:45 - Improved Sleep with WHOOP. “I think it shows the power of the product and all the good work that’s gone into it. I want to highlight that finding because I think it is so interesting and somewhat, I think, unique. I don’t think it’s something people think about as much. They think about quantifying everything but the fact that we’re driving positive behavior change is special.” Kristen adds that modifying behavior represents the “keys to the kingdom.” 14:54 - What is Polysomnography? Emily details what a Polysomnography is and why it’s considered the “gold standard” in sleep testing. 15:42 - Accuracy. “One outcome that was extremely exciting for us is that heart rate, heart rate variability, and respiratory rate were all extremely accurate, within one unit of truth on average throughout the sleep. It was really exciting. Those are the inputs out that our whole sleep algorithm is built on, so if those measurements weren’t accurate, nothing else could possibly be as accurate. We also measured our ability to detect sleep and wake … and then slow-wave sleep and REM sleep. All of those agreements to the gold-standard were really, really high. 17:00 - Old School. Emily describes how a Polysomnography is conducted. “Something you might not expect about a gold-standard measurement is that it’s actually done by hand. It’s an extremely manual process.” 18:04 - How Sleep is Difficult to Track. “A lot of sleep is in this grey area that’s not quite one sleep stage or another. Physiologically, the explanation for this is that it’s not really true that sleep stages are these discrete categories… It’s a spectrum.” 21:49 - The 4 Inputs That Measure Your Sleep on WHOOP. “Motion, heart rate, heart rate variability, and respiratory rate.” 23:05 - Validation for WHOOP. “It’s really great to see everybody’s hard work formally recognized in this way. It’s one thing when we do all of our own analysis … we have a lot of control over all of the internal validation that we do and then we have this idea of how good WHOOP is. But when you go to do this third-party validation study, you almost have to hand over your baby and all the keys and trust that everything you’ve been doing internally is actually going to hold up in this world where you have no control over it. To hand these keys over to [the University of Arizona researchers] and for them to publish and show that everything that we thought about the product really does stand up in this rigorous test of its accuracy and performance was really exciting.”