How Accurate Are Fitness Trackers?

Wearable fitness trackers like the Fitbit and Jawbone Up are all the rage, but are they accurate? We wear them to gauge how much we move and how many calories we burn. But if they’re not accurate, you might be sabotaging your weight loss goals by eating more than you really burn.

I previously looked at calorie counters on heart rate monitors and exercise machines at the gym and found they overestimated the calories you burn significantly: Watches with heart rate monitor straps overestimated by 28% and gym machines by 19%. Now that activity trackers are all the rage, I thought I would check them with the same methodology.

The Trackers
I’m trying out the Fitbit Flex, Nike Plus Fitness on my iPod Nano, BodyMedia’s FIT armband and the Jawbone UP. These trackers aim either to track workouts or make people more aware of how sedentary their daily routines are. They estimate distance and calories burned. All these devices use accelerometers – which use movement rather than GPS to determine the

distance you travel. None are marketed to be precise measures, but I think it’s important to know if the variance is a little or a lot. The more you know, the better off you are.

I read the instructions for all devices and set them up accordingly. Most ask for my age, weight, height, and gender. One asks if I’m left or right handed. None force or suggest any further calibration or more quantitative information about my stride or body measurements during the set-up.

Experiment #1 – Testing Distance: Two Miles Around The Track
I run the 8 laps plus 18 meters hugging the inside lane edge to get my distance as close to 2 miles as I can. I finish tired but hopeful that my activity monitors recognize the distance I’ve covered.

The BodyMedia only measures steps, and it says I took 3659. But even that number differs from the other two devices that also count steps.

And what really surprises me is not the variance in distance (they may all estimate my stride length differently and have slightly different distance algorithms) but it’s the variance in the step counts, which I imagined as a fairly standardized pedometer-like measure of movement. At the low end, the BodyMedia says I took 3659 steps, and at the high end the Jawbone UP thinks I took nearly 300 more steps at 3947. That’s a big discrepancy.

So what?
These devices are meant as relative trackers of motion – ways to quantify how much you move. Then, armed with that awareness, they motivate you to move more. So the exactness of the measures may not be critical. You moved more than yesterday by taking the stairs, going for a walk on your lunch break and playing with the kids – good for you!

Did you earn that candy bar?
But the problem is that these devices also display of how many calories they estimate you burn over the entire day. Weight loss happens when you eat fewer calories than you burn. So if you base your food intake on the amount of calories an exercise tracker says you burned, you want that caloric number to be as accurate as possible.

Experiment #2 - Testing Calorie Estimates
The variance in steps and distance has me concerned, so I head over to the San Francisco State University Kinesiology Lab. My friend, Professor Matt Lee performs an indirect calorimetry test on me to accurately measure how many calories I burn and compare the results to the numbers from the fitness trackers. I put on a snorkel-like apparatus hooked up to a computer that analyzes my breathing. The amount of oxygen I exhale is the true measure of my energy or caloric expenditure. I wear my activity trackers simultaneously to see how they compare to the lab’s gold standard for caloric measurement.

After 10 minutes of running with the snorkel gear on, Matt tells me the indirect calorimetry test shows I expended 96 calories. But that fewer calories than all four of my devices say I’ve burned – in some cases, far less.

I ask Matt how he would explain the differences? “Keep in mind you are one individual.” And he’s right; this is not a scientific study, just one person trying to figure it out. Matt says to truly know if the numbers are off for everyone, you’d have to use a much larger sampling of people, and look at averages.

Calibration

You can calibrate some of these devices, and I strongly recommend you do that to get as accurate a result as possible.

• Fitbit – I have to dig to find it on their website, but in my settings there is an option to input my stride length. I measure my walking and a running stride lengths, then enter them into my profile.

• With the Jawbone UP and the Nike, you can perform a calibration by going into settings and telling the devices the actual distance of the walk or run you just did. I set them after the track and do get better accuracy.

• I searched high and low on the BodyMedia site and in their app and can find no calibration options.

Final Word

These devices are incredibly handy and great for encouraging users to move more. So while I do advocate calibrating your device a little so you can get a feel for the margin of error. I think the true value is in the relative activity levels they monitor, not in the exact measurements. Matt Lee from San Francisco State says it best, “Take it for what it is. It’s a number. Use it to get off the couch and get moving.”

[Related: Worst Tech Rip-offs]