For the past several weeks, I have been wearing an Apple Watch (nerd bling!), a Fitbit Charge HR, and not exactly wearing but still carrying my mobile phone like all the other technology addicted almost-middle aged people out there. The phone has the Withings app on it because I track my weight so that I can cycle back and forth between exuberance (Alright! I lost 5 pounds!) to bummed out (And I just gained 7 pounds. PBBBBBBT!!!!). The Withings App tracks steps based on the M7 chip inside of my iPhone 5S (why yes, I give Apple a lot of my money, thanks for asking). So I wanted to see how the three compared to each other. This is a casual comparison - I'm not running statistical tests or anything this time around.
The ground rules
After getting my Apple Watch, I decided to do this comparison thing. I was going to wear both the Fitbit and the Watch each day as I normally would. What's normal? When I wake up in the morning, I put on each device. The Watch goes on my non dominant (left) arm. The fitbit goes on my dominant (right) arm. I wasn't picky about which went on first - usually, it was whichever I could fumble and grab first. I decided some time ago (about when I discovered my wrist was getting sort of smelly from the fitbit charge) that I was not going to wear any devices at night nor track my sleep. (Yeah, bad self tracker, I know.) I set the recording of each device to the correct arm according to their corresponding apps. I carried my phone with me as I would normally - which means it goes with me to breakfast and goes with me when I walk the dog in the morning. When I get to work, it stays with me for the most part, but I will forget and leave it on my desk. At the gym, it sometimes goes with me onto a treadmill but it might also linger in the locker. If I am going to be going into water, like the pool or the beach or into the shower, I take off all devices and keep the phone away from water. For the most part, I went about my life.
Normal life involves walking the above mentioned dog, going on outings with the family, doing simple social activities with friends, and working at a job that involves a lot of sitting near a computer and cursing at the photocopier. I occasionally forget my phone for some reason, panic, and then gradually accept that I am phoneless for several hours. That is normal, and I did not track what days I left my phone somewhere. That is just life. I was trying to run an experiment that was fairly true to my life.
For days tracked, I picked the window between 6/3/2015 and 7/21/2015. Are these special days? An anniversary or obscure holiday? No. I got the watch on the 2nd, but it was late in the day so I didn't have a full day of data on that whereas the other devices/app had more time to be attached to my person. I actually wanted to go all the way through July, but my phone had been having issues and I had to get it replaced (hooray applecare! and yes, more money to Apple for applecare), so I lost Withings step data after the 21st. That's why we end then.
Getting Data Out
QS Labs was kind enough to release an iOS app called QS Access that let me get a .csv file of my Apple Health data. (Note: when I peak at the data in the health app, it seems that some of the data points, like a span of a minute or two, is from the phone rather than the watch? But I didn't care enough to dig into it and assume this is apple being smart about getting a more thorough picture. I'm sure I can read some message board and get a lot more details, but I'm just calling data from the health app "watch data" even though it isn't 100% true).
Fibit data I grabbed from the dashboard of Fitbit. I know there are hacks to grab data - in fact, I'm associated with one of them that will grab it in minute increments - but I just wanted to compare daily totals. Maybe one day in the future I will look at minute by minute or hour by hour or some other increment to see if there is something cool going on.
Withings lets you just export .csv files from their web interface, so that was easy enough. Just a few clicks here and there. Then stick it all into a spreadsheet and make a few plots. Again, I'm being lazy about this and am not running any serious statistics. I have had the most experience with Fitbit devices (I once tested the zip against the flex and saw the flex undercounted quite a bit relative to the zip), so I figured I'd use the fitbit devices as a baseline. Anyway, here are the results.
Result 1: Fitbit Charge HR tends to count more steps
If you take into account my previous experience showing that the flex seemed to undercount relative to the hip based clip on zip (which is more similar to research grade pedometers that exercise science people use), then we might assume that the Charge HR undercounts relative to whatever is my true number of steps. (Also, published research suggests as much). No matter though - I'd rather have an undercount than overcount so that I push myself a little more. But what is interesting is that the Charge HR, assuming it was undercounting, was counting still more steps than the other two. See the poorly labeled graph that has not been cropped below.