Running Watch Accuracy Test

I just got home from Boston after running the weekend races. I took my trusty FR620, FR920XT, Fenix3 and SirfSTAR GPS logger with me and bought a TomTom Runner at the expo just to get one more data point for the comparison run that was not Garmin. My hypothesis before the running was that the Fenix 3 would do great with Boston given it's relatively straight line running course with very few turns and it would do better than most watches in downtown thanks to Glonass. I also suspected that the FR920XT would probably do just as good if not better. So how did they do?

I ran both the 5K then two days later the marathon. The 5K is strictly in downtown with heavy building cover and runs the last mile of the marathon course under these tough conditions.

The 5K weather was perfect, clear skies, full sun, as good as it gets for a race and for GPS. The marathon was full overcast with moderate to heavy rain throughout the race with stiff headwind, basically the worst you can have for a race or GPS or pretty close to it. It turned out pretty good to have both of those conditions as far as GPS reception goes as it really tells a better story. Please ignore my times and the horrible positive split in the marathon, given that I ran a total of 100 miles in the last 3 months with the longest run being a 10 miler due to injury has something to do with it. But let's look at the data...

Here are the distance differences relative to the FR920XT which I used as reference:

LoggerFR620Fenix3

You can see that the Fenix 3 measured the shortest while the logger measured the longest which is no surprise, this has been the case in all my testing so far. What you can see is how the Fenix 3 cut off the turnaround on Commonwealth while the other units did way better on that turn. The underpass did not pose a huge challenge to any of the units. The turn to Hereford looks pretty decent on all units even if the Fenix 3 and the FR620 were offset form the other two. Once on Boylston the story changed and you can see that GLONASS really came through and made the Fenix 3 and the FR920XT shine. Overall I think these tracks look great and as good as anyone could expect from these units, the only caveat being that it seems like the Fenix 3 has to resort to heavy filtering to achieve this look which ends up costing in turns.

Here are the distance differences relative to the FR920XT which I used as reference:

LoggerFR620Fenix3TomTom Runner

The logger measured way longer than anything else and it also seem to have lost its marbles at the 2 hour mark for about 7 minutes where I see no real reason for it. The logger was in a front pocket of my running shorts and it very well could have had something to do with my shorts being soaked from the rain by this point but it picked up and recovered after the incident.

If we zoom in and take the Logger's data out of the view then you can see that the rest of the units tracked close, within 500 ft of each other for the entire race with TomTom measuring the shortest, The FR920 the second shortest, then the Fenix 3 and finally the FR620. This is kind of what I expected since there were no turns for the Fenix to lose distance over the FR920 and I still thin kthe 920 is more accurate for distances. The TomTom's measurements while shorter than any of the others seem to be the closest to the actual distance and the track looks really good. Mind you, this is all in full overcast and pouring rain for the most part.

LoggerFR620Fenix3TomTom Runner

Things are looking good overall until we get past the Newton Hills and start rolling down past the cemetery where the Fenix 3 decided to take a detour thanks to multipath and some taller buildings on the right side. Once we got to Fenway Park the situation for most units became too much to bear and things got ugly. I'm actually quite shocked just how ugly they got. Boylston street was a disaster for pretty much all "GPS participants" in this race and the Fenix 3 fared just as bad as an FR620 without GLONASS. The FR920XT hanged on for dear life and did the best it could in the conditions and the TomTom unit really put on a valiant effort when compared to the abysmal performance of the GPS logger, the Fenix 3 and the FR620.

Even with all these route distractions the distance measured by the Fenix 3 on the watch were not bad, it claimed that I ran 26.38 miles while the FR920XT said it was 26.35 miles both of which are as close as anyone can get on a marathon. The TomTom said it was 26.42 miles while the FR620 reported 26.47 miles during the run. I prefer to concentrate on these numbers rather than the re-calculated numbers by mygpsfiles or SportTracks as those are not the ones I run/pace off of.

In SportTracks the distances were:

Conclusion

From last weekends running you can see that weather conditions can have a tremendous effect on GPS reception quality, especially when the GPS reception is already compromised by tall buildings or tree cover.

The Fenix 3 filtering is still the same (ran with the FW3.2 for the marathon, FW3.1 for the 5K) and it tries to help with the sub-par quality of the GPS track that it might produce otherwise.

A $100 TomTom without GLONASS performed almost as good as the FR920XT for the run and is definitely better than the Fenix 3. If you're the kind of guy who'd like to see the same number being calculated everywhere then TomTom is the watch for you as it was the only contender that matched its in-run distance with the post-run distance.

I think the FR920XT is still the king of Garmin watches and I'm definitely not ready to sell it just yet. The Fenix 3 has a long way to go to catch up to the FR920XT.

However a log from a friend of mine who ran a much better Boston than I did on Monday with the Fenix 3 both GPS and time wise so it can very well be a large variance between unit to unit on the Fenix 3. This is his second Fenix 3, the first one he returned because it was measuring short on his tree covered city park route and the new one's serial number was only a hundred or so different from the old one. This would also explain why some people report absolutely zero problems while others report serious problems. However on his run it seems like the ANT+ stack died 2 hours 11 minutes into his run as his cadence measurements took a serious change (switched off of HR+RUN) and his temperature rose (switched from Tempe to internal) and his HR data went away.

Speaking of Temperature, my Tempe stayed connected the whole way and I did lose it only a handful of times in the entire run. I think it should be an easy task to filter out these spikes from the log via a firmware update and I also think it is absolutely necessary for Garmin to do so as the spikes are annoying but not as annoying than the huge variance between the two Fenix 3 tracks I just posted which is the most concerning problem of all.

Original files: https://dl.dropboxusercontent.com/u/...Boston2015.zip

Another thing that also happened is that according to my Fenix 3 I set 8 new records on this run, one of them being the fastest mile in 0:00 (yes, zero zero) and the fastest 5K in 15:56 as well as 40:53 10K and 1:33 half, none if which were part of this run. So there is something seriously screwed up there too.

I wonder if the fact that I used a course navigation during this run had anything to do with any of these things, including the massive disparity between my GPS track and my friend's and the disparity between the log quality of Saturday's 5K and the marathon. Now that I looked at the track closer it almost looks like that GLONASS did not work even though I had it set to be on.

Read more Garmin Fenix 3 reviews on Amazon.com

This was a guest post of Greg Takacs