Tracker Delay & Tuesday Morning Resolution

Tracker update issue overnight being resolved Tuesday morning. Positions will begin updating throughout the morning, including the manually reported 0600 times direct from the fleets daily roll call. All boats checked in per daily morning check in protocal and all is well with the fleet. Estimating Rio100 as first monohull finisher after dark. YB Tracking issued a statement on the technological explanation for the tracker down time. They have been in contact with SDYC throughout the process and are working to keep the event running smoothly for the competitors and race fans.

Note from YB Tracking
March 6, 2018, 1000

An apology to all viewers and to SDYC from YB. We had a technical issue which caused all trackers to stop transmitting on-board the boats for an extended period. The problem has been resolved by a physical reset of the tracker on the yachts - this has taken some time to resolve since each skipper had to perform the reset on the trackers.

It isn't up to our usual standard. We have found the root cause of the issue (which initially happened with Timeshaver on Friday, and subsequently with the entire fleet overnight) and are confident that the same issue will not reoccur on this, or any other race.

As with all problems, it was actually caused by several factors:

  • The first was the particular mode the trackers were in (taking a fix every 20 minutes, transmitting 3 positions every 60). There was a bug in the way we were interpretting this data, which caused it to look odd in our own management system.
  • The second, there is a 60 minute delay implemented on the data, which was making transmissions look older than they actually were.
  • And the third, was the combination of the first two - our on-call engineer, late at night, was looking at the data and warnings from our monitoring system, and considered that a serious problem had occurred with all the trackers on the race. He therefore sent a reset command to the trackers remotely in order to resolve the problem. This reset command did indeed reset the trackers, but also put them back into standby mode waiting to be activated again - thus causing a massive problem out of a relatively minor one.

The bug in the special 60/20 mode data interpretation has been fixed, all engineers educated NOT to do that ever again without a second opinion, and the remote reset command protected from 'easy' use.

We try our best to always be honest in our explanation of a problem, and try to always learn from bad experiences. Hopefully everyone will agree we do a decent job the vast majority of the time, sometimes we mess up - sorry SDYC.