Part 1 of our Positioning Engine Deep Dive Series
TLDR: Synchronizing sensor data so all measurements can be used together is harder than it looks. Different sensors have different clocks, measurements arrive late and out of order, and syncing to GPS time breaks in parking garages. Here’s how we think about the problem—and one approach to solving it.
You’re building a positioning system. GNSS receiver, IMU, wheel encoders. Sensor fusion algorithm. You deploy it and the position jumps around in ways that don’t match your sensor quality.
Often the issue isn’t your sensors or your filter. It’s time synchronization.
This is probably the number one thing that most people do wrong if you’re building your own version of this system. This is the hardest thing to get right. Certainly one of the hardest things to get right. And it’s very easy to get wrong.
Tom and I recently walked through the complete architecture of a production positioning engine—everything between raw sensor data arriving and position estimates going out. Time synchronization is the foundation all of it depends on.
The Problem: Every Sensor Lives in Its Own Time Domain
A typical positioning stack starts with two core sensors: a GNSS receiver and an IMU. More capable systems add wheel odometry, barometric altimeters, computer vision, or other sensors—and the time synchronization problem gets more complicated with each one you add.
You have incoming sensors that have nothing to do with each other. Every sensor has its own internal clock. None of them agree.
Your GNSS receiver timestamps observations in GPS time—seconds since January 6, 1980, derived from atomic clocks on satellites. But that observation travels through a processing pipeline before reaching your fusion engine. That adds latency. Typically it might take an IMU measurement 10, 20, 30 milliseconds to get to you. And at the same time, if you captured a GNSS measurement at the exact same moment, in practice it might take the receiver 50, 60, 70 milliseconds to get that measurement to you. And in some receivers, we’ve seen it take longer than the measurement interval. So if you’ve got a 10 Hz receiver that’s measuring one update every 100 milliseconds, we’ve seen cases where it takes 120 milliseconds to get that measurement.
So it can take a really long time for the data to physically come out of the sensor and then ultimately make its way into our software.
Your IMU runs on its own oscillator. It might be sampling at 100 Hz, or 3,000 Hz. It has no concept of GPS time. Zero. It counts clock ticks from the moment it powered on and that’s all it knows.
Wheel speed data often arrives with no timestamp at all—just time of arrival. The sensor doesn’t tell you when the measurement was captured. All you know is when it showed up.
Picture this: your vehicle turns a corner. One physical event. Three sensors observe it. Data arrives at your fusion engine at three different times, with three different timestamps, from three completely independent clock domains.
If you fuse those measurements as if they happened simultaneously—which is what happens if you’re not explicitly handling this—you’re combining a position from 100 milliseconds ago with an acceleration from right now.
At highway speeds—call it 30 meters per second—50 milliseconds of error means you’re off by a meter and a half before you even start processing. Your filter incorporates it because, as far as the filter knows, the data is correct. The data is correct, but the timing is wrong.
The “right time” isn’t really the question. The point is that the time across these things must be synchronized.
And GNSS measurements are notoriously susceptible to timing issues.
The Parking Garage Problem
Cold start without GPS: your vehicle powers on inside a parking garage. No satellites. No GPS time reference. Nothing. Your IMU is running. Wheel sensors are running. But you have no absolute time base to anchor any of it to.
Some designs just wait for a GPS fix before they start fusing. And that works on a test track with open sky. It does not work when your customer’s delivery robot boots up in the basement of a warehouse and needs to navigate to the loading dock immediately. It does not work when your autonomous shuttle starts its route inside a parking structure.
That is a common problem we’ve seen in a lot of these systems, where everything is synchronized with GPS time, which sounds great until you start indoors.
You need an internal time reference that’s independent of GPS—one that works from the instant of power-on and reconciles with GPS time later, when satellites become available.
One Approach: Invent Your Own Time
It’s not an easy problem to correctly synchronize data so that all the measurements can be used together. There’s not just one answer. There are a couple of fairly good practices to approach the
problem. Depends on your system and the hardware that’s involved.
But ultimately something needs to be able to model the relationship between different sources of time, relate them, and order events correctly.
The way we do it in our system is we basically invent a time that we call P1 time.
P1 time is the common factor across everything. None of the sensors know what P1 time is. You can’t buy a GPS receiver that knows about P1 time. But you can’t buy an IMU that knows about GPS time
either.
Generally speaking, P1 time doesn’t have anything to do with an actual clock or the real world. It’s just a number that goes up. There are some things we do to discipline it and make it
consistent, but ultimately it’s just an arbitrary notion of time. And it allows us to synchronize things very readily.
The Time Provider’s job is to estimate the relationship between various sources of time. As measurements come in from different sensors, the Time Provider converts each sensor’s native timestamp
into P1 time:
- GNSS measurement with GPS time → Time Provider estimates the relationship → P1 timestamp
- IMU measurement with internal counter → Time Provider estimates the relationship → P1 timestamp
- Synchronize time from the operating system so we can timestamp data arriving over the network
- Same process for wheel speeds, camera frames, anything else
You might also have more sensors to estimate, maybe a computer vision system that’s running independently. You might have data coming in over a TCP socket and has to be timestamped by your local processor on arrival. You might not have an internet connection, so your processor might not know what time it is.
There’s a lot of complications, and the Time Provider software abstracts and handles this.
Including in cases where the system starts up and doesn’t have GNSS measurements.
Time Alignment: Putting Measurements Back in Order
Once everything is timestamped in P1 time, you still haven’t dealt with the fact that those measurements came in from the sensors with random and completely independent latency.
But the trick is we know precisely when the measurement was captured. The sensor told us: these GNSS measurements were captured at this GPS time. This IMU measurement was captured at this IMU clock time.
Through the Time Provider, we were able to relate those two through this P1 time notion. So we can say that the GPS time was this P1 timestamp. That IMU time was this P1 timestamp. Now, even though maybe the GNSS measurements came in much later than the IMU data, we can look at them, relate them, and put them back in the correct order.
This is super critical to get working well. The filter predicts the system state forward to the time of each measurement. If time synchronization is wrong, you’re predicting to the wrong moment—measurements will incorrectly imply you’re more wrong than you are.
If reordering is wrong, you might process a newer measurement before an older one. Your prediction is already ahead of that older measurement, and you can’t easily go back in time to use it.
What Timing Errors Look Like in Practice
That’s the theory. Here’s what actually happens when time synchronization breaks:
If you process measurements in arrival order rather than observation order, a delayed GNSS measurement might arrive after newer IMU data. Your filter predicts the system state forward to match the IMU timestamp, then processes the old GNSS measurement. The prediction is already ahead of where that measurement was captured—you can’t easily go back in time. The measurement either gets discarded or creates errors in your state estimate.
If you get the time synchronization wrong, you predict to the wrong moment. The measurement will incorrectly imply you’re more wrong than you actually are, corrupting your position solution.
The fix is in the time alignment layer—reordering measurements into correct temporal sequence before they reach the estimator.
Plan for Implementation Complexity
We made this sound straightforward. But there can be complexity to handle depending on your system and hardware.
You might have a computer vision system running independently. Data coming in over a TCP socket that needs to be timestamped by your local processor on arrival. You might not have an internet connection, so your processor might not know what time it is. The Time Provider needs to handle all of that.
When GPS time arrives and disagrees with your internal estimate, the system needs to reconcile them without creating jumps in position output. Even small discontinuities can cause problems for downstream autonomy software.
Clocks also drift over time. In some cases, they jump either forward or backward at random moments—they’re not guaranteed to be smooth. Many GNSS receivers do a “clock reset” to occasionally adjust their receiver time to GPS time once it has drifted far enough. How big the jump is, which direction time jumps, and how often it happens depends on the receiver. Operating systems like Linux do the same thing with some of their clocks. Modern operating systems have multiple different clocks—yet another layer to model, since different subsystems sync to different clocks.
These details matter enormously and vary significantly depending on platform and hardware constraints.
Questions Worth Asking
If you’re building your own positioning stack or evaluating a commercial system:
- What’s the actual latency distribution of your sensor inputs under real operating conditions? Not the datasheet number. The measured distribution.
- Are you processing measurements in observation order or arrival order?
- What happens when you cold start without GPS?
- Can you log and replay raw sensor data with correct timestamps for debugging?
- What’s your strategy when GPS time and your internal clock disagree?
- How does your system handle clock jumps (GNSS receiver resets, NTP adjustments)?
- Do you account for clock drift between different sensor time bases?
- If you’re using multiple processors or computers, how do you synchronize their clocks?
- Can you simulate variable sensor latency in your test environment?
- What’s your maximum tolerated latency before a measurement is considered too old?
- Do you have visibility into when time synchronization is degraded or failing?
- How do you handle sensors that report non-monotonic timestamps (time going backwards)?
These questions don’t have universal answers. The right approach depends on your specific system, sensors, and constraints. But they’re worth asking.
What's Next
Time synchronization gets all your measurements properly aligned in time. But that doesn’t mean they’re good measurements.
In the next post, we’ll cover pre-processing—the work that happens before data reaches your fusion algorithm. Detecting multipath (GPS signals bouncing off buildings). Filtering IMU vibration without losing important dynamics. Inferring environmental context from signal quality. Catching carrier phase cycle slips.
Because no matter how good your timing is, you still need to make sure your inputs aren’t corrupted.
This is Part 1 of our series on positioning system architecture. Watch the full video walkthrough where Tom and I go through the complete system architecture diagram.