[SciPy-User] synchronizing timestamps from different systems; unpaired linear regression

[Charles R Harris]

On Tue, Apr 10, 2012 at 10:16 PM, Chris Rodgers <xrodgers at gmail.com> wrote:

> Dear all
>
> Thanks very much for the suggestions!
>
> Re a new hardware implementation: I bet this would totally work and
> honestly is probably the fastest way to get it working. I think even a
> rough system clock would do the trick. The downsides are 1) many data
> have already been collected with the old setup; 2) I'm getting
> stubbornly interested in this problem for its own sake since it feel
> so solvable. So perhaps I'll change the hardware for future data and
> keep working on algorithms for the old data. (I'd never heard of
> Lamport timestamps. The wikipedia article is really interesting. If I
> understand it correctly, it would still require a hardware change
> though.)
>
>
> Re Nathaniel's suggestion:
> I think this is pretty similar to the algorithm I'm currently using.
> Pseudocode:
>
> current_guess = estimate_from_correlation(x, y)
> for timescale in decreasing_order:
>  xm, ym = find_matches(
>    x, y, current_guess, within=timescale)
>  current_guess = linfit(xm, ym)
>
> The problem is the local minima caused by mismatch errors. If the
> clockspeed estimate is off, then late events are incorrectly matched
> with a delay of one event. Then the updated guess moves closer to this
> incorrect solution. So by killing off the points that disagree, we
> reinforce the current orthodoxy!
>
> Actually the truest objective function would be the number of matches
> within some specified error.
> ERR = .1
> def objective(offset, clockspeed):
>   # adjust parametrization to suit
>   adj_y_times = y_times * clockspeed + offset
>   closest_x_times = np.searchsorted(x_midpoints, adj_y_times)
>    pred_err = abs(adj_y_times - x_midpoints[closest_x_times])
>   closest_good = closest_x_times[pred_err < ERR]
>   return len(unique(closest_good))
>
>
> That function has some ugly non-smoothness due to the
> len(unique(...)). Would something like optimize.brent work for this or
> am I on the wrong track?
>
>
>
You can also get some useful information from NTP (or chrony). For instance
on Fedora 16, which uses chrony instead of the ntp utilities to synchronize
the time,

chronyc> tracking
Reference ID    : 66.219.59.208 (mx1.mailfighter.net)
Stratum         : 3
Ref time (UTC)  : Wed Apr 11 15:17:33 2012
System time     : 0.000000349 seconds fast of NTP time
Frequency       : 41.465 ppm fast
Residual freq   : 0.001 ppm
Skew            : 0.104 ppm
Root delay      : 0.063986 seconds
Root dispersion : 0.023441 seconds

The Frequency might be useful to you.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20120411/e2f7437f/attachment.html>