Anuj NagarData analysis & science writing

Experiment 01 · flagship

MF4 Log Analytics

Dual-path controllers are hard to inspect by hand: two processors, separate time bases, hundreds of power-ups per test campaign. This pipeline reads the recordings and answers, for every start-up, which path came first, by how many milliseconds, and why.

Period
Built and published Oct 2026
Stack
Python, pandas, NumPy, asammdf, HTML/JS dashboard
Data
Synthetic demo recordings (no proprietary logs)
Licence
MIT, open source
Question

Which processor starts first, and can I trust the timestamps?

In a dual-path controller, a master and a slave processor each keep their own clock. To check a start-up rule you scroll through timestamps in a measurement tool, correct for the clock offset in your head, and repeat that for every power-up in every file.

A long-duration test campaign makes that expensive. My worked example: 10 recordings of about 4 GB each, with 57 to 60 start-ups per recording.

  • ≈585start-up cycles to check (570 to 600)
  • ≈10 minper cycle by hand (my estimate)
  • 95–100 hmanual analysis, about 12 working days
Method

Find every power-up, correct the clocks, then check the answer a second way

  1. Load. Open large MF4 files lazily with asammdf and find the two path channel groups from their acquisition source.
  2. Segment. Split the recording into power-ups by detecting init-flag falling edges, merged within one second.
  3. Measure. Count resets per path, detect reboot data gaps, find each path's first application update.
  4. Correct. Estimate the clock offset between the paths from matching inter-processor alive-counter pairs, with a linear fit to absorb drift.
  5. Cross-check. Compute a second start difference from each path's own clock, which needs no offset estimate at all.
  6. Judge. Combine both into a verdict (PA_FIRST, SAME_FRAME, PB_BEFORE_PA, INCONCLUSIVE) and a result (OK, NOK, CHECK, INFO).
  7. Report. One CSV row per power-up (60 columns), plus a local browser dashboard with charts, a timeline view and filters.

Why two methods? The corrected difference depends on the offset estimate; the offset-free one does not. A violation is only reported when both agree, so false alarms from timestamp artefacts get filtered out.

Result

About 95 hours of checking becomes about 2.5 hours of unattended analysis

  • 95 h → 2.5 hestimated, per 10-recording campaign
  • ≈39×faster (97% less time)
  • ≈15 sper cycle, against about 10 min by hand

The bigger win is what the method catches. On the synthetic demo recording that ships with the repo, start-up #6 looks fine in the raw timestamps (+2.0 ms) but is a real violation once the 3.2 ms clock offset between the paths is removed.

Corrected start difference (PB minus PA) for the 7 power-ups of the synthetic demo recording. Negative means path B started first. #6 is the violation the raw timestamps hid.

Everything runs from the command line or a dependency-free local dashboard (a small Python http.server backend and one HTML/JS page, bound to localhost). Results are cached by file, settings, signal map and script hash, so reopening a file is instant.

Caveat

What the numbers do and don’t say

  • The tool time was measured once, on a 3.4 GB recording (peak memory about 5.7 GB) while another analysis was running. Treat it as an upper bound.
  • The manual time per cycle (about 10 minutes) is my estimate, not a benchmark. Recording time and the time spent deciding what to fix are not counted, because they are needed either way.
  • The public repo uses a synthetic recording. Channel names are not hard-coded; a signal map file points the tool at real ones, and the real signal map stays out of git.

The interactive workflow page lets you change those assumptions and see the estimate move.

Contact

Got a messy dataset or a science question?

I read everything and answer when I can. The quickest way is email, and LinkedIn works too.