Welcome to the 21st century, it’s about time…
What’s happening
Since the beginning of January I have made significant progress with hardware timestamping work for ptpd. The code is currently in my 2.3.2-hwtest branch and hopefully will be pulled into the main ptpd git tree in the near future.
The code right now is stable and people are more than welcome to test it. One important note: all ptpd2 references have been changed to ptpd. You can automatically build RPM packages with the packagebuild/rpm-rh/rpmbuild.sh script, as before.
The man page is not fully updated yet with new and changed settings, but the built-in help is, so you may use ptpd -H, ptpd -O, and ptpd -e [setting] for help. The best thing to do for now is to grab the latest stable version, take the ptpd -O output from the last official version or git source and diff it against the new version. The names of the settings are descriptive and mostly self-explanatory. The best way to monitor is to watch the status file /var/run/ptpd.status (as in: watch -n 1 `cat /var/run/ptpd.status`). The status file shows the list of all clocks and each clock’s current status.
So far, this has been tested on Intel (igb, ixgb), Mellanox ConnectX3 (mlx4_en) and Solarflare (6xxx, 7xxx - sfc) - GE and 10GE with very decent results. No testing with embedded platforms yet, like i.mx6 or am335x, but in principle they should work. A sample config file has been added: test/linuxphc_hw.conf.
What’s new
First of all, full Linux PHC support with h/w timestamping and hardware clock control.
Clock driver API - the PTP engine now only computes an offset and serves as an external reference to its clock driver. No direct calls to any system time functions from PTP code. The clock API is quite rich, the clock model has a state machine that any time nut will recognise: init -> freerun -> tracking -> locked -> holdover.
(Linux) support for timestamping over VLAN interfaces and bonded interfaces (active standby only), with automatic tracking of the active members changing, being added, removed, etc. This is all done using native Linux APIs rather than parsing /proc or /sys files.
Simultaneous intelligent sync of an arbitrary number of PHC clocks - all members of a bond, plus any extra clocks you tell it to - and the system clock of course. When running as PTP master, you can select a designated clock to be the master clock source. No multiple tools or processes are required.
Clock stability tracking is handled by the clock driver and is based on its frequency offset’s Allan deviation. A best clock is selected to which all other clocks sync - unless they are read-only. The OS clock (and any clock) is protected from rapid changes of other clocks - if they are controlled too aggressively they go out of LOCKED state and other clocks detach from them, previously LOCKED clocks go into HOLDOVER and one of them takes over until the PTP-controlled clock is back in good sync. With some engineering it will become possible to take PTP from multiple sources and pick the best behaving one.
Protection from negative clock step - can be selectively enabled for h/w clocks and OS clock. Once a clock is in NEGSTEP state, manual intervention is required. Respective alarms are raised and log messages issued.
Optimal clock frequency is stored in files, one per clock device, stored and restored when needed (also notably when going into FREERUN state, so we are not left with a wildly drifting clock).
Major improvements to filtering, adapted to sync intervals as low as 1/sec with all filters enabled (sync outlier filter, sync stat filter, delay outlier filter, delay stat filter).
Caveats
Caveat #1: This code is currently Linux only - I’ll leave restoring portability for later.
Caveat #2: You need a kernel / OS with Linux PHC support, and some PTP-enabled NIC. Check ethtool -T output of your interface if in doubt.
Caveat #3: Leap second support and setting various kernel status flags are not working yet. They have to be rewritten into the clock API, a work in progress.
Caveat #4: Mixing HW PTP NICs with non-HW NICs in a bond is possible, but not well tested.
Caveat #5: The old NTP mode 7 integration is not yet compatible with the new reference clock model and, for now, should not be used. It is a legacy method anyway, and a successor is in development.
Caveat #6: For now, do not try transports other than IPv4. Transports will be rewritten into an API, which is the very next step after the clock driver.
Caveat #7: With multiple clocks running, ptpd can get quite chatty, but all clock-related log messages are prefixed with clock:.
Caveat #8: Currently clock stability estimation is based on Allan deviation only. If you choose very low servo kP and kI constants, a clock that has a high offset but changes very slowly will seem stable. A fix is underway which uses clock’s offset statistics as one of best clock selection criteria, so the best clock will really win.
Happy testing!
Regards,
Wojciech
Wojciech Owczarek