Distribution of clock synchronization using IEEE 1588 (Precision Timing Protocol, PTP)
Precise timing is increasingly used in a wide range of applications and commercial infrastructures such as mobile communications, networking, the smart grid, the network of Internet of Things (IoT), industrial automation and financial technologies (Fintech). There is an ever-expanding set of critical infrastructure that is required to provide precise time and frequency for such applications. This requirement is growing and becoming increasingly pervasive in worldwide communication networks and commercial infrastructure.
In addition, measurement and automation systems require event synchronization and data correlation especially when multiple devices are involved in the measurement. Such devices have direct access to timing signals from a common source, or the devices must have a way to interact and synchronize their individual clocks in order to share a common time base.
Devices located in close proximity to each other, sharing a common timing signal may be the easiest way of achieving time synchronization. For example, blades or cards in a common chassis could share a common clock signal from a backplane, enabling event synchronization to a high degree of accuracy. To accurately use such a common timing signal, some calibrations would be required to compensate for issues such as propagation delays, but since these are statically determinable, engineering such a method is relatively easy.
In some respects, GNSS based timing is a similar method used to derive clock from a common source (GNSS signal). Since GNSS is available world wide and access to such signals is almost global outdoors, it is the most commonly deployed method of time synchronization for most commercial purposes. So much so that GNSS based / derived signals are considered ‘fully trace-able’ and accepted as the standard way to “anchor” most time synchronization systems. Acceptance of GNSS as the backbone to provide synchronization is global. In addition, GNSS signals are used in car navigation systems and E911 (emergency) services in the US, and “112” services in Europe (especially EU). Precise time synchronization for the IoT infrastructure, financial transactions and the power grid are also ultimately based on the ubiquitous GNSS signals.
However, having a common timing signal becomes unfeasible when the distance between devices increase, or when devices are located indoors (where GNSS signals do not penetrate) or when such devices are nomadic, frequently changing their location.
In such situations, clock synchronization need to be distributed using either a dedicated network or an existing network. Examples of distributed clock synchronization a PC’s internal clock synchronized to an NTP time server, or a group of devices similarly synchronized using the IEEE 1588 (PTP) protocol from a PTP Grand Master. In both these protocols, these devices periodically exchange information and adjust their local timing sources to match each other.
Both these methods of time synchronization require a continuous process of clock adjustment. If two clocks were set fully aligned and their frequency sources ran at the exact same rate, they would remain synchronized indefinitely. In practice, however, clocks are set with limited precision, and their frequency sources run at slightly different rates, and rate of a frequency source varies over time, temperature and some other physical factors. Most modern electronic clocks use some form of a crystal oscillator as a frequency source. The inherent instability in all oscillators require that the clocks must continually be synchronized to match each other in frequency and phase.
IEEE 1588 Synchronization (PTP)
IEEE 1588 outlines a standard protocol for synchronizing clocks connected via a network, such as Ethernet. Released as a standard in 2002, IEEE 1588 was designed to provide fault tolerant synchronization among heterogeneous networked clocks requiring little network bandwidth overhead, processing power, and administrative setup. IEEE 1588 provides this by defining a protocol known as the precision time protocol, or PTP. The standard has undergone major improvements and has been used to cater to a multitude of applications by adding special features for such applications to evolve the basic IEEE 1588 into multiple ‘profiles’.
The PTP protocol provides a fault tolerant method of synchronizing all participating clocks of varying capabilities to use the highest quality clock in the network. IEEE 1588 defines a standard set of clock characteristics. By running a distributed algorithm, called the Best Master Clock algorithm (BMC), each clock in the network identifies the highest quality clock; and the self-adjusting network can run in the most optimal mode, using the best source of clock. The “highest ranking” clock, called the ‘grandmaster’ clock, and synchronizes all other ‘slave’ clocks. When or if the grandmaster clock is removed from the network, or if its characteristics change in a way such that it is no longer the ‘best’ clock, the BMC algorithm provides a way for the participating other slave clocks to automatically determine the current ‘best’ clock, which then becomes the new Grandmaster. This way, IEEE 1588 creates a simple self-adjusting network of clocks that always stays optimally synchronized.
Slave clocks synchronize to the 1588 grandmaster by using bidirectional communication (see Figure 1). The grandmaster clock periodically issues a packet called a ‘sync’ packet with the timestamp when the packet left the grandmaster clock. The grandmaster may also, issue a ‘follow up’ packet with the timestamp for the ‘sync’ packet. Use of this ‘follow up’ packet enables accurate time-stamping of the ‘sync’ packet on networks when the departure time of a packet cannot be known accurately beforehand. For example, the collision detection and random back off mechanism of Ethernet communication prevents the exact transmission time of a packet from being known until the packet has already been sent without a collision being detected, after which time it is impossible to alter the packet’s content.
Figure 1. The PTP Protocol
A slave clock receives the grandmaster’s ‘sync’ packet and timestamps the packet’s arrival time using its own clock. The difference in the ‘sync’ packet’s departure timestamp and the ‘sync’ packet’s arrival timestamp is the combination of the slave clock’s offset from the master and the network propagation delay. Using only the ‘sync’ message it’s possible to synchronize the frequency of the slaves clock to the master’s clock.
IEEE 1588 assumes that the network propagation delay is symmetrical or unknowable. That is, the delay of a packet sent from the master to the slave is assumed to be the same as the delay of a packet sent from the slave to the master. In this way, the slave can discover, and compensate for the propagation delay. It accomplishes this by issuing a ‘delay request’ packet which is time stamped on departure from the slave. The ‘delay request’ message is received and time stamped by the master clock, and the arrival timestamp is sent back to the slave clock in a ‘delay response’ packet. The difference in these two timestamps is the network propagation delay (refer to Figure 1).
By sending and receiving these synchronization packets, the slave clocks can accurately measure the offset between their local clock and the master’s clock. The slaves can then adjust their clocks by this offset to match the time of the master. This is where individual “servo algorithms” can show their differentiating value, since the IEEE 1588 specification does not include any standard implementation for adjusting a clock; it limits itself to providing a standard protocol for exchanging these messages, allowing devices from different manufacturers, and with different implementations to interoperate.
Many factors affect the accuracy of the synchronization levels achievable using IEEE 1588. During the time between synchronization packets, the individual clocks in a system will drift apart from each other due to frequency changes in their local timing source. Obviously, this drift can be reduced by using higher stability timing sources and by shortening the intervals between synchronization packets. In increasing order of stability – Temperature-controlled crystal oscillators (TCXOs), oven-controlled crystal oscillators (OCXOs) or atomic clocks provide higher stability than commercial standard crystal oscillators. In addition to this, a clock’s resolution and a good implementation of the timestamps transmitted in the PTP synchronization messages are important factors. Devices that have a higher resolution clock are able to more accurately timestamp messages. Significant contributors to problems are variations in network delay caused by packet delay variation introduced by variable traffic and intermediate networking devices such as hubs and switches all combine to reduce the achievable synchronization level.
Ethernet networking enables high bandwidth networks over long distances with relatively cheap cabling and infrastructure costs. Typical Ethernet based IEEE 1588 implementations provide synchronization at the tens of microsecond level. High quality servo algorithms can deliver sub-microsecond synchronization. Actual performance is highly implementation and topology specific due to these reasons.
The decision regarding the optimal method to use involves trade-offs between cost, accuracy, complexity of configuration, “hop-count” and distance requirements. Standard NTP synchronization over Ethernet offers millisecond level synchronization appropriate for lower-speed events that are not mission or time-critical. IEEE 1588 provides an important alternative for systems requiring precision sub-microsecond synchronization in geographically distributed systems, however, to achieve these levels a world-class synchronization implementation is required.
The level of precision achievable using IEEE 1588 is heavily dependent on the packet delay variation (the variation in latency) present in the underlying network. Of course, simple point-to-point connections provide the highest precision, however most practical applications require more than just point-to point connections. Hubs impose relatively little network packet delay variation. A realistic operational network, with traffic load will add delay variation to IEEE 1588 packets, which can only be overcome by use of sophisticated algorithms. Prioritization of packets help but do not fully solve the problem.
Figure 2. PTP slave with precision sync algorithm 'filters' out network effects
While using sophisticated algorithms is the most cost effective way to implement a IEEE 1588 timing network, another way to reduce the effect of packet delay variation is the use of IEEE 1588 boundary clocks or transparent switches (see Fig 2). A switch acting as a boundary clock (which is an integrated slave + grandmaster) runs the PTP protocol, and is synchronized to an attached master clock. This boundary clock in turn acts as a master clock to all attached slaves. With this approach, all internal latencies and packet delay variation in the switch can be compensated and do not affect synchronization accuracy.
Figure 3. PTP Network
Boundary clocks do not allow passage of Sync, Follow_Up, Delay_Req, or Delay_Resp messages from the grandmaster to the slaves directly. Within a subnet, a port of a boundary clock acts just like an ordinary clock with respect to synchronization and best master clock algorithm. The boundary clock may internally select the port that sees the ‘best clock’ as the single slave port. This port is a slave in the selected subnet. All other ports of the boundary clock internally synchronize to this slave port. If there are cyclic paths in the network topology the best master clock algorithm reduces the logical topology to an acyclic graph.
Transparent clocks are essentially switches that try to solve the same problem in a slightly different manner. A transparent switch is so called because it does not operate as a PTP node in a IEEE 1588 system, but instead it modifies the timing contents of PTP packets to account for the delay caused by the switch. Typically, a transparent switch calculates how much time a 'sync' packet spends inside the switch, and then modifies the timestamp of the associated 'follow up' packet to account for this delay. The use of transparent clocks are of limited benefit unless the entire network is ‘fully IEEE 1588 aware’ end to end. As such their use in mission critical networks such as Telecom have been limited.
Both Boundary clocks and Transparent clocks are limited in their effectiveness for networks that are not full ‘time aware’. A fully time aware network would be one where all nodes (switches, routers etc.) are fully enabled with either a IEEE 1588 Boundary Clock or Transparent Clock. A partially time aware network benefits most from high precision algorithms at the slave (or at least in intermediate boundary clocks) that enables derivation of a precise clock in spite of network packet delay variation.
Markets & Applications
These precise synchronization capabilities of IEEE 1588 is leading to interest and use for multiple applications world wide:
- Mobile & telecom infrastructure
- Smart power & power infrastructure
- Cyberphysical systems / Industrial Internet of Things
- Test & Measurement
- Factory Automation
- Robotic Control
- Financial networks