NTP users are strongly urged to take immediate action to ensure that their NTP daemons are not susceptible to being used in distributed denial-of-service (DDoS) attacks. Please also take this opportunity to defeat denial-of-service attacks by implementing Ingress and Egress filtering through BCP38.

ntp-4.2.8p18 was released on 25 May 2024 and addresses 40 bugs and provides 40 improvements.

Please see the NTP 4.2.8p18 Changelog for details.

Bug 3596 - ntpd uses highly predictable transmit timestamps
Summary: ntpd uses highly predictable transmit timestamps
Status: RESOLVED FIXED
Alias: None
Product: ntp
Classification: Unclassified
Component: ntpd (show other bugs)
Version: 4.2.8
Hardware: PC All
: P3 major
Assignee: Harlan Stenn
URL:
Depends on:
Blocks:
 
Reported: 2019-06-20 02:00 UTC by Harlan Stenn
Modified: 2022-12-24 11:28 UTC (History)
5 users (show)

See Also:
stenn: blocking4.2.8+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Harlan Stenn 2019-06-20 02:00:13 UTC

    
Comment 1 Harlan Stenn 2019-06-20 02:07:59 UTC
MIroslav emailed the following to security@ :

The discussion about random source port on the NTP WG mailing list
prompted me to try the off-attack on ntpd where the attacker is
predicting transmit timestamps in order to take control of the clock.

It works. I was able to force ntpd using three NTP sources to update
its clock with an arbitrary offset, predicting its transmissions
only from information that it provides as an NTP server.

The main issue here is that the sub-second part of the transmission
timestamps is very stable due to the fixed 1-second timer that ntpd
uses for all its requests, which makes the origin timestamp in server
responses highly predictable. The stability depends on many things
(e.g. CPU load, network load), but in my tests on Linux there were
only about 16 random bits.

In case of a stratum 2 server the attacker can determine the stable
bits of the sub-second part from the reference timestamp, root delay
and response time of the upstream server. In other cases the attacker
can try slowly adjusting the value until the server switches to
another source, indicating the offset was injected, which caused the
original source to be rejected in the source selection. When this
value is known, it is possible to spoof responses from all other
sources that the client is using (which can be determined from the
refid).

In my tests I was sending packets at a rate of about 750000 per
second, iterating over the 16-bit space several times per second. The
client could process less than half of it, which means most of the
genuine responses from the upstream servers were also dropped, so the
attack window wasn't limited to the round trip time.

Only one injected sample is needed to cause a source to be rejected
in the source selection. The clock filter has 8 samples, so 8 samples
need to be injected in a row in order to update the clock. The goal of
the attacker is to have the client one source (assuming default
minclock) with 8 injected samples and all other sources at least one
injected sample each.

In my tests with three sources polled at an 8-second interval (and the
stepout timeout adjusted accordingly) the attack took an hour. With
a 64-second polling interval it took whole night. I didn't try a
longer interval, but I expect it to scale linearly. There are some
optimizations I have not tried to implement, which may reduce the time
significantly.

Here is an ntpd log which shows a clock update reached the panic
threshold:
10.11.160.238 8014 84 reachable
10.5.26.10 8014 84 reachable
10.5.27.10 8014 84 reachable
10.11.160.238 901a 8a sys_peer
0.0.0.0 c615 05 clock_sync
10.5.26.10 901a 8a sys_peer
0.0.0.0 0618 08 no_sys_peer
10.11.160.238 902a 8a sys_peer
0.0.0.0 0628 08 no_sys_peer
10.5.27.10 901a 8a sys_peer
0.0.0.0 0638 08 no_sys_peer
10.5.26.10 902a 8a sys_peer
10.11.160.238 903a 8a sys_peer
10.5.26.10 903a 8a sys_peer
10.11.160.238 944a 8a sys_peer
10.5.27.10 942a 8a sys_peer
0.0.0.0 0648 08 no_sys_peer
10.5.27.10 903a 8a sys_peer
0.0.0.0 0617 07 panic_stop +10000 s; set clock manually within 1000 s.
0.0.0.0 061d 0d kern kernel time sync disabled


I think this is a serious issue that that needs to be fixed as soon as
possible. Random source port might make the attack impractical.
Ideally ntpd would be using fully random transmit timestamps as
recommended in the data minimization draft.

Please let me know if there are any questions. I'll see if I can write
a short article with more details about this vulnerability.
Comment 2 Harlan Stenn 2020-03-03 06:25:05 UTC
STAGED for ntp-4.2.8p14
Comment 3 Harlan Stenn 2020-03-04 05:00:22 UTC
Miroslav, thanks for your report.

Please mark this bug as VERIFIED or REOPENED, as appropriate.

Pearly, thanks for your work on this.
Comment 4 Miroslav Lichvar 2020-10-22 13:55:09 UTC
I did a quick test with ntp-4.2.8p15. I don't see any change in the predictability of the transmit timestamp. In the tcpdump log I see:

          Transmit Timestamp:   3812362068.379084938 (2020/10/22 15:27:48)
          Transmit Timestamp:   3812362070.379091346 (2020/10/22 15:27:50)
          Transmit Timestamp:   3812362072.379084193 (2020/10/22 15:27:52)
          Transmit Timestamp:   3812362074.379087482 (2020/10/22 15:27:54)
          Transmit Timestamp:   3812362076.379122541 (2020/10/22 15:27:56)
          Transmit Timestamp:   3812362078.379117626 (2020/10/22 15:27:58)
          Transmit Timestamp:   3812362134.379101956 (2020/10/22 15:28:54)
          Transmit Timestamp:   3812362202.379112421 (2020/10/22 15:30:02)
          Transmit Timestamp:   3812362268.379094490 (2020/10/22 15:31:08)
          Transmit Timestamp:   3812362336.379094767 (2020/10/22 15:32:16)
          Transmit Timestamp:   3812362405.379097638 (2020/10/22 15:33:25)
          Transmit Timestamp:   3812362473.379107612 (2020/10/22 15:34:33)
          Transmit Timestamp:   3812362540.379067952 (2020/10/22 15:35:40)
          Transmit Timestamp:   3812362609.379100428 (2020/10/22 15:36:49)
          Transmit Timestamp:   3812362677.379106131 (2020/10/22 15:37:57)
          Transmit Timestamp:   3812362743.379096065 (2020/10/22 15:39:03)
          Transmit Timestamp:   3812362811.379123724 (2020/10/22 15:40:11)
          Transmit Timestamp:   3812362880.379128079 (2020/10/22 15:41:20)
          Transmit Timestamp:   3812362948.379094403 (2020/10/22 15:42:28)
          Transmit Timestamp:   3812363014.379084829 (2020/10/22 15:43:34)

The sub-second part of all these transmit timestamps was between .3790841 and .3791281. That's only ~44 microseconds, or ~18 bits that the attacker needs to guess.

I'd reopen the bug, but my bugzilla account doesn't seem to the permissions for that.
Comment 5 kircher 2020-11-11 06:43:43 UTC
(In reply to comment #4)
> I did a quick test with ntp-4.2.8p15. I don't see any change in the
> predictability of the transmit timestamp. In the tcpdump log I see:
> 
>           Transmit Timestamp:   3812362068.379084938 (2020/10/22 15:27:48)
>           Transmit Timestamp:   3812362070.379091346 (2020/10/22 15:27:50)
>           Transmit Timestamp:   3812362072.379084193 (2020/10/22 15:27:52)
>           Transmit Timestamp:   3812362074.379087482 (2020/10/22 15:27:54)
>           Transmit Timestamp:   3812362076.379122541 (2020/10/22 15:27:56)
>           Transmit Timestamp:   3812362078.379117626 (2020/10/22 15:27:58)
>           Transmit Timestamp:   3812362134.379101956 (2020/10/22 15:28:54)
>           Transmit Timestamp:   3812362202.379112421 (2020/10/22 15:30:02)
>           Transmit Timestamp:   3812362268.379094490 (2020/10/22 15:31:08)
>           Transmit Timestamp:   3812362336.379094767 (2020/10/22 15:32:16)
>           Transmit Timestamp:   3812362405.379097638 (2020/10/22 15:33:25)
>           Transmit Timestamp:   3812362473.379107612 (2020/10/22 15:34:33)
>           Transmit Timestamp:   3812362540.379067952 (2020/10/22 15:35:40)
>           Transmit Timestamp:   3812362609.379100428 (2020/10/22 15:36:49)
>           Transmit Timestamp:   3812362677.379106131 (2020/10/22 15:37:57)
>           Transmit Timestamp:   3812362743.379096065 (2020/10/22 15:39:03)
>           Transmit Timestamp:   3812362811.379123724 (2020/10/22 15:40:11)
>           Transmit Timestamp:   3812362880.379128079 (2020/10/22 15:41:20)
>           Transmit Timestamp:   3812362948.379094403 (2020/10/22 15:42:28)
>           Transmit Timestamp:   3812363014.379084829 (2020/10/22 15:43:34)
> 
> The sub-second part of all these transmit timestamps was between .3790841 and
> .3791281. That's only ~44 microseconds, or ~18 bits that the attacker needs to
> guess.
> 
> I'd reopen the bug, but my bugzilla account doesn't seem to the permissions for
> that.


We would like to know how many randomization bits and the risk of attack are there for Transmit Timestamp, perhaps you could provide a program that simulates attacks? It can be Python3, shell, or C language code.

For example, we can randomize only the 32 bits of the decimal part of the Transmit Timestamp when sending a packet. I don't know if this can solve the problem of ntp predictability.

Sending a completely randomized 64-bit Transmit Timestamp will make this field invalid.
Comment 7 Raj 2022-09-12 18:19:09 UTC
@Harlan Stenn, can you share your code/proof-of-concept if possible at rajattan@usc.edu?

Also, which tool did you use to generate 750000 packets per second?