From owner-freebsd-stable@freebsd.org Fri Feb 28 06:58:26 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4479A25BDAD for ; Fri, 28 Feb 2020 06:58:26 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48TL3N5dhkz4LVg for ; Fri, 28 Feb 2020 06:58:24 +0000 (UTC) (envelope-from dewaynegeraghty@gmail.com) Received: by mail-wr1-x442.google.com with SMTP id j7so1631798wrp.13 for ; Thu, 27 Feb 2020 22:58:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=m0l8n12dMjwa1iFfbSt4bp0/Ruq8eQNdxUt8MN2YZGg=; b=fkri1GvAW7pF1moj7TnZSINCPZiq23zKt0SDz6E4agiTIB6ANYB5Q35on/FW5Qvyqj 8VFQmmSk9qtSMv0cyyRSm8ON8Q0chmW9rnoxF8S+Upfy1UPyMi6JTdnswfh0nDF998kz x3XNO0hrfAemseWszNIgXrNivVYMoWeEGPYTjUwXcnjM5mFuiD7OLuk4y6Eq0Uk+tVTZ XaLWY/jUx/GptKsZt5vYdkkLH/8ehkhXoSp3Wv+D2TGbTQk4R1ZQmd4TQcAdlcpTpiSQ tJ8CUcKcNT5G6N1gFaBeVLKv/VU8HJouy1afq4e33u+OaE9Qs6ATFeL5kIWZwQRWyQZZ DQkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=m0l8n12dMjwa1iFfbSt4bp0/Ruq8eQNdxUt8MN2YZGg=; b=kpttdx9YSf5OyKs7I0YT/U+joeiKs3dKZ1BljUDPVzkBVJRsjVZjls3jqSpWWgiLaj 4WTlbEjqccOeBZpYr2XOwCjDv/gQvCa+ehdf4IgY129lT8HHIfOKjM1Y60lbjWF4u6Wn lVLh2dpYm18yRxjh2/CTWtSAPNqtxd5VFswyq1PtW2n/puCAo0fsz5UTLtvflUC/t4tY jP4ItM2P387DYeHtRgowLJ4LBTngfUt9fjrWwUd7aUT9rOboUlkfLEDGgApM9vkPYEQh Y82nmVzLFt9HoA1xQN7KaD9YErJ0zgHrY9eQSQor3Cj0/3fccPw5amON9ap0Zohe3W09 dFXg== X-Gm-Message-State: APjAAAWMViSZhsSSKqhEqq9pGprbFyWTBm41NSVZVV+adfHjvdktEr9r osO3COKl2MrfiskUbCU3LePnR4mv7Fa9trLY/TQhyoAUfgs= X-Google-Smtp-Source: APXvYqzkb3MdCoyZ4Z80On24ZzLnX27T7sM1Nt13UrlTidsrfStIQuS/JOHxZr2+QmgS9Pz9xdQPXOMtWddTNNtTv/g= X-Received: by 2002:adf:f607:: with SMTP id t7mr3229341wrp.36.1582873102766; Thu, 27 Feb 2020 22:58:22 -0800 (PST) MIME-Version: 1.0 References: <20200226194329.GA85186@server.rulingia.com> In-Reply-To: <20200226194329.GA85186@server.rulingia.com> From: Dewayne Geraghty Date: Fri, 28 Feb 2020 17:56:14 +1100 Message-ID: Subject: Re: ntp problems stratum 2 to 14? To: Peter Jeremy Cc: freebsd-stable stable X-Rspamd-Queue-Id: 48TL3N5dhkz4LVg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=fkri1GvA; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dewaynegeraghty@gmail.com designates 2a00:1450:4864:20::442 as permitted sender) smtp.mailfrom=dewaynegeraghty@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2.4.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (2.57), ipnet: 2a00:1450::/32(-2.41), asn: 15169(-1.67), country: US(-0.05)]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2020 06:58:26 -0000 On Thu, 27 Feb 2020 at 06:43, Peter Jeremy wrote: > On 2020-Feb-26 16:37:43 +1100, Dewayne Geraghty > wrote: > >I usually run ntpd with both aslr and as user ntpd. While testing I > >noticed that my server with a direct network cable to my main time keeper, > >jumped from the expected stratum 2 to 14 as follows (I record the date so > I > >can synch with the debug log, also below): > > > >vm.loadavg={ 0.09 0.10 0.18 } > > > >Wed 26 Feb 2020 15:16:38 AEDT > > remote refid st t when poll reach delay offset > > jitter > > >============================================================================== > > 10.0.7.6 203.35.83.242 2 u 44 64 377 0.147 -227.12 > 33.560 > >*127.127.1.1 .LOCL. 14 l 59 128 377 0.000 0.000 > 0.000 > > >26 Feb 15:03:40 ntpd[8772]: LOCAL(1) 901a 8a sys_peer <== bad > > Why is this bad? You've specified that this is a valid clock source so > ntpd is free to use it if it decides it is the best source of time. > > >server 127.127.1.1 minpoll 7 maxpoll 7 > >fudge 127.127.1.1 stratum 14 > > Synchronizing to the local clock (ie using 127.127.1.x as a reference) is > almost never correct. What external (to NTP) source is being used to > synchronize the local clock? > > >I'm also very surprised that the jitter on the server (under testing) is > so > >poor. The internet facing time server is > >*x.y.z.t .ATOM. 1 u 73 512 7 23.776 34.905 95.961 > >but its very old and not running aslr. > > The 23ms distance to the peer suggests that this is over the Internet. > What > sort of link do you have to the Internet and how heavily loaded is it? The > NTP protocol includes the assumption that the client-server path delay is > symmetric - this is often untrue for SOHO connections. And SOHO > connections > will often wind up saturated in one direction - which skews the apparent > timestamps and shows up as high jitter values. > > > /usr/local/sbin/ntpd -c /etc/ntp.conf -g -g -u ntpd --nofork > ... > >I get similar results with /usr/sbin/ntpd, I've been testing both and > >happened to record details for the port ntpd. > > It's probably not relevant but it would be useful for you to say up front > which ntpd you are having problems with and which version of the port you > have installed. > > -- > Peter Jeremy > Hi Peter, I appreciate your thoughts. Yes, using LOCL is bad because it should only be used when the stratum 2 machine is unavailable, and it isn't (representative ping time average 0.15ms). The load is less than 10% on both devices and both the internet and internal traffic is typically less than 50Kb. :/ The use of LOCL clock was a fix as named failed if ntpd only used the timeserver and it lost sync (due to some ipsec work another story), I suspect kerberos had a part as it uses tkey-gssapi-keytab. I should investigate why the use of LOCL clock makes things work, but ... its a workaround and I'm ok with it. I'm at my wits end, I've systematically changed one variable from the list, and always the system clock reverts to LOCL within 20 minutes if not immediately. This is FreeBSD 12.1-STABLE #0 r356046M: Tue Dec 24. I think its time to try an earlier ntp to see if that helps (???) :( The variables tested, one changed at a time: - security.mac.ntpd.enabled - kern.elf64.aslr.enable kern.elf64.aslr.stack_gap changed as a pair - security.mac.portacl.rules - run as root or ntpd - use of proccontrol (which was changed with different combinations of aslr, stack_gap - all off and run as root - and of course changes to the command line (-g or -G or -g -x) I guess everyone else is using ntpd without a problem? (or not...) Cheers, Dewayne PS Apologies for delay in getting back, gmail placed your reply in the spam folder :/