From owner-freebsd-stable@freebsd.org Mon Jan 22 17:07:53 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5030ECAAD1 for ; Mon, 22 Jan 2018 17:07:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x233.google.com (mail-lf0-x233.google.com [IPv6:2a00:1450:4010:c07::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2852B84FC4 for ; Mon, 22 Jan 2018 17:07:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x233.google.com with SMTP id t139so11496646lff.0 for ; Mon, 22 Jan 2018 09:07:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:from:date:message-id:subject:to; bh=kqlyqTMDYc/aVrX1iI02jrMnJm8lHUSnmRnu3D9H94E=; b=hvzWJhvFIjkPRQbq8eX9WbGtqjnuxtm3nGCY1fYf1Fm/clNqp1nEapHzc1LDkfLhTA B1j+/TQEYEG0KyvJtcudt3OJV4A/RVXIABKS8O/kaqUY/d6lWvmk1BvF4Zl/dpdHDb9I ci/YK7ZxTBwDifaRXaoTjsAiFbK87hr48uKBNZZVFT2mXVzL9YdUyhwPnsyhu1ImW2+p UBOxemV8DFVc6D8ws3+frYyRWxSho3Q5+lvnCgY21BbvzU5/9kTwmYwEBy8/uQWcTCw6 nQQhVD2fF09QzW4xRdd96VWUqLSBkMj889PAj3IttBAHTLaV1PBNcfnC/HyqGHvBKDth 9Cdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=kqlyqTMDYc/aVrX1iI02jrMnJm8lHUSnmRnu3D9H94E=; b=XBvqhfxvh0HmgDWg76t2cHocQsXzfqZqOy7QxVjvesf+oeQ5FFsyo4MtY48dyc4J+L JAFJ3KdHeylPOOgWC1ybVgCQP6kYP8D2LZ/lHgrxiaF7ta0TtMa3Wy799Pu348uBCXT2 u1v8gvDRrxr7fXXrzrG35NFQP11UOqF4OyXnssqSe7azvY1P9foLHfrmcx3CAp4GFFK+ 1qVJTcjHNh48XzFQHJ3VDdfUVMoxLOSdBWMtoqMtyMgPEMhCyX31V6PvSa+phFZkb+7g eCEK/fvdwq0u/NUAmSMf2bynR2NebM2khqCocy4TYBfpBrpjOnSROQHoV+61nmZMTFr8 fQbA== X-Gm-Message-State: AKwxyteCpEsicQGBU1YdNGQi+Y9KrcZwZVdXvXLFuR/6Pe9R1c8il+jN mY1oMlm72BXK31SUyvE0flUrxvTsykJYwbxugk08kA== X-Google-Smtp-Source: AH8x225tuaksWFHEJZcZUBKxhYbXY3fxbLa3N80Dfdhx7Uux7qyYbZZVRGvtDU0Vok6pw1WdL/xW7pjOIAsHVKNR3nE= X-Received: by 10.46.93.210 with SMTP id v79mr3799243lje.80.1516640870905; Mon, 22 Jan 2018 09:07:50 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.87.131 with HTTP; Mon, 22 Jan 2018 09:07:50 -0800 (PST) From: Alan Somers Date: Mon, 22 Jan 2018 10:07:50 -0700 X-Google-Sender-Auth: Hpp8TvwsfeSUZzpvasnqX3AKle8 Message-ID: Subject: Clock occasionally jumps backwards on 11.1-RELEASE To: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jan 2018 17:07:53 -0000 Since upgrading my jail server to 11.1-RELEASE, the clock occasionally jumps backwards by 5-35 minutes for no apparent reason. Has anybody seen something like this? Details ===== * Happens about once a day on my jail server, and has happened at least once on a separate bhyve server. * The jumps almost always happen between 1 and 3 AM, but I've also seen them happen at 06:30 and 20:15. * The jumps are always backwards, never forwards. * Inspecting the logs of both the host and its jails shows nothing interesting that's correlated with the jumps. Sometimes I find Amanda doing a backup, but not always. * Sometimes the jumps happen immediately after ntpd adds a new server to its list, but not always. * I'm using the default ntp.conf file. * ntpd is running on both, and it should be the only process touching the clock. I have a script running "ntpq -c peers" once a minute, which shows the offset for one server suddenly jump to a large negative number. Then the offsets for other servers jump to the same value, then either ntpd fixes the clock or exits because the offset is too high. * Said script is sleeping using the monotonic clock, not the realtime clock. As expected, successive timestamps differ by about 6.5 minutes when ntpd corrects a 5.5 minute clock offset. However, when the clock presumably jumps backwards I _don't_ see successive timestamps go backwards too. They keep marching forward at the expected rate. This makes me wonder if the entire machine is hanging. But it would have to be a pretty serious hang to stop the clock from ticking. Any ideas?