From owner-svn-src-head@freebsd.org Sat Apr 1 04:42:36 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AED86D288CF; Sat, 1 Apr 2017 04:42:36 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 702E1A11; Sat, 1 Apr 2017 04:42:36 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v314gZ47045003; Sat, 1 Apr 2017 04:42:35 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v314gZB5044999; Sat, 1 Apr 2017 04:42:35 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201704010442.v314gZB5044999@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Sat, 1 Apr 2017 04:42:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r316342 - in head: etc/defaults etc/periodic/daily share/man/man5 usr.sbin/periodic X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Apr 2017 04:42:36 -0000 Author: asomers Date: Sat Apr 1 04:42:35 2017 New Revision: 316342 URL: https://svnweb.freebsd.org/changeset/base/316342 Log: Consolidate random sleeps in periodic scripts Multiple periodic scripts sleep for a random amount of time in order to mitigate the thundering herd problem. This is bad, because the sum of multiple uniformly distributed random variables approaches a normal distribution, so the problem isn't mitigated as effectively as it would be with a single sleep. This change creates a single configurable anticongestion sleep. periodic will only sleep if at least one script requires it, and it will never sleep more than once per invocation. It also won't sleep if periodic was run interactively, fixing an unrelated longstanding bug. PR: 217055 PR: 210188 Reviewed by: cy MFC after: 3 weeks Differential Revision: https://reviews.freebsd.org/D10211 Modified: head/etc/defaults/periodic.conf head/etc/periodic/daily/480.leapfile-ntpd head/share/man/man5/periodic.conf.5 head/usr.sbin/periodic/periodic.sh Modified: head/etc/defaults/periodic.conf ============================================================================== --- head/etc/defaults/periodic.conf Sat Apr 1 01:00:36 2017 (r316341) +++ head/etc/defaults/periodic.conf Sat Apr 1 04:42:35 2017 (r316342) @@ -22,6 +22,8 @@ periodic_conf_files="/etc/periodic.conf # periodic script dirs local_periodic="/usr/local/etc/periodic" +# Max time to sleep to avoid causing congestion on download servers +anticongestion_sleeptime=3600 # Daily options @@ -136,8 +138,6 @@ daily_status_mail_rejects_shorten="NO" # 480.leapfile-ntpd daily_ntpd_leapfile_enable="YES" # Fetch NTP leapfile -daily_ntpd_avoid_congestion="YES" # Avoid congesting - # leapfile sources # 480.status-ntpd daily_status_ntpd_enable="NO" # Check NTP status @@ -307,6 +307,18 @@ security_status_tcpwrap_period="daily" if [ -z "${source_periodic_confs_defined}" ]; then source_periodic_confs_defined=yes + # Sleep for a random amount of time in order to mitigate the thundering + # herd problem of multiple hosts running periodic simultaneously. + # Will not sleep when used interactively. + # Will sleep at most once per invocation of periodic + anticongestion() { + [ -n "$PERIODIC_IS_INTERACTIVE" ] && return + if [ -f "$PERIODIC_ANTICONGESTION_FILE" ]; then + rm -f $PERIODIC_ANTICONGESTION_FILE + sleep `jot -r 1 0 ${anticongestion_sleeptime}` + fi + } + # Compatibility with old daily variable names. # They can be removed in stable/11. security_daily_compat_var() { Modified: head/etc/periodic/daily/480.leapfile-ntpd ============================================================================== --- head/etc/periodic/daily/480.leapfile-ntpd Sat Apr 1 01:00:36 2017 (r316341) +++ head/etc/periodic/daily/480.leapfile-ntpd Sat Apr 1 04:42:35 2017 (r316342) @@ -13,16 +13,9 @@ fi case "$daily_ntpd_leapfile_enable" in [Yy][Ee][Ss]) - case "$daily_ntpd_avoid_congestion" in - [Yy][Ee][Ss]) - # Avoid dogpiling - (sleep $(jot -r 1 0 3600); service ntpd onefetch) & - ;; - *) - service ntpd onefetch - ;; - esac - ;; + anticongestion + service ntpd onefetch + ;; esac exit $rc Modified: head/share/man/man5/periodic.conf.5 ============================================================================== --- head/share/man/man5/periodic.conf.5 Sat Apr 1 01:00:36 2017 (r316341) +++ head/share/man/man5/periodic.conf.5 Sat Apr 1 04:42:35 2017 (r316342) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd March 26, 2015 +.Dd March 31, 2015 .Dt PERIODIC.CONF 5 .Os .Sh NAME @@ -133,6 +133,10 @@ respectively. Refer to the .Xr periodic 8 manual page for how script return codes are interpreted. +.It Va anticongestion_sleeptime +.Pq Vt int +The maximum number of seconds to randomly sleep in order to smooth bursty loads +on a shared resource, such as a download mirror. .El .Pp The following variables are used by the standard scripts that reside in Modified: head/usr.sbin/periodic/periodic.sh ============================================================================== --- head/usr.sbin/periodic/periodic.sh Sat Apr 1 01:00:36 2017 (r316341) +++ head/usr.sbin/periodic/periodic.sh Sat Apr 1 04:42:35 2017 (r316342) @@ -76,6 +76,12 @@ fi shift arg=$1 +if [ -z "$PERIODIC_ANTICONGESTION_FILE" ] ; then + export PERIODIC_ANTICONGESTION_FILE=`mktemp ${TMPDIR:-/tmp}/periodic.anticongestion.XXXXXXXXXX` +fi +if tty > /dev/null 2>&1; then + export PERIODIC_IS_INTERACTIVE=1 +fi tmp_output=`mktemp ${TMPDIR:-/tmp}/periodic.XXXXXXXXXX` context="$PERIODIC" export PERIODIC="$arg${PERIODIC:+ }${PERIODIC}" @@ -141,3 +147,4 @@ esac } | output_pipe $arg "$context" rm -f $tmp_output +rm -f $PERIODIC_ANTICONGESTION_FILE