From owner-freebsd-bugs Thu Jun 27 17:40:29 2002 Delivered-To: freebsd-bugs@hub.freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id B77D737B405 for ; Thu, 27 Jun 2002 17:40:01 -0700 (PDT) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.4/8.12.4) with ESMTP id g5S0e1JU064866 for ; Thu, 27 Jun 2002 17:40:01 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.4/8.12.4/Submit) id g5S0e1Z9064864; Thu, 27 Jun 2002 17:40:01 -0700 (PDT) Received: from www.freebsd.org (www.FreeBSD.org [216.136.204.117]) by hub.freebsd.org (Postfix) with ESMTP id DC65037B476 for ; Thu, 27 Jun 2002 17:33:13 -0700 (PDT) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.12.4/8.12.4) with ESMTP id g5S0RSOT059757 for ; Thu, 27 Jun 2002 17:27:28 -0700 (PDT) (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.12.4/8.12.4/Submit) id g5S0RSxR059756; Thu, 27 Jun 2002 17:27:28 -0700 (PDT) Message-Id: <200206280027.g5S0RSxR059756@www.freebsd.org> Date: Thu, 27 Jun 2002 17:27:28 -0700 (PDT) From: Guolin Cheng To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-1.0 Subject: bin/39941: /usr/sbin/periodic sends thousands of emails Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org >Number: 39941 >Category: bin >Synopsis: /usr/sbin/periodic sends thousands of emails >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Jun 27 17:40:01 PDT 2002 >Closed-Date: >Last-Modified: >Originator: Guolin Cheng >Release: FreeBSD 4.6 >Organization: ALexa Internet. Inc. >Environment: FreeBSD rontmp.alexa.com 4.6-RELEASE FreeBSD 4.6-RELEASE #0: Fri Jun 21 20:37:42 GMT 2002 root@rontmp.alexa.com:/usr/src/sys/compile/ALEXA i386 >Description: The problems are: 1, /usr/sbin/periodic sends tons of email, when running hourly running scripts whether or not the latter generate outputs. Then email server will be overwhelmed if we have hundreds or thousands FreeBSD boxes. 2, periodic can not tell normal shell scripts from backup files ended with '~' and ',' , which are normally created by CVS/RVS/Perforce. it tries to run all executable files under base directory specified. 3, the coufiguration file */periodic.conf, which contain variable definition entries including _{show|output|...}, will confuse Bourne Shell interpreter when the basedir contains character '.', which is normal for directory name but not appropriate for shell variable name substitution. >How-To-Repeat: just create a directory /etc/periodic/hourly, and create a simply output-nothing shell script hahaha.sh under it: #! /bin/sh # do nothing and output nothing # echo "hahahaa... wrong :)" Set its executable mode bit and run the periodic with hourly directory as its argument.. sh -x /usr/sbin/periodic hourly Then copy the hahaha.sh to hahaha.sh~ and run the same command above, you will find that both shell scripts are processed! at last change the hourly directory name to cron.hourly name, and it will report problem even at beginning when processing the periodic.conf file! >Fix: To fix the problem, we need to tell periodic 1, skip shell scripts ended with '~' and '.', 2, substitute the '.' character to '_' in the periodic.conf and inform users about that special case, so that no base diretories like cron.daily and cron_daily exists at the same time. or if so, then the setting will overlaps. 3, send emails only when necessary, normally that means scripts under base directory generating outputs. The diff between original edited version of /usr/sbin/periodic are attached. shell> diff -c /usr/sbin/periodic /var/tmp/periodic *** /usr/sbin/periodic Mon Jun 10 21:19:56 2002 --- /var/tmp/periodic Thu Jun 27 15:10:53 2002 *************** *** 28,33 **** --- 28,34 ---- host=`hostname` export host tmp_output=`mktemp ${TMPDIR:-/tmp}/periodic.XXXXXXXXXX` + tmp_email_output=`mktemp ${TMPDIR:-/tmp}/periodic.XXXXXXXXXX` # Execute each executable file in the directory list. If the x bit is not # set, assume the user didn't really want us to muck with it (it's a *************** *** 35,42 **** for arg do # Where's our output going ? ! eval output=\$${arg##*/}_output case "$output" in /*) pipe="cat >>$output";; "") pipe=cat;; --- 36,47 ---- for arg do + # for special cases that the base directory has a '.' character contained, + # and difficult to change, then we can replace '.' with '_' in */period.conf, + # to pass the shell parameter name check for */periodic.conf + arg_in_conf=`echo $arg | tr '.' '_'` # Where's our output going ? ! eval output=\$${arg_in_conf##*/}_output case "$output" in /*) pipe="cat >>$output";; "") pipe=cat;; *************** *** 46,52 **** success=YES info=YES badconfig=NO # Defaults when ${run}_* aren't YES/NO for var in success info badconfig do ! case $(eval echo "\$${arg##*/}_show_$var") in [Yy][Ee][Ss]) eval $var=YES;; [Nn][Oo]) eval $var=NO;; esac --- 51,57 ---- success=YES info=YES badconfig=NO # Defaults when ${run}_* aren't YES/NO for var in success info badconfig do ! case $(eval echo "\$${arg_in_conf##*/}_show_$var") in [Yy][Ee][Ss]) eval $var=YES;; [Nn][Oo]) eval $var=NO;; esac *************** *** 72,79 **** processed=0 for dir in $dirlist do ! for file in $dir/* do if [ -x $file -a ! -d $file ] then output=TRUE --- 77,91 ---- processed=0 for dir in $dirlist do ! for file in $dir/*[^~,] ; ## Ignore *~ and *, scripts do + # Don't run *.{rpmsave,rpmorig,rpmnew,swp} scripts + [ "${file%.rpmsave}" != "${file}" ] && continue + [ "${file%.rpmorig}" != "${file}" ] && continue + [ "${file%.rpmnew}" != "${file}" ] && continue + [ "${file%.swp}" != "${file}" ] && continue + [ "${file%,v}" != "${file}" ] && continue + if [ -x $file -a ! -d $file ] then output=TRUE *************** *** 93,106 **** fi done done ! if [ $empty = TRUE ] ! then ! [ $processed = 1 ] && plural= || plural=s ! echo "No output from the $processed file$plural processed" ! else ! echo "" ! echo "-- End of $arg output --" ! fi ! } | eval $pipe done rm -f $tmp_output --- 105,124 ---- fi done done ! ## If there is no output from scripts processed, then we will not sendmail ! #if [ $empty = TRUE ] ! #then ! # [ $processed = 1 ] && plural= || plural=s ! # echo "No output from the $processed file$plural processed" ! #else ! # echo "" ! # echo "-- End of $arg output --" ! #fi ! } > $tmp_email_output ! if [ -s $tmp_email_output ] ; ! then ! cat $tmp_email_output | eval $pipe ! fi done rm -f $tmp_output + rm -f $tmp_email_output >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message