From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 27 12:55:48 2008 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75CC716A420 for ; Sun, 27 Jan 2008 12:55:48 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.freebsd.org (Postfix) with ESMTP id 0332113C455 for ; Sun, 27 Jan 2008 12:55:47 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id A1A67208F for ; Sun, 27 Jan 2008 13:55:40 +0100 (CET) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: -0.2/3.0 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on tim.des.no Received: from ds4.des.no (des.no [80.203.243.180]) by smtp.des.no (Postfix) with ESMTP id 120D7208E for ; Sun, 27 Jan 2008 13:55:40 +0100 (CET) Received: by ds4.des.no (Postfix, from userid 1001) id E37CC844A2; Sun, 27 Jan 2008 13:55:39 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: hackers@freebsd.org Date: Sun, 27 Jan 2008 13:55:39 +0100 Message-ID: <86k5lv1l84.fsf@ds4.des.no> User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Cc: Subject: 'periodic daily' memory usage X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2008 12:55:48 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable 'periodic daily' runs my router out of swap every night, usually killing named as a result. A little sleuthing uncovered that the culprit is the 'sort -k 11' command in /etc/periodic/security/100.checksetuid. The easy solution would be to disable that script, but for obvious reasons, I'd rather not. Most of the time, named has the largest RSS of all the processes running on my router, by an order of magnitude. It's difficult to tell precisely since ssh'ing in to run 'top -o res' skews the results (how are you doing, mister Heisenberg?), but it's usually named followed by sshd and zsh. When 100.checksetuid is running, however, sort grows larger than even named. I tried modifying the script to feed considerably less data to sort, (only fields 2 and 11 from each line), but it doesn't seem to affect sort's memory usage. I'm starting to wonder if perhaps GNU sort uses a fixed-size buffer for each line of input, so reducing the length of the lines makes no difference. The solution I found that did work was to eliminate the loop over $MP and use 'find -s $MP ...' instead, which eliminates the need for sort. This reduces the memory requirement for 100.checksetuid by, oh, 80% or so, and greatly simplifies the logic. Note that 'find -s' and find | sort may not produce the same output, but this only means you'll get an ugly diff the first time you run the new script - it won't cause any trouble later. An entirely different issue is why named uses so much memory... does anybody know of a way to specify how much memory named may use for its cache? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=chksetuid.diff Index: etc/periodic/security/100.chksetuid =================================================================== RCS file: /home/ncvs/src/etc/periodic/security/100.chksetuid,v retrieving revision 1.9 diff -u -r1.9 100.chksetuid --- etc/periodic/security/100.chksetuid 23 Nov 2007 13:00:31 -0000 1.9 +++ etc/periodic/security/100.chksetuid 27 Jan 2008 12:54:38 -0000 @@ -43,22 +43,17 @@ [Yy][Ee][Ss]) echo "" echo 'Checking setuid files and devices:' - # XXX Note that there is the possibility of overrunning the args to ls - MP=`mount -t ufs,zfs | egrep -v " no(suid|exec)" | awk '{ print $3 }' | sort` - if [ -n "${MP}" ] - then - set ${MP} - while [ $# -ge 1 ]; do - mount=$1 - shift - find $mount -xdev -type f \ - \( -perm -u+x -or -perm -g+x -or -perm -o+x \) \ - \( -perm -u+s -or -perm -g+s \) -print0 - done | xargs -0 -n 20 ls -liTd | sed 's/^ *//' | sort -k 11 | - check_diff setuid - "${host} setuid diffs:" - rc=$? - fi;; - *) rc=0;; + MP=`mount -t ufs,zfs | awk '$0 !~ /no(suid|exec)/ { print $3 }'` + find -sx $MP -type f \ + \( -perm -u+x -or -perm -g+x -or -perm -o+x \) \ + \( -perm -u+s -or -perm -g+s \) -print0 | + xargs -0 ls -liTd | + check_diff setuid - "${host} setuid diffs:" + rc=$? + ;; + *) + rc=0 + ;; esac exit $rc --=-=-=--