From owner-freebsd-smp Tue Nov 16 12: 6:57 1999 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id A50DE15281; Tue, 16 Nov 1999 12:06:49 -0800 (PST) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.2) with ESMTP id VAA26898; Tue, 16 Nov 1999 21:06:43 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Date: Tue, 16 Nov 1999 21:06:42 +0100 Message-ID: <26892.942782802.1@critter.freebsd.dk> From: Poul-Henning Kamp Subject: Re: the ps/cmdline caching MIME-Version: 1.0 Content-Type: multipart/digest; boundary="----- =_aaaaaaaaaa" Content-Description: Blind Carbon Copy Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org ------- =_aaaaaaaaaa Content-Type: message/rfc822 Content-Description: Original Message To: core@freebsd.org Subject: Re: the ps/cmdline caching In-reply-to: Your message of "Tue, 16 Nov 1999 11:37:40 MST." <199911161837.LAA00345@caspian.plutotech.com> Date: Tue, 16 Nov 1999 21:06:42 +0100 Message-ID: <26892.942782802@critter.freebsd.dk> From: Poul-Henning Kamp MIME-Version: 1.0 In message <199911161837.LAA00345@caspian.plutotech.com>, "Justin T. Gibbs" wri tes: >I must have missed a thread somewhere. Can I get a reference to what >this problem is? Run this on a SMP box and it will die in seconds: i=0 while [ $i -lt 200 ] do i=`expr $i + 1` ( while true ; do ps xao pid,command > /dev/null 2>&1 ; done ) & done The problem is where ps using /proc/%pid/mem grovels around in the other process address room to get hold of argv. This gets particular nasty when we have multiple copies of ps(1) running on a SMP box: First CPU: running process 100 which is a "ps -ax" currently trying to get the argument list from process 101 Second CPU: running process 101 which is a "ps -ax" currently trying to get the argument list from process 101 (or 100 for that matter). The code in procfs_mem.c has significant problems in this scenario, and rather than try to hunt them down, something we should eventually do my solution is to hang a copy of of the arg list from struct proc and use a sysctl in the kern.proc family to access it. My implementation has a sysctl variable which sets an upper limit on how many bytes we use per process for this. If this limit is exceeded, we default to the current reality, obviously the limit can be set to zero as well. For anyone wanting to work on procfs_mem, setting the sysctl to zero will revert to current behaviour so that bug is not obscured. The arguments are stored in a separate structure which is shared across fork and replaced in exec, so the overhead in struct proc itself is only a pointer, and for daemons which fork a lot of children only one copy of the arguments are stored, until they set their own with setproctitle() that is. The side effects of this change are many and varied: Plus side: 1. ps(1) runs much faster and uses far fewer resources. 2. ps(1) don't need a /proc anymore. Particular nice for chroot and jail. 3. We get access to the full command line in kernel debuggers. 4. (untested/unimplemented) /bin/ps doesn't need to be setgid kmem anymore. 5. On the long run we use less memory because we don't need to i allocate a vnode and inode for /proc/%pid/mem. (This is not implemented yet, we need to figure out how much ps(1) should use libkvm and fix it accordingly). On the minus side: 1. Memory usage, if all your process have very long commandlines. (You can limit this with the sysctl.) 2. A process which writes to argv[0] rather than use setproctitle() doesn't have the desired effect. (Setting the sysctl to zero solves this problem as well.) 3. If you never run ps(1) there is a epsilon sized overhead in exec(2). (You can make that a epsilon-squared sized overhead by setting the sysctl to zero.) So expect to hear a happy Paul Saab sing our praise once again and expect to see my commit to -current in a few moments. -- Poul-Henning Kamp FreeBSD coreteam member phk@FreeBSD.ORG "Real hackers run -current on their laptop." FreeBSD -- It will take a long time before progress goes too far! ------- =_aaaaaaaaaa-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message