From owner-freebsd-current Sun Feb 16 17:54:14 2003 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0BB5837B401 for ; Sun, 16 Feb 2003 17:54:12 -0800 (PST) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id B72A443F75 for ; Sun, 16 Feb 2003 17:54:10 -0800 (PST) (envelope-from bde@zeta.org.au) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id MAA01749; Mon, 17 Feb 2003 12:53:57 +1100 Date: Mon, 17 Feb 2003 12:54:38 +1100 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Julian Elischer Cc: FreeBSD current users Subject: Re: resource usage overflow In-Reply-To: Message-ID: <20030217121418.V4039-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 16 Feb 2003, Julian Elischer wrote: > In the resource usage we have teh following values calculated.. > > long ru_ixrss; /* integral shared memory size */ > long ru_idrss; /* integral unshared data " */ > long ru_isrss; /* integral unshared stack " */ > > in statclock() we have: > ru->ru_ixrss += pgtok(vm->vm_tsize); > ru->ru_idrss += pgtok(vm->vm_dsize); > ru->ru_isrss += pgtok(vm->vm_ssize); > > in other words these ore incremented at the rate of statclock > (for example 1kHz in not unheard of now). > > Assuming that I have a 600MB process (e.g. fsck doing a 1TB fileesyste > uses 667MB in pass 1 alone). > this means that the number of ticks we can count this amount of memory > usage for is: > 4E9/6E5 = 7.1E3 or 7100 ticks or, Strictly half of that on machines with 32-bit longs (they overflow at 2G, not 4G), and 2 billion times that on machines with 64-bit longs. > in the realworld (TM)... seven seconds if I am doing stats and profiling > at 1kHz. You shouldn't do stats at 1kHz. Even the basic tick counters are in danger of overflow if the statclock frequency is large (the counters are 64-bit, but the algorithms that use them (in calcru()) only work if they aren't much larger than 2^32). The profiling clock is normally scaled down to get a statclock of nearly 128Hz. However, no scaling of the scheduling clock is done for arches that don't have a separate profiling or statistics clock and use the scheduling clock for both. Large scheduling clock frequencies are more useful and less harmful than large statclock frequencies. > I think I could make a case for these figures being extended to 64 bits > but: They are already 64-bits on arches with 64-bit longs. > 1/ is it worth it? what uses them? Easier to drop them. In /usr/src, they are used in csh(1) (time builtin), time(1), window(1), acct(2), and potentially anything that calls getrusage(2). I think they are worth keeping for time(1), but it doesn't matter a lot of they are wrong. > 2/ are these mandated by any standard? would making them 64 bits > break anything? They aren't in POSIX-1.2001 (draft). Making them 64-bits on arches with 32-bit longs would break binary compatibilty of getrusage() too and other syscalls that return a struct rusage. > 3/ would 64 bits be enough? We are getting both bigger and faster > 64000 times faster and 64000 times bigger and we are back at seven > seconds. 640 times faster and 640 times bigger and we are still only at > 70000 seconds (19 hours) before overflow. Er, 64 bit longs provide a factor of 4 billion (not 64000) for bloat. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message