From owner-freebsd-hackers Tue Jul 13 17:15:15 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id 1507715369 for ; Tue, 13 Jul 1999 17:15:10 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id RAA82237; Tue, 13 Jul 1999 17:12:30 -0700 (PDT) (envelope-from dillon) Date: Tue, 13 Jul 1999 17:12:30 -0700 (PDT) From: Matthew Dillon Message-Id: <199907140012.RAA82237@apollo.backplane.com> To: John-Mark Gurney Cc: Matthew Jacob , freebsd-hackers@FreeBSD.ORG Subject: Swap subsystem overhead (was Re: Replacement for grep(1) (part 2)) References: <199907131920.MAA80146@apollo.backplane.com> <19990713165520.08447@hydrogen.fircrest.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :hmmm... so this means that on my home server where I have: :Device 1K-blocks Used Avail Capacity Type :/dev/od0b 262144 31176 230840 12% Interleaved :/dev/da1b 393216 31136 361952 8% Interleaved :/dev/da2b 262144 31072 230944 12% Interleaved :/dev/da3b 131072 31180 99764 24% Interleaved :/dev/sd4b 393216 30916 362172 8% Interleaved :/dev/sd5b 65536 30992 34416 47% Interleaved :/dev/sd6b 131072 30580 100364 23% Interleaved :Total 1637504 217052 1420452 13% :FreeBSD metriclient-2.uoregon.edu 3.0-RELEASE FreeBSD 3.0-RELEASE #19: Sun May 16 18:36:07 PDT 1999 root@:/a/home/johng/FreeBSD-checkout/30r/sys/compile/hydrogen i386 : :does this mean that the kernel is using more wired memory than it should :be? I have been able to do extensive swapping and when I do, I can get :around 3-4meg/sec for EACH of swapping in and out... so the performance :is pretty decient... and I have 80megs of ram in the machine... : :I have: :options "NSWAPDEV=10" : :in my kernel config file... : John-Mark Gurney Voice: +1 541 684 8449 Ok, I will be more specific. Under FreeBSD-STABLE *AND* FreeBSD-CURRENT, FreeBSD allocates metadata structures that scale to the amount of swap space assigned to the system. However, it is not *precisely* the amount of swap space. What it is is the largest single swap area multiplied by the number of swap areas. Your largest single swap area is 393MB so meta-data is allocated to cover 393MB x 10 = 3.93 GB worth of swap even though you only actually have 1.6GB of swap. The reason things are done this way is simply because it is the easiest way for the swap subsystem to allocate swap. It interleaves the swap areas equally across one big linear swath, and then reserves the holes that are left over when one swap area terminates before another. The swap subsystem can then attempt to allocate a linear swath that then happens to inherently interleave across multiple devices. Under FreeBSD-stable, the kern/subr_rlist module is used. This module nominally allocates holes only as required, but due to the interleaving there are lots of holes even when no swap is in use. So it scales roughly the same as FreeBSD-current. The one exception is in the case where all the swap areas are exactly the same size. In this case FreeBSD-stable does a better job in regards to the memory it uses for the initial allocation. Under FreeBSD-current we use the kern/subr_blist module, which is a fixed radix tree. FreeBSD-current preallocates all the necessary bitmaps required to manage the swap area so there is a larger initial memory allocation. FreeBSD-current makes up for this by having a (usually) smaller dynamic memory component once swapping actually starts to happen. FreeBSD-current preallocates the metadata in order to prevent low-memory deadlocks from occuring (something FreeBSD-stable cannot guarentee using rlists). FreeBSD-current's swap subsystem is also an O(1)ish algorithm to both allocate and free swap while FreeBSD-stable's swap subsystem uses an O(N) algorithm to free swap and approximately O(1) to allocate it (unless there is fragmentation, in which case it is worse). In your case, if you are running STABLE I recommend that you throw away your smaller areas to reduce fragmentation /dev/da3b, /dev/sd5b, and /dev/da6b. If you are running CURRENT it doesn't really matter... you will only save memory if you throw away your largest two areas, which I do not recommend. The memory used by the swap subsystem is probably a drop in the bucket for you anyway, so you normally do not have to worry about it. Under FreeBSD-current you can use: vmstat -m | more (then search for "VM pgdata") and vmstat -z | more (then search for "SWAPMETA") To determine how much memory has been wired to support the swap subsystem. "VM pgdata" is the fixed allocation supporting the radix tree while SWAPMETA is the dynamic allocation supporting currently swapped-out goods. Under FreeBSD-stable, just look under "VM pgdata" to see how much memory is being wired to support the swap subsystem. This usage covers both the fixed and dynamic allocations. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message