Date: Wed, 19 Jul 2000 15:32:14 -0300 (ADT) From: The Hermit Hacker <scrappy@hub.org> To: freebsd-questions@freebsd.org Subject: Shared Memory and Semaphores under 4.0-STABLE Message-ID: <Pine.BSF.4.21.0007191323450.1745-100000@thelab.hub.org>
next in thread | raw e-mail | index | archive | help
First off, has anyone ever come up with a *good* resource on setting this
stuff up under FreeBSD? Mailing list archives, FAQs and Handbooks, so far
as I can tell, come up pretty much with alot of questions, few answers :(
I'm trying to increase the number of clients able to access a PostgreSQL
v7.x database server. The machine is a Dual-PIII with 512Meg of RAM and
I'm just building a kernel that looks like:
options SYSVSHM
options SHMMAXPGS=524288
options SHMSEG=128
options SYSVSEM
options SEMMNI=80
options SEMMNS=480
options SEMMNU=240
options SEMMAP=240
options SYSVMSG #SYSV-style message queues
but other then doubling hte old values, I haven't got a clue how
co-relates ...
For instance, I'm trying to get it so that I can start up 128 backends
simultaneously, since the 64 I have it at now keeps getting hit.
According to the PostgreSQL man page for the postmaster process, this
involves increasing the -N option to 128 (defaults to 32) ... but, that
also requires upgrading the -B option to 256 (must be double -N) ...
so, according to the man page, I need 256*8k bytes ... so I need 2Meg of
Shared Memory (ya, my SHMMAXPGS is overkill above) ... so, as far as
shared memory is concerned, I need to set SHMMAXPGS to what? What is a
PGS in this instance? the system PAGE_SIZE? If so, for the above
requirement of 256*8k, I'd have to set SHMMAXPGS to 512 (based on sysctl
-n hw.pagesize reporting a 4096 page size)?
So, I can reduce my shared memory from the atrocious value I have above on
my next kernel compile, if this is correct ...
Now, what about SHMSEG? Do I need a 1:1 ration to the number of
processes?
The problem I'm hitting, and am working towards rectifying, is:
IpcSemaphoreCreate: semget failed (No space left on device) key=5432021, num=16, permission=600
Based on the following doc that I found, I'm quite confused as to exactly
which part I want to set to get rid of this :(
Basically, if I want to have 128 processes going, I need one semaphore
identifier per process, correct? so SEMMNI needs to be 128 (or
larger)? At the same time, I need as many, or more, semaphores in the
system then identifiers ...?
Does this make sense, giving me room to grow?
================================
options SYSVSHM
options SHMMAXPGS=1024
options SHMSEG=256
options SYSVSEM
options SEMMNI=256
options SEMMNS=512
options SEMMNU=256
options SEMMAP=256
===============================
correctly, semmap defines
===========================================================================
Name Default Max Brief Description
------ ------- -------------- -------------------------------------
semmap 10 2147483647 Number of entries in semaphore map
semmni 10 65535 Number of semaphore sets (identifiers)
semmns 60 2147483647 Number of semaphores in the system
65535 (usage)
semmnu 30 2147483647 Number of "undo" structures in the system
semmsl 25 2147483647 Max number of semaphores, per semaphore id
65535 (usage)
semopm 10 2147483647 Max number of operations, per semaphore call
semume 10 2147483647 Max number of "undo" entries, per process
semusz 96 *see below* Size in bytes of "undo" structure
semvmx 32767 2147483647 Semaphore maximum value
65535 (usage)
semaem 16384 2147483647 Adjust on exit maximum value
32767 (usage)
Detailed Descriptions
---------------------
semmap
Defines the size of the semaphore resource map; each block of available,
contiguous semaphores requires one entry in this map. This is the pool from
which semget(2) acquires semaphore sets.
When a semaphore set is removed (deleted), if the block of semaphores to be
freed is adjacent to a block of semaphores already in the resource map, the
semaphores in the set being removed are added to the existing map entry;
no new map entry is required. If the semaphores in the removed set are not
adjacent to those in an existing map entry, then a new map entry is required
to track these semaphores; if there are no more map entries available, the
system has to discard an entry, 'permanently' losing a block of semaphores
(permanence is relative; a reboot fixes the problem). If this should occur,
a WARNING will be generated, the text of which will be something like
"rmallocmap: rmap overflow, lost ...". The end result is that a user could
later get ENOSPC errors from semget(2) even though it doesn't look like all
the semaphores are allocated.
semmni
Defines the number of semaphore sets (identifiers), system wide. Every
semaphore set in the system has a unique indentifier and control structure.
The system pre-allocates kernel memory for semmni control structures; each
control structure is 84 bytes. If no more identifiers are available,
semget(2) returns ENOSPC.
Attempting to set semmni to a value greater than 65535 will result in
generation of a WARNING, and the value will be set to 65535.
semmns
Defines the number of semaphores in the system; 16 bytes of kernel memory is
pre-allocated for each semaphores. If there is not a large enough block of
contiguous semaphores in the resource map (see semmap) to satisfy the request,
semget(2) returns ENOSPC.
Fragmentation of the semaphore map will result in ENOSPC errors, even though
there may appear to be ample free semaphores. Despite attempts by the system
to merge free sets (see semmap), the size of the clusters of free semaphores
generally decreases over time. For this reason, semmns frequently must be set
higher than the actual number of semaphores required.
semmnu
Defines the number of semaphore undo structures in the system. semusz (see
below) bytes of kernel memory are pre-allocated for each undo structure; one
undo structure is required for every process for which undo information must
be recorded. semop() will return ENOSPC if it is requested to record undo
information and there are no undo structures available.
semmsl
Limits the number of semaphores that can be created for a single semaphore id.
If semget(2) returns EINVAL, this limit should be increased. This parameter
is only used to validate the argument passed to semget(2). Logically, it
should be less than or equal to semmns (see above). Setting semmsl too high
might allow a few identifiers to hog all the semaphores in the system.
semopm
Limits the number of operations that are allowed in a single semop(2) call.
If semop(2) returns E2BIG, this limit should be increased. This parameter is
only used to validate the argument passed to semop(2).
semume
Limits the number of undo records that can exist for a process. If semop(2)
returns EINVAL, this limit should be increased. In addition to its use in
validating arguments to semop(2), this parameter is used to calculate the
value of semusz (see below).
semusz
Defines the size of the semaphore undo structure. Any attempt to modify this
parameter directly will be ignored; semusz is always calculated based upon
the value of semume (see above); semusz = 8 * (semume + 2).
semvmx
Limits the maximum value of a semaphore. Due to the interaction with undo
structures and semaem (see below), this tuneable should not be increased
beyond its default value of 32767, unless you can guarantee that SEM_UNDO is
never and will never be used. It can be safely reduced, but doing so provides
no savings.
semaem
Limits the maximum value of an adjust-on-exit undo element. No system
resources are allocated based on this value.
=======================================
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0007191323450.1745-100000>