Date: Tue, 2 Dec 1997 19:30:31 GMT From: nick@foobar.org To: FreeBSD-gnats-submit@FreeBSD.ORG Cc: nick@foobar.org Subject: bin/5193: Long rebuild times for pwd_mkdb with large master.passwd files Message-ID: <199712021930.TAA02446@iol.ie> Resent-Message-ID: <199712021940.LAA26928@hub.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 5193
>Category: bin
>Synopsis: It takes ages to rebuild master.passwd if it's large.
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Class: change-request
>Submitter-Id: current-users
>Arrival-Date: Tue Dec 2 11:40:01 PST 1997
>Last-Modified:
>Originator: Nick Hilliard
>Organization:
Ireland On-Line
>Release: FreeBSD 2.2.5-RELEASE i386
>Environment:
FreeBSD 2.2.5-RELEASE with large master.passwd file
>Description:
By default, FreeBSD doesn't cope well with rebuilding large
passwd files. On an unloaded P5/200 with 30000 passwd
entries with the default DB cache size of 2Mb, it takes
almost 30 minutes to do a complete passwd rebuild. Increasing
the cache size dramatically reduces the rebuild time.
I've included a patch below for implementing a "-s" option which
will specify the cache size in megabytes. This leads to
the following results:
# pwd_mkdb -p -s 2 /etc/master.passwd
14.772u 41.534s 26:11.28 3.5% 16+5850k 57+97145io 0pf+0w
# pwd_mkdb -p -s 4 /etc/master.passwd
13.720u 27.744s 17:45.82 3.8% 16+11227k 8+65377io 0pf+0w
# pwd_mkdb -p -s 6 /etc/master.passwd
11.786u 15.848s 9:49.47 4.6% 16+16253k 8+36842io 0pf+0w
# pwd_mkdb -p -s 8 /etc/master.passwd
10.224u 7.435s 4:30.88 6.5% 16+20428k 7+17582io 0pf+0w
# pwd_mkdb -p -s 10 /etc/master.passwd
9.927u 4.358s 2:34.70 9.2% 17+24852k 9+10910io 0pf+0w
# pwd_mkdb -p -s 12 /etc/master.passwd
10.253u 3.449s 1:45.71 12.9% 17+28356k 7+8043io 0pf+0w
# pwd_mkdb -p -s 14 /etc/master.passwd
10.814u 2.894s 1:24.57 16.1% 16+30982k 7+6657io 0pf+0w
# pwd_mkdb -p -s 16 /etc/master.passwd
12.406u 3.128s 1:13.88 21.0% 16+35270k 7+6153io 0pf+0w
# pwd_mkdb -p -s 18 /etc/master.passwd
12.515u 3.148s 1:12.56 21.5% 16+37672k 18+6303io 0pf+0w
# pwd_mkdb -p -s 20 /etc/master.passwd
13.733u 3.255s 1:17.34 21.9% 17+42232k 51+6223io 0pf+0w
As you can see, the rebuild time decreases assymptotically.
Is there any interest in putting this into the default distribution?
>How-To-Repeat:
Create large master.passwd file, type pwd_mkdb -p /etc/master.passwd,
and go on vacation for several weeks while waiting for the code to
execute.
>Fix:
diff -u -r1.1 pwd_mkdb.8
--- pwd_mkdb.8 1997/12/02 18:59:13 1.1
+++ pwd_mkdb.8 1997/12/02 19:14:03
@@ -76,6 +76,14 @@
Only update the record for the specified user. Utilities that
operate on a single user can use this option to avoid the
overhead of rebuilding the entire database.
+.It Fl s Ar cachesize
+Specify in megabytes the size of the memory cache used by the
+hashing library. On systems with a large user base, a small cache
+size can lead to prohibitively long database file rebuild times.
+As a rough guide, the memory usage of
+.Nm pwd_mkdb
+in megabytes will be a little bit more than twice the figure
+specified here. The default is 2 megabytes.
.El
.Pp
The two databases differ in that the secure version contains the user's
diff -u -r1.1 pwd_mkdb.c
--- pwd_mkdb.c 1997/12/02 18:59:13 1.1
+++ pwd_mkdb.c 1997/12/02 18:59:18
@@ -110,7 +110,7 @@
strcpy(prefix, _PATH_PWD);
makeold = 0;
username = NULL;
- while ((ch = getopt(argc, argv, "cd:pu:v")) != -1)
+ while ((ch = getopt(argc, argv, "cd:ps:u:v")) != -1)
switch(ch) {
case 'c': /* verify only */
cflag = 1;
@@ -123,6 +123,9 @@
break;
case 'u': /* only update this record */
username = optarg;
+ break;
+ case 's': /* change default cachesize */
+ openinfo.cachesize = atoi(optarg) * 1024 * 1024;
break;
case 'v': /* backward compatible */
break;
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199712021930.TAA02446>