From owner-freebsd-hackers Wed Aug 30 10:41:03 1995 Return-Path: hackers-owner Received: (from majordom@localhost) by freefall.FreeBSD.org (8.6.11/8.6.6) id KAA16137 for hackers-outgoing; Wed, 30 Aug 1995 10:41:03 -0700 Received: from ns1.win.net (ns1.win.net [204.215.209.3]) by freefall.FreeBSD.org (8.6.11/8.6.6) with ESMTP id KAA16125 for ; Wed, 30 Aug 1995 10:41:01 -0700 Received: (from bugs@localhost) by ns1.win.net (8.6.11/8.6.9) id NAA27537 for hackers@freebsd.org; Wed, 30 Aug 1995 13:46:43 -0400 From: Mark Hittinger Message-Id: <199508301746.NAA27537@ns1.win.net> Subject: re: re: pwd_mkdb is intense To: hackers@freebsd.org Date: Wed, 30 Aug 1995 13:46:43 -0400 (EDT) X-Mailer: ELM [version 2.4 PL23] Content-Type: text Content-Length: 2317 Sender: hackers-owner@freebsd.org Precedence: bulk I am resending this because I did not see it come through. Looks like the server there took another disk space hit, because the sup's appear to be a little messy today also. Apologies if you've seen this. Right now we have about 11552 lines in our password file. Changing a password or adding an account causes pwd_mkdb to run for about 2-3 minutes on a p90/pci consuming 20-30% of the cpu. In addition my swap space (which is rather large) goes up around 15% (say around 25 megabytes worth). Reviewing pwd_mkdb shows that the password records appear to be stored in 8 seperate places. 1. /etc/passwd 2. /etc/master.passwd 3. /etc/pwd.db pw record keyed by username 4. /etc/pwd.db pw record keyed by 'count' 5. /etc/pwd.db pw record keyed by uid 6. /etc/spwd.db pw record keyed by username 7. /etc/spwd.db pw record keyed by 'count' 8. /etc/spwd.db pw record keyed by uid We are devoting around 8.5 megabytes to this structure. I realize a lot of what pwd_mkdb does is for locking and maintaning compatiblity. My beef is with the way we are using db for this. On a more vanilla level my experience with mixing keys and data in the same file has nearly always been bad. This is both from a production performance and a development/debugging perspective. It looks like we are doing weird things with the keys to prevent a duplicate key scenario. Therefore, since I am permitted to wish out loud, I would prefer to see two flat binary pw record files, one for public and one for shadow. I would like to see three entirely seperate files for the keys, each keyed file record simply has the key and a pointer to the flat binary file's record. We add between 20 and 30 accounts per day right now. It would be slick if I could simply append the records to the flat files and add the three keys to the key files. When I need to update a pw record, I could just update the flat pw record in place. I would not need to recreate the entire key structure each time. Wish Wish - Now tell me why we can't do it because we have to maintain compatibility with some other bogus implementation :-) I am playing around with buffers and caches now trying to pump it up, but I suspect we are just doing something that is not scalable to these levels and beyond. Regards, Mark Hittinger bugs@win.net