From owner-freebsd-questions@freebsd.org Tue Oct 18 14:27:02 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 286AAC16DD8 for ; Tue, 18 Oct 2016 14:27:02 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: from cosmo.uchicago.edu (cosmo.uchicago.edu [128.135.70.90]) by mx1.freebsd.org (Postfix) with ESMTP id DC53FD0F for ; Tue, 18 Oct 2016 14:27:01 +0000 (UTC) (envelope-from galtsev@kicp.uchicago.edu) Received: by cosmo.uchicago.edu (Postfix, from userid 48) id 0285FCB8CBC; Tue, 18 Oct 2016 09:27:00 -0500 (CDT) Received: from 128.135.52.6 (SquirrelMail authenticated user valeri) by cosmo.uchicago.edu with HTTP; Tue, 18 Oct 2016 09:27:00 -0500 (CDT) Message-ID: <54165.128.135.52.6.1476800820.squirrel@cosmo.uchicago.edu> In-Reply-To: References: <20161018074415.0f6575675db30d0b83d9b8ec@sohara.org> <60719.69.209.239.247.1476794773.squirrel@cosmo.uchicago.edu> <20161018140531.GA1080@lime.woodcruft.co.uk> Date: Tue, 18 Oct 2016 09:27:00 -0500 (CDT) Subject: Re: pw add user slow From: "Valeri Galtsev" To: "Paul Macdonald" Cc: freebsd-questions@freebsd.org Reply-To: galtsev@kicp.uchicago.edu User-Agent: SquirrelMail/1.4.8-5.el5.centos.7 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 14:27:02 -0000 On Tue, October 18, 2016 9:17 am, Paul Macdonald wrote: > > > On 18/10/2016 15:05, Frank Shute wrote: >> On Tue, Oct 18, 2016 at 07:46:13AM -0500, Valeri Galtsev wrote: >>> >>> On Tue, October 18, 2016 1:44 am, Steve O'Hara-Smith wrote: >>>> On Mon, 17 Oct 2016 23:36:33 +0100 >>>> Paul Macdonald wrote: >>>> >>>>> Hi, >>>>> >>>>> I have to migrate an imap server, which uses about 6000 local >>>>> accounts >>>>> for mail delivery >>>>> >>>>> I'm using a script that creates system acounts using: >>>>> >>>>> /usr/sbin/pw adduser $LOCAL_USER -u$UID-gGID-d $PATH -s/bin/false >>>>> >>>>> I'm finding this grows ever more slow, up to 10 seconds per account >>>>> on a >>>>> SATA box , and on a test SSD box ( 20 acc's per second initially) >>>>> even >>>>> that slows to about 1 account created per second. >>>>> >>>>> why does this slow down as more accounts are added? >>>> Because for each one it updates text files (passwd and >>>> master.passwd) and regenerates databases (pwd.db and spwd.db) from the >>>> text >>>> files. The fastest way to do this sort of thing is to directly update >>>> passwd and master.passwd and then run pwd_mkdb. >>> It appears to be sufficient to only edit master.passwd; pwd_mkdb will >>> re-generate passwd file from master.passwd. >>> >> There's a tool for that: vipw(8) >> >> It uses $EDITOR if set, vi as default. >> >> If using vi/vim then fire up vipw and: >> >> :r!myscript >> >> will stick stdout from myscript into the buffer where the cursor is. >> Then: >> >> :wq >> >> and vipw will write it if it's conformant and then call pwd_mkdb to do >> it's >> business. >> >> The OP will also have to edit /etc/group to add his users to the >> appropriate >> group(s). >> >> Doesn't the imap server have an option for using a relational database >> rather >> than filling passwd up? I'd go for that option if it does. >> >> >> Regards, >> > manually writing to the master passwd and running pwd_mkdb was a lot > quicker, thanks There is one important thing Frank said: using vipw instead of any of regular text editors: vipw will set locks, thus preventing mess if anyone on live system uses password tools while you are editing master.passwd. Thanks, Frank and Steve, all you said was very instructive! Valeri > > at 10 seconds per account, 6500 would take a long time... (16hrs!) > > the system accounts are for sendmail delivery to mailboxes, possibly a > good time to take a look at exim/postfix. > > thanks > Paul > > -- > ------------------------- > Paul Macdonald > IFDNRG Ltd > Web and video hosting > ------------------------- > t: 0131 5548070 > m: 07970339546 > e: paul@ifdnrg.com > w: http://www.ifdnrg.com > ------------------------- > IFDNRG > 40 Maritime Street > Edinburgh > EH6 6SA > ---------------------------------------------------- > > Virtual Servers from £50.00pm > High specification Dedicated Servers from £150.00pm > > ---------------------------------------------------- > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscribe@freebsd.org" > ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++