From owner-svn-src-all@freebsd.org Thu Oct 13 21:42:32 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD5F9C10F75; Thu, 13 Oct 2016 21:42:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x22c.google.com (mail-qk0-x22c.google.com [IPv6:2607:f8b0:400d:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3E4533D; Thu, 13 Oct 2016 21:42:32 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x22c.google.com with SMTP id f128so110741020qkb.1; Thu, 13 Oct 2016 14:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=eqPNMl7awoy8ADNBfv6wfK42vX3Ji7yPHPvBfq5T4t8=; b=IFf47xw5j3HVFYqqsgimGXu69ahOmu4AV5lAZQVVnkSqOSplsplUGMUg0BRX0lXBPV tfCoGL7enPMPO23CR69kA6GKxdzzKMzxxXdnVAGU9w4FMM1WtOr+qVTKg4pBqs5Vmo58 dM7g/NICl4eHIrBRKddiYDLZ9V4AD1fZxc2GJpvrFT/47LkW/IrVjtoVBrviGm7WGhjp UeDERfoIUGmORqgmTolHTMbWHLR22Jaetqw2pYu+gVStV8zOpbbBhnr1PHEeCLwG/ING abZaD8bARdEJ8E2CaTo/gA8HslktWOKM6WYAUAxR7sFX/szEGTwu7qQ7D7CDa4RpzNZk Lvow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=eqPNMl7awoy8ADNBfv6wfK42vX3Ji7yPHPvBfq5T4t8=; b=lx/OYtAYk34bylQnsDtxl2GOqMUISCWT861UfZGDm+XB32o8fByJCFBMGu06dYc0/h RnC9voWLRIUe2NeYG5eFS9Nm8vdcXc5KxROWCQDlJUqsgD4FIbgJcIz8+oq14esGEy5T bocNywTy9U93blRqAClRj+8wk68YNeYIsi+lGDQozQqsIFR9f3nZj0txmgvjDsLjpJ4M dwCD4RwvdHDWqVlOhckACe6BDkeKL4kXHnrCXyECjTtD4oxPSEPlG3LqDbRTHGNYlMqd IkkI6Ped6PG3yacdecco/riYcNzrrCOD/VaIo7WdW2py+pkB7nKQPR0RwuXReoYx+09g GMDw== X-Gm-Message-State: AA6/9RmZg+NWOKv3HMrFFIUt1p/nYCz0dEKqW3GObDRS1qMdaGK3rskRZV7M9bfDNosocjbq5ZXHNm0qLQQhZw== X-Received: by 10.55.80.136 with SMTP id e130mr8396419qkb.119.1476394951700; Thu, 13 Oct 2016 14:42:31 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.138.251 with HTTP; Thu, 13 Oct 2016 14:42:31 -0700 (PDT) In-Reply-To: <201507021731.t62HV074085188@repo.freebsd.org> References: <201507021731.t62HV074085188@repo.freebsd.org> From: Alan Somers Date: Thu, 13 Oct 2016 15:42:31 -0600 X-Google-Sender-Auth: Zi29YmTRfK1BpHLcf0OjKwSW8iQ Message-ID: Subject: Re: svn commit: r285050 - in head: lib/libutil usr.sbin/pwd_mkdb To: Renato Botelho Cc: "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 21:42:33 -0000 On Thu, Jul 2, 2015 at 11:31 AM, Renato Botelho wrote: > Author: garga (ports committer) > Date: Thu Jul 2 17:30:59 2015 > New Revision: 285050 > URL: https://svnweb.freebsd.org/changeset/base/285050 > > Log: > When passwd or group information is changed (by pw, vipw, chpass, ...) > temporary file is created and then a rename() call move it to official file. > This operation didn't have any check to make sure data was written to disk > and if a power cycle happens system could end up with a 0 length passwd > or group database. > > There is a pfSense bug with more infor about it: > > https://redmine.pfsense.org/issues/4523 > > The following changes were made to protect passwd and group operations: > > * lib/libutil/gr_util.c: > - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file > - After rename(), fsync() call on directory for faster result > > * lib/libutil/pw_util.c > - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file > > * usr.sbin/pwd_mkdb/pwd_mkdb.c > - Added O_SYNC flag on dbopen() calls > - After rename(), fsync() call on directory for faster result > > * lib/libutil/pw_util.3 > - pw_lock() returns a file descriptor to master password file on success > > Differential Revision: https://reviews.freebsd.org/D2978 > Approved by: bapt > Sponsored by: Netgate > > Modified: > head/lib/libutil/gr_util.c > head/lib/libutil/pw_util.3 > head/lib/libutil/pw_util.c > head/usr.sbin/pwd_mkdb/pwd_mkdb.c This change is making certain pw operations very slow on ZFS root systems. The problem is that when you open a file with O_SYNC, every single write(2) call turns into a zil_commit on ZFS, which is fairly expensive. Did you consider using fsync(2) on the temporary files instead of opening them with O_SYNC? I just tried that now, and I see a considerable speedup when running the tests in /usr/tests/usr.sbin/pw: Using O_SYNC, as CURRENT does: 4 minutes 5.2 seconds No synchronous operations at all: 49.5 seconds Using fsync(2): 56.0 seconds -Alan