From owner-svn-src-head@freebsd.org Fri Oct 21 20:36:35 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24DC5C1C117; Fri, 21 Oct 2016 20:36:35 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8DF09FB; Fri, 21 Oct 2016 20:36:34 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qt0-x234.google.com with SMTP id m5so95772792qtb.3; Fri, 21 Oct 2016 13:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Jdd66SUYt7qgdg+FLVUXV1F4mrrOfnHbxDpIY4CzZq0=; b=UZ3sot7UsyTtElAIn3iASYMb3lIOdrotkmMckXislJX/i5C7DSA95DYPNhskyVodg5 eu46xiCcJ72dKigMfFKETV3tvDLJEAN6C9fhEcvPbD+mNBmbxxTD/02Kepor8Yv0Swgl 5UPPKL5xtvBMou4Zgmo26d2NAJot/1+SxVnbBvuJ9PUasjF00POJ8eZLaiX3JUEZ7uTm r4SBeUFa/KvW2rGbJnmORmbnbi6frzG2/OKLHcGw96LeARovI3cut4bDL0lp5vLOzKL1 YsLdoaXk16ZXDtH4gLjh1To0XxJ58xdXdyJMfPa2WyXr7o9C5HK+GQuZGB8gfBCq5Wiw wGzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Jdd66SUYt7qgdg+FLVUXV1F4mrrOfnHbxDpIY4CzZq0=; b=m0LScM8YWqZRJg+O0B+SgpWWJFBsrcbwSQjFORzhCrZ3q/HZO/XmRTPOmoQ/v+K0Gz xnvPlEcS9GIUWPfS22+lBogzx6uCOd1b5rchG5fGt+csMo/w6p/v1/8ANIATVqyGGcBR MZTG4OWAuHJr99hfwOUgzjE6I5B0F2VZ9NFrUPygTyFhYiE6BBrAHF/5AkZ9/kFEtpZl K+wXkk6fih/WcxNqh0YsAeQG9SR9QJdi/pyeidTzd6xeDuWvqWmTQ26N+3Ycqocw4E4Y RTtXC1dWcGaTJeWJ6YJDWYQf9FpKpf6qsNnAUfBX28sC9EGTubEtPNH13xGuk/8Sykhc kUBQ== X-Gm-Message-State: ABUngvcjWPySIm7Y4bYfpG/Tf2Eipvpm2Ryz6+B2HK50ctxfWrgMvFFu2geiVyELiYrYn+cw+Q0GmDESPOxUEw== X-Received: by 10.200.51.20 with SMTP id t20mr3354449qta.48.1477082193870; Fri, 21 Oct 2016 13:36:33 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.138.251 with HTTP; Fri, 21 Oct 2016 13:36:33 -0700 (PDT) In-Reply-To: <201610132306.u9DN6Aod023917@gw.catspoiler.org> References: <201610132306.u9DN6Aod023917@gw.catspoiler.org> From: Alan Somers Date: Fri, 21 Oct 2016 14:36:33 -0600 X-Google-Sender-Auth: kgwL3LL6ir1A0sjogb3Ffc-5Xic Message-ID: Subject: Re: svn commit: r285050 - in head: lib/libutil usr.sbin/pwd_mkdb To: Don Lewis Cc: Renato Botelho , "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 20:36:35 -0000 On Thu, Oct 13, 2016 at 5:06 PM, Don Lewis wrote: > On 13 Oct, Alan Somers wrote: >> On Thu, Jul 2, 2015 at 11:31 AM, Renato Botelho wrote: >>> Author: garga (ports committer) >>> Date: Thu Jul 2 17:30:59 2015 >>> New Revision: 285050 >>> URL: https://svnweb.freebsd.org/changeset/base/285050 >>> >>> Log: >>> When passwd or group information is changed (by pw, vipw, chpass, ...) >>> temporary file is created and then a rename() call move it to official file. >>> This operation didn't have any check to make sure data was written to disk >>> and if a power cycle happens system could end up with a 0 length passwd >>> or group database. >>> >>> There is a pfSense bug with more infor about it: >>> >>> https://redmine.pfsense.org/issues/4523 >>> >>> The following changes were made to protect passwd and group operations: >>> >>> * lib/libutil/gr_util.c: >>> - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file >>> - After rename(), fsync() call on directory for faster result >>> >>> * lib/libutil/pw_util.c >>> - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file >>> >>> * usr.sbin/pwd_mkdb/pwd_mkdb.c >>> - Added O_SYNC flag on dbopen() calls >>> - After rename(), fsync() call on directory for faster result >>> >>> * lib/libutil/pw_util.3 >>> - pw_lock() returns a file descriptor to master password file on success >>> >>> Differential Revision: https://reviews.freebsd.org/D2978 >>> Approved by: bapt >>> Sponsored by: Netgate >>> >>> Modified: >>> head/lib/libutil/gr_util.c >>> head/lib/libutil/pw_util.3 >>> head/lib/libutil/pw_util.c >>> head/usr.sbin/pwd_mkdb/pwd_mkdb.c >> >> This change is making certain pw operations very slow on ZFS root >> systems. The problem is that when you open a file with O_SYNC, every >> single write(2) call turns into a zil_commit on ZFS, which is fairly >> expensive. Did you consider using fsync(2) on the temporary files >> instead of opening them with O_SYNC? I just tried that now, and I see >> a considerable speedup when running the tests in >> /usr/tests/usr.sbin/pw: >> >> Using O_SYNC, as CURRENT does: 4 minutes 5.2 seconds >> No synchronous operations at all: 49.5 seconds >> Using fsync(2): 56.0 seconds > > pwd_mkdb was fixed back in February with by switching to fsync() in > r295925. It looks like libutil was not fixed, though. https://reviews.freebsd.org/D8319