From owner-svn-src-all@freebsd.org Thu Oct 13 23:06:21 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EE5CC1014B; Thu, 13 Oct 2016 23:06:21 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D484F1890; Thu, 13 Oct 2016 23:06:20 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u9DN6Aod023917; Thu, 13 Oct 2016 16:06:14 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201610132306.u9DN6Aod023917@gw.catspoiler.org> Date: Thu, 13 Oct 2016 16:06:10 -0700 (PDT) From: Don Lewis Subject: Re: svn commit: r285050 - in head: lib/libutil usr.sbin/pwd_mkdb To: asomers@freebsd.org cc: garga@freebsd.org, src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 23:06:21 -0000 On 13 Oct, Alan Somers wrote: > On Thu, Jul 2, 2015 at 11:31 AM, Renato Botelho wrote: >> Author: garga (ports committer) >> Date: Thu Jul 2 17:30:59 2015 >> New Revision: 285050 >> URL: https://svnweb.freebsd.org/changeset/base/285050 >> >> Log: >> When passwd or group information is changed (by pw, vipw, chpass, ...) >> temporary file is created and then a rename() call move it to official file. >> This operation didn't have any check to make sure data was written to disk >> and if a power cycle happens system could end up with a 0 length passwd >> or group database. >> >> There is a pfSense bug with more infor about it: >> >> https://redmine.pfsense.org/issues/4523 >> >> The following changes were made to protect passwd and group operations: >> >> * lib/libutil/gr_util.c: >> - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file >> - After rename(), fsync() call on directory for faster result >> >> * lib/libutil/pw_util.c >> - Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file >> >> * usr.sbin/pwd_mkdb/pwd_mkdb.c >> - Added O_SYNC flag on dbopen() calls >> - After rename(), fsync() call on directory for faster result >> >> * lib/libutil/pw_util.3 >> - pw_lock() returns a file descriptor to master password file on success >> >> Differential Revision: https://reviews.freebsd.org/D2978 >> Approved by: bapt >> Sponsored by: Netgate >> >> Modified: >> head/lib/libutil/gr_util.c >> head/lib/libutil/pw_util.3 >> head/lib/libutil/pw_util.c >> head/usr.sbin/pwd_mkdb/pwd_mkdb.c > > This change is making certain pw operations very slow on ZFS root > systems. The problem is that when you open a file with O_SYNC, every > single write(2) call turns into a zil_commit on ZFS, which is fairly > expensive. Did you consider using fsync(2) on the temporary files > instead of opening them with O_SYNC? I just tried that now, and I see > a considerable speedup when running the tests in > /usr/tests/usr.sbin/pw: > > Using O_SYNC, as CURRENT does: 4 minutes 5.2 seconds > No synchronous operations at all: 49.5 seconds > Using fsync(2): 56.0 seconds pwd_mkdb was fixed back in February with by switching to fsync() in r295925. It looks like libutil was not fixed, though.