From owner-svn-src-all@freebsd.org Tue Dec 11 17:40:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3E32130C03A; Tue, 11 Dec 2018 17:40:55 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from shxd.cx (mail.shxd.cx [64.201.244.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDDDB82A8A; Tue, 11 Dec 2018 17:40:54 +0000 (UTC) (envelope-from devin@shxd.cx) Received: from [76.77.180.168] (port=60367 helo=eskarina.lan) by shxd.cx with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1gWm1W-000C5t-ME; Tue, 11 Dec 2018 09:40:46 -0800 From: Devin Teske Message-Id: <98481565-CDD7-4301-B86B-072D5B984AF7@FreeBSD.org> Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: svn commit: r341803 - head/libexec/rc Date: Tue, 11 Dec 2018 09:40:45 -0800 In-Reply-To: <2a76b295-b2da-3015-c201-dbe0ec63ca5a@FreeBSD.org> Cc: Devin Teske , Conrad Meyer , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org To: John Baldwin References: <201812110138.wBB1cp1p006660@repo.freebsd.org> <2a76b295-b2da-3015-c201-dbe0ec63ca5a@FreeBSD.org> X-Mailer: Apple Mail (2.3445.9.1) Sender: devin@shxd.cx X-Rspamd-Queue-Id: CDDDB82A8A X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[FreeBSD.org]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[6]; IP_SCORE(0.29)[ip: (0.80), ipnet: 64.201.240.0/20(0.40), asn: 36734(0.32), country: US(-0.09)]; NEURAL_SPAM_SHORT(0.19)[0.195,0]; MX_GOOD(-0.01)[mail.shxd.cx]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[dteske@FreeBSD.org,devin@shxd.cx]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:36734, ipnet:64.201.240.0/20, country:US]; FROM_NEQ_ENVFROM(0.00)[dteske@FreeBSD.org,devin@shxd.cx]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2018 17:40:56 -0000 > On Dec 11, 2018, at 9:23 AM, John Baldwin wrote: >=20 > On 12/10/18 5:38 PM, Conrad Meyer wrote: >> Author: cem >> Date: Tue Dec 11 01:38:50 2018 >> New Revision: 341803 >> URL: https://svnweb.freebsd.org/changeset/base/341803 >>=20 >> Log: >> rc.subr: Implement list_vars without using 'read' >>=20 >> 'read' pessimistically read(2)s one byte at a time, which can be = quite >> silly for large environments in slow emulators. >>=20 >> In my boring user environment, truss shows that the number of read() >> syscalls to source rc.subr and invoke list_vars is reduced by = something like >> 3400 to 60. ministat(1) shows a significant time difference of = about -71% >> for my environment. >>=20 >> Suggested by: jilles >> Discussed with: dteske, jhb, jilles >> Differential Revision: https://reviews.freebsd.org/D18481 >=20 > For some background, one my colleagues reported that it was taking = hours in > (an admittedly slow) CPU simulator to get through '/etc/rc.d/netif = start'. > I ended up running that script under truss in a RISC-V qemu machine. = The > entire run took 212 seconds (truss did slow it down quite a bit). Of = that > 212 seconds, the read side of each list_vars invocation took ~25.5 = seconds, > and with lo0 and vtnet0 there were 8 list_vars invocations, so 204 out = of > the 212 seconds were spent in the single-byte read() syscalls in = 'while read'. >=20 > Even on qemu without truss during bootup 'netif start' took a couple = of > seconds (long enough to get 2-3 Ctrl-T's in) before this change and is = now > similar to bare metal with the change. list_vars is rarely used = outside of > 'netif', so it probably doesn't make a measurable difference on bare = metal. >=20 Thank you for the background which was lost by the time I got to the = phab. I can't help but ask though,... If it was noticed that read(2) processes the stream one byte at a time, why not just optimize read(2)? I'm afraid of the prospect of having to hunt down every instance of = while-read, but if we can fix the underlying read(2) inefficiency then we make = while-read OK. --=20 Devin=