From owner-svn-src-head@freebsd.org Tue Dec 11 17:55:01 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1A0D130C797; Tue, 11 Dec 2018 17:55:00 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65DBF83588; Tue, 11 Dec 2018 17:55:00 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 6D94A10A87D; Tue, 11 Dec 2018 12:54:59 -0500 (EST) Subject: Re: svn commit: r341803 - head/libexec/rc To: Devin Teske Cc: Conrad Meyer , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201812110138.wBB1cp1p006660@repo.freebsd.org> <2a76b295-b2da-3015-c201-dbe0ec63ca5a@FreeBSD.org> <98481565-CDD7-4301-B86B-072D5B984AF7@FreeBSD.org> From: John Baldwin Openpgp: preference=signencrypt Autocrypt: addr=jhb@FreeBSD.org; keydata= xsDiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg80eSm9obiBCYWxk d2luIDxqb2huQGJhbGR3aW4uY3g+wmMEExECACMCGwMGCwkIBwMCBBUCCAMEFgIDAQIeAQIX gAUCRND5wwIZAQAKCRBy3lIGd+N/BNLXAJ9KIb6teuDL1W+FkCgvv+y8PxKTkACeIUfbn3sl cueBzqTcf09idwa8YTbOwU0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Ds gnr31AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh +GojXlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cM SOrHYUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOF QVHOEVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq 1tqzhltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZ TwtXsNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m 7Z164yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioI AjjHaIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbU KWwxQ4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjH uW/CSQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZN wwCfafMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: Date: Tue, 11 Dec 2018 09:54:58 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Thunderbird/60.3.3 MIME-Version: 1.0 In-Reply-To: <98481565-CDD7-4301-B86B-072D5B984AF7@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Tue, 11 Dec 2018 12:54:59 -0500 (EST) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-Rspamd-Queue-Id: 65DBF83588 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-0.50 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_SHORT(-0.50)[-0.499,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2018 17:55:01 -0000 On 12/11/18 9:40 AM, Devin Teske wrote: > > >> On Dec 11, 2018, at 9:23 AM, John Baldwin > wrote: >> >> On 12/10/18 5:38 PM, Conrad Meyer wrote: >>> Author: cem >>> Date: Tue Dec 11 01:38:50 2018 >>> New Revision: 341803 >>> URL: https://svnweb.freebsd.org/changeset/base/341803 >>> >>> Log: >>>  rc.subr: Implement list_vars without using 'read' >>> >>>  'read' pessimistically read(2)s one byte at a time, which can be quite >>>  silly for large environments in slow emulators. >>> >>>  In my boring user environment, truss shows that the number of read() >>>  syscalls to source rc.subr and invoke list_vars is reduced by something like >>>  3400 to 60.  ministat(1) shows a significant time difference of about -71% >>>  for my environment. >>> >>>  Suggested by:jilles >>>  Discussed with:dteske, jhb, jilles >>>  Differential Revision:https://reviews.freebsd.org/D18481 >> >> For some background, one my colleagues reported that it was taking hours in >> (an admittedly slow) CPU simulator to get through '/etc/rc.d/netif start'. >> I ended up running that script under truss in a RISC-V qemu machine.  The >> entire run took 212 seconds (truss did slow it down quite a bit).  Of that >> 212 seconds, the read side of each list_vars invocation took ~25.5 seconds, >> and with lo0 and vtnet0 there were 8 list_vars invocations, so 204 out of >> the 212 seconds were spent in the single-byte read() syscalls in 'while read'. >> >> Even on qemu without truss during bootup 'netif start' took a couple of >> seconds (long enough to get 2-3 Ctrl-T's in) before this change and is now >> similar to bare metal with the change.  list_vars is rarely used outside of >> 'netif', so it probably doesn't make a measurable difference on bare metal. >> > > Thank you for the background which was lost by the time I got to the phab. > > I can't help but ask though,... > > If it was noticed that read(2) processes the stream one byte at a time, > why not just optimize read(2)? > > I'm afraid of the prospect of having to hunt down every instance of while-read, > but if we can fix the underlying read(2) inefficiency then we make while-read OK. It's a system call. A CPU emulator has to do a lot of work for a system call because it involves two mode switches (user -> kernel and back again). You can't "fix" that as it's just a part of the CPU architecture. There's a reason that stdio uses buffering by default, it's because system calls have overhead. The 'read' builtin in sh can't use buffering, so it is always going to be inefficient. -- John Baldwin