Date: Tue, 11 Dec 2018 11:57:55 -0800 From: Conrad Meyer <cem@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: src-committers <src-committers@freebsd.org>, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r341803 - head/libexec/rc Message-ID: <CAG6CVpVtWL448-pxW9EK7SEe7Cfk1mkCvPNmtE=YuFWM3Bn0tA@mail.gmail.com> In-Reply-To: <CANCZdfqtyvxCBwdwDQ4Raeven2LmaPd3C-c---pVFhHDWBKfxA@mail.gmail.com> References: <201812110138.wBB1cp1p006660@repo.freebsd.org> <2a76b295-b2da-3015-c201-dbe0ec63ca5a@FreeBSD.org> <98481565-CDD7-4301-B86B-072D5B984AF7@FreeBSD.org> <dafbcc18-146f-2e4f-e1e9-346d7c05b096@FreeBSD.org> <CANCZdfqtyvxCBwdwDQ4Raeven2LmaPd3C-c---pVFhHDWBKfxA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 11, 2018 at 10:04 AM Warner Losh <imp@bsdimp.com> wrote: > On Tue, Dec 11, 2018, 9:55 AM John Baldwin <jhb@freebsd.org wrote: >> >> The 'read' builtin in sh can't use buffering, so it is always going to b= e slow > > It can't use it because of pipes. The example from the parts of this that= was on IRC was basically: > > foo | (read bar; baz) > > Which reads one line into the bar variable and then sends the rest to the= bar command. It can't trivially, but it's not impossible. sh could play games and buffer its own use of stdin, and then open a fresh pipe for stdin of subsequent non-builtins, writing out unused portions of the buffer.[A] Some other alternatives that would require kernel support but are things we've talked about doing in the kernel before anyway: * If we had something like eBPF programs attached to IO, maybe sh's read built-in could push a small eBPF program into the kernel that determined how many bytes could be read from the pipe in a single syscall without reading too far. It's fairly trivial. Simply returning a number of bytes up to and including the first '\n' would be a fine, if sometimes conservative amount. (Input lines can be continued with a trailing backslash, except in -r mode, but as a first-cut approximation, reading-until-newline is probably good enough.)[B] * Heck, even just a read_until_newline(2) syscall would work and probably be more broadly useful than just sh(1). I don't think it passes the sniff test =E2=80=94 not general enough, and probably not someth= ing you want beginners stumbling across instead of fgets(3) =E2=80=94 but it'd = be fine, and there are other pipe-abusing programs that care about reading ASCII text lines without overconsuming input than just sh(1).[C] * If we had something like Linux's tee(2) system call (which is as it sounds =E2=80=94 tee(1) for pipes), sh(1)'s read built-in could tee(2) for buffering without impacting stdin, and read(2) stdin only when it knew how many bytes were consumed (or when the pipe buffer became full).[D] I suspect (C) would be the easiest to implement correctly, followed by (D). (B) is requires some architectural design and bikeshedding and the details on the kernel side are tricky. (A) would be a little tricky and probably require extensive changes to sh(1) itself, which is a risk to the base system. But it would not impact the kernel. Is there any interest in a tee(2)-like syscall? Thanks, Conrad
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpVtWL448-pxW9EK7SEe7Cfk1mkCvPNmtE=YuFWM3Bn0tA>