From owner-svn-src-all@freebsd.org Wed Sep 23 17:24:04 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BA7273FFA7A for ; Wed, 23 Sep 2020 17:24:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qk1-x743.google.com (mail-qk1-x743.google.com [IPv6:2607:f8b0:4864:20::743]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BxQ5J196Vz3f9m for ; Wed, 23 Sep 2020 17:24:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qk1-x743.google.com with SMTP id g72so441102qke.8 for ; Wed, 23 Sep 2020 10:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=UPabJJuKD9GRQanq1ky0iZoB5DN1Vc+HSm4+07QWeds=; b=IYOgZAED+/XdwAsA4JW0BtDzp9qX4+uSgOM5Z4KNbIwvSTPrlEqiFlDT+Yloc0BEpt jV9sFmDDbAwHd+umpUfjDdUHTvpbEVkqPZ6dIBF1qasHsTGBPShdj7XRi0Ec+H8cFYAy omuRLl/dS+/ux9xMaMtrkgp0X5/S9C7qyGiK1N6bmgZSMvrWz4kGYi1gcGnf8Xsv107T jEXE3c+Jm/09vn/ZaVQvcLoN85DLGrAtOvqPSS16f52+HdcjFalfBNVwE22hgOlDdj4t wj6yVAKX+UvU7cW8tuQurwDpV9MuDAaVVIL1iLIY0TaPAqV3S2s+7ZfaeLOwTL8pebkK RPKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=UPabJJuKD9GRQanq1ky0iZoB5DN1Vc+HSm4+07QWeds=; b=tmGxX0qxi9Z9BoBRBrkHHbk2dES1ryTQQ6Xfcr6f2WB93KgP3pGR3OdYG/jDFnVmqq 3+LsqZJsQ18y0M6oBFXuAeSInDvV4NFm4vLAdwSss4MUFFiC0uLDqN5bdzzz8UUpJurF yw8Z2xqx6b9c61a94gyV/SkNc4M56h3QCdyJj7Ctrd2gNg4nyQc3WYjyjDgRBrpT1doL DsjETyLoKBtLz8ySLwpkxghI8xXwHGOIDlBG3wQBClR7Akt+ZHoD51kLirp/dDpbKGiA vOM/0rF8zYa4JWmpqn4FvX+mTpCT9o3G3VNuR/kyC0NU1Iw4qLzT8LhBrzoGbDSZyRa6 /YLg== X-Gm-Message-State: AOAM53025uW+EPYiO+gDWv+CNWNBaRiC/7n0lM5eA/D8ATTApPobtcAk XX2/kxdDBikJ7sw5Grdg8xPQu6fLj+GzS4+5EiIhmQ== X-Google-Smtp-Source: ABdhPJw+aTYgLZI+mPuPnZtDDWD4vIz4WuACEYNa/UrybmrCduhLzqsqlkVR49aDleBRSUSjO9AMe1nU7435RsdKdG0= X-Received: by 2002:a05:620a:1583:: with SMTP id d3mr856442qkk.495.1600881843024; Wed, 23 Sep 2020 10:24:03 -0700 (PDT) MIME-Version: 1.0 References: <202009231656.08NGujEs042900@gndrsh.dnsmgr.net> In-Reply-To: <202009231656.08NGujEs042900@gndrsh.dnsmgr.net> From: Warner Losh Date: Wed, 23 Sep 2020 11:23:51 -0600 Message-ID: Subject: Re: svn commit: r365643 - head/bin/cp To: "Rodney W. Grimes" Cc: Kyle Evans , Alan Somers , Mateusz Guzik , src-committers , svn-src-all , svn-src-head X-Rspamd-Queue-Id: 4BxQ5J196Vz3f9m X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=IYOgZAED; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::743) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-1.72 / 15.00]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[svn-src-all]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-0.86)[-0.858]; NEURAL_HAM_LONG(-0.94)[-0.943]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[svn-src-all@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; NEURAL_SPAM_SHORT(0.08)[0.082]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; RCPT_COUNT_SEVEN(0.00)[7]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::743:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2020 17:24:04 -0000 On Wed, Sep 23, 2020, 10:56 AM Rodney W. Grimes wrote: > > cp is already fixed, people are still feeling the fallout of being > > within those revisions and needing to bootstrap their own cp. We can > > reduce the number of components these invocations rely on trivially to > > shell built-in mechanics, why not do so? > > I would even go further, I would like to see the dependency on > /dev/null removed from the build, and the boot process. > A worthy goal, but one I'm afraid is out of our reach. A quick grep shows just over 200 instances of /dev/null in the Makefile and mk file (800 if you don't filter Makefile.in and Makefile.am). Maybe a third of those are due to tests and other false positives. It would be quite the effort to eliminate them all. And /dev/tty and /dev/zero likely will be troublesome too, as they are used by running programs. and how would you throw away output you know is bad / bogus without /dev/null? >From the build because it means I would no longer have to > mount /dev in my chroots, and from the boot because I > hate to say it, but we often scribble in /dev before > devfs is mounted and if you look at root file systems > mounted on other systems you well often find a /dev/null > FILE that got created during the boot process from a >/dev/null > before devfs was mounted. > But for this issue, we're not mounting devfs early enough. We should fix that. Removing /dev/null from the boot process likely is never going to happen because we use it all over the place to discard output... There's ~200 instances of it in the boot rc scripts, so getting rid of it there would also be quite the effort, with the same question. Warner > > > On Tue, Sep 22, 2020 at 4:40 PM Warner Losh wrote: > > > > > > So why do we need a workaround at all? cp /dev/null has been fixed, > and that's way more important to get right. > > > > > > I don't want to paper-over issues with this at all, though if we use > the host's (now broken) cp, I suppose that might be OK in the short term. > If that's the case, then maybe this is OK. > > > > > > Otherwise, I'd strongly prefer we fix cp. > > > > > > Warner > > > > > > On Tue, Sep 22, 2020 at 3:31 PM Alan Somers > wrote: > > >> > > >> +1. > > >> > > >> On Tue, Sep 22, 2020 at 3:27 PM Kyle Evans > wrote: > > >>> > > >>> I'm running a build at the suggestion of mjg to confirm there aren't > > >>> any others hiding that can be converted, and I will commit after I've > > >>> verified that this is it. > > >>> > > >>> On Tue, Sep 22, 2020 at 4:02 PM Alan Somers > wrote: > > >>> > > > >>> > LGTM. > > >>> > > > >>> > On Tue, Sep 22, 2020 at 2:59 PM Kyle Evans > wrote: > > >>> >> > > >>> >> Perhaps: > > >>> >> > > >>> >> diff --git a/stand/i386/zfsboot/Makefile > b/stand/i386/zfsboot/Makefile > > >>> >> index ff315abc0ef..7e362b43a39 100644 > > >>> >> --- a/stand/i386/zfsboot/Makefile > > >>> >> +++ b/stand/i386/zfsboot/Makefile > > >>> >> @@ -81,7 +81,7 @@ zfsboot.ld: zfsboot.ldr zfsboot.bin ${BTXKERN} > > >>> >> -o ${.TARGET} -P 1 zfsboot.bin > > >>> >> > > >>> >> zfsboot.ldr: > > >>> >> - cp /dev/null ${.TARGET} > > >>> >> + :> ${.TARGET} > > >>> >> > > >>> >> zfsboot.bin: zfsboot.out > > >>> >> ${OBJCOPY} -S -O binary zfsboot.out ${.TARGET} > > >>> >> diff --git a/stand/libsa/Makefile b/stand/libsa/Makefile > > >>> >> index effece9e01b..63cd46a9c54 100644 > > >>> >> --- a/stand/libsa/Makefile > > >>> >> +++ b/stand/libsa/Makefile > > >>> >> @@ -122,7 +122,7 @@ beforedepend: > > >>> >> ln -sf ${SRCTOP}/include/arpa/inet.h arpa/inet.h; \ > > >>> >> ln -sf ${SRCTOP}/include/arpa/tftp.h arpa/tftp.h; \ > > >>> >> for i in _time.h _strings.h _string.h; do \ > > >>> >> - [ -f xlocale/$$i ] || cp /dev/null xlocale/$$i; \ > > >>> >> + [ -f xlocale/$$i ] || :> xlocale/$$i; \ > > >>> >> done; \ > > >>> >> for i in ${STAND_H_INC}; do \ > > >>> >> ln -sf ${SASRC}/stand.h $$i; \ > > >>> >> > > >>> >> > > >>> >> On Tue, Sep 22, 2020 at 3:58 PM Alan Somers > wrote: > > >>> >> > > > >>> >> > Looks like two places in stand. Is there any reason why > Mateusz's suggestion wouldn't work? > > >>> >> > > > >>> >> > > rg -g Makefile 'cp.*/dev/null' > > >>> >> > stand/libsa/Makefile > > >>> >> > 125: [ -f xlocale/$$i ] || cp /dev/null xlocale/$$i; \ > > >>> >> > > > >>> >> > stand/i386/zfsboot/Makefile > > >>> >> > 82: cp /dev/null ${.TARGET} > > >>> >> > > > >>> >> > On Tue, Sep 22, 2020 at 2:54 PM Mateusz Guzik < > mjguzik@gmail.com> wrote: > > >>> >> >> > > >>> >> >> Can we instead add a workaround to the build tree? > > >>> >> >> > > >>> >> >> Where is cp /dev/null coming from anyway? Perhaps this can be > patched > > >>> >> >> to touch the target file. > > >>> >> >> > > >>> >> >> On 9/22/20, Alan Somers wrote: > > >>> >> >> > On Tue, Sep 22, 2020 at 2:48 PM Kyle Evans < > kevans@freebsd.org> wrote: > > >>> >> >> > > > >>> >> >> >> On Fri, Sep 11, 2020 at 3:49 PM Alan Somers < > asomers@freebsd.org> wrote: > > >>> >> >> >> > > > >>> >> >> >> > Author: asomers > > >>> >> >> >> > Date: Fri Sep 11 20:49:36 2020 > > >>> >> >> >> > New Revision: 365643 > > >>> >> >> >> > URL: https://svnweb.freebsd.org/changeset/base/365643 > > >>> >> >> >> > > > >>> >> >> >> > Log: > > >>> >> >> >> > cp: fall back to read/write if copy_file_range fails > > >>> >> >> >> > > > >>> >> >> >> > Even though copy_file_range has a file-system agnostic > version, it > > >>> >> >> >> still > > >>> >> >> >> > fails on devfs (perhaps because the file descriptor is > non-seekable?) > > >>> >> >> >> In > > >>> >> >> >> > that case, fallback to old-fashioned read/write. Fixes > > >>> >> >> >> > "cp /dev/null /tmp/null" > > >>> >> >> >> > > > >>> >> >> >> > > >>> >> >> >> Hi, > > >>> >> >> >> > > >>> >> >> >> Any objection to adding a quick UPDATING entry for this? > I'm seeing > > >>> >> >> >> occasional reports of this breakage as recent as today on > IRC from > > >>> >> >> >> folks that were a little bit thrown off by this because it > throws up > > >>> >> >> >> fairly far into the build and looks like a stand build > regression > > >>> >> >> >> instead of a cp regression. > > >>> >> >> >> > > >>> >> >> >> Thanks, > > >>> >> >> >> > > >>> >> >> >> Kyle Evans > > >>> >> >> >> > > >>> >> >> > > > >>> >> >> > No objection. Can you suggest the proper wording? > > >>> >> >> > _______________________________________________ > > >>> >> >> > svn-src-all@freebsd.org mailing list > > >>> >> >> > https://lists.freebsd.org/mailman/listinfo/svn-src-all > > >>> >> >> > To unsubscribe, send any mail to " > svn-src-all-unsubscribe@freebsd.org" > > >>> >> >> > > > >>> >> >> > > >>> >> >> > > >>> >> >> -- > > >>> >> >> Mateusz Guzik > > > > -- > Rod Grimes > rgrimes@freebsd.org >