From owner-freebsd-hackers@freebsd.org Sun Sep 20 16:04:10 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AEF763E6A0E for ; Sun, 20 Sep 2020 16:04:10 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BvXST598Mz3V24 for ; Sun, 20 Sep 2020 16:04:09 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f48.google.com with SMTP id n61so10139138ota.10 for ; Sun, 20 Sep 2020 09:04:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=N7a7M4d7Cq3PaaABjt3bMm+CTTTplS5I+/P0bqN0IGc=; b=NgJVA2znW+qOhOUgoMTfhct07hfc8jLd81kfti8qw+yFokN5Omd/CmODZnAvaRIg8k A588XfmK7yqpvSH0OxWEBFivXWTL/CVvWIP1jUSi8dQNnsBFaoAs5CbGIq0Z4ceMNOKY EO5uCEkKDP8B2CcbrlzUl6MDn4LHU0y6+FQGVg3Gnyay5DjhP4Qwr/jbqA2k3btMjYf2 wlzMDIRhQlb2rP64S3eq+VCYMi2DTekFruFqZFwy9eix+lcUeBbVUfptseJBUcicMxlM u/AoVeF7RWer236i9XBqgjh75wyToEKlxuRefzHPFpv777ogDDqvIQS9DJaXtTbfC2xf liOw== X-Gm-Message-State: AOAM5309h0WKD7oEeMgyzvDyg1WudBMV3iwK9zOsC15lEaEd/v0/KSUD JeLD8rUHRK+S8mw3iPWNy2mNbRD7a8GeLb6lsuo= X-Google-Smtp-Source: ABdhPJxqNX+Km1WvwI0eXAYYS/X8XnmMZCbrBUDx6kfqXpJPhLt0Jx4fPRgQ/O/MnbDXhfeFZNIZsRMGaAxtIfUbk30= X-Received: by 2002:a05:6830:1286:: with SMTP id z6mr27893919otp.291.1600617848599; Sun, 20 Sep 2020 09:04:08 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sun, 20 Sep 2020 10:03:57 -0600 Message-ID: Subject: Re: RFC: copy_file_range(3) To: Rick Macklem Cc: FreeBSD Hackers X-Rspamd-Queue-Id: 4BvXST598Mz3V24 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.48 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.89 / 15.00]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[209.85.210.48:from]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_LONG(-1.08)[-1.079]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.79)[-0.791]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.48:from]; NEURAL_HAM_MEDIUM(-1.01)[-1.015]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2020 16:04:10 -0000 On Sun, Sep 20, 2020 at 9:58 AM Rick Macklem wrote: > Alan Somers wrote: > >copy_file_range(2) is nifty, but it has a few sharp edges: > >1) Certain file systems don't support it, necessitating a write/read based > >fallback > >2) It doesn't handle sparse files as well as SEEK_HOLE/SEEK_DATA > >3) It's slightly tricky to both efficiently deal with holes and also > >promptly respond to signals > > > >These problems aren't terribly hard, but it seems to me like most > >applications that use copy_file_range would share the exact same > >solutions. In particular, I'm thinking about cp(1), dd(1), and > >install(8). Those three could benefit from sharing a userland wrapper > that > >handles the above problems. > > > >Should we add such a wrapper to libc? If so, what should it be called, > and > >should it be public or just private to /usr/src ? > There has been a discussion on src-committers which I suggested should > be taken to a public mailing list. > > The basic question is... > Whether or not the copy_file_range(2) syscall should be compatible with > the Linux one. > When I did the syscall, I tried to make it Linux-compatible, arguing that > Linux is now a de-facto standard. > The Linux syscall only works on regular files, which is why Alan's patch > for > cp required a "fallback to the old way" for VCHR files like /dev/null. > > He is considering a wrapper in libc to provide FreeBSD specific semantics, > which I have no problem with, so long as the naming and man page make > it clear that it is not compatible with the Linux syscall. > (Personally, I'd prefer a wrapper in libc to making the actual syscall > non-Linux > compatible, but that is just mho.) > > Hopefully this helps clarify what Alan is asking, rick > I don't think the two questions are equivalent. I think that copy_file_range(2) ought to work on character devices. Separately, even it does, I think a userland wrapper would still be useful. It would still be able to handle sparse files more efficiently than the kernel-based vn_generic_copy_file_range. -Alan