From owner-freebsd-current@FreeBSD.ORG Mon Feb 17 13:07:59 2014 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99A0E562; Mon, 17 Feb 2014 13:07:59 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 320C91E25; Mon, 17 Feb 2014 13:07:59 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so23396189qcq.22 for ; Mon, 17 Feb 2014 05:07:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=jkEKxMsz1JDaKEBV0I+VwQWk3eZMeyl1dYoT9Tdu4YU=; b=Gws4HnpyteVhycdKk1ZVowa04jLT4Cnrcf/yXkfMx5wkOBSWrveCMcs2lfQ3AR8+Wg h/S5WdAcw0rlGAG1oFHy31m0FA/u5z/RdRS9qpIZIdKjJ4JCJy7SY/+L62Ot8xVv4Mqe rDnjHGJfzXNC0tTvF3HOmcMfOYrUCuO02aDxzuElY7WMogHwJ239sw6st8/JmabvC0yH tFVdh9ZD2w1UvzhkP0GF1n6LH2A4OQa1u/HDgBuQ72ebhzxXYqsrKhCyxLQeBxe8aJ26 SxuCDLroAid1TWoKP2duHvtUS8O2nhoZwFh20jIQPgYAQS+GG9h8Q5PMpFEWSxxThfTw KFkw== MIME-Version: 1.0 X-Received: by 10.224.16.72 with SMTP id n8mr2296563qaa.76.1392642478420; Mon, 17 Feb 2014 05:07:58 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Mon, 17 Feb 2014 05:07:58 -0800 (PST) In-Reply-To: <20140217113253.GM26785@glebius.int.ru> References: <20140217111635.GL26785@glebius.int.ru> <20140217113253.GM26785@glebius.int.ru> Date: Mon, 17 Feb 2014 05:07:58 -0800 X-Google-Sender-Auth: jQFXHYxv-oXS1Zec9tnh3fS_1uc Message-ID: Subject: Re: [CFT] new sendfile(2) From: Adrian Chadd To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: David Chisnall , "current@freebsd.org" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Feb 2014 13:07:59 -0000 On 17 February 2014 03:32, Gleb Smirnoff wrote: > On Mon, Feb 17, 2014 at 11:24:21AM +0000, David Chisnall wrote: > D> > Now for the new sendfile. The core idea is that sendfile() > D> > schedules the I/O, but doesn't wait for it to complete. It > D> > returns immediately to the process, and I/O completion is > D> > processed in kernel context. Unlike aio(4), no additional > D> > threads in kernel are created. The new sendfile is a drop-in > D> > replacement for the old one. Applications (like nginx) doesn't > D> > need recompile, neither configuration change. The SF_NODISKIO is > D> > ignored. > D> > D> Doesn't this introduce a race? If I do a sendfile now, then I am at l= iberty to modify the underlying file as soon as it returns. With this versi= on, I not only am not free to modify the file, I have no notification that = it is finished so I can't ever safely use this call on a file that I might = eventually modify. > D> > D> Wouldn't it be better to provide an aio_sendfile() that would deliver = completion notifications via the normal aio mechanism? > > This race actually exists with the classical sendfile and existed > ever since. While pages are in socket buffer, and not yet sent > out wire, any modification to them would affect data sent. > > To know when you can modify data you need to use SF_SYNC flag. > The flag should work with new sendfile as well, effectively > making it blocking on socket. I haven't tested this, but > should work. > > Adrian have also committed the kqueue notification of SF_SYNC > event. This should work in combination with new sendfile, but > that hasn't been tested yet. This is going to be tested soon, > since we need this functionality. Yup, that's exactly why I committed the sendfile kqueue completion notification stuff. It's main caveat is that it requires that the ethernet driver immediately free the mbuf upon TX completion - which the chelsio driver doesn't yet do. That has to be fixed for the notification completion stuff to behave well under lower traffic loads. -a