From owner-freebsd-current@FreeBSD.ORG  Fri Jul 12 02:04:28 2013
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by hub.freebsd.org (Postfix) with ESMTP id 6ACCED25;
 Fri, 12 Jul 2013 02:04:28 +0000 (UTC)
 (envelope-from davidxu@freebsd.org)
Received: from freefall.freebsd.org (freefall.freebsd.org
 [IPv6:2001:1900:2254:206c::16:87])
 by mx1.freebsd.org (Postfix) with ESMTP id 5C4C111EC;
 Fri, 12 Jul 2013 02:04:28 +0000 (UTC)
Received: from xyf.my.dom (localhost [127.0.0.1])
 by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r6C24QgH053166;
 Fri, 12 Jul 2013 02:04:27 GMT (envelope-from davidxu@freebsd.org)
Message-ID: <51DF6450.7050204@freebsd.org>
Date: Fri, 12 Jul 2013 10:05:04 +0800
From: David Xu <davidxu@freebsd.org>
User-Agent: Mozilla/5.0 (X11; FreeBSD i386;
 rv:17.0) Gecko/20130416 Thunderbird/17.0.5
MIME-Version: 1.0
To: Konstantin Belousov <kostikbel@gmail.com>
Subject: Re: hacking - aio_sendfile()
References: <CAJ-Vmo=icr6bda+WMNUarc3WbdqJ+MdauX6kByxxdTx8oSovBg@mail.gmail.com>
 <20130711061753.GK91021@kib.kiev.ua>
In-Reply-To: <20130711061753.GK91021@kib.kiev.ua>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Adrian Chadd <adrian@freebsd.org>,
 freebsd-current <freebsd-current@freebsd.org>
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Jul 2013 02:04:28 -0000

On 2013/07/11 14:17, Konstantin Belousov wrote:
> On Wed, Jul 10, 2013 at 04:36:23PM -0700, Adrian Chadd wrote:
>> Hiya,
>>
>> I've started writing an aio_sendfile() syscall.
>>
>> http://people.freebsd.org/~adrian/ath/20130710-aio-sendfile-3.diff
>>
>> Yes, the diff is against -HEAD and not stable/9.
>>
>> It's totally horrible, hackish and likely bad. I've only done some
>> very, very basic testing to ensure it actually works; i haven't at all
>> stress tested it out yet. It's also very naive - I'm not at all doing
>> any checks to see whether I can short-cut to do the aio there and
>> then; I'm always queuing the sendfile() op through the worker threads.
>> That's likely stupid and inefficient in a lot of cases, but it at
>> least gets the syscall up and working.
> Yes, it is naive, but for different reason.
>
> The kern_sendfile() is synchronous function, it only completes after
> the other end of the network communication allows it. This means
> that calling kern_sendfile() from the aio thread blocks the thread
> indefinitely by unbounded sleep.
>
> Your implementation easily causes exhaustion of the aio thread pool,
> blocking the whole aio subsystem. It is known that our aio does not work
> for sockets for the same reason. I object against adding more code with
> the same defect.
>
> Proper route seems to rewrite aio for sockets using the upcalls.  The same
> should be done for sendfile, if sendfile is given aio flavor.
>

Yes, current aio implementation is horrible, it only works for fast 
disk I/O, I think the thread pool size is enough to saturate disks,
but for socket or pipe I/O, it does not work well, the thread pool is
too easy to be exhausted.

I even think the support for socket and pipe in aio code should be
cut and thrown away, because you can always use kqueue + non-blocking
I/O.