From owner-freebsd-current@FreeBSD.ORG Wed Oct 20 19:10:01 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D298816A4D1 for ; Wed, 20 Oct 2004 19:10:01 +0000 (GMT) Received: from mail.ambrisko.com (adsl-64-174-51-43.dsl.snfc21.pacbell.net [64.174.51.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id A171B43D46 for ; Wed, 20 Oct 2004 19:10:01 +0000 (GMT) (envelope-from ambrisko@ambrisko.com) Received: from server2.ambrisko.com (HELO www.ambrisko.com) (192.168.1.2) by mail.ambrisko.com with ESMTP; 20 Oct 2004 12:10:01 -0700 Received: from ambrisko.com (localhost [127.0.0.1]) by www.ambrisko.com (8.12.9p2/8.12.9) with ESMTP id i9KJA1El017233 for ; Wed, 20 Oct 2004 12:10:01 -0700 (PDT) (envelope-from ambrisko@ambrisko.com) Received: (from ambrisko@localhost) by ambrisko.com (8.12.9p2/8.12.9/Submit) id i9KJA1wQ017232 for current@freebsd.org; Wed, 20 Oct 2004 12:10:01 -0700 (PDT) (envelope-from ambrisko) From: Doug Ambrisko Message-Id: <200410201910.i9KJA1wQ017232@ambrisko.com> To: current@freebsd.org Date: Wed, 20 Oct 2004 12:10:01 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Subject: lio_listio fixes and adding kqueue notification patch X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Oct 2004 19:10:02 -0000 Here's a patch to fix some lio_listio bugs and add kqueue support to it. BTW kqueue & aio_read/write is broken and that seems to have happened during the locking changes. I could use some more testers for the signal causes. http://www.ambrisko.com/doug/listio_kqueue/listio_kqueue.patch The summary is: kqueue changes: - kqueue locking removed/broke knote_remove (transformed into knlist_clear??) which no longer deletes and clean up the the knotes so they are left hanging around and can break the next transfer since they are already triggered etc. - add knlist_delete to restore knote_remove functionality to totally take it of the system. When you do an aio_return (which implies aio_free_entry) the kevent can never trigger again since the context is lost. - hack, remove the mtx_assert in knlist_empty so it can be used to see if there are kqueue's on an aio task. aio (mostly lio_listio) changes: - add kqueue support to lio_listio - fixed semantic race conditions in which tracking operations were not being compared to total number of requests. The existing code compared the finished to the count that the listio task had got submitted so far and not to the actually number of tasks that will be submitted. - aio_physwakeup did not account for jobs being split across the fast path (aio_qphysio) and the work queue (aio_daemon). This would result in jobs claiming to be done before they were. - fix another condition in which we notify the process (via signals or kqueue) the total job is done before moving the final task to the done queue. - make aio_daemon signal notification code paths use process signal so we don't send a kqueue task a signal! This check lives in process_signal so we just use that. - Don't knote or signal based on a single aio task complete if it belongs to listio job and only notify a listio if the job is complete. - Make it a failure to do an aio I/O op on raw disk bigger then the max I/O size of that driver. - use knlist_empty to see if there are aio task. Added some guards to prevent panic due to SLIST ops on an emply list. A review of this would be great with suggestions on things to change so we can get this into the tree. Thanks, Doug A.