From owner-freebsd-arch@FreeBSD.ORG Mon Oct 6 19:33:17 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE0F816A4B3 for ; Mon, 6 Oct 2003 19:33:17 -0700 (PDT) Received: from smtp.mho.com (smtp.mho.net [64.58.4.6]) by mx1.FreeBSD.org (Postfix) with SMTP id 9878F43FDF for ; Mon, 6 Oct 2003 19:33:16 -0700 (PDT) (envelope-from scottl@freebsd.org) Received: (qmail 99772 invoked by uid 1002); 7 Oct 2003 02:33:15 -0000 Received: from unknown (HELO freebsd.org) (64.58.1.252) by smtp.mho.net with SMTP; 7 Oct 2003 02:33:15 -0000 Message-ID: <3F8225EB.5020902@freebsd.org> Date: Mon, 06 Oct 2003 20:33:15 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3) Gecko/20030425 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Don Lewis References: <200310070049.h970mvN1052990@gw.catspoiler.org> In-Reply-To: <200310070049.h970mvN1052990@gw.catspoiler.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: arch@FreeBSD.org cc: phk@phk.freebsd.dk cc: wollman@khavrinen.lcs.mit.edu Subject: Re: Alignment of disk-I/O from userland. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 02:33:17 -0000 Don Lewis wrote: > On 7 Oct, Poul-Henning Kamp wrote: > > >>But I also just realized a complication I had not thought of earlier, >>and which may modify our thinking further: This is an issue for >>all physread()/physwrite() drivers, not just disks. >> >>In other words, if I want to write to 1MB blocks to a SCSI tape, >>and I don't align my in memory buffer sufficiently for the hardware, >>busdma would have to allocate 1MB of memory (it may _possibly_ be >>able to do so as disjunct pages rather than consequtively) and copy >>the entire request over. >> >>For disks we can chop the request at sector boundaries or multiple >>thereoff and deal with it that way, but we don't have that option >>for scsi_sa or even scsi_pt devices. >> >>Currently we impose a 128k upper limit on I/O requests, but we have >>already more or less agreed that needs to grow into the 4-16MB range >>soon. > > > There might be another reason to other than alignment to copy the data. > What are the limits on the size of the scatter-gather lists that are > supported by the DMA engines of commonly available controllers? It is > unlikely that the user's buffer will be in contiguous memory even if it > is properly aligned. It might be necessary to copy the the memory into > some number of buffers, where each buffer is in contiguous physical > memory, so that reasonable size I/O requests can be supported on > commonly available controllers. A 4MB request might require 1K > scatter-gather list entries on a machine with a 4K page size. Ick ... > This again is a job for busdma. Again, it isn't implemented, but the API details are already defined and waiting for the back-end code. With more and more drivers switching to busdma, it really makes much more sense to finish the implementation there rather than invent several new implementations elsewhere. Scott