From owner-freebsd-arch@FreeBSD.ORG  Mon Oct  6 17:49:09 2003
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: freebsd-arch@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 1135616A4B3; Mon,  6 Oct 2003 17:49:09 -0700 (PDT)
Received: from gw.catspoiler.org (217-ip-163.nccn.net [209.79.217.163])
	by mx1.FreeBSD.org (Postfix) with ESMTP
	id E8D3643FBD; Mon,  6 Oct 2003 17:49:07 -0700 (PDT)
	(envelope-from truckman@FreeBSD.org)
Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2])
	by gw.catspoiler.org (8.12.9/8.12.9) with ESMTP id h970mvN1052990;
	Mon, 6 Oct 2003 17:49:01 -0700 (PDT)
	(envelope-from truckman@FreeBSD.org)
Message-Id: <200310070049.h970mvN1052990@gw.catspoiler.org>
Date: Mon, 6 Oct 2003 17:48:57 -0700 (PDT)
From: Don Lewis <truckman@FreeBSD.org>
To: phk@phk.freebsd.dk
In-Reply-To: <27374.1065481871@critter.freebsd.dk>
MIME-Version: 1.0
Content-Type: TEXT/plain; charset=us-ascii
cc: arch@FreeBSD.org
cc: scottl@FreeBSD.org
cc: wollman@khavrinen.lcs.mit.edu
Subject: Re: Alignment of disk-I/O from userland. 
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussion related to FreeBSD architecture
	<freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Oct 2003 00:49:09 -0000

On  7 Oct, Poul-Henning Kamp wrote:

> But I also just realized a complication I had not thought of earlier,
> and which may modify our thinking further:  This is an issue for
> all physread()/physwrite() drivers, not just disks.
> 
> In other words, if I want to write to 1MB blocks to a SCSI tape,
> and I don't align my in memory buffer sufficiently for the hardware,
> busdma would have to allocate 1MB of memory (it may _possibly_ be
> able to do so as disjunct pages rather than consequtively) and copy
> the entire request over.
> 
> For disks we can chop the request at sector boundaries or multiple
> thereoff and deal with it that way, but we don't have that option
> for scsi_sa or even scsi_pt devices.
> 
> Currently we impose a 128k upper limit on I/O requests, but we have
> already more or less agreed that needs to grow into the 4-16MB range
> soon.

There might be another reason to other than alignment to copy the data.
What are the limits on the size of the scatter-gather lists that are
supported by the DMA engines of commonly available controllers?  It is
unlikely that the user's buffer will be in contiguous memory even if it
is properly aligned.  It might be necessary to copy the the memory into
some number of buffers, where each buffer is in contiguous physical
memory, so that reasonable size I/O requests can be supported on
commonly available controllers.  A 4MB request might require 1K
scatter-gather list entries on a machine with a 4K page size.   Ick ...