From owner-freebsd-stable@FreeBSD.ORG Mon Oct 27 17:10:55 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F38F6760; Mon, 27 Oct 2014 17:10:54 +0000 (UTC) Received: from mithlond.kdm.org (mithlond.kdm.org [70.56.43.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "A1-33714", Issuer "A1-33714" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BC10264E; Mon, 27 Oct 2014 17:10:54 +0000 (UTC) Received: from mithlond.kdm.org (localhost [127.0.0.1]) by mithlond.kdm.org (8.14.9/8.14.9) with ESMTP id s9RHAqiM092448 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 27 Oct 2014 11:10:52 -0600 (MDT) (envelope-from ken@mithlond.kdm.org) Received: (from ken@localhost) by mithlond.kdm.org (8.14.9/8.14.9/Submit) id s9RHAq14092447; Mon, 27 Oct 2014 11:10:52 -0600 (MDT) (envelope-from ken) Date: Mon, 27 Oct 2014 11:10:52 -0600 From: "Kenneth D. Merry" To: Konstantin Belousov Subject: Re: sa(4) 9.2->10.1, nsa0.0: request ptr 0x803135040 is not on a page boundary; cannot split request Message-ID: <20141027171052.GA92347@mithlond.kdm.org> References: <54494E92.5010007@omnilan.de> <20141024230725.GA50845@mithlond.kdm.org> <20141025175350.GK1877@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141025175350.GK1877@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mithlond.kdm.org [127.0.0.1]); Mon, 27 Oct 2014 11:10:52 -0600 (MDT) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mithlond.kdm.org Cc: Harald Schmalzbauer , royger@freebsd.org, FreeBSD Stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Oct 2014 17:10:55 -0000 On Sat, Oct 25, 2014 at 20:53:50 +0300, Konstantin Belousov wrote: > On Fri, Oct 24, 2014 at 05:07:26PM -0600, Kenneth D. Merry wrote: > > On Thu, Oct 23, 2014 at 20:53:06 +0200, Harald Schmalzbauer wrote: > > > Hello, > > > > > > I read about the changes in sa(4) regarding large-block-split changes > > > and the transitional 'kern.cam.sa.allow_io_split' workarround. > > > > > > I'm using bacula (7.0.5) and my previous neccessarry multi-blocking > > > adjustmets like "Minimum block size = 2097152" obviously didn't work > > > with FreebSD 10.1 anymore. > > > Good news is, they are not needed any more! > > > With the default of 126 blocks (64512) I get 60-140MB/s with btape(8)'s > > > speed test on my LTO4 (HH) drive and another quick test showed that > > > using mbuffer(1) for zfs(8) 'send' isn't needed anymore (| dd > > > of=/dev/nsa0 bs=64512 seems to max out LTO4 speed). [with FreeBSD 9 the > > > transfer rates were some magnitudes lower with these block size settings!] > > > > > > Not so good news is, that bacula can't read the tape's label. > > > 'Labeling a tape (with 'label' at bconsole(8) or btape(8)) is > > > successful, and btape(8)'s 'readlabel' partially displays the correct > > > label, but not the very beginning of the label: > > > Volume Label: > > > Id : **error**VerNo > > > ?rest OK > > > > > > While it should read: > > > Volume Label: > > > Id : Bacula 1.0 immortal > > > VerNo : 11 > > > ? > > > > > > When btape(8) starts to read the label, the _subject's error is reported_: > > > *nsa0.0: request ptr 0x803135040 is not on a page boundary; cannot split > > > request* > > > > What blocksize are you using with btape(8)? > > > > What kind of controller are you using? > > > > The reason you get that error message is that the sa(4) driver goes through > > physio(9) to get buffers from userland into the kernel. physio(9) relies > > on the vmapbuf()/vunmapbuf() routines to map buffers in and out of the > > kernel. > > > > vmapbuf() operates with a page granularity. The address to be mapped has > > to start on a page boundary. It also uses kernel virtual address segments > > that are MAXPHYS in size. On x86 boxes at least, MAXPHYS is 128KB. > > > > So if you use a blocksize of 128KB, but pass in a pointer that doesn't > > start on a page boundary, vmapbuf() will have to map 33 pages instead of > > 32. In your case, it will have to start at page address 0x803135000, and > > will need 33 4KB pages, which is greater than 128KB. > I want to disable unaligned physio at all. > > See https://reviews.freebsd.org/D888 for yet another case where this beats. > > Obvious thing which stops us from doing this is binary compatibility. > I need some form of wide support to make this change. I don't see how we can remove support for non-page-aligned I/O through the various block and character devices. It would break all kinds of things, and Bacula is just the start. Perhaps another way to handle it would be that requests with unaligned buffers would just go through a non-optimal path. We could just malloc buffers and copy the data in and out. If the application sends in aligned buffers, it can go through the optimal path. Ken -- Kenneth Merry ken@FreeBSD.ORG