From owner-svn-src-all@FreeBSD.ORG Thu Aug 28 18:58:19 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41FDCC25; Thu, 28 Aug 2014 18:58:19 +0000 (UTC) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27B7610A4; Thu, 28 Aug 2014 18:58:17 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id gl10so1487698lab.7 for ; Thu, 28 Aug 2014 11:58:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=QJLFY4UWBHKYk+UMn53YMVTetgxUC+Wi2N1ytpGo+FQ=; b=QxLN42WTj5q3nQo77wo8lDs8zHSDM6fg+A0hlDp9c5foc1QqRWoE6uSaEQwVQxxrdD QnTHZEoRisCqmznlLuKkabrXZ4cpxeY/BtIqPRODkjffvQpNmYv6x4gdtPriQCt0RnH9 jXOgxVzejtlY8Utl/afmo6Gcg9q34S/qlYNNXR4bVE8pJnoWE+eNh/ttYXQgBvx6Fx8G 1Z71VdNHeRUAA0VSKoXB7TuVq//JFVp60z+TaSzDLJTYAPR1tDUtgHupl3H++qLRKvrw SUaYk441dMzI+2Z01lVmq7aFTANmtmvfjK5KYGyHARfNTiaWJYJSWbhZOtMI0yPFy7BQ vQqg== X-Received: by 10.152.179.229 with SMTP id dj5mr4893968lac.97.1409252295967; Thu, 28 Aug 2014 11:58:15 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id h9sm2889224lae.40.2014.08.28.11.58.13 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 28 Aug 2014 11:58:15 -0700 (PDT) Sender: Alexander Motin Message-ID: <53FF7BC4.6050801@FreeBSD.org> Date: Thu, 28 Aug 2014 21:58:12 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John-Mark Gurney Subject: Re: svn commit: r269814 - head/sys/dev/xen/blkfront References: <53e8e31e.2179.30c1c657@svn.freebsd.org> <53FF7386.3050804@FreeBSD.org> <20140828184515.GV71691@funkthat.com> In-Reply-To: <20140828184515.GV71691@funkthat.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= , src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Aug 2014 18:58:19 -0000 On 28.08.2014 21:45, John-Mark Gurney wrote: > Alexander Motin wrote this message on Thu, Aug 28, 2014 at 21:23 +0300: >> Hi, Roger. >> >> It looks to me like this commit does not work as it should. I got >> problem when I just tried `newfs /dev/ada0 ; mount /dev/ada0 /mnt`. >> Somehow newfs does not produce valid filesystem. Problem is reliably >> repeatable and reverting this commit fixes it. >> >> I found at least one possible cause there: If original data buffer is >> unmapped, misaligned and not physically contiguous, then present x86 >> bus_dmamap_load_bio() implementation will process each physically >> contiguous segment separately. Due to the misalignment first and last >> physical segments may have size not multiple to 512 bytes. Since each >> segment processed separately, they are not joined together, and >> xbd_queue_cb() is getting segments not multiple to 512 bytes. Attempt to >> convert them to exact number of sectors in the driver cause data corruption. > > Are you sure this isn't a problem w/ the tag not properly specifying > the correct alignement? I don't know how to specify it stronger then this: error = bus_dma_tag_create( bus_get_dma_tag(sc->xbd_dev), /* parent */ 512, PAGE_SIZE, /* algnmnt, boundary */ BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ sc->xbd_max_request_size, sc->xbd_max_request_segments, PAGE_SIZE, /* maxsegsize */ BUS_DMA_ALLOCNOW, /* flags */ busdma_lock_mutex, /* lockfunc */ &sc->xbd_io_lock, /* lockarg */ &sc->xbd_io_dmat); > Also, I don't think there is a way for busdma > to say that you MUST have a segment be a multiple of 512, though you > could use a 512 boundary, but that would force all segments to only be > 512 bytes... As I understand, that is mandatory requirement for this "hardware". Alike 4K alignment requirement also exist at least for SDHCI, and IIRC UHCI/OHCI hardware. Even AHCI requires both segment addresses and lengths to be even. I may be wrong, but I think it is quite likely that hardware that requires segment address alignment quite likely will have the same requirements for segments length. >> This is a bug of the busdma code, and until it is fixed I don't see >> solution other then temporary reverting this commit. >> >> On 11.08.2014 18:37, Roger Pau Monné wrote: >>> Author: royger >>> Date: Mon Aug 11 15:37:02 2014 >>> New Revision: 269814 >>> URL: http://svnweb.freebsd.org/changeset/base/269814 >>> >>> Log: >>> blkfront: add support for unmapped IO >>> >>> Using unmapped IO is really beneficial when running inside of a VM, >>> since it avoids IPIs to other vCPUs in order to invalidate the >>> mappings. >>> >>> This patch adds unmapped IO support to blkfront. The following tests >>> results have been obtained when running on a Xen host without HAP: >>> >>> PVHVM >>> 3165.84 real 6354.17 user 4483.32 sys >>> PVHVM with unmapped IO >>> 2099.46 real 4624.52 user 2967.38 sys >>> >>> This is because when running using shadow page tables TLB flushes and >>> range invalidations are much more expensive, so using unmapped IO >>> provides a very important performance boost. >>> >>> Sponsored by: Citrix Systems R&D >>> Tested by: robak >>> MFC after: 1 week >>> PR: 191173 >>> >>> dev/xen/blkfront/blkfront.c: >>> - Add and announce support for unmapped IO. >>> >>> Modified: >>> head/sys/dev/xen/blkfront/blkfront.c >>> >>> Modified: head/sys/dev/xen/blkfront/blkfront.c >>> ============================================================================== >>> --- head/sys/dev/xen/blkfront/blkfront.c Mon Aug 11 15:06:07 2014 (r269813) >>> +++ head/sys/dev/xen/blkfront/blkfront.c Mon Aug 11 15:37:02 2014 (r269814) >>> @@ -272,8 +272,12 @@ xbd_queue_request(struct xbd_softc *sc, >>> { >>> int error; >>> >>> - error = bus_dmamap_load(sc->xbd_io_dmat, cm->cm_map, cm->cm_data, >>> - cm->cm_datalen, xbd_queue_cb, cm, 0); >>> + if (cm->cm_bp != NULL) >>> + error = bus_dmamap_load_bio(sc->xbd_io_dmat, cm->cm_map, >>> + cm->cm_bp, xbd_queue_cb, cm, 0); >>> + else >>> + error = bus_dmamap_load(sc->xbd_io_dmat, cm->cm_map, >>> + cm->cm_data, cm->cm_datalen, xbd_queue_cb, cm, 0); >>> if (error == EINPROGRESS) { >>> /* >>> * Maintain queuing order by freezing the queue. The next >>> @@ -333,8 +337,6 @@ xbd_bio_command(struct xbd_softc *sc) >>> } >>> >>> cm->cm_bp = bp; >>> - cm->cm_data = bp->bio_data; >>> - cm->cm_datalen = bp->bio_bcount; >>> cm->cm_sector_number = (blkif_sector_t)bp->bio_pblkno; >>> >>> switch (bp->bio_cmd) { >>> @@ -993,7 +995,7 @@ xbd_instance_create(struct xbd_softc *sc >>> >>> sc->xbd_disk->d_mediasize = sectors * sector_size; >>> sc->xbd_disk->d_maxsize = sc->xbd_max_request_size; >>> - sc->xbd_disk->d_flags = 0; >>> + sc->xbd_disk->d_flags = DISKFLAG_UNMAPPED_BIO; >>> if ((sc->xbd_flags & (XBDF_FLUSH|XBDF_BARRIER)) != 0) { >>> sc->xbd_disk->d_flags |= DISKFLAG_CANFLUSHCACHE; >>> device_printf(sc->xbd_dev, > -- Alexander Motin