From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 18 20:12:17 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A04D3CA0 for ; Sat, 18 Apr 2015 20:12:17 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24A2FC4B for ; Sat, 18 Apr 2015 20:12:17 +0000 (UTC) Received: by lagv1 with SMTP id v1so102059979lag.3 for ; Sat, 18 Apr 2015 13:12:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=0RBrklDHTmD2fvLx+t+bXV+k7DU6ij7sXUan+QWX+nA=; b=l2XRGUiDfE81XaDsMSoUZwPSnM/PkmoYBbxtUltM0494V2WCOvhJ1OEvrPizBQJ7tw J9YPCe17V6sPBlOewpQN71R5zazKA/P6xhQ8LdtJNLftXf01uEzQKdzP/cz24ZEK6UFJ j9vsk9FBWJPIK+1l2V2n0EkovHo6kGeQlILyZwxVdvtJnuFADqKX/azCf4QVgZ3tS0wX G1gfEbwGiLVjRRS9d7KHbGdaZG9zPIbVrxiPdQdWYps4OTGT8nz01HpAfIr5swmtJb8E QLK5yO1xOus+GUrFhBdTzI4275rRP0hvRmUHhgKTJI26GFDdqAkPhuyGCQXM4CbBlS3i k7hQ== X-Received: by 10.112.134.202 with SMTP id pm10mr9638274lbb.34.1429387935196; Sat, 18 Apr 2015 13:12:15 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id i3sm1143181lbs.21.2015.04.18.13.12.13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 18 Apr 2015 13:12:14 -0700 (PDT) Sender: Alexander Motin Message-ID: <5532BA9B.80600@FreeBSD.org> Date: Sat, 18 Apr 2015 23:12:11 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Julian Hsiao , "freebsd-virtualization@freebsd.org" Subject: Re: Bhyve storage improvements (was: Several bhyve quirks) References: <5522EEE0.5010807@FreeBSD.org> In-Reply-To: <5522EEE0.5010807@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Apr 2015 20:12:17 -0000 On 06.04.2015 23:38, Alexander Motin wrote: >> I had some time to try it out today, but I'm still having issues: > > I've just made experiment alike to your with making bhyve to work on top > of GEOM device instead of preferable "dev" mode of ZVOL. And I indeed > reproduced the problem. But the problem that I see is not related to the > block size. The block size is reported to the guest correctly as 4K, and > as I can see it works as such at least in FreeBSD guest. > > The problem is in the way how bhyve inter-operates with block/GEOM > devices. bhyve sends requests to the kernel with preadv()/pwritev() > calls, specifying scatter/gather lists of buffer addresses provided by > the guest. But GEOM code can not handle scatter/gather lists, only > sequential buffer, and so single request is split into several. The > problem is that splitting happens according to scatter/gather elements, > and those elements in general case may not be multiple to the block > size, that is fatal for GEOM and any block device. > > I am not yet sure how to fix this problem. The most straightforward way > is to copy the data at some point to collect elements of scatter/gather > list into something sequential to pass to GEOM, but that requires > additional memory allocation, and the copying is not free. May be some > cases could be optimized to work without copying but with some clever > page mapping, but that seems absolutely not trivial. I've committed the workaround to FreeBSD head at r281700. -- Alexander Motin