From owner-freebsd-virtualization@FreeBSD.ORG Fri Mar 27 17:43:14 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E91B236C for ; Fri, 27 Mar 2015 17:43:13 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7866AA53 for ; Fri, 27 Mar 2015 17:43:13 +0000 (UTC) Received: by wgra20 with SMTP id a20so106719753wgr.3 for ; Fri, 27 Mar 2015 10:43:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=cM+kxrYoSZJb4DbLSIGitmSkpjPvS1m/0kX0TCFH9aU=; b=bsJC2SybFjpYfMheekVe3fKMRSETvQAq2Qv51Ig4D2FND2gjopSacii9llqMJPPPeW EIu0cZ8wU74PldDwCODML7nuMfUeklV8kpKeFUMaK3ClEt711kHFzHqJPHgRyfy+9QJl EkwsapDoBEpZGE1NOZGFR0eVQnB8GGx7d8UJWDG/CBzy+luRFZVfbV6/NHeoNAGzfCPI JThiSOV6r5rSlAP19SEi/WEqDuLQVhYrw5niwB+rm4U2WNWAORjE48ltIah8KZ7xjadr wxlENjWzTic1LQVlj/VkKbi93OgE9A5WW+rRCPwmAxPuWm5JIUqEPDfTMHe1YADmIHM+ e07Q== X-Received: by 10.180.20.233 with SMTP id q9mr58535418wie.75.1427478191930; Fri, 27 Mar 2015 10:43:11 -0700 (PDT) Received: from mavbook.mavhome.dp.ua ([134.249.139.101]) by mx.google.com with ESMTPSA id bd1sm4375740wib.13.2015.03.27.10.43.10 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Mar 2015 10:43:11 -0700 (PDT) Sender: Alexander Motin Message-ID: <551596AD.8070202@FreeBSD.org> Date: Fri, 27 Mar 2015 19:43:09 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: John Nielsen Subject: Re: Bhyve storage improvements References: <5515270A.7050408@FreeBSD.org> <98136D5B-297B-4538-8EF4-EA2872C6640B@jnielsen.net> In-Reply-To: <98136D5B-297B-4538-8EF4-EA2872C6640B@jnielsen.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2015 17:43:14 -0000 On 27.03.2015 18:47, John Nielsen wrote: > Does anyone have plans (or know about any) to implement virtio-scsi support in bhyve? That API does support TRIM and should retain most or all of the low-overhead virtio goodness. I was thinking about that (not really a plans yet, just some thoughts), but haven't found a good motivation and understanding of whole possible infrastructure. I am not sure it worth to emulate SCSI protocol in addition to already done ATA in ahci-hd and simple block in virtio-blk just to get another, possibly faster then AHCI, block storage with TRIM/UNMAP. Really good SCSI disk emulation in CTL in kernel takes about 20K lines of code. It is pointless to duplicate it, and may be complicated for administration to just interface to it. Indeed I've seen virtio-blk being faster then ahci-hd in some tests, but those tests were highly synthetic. I haven't tested it on real workloads, but I have feeling that real difference may be not that large. If somebody wants to check -- more benchmarks are highly welcome! From the theoretical side I'd like to notice that both ATA and SCSI protocols on guests go through additional ATA/SCSI infrastructure (CAM in FreeBSD), absent in case pure block virtio-blk, so they have some more overhead by definition. Main potential benefit I see from using virtio-scsi is a possibility to pass through to client not a block device, but some real SCSI device. It can be some local DVD writer, or remote iSCSI storage. The last would be especially interesting for large production installations. But the main problem I see here is booting. To make user-level loader boot the kernel from DVD or iSCSI, bhyve has to implement its own SCSI initiator, like small second copy of CAM in user-level. Booting kernel from some other local block storage and then attaching to remote iSCSI storage for data can be much easier, but it is not convenient. It is possible to nt connect to iSCSI directly from user-level, but to make kernel CAM do it, and then make CAM provide both block layer for booting and SCSI layer for virtio-scsi, but I am not sure that it is very good from security point to make host system to see virtual disks. Though may be it could work if CAM could block kernel/GEOM access to them, alike it is done for ZVOLs now, supporting "geom" and "dev" modes. Though that complicates CAM and the whole infrastructure. -- Alexander Motin