From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 24 16:59:15 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 54B1FD86; Wed, 24 Apr 2013 16:59:15 +0000 (UTC) (envelope-from larrymelia@gmail.com) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 1FAE110A7; Wed, 24 Apr 2013 16:59:15 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id 16so2367024iea.38 for ; Wed, 24 Apr 2013 09:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=qvC74sN80guQq4ueEBdv7M1C4queNXdCmRw5ID04DLk=; b=n9Ri3dKyMv9vkJUWY1n/EEKFx70DGabkf2aHAyISKR9VHCVaY289OD8qiM4B9JgV9k i0n3VyDcqq0qLp3Wgn6vgkrg9NLjouQF/TlpIbyaVO/e1vWQ81R4GjHFjlBGvgF5pjIL q/iFDEIrWMY/YmB7+AF5/Vp2Kad8ooMIno45c8tsF2kIsO8LLlJOmJ32RVfTwOU48YwC y6CAC41qBFdewh6cqi0UFp3Xzh7x2ZWHenRLCfwkgTl+ORoMG/juVWgC6jkSqUqccCsr fndxV+eb1rVchJY53zNQA74vu/AvhBhAWRS2Z79VLQdcwOiQsyY/uoyCowCaeS2HBaqz 4DqA== X-Received: by 10.50.138.166 with SMTP id qr6mr27790090igb.45.1366822754820; Wed, 24 Apr 2013 09:59:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.68.196 with HTTP; Wed, 24 Apr 2013 09:58:34 -0700 (PDT) In-Reply-To: <5176C98B.4050607@FreeBSD.org> References: <5176C98B.4050607@FreeBSD.org> From: Larry Melia Date: Wed, 24 Apr 2013 09:58:34 -0700 Message-ID: Subject: Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time To: Alexander Motin Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "Abhishek Gupta \(LIS\)" , KY Srinivasan , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 16:59:15 -0000 Thanks, Alexander, we'll try your approach. It seems very reasonable. On Tue, Apr 23, 2013 at 10:48 AM, Alexander Motin wrote: > On 23.04.2013 17:07, Larry Melia wrote: > >> I=92m on a team of developers working on synthetic device drivers for >> FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list) >> that we're trying to get our drivers included with the FreeBSD >> distribution. Also noted were some necessary patches to the ATA driver. >> The changes are necessary to achieve significant performance gains by >> replacing the native ATA driver with our synthetic storage driver >> when when Hyper-V is detected. Alexander Motin, the maintainer of the >> ATA code-base, however, expressed some concerns about making these >> modifications that are so specific to Hyper-V and the AMD64 build. We >> understand his concerns and have subsequently removed these patches from >> our vendor branch. Our drivers are now completely independent and >> require no changes to the kernel or other drivers. The only touch-point >> is the common makefile for modules/drivers. >> >> Removing our ATA patches, on the other hand, results in a huge >> performance loss. This is because the root file system is managed by the >> ATA driver, which is emulated under Hyper-V. Furthermore, there may be >> other native drivers that we wish to replace with synthetic ones at boot >> time (e.g., mouse driver, video driver), but, there appears to be no >> easy way to do this.Therefore, we would like to work with the developer >> community to come up with better solution that would improve support for >> synthetic drivers at boot-time. >> >> One possible solution: >> >> (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is >> used for hypervisor detection, to an earlier point in the boot process. >> Presently, it appears to be called after the ATA driver is selected, >> which is too late in the boot process. (This was discovered after some >> testing with the ATA driver.) Therefore, before the bus drivers and >> native controllers are detected and selected, discovery of a host >> hypervisor should be done first. >> >> (2) Extend the boot process to dynamically insert and prioritize >> synthetic drivers over native ones. Presently, FreeBSD supports the >> selection of generic drivers for a given device, but these can be >> overridden when a more specific driver is available. This priority >> scheme could be extended by first detecting the presence of a >> hypervisor, then allowing this priority mechanism to override the native >> drivers with their synthetic cousins (i.e., we only want to override a >> native driver when a specific hypervisor is detected and synthetic >> drivers for it have been configured). This could be arranged in a >> separate, table-driven, input file or perhaps, by extending the existing >> driver/module configuration file with additional mappings of native to >> synthetic drivers. >> >> (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use >> the more recent approach to hypervisor detection. This approach uses the >> CPU-identify functions to retrieve a unique signature consisting of a >> fixed string of ASCII characters. This was done on Linux about five >> years. For backward compatibility, however, the existing logic would be >> retained, but augmented with this new approach. It would also be >> conditionally added only for x86/AMD64 builds. >> >> When reviewing the changes we're not looking at a lot of lines of >> code.The difficultly lies, however, in where the changes are made. >> Obviously, we need to work closely with the committers responsible for >> the kernel modules we're looking to touch. Retrospectively, these >> upgrades only bring FreeBSD up to the same level of hypervisor support >> already present in Linux.There are other issues that we would also like >> to discuss, but the improvements described here are on the top of the >> list. >> >> We look forward to working more closely with the FreeBSD community and >> welcome your remarks and feedback on this proposal. >> > > I am sorry, I don't really understand what is your proposition, so I just > describe my vision of the solution here, hoping it will be useful. > > You are right that FreeBSD has priority mechanism to select best driver > for each device. That mechanism should be sufficient to prevent default O= S > driver from attaching to emulated ATA controller when Hyper-V environment > is detected and paravirtual drivers are installed. To do that you should > just create a very small dummy driver for PCI bus, implementing single > probe() method. That probe() method should in your preferable way check > that Hyper-V environment is active and match PCI ID of the device against > the list of used by Hyper-V for emulated ATA controller. If both conditio= ns > match, the probe() method should return highest priority value to overrid= e > any other ATA driver in system for this specific device. After that the > dummy driver should just have empty attach() and detach() routines, not > doing anything with the emulated device, relying on paravitrual driver to > handle the disks. > > Since the dummy driver can be made machine-dependent, you may use there > any VM detection mechanisms you like, even direct MSR access. Dummy drive= r > can be made completely independent from the rest of the system, so it can > even be loaded as kernel module without requiring any base system > modifications. If you still prefer it for some reason (for example to not > do some complicated checks multiple times in multiple drivers), you may > move Hyper-V detection logic out of that driver to some single place in > base system. But you don't need to touch OS-native ATA driver in any way. > > -- > Alexander Motin >