From owner-freebsd-virtualization@FreeBSD.ORG Mon Apr 22 11:06:54 2013 Return-Path: Delivered-To: freebsd-virtualization@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 434FF622 for ; Mon, 22 Apr 2013 11:06:54 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 0C7101097 for ; Mon, 22 Apr 2013 11:06:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.6/8.14.6) with ESMTP id r3MB6ruZ089317 for ; Mon, 22 Apr 2013 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.6/8.14.6/Submit) id r3MB6rfh089315 for freebsd-virtualization@FreeBSD.org; Mon, 22 Apr 2013 11:06:53 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Apr 2013 11:06:53 GMT Message-Id: <201304221106.r3MB6rfh089315@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-virtualization@FreeBSD.org Subject: Current problem reports assigned to freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Apr 2013 11:06:54 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/170096 virtualization[vimage] Dynamically-attached network interface will c o kern/169991 virtualization[run] [vimage] panic after device plugged in o kern/165252 virtualization[vimage] [pf] [panic] kernel panics with VIMAGE and PF o kern/161094 virtualization[vimage] [pf] [panic] kernel panic with pf + VIMAGE wh o kern/160541 virtualization[vimage][pf][patch] panic: userret: Returning on td 0x o kern/160496 virtualization[vimage] [pf] [patch] kernel panic with pf + VIMAGE o kern/148155 virtualization[vimage] [pf] Kernel panic with PF + VIMAGE kernel opt a kern/147950 virtualization[vimage] [carp] VIMAGE + CARP = kernel crash s kern/143808 virtualization[pf] pf does not work inside jail a kern/141696 virtualization[rum] [vimage] [panic] rum(4)+ vimage = kernel panic 10 problems total. From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 23 14:07:44 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 97A86EAF; Tue, 23 Apr 2013 14:07:44 +0000 (UTC) (envelope-from larrymelia@gmail.com) Received: from mail-ia0-x22d.google.com (mail-ia0-x22d.google.com [IPv6:2607:f8b0:4001:c02::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 64BCA153E; Tue, 23 Apr 2013 14:07:44 +0000 (UTC) Received: by mail-ia0-f173.google.com with SMTP id j5so579293iaf.4 for ; Tue, 23 Apr 2013 07:07:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:from:date:message-id:subject:to:cc :content-type; bh=JndvOZSXBzNlhHCepUKnUYBhz++iUBy1805d7dL7sZo=; b=HRGD50WpDM/YhoMTCIJ8wNTeqf1jvkQEmv4LRbTTATP2GdbXxYiqehw/mxFVzPSoqB pvRvv2lhBR0SqFZr6UZTtYyLsvQup55mJuNOFFcz7ycKb8WfZ3i5AlFs1bRenWL5DvqM 6ILzzA1DmAGYkc8ZpR99wm59DTxN2Xy9rFRQ5ZOrtQ/sui2cKjM70oYLCBg9Qz5oIwhj W1U81DSpLqjJw+oY9/JQuMTDYR4DkNgxY+qqr8KwF7/6iFR/ZHsVPr/qkNAV53O+3xri qDIjfiIiGMDmX3oKrbRpq/37nBuE11UHUV/KdvJXOs4XkxLmHy2B7L8aL88VD/Afjhj2 5qug== X-Received: by 10.50.83.102 with SMTP id p6mr1341934igy.45.1366726064038; Tue, 23 Apr 2013 07:07:44 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.68.196 with HTTP; Tue, 23 Apr 2013 07:07:03 -0700 (PDT) From: Larry Melia Date: Tue, 23 Apr 2013 07:07:03 -0700 Message-ID: Subject: Proposal for better support of hypervisors and their synthetic drivers at boot-time To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Alexander Motin , KY Srinivasan , "Abhishek Gupta \(LIS\)" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 14:07:44 -0000 I=92m on a team of developers working on synthetic device drivers for FreeB= SD on Hyper-V. In late March, I briefly mentioned (on this list) that we're trying to get our drivers included with the FreeBSD distribution. Also noted were some necessary patches to the ATA driver. The changes are necessary to achieve significant performance gains by replacing the native ATA driver with our synthetic storage driver when when Hyper-V is detected. Alexander Motin, the maintainer of the ATA code-base, however, expressed some concerns about making these modifications that are so specific to Hyper-V and the AMD64 build. We understand his concerns and have subsequently removed these patches from our vendor branch. Our drivers are now completely independent and require no changes to the kernel or other drivers. The only touch-point is the common makefile for modules/drivers. Removing our ATA patches, on the other hand, results in a huge performance loss. This is because the root file system is managed by the ATA driver, which is emulated under Hyper-V. Furthermore, there may be other native drivers that we wish to replace with synthetic ones at boot time (e.g., mouse driver, video driver), but, there appears to be no easy way to do this.Therefore, we would like to work with the developer community to come up with better solution that would improve support for synthetic drivers at boot-time. One possible solution: (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is used for hypervisor detection, to an earlier point in the boot process. Presently, it appears to be called after the ATA driver is selected, which is too late in the boot process. (This was discovered after some testing with the ATA driver.) Therefore, before the bus drivers and native controllers are detected and selected, discovery of a host hypervisor should be done first. (2) Extend the boot process to dynamically insert and prioritize synthetic drivers over native ones. Presently, FreeBSD supports the selection of generic drivers for a given device, but these can be overridden when a more specific driver is available. This priority scheme could be extended by first detecting the presence of a hypervisor, then allowing this priority mechanism to override the native drivers with their synthetic cousins (i.e., we only want to override a native driver when a specific hypervisor is detected and synthetic drivers for it have been configured). This could be arranged in a separate, table-driven, input file or perhaps, by extending the existing driver/module configuration file with additional mappings of native to synthetic drivers. (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use the more recent approach to hypervisor detection. This approach uses the CPU-identify functions to retrieve a unique signature consisting of a fixed string of ASCII characters. This was done on Linux about five years. For backward compatibility, however, the existing logic would be retained, but augmented with this new approach. It would also be conditionally added only for x86/AMD64 builds. When reviewing the changes we're not looking at a lot of lines of code.The difficultly lies, however, in where the changes are made. Obviously, we need to work closely with the committers responsible for the kernel modules we're looking to touch. Retrospectively, these upgrades only bring FreeBSD up to the same level of hypervisor support already present in Linux.There are other issues that we would also like to discuss, but the improvements described here are on the top of the list. We look forward to working more closely with the FreeBSD community and welcome your remarks and feedback on this proposal. Larry Melia From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 23 16:43:18 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 61477A41 for ; Tue, 23 Apr 2013 16:43:18 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 36A671029 for ; Tue, 23 Apr 2013 16:43:18 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id at1so938144iec.29 for ; Tue, 23 Apr 2013 09:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=XyHwSU8VHB+OYvdEhluLAWiEsM/4Zpwfr8jUk1wPOSI=; b=v0VclTfNuQh7G0Vv55t+LHSMOGeTGh+7FNRN/u/wKsgyqcDKZ/7aYmVtRxdVY64fJE xWjDmWn7cNt0IacXLn2K47iYWbuhc5umu8RnFftb7AAWLOIO3eAk+fn9Qw2lQ07Viwki 1iJoSGiRuaQaru8YlZSy9uYPMC8ytNS8JUGImUdE9AvJDZ4ORQZRIBI5XHegEXNEIOUx ZNEn7iaGVDOx1JEDk4T39Sb7Qj3GVTYHsOZeBplRujrMzFka5TqvACnOmKU8x3n94EvZ UHTCSVTe5meHTpTQ6MrdoaqT3d5XzliU2+9K0itu/lASQlA82uoeDfiZ8vT/PXN/WQx5 Bqrg== MIME-Version: 1.0 X-Received: by 10.50.100.167 with SMTP id ez7mr24752061igb.3.1366735397886; Tue, 23 Apr 2013 09:43:17 -0700 (PDT) Received: by 10.43.9.138 with HTTP; Tue, 23 Apr 2013 09:43:17 -0700 (PDT) In-Reply-To: <4FC78CC2-AAF3-4802-ACFC-3F3281DC45DE@pluribusnetworks.com> References: <4FC78CC2-AAF3-4802-ACFC-3F3281DC45DE@pluribusnetworks.com> Date: Tue, 23 Apr 2013 09:43:17 -0700 Message-ID: Subject: Re: direct descriptor support for virtio block device From: Neel Natu To: Tycho Nightingale Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 16:43:18 -0000 Hi Tycho, On Fri, Apr 19, 2013 at 8:46 AM, Tycho Nightingale < tycho.nightingale@pluribusnetworks.com> wrote: > > The current virtio block device advertises support for the indirect > descriptors feature yet is unable to cope if the guest elects not to use > them. Attached is a patch for direct descriptors along with an > implementation of device reset. > > Thanks for the patch - committed as r249813. best Neel > Tycho > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 23 17:49:04 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3D882973 for ; Tue, 23 Apr 2013 17:49:04 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x22b.google.com (mail-ea0-x22b.google.com [IPv6:2a00:1450:4013:c01::22b]) by mx1.freebsd.org (Postfix) with ESMTP id C6F2413AE for ; Tue, 23 Apr 2013 17:49:03 +0000 (UTC) Received: by mail-ea0-f171.google.com with SMTP id b15so389826eae.2 for ; Tue, 23 Apr 2013 10:49:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=Tc5OiymHPPvqLmvUusoQDwncMdLDWPtMrHm3hX+YPUE=; b=kwiUJ+qORlWtu/cSUsQl/MwJ4OT4AxQhiI9PBT2oP2Fk6edQY9AbLGMrvFAjzDF863 HcOB7ZJTrjUhLZaxdZ+qNwoqXZtkZt9wd4L5/euvZHwCEnI4pVsMwqDUur+NuvnRoMcE /kcN/voRFDSQSYgLPvwcc1V2oqaodTrQmUfpT0C17a4pV2fWm2xWM4qSGzBxILIW2Tye /cSnG/DuKnS31G2uW7CN0jAwLZKLyx6fb9X3aWlUlvyRUBWgcKtofatLq1UJGcxRCbME /3yTA8eUyhAnERvCcF0CrYlzyETknC94YbMCbCfh6blYqe9mdoAKxk0xaLmXWKLRd8LE m1Lw== X-Received: by 10.14.221.67 with SMTP id q43mr9073925eep.20.1366739342941; Tue, 23 Apr 2013 10:49:02 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id w51sm48335169eev.13.2013.04.23.10.49.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 23 Apr 2013 10:49:01 -0700 (PDT) Sender: Alexander Motin Message-ID: <5176C98B.4050607@FreeBSD.org> Date: Tue, 23 Apr 2013 20:48:59 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Larry Melia Subject: Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: "Abhishek Gupta \(LIS\)" , KY Srinivasan , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 17:49:04 -0000 On 23.04.2013 17:07, Larry Melia wrote: > I’m on a team of developers working on synthetic device drivers for > FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list) > that we're trying to get our drivers included with the FreeBSD > distribution. Also noted were some necessary patches to the ATA driver. > The changes are necessary to achieve significant performance gains by > replacing the native ATA driver with our synthetic storage driver > when when Hyper-V is detected. Alexander Motin, the maintainer of the > ATA code-base, however, expressed some concerns about making these > modifications that are so specific to Hyper-V and the AMD64 build. We > understand his concerns and have subsequently removed these patches from > our vendor branch. Our drivers are now completely independent and > require no changes to the kernel or other drivers. The only touch-point > is the common makefile for modules/drivers. > > Removing our ATA patches, on the other hand, results in a huge > performance loss. This is because the root file system is managed by the > ATA driver, which is emulated under Hyper-V. Furthermore, there may be > other native drivers that we wish to replace with synthetic ones at boot > time (e.g., mouse driver, video driver), but, there appears to be no > easy way to do this.Therefore, we would like to work with the developer > community to come up with better solution that would improve support for > synthetic drivers at boot-time. > > One possible solution: > > (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is > used for hypervisor detection, to an earlier point in the boot process. > Presently, it appears to be called after the ATA driver is selected, > which is too late in the boot process. (This was discovered after some > testing with the ATA driver.) Therefore, before the bus drivers and > native controllers are detected and selected, discovery of a host > hypervisor should be done first. > > (2) Extend the boot process to dynamically insert and prioritize > synthetic drivers over native ones. Presently, FreeBSD supports the > selection of generic drivers for a given device, but these can be > overridden when a more specific driver is available. This priority > scheme could be extended by first detecting the presence of a > hypervisor, then allowing this priority mechanism to override the native > drivers with their synthetic cousins (i.e., we only want to override a > native driver when a specific hypervisor is detected and synthetic > drivers for it have been configured). This could be arranged in a > separate, table-driven, input file or perhaps, by extending the existing > driver/module configuration file with additional mappings of native to > synthetic drivers. > > (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use > the more recent approach to hypervisor detection. This approach uses the > CPU-identify functions to retrieve a unique signature consisting of a > fixed string of ASCII characters. This was done on Linux about five > years. For backward compatibility, however, the existing logic would be > retained, but augmented with this new approach. It would also be > conditionally added only for x86/AMD64 builds. > > When reviewing the changes we're not looking at a lot of lines of > code.The difficultly lies, however, in where the changes are made. > Obviously, we need to work closely with the committers responsible for > the kernel modules we're looking to touch. Retrospectively, these > upgrades only bring FreeBSD up to the same level of hypervisor support > already present in Linux.There are other issues that we would also like > to discuss, but the improvements described here are on the top of the list. > > We look forward to working more closely with the FreeBSD community and > welcome your remarks and feedback on this proposal. I am sorry, I don't really understand what is your proposition, so I just describe my vision of the solution here, hoping it will be useful. You are right that FreeBSD has priority mechanism to select best driver for each device. That mechanism should be sufficient to prevent default OS driver from attaching to emulated ATA controller when Hyper-V environment is detected and paravirtual drivers are installed. To do that you should just create a very small dummy driver for PCI bus, implementing single probe() method. That probe() method should in your preferable way check that Hyper-V environment is active and match PCI ID of the device against the list of used by Hyper-V for emulated ATA controller. If both conditions match, the probe() method should return highest priority value to override any other ATA driver in system for this specific device. After that the dummy driver should just have empty attach() and detach() routines, not doing anything with the emulated device, relying on paravitrual driver to handle the disks. Since the dummy driver can be made machine-dependent, you may use there any VM detection mechanisms you like, even direct MSR access. Dummy driver can be made completely independent from the rest of the system, so it can even be loaded as kernel module without requiring any base system modifications. If you still prefer it for some reason (for example to not do some complicated checks multiple times in multiple drivers), you may move Hyper-V detection logic out of that driver to some single place in base system. But you don't need to touch OS-native ATA driver in any way. -- Alexander Motin From owner-freebsd-virtualization@FreeBSD.ORG Tue Apr 23 19:40:11 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 43E10D48; Tue, 23 Apr 2013 19:40:11 +0000 (UTC) (envelope-from scrappy@hub.org) Received: from hub.org (hub.org [200.46.208.146]) by mx1.freebsd.org (Postfix) with ESMTP id 124C31AD0; Tue, 23 Apr 2013 19:40:10 +0000 (UTC) Received: from maia.hub.org (unknown [200.46.151.188]) by hub.org (Postfix) with ESMTP id 460E3A12C3F; Tue, 23 Apr 2013 16:40:10 -0300 (ADT) Received: from hub.org ([200.46.208.146]) by maia.hub.org (mx1.hub.org [200.46.151.188]) (amavisd-maia, port 10024) with ESMTP id 26632-01; Tue, 23 Apr 2013 19:40:10 +0000 (UTC) Received: from [10.5.250.150] (remote.ilcs.sd63.bc.ca [142.31.148.2]) by hub.org (Postfix) with ESMTPA id 87CD8A12C3E; Tue, 23 Apr 2013 16:40:09 -0300 (ADT) From: "Marc G. Fournier" Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: vbox + bce == sporactic ethernet hangs Date: Tue, 23 Apr 2013 12:40:07 -0700 Message-Id: <42DB312E-0DE5-4E89-BCC5-A1509A86DF12@hub.org> To: freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Apr 2013 19:40:11 -0000 I am running FreeBSD 9-STABLE (updated yesterday: FreeBSD 9.1-STABLE = #15: Mon Apr 22 07:45:07 UTC 2013) with VirtualBox 4.2.6 from ports =85 = the hardware is using a Broadcom ethernet: bce0: mem = 0xf4000000-0xf5ffffff irq 16 at device 0.0 on pci7 miibus0: on bce0 bce0: Ethernet address: 00:22:19:5b:20:bd bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C = (4.4.1); Bufs (RX:2;TX:2;PG:8); Flags (SPLT|MSI|MFW); MFW (UMP 1.1.9) bce0: bce_pulse(): Warning: bootcode thinks driver is absent! (bc_state = =3D 0x00004006) Running with simple jail's on it, the server runs flawlessly until = reboot =85 but as soon as I start running Virtualbox on it, I get = sporadic server 'hangs' =85 never the same time, usually can be = triggered by heavier then normal load on the virtual box (ie. running an = rsync session from the base server into the vbox environment) =85=20 When it happens, I can *usually* connect via the DRAC / remote console = and login =85 but doing an 'ifconfig down' on the device and then back = up makes no difference =85 if I send a ctl-alt-del through the remote = console, more often then not, it will free up whatever is going on, so = that pinging works again, but, of course, I've already hit ctl-alt-del, = so its rebooting even though now I don't need it to =85 Based on a page on the wiki about tuning for vbox, I have set: net.graph.maxdata=3D65536 but I've seen this happen even with that set, so not sure if I'm just = still triggering it, or its something else I'm experiencing =85 So, two questions: 1. is there something I can run to see if I *am* in fact hitting that = limit? =20 2. is there something I can do, like ctl-alt-del, but without the = reboot, to 'free' the ethernet? Thx From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 24 16:59:15 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 54B1FD86; Wed, 24 Apr 2013 16:59:15 +0000 (UTC) (envelope-from larrymelia@gmail.com) Received: from mail-ie0-x233.google.com (mail-ie0-x233.google.com [IPv6:2607:f8b0:4001:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 1FAE110A7; Wed, 24 Apr 2013 16:59:15 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id 16so2367024iea.38 for ; Wed, 24 Apr 2013 09:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=qvC74sN80guQq4ueEBdv7M1C4queNXdCmRw5ID04DLk=; b=n9Ri3dKyMv9vkJUWY1n/EEKFx70DGabkf2aHAyISKR9VHCVaY289OD8qiM4B9JgV9k i0n3VyDcqq0qLp3Wgn6vgkrg9NLjouQF/TlpIbyaVO/e1vWQ81R4GjHFjlBGvgF5pjIL q/iFDEIrWMY/YmB7+AF5/Vp2Kad8ooMIno45c8tsF2kIsO8LLlJOmJ32RVfTwOU48YwC y6CAC41qBFdewh6cqi0UFp3Xzh7x2ZWHenRLCfwkgTl+ORoMG/juVWgC6jkSqUqccCsr fndxV+eb1rVchJY53zNQA74vu/AvhBhAWRS2Z79VLQdcwOiQsyY/uoyCowCaeS2HBaqz 4DqA== X-Received: by 10.50.138.166 with SMTP id qr6mr27790090igb.45.1366822754820; Wed, 24 Apr 2013 09:59:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.64.68.196 with HTTP; Wed, 24 Apr 2013 09:58:34 -0700 (PDT) In-Reply-To: <5176C98B.4050607@FreeBSD.org> References: <5176C98B.4050607@FreeBSD.org> From: Larry Melia Date: Wed, 24 Apr 2013 09:58:34 -0700 Message-ID: Subject: Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time To: Alexander Motin Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "Abhishek Gupta \(LIS\)" , KY Srinivasan , freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 16:59:15 -0000 Thanks, Alexander, we'll try your approach. It seems very reasonable. On Tue, Apr 23, 2013 at 10:48 AM, Alexander Motin wrote: > On 23.04.2013 17:07, Larry Melia wrote: > >> I=92m on a team of developers working on synthetic device drivers for >> FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list) >> that we're trying to get our drivers included with the FreeBSD >> distribution. Also noted were some necessary patches to the ATA driver. >> The changes are necessary to achieve significant performance gains by >> replacing the native ATA driver with our synthetic storage driver >> when when Hyper-V is detected. Alexander Motin, the maintainer of the >> ATA code-base, however, expressed some concerns about making these >> modifications that are so specific to Hyper-V and the AMD64 build. We >> understand his concerns and have subsequently removed these patches from >> our vendor branch. Our drivers are now completely independent and >> require no changes to the kernel or other drivers. The only touch-point >> is the common makefile for modules/drivers. >> >> Removing our ATA patches, on the other hand, results in a huge >> performance loss. This is because the root file system is managed by the >> ATA driver, which is emulated under Hyper-V. Furthermore, there may be >> other native drivers that we wish to replace with synthetic ones at boot >> time (e.g., mouse driver, video driver), but, there appears to be no >> easy way to do this.Therefore, we would like to work with the developer >> community to come up with better solution that would improve support for >> synthetic drivers at boot-time. >> >> One possible solution: >> >> (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is >> used for hypervisor detection, to an earlier point in the boot process. >> Presently, it appears to be called after the ATA driver is selected, >> which is too late in the boot process. (This was discovered after some >> testing with the ATA driver.) Therefore, before the bus drivers and >> native controllers are detected and selected, discovery of a host >> hypervisor should be done first. >> >> (2) Extend the boot process to dynamically insert and prioritize >> synthetic drivers over native ones. Presently, FreeBSD supports the >> selection of generic drivers for a given device, but these can be >> overridden when a more specific driver is available. This priority >> scheme could be extended by first detecting the presence of a >> hypervisor, then allowing this priority mechanism to override the native >> drivers with their synthetic cousins (i.e., we only want to override a >> native driver when a specific hypervisor is detected and synthetic >> drivers for it have been configured). This could be arranged in a >> separate, table-driven, input file or perhaps, by extending the existing >> driver/module configuration file with additional mappings of native to >> synthetic drivers. >> >> (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use >> the more recent approach to hypervisor detection. This approach uses the >> CPU-identify functions to retrieve a unique signature consisting of a >> fixed string of ASCII characters. This was done on Linux about five >> years. For backward compatibility, however, the existing logic would be >> retained, but augmented with this new approach. It would also be >> conditionally added only for x86/AMD64 builds. >> >> When reviewing the changes we're not looking at a lot of lines of >> code.The difficultly lies, however, in where the changes are made. >> Obviously, we need to work closely with the committers responsible for >> the kernel modules we're looking to touch. Retrospectively, these >> upgrades only bring FreeBSD up to the same level of hypervisor support >> already present in Linux.There are other issues that we would also like >> to discuss, but the improvements described here are on the top of the >> list. >> >> We look forward to working more closely with the FreeBSD community and >> welcome your remarks and feedback on this proposal. >> > > I am sorry, I don't really understand what is your proposition, so I just > describe my vision of the solution here, hoping it will be useful. > > You are right that FreeBSD has priority mechanism to select best driver > for each device. That mechanism should be sufficient to prevent default O= S > driver from attaching to emulated ATA controller when Hyper-V environment > is detected and paravirtual drivers are installed. To do that you should > just create a very small dummy driver for PCI bus, implementing single > probe() method. That probe() method should in your preferable way check > that Hyper-V environment is active and match PCI ID of the device against > the list of used by Hyper-V for emulated ATA controller. If both conditio= ns > match, the probe() method should return highest priority value to overrid= e > any other ATA driver in system for this specific device. After that the > dummy driver should just have empty attach() and detach() routines, not > doing anything with the emulated device, relying on paravitrual driver to > handle the disks. > > Since the dummy driver can be made machine-dependent, you may use there > any VM detection mechanisms you like, even direct MSR access. Dummy drive= r > can be made completely independent from the rest of the system, so it can > even be loaded as kernel module without requiring any base system > modifications. If you still prefer it for some reason (for example to not > do some complicated checks multiple times in multiple drivers), you may > move Hyper-V detection logic out of that driver to some single place in > base system. But you don't need to touch OS-native ATA driver in any way. > > -- > Alexander Motin > From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 24 18:08:00 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A5689C37 for ; Wed, 24 Apr 2013 18:08:00 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 62A0C13C3 for ; Wed, 24 Apr 2013 18:08:00 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 2E089121DE; Thu, 25 Apr 2013 04:07:58 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro.local (c-67-190-11-104.hsd1.co.comcast.net [67.190.11.104]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BLR36910 (AUTH peterg@ptree32.com.au); Thu, 25 Apr 2013 04:07:56 +1000 Message-ID: <51781F73.1070605@freebsd.org> Date: Wed, 24 Apr 2013 12:07:47 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Importing the Microsoft hyper-v PV drivers into CURRENT References: <5176C98B.4050607@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Junkmail-Info: RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,SPF_SOFTFAIL X-Junkmail-Status: score=24/51, host=dommail.onthenet.com.au Cc: "Abhishek Gupta \(LIS\)" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Apr 2013 18:08:00 -0000 Hi all, I'd like to start the process of bringing in Microsoft's hyper-v paravirtualized drivers into CURRENT. Here're the steps I'll follow: 1. Pull the hyperv driver-only git export (https://github.com/FreeBSDonHyper-V/VendorBranchForFreeBSDonHyper-V) into a vendor branch 2. A project branch off CURRENT will be created, and the vendor branch pulled into that. Folk can use that for testing/review until it's considered OK. 3. Lastly, when it's deemed ready, I'll drop the vendor branch into CURRENT and retire the project branch. Let me know if that sounds OK. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 27 21:00:01 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A8526A51 for ; Sat, 27 Apr 2013 21:00:01 +0000 (UTC) (envelope-from mack@macktronics.com) Received: from coco.macktronics.com (coco.macktronics.com [209.181.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 8A4971A6C for ; Sat, 27 Apr 2013 21:00:01 +0000 (UTC) Received: from coco.macktronics.com (coco.macktronics.com [209.181.253.65]) by coco.macktronics.com (Postfix) with ESMTP id A29CA4AC40 for ; Sat, 27 Apr 2013 15:51:52 -0500 (CDT) Date: Sat, 27 Apr 2013 15:51:52 -0500 (CDT) From: Dan Mack To: freebsd-virtualization@freebsd.org Subject: bhyve console question Message-ID: <20130427154625.X29498@coco.macktronics.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 21:00:01 -0000 For those of you tinkering with bhyve; quick question. I've noticed that everything I type on the console ends up in /var/log/messages on the guest. This includes all shell activity and login stuff. Is this expected behaviour ? When following the standard bhyve recipes on the interwebs, it appears as if all console activity is categorized as kernel.debug level messages, for example: root@mcp:~ # tail /var/log/messages Apr 27 15:50:02 mcp kernel: / Apr 27 15:50:02 mcp kernel: va Apr 27 15:50:03 mcp kernel: r/ Apr 27 15:50:03 mcp kernel: l Apr 27 15:50:03 mcp kernel: o Apr 27 15:50:03 mcp kernel: g Apr 27 15:50:04 mcp kernel: /m Apr 27 15:50:04 mcp kernel: e Apr 27 15:50:04 mcp kernel: s Apr 27 15:50:04 mcp kernel: sages root@mcp:~ # echo 'weird, huh?' weird, huh? root@mcp:~ # !tail tail /var/log/messages Apr 27 15:50:25 mcp kernel: , Apr 27 15:50:25 mcp kernel: h Apr 27 15:50:26 mcp kernel: u Apr 27 15:50:26 mcp kernel: h Apr 27 15:50:26 mcp kernel: ? Apr 27 15:50:27 mcp kernel: ' Apr 27 15:50:27 mcp kernel: Apr 27 15:50:29 mcp kernel: ! Apr 27 15:50:30 mcp kernel: ta Apr 27 15:50:30 mcp kernel: il root@mcp:~ # From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 27 22:22:37 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CA60971A for ; Sat, 27 Apr 2013 22:22:37 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 896F21D1A for ; Sat, 27 Apr 2013 22:22:37 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id EB20F11F76; Sun, 28 Apr 2013 08:22:29 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro.local (c-71-196-188-222.hsd1.co.comcast.net [71.196.188.222]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BLS48429 (AUTH peterg@ptree32.com.au); Sun, 28 Apr 2013 08:22:27 +1000 Message-ID: <517C4F9C.9020200@freebsd.org> Date: Sat, 27 Apr 2013 16:22:20 -0600 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: Dan Mack Subject: Re: bhyve console question References: <20130427154625.X29498@coco.macktronics.com> In-Reply-To: <20130427154625.X29498@coco.macktronics.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Junkmail-Info: RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,SPF_SOFTFAIL X-Junkmail-Status: score=24/51, host=dommail.onthenet.com.au Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Apr 2013 22:22:37 -0000 Hi Dan, > For those of you tinkering with bhyve; quick question. I've noticed that > everything I type on the console ends up in /var/log/messages on the > guest. This includes all shell activity and login stuff. > > Is this expected behaviour ? What does your /etc/ttys/ look like ? I think if /dev/console is being used, then all traffic ends up in /var/log/messages. later, Peter.