From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 01:37:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D2CF203; Sun, 30 Nov 2014 01:37:14 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 174E5FBE; Sun, 30 Nov 2014 01:37:14 +0000 (UTC) Received: by mail-wi0-f182.google.com with SMTP id h11so14159160wiw.15 for ; Sat, 29 Nov 2014 17:37:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Lv0RSQjmHUXoCrOx6D6fZ40c1sA01waDWrmGt2ZweeU=; b=LMdI7lHEWOKotUBzrArwFyjIcz7bEp2ivXxYShA4RWSOTgte+CtDUCMuJDAB3rtoLO 9kVBpib2L3WlT32ce4gHHULTlFnbVyKbEsqxoT2JOhb+g0/zbmGtwMA3u4y6PLdgzU/f BFzHzL+lBqrAkenp96RUIxBDmNH5O6of4PTbI58LzJdWYILkIzoEc3Q+0y5R9Gnwkoj1 BpPQGFvShcuxaS+Ui6OsTgXc10OedMWu78E09ef/umvFp+ly4J0Ky0pKyVSTz84DolSF S0gHd7XZnbGWDmvX29iFL3br8HrUSm+UYY36gfoXtEVrCSKWxhSgSOT35jGnFQ2Ukzi/ irkw== MIME-Version: 1.0 X-Received: by 10.194.238.3 with SMTP id vg3mr43638223wjc.69.1417311431969; Sat, 29 Nov 2014 17:37:11 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sat, 29 Nov 2014 17:37:11 -0800 (PST) In-Reply-To: <5479D1BE.2010106@gmx.com> References: <5478E4C4.3080203@gmx.com> <5479D1BE.2010106@gmx.com> Date: Sat, 29 Nov 2014 17:37:11 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 01:37:14 -0000 Hi Nikos, On Sat, Nov 29, 2014 at 6:01 AM, Nikos Vassiliadis wrote: > > > On 11/29/14 02:05, Craig Rodrigues wrote: >> >> On Fri, Nov 28, 2014 at 1:10 PM, Nikos Vassiliadis > > wrote: >> >> Hi, >> >> I am trying to use PCI passthrough to pass the bge device to a guest >> but I am doing something wrong. I have tried several combinations of >> pptdevs like "2/0/0 2/0/1 2/0/2 2/0/3" but it always fails on 2/0/0. >> Any ideas what I could try? >> >> >> Did you read this: https://wiki.freebsd.org/bhyve/pci_passthru ? >> >> After following those instructions, what is the output of pciconf -l -v >> before and after enabling PCI passthru? >> On the host and in the guest? > > > > Before loading vmm.ko: >> >> hostb0@pci0:0:0:0: class=0x060000 card=0x064b1025 chip=0x01548086 >> rev=0x09 hdr=0x00 >> pcib1@pci0:0:1:0: class=0x060400 card=0x064b1025 chip=0x01518086 >> rev=0x09 hdr=0x01 >> vgapci1@pci0:0:2:0: class=0x030000 card=0x064b1025 chip=0x01668086 >> rev=0x09 hdr=0x00 >> none0@pci0:0:22:0: class=0x078000 card=0x064b1025 chip=0x1e3a8086 >> rev=0x04 hdr=0x00 >> ehci0@pci0:0:26:0: class=0x0c0320 card=0x064b1025 chip=0x1e2d8086 >> rev=0x04 hdr=0x00 >> hdac0@pci0:0:27:0: class=0x040300 card=0x064b1025 chip=0x1e208086 >> rev=0x04 hdr=0x00 >> pcib2@pci0:0:28:0: class=0x060400 card=0x064b1025 chip=0x1e108086 >> rev=0xc4 hdr=0x01 >> pcib3@pci0:0:28:1: class=0x060400 card=0x064b1025 chip=0x1e128086 >> rev=0xc4 hdr=0x01 >> ehci1@pci0:0:29:0: class=0x0c0320 card=0x064b1025 chip=0x1e268086 >> rev=0x04 hdr=0x00 >> isab0@pci0:0:31:0: class=0x060100 card=0x064b1025 chip=0x1e578086 >> rev=0x04 hdr=0x00 >> ahci0@pci0:0:31:2: class=0x010601 card=0x064b1025 chip=0x1e038086 >> rev=0x04 hdr=0x00 >> none1@pci0:0:31:3: class=0x0c0500 card=0x064b1025 chip=0x1e228086 >> rev=0x04 hdr=0x00 >> vgapci0@pci0:1:0:0: class=0x030000 card=0x06911025 chip=0x114010de >> rev=0xa1 hdr=0x00 >> none2@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >> rev=0x10 hdr=0x00 >> none3@pci0:2:0:1: class=0x080501 card=0x06471025 chip=0x16bc14e4 >> rev=0x10 hdr=0x00 >> none4@pci0:2:0:2: class=0x088000 card=0x06471025 chip=0x16be14e4 >> rev=0x10 hdr=0x00 >> none5@pci0:2:0:3: class=0x088000 card=0x06471025 chip=0x16bf14e4 >> rev=0x10 hdr=0x00 >> none6@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 > > > After loading vmm.ko: >> >> hostb0@pci0:0:0:0: class=0x060000 card=0x064b1025 chip=0x01548086 >> rev=0x09 hdr=0x00 >> pcib1@pci0:0:1:0: class=0x060400 card=0x064b1025 chip=0x01518086 >> rev=0x09 hdr=0x01 >> vgapci1@pci0:0:2:0: class=0x030000 card=0x064b1025 chip=0x01668086 >> rev=0x09 hdr=0x00 >> none0@pci0:0:22:0: class=0x078000 card=0x064b1025 chip=0x1e3a8086 >> rev=0x04 hdr=0x00 >> ehci0@pci0:0:26:0: class=0x0c0320 card=0x064b1025 chip=0x1e2d8086 >> rev=0x04 hdr=0x00 >> hdac0@pci0:0:27:0: class=0x040300 card=0x064b1025 chip=0x1e208086 >> rev=0x04 hdr=0x00 >> pcib2@pci0:0:28:0: class=0x060400 card=0x064b1025 chip=0x1e108086 >> rev=0xc4 hdr=0x01 >> pcib3@pci0:0:28:1: class=0x060400 card=0x064b1025 chip=0x1e128086 >> rev=0xc4 hdr=0x01 >> ehci1@pci0:0:29:0: class=0x0c0320 card=0x064b1025 chip=0x1e268086 >> rev=0x04 hdr=0x00 >> isab0@pci0:0:31:0: class=0x060100 card=0x064b1025 chip=0x1e578086 >> rev=0x04 hdr=0x00 >> ahci0@pci0:0:31:2: class=0x010601 card=0x064b1025 chip=0x1e038086 >> rev=0x04 hdr=0x00 >> none1@pci0:0:31:3: class=0x0c0500 card=0x064b1025 chip=0x1e228086 >> rev=0x04 hdr=0x00 >> vgapci0@pci0:1:0:0: class=0x030000 card=0x06911025 chip=0x114010de >> rev=0xa1 hdr=0x00 >> ppt0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >> rev=0x10 hdr=0x00 >> ppt1@pci0:2:0:1: class=0x080501 card=0x06471025 chip=0x16bc14e4 >> rev=0x10 hdr=0x00 >> ppt2@pci0:2:0:2: class=0x088000 card=0x06471025 chip=0x16be14e4 >> rev=0x10 hdr=0x00 >> ppt3@pci0:2:0:3: class=0x088000 card=0x06471025 chip=0x16bf14e4 >> rev=0x10 hdr=0x00 >> ppt4@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 > > > Then I start "bhyve ... -s 10,passthru,2/0/0 ..." and it spits: > "Unsupported MSI-X configuration: 2/0/0" > and quits. > The "Unsupported MSI-X configuration" referred to here is that bhyve doesn't emulate the 'Pending Bit Array'. In most cases this is not relevant because the PBA and the MSI-X tables are in different page frames. In this case the MSI-X tables are emulated and the pending bit array page is passed through to the guest. If the PBA and MSI-X tables are on the same page frame then you see this error. You can confirm this by doing 'pciconf -lvbc pci0:2:0:0'. Having said that the drivers I have seen don't rely at all on the PBA. So it may be possible that this works just fine with this patch: https://people.freebsd.org/~neel/patches/bhyve_ignore_unsupported_pba.patch If it does then I can add a tunable to bhyve to ignore this check. best Neel > This happens *only* with 2/0/0, that is the bge device. When I try to > passthrough this: >> >> none6@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 >> vendor = 'Broadcom Corporation' >> device = 'BCM4313 802.11b/g/n Wireless LAN Controller' >> class = network > > > it goes further, bhyve starts and the device is passed to the host but I > cannot use it from the VM. Actually my goal is to use the wifi adapter from > a VM. The bge device is just testing and I thought I should report it/ask > about it. > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 01:43:51 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C2B633C for ; Sun, 30 Nov 2014 01:43:51 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D31AB9 for ; Sun, 30 Nov 2014 01:43:51 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id l15so21537031wiw.2 for ; Sat, 29 Nov 2014 17:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6odazPFO7RjG6wVZziBEk/5LtxR7G7BZXFkdCFO82LA=; b=c7HbeY4TJg7GcUeChGRn4LopYozCdrGAMaMrTQWovGqOyZGz8C3t1Y6rLFfMuxSkDt jY45wfRk+iRcRThKpjmf+RXvAj6gOjfRD9LI9hx2sMqx4P5krEL1aWCoq9NKNbjEiJ2j MZ3syLd6LMZACRUKmY36xlRAjWx8MsB+BA2ifW+Gd4/DZDY5lSsysGB+a2Y6eDF6Qv71 OcZIGmeMIodfBtq+hUkkMORqoIlzf664pJgDeyxpVIRFCv9uyRL2QqGeTlIZmp4w0Hos z1WaohPWOmKoA0qYkVGihQtpuCGuZj+DbX9U29CIDk1nhWFcyazWwplr4IGSglIOgOWU 4L/w== MIME-Version: 1.0 X-Received: by 10.180.75.237 with SMTP id f13mr71469207wiw.69.1417311829520; Sat, 29 Nov 2014 17:43:49 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sat, 29 Nov 2014 17:43:49 -0800 (PST) In-Reply-To: <547A1454.2030901@gmx.com> References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> Date: Sat, 29 Nov 2014 17:43:49 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 01:43:51 -0000 Hi Nikos, On Sat, Nov 29, 2014 at 10:45 AM, Nikos Vassiliadis wrote: > > > On 11/29/14 18:38, Anish Gupta wrote: >> >> >Another thing I should mention is that "acpidump -t | grep DMAR" >> reports nothing. It is mentioned in the documentation but I didn't see >> until now. >> >> DMAR/VT-d has been supported for sometime now by Intel chipset. You want >> to check your BIOS set-up to enable VT-d. > > > There are very few setting in the BIOS setup and nothing related to VT-d. My > CPU and CPU settings are these: > >> CPU: Intel(R) Core(TM) i3-3110M CPU @ 2.40GHz (2394.61-MHz K8-class CPU) >> Origin = "GenuineIntel" Id = 0x306a9 Family = 0x6 Model = 0x3a >> Stepping = 9 >> >> Features=0xbfebfbff >> >> Features2=0x3dbae3bf >> AMD Features=0x28100800 >> AMD Features2=0x1 >> Structured Extended Features=0x281 >> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID >> TSC: P-state invariant, performance statistics > > > I have attached the output of "acpidump -t" in case you'd like to take a > look. > >> >Actually my goal is to use the wifi adapter from a VM. The bge device >> is just testing and I thought I should report it/ask about it. >> What is the error you see in VM for enabling wifi adapter? Is the >> corresponding driver getting attached to WiFi adapter? > > > When I load the kernel module, I get this: >> >> [ 542.919615] cfg80211: Calling CRDA to update world regulatory domain >> [ 542.932441] wl 0000:00:07.0: can't derive routing for PCI INT A >> [ 542.932449] wl 0000:00:07.0: PCI INT A: no GSI >> [ 543.002253] wlan0: Broadcom BCM4727 802.11 Hybrid Wireless Controller >> 6.30.223.248 (r487574) > Can you provide the output of 'pciconf -lvbc pci0:3:0:0' on the FreeBSD host? This is assuming pci0:3:0:0 is the wlan device being passed through based on an earlier email. best Neel > > After that, there is a wlan0 device but I am not sure whether it works. > There are no new kernel warnings or error messages but no matter what I do > (scan, create an AP etc) there is zero reaction. But that might a > problematic driver that the VM is using. That's on Linux 3.16.something. > > I have tried to load another driver and there is a complain regarding PCI > which is common to both drivers: >> >> [ 1515.692558] bcma-pci-bridge 0000:00:07.0: can't derive routing for PCI >> INT A >> [ 1515.692568] bcma-pci-bridge 0000:00:07.0: PCI INT A: no GSI > > Didnt test further with this driver. > > On FreeBSD, I tried to do the same with an ndis windows driver. It didn't > get very far because some windows drivers and ndis and amd64 don't like each > other very much:) > >> >Then I start "bhyve ... -s 10,passthru,2/0/0 ..." and it spits: >> "Unsupported MSI-X configuration: 2/0/0" >> and quits. >>> >>> ppt0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >>> rev=0x10 hdr=0x00 >> >> >> I will try to reproduce the problem if I get hold of this device. > > > Great, thank you. > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 02:26:46 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9EEC26F6 for ; Sun, 30 Nov 2014 02:26:46 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FCB75E5 for ; Sun, 30 Nov 2014 02:26:46 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id r20so14229527wiv.14 for ; Sat, 29 Nov 2014 18:26:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Jj6dsPq3MTQVQUkcefq6HyXOpGYruXT3X53lg4fC6UM=; b=I12goAcZ84CUufD4dEiSz+1fWVWKt/pSSmj8/DBKAfjo39R7alvnbh6/zdzK5JqiN+ EBmoguT8DBToKbhRgkepNd7IY97+Iwnc2OEkAyMtilqdh2zp5S6aAZOhq8wcO3zQFXfc 2KBIGWb/3Rw6YAI/1Xv92gbNGCnMYK0PwYvw34H32Gi5Sn7vWKzrBGD/tmXNoxM+ULgt Pp8uu3TtQ/zilobCmkI8WPQ3IU4NFDR5ZHEy6h4L/qc+x6AfMXSJ5KHvdt32jQpPjj74 Qk59bDr6ZBTiHD2tQulaO+kBxHoZQlTLBqooWcPMlGLMvoGXupJrP2aSB8IO7isS7E35 7HQQ== MIME-Version: 1.0 X-Received: by 10.180.75.237 with SMTP id f13mr71654497wiw.69.1417314404568; Sat, 29 Nov 2014 18:26:44 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sat, 29 Nov 2014 18:26:44 -0800 (PST) In-Reply-To: References: Date: Sat, 29 Nov 2014 18:26:44 -0800 Message-ID: Subject: Re: Unsupported cmd:c4 (ada0:ahcich0:0:0:0): READ_MUL. ACB: c4 00 00 00 00 40 00 00 00 00 01 00 From: Neel Natu To: =?UTF-8?Q?Ask_Bj=C3=B8rn_Hansen?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 02:26:46 -0000 Hi Bjorn, On Fri, Nov 28, 2014 at 2:42 AM, Ask Bj=C3=B8rn Hansen = wrote: > Hi everyone, > > I=E2=80=99ve spent some time off this week upgrading my build-my-personal= -NanoBSD-image tool from a VMware virtual machine to a physical box running= a 32-bit build-box under bhyve. > > So far so good =E2=80=94 all that has been working really well. > > My nanobsd image is only used on Soekris and PC Engines boxes and my prod= uction images are really small (some <100MB storage and ~32MB memory), so I= have a crufty kernel config that excluded a bunch of stuff I (thought I) d= on=E2=80=99t need. > > Now I=E2=80=99d like to run my built image under bhyve for testing. It=E2= =80=99s easier for me if the disk comes up as an ada device, so I=E2=80=99m= using ahci-hd instead of virtio-blk. With virtio-blk the system boots but = the device has the =E2=80=9Cwrong" name. > > With ahci-hd it can=E2=80=99t read my partition for some reason. > > I=E2=80=99ve tried with both file and zvol backing store. > > This is the error I=E2=80=99m getting: > > ada0: ATA-8 SATA 2.x device > ada0: Serial Number 123456 > ada0: 300.000MB/s transfers (SATA 2.x, PIO4, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 260MB (532480 512 byte sectors: 16H 63S/T 528C) > ada0: Previously was known as ad4 > Unsupported cmd:c4 > (ada0:ahcich0:0:0:0): READ_MUL. ACB: c4 00 c1 1f 08 40 = 00 00 00 00 04 00 > (ada0:ahcich0:0:0:0): CAM status: ATA Status Error > (ada0:ahcich0:0:0:0): ATA status: 41 (DRDY ERR), error: 04 (ABRT ) > (ada0:ahcich0:0:0:0): RES: 41 04 00 00 00 40 00 00 00 00 00 > (ada0:ahcich0:0:0:0): Retrying command > Unsupported cmd:c4 > > > Any suggestion for what I=E2=80=99m doing wrong? Did I compile my kernel = with some idiotic set of not-working options? > Are you, by any chance, setting the tunable 'hw.ata.ata_dma' to 0 to disable DMA? I ask this because in the dmesg above you have: ada0: 300.000MB/s transfers (SATA 2.x, PIO4, PIO 8192bytes) In my setup, I get this instead: ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) best Neel > > Ask > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@free= bsd.org" From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 02:31:29 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4596841; Sun, 30 Nov 2014 02:31:29 +0000 (UTC) Received: from mail-wi0-x230.google.com (mail-wi0-x230.google.com [IPv6:2a00:1450:400c:c05::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68839694; Sun, 30 Nov 2014 02:31:29 +0000 (UTC) Received: by mail-wi0-f176.google.com with SMTP id ex7so21482565wid.15 for ; Sat, 29 Nov 2014 18:31:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=KR8vMwGwVgQ9p04EZcxQTYi1uQa9ZQQKT9XTDdFXw9A=; b=ygvtKqM86WAOgrbR3VFh1ffJW3yuGGuOwYFWR+Nf2lJ9uqSbR5CKPa43Ebnp1Zorj6 RtnY62gYP215kiwAetHSKmt50x/Psg4GoPXgvRlCofYHBxFdOlRTipeT/8w0kZ9s7qNB cFXMOyHSDCjLD4rh4FkwfmO31mpnbJtbr5LS/auhMzOgKvPfH1qHqCSM0Wlj2X0Kug8v yULv0hp9EHWBIuNQ3I9aUUuvcx4nwOfaIGjq3kBl5ONejAiIHWHxt2/dIeezfRjghAbi /69jvGAUkGZK7FesFE2NPfHU/8FadWe5Dsl0sng6BbdJCn4E+Prs0ltQwvVHYof6mexu Qm9w== MIME-Version: 1.0 X-Received: by 10.194.243.164 with SMTP id wz4mr83201469wjc.129.1417314687606; Sat, 29 Nov 2014 18:31:27 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sat, 29 Nov 2014 18:31:27 -0800 (PST) In-Reply-To: <54789699.7030507@freebsd.org> References: <54773CF8.2020306@freebsd.org> <54789699.7030507@freebsd.org> Date: Sat, 29 Nov 2014 18:31:27 -0800 Message-ID: Subject: Re: bhyve failure From: Neel Natu To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 02:31:29 -0000 Hi Julian, On Fri, Nov 28, 2014 at 7:36 AM, Julian Elischer wrote: > On 11/28/14, 3:12 AM, Neel Natu wrote: >> >> Hi Julian, >> >> On Thu, Nov 27, 2014 at 7:02 AM, Julian Elischer >> wrote: >>> >>> I added WITNESS to a kernel I've been running and during boot I got this >>> result: >>> >>> vm exit[0] >>> reason VMX >>> rip 0xffffffff8078c1a7 >>> inst_length 7 >>> status 0 >>> exit_reason 2 >> >> That's a triple fault (from sys/amd64/vmm/intel/vmcs.h) >> >> #define EXIT_REASON_TRIPLE_FAULT 2 >> >> >>> qualification 0x0000000000000000 >>> inst_type 0 >>> inst_error 0 >>> Abort trap (core dumped) >>> >>> anyone have a decoder ring? >>> >> A few questions to narrow this down: >> >> Did you add WITNESS to the guest kernel or the host kernel when this >> happened? > > yes I added it to the guest kernel.. I suspect it ay be related to this, > especially since there was a kernel module loaded that was not recompiled. > I'm trying to ascertain now whether a withness kernel requires special > handling for its modules. > > >> >> Is this readily reproducible? > > don't know yet. > Its interesting that the guest went all the way to a triple fault instead of stopping at the double fault handler. I'll be happy to track this down if there is a reliably repro case (guest binaries would be sufficient). best Neel >> >> Can you share the host and guest revision numbers? > > > guest is custom sources based on stable-10 on the same day as 10.0 was > released.basically whatever svn revision# was on 10.0 but on the stable > branch. > host is > FreeBSD 10.1-BETA1 #0 r271671: Tue Sep 16 09:31:23 PDT 2014 > > > > >> >> best >> Neel >> >>> julian >>> _______________________________________________ >>> freebsd-virtualization@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >>> To unsubscribe, send any mail to >>> "freebsd-virtualization-unsubscribe@freebsd.org" > > From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 10:04:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 74A466A1; Sun, 30 Nov 2014 10:04:14 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 48C32F68; Sun, 30 Nov 2014 10:04:13 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sAUA48Aa039770 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 30 Nov 2014 02:04:11 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <547AEB93.3050600@freebsd.org> Date: Sun, 30 Nov 2014 18:04:03 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Craig Rodrigues Subject: Re: RFC: Enabling VIMAGE in GENERIC References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 10:04:14 -0000 On 11/29/14, 5:28 PM, Craig Rodrigues wrote: > On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer > wrote: > > > > > > also look at the following: (a little dated) > > > > > http://p4web.freebsd.org/@md=d&cd=//depot/projects/vimage/&cdf=//depot/projects/vimage/porting_to_vimage.txt&c=tO0@//depot/projects/vimage/porting_to_vimage.txt?ac=22 > > > This is a useful document. I put it on the wiki: > https://wiki.freebsd.org/VIMAGE/porting-to-vimage Thanks.. wow, did I actually know ALL that only 5 years ago? Scary. probbaly worth having someone who is currently active and up to date look at it to see if it's all still correct.. especially the module load/unload stuff. > > -- > Craig From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 15:37:28 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 133D2E48 for ; Sun, 30 Nov 2014 15:37:28 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E2AFF8C for ; Sun, 30 Nov 2014 15:37:27 +0000 (UTC) Received: from moby.local ([109.193.238.174]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MXIGf-1XR9VS3Cp3-00WEsN; Sun, 30 Nov 2014 16:37:19 +0100 Message-ID: <547B39A8.5040106@gmx.com> Date: Sun, 30 Nov 2014 16:37:12 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Craig Rodrigues , neel Natu Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:G1kr+PjmXp6Dm68fQd4fztYPeZ9rafZV2xB0YbdEc43viS7oKa+ RAKJp263U0PyVXR+jlxelkRRistUoFD5PrpAXvf/BPpbM7+nwdEzrTiBpzHXmpU4NRPSn5z ALEIhUh5/hIatU8gJzh4hCu8C9PqzwcMNwO+lmXXhvqESqguErs95xajcHB0fB00afo39Q0 iryUHesCqOsyTe9q+qWXg== X-UI-Out-Filterresults: notjunk:1; Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 15:37:28 -0000 Hi, On 11/30/14 00:48, Craig Rodrigues wrote: > I'm not sure, but according to this datasheet: > > http://ark.intel.com/products/65700/Intel-Core-i3-3110M-Processor-3M-Cache-2_40-GHz > > this CPU does not support VT-d, which is needed for PCI passthru. Indeed. Good catch, Neel should this be the case? From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 15:50:36 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 45F72E6; Sun, 30 Nov 2014 15:50:36 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B80DFC2; Sun, 30 Nov 2014 15:50:35 +0000 (UTC) Received: from moby.local ([109.193.238.174]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MJByE-1XsLsc0Fv0-002s6q; Sun, 30 Nov 2014 16:50:28 +0100 Message-ID: <547B3CC2.4040301@gmx.com> Date: Sun, 30 Nov 2014 16:50:26 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Neel Natu Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 References: <5478E4C4.3080203@gmx.com> <5479D1BE.2010106@gmx.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:6EQ43SN6GMvcGIRmr5d6BQQShsqV8GUQKIbgNwd1N0PwtK1ZJ13 E+3p5Rr8vQ3OAnb0Q86U1kpCKqNjYnpJxj0aaF9QgL/lrAZEEAgY9DGsUKAGuf5VdcmNknX +UXvEVDqt049wMfLGmSu3Zhs2e78QK263MoaDdK3YcS0OIDyqyRJaj2hbRvhpE5sPQHnqDu 7k4ARYQsMat44ynEVszug== X-UI-Out-Filterresults: notjunk:1; Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 15:50:36 -0000 Hi, On 11/30/14 02:37, Neel Natu wrote: > The "Unsupported MSI-X configuration" referred to here is that bhyve > doesn't emulate the 'Pending Bit Array'. > > In most cases this is not relevant because the PBA and the MSI-X > tables are in different page frames. In this case the MSI-X tables are > emulated and the pending bit array page is passed through to the > guest. > > If the PBA and MSI-X tables are on the same page frame then you see > this error. You can confirm this by doing 'pciconf -lvbc pci0:2:0:0'. Here is the ouput: > bge0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 rev=0x10 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetLink BCM57785 Gigabit Ethernet PCIe' > class = network > subclass = ethernet > bar [10] = type Prefetchable Memory, range 64, base 0xb3430000, size 65536, enabled > bar [18] = type Prefetchable Memory, range 64, base 0xb3440000, size 65536, enabled > cap 01[48] = powerspec 3 supports D0 D3 current D0 > cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message > cap 11[a0] = MSI-X supports 5 messages > Table in map 0x18[0x0], PBA in map 0x18[0x120] > cap 10[ac] = PCI-Express 2 endpoint max data 128(128) link x1(x1) > speed 2.5(2.5) ASPM L1(L0s/L1) > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected > ecap 0003[13c] = Serial 1 0000208984c6b5ed > ecap 0004[150] = Power Budgeting 1 > ecap 0002[160] = VC 1 max VC0 > Having said that the drivers I have seen don't rely at all on the PBA. > So it may be possible that this works just fine with this patch: > https://people.freebsd.org/~neel/patches/bhyve_ignore_unsupported_pba.patch > > If it does then I can add a tunable to bhyve to ignore this check. It seems a tunable can help. I tried the patch and now bhyve continues and indeed I see the bge device in my VM. What is interesting/strange is that the device is partly functioning. For example, it does detect a link change when I remove the cable but that's all. The kernel also complains with a "bge watchdog timeout" message. In case there is no hardware support for VT-d, as Craig noticed, how could this behaviour be explained? Why does link detection work? Again, thank you both! From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 15:58:34 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDDCA253 for ; Sun, 30 Nov 2014 15:58:34 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8448E17D for ; Sun, 30 Nov 2014 15:58:34 +0000 (UTC) Received: from moby.local ([109.193.238.174]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0MHoWj-1Xtiuj1LmX-003hcO; Sun, 30 Nov 2014 16:58:32 +0100 Message-ID: <547B3EA6.1070200@gmx.com> Date: Sun, 30 Nov 2014 16:58:30 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Neel Natu Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:b8PfJBfpgfE4W/4hFxMkLFokaII/dX14q25gKI4cbf1kqcAmi2N aI2xFQ3dUNy8COWnvy5AECka6JThpJN9MQk4tEK2hxjd/rxC/w9YbCcz4PnffGpgJAfGmyH zH1HcPj7bczt0kjQxxxMsJhbs3rXd7fKvv6+udEq6vSE4YWeHG5QjZN6udWNZ3RVC9zcFzn cgOywgs9dOKwShxyfYXUw== X-UI-Out-Filterresults: notjunk:1; Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 15:58:35 -0000 Hi, On 11/30/14 02:43, Neel Natu wrote: > Can you provide the output of 'pciconf -lvbc pci0:3:0:0' on the FreeBSD host? > > This is assuming pci0:3:0:0 is the wlan device being passed through > based on an earlier email. Yes, it is. Here is the output: > none0@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 rev=0x01 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'BCM4313 802.11b/g/n Wireless LAN Controller' > class = network > bar [10] = type Memory, range 64, base 0xb3500000, size 16384, enabled > cap 01[40] = powerspec 3 supports D0 D1 D2 D3 current D0 > cap 09[58] = vendor (length 120) > cap 05[48] = MSI supports 1 message, 64 bit > cap 10[d0] = PCI-Express 1 endpoint max data 128(128) link x1(x1) > speed 2.5(2.5) ASPM L1(L1) > ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected > ecap 0002[13c] = VC 1 max VC0 > ecap 0003[160] = Serial 1 0000dcffff210c84 > ecap 0004[16c] = Power Budgeting 1 Thanks, Nikos From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 19:30:48 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D446B14 for ; Sun, 30 Nov 2014 19:30:48 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C728947 for ; Sun, 30 Nov 2014 19:30:48 +0000 (UTC) Received: by mail-wg0-f43.google.com with SMTP id l18so12399480wgh.30 for ; Sun, 30 Nov 2014 11:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=P3fFCXX0K8+MuWCGb+itXU7+ICEpkTspVB8bKeQdXY4=; b=nrcqA79K4S4F2gnawalyX/jHTqg9xJqjzCipZIPBv6WcgSdFUprZxnOyV9VpoWXEHw qJacEHYe3v72Ds811M7z5D4pGrQ2YgtclJzeIIkR9bcmDmCmQ4sxo/bo8qr3Lc1MZu1i 9wLuD30kFi11d49w440/stsZMi5tMBPRxEQW+iEkJpVB0mXWZ3TmGEbhSMCoFwvjRKiI xA5CH3aQWXFCExy3TCE8ifOIMLMOXAh45wPgCQb1cpXM9o9/bXFGCy9+jzAqfNagdplg Yg3/DTEAXKrn/ImzIAA85qTVOlS8o6zEJ8Sztole9XOZjb0u6JR3lIztzFkFT7VKzDjz mOww== MIME-Version: 1.0 X-Received: by 10.180.75.237 with SMTP id f13mr76995158wiw.69.1417375846550; Sun, 30 Nov 2014 11:30:46 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sun, 30 Nov 2014 11:30:46 -0800 (PST) In-Reply-To: <547B3EA6.1070200@gmx.com> References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> <547B3EA6.1070200@gmx.com> Date: Sun, 30 Nov 2014 11:30:46 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 19:30:48 -0000 Hi Nikos, On Sun, Nov 30, 2014 at 7:58 AM, Nikos Vassiliadis wrote: > Hi, > > On 11/30/14 02:43, Neel Natu wrote: >> >> Can you provide the output of 'pciconf -lvbc pci0:3:0:0' on the FreeBSD >> host? >> >> This is assuming pci0:3:0:0 is the wlan device being passed through >> based on an earlier email. > > > Yes, it is. Here is the output: >> >> none0@pci0:3:0:0: class=0x028000 card=0xe042105b chip=0x472714e4 >> rev=0x01 hdr=0x00 >> vendor = 'Broadcom Corporation' >> device = 'BCM4313 802.11b/g/n Wireless LAN Controller' >> class = network >> bar [10] = type Memory, range 64, base 0xb3500000, size 16384, >> enabled >> cap 01[40] = powerspec 3 supports D0 D1 D2 D3 current D0 >> cap 09[58] = vendor (length 120) >> cap 05[48] = MSI supports 1 message, 64 bit Ok, this device does have MSI which is what I wanted to confirm. Its very likely that the device doesn't work when passed to the guest because your system doesn't have VT-d. best Neel >> cap 10[d0] = PCI-Express 1 endpoint max data 128(128) link x1(x1) >> speed 2.5(2.5) ASPM L1(L1) >> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 0 corrected >> ecap 0002[13c] = VC 1 max VC0 >> ecap 0003[160] = Serial 1 0000dcffff210c84 >> ecap 0004[16c] = Power Budgeting 1 > > > Thanks, Nikos From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 19:37:29 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09A95E52; Sun, 30 Nov 2014 19:37:29 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F918A2A; Sun, 30 Nov 2014 19:37:28 +0000 (UTC) Received: by mail-wi0-f177.google.com with SMTP id l15so15332673wiw.10 for ; Sun, 30 Nov 2014 11:37:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K1I8eTkPZwBXO9CL2wi4n65bSsxxWR7/W6NAxESdyzg=; b=ETZDDr6WWuaBAkY2QwuFuHoI8YmL7HAIfeYCsYhEnap2W9nZ8rtlLvd/6jxfCsYXw7 3xWCoXDHpTtsR9t8t8lt06B2cLPHwkFXKshCTBh0vCIISKXgHLb7sbCmBBdSlORKtEL5 noPvlFrmYspCREzeSx96XhURiKMJ5Lx7fsHXzHq/RDBfAGajUC/FF7fr4xrpeIBUU6do 8wjVHbYhIbLuBViZdlEFgQNMR8z2+PSKyNACz8gP38Ya+TrUWEWBIvaj/Jg1qT9pqRvM ViPCGhMfIF4yc3ZRO3Fg/lvouwB7X46zNDWID6WTOZvcKlmjaSFymLsmEKbmlPR4bIRh r3wA== MIME-Version: 1.0 X-Received: by 10.180.98.100 with SMTP id eh4mr44953790wib.54.1417376247109; Sun, 30 Nov 2014 11:37:27 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sun, 30 Nov 2014 11:37:27 -0800 (PST) In-Reply-To: <547B3CC2.4040301@gmx.com> References: <5478E4C4.3080203@gmx.com> <5479D1BE.2010106@gmx.com> <547B3CC2.4040301@gmx.com> Date: Sun, 30 Nov 2014 11:37:27 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 19:37:29 -0000 Hi Nikos, On Sun, Nov 30, 2014 at 7:50 AM, Nikos Vassiliadis wrote: > Hi, > > On 11/30/14 02:37, Neel Natu wrote: >> >> The "Unsupported MSI-X configuration" referred to here is that bhyve >> doesn't emulate the 'Pending Bit Array'. >> >> In most cases this is not relevant because the PBA and the MSI-X >> tables are in different page frames. In this case the MSI-X tables are >> emulated and the pending bit array page is passed through to the >> guest. >> >> If the PBA and MSI-X tables are on the same page frame then you see >> this error. You can confirm this by doing 'pciconf -lvbc pci0:2:0:0'. > > > Here is the ouput: >> >> bge0@pci0:2:0:0: class=0x020000 card=0x06471025 chip=0x16b514e4 >> rev=0x10 hdr=0x00 >> vendor = 'Broadcom Corporation' >> device = 'NetLink BCM57785 Gigabit Ethernet PCIe' >> class = network >> subclass = ethernet >> bar [10] = type Prefetchable Memory, range 64, base 0xb3430000, size >> 65536, enabled >> bar [18] = type Prefetchable Memory, range 64, base 0xb3440000, size >> 65536, enabled >> cap 01[48] = powerspec 3 supports D0 D3 current D0 >> cap 05[58] = MSI supports 8 messages, 64 bit enabled with 1 message >> cap 11[a0] = MSI-X supports 5 messages >> Table in map 0x18[0x0], PBA in map 0x18[0x120] Indeed, the MSI-X tables and PBA are both in the first page in the BAR. >> cap 10[ac] = PCI-Express 2 endpoint max data 128(128) link x1(x1) >> speed 2.5(2.5) ASPM L1(L0s/L1) >> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected >> ecap 0003[13c] = Serial 1 0000208984c6b5ed >> ecap 0004[150] = Power Budgeting 1 >> ecap 0002[160] = VC 1 max VC0 > > > > >> Having said that the drivers I have seen don't rely at all on the PBA. >> So it may be possible that this works just fine with this patch: >> >> https://people.freebsd.org/~neel/patches/bhyve_ignore_unsupported_pba.patch >> >> If it does then I can add a tunable to bhyve to ignore this check. > > > It seems a tunable can help. > > I tried the patch and now bhyve continues and indeed I see the bge device in > my VM. What is interesting/strange is that the device is partly functioning. > For example, it does detect a link change when I remove the cable but that's > all. The kernel also complains with a "bge watchdog timeout" message. > > In case there is no hardware support for VT-d, as Craig noticed, how could > this behaviour be explained? Why does link detection work? > I suspect that link status detection is done by polling device registers in its MMIO space which is mapped in the guest's address space. The MMIO accesses initiated by the guest will work independent of VT-d. It is only the reads/writes to system memory initiated by the device that won't work properly without VT-d. best Neel > Again, thank you both! From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 19:42:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CE3DEF7; Sun, 30 Nov 2014 19:42:06 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97A0BADE; Sun, 30 Nov 2014 19:42:05 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id r20so15372119wiv.2 for ; Sun, 30 Nov 2014 11:42:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BvAnlLyDHW9diwLT7k4E+pziywhz6U0rABooc1F7XgY=; b=ptXtKrly6vpOsHJOUV5yBOZnIUvwWlRAP8ByuRSnym2p5GrN2rq/AgmfBv2fvdFmuz m/6BlBKo6LJyqITAWruKNxuLoLouTCAsW4SnF/iVWYiqbXIXEzdet/VFpqlKwYbVGrAE CnVtsIA9yXpequoxOfPmAX5MnjvfGFeSKcEvCC0ByXFgGOhRIdyve1X4km/n0Z97Mlpu MfKZp5iocB3YZvzKWYPVb/Nx+mZfG25bkoiAsjQxLDLxJbNwkgpdYUCA/5HkpBxM2SrK dNPt9L2lLDlU6S4jLhpfqvsKLL2T2JCyyWVP2pwrKj1k8l9CGhdKbMFx2pCTnvDGmY5Y H7fA== MIME-Version: 1.0 X-Received: by 10.194.243.164 with SMTP id wz4mr89072949wjc.129.1417376524066; Sun, 30 Nov 2014 11:42:04 -0800 (PST) Received: by 10.27.12.168 with HTTP; Sun, 30 Nov 2014 11:42:04 -0800 (PST) In-Reply-To: <547B39A8.5040106@gmx.com> References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> <547B39A8.5040106@gmx.com> Date: Sun, 30 Nov 2014 11:42:04 -0800 Message-ID: Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 From: Neel Natu To: Nikos Vassiliadis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 19:42:06 -0000 Hi Nikos, On Sun, Nov 30, 2014 at 7:37 AM, Nikos Vassiliadis wrote: > Hi, > > On 11/30/14 00:48, Craig Rodrigues wrote: >> >> I'm not sure, but according to this datasheet: >> >> >> http://ark.intel.com/products/65700/Intel-Core-i3-3110M-Processor-3M-Cache-2_40-GHz >> >> this CPU does not support VT-d, which is needed for PCI passthru. > > > Indeed. Good catch, Neel should this be the case? Definitely! vmm.ko should probably not even attach the ppt driver to "pptdevs" if an IOMMU is absent. best Neel From owner-freebsd-virtualization@FreeBSD.ORG Sun Nov 30 23:17:44 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48B47892; Sun, 30 Nov 2014 23:17:44 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D40C7F2; Sun, 30 Nov 2014 23:17:43 +0000 (UTC) Received: from moby.local ([109.193.238.174]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0MQvDO-1XTDJb29Hy-00UIkm; Mon, 01 Dec 2014 00:17:41 +0100 Message-ID: <547BA58B.9050204@gmx.com> Date: Mon, 01 Dec 2014 00:17:31 +0100 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Neel Natu Subject: Re: bhyve: Unsupported MSI-X configuration: 2/0/0 References: <5478E4C4.3080203@gmx.com> <1B9C5228-5549-412C-AAE9-5053C86EDB9D@gmail.com> <5479DBD8.1090009@gmx.com> <547A1454.2030901@gmx.com> <547B39A8.5040106@gmx.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:SefSw2+Q/tvDHP4YcFb94GYwePeUKHjd+DR09yrAdsuTzaQVgSS z4hgVVMlLdf/W/duGyEFYkvZQjrNH4av8pZnlpMEKzewL3z0LNXFTDEftkbdChDcIVt4qbw KoAef63etLsHHXpiCg0Gaz4mdjXqZSavvZtvNtMuXaEx1L65SNHWhmB3rQ0avbIlE+qcjEw HTCeM2XEeO8tmMf+bUpTQ== X-UI-Out-Filterresults: notjunk:1; Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Nov 2014 23:17:44 -0000 On 11/30/14 20:42, Neel Natu wrote: >>> this CPU does not support VT-d, which is needed for PCI passthru. >> >> >> Indeed. Good catch, Neel should this be the case? > > Definitely! > > vmm.ko should probably not even attach the ppt driver to "pptdevs" if > an IOMMU is absent. OK, the mystery is solved then. Thank you all for your answers PS: I just started using bhyve and it's amazing to say the least! Amazing work people! From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 2 04:07:38 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A00E726; Tue, 2 Dec 2014 04:07:38 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE172980; Tue, 2 Dec 2014 04:07:37 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:57887 helo=[172.27.111.3]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82) (envelope-from ) id 1XvekQ-0006ZE-Ts; Mon, 01 Dec 2014 23:07:35 -0500 From: "George Neville-Neil" To: "Julian Elischer" Subject: Re: Enabling VIMAGE in GENERIC Date: Mon, 01 Dec 2014 23:07:34 -0500 Message-ID: In-Reply-To: <547AEB93.3050600@freebsd.org> References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: MailMate (1.8r4576) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 04:07:38 -0000 On 30 Nov 2014, at 5:04, Julian Elischer wrote: > On 11/29/14, 5:28 PM, Craig Rodrigues wrote: >> On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer > > wrote: >> > >> > >> > also look at the following: (a little dated) >> > >> > = >> http://p4web.freebsd.org/@md=3Dd&cd=3D//depot/projects/vimage/&cdf=3D/= /depot/projects/vimage/porting_to_vimage.txt&c=3DtO0@//depot/projects/vim= age/porting_to_vimage.txt?ac=3D22 >> >> >> This is a useful document. I put it on the wiki: = >> https://wiki.freebsd.org/VIMAGE/porting-to-vimage > > Thanks.. wow, did I actually know ALL that only 5 years ago? > Scary. probbaly worth having someone who is currently active and up = > to date look at it to see if it's all still correct.. > especially the module load/unload stuff. > >> >> -- >> Craig > On a slight tangent. I ran VIMAGE kernels vs. non VIMAGE kernels for = both a VANILLA kernel and a PF kernel (PF on but no rules) as a quick smoke test today. The = raw forwarding performance was unchanged between kernels with and without VIMAGE on a 10G based = system in the Sentex lab (lion1). I will be doing a bit more work in this area and will then put = up some results in my netperf github repo. The tests are easy enough to run if you have 3 systems, and Conductor = installed. The source, sink and dut config files are all there to be updated and tried. Best, George From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 2 04:13:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B3828E2; Tue, 2 Dec 2014 04:13:00 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CF8F9A41; Tue, 2 Dec 2014 04:12:59 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-246-72.lns20.per2.internode.on.net [121.45.246.72]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sB24CliN048450 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 1 Dec 2014 20:12:50 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <547D3C3A.5030408@freebsd.org> Date: Tue, 02 Dec 2014 12:12:42 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: George Neville-Neil Subject: Re: Enabling VIMAGE in GENERIC References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 04:13:00 -0000 On 12/2/14, 12:07 PM, George Neville-Neil wrote: > On 30 Nov 2014, at 5:04, Julian Elischer wrote: > >> On 11/29/14, 5:28 PM, Craig Rodrigues wrote: >>> On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer >>> > wrote: >>> > >>> > >>> > also look at the following: (a little dated) >>> > >>> > >>> http://p4web.freebsd.org/@md=d&cd=//depot/projects/vimage/&cdf=//depot/projects/vimage/porting_to_vimage.txt&c=tO0@//depot/projects/vimage/porting_to_vimage.txt?ac=22 >>> >>> >>> This is a useful document. I put it on the wiki: >>> https://wiki.freebsd.org/VIMAGE/porting-to-vimage >> >> Thanks.. wow, did I actually know ALL that only 5 years ago? >> Scary. probbaly worth having someone who is currently active and >> up to date look at it to see if it's all still correct.. >> especially the module load/unload stuff. >> >>> >>> -- >>> Craig >> > > > On a slight tangent. I ran VIMAGE kernels vs. non VIMAGE kernels > for both a VANILLA kernel > and a PF kernel (PF on but no rules) as a quick smoke test today. > The raw forwarding performance > was unchanged between kernels with and without VIMAGE on a 10G based > system in the Sentex lab > (lion1). I will be doing a bit more work in this area and will then > put up some results in my > netperf github repo. > > The tests are easy enough to run if you have 3 systems, and > Conductor installed. The source, sink > and dut config files are all there to be updated and tried. > > Best, > George > > the interesting benchmarks are if you have multiple sessions and spread them across multiple vimage jails, and compare that with the same number of sessions crowded onto a single machine.. lock contention goes down of course so things can actually get faster. From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 2 05:00:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24E6B88D; Tue, 2 Dec 2014 05:00:00 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E651FDD8; Tue, 2 Dec 2014 04:59:59 +0000 (UTC) Received: from pool-96-250-5-187.nycmny.fios.verizon.net ([96.250.5.187]:58967 helo=[172.27.111.3]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82) (envelope-from ) id 1XvfZ7-0003nJ-KW; Mon, 01 Dec 2014 23:59:57 -0500 From: "George Neville-Neil" To: "Julian Elischer" Subject: Re: Enabling VIMAGE in GENERIC Date: Mon, 01 Dec 2014 23:59:56 -0500 Message-ID: <84B51B87-22E2-4A8D-BB31-DFB1ADBAD05E@neville-neil.com> In-Reply-To: <547D3C3A.5030408@freebsd.org> References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> <547D3C3A.5030408@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; format=flowed Content-Transfer-Encoding: quoted-printable X-Mailer: MailMate (1.8r4576) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 05:00:00 -0000 On 1 Dec 2014, at 23:12, Julian Elischer wrote: > On 12/2/14, 12:07 PM, George Neville-Neil wrote: >> On 30 Nov 2014, at 5:04, Julian Elischer wrote: >> >>> On 11/29/14, 5:28 PM, Craig Rodrigues wrote: >>>> On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer = >>>> > wrote: >>>> > >>>> > >>>> > also look at the following: (a little dated) >>>> > >>>> > = >>>> http://p4web.freebsd.org/@md=3Dd&cd=3D//depot/projects/vimage/&cdf=3D= //depot/projects/vimage/porting_to_vimage.txt&c=3DtO0@//depot/projects/vi= mage/porting_to_vimage.txt?ac=3D22 >>>> >>>> >>>> This is a useful document. I put it on the wiki: = >>>> https://wiki.freebsd.org/VIMAGE/porting-to-vimage >>> >>> Thanks.. wow, did I actually know ALL that only 5 years ago? >>> Scary. probbaly worth having someone who is currently active and up = >>> to date look at it to see if it's all still correct.. >>> especially the module load/unload stuff. >>> >>>> >>>> -- = >>>> Craig >>> >> >> >> On a slight tangent. I ran VIMAGE kernels vs. non VIMAGE kernels for = >> both a VANILLA kernel >> and a PF kernel (PF on but no rules) as a quick smoke test today. The = >> raw forwarding performance >> was unchanged between kernels with and without VIMAGE on a 10G based = >> system in the Sentex lab >> (lion1). I will be doing a bit more work in this area and will then = >> put up some results in my >> netperf github repo. >> >> The tests are easy enough to run if you have 3 systems, and Conductor = >> installed. The source, sink >> and dut config files are all there to be updated and tried. >> >> Best, >> George >> >> > the interesting benchmarks are if you have multiple sessions and = > spread them across multiple vimage jails, and compare that with the = > same number of sessions crowded onto a single machine.. > > lock contention goes down of course so things can actually get faster. All in good time. Best, George From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 2 11:31:43 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5017160C; Tue, 2 Dec 2014 11:31:43 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F3D40BBB; Tue, 2 Dec 2014 11:31:42 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 94C5D25D388C; Tue, 2 Dec 2014 11:31:39 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id BFE3FC77070; Tue, 2 Dec 2014 11:31:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id RCyHi4c-dWrR; Tue, 2 Dec 2014 11:31:37 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6] (orange-tun0-ula.sbone.de [IPv6:fde9:577b:c1a9:4420:cabc:c8ff:fe8b:4fe6]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 2B412C7706C; Tue, 2 Dec 2014 11:31:35 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: RFC: Enabling VIMAGE in GENERIC From: "Bjoern A. Zeeb" In-Reply-To: <547AEB93.3050600@freebsd.org> Date: Tue, 2 Dec 2014 11:31:34 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.1878.6) Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 11:31:43 -0000 On 30 Nov 2014, at 10:04 , Julian Elischer wrote: > On 11/29/14, 5:28 PM, Craig Rodrigues wrote: >> On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer > wrote: >> > >> > >> > also look at the following: (a little dated) >> > >> > = http://p4web.freebsd.org/@md=3Dd&cd=3D//depot/projects/vimage/&cdf=3D//dep= ot/projects/vimage/porting_to_vimage.txt&c=3DtO0@//depot/projects/vimage/p= orting_to_vimage.txt?ac=3D22 >>=20 >>=20 >> This is a useful document. I put it on the wiki: = https://wiki.freebsd.org/VIMAGE/porting-to-vimage >=20 > Thanks.. wow, did I actually know ALL that only 5 years ago? > Scary. probbaly worth having someone who is currently active and up = to date look at it to see if it's all still correct.. > especially the module load/unload stuff. Yeah I popped it up in a browser window to read through it once I have a = short break to do that. =97=20 Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 From owner-freebsd-virtualization@FreeBSD.ORG Tue Dec 2 15:14:39 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44DCE7BD; Tue, 2 Dec 2014 15:14:39 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EA01752; Tue, 2 Dec 2014 15:14:38 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-246-72.lns20.per2.internode.on.net [121.45.246.72]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id sB2FEWpa050896 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 2 Dec 2014 07:14:35 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <547DD753.1060105@freebsd.org> Date: Tue, 02 Dec 2014 23:14:27 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: "Bjoern A. Zeeb" Subject: Re: RFC: Enabling VIMAGE in GENERIC References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 15:14:39 -0000 On 12/2/14, 7:31 PM, Bjoern A. Zeeb wrote: > On 30 Nov 2014, at 10:04 , Julian Elischer wrote: > >> On 11/29/14, 5:28 PM, Craig Rodrigues wrote: >>> On Mon, Nov 24, 2014 at 9:03 AM, Julian Elischer > wrote: >>>> >>>> also look at the following: (a little dated) >>>> >>>> http://p4web.freebsd.org/@md=d&cd=//depot/projects/vimage/&cdf=//depot/projects/vimage/porting_to_vimage.txt&c=tO0@//depot/projects/vimage/porting_to_vimage.txt?ac=22 >>> >>> This is a useful document. I put it on the wiki: https://wiki.freebsd.org/VIMAGE/porting-to-vimage >> Thanks.. wow, did I actually know ALL that only 5 years ago? >> Scary. probbaly worth having someone who is currently active and up to date look at it to see if it's all still correct.. >> especially the module load/unload stuff. > Yeah I popped it up in a browser window to read through it once I have a short break to do that. thanks.. I remember learning most of that stuff the hard way. but kids and work have pretty much made me forget it again and it may have gotten out of date. > > — > Bjoern A. Zeeb "Come on. Learn, goddamn it.", WarGames, 1983 > > > > From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 02:41:14 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 034F9B5D for ; Wed, 3 Dec 2014 02:41:14 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CB631B88 for ; Wed, 3 Dec 2014 02:41:13 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sB32fChW096483 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 2 Dec 2014 18:41:13 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sB32fCTw096482 for freebsd-virtualization@freebsd.org; Tue, 2 Dec 2014 18:41:12 -0800 (PST) (envelope-from jmg) Date: Tue, 2 Dec 2014 18:41:12 -0800 From: John-Mark Gurney To: freebsd-virtualization@freebsd.org Subject: FreeBSD i386 guest doesn't see console... Message-ID: <20141203024112.GF99957@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 02 Dec 2014 18:41:13 -0800 (PST) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 02:41:14 -0000 So, I tried to boot a i386 guest under bhyve, and no console was detected... an amd64 guest has console fine... This is w/ a resonably recent current i386 kernel (r275062)... Any suggestions? i386 guest dmesg: Hypervisor: Origin = "bhyve bhyve " real memory = 536870912 (512 MB) avail memory = 495017984 (472 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: [...] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 on acpi0 (no second line, no output after loading kernel when running vmrun.sh)... amd64 guest dmesg: Hypervisor: Origin = "bhyve bhyve " real memory = 18253611008 (17408 MB) avail memory = 16573952000 (15806 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: [...] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) I can provide more if you'd like... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 03:27:33 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3F1A81B for ; Wed, 3 Dec 2014 03:27:33 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id 979BCD2 for ; Wed, 3 Dec 2014 03:27:32 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 5130312648; Wed, 3 Dec 2014 13:27:25 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAF10028 (AUTH peterg@ptree32.com.au); Wed, 3 Dec 2014 13:27:24 +1000 Message-ID: <547E8319.9070509@freebsd.org> Date: Tue, 02 Dec 2014 19:27:21 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John-Mark Gurney , freebsd-virtualization@freebsd.org Subject: Re: FreeBSD i386 guest doesn't see console... References: <20141203024112.GF99957@funkthat.com> In-Reply-To: <20141203024112.GF99957@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 03:27:33 -0000 Hi John-Mark, > This is w/ a resonably recent current i386 kernel (r275062) I verified the r274463 i386 snapshot boots fine. That's not quite as recent, but: >... Any suggestions? > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 on acpi0 ... > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (9600,n,8,1) The "flags 0x10" text isn't being displayed in the i386 dmesg. This is derived from /boot/device.hints i.e. ... hint.uart.0.flags=0x10 ... You can see this from bhyveload by issuing a 'show' command to dump all the environment, including what was read in from device.hints by the loader, or "more /boot/device.hints". It may be possible that this file isn't being populated in your image, or being read in by bhyveload. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 15:11:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C0E6B36 for ; Wed, 3 Dec 2014 15:11:00 +0000 (UTC) Received: from potato.growveg.org (potato.growveg.org [62.49.247.163]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DA816CF for ; Wed, 3 Dec 2014 15:11:00 +0000 (UTC) Received: from john by potato.growveg.org with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1XwB0I-0005cn-Jv for freebsd-virtualization@freebsd.org; Wed, 03 Dec 2014 14:34:06 +0000 Date: Wed, 3 Dec 2014 14:34:06 +0000 From: John To: freebsd-virtualization@freebsd.org Subject: odd bhyve <-> ubuntu 14.04 problem Message-ID: <20141203143406.GA29037@potato.growveg.org> Reply-To: freebsd-lists@potato.growveg.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Sender: John X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: john@potato.growveg.org X-SA-Exim-Scanned: No (on potato.growveg.org); SAEximRunCond expanded to false X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 15:11:00 -0000 Hello list, I upgraded the host from freebsd-10-RC3 to 10.1-R and after that upgraded bhyve to grub2-bhyve-0.30 today and since then I've found that when booting ubuntu 14.04.1 LTS guests, the ethernet interface (ethN where N is a number) is now variable. This is an issue because if I stop a VM and say reboot the host, I have to hack about with the VM instead of just booting it. Previously the VM always saw the virtual network interface as eth0. Now, it's anything but that. bhyve has been upgraded, the VM hasn't. How can I control this behaviour? I want eth0 to remain as eth0. These are all VMs that were eth0 before the bhyve upgrade. It doesn't appear to matter what tap interface is used. I don't have this issue with freebsd guests on the freebsd host, just ubuntu 14.04.1 LTS. I define the bridge and 9 tap interfaces at boot, in case this is relevant. Please can anyone shed some light? thanks, -- John From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 17:49:50 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF8EF237; Wed, 3 Dec 2014 17:49:50 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ACD421C; Wed, 3 Dec 2014 17:49:50 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sB3HnmAl007997 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Dec 2014 09:49:48 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sB3HnmHV007996; Wed, 3 Dec 2014 09:49:48 -0800 (PST) (envelope-from jmg) Date: Wed, 3 Dec 2014 09:49:48 -0800 From: John-Mark Gurney To: Peter Grehan Subject: Re: FreeBSD i386 guest doesn't see console... Message-ID: <20141203174947.GG99957@funkthat.com> References: <20141203024112.GF99957@funkthat.com> <547E8319.9070509@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <547E8319.9070509@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 03 Dec 2014 09:49:48 -0800 (PST) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 17:49:50 -0000 Peter Grehan wrote this message on Tue, Dec 02, 2014 at 19:27 -0800: > >This is w/ a resonably recent current i386 kernel (r275062) > > I verified the r274463 i386 snapshot boots fine. That's not quite as > recent, but: > > >... Any suggestions? > > >uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 on acpi0 > ... > >uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > >uart0: console (9600,n,8,1) > > The "flags 0x10" text isn't being displayed in the i386 dmesg. This is > derived from /boot/device.hints i.e. > > ... > hint.uart.0.flags=0x10 > ... > > You can see this from bhyveload by issuing a 'show' command to dump > all the environment, including what was read in from device.hints by the > loader, or "more /boot/device.hints". So, checking /boot/device.hints shows: hint.uart.0.at="isa" hint.uart.0.port="0x3F8" hint.uart.0.flags="0x10" hint.uart.0.irq="4" hint.uart.1.at="isa" hint.uart.1.port="0x2F8" hint.uart.1.irq="3" > It may be possible that this file isn't being populated in your image, > or being read in by bhyveload. show from loader: OK show LINES=24 boot_serial=1 console=userboot currdev=disk0p3: interpret=OK kernelname=/boot/kernel/kernel loaddev=disk0p3: prompt=${interpret} smbios.bios.vendor=BHYVE Any ideas? Oh, I did verify that uart0 is functional by logging into the system and doing: echo foobar > /dev/cuau0 So, it's not like the console is not working, it's clearly a problem w/ the kernel deciding to use it as console.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 18:30:06 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A7BD0EDB for ; Wed, 3 Dec 2014 18:30:06 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id 6B6978FC for ; Wed, 3 Dec 2014 18:30:05 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id A67CE124A8; Thu, 4 Dec 2014 04:30:03 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAF44467 (AUTH peterg@ptree32.com.au); Thu, 4 Dec 2014 04:30:02 +1000 Message-ID: <547F56A8.6020607@freebsd.org> Date: Wed, 03 Dec 2014 10:30:00 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John-Mark Gurney Subject: Re: FreeBSD i386 guest doesn't see console... References: <20141203024112.GF99957@funkthat.com> <547E8319.9070509@freebsd.org> <20141203174947.GG99957@funkthat.com> In-Reply-To: <20141203174947.GG99957@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 18:30:06 -0000 Hi John-Mark, >> It may be possible that this file isn't being populated in your image, >> or being read in by bhyveload. > > show from loader: > OK show > LINES=24 > boot_serial=1 > console=userboot > currdev=disk0p3: > interpret=OK > kernelname=/boot/kernel/kernel > loaddev=disk0p3: > prompt=${interpret} > smbios.bios.vendor=BHYVE > > Any ideas? Looks like device.hints isn't being sourced in by the loader (aka bhyveload in this case). This is done on a stock system from: % grep device.hints /boot/defaults/loader.conf loader_conf_files="/boot/device.hints /boot/loader.conf /boot/loader.conf.local" Any chance on your system that you've overridden loader_conf_files, or are using a custom setup for loader config ? later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 18:38:03 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81592296 for ; Wed, 3 Dec 2014 18:38:03 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id 451FAA00 for ; Wed, 3 Dec 2014 18:38:02 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id 0A15412607; Thu, 4 Dec 2014 04:38:01 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAF44570 (AUTH peterg@ptree32.com.au); Thu, 4 Dec 2014 04:38:00 +1000 Message-ID: <547F5886.2060208@freebsd.org> Date: Wed, 03 Dec 2014 10:37:58 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-lists@potato.growveg.org, freebsd-virtualization@freebsd.org Subject: Re: odd bhyve <-> ubuntu 14.04 problem References: <20141203143406.GA29037@potato.growveg.org> In-Reply-To: <20141203143406.GA29037@potato.growveg.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 18:38:03 -0000 Hi John, > I upgraded the host from freebsd-10-RC3 to 10.1-R and after that > upgraded bhyve to grub2-bhyve-0.30 today and since then I've found that > when booting ubuntu 14.04.1 LTS guests, the ethernet interface (ethN > where N is a number) is now variable. This is an issue because if I stop > a VM and say reboot the host, I have to hack about with the VM instead > of just booting it. > > Previously the VM always saw the virtual network interface as eth0. Now, > it's anything but that. bhyve has been upgraded, the VM hasn't. How can > I control this behaviour? I want eth0 to remain as eth0. These are all > VMs that were eth0 before the bhyve upgrade. It doesn't appear to matter > what tap interface is used. I don't have this issue with freebsd guests > on the freebsd host, just ubuntu 14.04.1 LTS. > > I define the bridge and 9 tap interfaces at boot, in case this is > relevant. Please can anyone shed some light? Was the name of the VM changed when doing the upgrade ? If so, that will modify the MAC address, which confuses Ubuntu. The MAC addresses generated for an interface are derived from an md5 hash of the PCI slot/function and VM name, so they should be unique for each port and VM on a system. If the VM name is changed, or the slot configuration of a virtio-net device is changed, the MAC will also be changed. bhyve does provide the ability to set the MAC address for an interface, but granted this isn't available as an option with vmrun.sh :( later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 18:50:07 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A0EA861A; Wed, 3 Dec 2014 18:50:07 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D511B3E; Wed, 3 Dec 2014 18:50:06 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sB3Io5dE008656 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Dec 2014 10:50:05 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sB3Io5xS008655; Wed, 3 Dec 2014 10:50:05 -0800 (PST) (envelope-from jmg) Date: Wed, 3 Dec 2014 10:50:05 -0800 From: John-Mark Gurney To: Peter Grehan Subject: Re: FreeBSD i386 guest doesn't see console... Message-ID: <20141203185005.GJ99957@funkthat.com> References: <20141203024112.GF99957@funkthat.com> <547E8319.9070509@freebsd.org> <20141203174947.GG99957@funkthat.com> <547F56A8.6020607@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <547F56A8.6020607@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 03 Dec 2014 10:50:06 -0800 (PST) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 18:50:07 -0000 Peter Grehan wrote this message on Wed, Dec 03, 2014 at 10:30 -0800: > >> It may be possible that this file isn't being populated in your image, > >>or being read in by bhyveload. > > > >show from loader: > >OK show > >LINES=24 > >boot_serial=1 > >console=userboot > >currdev=disk0p3: > >interpret=OK > >kernelname=/boot/kernel/kernel > >loaddev=disk0p3: > >prompt=${interpret} > >smbios.bios.vendor=BHYVE > > > >Any ideas? > > Looks like device.hints isn't being sourced in by the loader (aka > bhyveload in this case). This is done on a stock system from: > > % grep device.hints /boot/defaults/loader.conf > loader_conf_files="/boot/device.hints /boot/loader.conf > /boot/loader.conf.local" > > Any chance on your system that you've overridden loader_conf_files, or > are using a custom setup for loader config ? Nope, my /boot/loader.conf looks like: aesni_load="YES" That's all... And /boot/defaults/loader.conf had the line: loader_conf_files="/boot/device.hints /boot/loader.conf /boot/loader.conf.local" and, no, I don't have a loader.conf.local file.. Hmmm... I just realized that my i386 system doesn't have the beastie menu when it boots, but my amd64 system does... This image was recently built using installworld/distribution w/ -DNO_ROOT, but the steps to produce the image are exactly the same wrt amd64... Ok, if I do a: set hint.uart.0.flags="0x10" in loader, and boot... serial console still doesn't work, but the flag is now seen by uart0: uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 So, it's not just that flag that's the issue.. I also just did a diff between the amd64 and i386 /boot dir, and besides the expected, boot blocks and loader differing, the only differences are if_faith_load got removed and ata and apm hints are now present.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 18:59:25 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 686D38D9 for ; Wed, 3 Dec 2014 18:59:25 +0000 (UTC) Received: from alto.onthenet.com.au (alto2.onthenet.com.au [203.13.68.14]) by mx1.freebsd.org (Postfix) with ESMTP id 27D60C56 for ; Wed, 3 Dec 2014 18:59:24 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTP id B4F27127D5; Thu, 4 Dec 2014 04:59:22 +1000 (EST) Received: from Peters-MacBook-Pro.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.4.4-GA) with ESMTP id CAF44818 (AUTH peterg@ptree32.com.au); Thu, 4 Dec 2014 04:59:21 +1000 Message-ID: <547F5D88.40301@freebsd.org> Date: Wed, 03 Dec 2014 10:59:20 -0800 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: John-Mark Gurney Subject: Re: FreeBSD i386 guest doesn't see console... References: <20141203024112.GF99957@funkthat.com> <547E8319.9070509@freebsd.org> <20141203174947.GG99957@funkthat.com> <547F56A8.6020607@freebsd.org> <20141203185005.GJ99957@funkthat.com> In-Reply-To: <20141203185005.GJ99957@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 18:59:25 -0000 Hi John-Mark, > Nope, my /boot/loader.conf looks like: > aesni_load="YES" Try taking that out and see what happens. If the system boots fine without it, then you may need to upgrade your host to r274407, which fixes the issue with boot-time loading of 32-bit kernel modules. later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Dec 3 20:29:10 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28312406; Wed, 3 Dec 2014 20:29:10 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F0A688FB; Wed, 3 Dec 2014 20:29:09 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id sB3KT10x009903 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Dec 2014 12:29:02 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id sB3KT1cg009902; Wed, 3 Dec 2014 12:29:01 -0800 (PST) (envelope-from jmg) Date: Wed, 3 Dec 2014 12:29:01 -0800 From: John-Mark Gurney To: Peter Grehan Subject: Re: FreeBSD i386 guest doesn't see console... Message-ID: <20141203202901.GK99957@funkthat.com> References: <20141203024112.GF99957@funkthat.com> <547E8319.9070509@freebsd.org> <20141203174947.GG99957@funkthat.com> <547F56A8.6020607@freebsd.org> <20141203185005.GJ99957@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141203185005.GJ99957@funkthat.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 03 Dec 2014 12:29:02 -0800 (PST) Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 20:29:10 -0000 John-Mark Gurney wrote this message on Wed, Dec 03, 2014 at 10:50 -0800: > I also just did a diff between the amd64 and i386 /boot dir, and besides > the expected, boot blocks and loader differing, the only differences are > if_faith_load got removed and ata and apm hints are now present.. Don't know how my diff missed loader.rc missing, but it did... So, after some investigation, it appears that loader.rc and friends weren't installed, though loader and many other things were... I believe I have run into this issue before multiple times, but didn't know what was the problem, and a new installworld fixed the issue... So, there is an issue w/ our installworld, but tracking it down will probably be hard.. I've had this happen on both local UFS, and NFS backed by ZFS... It could be related to the fact that bmake doesn't always error out when an error occurs (reported), and so it appears that installworld completes successfully, but doesn't... Also, I was able to reliably reproduce (in different locations) a failure in installing tests that failed w/ a permission denied... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-virtualization@FreeBSD.ORG Thu Dec 4 14:58:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50439F3F; Thu, 4 Dec 2014 14:58:00 +0000 (UTC) Received: from cyrus.watson.org (cyrus.watson.org [198.74.231.69]) by mx1.freebsd.org (Postfix) with ESMTP id 22B98F6A; Thu, 4 Dec 2014 14:58:00 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [198.74.231.63]) by cyrus.watson.org (Postfix) with ESMTPS id 4DCAE46B2C; Thu, 4 Dec 2014 09:57:59 -0500 (EST) Date: Thu, 4 Dec 2014 14:57:59 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: George Neville-Neil Subject: Re: Enabling VIMAGE in GENERIC In-Reply-To: Message-ID: References: <1423616F-F44D-47E5-8595-DE862DC04464@bsdimp.com> <546A34C8.6060004@freebsd.org> <546C8812.2070904@FreeBSD.org> <20141119195923.GS24601@funkthat.com> <69A8C06F-A7F6-49EC-8601-91AC4CDBFB13@FreeBSD.org> <547364EB.7090505@freebsd.org> <547AEB93.3050600@freebsd.org> User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Net , "freebsd-virtualization@freebsd.org" , freebsd-arch X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Dec 2014 14:58:00 -0000 On Mon, 1 Dec 2014, George Neville-Neil wrote: > On a slight tangent. I ran VIMAGE kernels vs. non VIMAGE kernels for both a > VANILLA kernel and a PF kernel (PF on but no rules) as a quick smoke test > today. The raw forwarding performance was unchanged between kernels with > and without VIMAGE on a 10G based system in the Sentex lab (lion1). I will > be doing a bit more work in this area and will then put up some results in > my netperf github repo. Was this a CPU-bound or network-bound workload? In general, I'd expect VIMAGE to have a modest overhead for most measurable workloads .. unless you are CPU-bound, in which case per-packet processing overheads might become (potentially) quite visible. They will also be more visible on simpler pipelines and with less cache-rich designs -- e.g., SoCs of various sorts. Doing a bit of CPU-bound networking on a modest ARM core might show off the effects better. Robert