From owner-freebsd-xen@FreeBSD.ORG Mon Aug 1 16:32:49 2011 Return-Path: Delivered-To: freebsd-xen@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E6461065781 for ; Mon, 1 Aug 2011 16:32:49 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout1-b.corp.re1.yahoo.com (mrout1-b.corp.re1.yahoo.com [69.147.107.20]) by mx1.freebsd.org (Postfix) with ESMTP id DDF7C8FC19 for ; Mon, 1 Aug 2011 16:32:48 +0000 (UTC) Received: from [127.0.0.1] (cheese.corp.yahoo.com [216.145.50.99]) by mrout1-b.corp.re1.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id p71GWVfp028582; Mon, 1 Aug 2011 09:32:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1312216352; bh=ODeqgBllQMhNMOHhVEA2/I9Z81ydNWXxM+mfqf+YyOA=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=GX332ljT73t86mLx3w0qOP7jBocls37d3+N2zJvU4sM+eESCO1ZFFgk6qteUwzkd/ 8XvxMlgE7obk41UrStrvLHXYDHvBQ/xLTlEikFZ+4OYamPE3BQFHonK5n1xptswaXx S3AU3KNphI1fIuGDmc2qid5+fRDkANyrgOB60ZUI= From: Sean Bruno To: "Ben C." In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Date: Mon, 01 Aug 2011 09:32:31 -0700 Message-ID: <1312216351.2872.4.camel@hitfishpass-lx.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.2 (2.32.2-1.fc14) Content-Transfer-Encoding: 7bit Cc: "freebsd-xen@freebsd.org" Subject: Re: Networking issues with FreeBSD HVM + SMP X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2011 16:32:49 -0000 On Sun, 2011-07-31 at 20:22 -0700, Ben C. wrote: > Hello all, > > Thanks for taking time to read this post. It's a fairly long winded > tale of confusion, but here goes: > > * I originally installed FreeBSD 8.2-RELEASE fine working under 100% > HVM. Using the GENERIC kernel with xen's qemu > * I trashed the box (kind of willingly, it's been a while since I've > had the pleasure of FreeBSD and think of it as my first pancake) > * Reinstalled. Networking failed. I had no idea why. Several weeks > past, I finally got networking back working after about 20-30 install > attempts. > * Setup the box properly. > * Did some changes throughout the dom0/domu's - I had originally had > the vcpu's pinned to physical cpu's and all this mojo. I found that > my load averages on the dom0 were significantly lower when the vcpu's > were bound to 'any physical' -- but that's beside the point. The > point is, FreeBSD changed vcpu's from 1 to 2 - and networking stopped. > * I honestly thought it surely was something else. I triple-checked > and confirmed, when vcpu=1, networking worked as expected. When vcpu > was >1, networking failed with a vauge .. em0 (or re0, see below) .. > watchdog timeout spewing all over. > > If you take a look at my configuration below, I originally had > model=e1000 in the vif, which I *thought* was what made it work > originally. I was apprently wrong. vcpu pinning doesn't matter. > Essentially, if it has more than 1 (v)cpu's, networking fails. > > The dom0 is running NetBSD -CURRENT [current there is just like > current here], on an Intel i7. I could try the -current branch or > using the xenhvm kernel/drivers. I'm fairly sure the latter may help > the situation, but honestly I'm just a bit bothered by the strangeness > of this "bug". Does anyone else have any suggestions on what I > should try next? > > Thanks for your time, Ben > > > name ="a5freebsd" > memory = 2048 > > kernel = "/usr/pkg/lib/xen/boot/hvmloader" > builder='hvm' > device_model = '/usr/pkg/libexec/qemu-dm' > > disk = [ > 'phy:/dev/mapper/rxenpool-a5freebsd_root,ioemu:hda,w', > 'file:/home/xen/iso/FreeBSD-8.2-RELEASE-amd64-disc1.iso,ioemu:hdc:cdrom,r' > ] > boot = 'cd' > > vcpus=2 > #cpus=['2'] > > > #vif=[ 'type=ioemu,bridge=bridge0,model=e1000,mac=00:16:3e:4f:bb:78' ] > vif=[ 'type=ioemu,bridge=bridge0,mac=00:16:3e:4f:bb:78' ] > > vnc = 1 > vncdisplay = 2 > vnclisten = "1.2.3.4" > vncpasswd = "removed" > > on_reboot = 'restart' > on_crash = 'restart' > > usbdevice = 'tablet' I haven't tried the e1000 emulation on the refernce build Xen images in the cluster yet, but when I use the stock Generic kernel I seem to be using re(4). Not that this really matters, but its a reference point. Also, I kind of gave up on the NetBSD dom0 a while ago so I could get stuff working in the cluster. Currently I'm utilizing a CentOS 5.6 Dom0 with Xen 3.4.3 from http://www.gitco.de/repo/ Here's the config I'm using for ref8-xen64.freebsd.org that seems to be the most likely thing to work: #============================================================================ # Python configuration setup for 'xm create'. # This script sets the parameters used when a domain is created using 'xm create'. # You use a separate script for each domain you want to create, or # you can set the parameters for the domain on the xm command line. #============================================================================ #---------------------------------------------------------------------------- # Kernel image file. kernel = "/usr/lib/xen/boot/hvmloader" #---------------------------------------------------------------------------- # device model to use: only qemu-dm available for now device_model = '/usr/lib64/xen/bin/qemu-dm' builder='hvm' # Initial memory allocation (in megabytes) for the new domain. memory = 1024 # number of CPUS vcpus = 2 # A name for your domain. All domains must have different names. name = "ref8-xen64" arch = "x86_64" #Network interface. By default emules a realtek 8139. For a NetBSD guest you # have to disable re(4) and let rtk attach to use it. # ne2k_pci emulates a pci ne2000 clone; this his cpu-hungry in dom0 # pcnet emulates a AMD PCnet-PCI controller; but it corrupts packets with # pcn(4) under NetBSD. vif = [ 'mac=00:16:3e:00:00:02, bridge=xenbr0, type=ioemu' ] #vif = [ 'mac=00:16:3e:00:00:02, bridge=xenbr0, type=vbd' ] # Define the disk devices you want the domain to have access to, and # what you want them accessible as. # Each disk entry is of the form phy:UNAME,DEV,MODE # where UNAME is the device, DEV is the device name the domain will see, # and MODE is r for read-only, w for read-write. # For hvm domains you can only use hda to hdd. You can set extra types # (e.g. cdrom) disk = [ 'file:/var/virt/FreeBSD-8.2-RELEASE-amd64-disc1.iso,hdc:cdrom,r', 'file:/var/virt/ref8-xen64.bin,hda,w' ] # floppy images; this doesn't seem to work currently. Use a iso image instead. #fda = '/home/domains/boot1.fs' # boot device: a = floppy, c= hard drive, d= cdrom (with the disk entry # before) # # boot CDROM image #boot='d' # boot from DISK file boot='c' # By default, 'xm create' will try to open an X window on the current display # for the virtal framebuffer. You can have the virtal framebuffer in vnc # instead, and connect using a vnc client (using localhost:$vncdisplay) # If vncunused is set to 1 (this is the default value), vncdisplay # will be set to the first unused port; so it's recommended to vnc = 1 vncdisplay = 2 vncunused = 1 vncpasswd='' #Xen emulates a PS/2 mouse, but the pointer in the guest has difficulties # tracking the absolute position. Xen can emulate a USB tablet in addition # to the mouse which will report the absolute position of the pointer, # and make the mouse much easier to use. # usb=1 usbdevice='tablet' #usbdevice='mouse' serial='pty' on_reboot='restart' #============================================================================