From owner-freebsd-stable@freebsd.org Fri Jan 20 18:39:49 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51BABCB9356; Fri, 20 Jan 2017 18:39:49 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36BEE1147; Fri, 20 Jan 2017 18:39:49 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.88) (envelope-from ) id 1cUe5S-000PAF-Ia; Fri, 20 Jan 2017 11:38:58 -0700 Date: Fri, 20 Jan 2017 11:38:58 -0700 From: The Doctor To: SK Cc: "Andrey V. Elsukov" , freebsd-xen@freebsd.org, "freebsd-stable@freebsd.org" , freebsd-virtualization@freebsd.org Subject: Re: FreeBSD 11 Stable on a Xen :: bridge0 crashing server Message-ID: <20170120183858.GB93874@doctor.nl2k.ab.ca> References: <7c06aea6-0b10-e9c0-d750-f42d4a406258@cps-intl.org> <20170120164306.GA53685@doctor.nl2k.ab.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jan 2017 18:39:49 -0000 On Fri, Jan 20, 2017 at 05:36:37PM +0000, SK wrote: > On 20/01/2017 16:43, The Doctor wrote: > > On Fri, Jan 20, 2017 at 07:20:15PM +0300, Andrey V. Elsukov wrote: > >> On 20.01.2017 18:57, Trond Endrest??l wrote: > >>>> Here is the situation: > >>>> I have a VPS server from a well reputed provider (and they deserve the > >>>> reputation), running FreeBSD 11 stable x64 under Xen Full Virtualization > >>>> (HVM). I have the xn0 interface which is working fine. I intend to use VIMAGE, > >>>> so I compiled the kernel, added cloned_interface="bridge0" and restarted the > >>>> server. But as soon as I am attaching the xn0 to bridge0, the kernel is > >>>> panicking and the server restarting. > >>>> Any suggestion/pointer/test-instruction is highly appreciated. > >>> The code crashes at line 427 of sys/netinet/if_ether.c: > >>> > >>> ARPSTAT_INC(txrequests); > >>> > >>> See > >>> https://svnweb.freebsd.org/base/stable/11/sys/netinet/if_ether.c?view=annotate#l427 > >>> > >>> stable/11 has problems accounting the outgoing octets of any xn > >>> interface, although this isn't connected to your case. > >>> > >>> Just to rule out any uncertainty, try this patch: > >>> > >>> https://svnweb.freebsd.org/base?view=revision&revision=308126 > >>> > >>> See PR 213439, > >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213439 > >>> > >>> Note, I'm not a developer nor a committer, just a humble sysadmin. > >> This problem is unrelated. ARP statistics is global and isn't related to > >> some specific interface. IMHO, the kernel panics due to missing VNET > >> context. As I see from the code in sys/dev/xen, it is not capable with > >> VIMAGE. > >> > >> -- > >> WBR, Andrey V. Elsukov > > Still we need to address this issue! > > > > Even here in FreeBSD-11 stable I see crashing > > of the bhyve. > > > > What I need to help you is what is needed for running gdb ../bhyve ../bhyve.core > > > Apologies, but I might be missing something here. I am not trying to run > bhyve, bhyve is not crashing. FreeBSD 11 in my case (FreeBSD > testBox.test.domain 11.0-STABLE FreeBSD 11.0-STABLE #0 r312349: Fri Jan > 20 12:29:17 GMT 2017 > root@testBox.test.domain:/usr/obj/usr/src/sys/vnet-test amd64) is > running on a Xen HVM, and as I am trying to attach bridge0 to any > interface (either xn0, or epair0a, or so on), the server is crashing and > rebooting. I had the line ifconfig_bridge0="addm xn0 addm epair0a up" on > the /etc/rc.conf , and the server went into an infinite crash-reboot loop. > > What I would like to have, if possible, is running FreeBSD 11 stable on > the server, use vnet/vimage for my jails so I can isolate each of them > from one another. > > Currently I am building world with the patch suggested by Trond (started > that before I read the mail from Andrey). But if that is unlikely to > bring a solution, that would mean most people on VPS with jails that are > using Xen are going to suffer from any upgrade to 11 (or having new > servers running 11) -- which is definitely undesirable. > > If anyone would like me to run further test on the box, please let me > know what you'd like me to run, and I can come back with the response. > Having a reliable/stable solution is the objective from my end. > > Thanks to all for your suggestions/comments. > > Regards > SK Well in my case using gdb /usr/sbin/bhyve /path/to/bhyve.core I get GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `bhyve'. Program terminated with signal 6, Aborted. Reading symbols from /usr/lib/libvmmapi.so.5...Reading symbols from /usr/lib/debug//usr/lib/libvmmapi.so.5.debug...done. done. Loaded symbols for /usr/lib/libvmmapi.so.5 Reading symbols from /lib/libmd.so.6...Reading symbols from /usr/lib/debug//lib/libmd.so.6.debug...done. done. Loaded symbols for /lib/libmd.so.6 Reading symbols from /lib/libz.so.6...Reading symbols from /usr/lib/debug//lib/libz.so.6.debug...done. done. Loaded symbols for /lib/libz.so.6 Reading symbols from /lib/libthr.so.3...Reading symbols from /usr/lib/debug//lib/libthr.so.3.debug...done. done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /lib/libutil.so.9...Reading symbols from /usr/lib/debug//lib/libutil.so.9.debug...done. done. Loaded symbols for /lib/libutil.so.9 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x000000080119755a in thr_kill () from /lib/libc.so.7 [New Thread 945218000 (LWP 102104/)] [New Thread 801a77400 (LWP 102102/)] [New Thread 801a76a00 (LWP 102087/)] [New Thread 801a76500 (LWP 102086/)] [New Thread 801a76000 (LWP 102085/)] [New Thread 801a1cb00 (LWP 102084/)] [New Thread 801a1c600 (LWP 102082/)] [New Thread 801a1c100 (LWP 102081/)] [New Thread 801a1bc00 (LWP 102080/)] [New Thread 801a1b700 (LWP 102079/)] [New Thread 801a1b200 (LWP 102078/)] [New Thread 801a1ad00 (LWP 102070/)] [New Thread 801a1a800 (LWP 102069/)] [New Thread 801a1a300 (LWP 102068/)] [New Thread 801a19e00 (LWP 102067/)] [New Thread 801a19900 (LWP 101469/)] [New Thread 801a19400 (LWP 100683/)] [New Thread 801a18f00 (LWP 100671/)] [New Thread 801a18a00 (LWP 100667/)] [New Thread 801a18500 (LWP 100666/)] [New Thread 801a18000 (LWP 100989/)] (Gdb) Where do I go from here? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! http://www.fullyfollow.me/rootnl2k Look at Psalms 14 and 53 on Atheism Birthdate 29 Jan 1969 Redhill Surrey England