From owner-freebsd-virtualization@FreeBSD.ORG Wed Jun 11 18:23:16 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14E1797C; Wed, 11 Jun 2014 18:23:16 +0000 (UTC) Received: from mail-la0-x22c.google.com (mail-la0-x22c.google.com [IPv6:2a00:1450:4010:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 222192557; Wed, 11 Jun 2014 18:23:14 +0000 (UTC) Received: by mail-la0-f44.google.com with SMTP id hr17so72994lab.31 for ; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=2bXKTQjGBs4PbPFOr1Va9Rjgi4hItZb44w0vqmOpPMo=; b=hk529SRVnDtGWRJUp6Gy9iv/hZ+NU9l2zdt1Vb+BzC1ritE08Bd9r+5OZwx2nO82+v je48n3X7iL3h+ldbhmzo62Svh1e9doNVTQOEIv8hkFry5FvlKc+0rfvrws/3SdUb87oh C+IT5V6MvNQ434fmp1IFTnUvnNN8Sc7ABHX2VRGi7ThLUhH6cQu5snbFTNIJEwtwzbxB r2hfku985FQDrCRJE1ZRgVtcWHig9rjt4y3m+HOeoKUC7CM8OxpWIF7N3K5TzvCd+wgn wpcr0wcDU9y0YJJD1ARsNkHnAqmDCfVSXuiWVdXFjFUEbk2LTwBEf/4tHxhcXBM1aG9D 5jMw== X-Received: by 10.152.26.99 with SMTP id k3mr2828239lag.68.1402510992576; Wed, 11 Jun 2014 11:23:12 -0700 (PDT) Received: from kloomba ([95.104.132.28]) by mx.google.com with ESMTPSA id d9sm22196463lag.19.2014.06.11.11.23.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jun 2014 11:23:11 -0700 (PDT) Sender: Roman Bogorodskiy Date: Wed, 11 Jun 2014 22:21:02 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140611182101.GD2855@kloomba> References: <20140610114342.GA11194@dev.san.ru> <20140610175420.GC61743@kloomba> <20140610191439.GA220@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W5WqUoFLvi1M7tJE" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:23:16 -0000 --W5WqUoFLvi1M7tJE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Craig Rodrigues wrote: > On Tue, Jun 10, 2014 at 1:54 PM, Craig Rodrigues wr= ote: > > On Tue, Jun 10, 2014 at 12:25 PM, Craig Rodrigues = wrote: > > lldb /usr/local/sbin/libvirtd libvirtd.core > > > > (lldb) bt > > * thread #1: tid =3D 0, 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583, name =3D 'libvirtd', stop reaso= n =3D > > signal SIGSEGV > > * frame #0: 0x0000000801aa8513 > > libvirt.so.1002`virNetDevTapCreateInBridgePort(brname=3D0x000000080a421= 1c8, > > ifname=3D0x000000080a88d1e0, macaddr=3D0x000000080a88d144, > > vmuuid=3D0x000000080a81b808, tapfd=3D0x0000000000000000, tapfdSize=3D1, > > virtPortProfile=3D0x0000000000000000, virtVlan=3D0x0000000000000000, > > flags=3D9) + 547 at virnetdevtap.c:583 > > frame #1: 0x0000000809e20569 > > libvirt_driver_bhyve.so`bhyveBuildNetArgStr(def=3D0x000000080a81b800, > > cmd=3D0x000000080a4db200, dryRun=3Dfalse) + 841 at bhyve_command.c:82 > > frame #2: 0x0000000809e20196 > > libvirt_driver_bhyve.so`virBhyveProcessBuildBhyveCmd(driver=3D0x0000000= 80a48a040, > > def=3D0x000000080a81b800, dryRun=3Dfalse) + 310 at bhyve_command.c:248 > > frame #3: 0x0000000809e24d6a > > libvirt_driver_bhyve.so`virBhyveProcessStart(conn=3D0x000000080a4db100, > > driver=3D0x000000080a48a040, vm=3D0x000000080a48b0f0, > > reason=3DVIR_DOMAIN_RUNNING_BOOTED, flags=3D0) + 746 at > > bhyve_process.c:134 > > frame #4: 0x0000000809e2334a > > libvirt_driver_bhyve.so`bhyveDomainCreateWithFlags(dom=3D0x000000080a48= a280, > > flags=3D0) + 490 at bhyve_driver.c:847 > > frame #5: 0x0000000809e2315a > > libvirt_driver_bhyve.so`bhyveDomainCreate(dom=3D0x000000080a48a280) + 26 > > at bhyve_driver.c:867 > > frame #6: 0x0000000801b96aad > > libvirt.so.1002`virDomainCreate(domain=3D0x000000080a48a280) + 637 at > > libvirt.c:8968 > > frame #7: 0x0000000001075427 > > libvirtd`remoteDispatchDomainCreate(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340) + 231 at > > remote_dispatch.h:3048 > > frame #8: 0x000000000103b84e > > libvirtd`remoteDispatchDomainCreateHelper(server=3D0x000000080781c400, > > client=3D0x00000008078df480, msg=3D0x00000008078791d0, > > rerr=3D0x00007fffff9fcc30, args=3D0x0000000807934340, > > ret=3D0x000000080a421058) + 254 at remote_dispatch.h:3026 > > frame #9: 0x0000000801c26b20 > > libvirt.so.1002`virNetServerProgramDispatchCall(prog=3D0x000000080790f4= 00, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 784 at virnetserverprogram.c:437 > > frame #10: 0x0000000801c26641 > > libvirt.so.1002`virNetServerProgramDispatch(prog=3D0x000000080790f400, > > server=3D0x000000080781c400, client=3D0x00000008078df480, > > msg=3D0x00000008078791d0) + 705 at virnetserverprogram.c:307 > > frame #11: 0x000000000108f97c > > libvirtd`virNetServerProcessMsg(srv=3D0x000000080781c400, > > client=3D0x00000008078df480, prog=3D0x000000080790f400, > > msg=3D0x00000008078791d0) + 332 at virnetserver.c:172 > > frame #12: 0x000000000108cd9c > > libvirtd`virNetServerHandleJob(jobOpaque=3D0x0000000807934360, > > opaque=3D0x000000080781c400) + 204 at virnetserver.c:193 > > frame #13: 0x0000000801ac84b8 > > libvirt.so.1002`virThreadPoolWorker(opaque=3D0x0000000807901d60) + 680 > > at virthreadpool.c:145 > > frame #14: 0x0000000801ac7b9b > > libvirt.so.1002`virThreadHelper(data=3D0x000000080781b520) + 75 at > > virthread.c:197 > > frame #15: 0x0000000803c614f5 libthr.so.3`??? + 277 > > > > -- > > Craig >=20 >=20 >=20 > OK, I think I found the problem. >=20 > In this file: http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3D= src/bhyve/bhyve_command.c;hb=3DHEAD >=20 > We have: >=20 > static int > bhyveBuildNetArgStr(const virDomainDef *def, virCommandPtr cmd, bool dryR= un) > { > virDomainNetDefPtr net =3D NULL; > char *brname =3D NULL; > char *realifname =3D NULL; > int *tapfd =3D NULL; >=20 >=20 > tapfd is never initialized after this. >=20 >=20 > In the same function, we have this: >=20 > if (!dryRun) > if (virNetDevTapCreateInBridgePort(brname, &net->ifname, &net= ->mac, > def->uuid, tapfd, 1, >=20 > virDomainNetGetActualVirtPortProfile(net), > virDomainNetGetActualVlan(= net), >=20 > VIR_NETDEV_TAP_CREATE_IFUP | VIR_NETDEV_TAP_CREATE_PERSIST) < 0) { > VIR_FREE(net->ifname); > VIR_FREE(brname); > return -1; > } >=20 >=20 >=20 > virNetDevTapCreateinBridgePort() is being passed a NULL tapfd, but the > next parameter is 1 instead of 0. >=20 >=20 >=20 > Then further along in this file: > http://libvirt.org/git/?p=3Dlibvirt.git;a=3Dblob_plain;f=3Dsrc/util/virne= tdevtap.c;hb=3DHEAD > we have: >=20 > error: > for (i =3D 0; i < tapfdSize && tapfd[i] >=3D 0; i++) > VIR_FORCE_CLOSE(tapfd[i]); >=20 >=20 > tapfd is NULL, but tapfdSize is 1, so the coredump happens here. >=20 > What is the best way to fix this? Thanks for the analysis! I'll think what would be a proper fix for that problem. Also, while thinking how to reproduce it, I did 'kldunload if_bridge' and started to see the same problem. So, as a temporary fix, could you check if you have bridge support available (in kernel or via module)? Bridges are needed to get networking anyway. Thanks, Roman Bogorodskiy --W5WqUoFLvi1M7tJE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTmJ4NAAoJEMltX/4IwiJqXJUH/j+Oy+peyJse6WJYItF+CHnF DfQcz9YsuPxEOrFJIHMVz6MzTbfyXqXaxYLPtMfa0ly8qqgBuGWW3sCnQtYeZokG UxroKlFVKdgxBx5FlByNTHUn0Onj3sqK3oECp6lSKk/3e9HXTFUTsNF2q/5dueDN 66axiddJA4aKhb4RxxOsjBKU/jLSmDATfGp6d3GUentp7tqs4LGlgle4llNd+zNn LjFSN5ncXnCSKiQ3tCWqtGlBUQW5VfY8MhA5EeZe/8Hp65OxNurwDi7eWHdUQQD9 BfxysIiJiFWM1cJ2LBwTsdpbjxYv8JcuVvxLOB7epYQB+PJKQSxYJXNypn/WEWk= =lg/Q -----END PGP SIGNATURE----- --W5WqUoFLvi1M7tJE--