From owner-freebsd-current@freebsd.org Sun Aug 5 15:36:09 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAE801055D6D for ; Sun, 5 Aug 2018 15:36:08 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B2A5934B1 for ; Sun, 5 Aug 2018 15:36:08 +0000 (UTC) (envelope-from bogorodskiy@gmail.com) Received: by mail-ed1-x542.google.com with SMTP id s24-v6so3890437edr.8 for ; Sun, 05 Aug 2018 08:36:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=uelmaz0hXmbzPcpjLITiaW+u/7mYsjVjfFtivKPrbrA=; b=c28LUSC31YXB1tUwQ2ucfLdRxxcdVErfdZ83ZW7SeEq+OKJVi9L4L0TNx0XThcUY3I c4tjbZNauRw2sjRahblE6QIj8oVuzs3pnwuiMuE033+EdOVqnECR1B8Mnq7Su+kLASN2 eX9uPiOtDtd9qgr5tJa8vxhs1+P/cmG9rlmQ0graA2NWKcFbRdAIF2imsrm2s7DnsonX N19YVIzuoYhiHySowXGaGN0fx9aQSKwr340v1gc8hnJm4OOMJ1Xf6Rr9eocgdHMToG81 xU+k+evCrCnFGz+JvGs2Yz37Vluosxbr+fSp8Ro0QInROcri0MaiFvO7ZUJqJW3aHDVC IrpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:subject:message-id :mime-version:content-disposition:user-agent; bh=uelmaz0hXmbzPcpjLITiaW+u/7mYsjVjfFtivKPrbrA=; b=HpyttvTSvIWCpY3XShJk/a99by0AjAmIF1wA0dirTIKPI8zYegwWtrTfEVX5sdvhDY IM8YqkTkq9Dxt3dqgGVo7YH+dhjnbmFjWNXCbAotNTCOs1eiEOyvkkAsCQX0kgocxyAl V16EHDtRTpltkYIGO8lqwwr2bp9nlz6YylAY1gEE3LdyW5hz6wdvWRDFk+0IqvPKqoK7 KhvaHIK9WzNhf6TuYGEiClYkmZnmuFH+Lkorkf45iRkXKNd7+6Z63umryx08WvfL0CaZ 4xVq6EIJWq50TDYo1Mv5J4CN/gdJ+AsQbezb3Xjn3Jkt+4b17DFy7po8jxxr/Rnhi+1f cUig== X-Gm-Message-State: AOUpUlEt/prfBEx5eV+aM7jPgluFlJJES82LHclO7ORNcb/HLylkNIJd EX6DISY08vOdf8LaV13OBAH6vueV X-Google-Smtp-Source: AAOMgpdTmEA7ZNq7xr9T02t375L9m04Ixo1IJnpiJ9g1rMO9wLH1hAu5Zn8b+88HA78kH/P7kEfcEQ== X-Received: by 2002:a50:aa83:: with SMTP id q3-v6mr14825451edc.64.1533483366105; Sun, 05 Aug 2018 08:36:06 -0700 (PDT) Received: from kloomba ([95.104.131.95]) by smtp.gmail.com with ESMTPSA id e2-v6sm4079554edn.11.2018.08.05.08.36.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 05 Aug 2018 08:36:05 -0700 (PDT) Sender: Roman Bogorodskiy Date: Sun, 5 Aug 2018 19:35:57 +0400 From: Roman Bogorodskiy To: freebsd-current@freebsd.org Subject: panic after ifioctl/if_clone_destroy Message-ID: <20180805153556.GA1957@kloomba> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="FL5UXtIhxfXey3p5" Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Aug 2018 15:36:09 -0000 --FL5UXtIhxfXey3p5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Running -CURRENT r336863 on amd64. Get the following panic right after (or during) boot: Fatal trap 12: page fault while in kernel mode cpuid =3D 2; apic id =3D 04 fault virtual address =3D 0xdeadc2ff fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff80bd7858 stack pointer =3D 0x28:0xfffffe008b445580 frame pointer =3D 0x28:0xfffffe008b4455c0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 903 (libvirtd) Traceback is: (kgdb) #0 doadump (textdump=3D0) at pcpu.h:230 #1 0xffffffff8043dc7b in db_dump (dummy=3D, dummy2=3D, dummy3=3D, dummy4=3D) at /usr/src/sys/ddb/db_command.c:574 #2 0xffffffff8043da49 in db_command (cmd_table=3D) at /usr/src/sys/ddb/db_command.c:481 #3 0xffffffff8043d7c4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:534 #4 0xffffffff804409ef in db_trap (type=3D, code=3D) at /usr/src/sys/ddb/db_main.c:252 #5 0xffffffff80bdd513 in kdb_trap (type=3D12, code=3D0, tf=3D) at /usr/src/sys/kern/subr_kdb.c:693 #6 0xffffffff810769f1 in trap_fatal (frame=3D0xfffffe008b4454c0, eva=3D373= 5929599) at /usr/src/sys/amd64/amd64/trap.c:884 #7 0xffffffff81076b12 in trap_pfault (frame=3D0xfffffe008b4454c0, usermode=3D) at pcpu.h:230 #8 0xffffffff8107611a in trap (frame=3D0xfffffe008b4454c0) at /usr/src/sys/amd64/amd64/trap.c:427 #9 0xffffffff810518ac in calltrap () at /usr/src/sys/amd64/amd64/exception.S:230 #10 0xffffffff80bd7858 in epoch_block_handler_preempt ( global=3D, cr=3D0xfffffe00760c3a00, arg=3D) at /usr/src/sys/kern/subr_epoch.c:256 #11 0xffffffff803994fd in ck_epoch_synchronize_wait ( global=3D0xfffff800030c5680, cb=3D0xffffffff80bd77a0 , ct=3D0x0) at /usr/src/sys/contrib/ck/src/ck_epoch.c:407 #12 0xffffffff80bd7630 in epoch_wait_preempt (epoch=3D0xfffff800030c5680) at /usr/src/sys/kern/subr_epoch.c:389 #13 0xffffffff80c983bf in if_delgroup (ifp=3D0xfffff80003aab800, groupname=3D0xfffff80005ff5e00 "bridge") at /usr/src/sys/net/if.c:1514 #14 0xffffffff80c9f2b2 in if_clone_destroyif (ifc=3D0xfffff80005ff5e00, ifp=3D0xfffff80003aab800) at /usr/src/sys/net/if_clone.c:325 #15 0xffffffff80c9f0d5 in if_clone_destroy (name=3D0xfffffe008b4458d0 "virb= r0") at /usr/src/sys/net/if_clone.c:288 #16 0xffffffff80c9a2c3 in ifioctl (so=3D0xfffff80007edca38, cmd=3D214960780= 1, data=3D, td=3D) at /usr/src/sys/net/if.c:3053 #17 0xffffffff80c04259 in kern_ioctl (td=3D0xfffff80007c1a580, fd=3D, com=3D, data=3D) at file.h:330 #18 0xffffffff80c03f2e in sys_ioctl (td=3D0xfffff80007c1a580, uap=3D0xfffff80007c1a940) at /usr/src/sys/kern/sys_generic.c:712 #19 0xffffffff81077401 in amd64_syscall (td=3D0xfffff80007c1a580, traced=3D= 0) at subr_syscall.c:135 #20 0xffffffff8105218d in fast_syscall_common () at /usr/src/sys/amd64/amd64/exception.S:500 #21 0x00000008028f4c0a in ?? () = = =20 Previous frame inner to this frame (corrupt stack?) = = =20 Current language: auto; currently minimal = = =20 (kgdb) It looks like panic happens during network interfaces related operations. Couple of dmesg lines before panic: Aug 5 19:02:42 romashka rtsold[585]: interface b= ridge0 removed Aug 5 19:02:42 romashka kernel: bridge0: Ethernet address: 02:af:41:48:c7:= 00 Aug 5 19:02:42 romashka kernel: bridge0: changing name to 'virbr-ab' Aug 5 19:02:42 romashka kernel: tap0: Ethernet address: 00:bd:8d:11:f7:00 Aug 5 19:02:42 romashka kernel: tap0: link state changed to UP Aug 5 19:02:42 romashka kernel: tap0: changing name to 'virbr-ab-nic' Aug 5 19:02:42 romashka kernel: virbr-ab-nic: promiscuous mode enabled Aug 5 19:02:42 romashka kernel: virbr-ab: link state changed to UP Aug 5 19:02:42 romashka rtsold[585]: interface t= ap0 removed Aug 5 19:02:43 romashka dnsmasq[1047]: setting --bind-interfaces option be= cause of OS limitations Aug 5 19:02:43 romashka dnsmasq[1047]: warning: no upstream servers config= ured Aug 5 19:02:43 romashka kernel: virbr-ab-nic: link state changed to DOWN Aug 5 19:02:43 romashka kernel: virbr-ab: link state changed to DOWN Aug 5 19:02:43 romashka kernel: bridge1: Ethernet address: 02:af:41:48:c7:= 01 Aug 5 19:02:43 romashka kernel: bridge1: changing name to 'virbr0' Aug 5 19:02:43 romashka rtsold[585]: interface b= ridge1 removed Aug 5 19:02:43 romashka kernel: tap1: Ethernet address: 00:bd:53:14:f7:01 Aug 5 19:02:43 romashka kernel: tap1: link state changed to UP Aug 5 19:02:43 romashka kernel: tap1: changing name to 'virbr0-nic' Aug 5 19:02:43 romashka kernel: virbr0: link state changed to UP Aug 5 19:02:43 romashka kernel: virbr0-nic: promiscuous mode enabled Aug 5 19:02:43 romashka rtsold[585]: interface t= ap1 removed Aug 5 19:05:03 romashka syslogd: kernel boot file is /boot/kernel/kernel Aug 5 19:05:03 romashka kernel: Aug 5 19:05:03 romashka syslogd: last message repeated 1 times Aug 5 19:05:03 romashka kernel: Fatal trap 12: page fault while in kernel = mode If I disable libvirt service, system completes booting fine. What it tries to do on start, it creates a couple of bridge(4) and tap(4) devices, adds tap devices to bridges it created, and possibly destroy these interfaces in case of errors. It also starts dnsmasq on some of these interfaces. This problem started to appear about 2-4 weeks ago. Roman Bogorodskiy --FL5UXtIhxfXey3p5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJbZxlcAAoJEMltX/4IwiJqz14H/iUeN+2tp1x1c41ONRs9DwIU ahv5/cpAVLUGouo3KJru1SxLzBa8jXauMEC/zo0xDOBOcDsGAXCZWFwIXvErjmUw +4pHQtKVlWF+H5jkAc7t1XCcYMP1Mj0+BM1UFKX8/9//oZ35ti+f4c/GEkRQlXwU hLLTU+dIW4XExKkXR5DgolbRhbknN7LeZvceQ4RF1XB6HTLdMKIcIU/m+plb0SMX Um7dxCKuxkZAj6sCUo/9hiFvzr4vDBEajdlwGLiRagAaHXKa0euZ7C96B8skYHAh ABTkKZUTcnbL6XH6AWX5CdUsiNN1K4uxJ486Xhsq8BvIjfk+syiw8l2yOOqttcA= =J8Du -----END PGP SIGNATURE----- --FL5UXtIhxfXey3p5--