From owner-freebsd-net@freebsd.org Sun Jul 10 06:22:56 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F316B83650 for ; Sun, 10 Jul 2016 06:22:56 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) Received: from nm39-vm8.bullet.mail.gq1.yahoo.com (nm39-vm8.bullet.mail.gq1.yahoo.com [98.136.217.111]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28BA01F3D for ; Sun, 10 Jul 2016 06:22:55 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1468131769; bh=wHlBdASmMxyxJs1jAJciJblXXiQ7Q5KZpSvFAk0MIl8=; h=Date:From:Reply-To:To:Subject:References:From:Subject; b=IwvSGbM1yZ5ta1EbXY8VaCexrU3G26QPuoWm7HDGooj5VU1MB6wIP4JU5wNABc5P45wPuq0iq2/XNN0rlDu8+QWg5qreS0gMbOdS+RJmBfgg1u5m0BPxNIFmqguTA8Ejeyto5DQPxCmYhTcqd9keZcP3029nYsYzZrdptjxArr/IX7OsnJDp8QVpMYJf5GSGnLNyKzDgt/GMYCQ4+tF9qDznS3yMM1xFbi55CXy324+XP3pW5tdGl1NQR3kDuS6uKVdUtHw10rJr6mZ0MyN10QdllwdjQtWrdjGpa6oP7q8Dp9ekXE8OXRI+N+mN1inf59qru4ZCRBmYivYXlSu3eQ== Received: from [127.0.0.1] by nm39.bullet.mail.gq1.yahoo.com with NNFMP; 10 Jul 2016 06:22:49 -0000 Received: from [98.137.12.59] by nm39.bullet.mail.gq1.yahoo.com with NNFMP; 10 Jul 2016 06:19:49 -0000 Received: from [98.139.170.180] by tm4.bullet.mail.gq1.yahoo.com with NNFMP; 10 Jul 2016 06:19:49 -0000 Received: from [98.139.212.245] by tm23.bullet.mail.bf1.yahoo.com with NNFMP; 10 Jul 2016 06:19:48 -0000 Received: from [127.0.0.1] by omp1054.mail.bf1.yahoo.com with NNFMP; 10 Jul 2016 06:19:48 -0000 X-Yahoo-Newman-Property: ymail-4 X-Yahoo-Newman-Id: 919164.57285.bm@omp1054.mail.bf1.yahoo.com X-YMail-OSG: _mMGMGMVM1kOWovfYykAnm8rI0BhqHqSHFEC3i8UzXSH19UkZ8hIuPm8z7tVXAu ybiNZoBOVcFojhxYu6_8085jx84_ObUuFRfkKx4P2tnSx1KBxQAcYxMivFd0wIawp3IHIS8OqzG6 H1FYkUFYANd56Az09y206FVWJDbTGNrM2TTqSq167idNnVtQ83OBpa5VRTzMtDXP2AdvHzQAw38a ruU1hoPJBeGhgIEo.lTDS076sQ9qeVRDRK0lHohnIEnvS9NXdJjPXPl7aZN_wsECk_00QwIGRtKt swdMRQeaI31fTtBlhEKaUOq_weVcR0p6oPWCrqDTNfjO43niFVxq_Y9IO7yYXwdWpu2E_p1Pz_vW EEjpGosmEWlhHaeRbisgTDlvWWa.Jjfxz5DsGe2AoECrHi010ujmeb8v5mcxLCQ6CoonCpVNI8it gF7yY7GngtVSd7f5W.os3vu8cfJI2dLWKegNNiQs5320GCuiWiyOP_89vlk8gHKRxiHzNhE2A99. KFkIGt.zuYD5qdhMJ.BsFb4ieyJEJuIEyYX8iK7Wd7n0OLYna3qwVY0jNNLh6eNjHh3yIphwOnS5 HfIYetHA- Received: from jws10687.mail.bf1.yahoo.com by sendmailws138.mail.bf1.yahoo.com; Sun, 10 Jul 2016 06:19:48 +0000; 1468131588.464 Date: Sun, 10 Jul 2016 06:19:48 +0000 (UTC) From: Reply-To: To: "freebsd-net@freebsd.org" Message-ID: <58413371.417578.1468131588156.JavaMail.yahoo@mail.yahoo.com> Subject: kernel panic with netgraph and mpd3.8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit References: <58413371.417578.1468131588156.JavaMail.yahoo.ref@mail.yahoo.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jul 2016 06:22:56 -0000 Hi I'm running an l2tp lns through mpd3.8 and it's been crashing twice in 24h. This is a new project replacing a cisco 7206, 700-sessions 800mbit/s I am not familiar with troubleshooting kernel panic's, I am suspecting that the crash is happening inside the netgraph module because the crash is happening at the instruction pointer = 0x20:0xffffffff81c38283 I included the 2 two crash logs. I need some help to to figure out what to do next. -Dbaud ============================ The box is a: # uname -a FreeBSD mybox.example.com 10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat May 28 12:23:44 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 # kldstat Id Refs Address Size Name 1 34 0xffffffff80200000 17bc6a8 kernel 2 2 0xffffffff81c11000 114db ipfw.ko 3 1 0xffffffff81c23000 d32f dummynet.ko 4 1 0xffffffff81c31000 3831 ng_socket.ko 5 9 0xffffffff81c35000 ba02 netgraph.ko 6 1 0xffffffff81c41000 2b99 ng_mppc.ko 7 1 0xffffffff81c44000 80c rc4.ko 8 1 0xffffffff81c45000 23dc vmmemctl.ko 9 1 0xffffffff81c48000 397d ng_l2tp.ko 10 1 0xffffffff81c4c000 4b04 ng_ksocket.ko 11 1 0xffffffff81c51000 17d6 ng_tee.ko 12 1 0xffffffff81c53000 40d2 ng_iface.ko 13 1 0xffffffff81c58000 5829 ng_ppp.ko 14 1 0xffffffff81c5e000 18b1 ng_tcpmss.ko 15 1 0xffffffff81c60000 2df7 ng_vjc.ko ======================================= First crash dump: Jul 8 08:09:04 mybox syslogd: kernel boot file is /boot/kernel/kernel Jul 8 08:09:04 mybox kernel: Jul 8 08:09:04 mybox kernel: Jul 8 08:09:04 mybox kernel: Fatal trap 12: page fault while in kernel mode Jul 8 08:09:04 mybox kernel: cpuid = 1; apic id = 01 Jul 8 08:09:04 mybox kernel: fault virtual address = 0x28 Jul 8 08:09:04 mybox kernel: fault code = supervisor read data, page not present Jul 8 08:09:04 mybox kernel: instruction pointer = 0x20:0xffffffff81c38283 Jul 8 08:09:04 mybox kernel: stack pointer = 0x28:0xfffffe0174d85540 Jul 8 08:09:04 mybox kernel: frame pointer = 0x28:0xfffffe0174d85580 Jul 8 08:09:04 mybox kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Jul 8 08:09:04 mybox kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Jul 8 08:09:04 mybox kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Jul 8 08:09:04 mybox kernel: current process = 628 (ng_queue2) Jul 8 08:09:04 mybox kernel: trap number = 12 Jul 8 08:09:04 mybox kernel: panic: page fault Jul 8 08:09:04 mybox kernel: cpuid = 1 Jul 8 08:09:04 mybox kernel: KDB: stack backtrace: Jul 8 08:09:04 mybox kernel: #0 0xffffffff8098e390 at kdb_backtrace+0x60 Jul 8 08:09:04 mybox kernel: #1 0xffffffff80951066 at vpanic+0x126 Jul 8 08:09:04 mybox kernel: #2 0xffffffff80950f33 at panic+0x43 Jul 8 08:09:04 mybox kernel: #3 0xffffffff80d55f7b at trap_fatal+0x36b Jul 8 08:09:04 mybox kernel: #4 0xffffffff80d5627d at trap_pfault+0x2ed Jul 8 08:09:04 mybox kernel: #5 0xffffffff80d558fa at trap+0x47a Jul 8 08:09:04 mybox kernel: #6 0xffffffff80d3b8d2 at calltrap+0x8 Jul 8 08:09:04 mybox kernel: #7 0xffffffff81c5e509 at ng_tcpmss_rcvdata+0x2d9 Jul 8 08:09:04 mybox kernel: #8 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 08:09:04 mybox kernel: #9 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 08:09:04 mybox kernel: #10 0xffffffff81c5a1c8 at ng_ppp_comp_recv+0x148 Jul 8 08:09:04 mybox kernel: #11 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 08:09:04 mybox kernel: #12 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 08:09:04 mybox kernel: #13 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 08:09:04 mybox kernel: #14 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 08:09:04 mybox kernel: #15 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 08:09:04 mybox kernel: #16 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 08:09:04 mybox kernel: #17 0xffffffff81c4d3e2 at ng_ksocket_incoming2+0x2f2 Jul 8 08:09:04 mybox kernel: Uptime: 5d17h47m38s Jul 8 08:09:04 mybox kernel: Copyright (c) 1992-2016 The FreeBSD Project. Jul 8 08:09:04 mybox kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Jul 8 08:09:04 mybox kernel: The Regents of the University of California. All rights reserved. Jul 8 08:09:04 mybox kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Jul 8 08:09:04 mybox kernel: FreeBSD 10.3-RELEASE-p4 #0: Sat May 28 12:23:44 UTC 2016 Jul 8 08:09:04 mybox kernel: root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Jul 8 08:09:04 mybox kernel: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Jul 8 08:09:04 mybox kernel: CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (2000.00-MHz K8-class CPU) Jul 8 08:09:04 mybox kernel: Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Jul 8 08:09:04 mybox kernel: Features=0x1fa3fbff Jul 8 08:09:04 mybox kernel: Features2=0x9fba2203 Jul 8 08:09:04 mybox kernel: AMD Features=0x28100800 Jul 8 08:09:04 mybox kernel: AMD Features2=0x1 Jul 8 08:09:04 mybox kernel: Structured Extended Features=0x2 Jul 8 08:09:04 mybox kernel: TSC: P-state invariant Jul 8 08:09:04 mybox kernel: Hypervisor: Origin = "VMwareVMware" Jul 8 08:09:04 mybox kernel: real memory = 6442450944 (6144 MB) Jul 8 08:09:04 mybox kernel: avail memory = 6183518208 (5897 MB) Jul 8 08:09:04 mybox kernel: Event timer "LAPIC" quality 600 Jul 8 08:09:04 mybox kernel: ACPI APIC Table: Jul 8 08:09:04 mybox kernel: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs Jul 8 08:09:04 mybox kernel: FreeBSD/SMP: 1 package(s) x 4 core(s) Jul 8 08:09:04 mybox kernel: cpu0 (BSP): APIC ID: 0 Jul 8 08:09:04 mybox kernel: cpu1 (AP): APIC ID: 1 Jul 8 08:09:04 mybox kernel: cpu2 (AP): APIC ID: 2 Jul 8 08:09:04 mybox kernel: cpu3 (AP): APIC ID: 3 Jul 8 08:09:04 mybox kernel: random: initialized Jul 8 08:09:04 mybox kernel: MADT: Forcing active-low polarity and level trigger for SCI Jul 8 08:09:04 mybox kernel: ioapic0 irqs 0-23 on motherboard Jul 8 08:09:04 mybox kernel: kbd1 at kbdmux0 Jul 8 08:09:04 mybox kernel: acpi0: on motherboard Jul 8 08:09:04 mybox kernel: acpi0: Power Button (fixed) Jul 8 08:09:04 mybox kernel: Timecounter "HPET" frequency 14318180 Hz quality 950 Jul 8 08:09:04 mybox kernel: cpu0: on acpi0 Jul 8 08:09:04 mybox kernel: cpu1: on acpi0 Jul 8 08:09:04 mybox kernel: cpu2: on acpi0 Jul 8 08:09:04 mybox kernel: cpu3: on acpi0 Jul 8 08:09:04 mybox kernel: attimer0: port 0x40-0x43 irq 0 on acpi0 Jul 8 08:09:04 mybox kernel: Timecounter "i8254" frequency 1193182 Hz quality 0 Jul 8 08:09:04 mybox kernel: Event timer "i8254" frequency 1193182 Hz quality 100 Jul 8 08:09:04 mybox kernel: atrtc0: port 0x70-0x71 irq 8 on acpi0 Jul 8 08:09:04 mybox kernel: Event timer "RTC" frequency 32768 Hz quality 0 Jul 8 08:09:04 mybox kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 Jul 8 08:09:04 mybox kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 Jul 8 08:09:04 mybox kernel: pcib0: port 0xcf8-0xcff on acpi0 Jul 8 08:09:04 mybox kernel: pci0: on pcib0 Jul 8 08:09:04 mybox kernel: pcib1: at device 1.0 on pci0 Jul 8 08:09:04 mybox kernel: pci1: on pcib1 Jul 8 08:09:04 mybox kernel: isab0: at device 7.0 on pci0 Jul 8 08:09:04 mybox kernel: isa0: on isab0 Jul 8 08:09:04 mybox kernel: atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0 Jul 8 08:09:04 mybox kernel: ata0: at channel 0 on atapci0 Jul 8 08:09:04 mybox kernel: ata1: at channel 1 on atapci0 Jul 8 08:09:04 mybox kernel: pci0: at device 7.3 (no driver attached) Jul 8 08:09:04 mybox kernel: vgapci0: port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0 Jul 8 08:09:04 mybox kernel: vgapci0: Boot video device Jul 8 08:09:04 mybox kernel: mpt0: port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0 Jul 8 08:09:04 mybox kernel: mpt0: MPI Version=1.2.0.0 Jul 8 08:09:04 mybox kernel: pcib2: at device 17.0 on pci0 Jul 8 08:09:04 mybox kernel: pci2: on pcib2 Jul 8 08:09:04 mybox kernel: em0: port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2 Jul 8 08:09:04 mybox kernel: em0: Ethernet address: 00:50:56:8d:d7:57 Jul 8 08:09:04 mybox kernel: pcib3: at device 21.0 on pci0 Jul 8 08:09:04 mybox kernel: pci3: on pcib3 Jul 8 08:09:04 mybox kernel: vmx0: port 0x4000-0x400f mem 0xfd4fc000-0xfd4fcfff,0xfd4fd000-0xfd4fdfff,0xfd4fe000-0xfd4fffff irq 18 at device 0.0 on pci3 Jul 8 08:09:04 mybox kernel: vmx0: Ethernet address: 00:50:56:8d:4c:68 Jul 8 08:09:04 mybox kernel: pcib4: at device 21.1 on pci0 Jul 8 08:09:04 mybox kernel: pci4: on pcib4 Jul 8 08:09:04 mybox kernel: pcib5: at device 21.2 on pci0 Jul 8 08:09:04 mybox kernel: pci5: on pcib5 Jul 8 08:09:04 mybox kernel: pcib6: at device 21.3 on pci0 Jul 8 08:09:04 mybox kernel: pci6: on pcib6 Jul 8 08:09:04 mybox kernel: pcib7: at device 21.4 on pci0 Jul 8 08:09:04 mybox kernel: pci7: on pcib7 Jul 8 08:09:04 mybox kernel: pcib8: at device 21.5 on pci0 Jul 8 08:09:04 mybox kernel: pci8: on pcib8 Jul 8 08:09:04 mybox kernel: pcib9: at device 21.6 on pci0 Jul 8 08:09:04 mybox kernel: pci9: on pcib9 Jul 8 08:09:04 mybox kernel: pcib10: at device 21.7 on pci0 Jul 8 08:09:04 mybox kernel: pci10: on pcib10 Jul 8 08:09:04 mybox kernel: pcib11: at device 22.0 on pci0 Jul 8 08:09:04 mybox kernel: pci11: on pcib11 Jul 8 08:09:04 mybox kernel: vmx1: port 0x5000-0x500f mem 0xfd3fc000-0xfd3fcfff,0xfd3fd000-0xfd3fdfff,0xfd3fe000-0xfd3fffff irq 19 at device 0.0 on pci11 Jul 8 08:09:04 mybox kernel: vmx1: Ethernet address: 00:50:56:8d:85:29 Jul 8 08:09:04 mybox kernel: pcib12: at device 22.1 on pci0 Jul 8 08:09:04 mybox kernel: pci12: on pcib12 Jul 8 08:09:04 mybox kernel: pcib13: at device 22.2 on pci0 Jul 8 08:09:04 mybox kernel: pci13: on pcib13 Jul 8 08:09:04 mybox kernel: pcib14: at device 22.3 on pci0 Jul 8 08:09:04 mybox kernel: pci14: on pcib14 Jul 8 08:09:04 mybox kernel: pcib15: at device 22.4 on pci0 Jul 8 08:09:04 mybox kernel: pci15: on pcib15 Jul 8 08:09:04 mybox kernel: pcib16: at device 22.5 on pci0 Jul 8 08:09:04 mybox kernel: pci16: on pcib16 Jul 8 08:09:04 mybox kernel: pcib17: at device 22.6 on pci0 Jul 8 08:09:04 mybox kernel: pci17: on pcib17 Jul 8 08:09:04 mybox kernel: pcib18: at device 22.7 on pci0 Jul 8 08:09:04 mybox kernel: pci18: on pcib18 Jul 8 08:09:04 mybox kernel: pcib19: at device 23.0 on pci0 Jul 8 08:09:04 mybox kernel: pci19: on pcib19 Jul 8 08:09:04 mybox kernel: pcib20: at device 23.1 on pci0 Jul 8 08:09:04 mybox kernel: pci20: on pcib20 Jul 8 08:09:04 mybox kernel: pcib21: at device 23.2 on pci0 Jul 8 08:09:04 mybox kernel: pci21: on pcib21 Jul 8 08:09:04 mybox kernel: pcib22: at device 23.3 on pci0 Jul 8 08:09:04 mybox kernel: pci22: on pcib22 Jul 8 08:09:04 mybox kernel: pcib23: at device 23.4 on pci0 Jul 8 08:09:04 mybox kernel: pci23: on pcib23 Jul 8 08:09:04 mybox kernel: pcib24: at device 23.5 on pci0 Jul 8 08:09:04 mybox kernel: pci24: on pcib24 Jul 8 08:09:04 mybox kernel: pcib25: at device 23.6 on pci0 Jul 8 08:09:04 mybox kernel: pci25: on pcib25 Jul 8 08:09:04 mybox kernel: pcib26: at device 23.7 on pci0 Jul 8 08:09:04 mybox kernel: pci26: on pcib26 Jul 8 08:09:04 mybox kernel: pcib27: at device 24.0 on pci0 Jul 8 08:09:04 mybox kernel: pci27: on pcib27 Jul 8 08:09:04 mybox kernel: pcib28: at device 24.1 on pci0 Jul 8 08:09:04 mybox kernel: pci28: on pcib28 Jul 8 08:09:04 mybox kernel: pcib29: at device 24.2 on pci0 Jul 8 08:09:04 mybox kernel: pci29: on pcib29 Jul 8 08:09:04 mybox kernel: pcib30: at device 24.3 on pci0 Jul 8 08:09:04 mybox kernel: pci30: on pcib30 Jul 8 08:09:04 mybox kernel: pcib31: at device 24.4 on pci0 Jul 8 08:09:04 mybox kernel: pci31: on pcib31 Jul 8 08:09:04 mybox kernel: pcib32: at device 24.5 on pci0 Jul 8 08:09:04 mybox kernel: pci32: on pcib32 Jul 8 08:09:04 mybox kernel: pcib33: at device 24.6 on pci0 Jul 8 08:09:04 mybox kernel: pci33: on pcib33 Jul 8 08:09:04 mybox kernel: pcib34: at device 24.7 on pci0 Jul 8 08:09:04 mybox kernel: pci34: on pcib34 Jul 8 08:09:04 mybox kernel: acpi_acad0: on acpi0 Jul 8 08:09:04 mybox kernel: atkbdc0: port 0x60,0x64 irq 1 on acpi0 Jul 8 08:09:04 mybox kernel: atkbd0: irq 1 on atkbdc0 Jul 8 08:09:04 mybox kernel: kbd0 at atkbd0 Jul 8 08:09:04 mybox kernel: atkbd0: [GIANT-LOCKED] Jul 8 08:09:04 mybox kernel: psm0: irq 12 on atkbdc0 Jul 8 08:09:04 mybox kernel: psm0: [GIANT-LOCKED] Jul 8 08:09:04 mybox kernel: psm0: model IntelliMouse, device ID 3 Jul 8 08:09:04 mybox kernel: fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 Jul 8 08:09:04 mybox kernel: fd0: <1440-KB 3.5" drive> on fdc0 drive 0 Jul 8 08:09:04 mybox kernel: orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 Jul 8 08:09:04 mybox kernel: sc0: at flags 0x100 on isa0 Jul 8 08:09:04 mybox kernel: sc0: VGA <16 virtual consoles, flags=0x300> Jul 8 08:09:04 mybox kernel: vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Jul 8 08:09:04 mybox kernel: ppc0: cannot reserve I/O port range Jul 8 08:09:04 mybox kernel: Timecounters tick every 1.000 msec Jul 8 08:09:04 mybox kernel: em0: link state changed to UP Jul 8 08:09:04 mybox kernel: random: unblocking device. Jul 8 08:09:04 mybox kernel: cd0 at ata1 bus 0 scbus1 target 0 lun 0 Jul 8 08:09:04 mybox kernel: cd0: Removable CD-ROM SCSI device Jul 8 08:09:04 mybox kernel: cd0: Serial Number 10000000000000000001 Jul 8 08:09:04 mybox kernel: cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) Jul 8 08:09:04 mybox kernel: cd0: Attempt to query device size failed: NOT READY, Medium not present Jul 8 08:09:04 mybox kernel: cd0: quirks=0x40 Jul 8 08:09:04 mybox kernel: da0 at mpt0 bus 0 scbus2 target 0 lun 0 Jul 8 08:09:04 mybox kernel: da0: Fixed Direct Access SCSI-2 device Jul 8 08:09:04 mybox kernel: da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) Jul 8 08:09:04 mybox kernel: da0: Command Queueing enabled Jul 8 08:09:04 mybox kernel: da0: 12288MB (25165824 512 byte sectors) Jul 8 08:09:04 mybox kernel: da0: quirks=0x40 Jul 8 08:09:04 mybox kernel: SMP: AP CPU #3 Launched! Jul 8 08:09:04 mybox kernel: SMP: AP CPU #1 Launched! Jul 8 08:09:04 mybox kernel: SMP: AP CPU #2 Launched! Jul 8 08:09:04 mybox kernel: Trying to mount root from ufs:/dev/da0p2 [rw]... Jul 8 08:09:04 mybox kernel: WARNING: / was not properly dismounted Jul 8 08:09:04 mybox kernel: vmx0: link state changed to UP Jul 8 08:09:04 mybox kernel: vmx1: link state changed to UP Jul 8 08:09:04 mybox kernel: Limiting icmp unreach response from 1816 to 200 packets/sec Jul 8 08:09:04 mybox kernel: Limiting icmp unreach response from 1982 to 200 packets/sec Jul 8 08:09:04 mybox kernel: ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled Jul 8 08:09:04 mybox kernel: DUMMYNET 0 with IPv6 initialized (100409) Jul 8 08:09:04 mybox kernel: load_dn_sched dn_sched FIFO loaded Jul 8 08:09:04 mybox kernel: load_dn_sched dn_sched QFQ loaded Jul 8 08:09:04 mybox kernel: load_dn_sched dn_sched RR loaded Jul 8 08:09:04 mybox kernel: load_dn_sched dn_sched WF2Q+ loaded Jul 8 08:09:04 mybox kernel: load_dn_sched dn_sched PRIO loaded Jul 8 08:09:04 mybox kernel: Limiting icmp unreach response from 2125 to 200 packets/sec Jul 8 08:09:05 mybox kernel: WARNING: attempt to domain_add(netgraph) after domainfinalize() Jul 8 08:09:06 mybox kernel: done Jul 8 08:09:07 mybox kernel: VMware memory control driver initialized Jul 8 08:09:07 mybox kernel: done =========================================== Second crash dump Jul 8 20:02:30 mybox syslogd: kernel boot file is /boot/kernel/kernel Jul 8 20:02:30 mybox kernel: Jul 8 20:02:30 mybox kernel: Jul 8 20:02:30 mybox kernel: Fatal trap 12: page fault while in kernel mode Jul 8 20:02:30 mybox kernel: cpuid = 2; apic id = 02 Jul 8 20:02:30 mybox kernel: fault virtual address = 0x28 Jul 8 20:02:30 mybox kernel: fault code = supervisor read data, page not present Jul 8 20:02:30 mybox kernel: instruction pointer = 0x20:0xffffffff81c38283 Jul 8 20:02:30 mybox kernel: stack pointer = 0x28:0xfffffe0174d99660 Jul 8 20:02:30 mybox kernel: frame pointer = 0x28:0xfffffe0174d996a0 Jul 8 20:02:30 mybox kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Jul 8 20:02:30 mybox kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Jul 8 20:02:30 mybox kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Jul 8 20:02:30 mybox kernel: current process = 627 (ng_queue3) Jul 8 20:02:30 mybox kernel: trap number = 12 Jul 8 20:02:30 mybox kernel: panic: page fault Jul 8 20:02:30 mybox kernel: cpuid = 2 Jul 8 20:02:30 mybox kernel: KDB: stack backtrace: Jul 8 20:02:30 mybox kernel: #0 0xffffffff8098e390 at kdb_backtrace+0x60 Jul 8 20:02:30 mybox kernel: #1 0xffffffff80951066 at vpanic+0x126 Jul 8 20:02:30 mybox kernel: #2 0xffffffff80950f33 at panic+0x43 Jul 8 20:02:30 mybox kernel: #3 0xffffffff80d55f7b at trap_fatal+0x36b Jul 8 20:02:30 mybox kernel: #4 0xffffffff80d5627d at trap_pfault+0x2ed Jul 8 20:02:30 mybox kernel: #5 0xffffffff80d558fa at trap+0x47a Jul 8 20:02:30 mybox kernel: #6 0xffffffff80d3b8d2 at calltrap+0x8 Jul 8 20:02:30 mybox kernel: #7 0xffffffff81c5a042 at ng_ppp_bypass+0x62 Jul 8 20:02:30 mybox kernel: #8 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 20:02:30 mybox kernel: #9 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 20:02:30 mybox kernel: #10 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 20:02:30 mybox kernel: #11 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 20:02:30 mybox kernel: #12 0xffffffff81c370ca at ng_apply_item+0x21a Jul 8 20:02:30 mybox kernel: #13 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 8 20:02:30 mybox kernel: #14 0xffffffff81c4d3e2 at ng_ksocket_incoming2+0x2f2 Jul 8 20:02:30 mybox kernel: #15 0xffffffff81c36f62 at ng_apply_item+0xb2 Jul 8 20:02:30 mybox kernel: #16 0xffffffff81c38d39 at ngthread+0x1b9 Jul 8 20:02:30 mybox kernel: #17 0xffffffff8091a4ea at fork_exit+0x9a Jul 8 20:02:30 mybox kernel: Uptime: 11h52m56s Jul 8 20:02:30 mybox kernel: Dumping 591 out of 6119 MB:..3%..11%..22%..33%..41%..52%..63%..71%..82%..92% Jul 8 20:02:30 mybox kernel: Dump complete Jul 8 20:02:30 mybox kernel: Copyright (c) 1992-2016 The FreeBSD Project. Jul 8 20:02:30 mybox kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Jul 8 20:02:30 mybox kernel: The Regents of the University of California. All rights reserved. Jul 8 20:02:30 mybox kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Jul 8 20:02:30 mybox kernel: FreeBSD 10.3-RELEASE-p4 #0: Sat May 28 12:23:44 UTC 2016 Jul 8 20:02:30 mybox kernel: root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Jul 8 20:02:30 mybox kernel: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Jul 8 20:02:30 mybox kernel: CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (2000.00-MHz K8-class CPU) Jul 8 20:02:30 mybox kernel: Origin="GenuineIntel" Id=0x206d7 Family=0x6 Model=0x2d Stepping=7 Jul 8 20:02:30 mybox kernel: Features=0x1fa3fbff Jul 8 20:02:30 mybox kernel: Features2=0x9fba2203 Jul 8 20:02:30 mybox kernel: AMD Features=0x28100800 Jul 8 20:02:30 mybox kernel: AMD Features2=0x1 Jul 8 20:02:30 mybox kernel: Structured Extended Features=0x2 Jul 8 20:02:30 mybox kernel: TSC: P-state invariant Jul 8 20:02:30 mybox kernel: Hypervisor: Origin = "VMwareVMware" Jul 8 20:02:30 mybox kernel: real memory = 6442450944 (6144 MB) Jul 8 20:02:30 mybox kernel: avail memory = 6183518208 (5897 MB) Jul 8 20:02:30 mybox kernel: Event timer "LAPIC" quality 600 Jul 8 20:02:30 mybox kernel: ACPI APIC Table: Jul 8 20:02:30 mybox kernel: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs Jul 8 20:02:30 mybox kernel: FreeBSD/SMP: 1 package(s) x 4 core(s) Jul 8 20:02:30 mybox kernel: cpu0 (BSP): APIC ID: 0 Jul 8 20:02:30 mybox kernel: cpu1 (AP): APIC ID: 1 Jul 8 20:02:30 mybox kernel: cpu2 (AP): APIC ID: 2 Jul 8 20:02:30 mybox kernel: cpu3 (AP): APIC ID: 3 Jul 8 20:02:30 mybox kernel: random: initialized Jul 8 20:02:30 mybox kernel: MADT: Forcing active-low polarity and level trigger for SCI Jul 8 20:02:30 mybox kernel: ioapic0 irqs 0-23 on motherboard Jul 8 20:02:30 mybox kernel: kbd1 at kbdmux0 Jul 8 20:02:30 mybox kernel: acpi0: on motherboard Jul 8 20:02:30 mybox kernel: acpi0: Power Button (fixed) Jul 8 20:02:30 mybox kernel: Timecounter "HPET" frequency 14318180 Hz quality 950 Jul 8 20:02:30 mybox kernel: cpu0: on acpi0 Jul 8 20:02:30 mybox kernel: cpu1: on acpi0 Jul 8 20:02:30 mybox kernel: cpu2: on acpi0 Jul 8 20:02:30 mybox kernel: cpu3: on acpi0 Jul 8 20:02:30 mybox kernel: attimer0: port 0x40-0x43 irq 0 on acpi0 Jul 8 20:02:30 mybox kernel: Timecounter "i8254" frequency 1193182 Hz quality 0 Jul 8 20:02:30 mybox kernel: Event timer "i8254" frequency 1193182 Hz quality 100 Jul 8 20:02:30 mybox kernel: atrtc0: port 0x70-0x71 irq 8 on acpi0 Jul 8 20:02:30 mybox kernel: Event timer "RTC" frequency 32768 Hz quality 0 Jul 8 20:02:30 mybox kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 Jul 8 20:02:30 mybox kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 Jul 8 20:02:30 mybox kernel: pcib0: port 0xcf8-0xcff on acpi0 Jul 8 20:02:30 mybox kernel: pci0: on pcib0 Jul 8 20:02:30 mybox kernel: pcib1: at device 1.0 on pci0 Jul 8 20:02:30 mybox kernel: pci1: on pcib1 Jul 8 20:02:30 mybox kernel: isab0: at device 7.0 on pci0 Jul 8 20:02:30 mybox kernel: isa0: on isab0 Jul 8 20:02:30 mybox kernel: atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1060-0x106f at device 7.1 on pci0 Jul 8 20:02:30 mybox kernel: ata0: at channel 0 on atapci0 Jul 8 20:02:30 mybox kernel: ata1: at channel 1 on atapci0 Jul 8 20:02:30 mybox kernel: pci0: at device 7.3 (no driver attached) Jul 8 20:02:30 mybox kernel: vgapci0: port 0x1070-0x107f mem 0xe8000000-0xefffffff,0xfe000000-0xfe7fffff irq 16 at device 15.0 on pci0 Jul 8 20:02:30 mybox kernel: vgapci0: Boot video device Jul 8 20:02:30 mybox kernel: mpt0: port 0x1400-0x14ff mem 0xfeba0000-0xfebbffff,0xfebc0000-0xfebdffff irq 17 at device 16.0 on pci0 Jul 8 20:02:30 mybox kernel: mpt0: MPI Version=1.2.0.0 Jul 8 20:02:30 mybox kernel: pcib2: at device 17.0 on pci0 Jul 8 20:02:30 mybox kernel: pci2: on pcib2 Jul 8 20:02:30 mybox kernel: em0: port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2 Jul 8 20:02:30 mybox kernel: em0: Ethernet address: 00:50:56:8d:d7:57 Jul 8 20:02:30 mybox kernel: pcib3: at device 21.0 on pci0 Jul 8 20:02:30 mybox kernel: pci3: on pcib3 Jul 8 20:02:30 mybox kernel: vmx0: port 0x4000-0x400f mem 0xfd4fc000-0xfd4fcfff,0xfd4fd000-0xfd4fdfff,0xfd4fe000-0xfd4fffff irq 18 at device 0.0 on pci3 Jul 8 20:02:30 mybox kernel: vmx0: Ethernet address: 00:50:56:8d:4c:68 Jul 8 20:02:30 mybox kernel: pcib4: at device 21.1 on pci0 Jul 8 20:02:30 mybox kernel: pci4: on pcib4 Jul 8 20:02:30 mybox kernel: pcib5: at device 21.2 on pci0 Jul 8 20:02:30 mybox kernel: pci5: on pcib5 Jul 8 20:02:30 mybox kernel: pcib6: at device 21.3 on pci0 Jul 8 20:02:30 mybox kernel: pci6: on pcib6 Jul 8 20:02:30 mybox kernel: pcib7: at device 21.4 on pci0 Jul 8 20:02:30 mybox kernel: pci7: on pcib7 Jul 8 20:02:30 mybox kernel: pcib8: at device 21.5 on pci0 Jul 8 20:02:30 mybox kernel: pci8: on pcib8 Jul 8 20:02:30 mybox kernel: pcib9: at device 21.6 on pci0 Jul 8 20:02:30 mybox kernel: pci9: on pcib9 Jul 8 20:02:30 mybox kernel: pcib10: at device 21.7 on pci0 Jul 8 20:02:30 mybox kernel: pci10: on pcib10 Jul 8 20:02:30 mybox kernel: pcib11: at device 22.0 on pci0 Jul 8 20:02:30 mybox kernel: pci11: on pcib11 Jul 8 20:02:30 mybox kernel: vmx1: port 0x5000-0x500f mem 0xfd3fc000-0xfd3fcfff,0xfd3fd000-0xfd3fdfff,0xfd3fe000-0xfd3fffff irq 19 at device 0.0 on pci11 Jul 8 20:02:30 mybox kernel: vmx1: Ethernet address: 00:50:56:8d:85:29 Jul 8 20:02:30 mybox kernel: pcib12: at device 22.1 on pci0 Jul 8 20:02:30 mybox kernel: pci12: on pcib12 Jul 8 20:02:30 mybox kernel: pcib13: at device 22.2 on pci0 Jul 8 20:02:30 mybox kernel: pci13: on pcib13 Jul 8 20:02:30 mybox kernel: pcib14: at device 22.3 on pci0 Jul 8 20:02:30 mybox kernel: pci14: on pcib14 Jul 8 20:02:30 mybox kernel: pcib15: at device 22.4 on pci0 Jul 8 20:02:30 mybox kernel: pci15: on pcib15 Jul 8 20:02:30 mybox kernel: pcib16: at device 22.5 on pci0 Jul 8 20:02:30 mybox kernel: pci16: on pcib16 Jul 8 20:02:30 mybox kernel: pcib17: at device 22.6 on pci0 Jul 8 20:02:30 mybox kernel: pci17: on pcib17 Jul 8 20:02:30 mybox kernel: pcib18: at device 22.7 on pci0 Jul 8 20:02:30 mybox kernel: pci18: on pcib18 Jul 8 20:02:30 mybox kernel: pcib19: at device 23.0 on pci0 Jul 8 20:02:30 mybox kernel: pci19: on pcib19 Jul 8 20:02:30 mybox kernel: pcib20: at device 23.1 on pci0 Jul 8 20:02:30 mybox kernel: pci20: on pcib20 Jul 8 20:02:30 mybox kernel: pcib21: at device 23.2 on pci0 Jul 8 20:02:30 mybox kernel: pci21: on pcib21 Jul 8 20:02:30 mybox kernel: pcib22: at device 23.3 on pci0 Jul 8 20:02:30 mybox kernel: pci22: on pcib22 Jul 8 20:02:30 mybox kernel: pcib23: at device 23.4 on pci0 Jul 8 20:02:30 mybox kernel: pci23: on pcib23 Jul 8 20:02:30 mybox kernel: pcib24: at device 23.5 on pci0 Jul 8 20:02:30 mybox kernel: pci24: on pcib24 Jul 8 20:02:30 mybox kernel: pcib25: at device 23.6 on pci0 Jul 8 20:02:30 mybox kernel: pci25: on pcib25 Jul 8 20:02:30 mybox kernel: pcib26: at device 23.7 on pci0 Jul 8 20:02:30 mybox kernel: pci26: on pcib26 Jul 8 20:02:30 mybox kernel: pcib27: at device 24.0 on pci0 Jul 8 20:02:30 mybox kernel: pci27: on pcib27 Jul 8 20:02:30 mybox kernel: pcib28: at device 24.1 on pci0 Jul 8 20:02:30 mybox kernel: pci28: on pcib28 Jul 8 20:02:30 mybox kernel: pcib29: at device 24.2 on pci0 Jul 8 20:02:30 mybox kernel: pci29: on pcib29 Jul 8 20:02:30 mybox kernel: pcib30: at device 24.3 on pci0 Jul 8 20:02:30 mybox kernel: pci30: on pcib30 Jul 8 20:02:30 mybox kernel: pcib31: at device 24.4 on pci0 Jul 8 20:02:30 mybox kernel: pci31: on pcib31 Jul 8 20:02:30 mybox kernel: pcib32: at device 24.5 on pci0 Jul 8 20:02:30 mybox kernel: pci32: on pcib32 Jul 8 20:02:30 mybox kernel: pcib33: at device 24.6 on pci0 Jul 8 20:02:30 mybox kernel: pci33: on pcib33 Jul 8 20:02:30 mybox kernel: pcib34: at device 24.7 on pci0 Jul 8 20:02:30 mybox kernel: pci34: on pcib34 Jul 8 20:02:30 mybox kernel: acpi_acad0: on acpi0 Jul 8 20:02:30 mybox kernel: atkbdc0: port 0x60,0x64 irq 1 on acpi0 Jul 8 20:02:30 mybox kernel: atkbd0: irq 1 on atkbdc0 Jul 8 20:02:30 mybox kernel: kbd0 at atkbd0 Jul 8 20:02:30 mybox kernel: atkbd0: [GIANT-LOCKED] Jul 8 20:02:30 mybox kernel: psm0: irq 12 on atkbdc0 Jul 8 20:02:30 mybox kernel: psm0: [GIANT-LOCKED] Jul 8 20:02:30 mybox kernel: psm0: model IntelliMouse, device ID 3 Jul 8 20:02:30 mybox kernel: fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 Jul 8 20:02:30 mybox kernel: fd0: <1440-KB 3.5" drive> on fdc0 drive 0 Jul 8 20:02:30 mybox kernel: orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xdc000-0xdffff,0xe0000-0xe7fff on isa0 Jul 8 20:02:30 mybox kernel: sc0: at flags 0x100 on isa0 Jul 8 20:02:30 mybox kernel: sc0: VGA <16 virtual consoles, flags=0x300> Jul 8 20:02:30 mybox kernel: vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Jul 8 20:02:30 mybox kernel: ppc0: cannot reserve I/O port range Jul 8 20:02:30 mybox kernel: Timecounters tick every 1.000 msec Jul 8 20:02:30 mybox kernel: em0: link state changed to UP Jul 8 20:02:30 mybox kernel: random: unblocking device. Jul 8 20:02:30 mybox kernel: cd0 at ata1 bus 0 scbus1 target 0 lun 0 Jul 8 20:02:30 mybox kernel: cd0: Removable CD-ROM SCSI device Jul 8 20:02:30 mybox kernel: cd0: Serial Number 10000000000000000001 Jul 8 20:02:30 mybox kernel: cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) Jul 8 20:02:30 mybox kernel: cd0: Attempt to query device size failed: NOT READY, Medium not present Jul 8 20:02:30 mybox kernel: cd0: quirks=0x40 Jul 8 20:02:30 mybox kernel: da0 at mpt0 bus 0 scbus2 target 0 lun 0 Jul 8 20:02:30 mybox kernel: da0: Fixed Direct Access SCSI-2 device Jul 8 20:02:30 mybox kernel: da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) Jul 8 20:02:30 mybox kernel: da0: Command Queueing enabled Jul 8 20:02:30 mybox kernel: da0: 12288MB (25165824 512 byte sectors) Jul 8 20:02:30 mybox kernel: da0: quirks=0x40 Jul 8 20:02:30 mybox kernel: SMP: AP CPU #1 Launched! Jul 8 20:02:30 mybox kernel: SMP: AP CPU #2 Launched! Jul 8 20:02:30 mybox kernel: SMP: AP CPU #3 Launched! Jul 8 20:02:30 mybox kernel: Trying to mount root from ufs:/dev/da0p2 [rw]... Jul 8 20:02:30 mybox kernel: WARNING: / was not properly dismounted Jul 8 20:02:30 mybox kernel: vmx0: link state changed to UP Jul 8 20:02:30 mybox kernel: vmx1: link state changed to UP Jul 8 20:02:30 mybox kernel: Limiting icmp unreach response from 2687 to 200 packets/sec Jul 8 20:02:30 mybox kernel: Limiting icmp unreach response from 2254 to 200 packets/sec Jul 8 20:02:30 mybox kernel: ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to deny, logging disabled Jul 8 20:02:30 mybox kernel: DUMMYNET 0 with IPv6 initialized (100409) Jul 8 20:02:30 mybox kernel: load_dn_sched dn_sched FIFO loaded Jul 8 20:02:30 mybox kernel: load_dn_sched dn_sched QFQ loaded Jul 8 20:02:30 mybox kernel: load_dn_sched dn_sched RR loaded Jul 8 20:02:30 mybox kernel: load_dn_sched dn_sched WF2Q+ loaded Jul 8 20:02:30 mybox kernel: load_dn_sched dn_sched PRIO loaded Jul 8 20:02:30 mybox kernel: Limiting icmp unreach response from 2374 to 200 packets/sec Jul 8 20:02:30 mybox kernel: Limiting icmp unreach response from 2602 to 200 packets/sec Jul 8 20:02:31 mybox kernel: Limiting icmp unreach response from 2283 to 200 packets/sec Jul 8 20:02:32 mybox kernel: Limiting icmp unreach response from 2333 to 200 packets/sec Jul 8 20:02:33 mybox kernel: Limiting icmp unreach response from 2217 to 200 packets/sec Jul 8 20:02:34 mybox kernel: Limiting icmp unreach response from 2134 to 200 packets/sec Jul 8 20:02:35 mybox kernel: Limiting icmp unreach response from 2064 to 200 packets/sec Jul 8 20:02:36 mybox kernel: Limiting icmp unreach response from 1662 to 200 packets/sec Jul 8 20:02:37 mybox kernel: Limiting icmp unreach response from 1704 to 200 packets/sec Jul 8 20:02:38 mybox kernel: Limiting icmp unreach response from 1632 to 200 packets/sec Jul 8 20:02:39 mybox kernel: Limiting icmp unreach response from 1472 to 200 packets/sec Jul 8 20:02:41 mybox kernel: Limiting icmp unreach response from 1494 to 200 packets/sec Jul 8 20:02:42 mybox kernel: Limiting icmp unreach response from 1474 to 200 packets/sec Jul 8 20:02:43 mybox kernel: Limiting icmp unreach response from 1520 to 200 packets/sec Jul 8 20:02:44 mybox kernel: Limiting icmp unreach response from 1409 to 200 packets/sec Jul 8 20:02:45 mybox kernel: Limiting icmp unreach response from 1227 to 200 packets/sec Jul 8 20:02:46 mybox kernel: Limiting icmp unreach response from 1264 to 200 packets/sec Jul 8 20:02:47 mybox kernel: Limiting icmp unreach response from 1105 to 200 packets/sec Jul 8 20:02:48 mybox kernel: Limiting icmp unreach response from 1049 to 200 packets/sec Jul 8 20:02:49 mybox kernel: Limiting icmp unreach response from 971 to 200 packets/sec Jul 8 20:02:50 mybox kernel: Limiting icmp unreach response from 882 to 200 packets/sec Jul 8 20:02:51 mybox kernel: Limiting icmp unreach response from 906 to 200 packets/sec Jul 8 20:02:52 mybox kernel: Limiting icmp unreach response from 826 to 200 packets/sec Jul 8 20:02:53 mybox kernel: Limiting icmp unreach response from 721 to 200 packets/sec Jul 8 20:02:54 mybox kernel: Limiting icmp unreach response from 666 to 200 packets/sec Jul 8 20:02:55 mybox kernel: Limiting icmp unreach response from 582 to 200 packets/sec Jul 8 20:02:56 mybox kernel: Limiting icmp unreach response from 551 to 200 packets/sec Jul 8 20:02:57 mybox kernel: Limiting icmp unreach response from 514 to 200 packets/sec Jul 8 20:02:57 mybox kernel: . Jul 8 20:02:57 mybox kernel: WARNING: attempt to domain_add(netgraph) after domainfinalize() Jul 8 20:02:58 mybox kernel: ipfw: rule 50: setsockopt(IP_FW_DEL): Invalid argument Jul 8 20:02:58 mybox kernel: done Jul 8 20:02:59 mybox kernel: Jul 8 20:02:59 mybox kernel: VMware memory control driver initialized Jul 8 20:02:59 mybox kernel: done ============================ From owner-freebsd-net@freebsd.org Sun Jul 10 21:14:17 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07B4BB83C40 for ; Sun, 10 Jul 2016 21:14:17 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FA1418DA for ; Sun, 10 Jul 2016 21:14:16 +0000 (UTC) (envelope-from hoomanfazaeli@gmail.com) Received: by mail-wm0-x234.google.com with SMTP id k123so72188192wme.0 for ; Sun, 10 Jul 2016 14:14:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=meoUsG+Hvc59Q4NrMgX28f1fqPk14f1UgiZQy9uSW+k=; b=qg27N8sNiTYotVIGdY54MeM9UlvknhfR/CWKuQRlQHEJIN5OhcejOwzCgL+meEw+GD igwf5NPZiENpNOlMddEWRm00v2UzdBZxCdltemnxTalLbo0c1sqGos34REDTRdzFEN+u 7jk2NgbuLj86ezbO2pw/zZ8XUHUk/hDWu9j0QlQVvgzLlWQTRjbWf+p4uenVWTWpZEEF qE6Ta4iFRzSvISgfV4+MA90qMjulozDSvvPPHPTo8HEIasS6yHS5ZkOaj+wQcXGYAmsl S82Prq7TDvFG8rVMjVGR0lM+P/yiP2yrYn17a23z7NRAl1kf2nGhHVSm2Ym2OyunGE3q lYZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=meoUsG+Hvc59Q4NrMgX28f1fqPk14f1UgiZQy9uSW+k=; b=hPF5CaOe+Xw3wDQh/MnLMLnePI/aUbIGClPG4QrFfDbKyMfOURTEc5ob1gMw6jtiIH 9kD0fYCVhpySC66mJjL7HoMV50KbN3J6aj+3tUTtRYLoeK9OB4ZFPpc8UC9iRGC9VHY3 yKk8PN9PidXC9e1O75w4dmrPfMYnmILQHl+j9flByHnTVucFs5Jm+qVz72npWuilOcbw uVCEzs56oDQzDPcilQeEEJgS20y+ZyHVu0FxMSXZua2/GXpYt0hq2MrPcJYko9BqKqls nD1uA7UoEgezufiDtjbIA4rftyUbB8K3wmjj2d1lf5DiTqMDFlspIttr7+ioT1MjgRww xsAw== X-Gm-Message-State: ALyK8tL7aEcUPt690DN4Qy+QejCGklDP6DPXvIojRyOrRLlRifhBH5hUGXl2Ld/uQKLCsA== X-Received: by 10.28.228.86 with SMTP id b83mr13603002wmh.11.1468185255209; Sun, 10 Jul 2016 14:14:15 -0700 (PDT) Received: from [192.168.2.30] ([2.190.254.38]) by smtp.googlemail.com with ESMTPSA id a198sm17085009wme.16.2016.07.10.14.14.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 10 Jul 2016 14:14:14 -0700 (PDT) Message-ID: <5782BAA4.2090606@gmail.com> Date: Mon, 11 Jul 2016 01:44:12 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: donaldbaud@yahoo.com CC: freebsd-net Subject: Re: kernel panic with netgraph and mpd3.8 References: <58413371.417578.1468131588156.JavaMail.yahoo.ref@mail.yahoo.com> <58413371.417578.1468131588156.JavaMail.yahoo@mail.yahoo.com> In-Reply-To: <58413371.417578.1468131588156.JavaMail.yahoo@mail.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jul 2016 21:14:17 -0000 On 2016-07-10 10:49, Donald Baud via freebsd-net wrote: > Hi I'm running an l2tp lns through mpd3.8 and it's been crashing twice in 24h. > This is a new project replacing a cisco 7206, 700-sessions 800mbit/s > > > I am not familiar with troubleshooting kernel panic's, > > I am suspecting that the crash is happening inside the netgraph module because the crash is happening at the > > instruction pointer = 0x20:0xffffffff81c38283 > I included the 2 two crash logs. I need some help to to figure out what to do next. > > -Dbaud > > - Upgrade to mpd 5 (/usr/ports/net/mpd5) - Try below workarounds: https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056548.html https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056549.html https://lists.freebsd.org/pipermail/freebsd-net/2014-June/038954.html -- Best regards Hooman Fazaeli From owner-freebsd-net@freebsd.org Mon Jul 11 00:43:16 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93EB5B84966 for ; Mon, 11 Jul 2016 00:43:16 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) Received: from nm26-vm0.bullet.mail.bf1.yahoo.com (nm26-vm0.bullet.mail.bf1.yahoo.com [98.139.213.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B6951D3F for ; Mon, 11 Jul 2016 00:43:15 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1468197789; bh=91GC+Q83m2jDD+E3XQUQx+e9L1mYI5HUJu/0HzkqzLg=; h=Subject:To:References:From:Date:In-Reply-To:From:Subject; b=cG/gajeBd1fPjekzbBICxLuj93Czpq2l4aoz3MifLNaQw/ylq7uRxUTFDs6HrysJqpWXdu0wz0c9iQ5opCSc1dgCk2TlcyhSehkar8w3/3DyhifhaTaa6OArZMxZpav4WutaUUmX4FwXY605kmAeaw75tX0W4LQxeVM01mpc/7LfIMEcyHlpjdxkODX9QXufq0f2h5VYeJoSW1/o6IGwQi6P/NJRBYXX/l7J/QVqXh4dGjJMY5FayG5onocZ82ta9DJbFMxYARJL0w01PclOsliV3WILZWfpWDbkaWOzNVCHXstBuPh448To/TmpL70dz50nhtp80tkLba0p5DyOOw== Received: from [98.139.215.142] by nm26.bullet.mail.bf1.yahoo.com with NNFMP; 11 Jul 2016 00:43:09 -0000 Received: from [98.139.211.205] by tm13.bullet.mail.bf1.yahoo.com with NNFMP; 11 Jul 2016 00:43:09 -0000 Received: from [127.0.0.1] by smtp214.mail.bf1.yahoo.com with NNFMP; 11 Jul 2016 00:43:09 -0000 X-Yahoo-Newman-Id: 370315.65779.bm@smtp214.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: Va6DLhgVM1kaGuUcflz9XGtTCgQKUisWPVvmcQ.yunYzkvU VhOEytOX41BDZ_wCE01bmVZNuLKT1vWAbiT5dprKfGAajxl8IABgs13_Bufg zd.7aC8LSivwajrvp.3S1s4JLAnwlApaOAYsQWN2AFdmdmZWtqkmrI2Q79Yd .WJ.nmpHjuQ2nlOpIGS9jOGMhyRD0kLK6BUIrR9VoD43sU57Rd6ZrTxf8mZY kJGJN7AQWb5jmOTw_jotBkQ9rziSaDB17AFQ2Y.gs5wMsTAVhWTHlCbt2yN2 Cgu1XDHDf0sPgwGs2KQVdWPlerhH.6NED3MG8HBqo4KPX6Ck5pmE7dnyv1zk MEmbrBcAJq5x8u53V8w4_EE1gUf8W5L2ntjmXT.dKrBARvprCQbYOLk6CArD yiTlaDFvSKj_Vs5FJ5NjgvH74s4yOnSdYQ2C1DRHzE23Y6WPr2IA2Ds53JMZ JBnDjm8O734415Myhg9UWtytWxgSYeU5j_pJjuu4Z1fyGemQw7DXZ5_FoIPK ObnMoUeY8FHu62vkflXcIL4AoriUQB_Uop2S0_NyXameAfJw0BXZvn5w- X-Yahoo-SMTP: fRRiyyGswBB1GYfBhw8frTz_QYRdpxY- Subject: Re: kernel panic with netgraph and mpd3.8 To: freebsd-net References: <58413371.417578.1468131588156.JavaMail.yahoo.ref@mail.yahoo.com> <58413371.417578.1468131588156.JavaMail.yahoo@mail.yahoo.com> <5782BAA4.2090606@gmail.com> From: Donald Baud Message-ID: <3b3c7a0a-35d4-93c5-c283-1b03f627ac38@yahoo.com> Date: Sun, 10 Jul 2016 20:43:11 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <5782BAA4.2090606@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2016 00:43:16 -0000 > On 2016-07-10 10:49, Donald Baud via freebsd-net wrote: >> Hi I'm running an l2tp lns through mpd3.8 and it's been crashing >> twice in 24h. >> This is a new project replacing a cisco 7206, 700-sessions 800mbit/s >> >> >> I am not familiar with troubleshooting kernel panic's, >> >> I am suspecting that the crash is happening inside the netgraph >> module because the crash is happening at the >> >> instruction pointer = 0x20:0xffffffff81c38283 >> I included the 2 two crash logs. I need some help to to figure out >> what to do next. >> >> -Dbaud >> >> > On 7/10/16 5:14 PM, Hooman Fazaeli wrote: > - Upgrade to mpd 5 (/usr/ports/net/mpd5) > - Try below workarounds: > https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056548.html > https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056549.html > https://lists.freebsd.org/pipermail/freebsd-net/2014-June/038954.html > > - I'm already using the latest mpd5: > mpd5 --version Version 5.8 (root@101amd64-quarterly-job-15 12:36 5-Jun-2016) - I had already reviewed those links you mentioned. Here is a summary of the main suggestions in them. * Add a "sleep 1" to up-down interface events. * Revert to RELENG8 or 9 * boost net.graph sysctl/loader.conf net.graph.maxdata=262140 # /boot/loader.conf net.graph.maxalloc=262140 # /boot.loader.conf I was using the following tunings net.graph.maxdgram=524288 (via sysctl.conf default=20480) net.graph.recvspace=524288 (via sysctl.conf default=20480) net.graph.maxdata=65536 (via loader.conf default=4096 ) net.graph.maxalloc=65536 (via loader.conf default=4096 ) I am suspecting that the panic might be caused by a too high maxdata and maxalloc values: I reduced the value to 20480, I'll report back if that will reduce the occurence of kernel panics. > vmstat -z | head -1 ; vmstat -z | grep -i graph ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP NetGraph items: 72, 20491, 2, 1672,467166841, 0, 0 NetGraph data items: 72, 20491, 0, 1643,1240166475, 0, 0 From owner-freebsd-net@freebsd.org Mon Jul 11 08:58:45 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EC5DB91C95 for ; Mon, 11 Jul 2016 08:58:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0DA8818BA for ; Mon, 11 Jul 2016 08:58:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6B8wi4j065549 for ; Mon, 11 Jul 2016 08:58:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 206186] [patch][netgraph] New netgraph node for calculate IP IP6 TCP UDP checksums Date: Mon, 11 Jul 2016 08:58:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: daemon.hammer@ya.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2016 08:58:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206186 Dmitry Vagin changed: What |Removed |Added ---------------------------------------------------------------------------- Version|10.2-STABLE |10.3-STABLE --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 11 10:06:23 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1E35B84708 for ; Mon, 11 Jul 2016 10:06:23 +0000 (UTC) (envelope-from no-reply@x55.vip.6pm-coupon.com) Received: from x55.vip.6pm-coupon.com (x55.vip.6pm-coupon.com [104.148.25.55]) by mx1.freebsd.org (Postfix) with ESMTP id 962731C5C for ; Mon, 11 Jul 2016 10:06:23 +0000 (UTC) (envelope-from no-reply@x55.vip.6pm-coupon.com) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=6pm-coupon; d=x55.vip.6pm-coupon.com; h=MIME-Version:From:To:Date:Subject:Content-Type:Content-Transfer-Encoding; i=no-reply@x55.vip.6pm-coupon.com; bh=1ctpmRzknA8TbVM+PgRvUQpEek0=; b=cM5nShV46KSAqIK4M0KtskFdO7do/q+fWyGTgpspJGSTKAS46+MgWzyxrrgdOyj7RipmlTJBdIS3 Rw393Qclwku/WzrFmu4rZ5PBwQkJGfF0m/rQfaukD981ffp6Manwyr27Wp8u4vSESnQR/Y8gLzUI BaJ+GRSWBhg1PXwzboY= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=6pm-coupon; d=x55.vip.6pm-coupon.com; b=oSnxmtGbX1MXRDkf3H3pQrgxP+UKkGn2i4HpDd+IU5X/1lBgxVDtwzuaP9VUF2NqzAxo9hYMgeah 3fGiE746XO8ejZ9ydu2cq+WlHlvJQdA44DUaMvjK23LLO/kPXqaJO3rBw0nEgKP75+FqffrfWQxK VOJ6LunbaIiI60URp18=; From: "Ray.Ban Sunglasses" To: freebsd-net@freebsd.org Date: 11 Jul 2016 17:46:00 +0800 Subject: Direct Sale:Free Delivery and best-selling items update724 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2016 10:06:24 -0000 From owner-freebsd-net@freebsd.org Mon Jul 11 21:08:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B642B920ED for ; Mon, 11 Jul 2016 21:08:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3ACCB1ED5 for ; Mon, 11 Jul 2016 21:08:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6BL8HMV085631 for ; Mon, 11 Jul 2016 21:08:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210901] em0 stores packets somewhere and lets them out slowly under load. Date: Mon, 11 Jul 2016 21:08:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: hiren@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2016 21:08:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210901 Hiren Panchasara changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |IntelNetworking CC| |sbruno@FreeBSD.org Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 12 03:33:18 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A62DEB91E00 for ; Tue, 12 Jul 2016 03:33:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95B6915FB for ; Tue, 12 Jul 2016 03:33:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6C3XInM059063 for ; Tue, 12 Jul 2016 03:33:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210901] em0 stores packets somewhere and lets them out slowly under load. Date: Tue, 12 Jul 2016 03:33:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: IntelNetworking X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: sbruno@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2016 03:33:18 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210901 --- Comment #1 from Sean Bruno --- A couple of things to try while we are doing an overhaul to em(4): hw.em.rx_process_limit: -1 hw.em.txd: 4096 hw.em.rxd: 4096 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Jul 12 10:28:33 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09D22B91F87 for ; Tue, 12 Jul 2016 10:28:33 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) Received: from nm13-vm0.bullet.mail.bf1.yahoo.com (nm13-vm0.bullet.mail.bf1.yahoo.com [98.139.213.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C3F701D07 for ; Tue, 12 Jul 2016 10:28:32 +0000 (UTC) (envelope-from donaldbaud@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1468319130; bh=s5T5qPp2Jv8hhamb19udnqNLjYlo9RKdo6jE9fsFSjY=; h=Subject:To:References:From:Date:In-Reply-To:From:Subject; b=C5LAPrLX0SMzxZJnXX3A4RmWZ3rKx1Ygr0B0o5S/17uHIH3RgfDggodnzlRRisDDfW4ob4hQs2KjvnFIwng7DytiCaW2MlLKYHVEF5dewPknVUgi4DhZAIZUWIMH1ozVEHehcE3sTGo+FlNFZWU5soPLGo0cTlwamRUugNRzvCWmFwsE1LyFjnoql2AjACLAx0/fKG3OEjC58Stl3eDIAIigCFDW5/X8uLOWSyuurGcHeuJOgxQpP/jzc2whl/DNtcJfxdose5xlwixPFC0ZZN7QQoZaHwYbPFd4eDeaaLQrxNc5a0EWPuoCpVpc9HpJP08DQwz/VywnHqQC61oWJA== Received: from [66.196.81.171] by nm13.bullet.mail.bf1.yahoo.com with NNFMP; 12 Jul 2016 10:25:30 -0000 Received: from [68.142.230.76] by tm17.bullet.mail.bf1.yahoo.com with NNFMP; 12 Jul 2016 10:25:30 -0000 Received: from [127.0.0.1] by smtp233.mail.bf1.yahoo.com with NNFMP; 12 Jul 2016 10:25:30 -0000 X-Yahoo-Newman-Id: 751801.52722.bm@smtp233.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: J9ZSep0VM1lzvDeejmjdp0oZf09RVQdUY35aiOxN7uLXSOC E89rq9YTdXd9QCZhRP..e7XiXpoGOTGMeGQVrQbq95lBKkUNVAsLkzGbAf2R wHvNe0aCJFXyVotn0LOUMcPk2QG8ur3Mdsntk9uBsCpa0JJZAe6nRunUTQs2 guL9fHkOCqtQ2zoZwcjbhg4czuKDnQsFBb_eejvmOqE_B1VakeNEl8Ci0_pO jLXSZwfCaQY26OTQmEtNwihofAVw.u_Ijiq9ZyPgI.iXNw8.F.5VYzPvY.vj oLPDwUAUpUSiT2dmzSOYdwOoQLY3.9qiPpJ2NrzfWUGqp_q9qE4V4R9exFVL VeI5ndKZO8mxBHhjcbxgkq6579oXh8Hn1WGBcGq1n7r9KSgq_Ib3pcLDF7Uo Cr3qCyNave98ZUy7GfhZhepAmyDC6AeLJNnAm_QcUo9VV_P3RFFRPNMJHDir fZ7ZkgofAqpiE5xJQJU0SRddOOlCMUNmvoyisqKaudNdRKaQGq5QEiRwjMOK UDo.sQ.e3yrvQrRaKOqrhscf_hIsD.iDsxuttChPKyT47FJyX.m7A1A-- X-Yahoo-SMTP: fRRiyyGswBB1GYfBhw8frTz_QYRdpxY- Subject: Re: kernel panic with netgraph and mpd5.8 To: freebsd-net@freebsd.org References: <58413371.417578.1468131588156.JavaMail.yahoo.ref@mail.yahoo.com> <58413371.417578.1468131588156.JavaMail.yahoo@mail.yahoo.com> <5782BAA4.2090606@gmail.com> <3b3c7a0a-35d4-93c5-c283-1b03f627ac38@yahoo.com> From: Donald Baud Message-ID: <2c786d19-7ba2-10ad-89a0-9a78abfe7146@yahoo.com> Date: Tue, 12 Jul 2016 06:25:36 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <3b3c7a0a-35d4-93c5-c283-1b03f627ac38@yahoo.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2016 10:28:33 -0000 >> On 2016-07-10 10:49, Donald Baud via freebsd-net wrote: >>> Hi I'm running an l2tp lns through mpd5.8 and it's been crashing >>> twice in 24h. >>> This is a new project replacing a cisco 7206, 700-sessions 800mbit/s >>> >>> >>> I am not familiar with troubleshooting kernel panic's, >>> >>> I am suspecting that the crash is happening inside the netgraph >>> module because the crash is happening at the >>> >>> instruction pointer = 0x20:0xffffffff81c38283 >>> I included the 2 two crash logs. I need some help to to figure out >>> what to do next. >>> >>> -Dbaud >>> >>> >> On 7/10/16 5:14 PM, Hooman Fazaeli wrote: >> - Upgrade to mpd 5 (/usr/ports/net/mpd5) >> - Try below workarounds: >> https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056548.html >> https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056549.html >> https://lists.freebsd.org/pipermail/freebsd-net/2014-June/038954.html >> >> > On 7/10/16 8:43 PM, Donald Baud via freebsd-net wrote: > - I'm already using the latest mpd5: > > mpd5 --version > Version 5.8 (root@101amd64-quarterly-job-15 12:36 5-Jun-2016) > > - I had already reviewed those links you mentioned. > Here is a summary of the main suggestions in them. > * Add a "sleep 1" to up-down interface events. > * Revert to RELENG8 or 9 > * boost net.graph sysctl/loader.conf > net.graph.maxdata=262140 # /boot/loader.conf > net.graph.maxalloc=262140 # /boot.loader.conf > > I was using the following tunings > net.graph.maxdgram=524288 (via sysctl.conf default=20480) > net.graph.recvspace=524288 (via sysctl.conf default=20480) > net.graph.maxdata=65536 (via loader.conf default=4096 ) > net.graph.maxalloc=65536 (via loader.conf default=4096 ) > > I am suspecting that the panic might be caused by a too high maxdata > and maxalloc values: > I reduced the value to 20480, I'll report back if that will reduce the > occurence of kernel panics. > >> vmstat -z | head -1 ; vmstat -z | grep -i graph > ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP > NetGraph items: 72, 20491, 2, 1672,467166841, 0, 0 > NetGraph data items: 72, 20491, 0, 1643,1240166475, 0, 0 > > The server crashed again this morning. It looks like it crashes somewhere in the netgraph.ko module Could someone please help me troubleshoot this issue, it crashes around the same location instruction pointer = 0x20:0xffffffff81c3828d The crash happens at random times not necessarily under heavy load. - using plain GENERIC kernel 10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat May 28 12:23:44 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 - # kldstat Id Refs Address Size Name 1 32 0xffffffff80200000 17bc6a8 kernel 2 2 0xffffffff81c11000 114db ipfw.ko 3 1 0xffffffff81c23000 d32f dummynet.ko 4 1 0xffffffff81c31000 3831 ng_socket.ko 5 8 0xffffffff81c35000 ba02 netgraph.ko 6 1 0xffffffff81c41000 2b99 ng_mppc.ko 7 1 0xffffffff81c44000 80c rc4.ko 8 1 0xffffffff81c45000 23dc vmmemctl.ko 9 1 0xffffffff81c48000 397d ng_l2tp.ko 10 1 0xffffffff81c4c000 4b04 ng_ksocket.ko 11 1 0xffffffff81c51000 17d6 ng_tee.ko 12 1 0xffffffff81c53000 40d2 ng_iface.ko 13 1 0xffffffff81c58000 5829 ng_ppp.ko 14 1 0xffffffff81c5e000 18b1 ng_tcpmss.ko - /etc/rc.conf mpd_enable="YES" quagga_daemons="zebra ospfd" devd_enable="NO" ipv6_network_interfaces="none" ip6addrctl_enable="NO" - /etc/sysctl.conf net.inet.ip.fastforwarding=1 hw.intr_storm_threshold=40000 net.graph.maxdgram=524288 net.graph.recvspace=524288 - /boot/loader.conf net.graph.maxdata=20480 net.graph.maxalloc=20480 - grep kernel: /var/log/messages Jul 12 04:18:05 mybox syslogd: kernel boot file is /boot/kernel/kernel Jul 12 04:18:05 mybox kernel: Jul 12 04:18:05 mybox kernel: Jul 12 04:18:05 mybox kernel: Fatal trap 9: general protection fault while in kernel mode Jul 12 04:18:05 mybox kernel: cpuid = 0; apic id = 00 Jul 12 04:18:05 mybox kernel: instruction pointer = 0x20:0xffffffff81c3828d Jul 12 04:18:05 mybox kernel: stack pointer = 0x28:0xfffffe0174da8380 Jul 12 04:18:05 mybox kernel: frame pointer = 0x28:0xfffffe0174da83c0 Jul 12 04:18:05 mybox kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Jul 12 04:18:05 mybox kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Jul 12 04:18:05 mybox kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Jul 12 04:18:05 mybox kernel: current process = 659 (ng_queue3) Jul 12 04:18:05 mybox kernel: trap number = 9 Jul 12 04:18:05 mybox kernel: panic: general protection fault Jul 12 04:18:05 mybox kernel: cpuid = 0 Jul 12 04:18:05 mybox kernel: KDB: stack backtrace: Jul 12 04:18:05 mybox kernel: #0 0xffffffff8098e390 at kdb_backtrace+0x60 Jul 12 04:18:05 mybox kernel: #1 0xffffffff80951066 at vpanic+0x126 Jul 12 04:18:05 mybox kernel: #2 0xffffffff80950f33 at panic+0x43 Jul 12 04:18:05 mybox kernel: #3 0xffffffff80d55f7b at trap_fatal+0x36b Jul 12 04:18:05 mybox kernel: #4 0xffffffff80d55bfd at trap+0x77d Jul 12 04:18:05 mybox kernel: #5 0xffffffff80d3b8d2 at calltrap+0x8 Jul 12 04:18:05 mybox kernel: #6 0xffffffff81c49606 at ng_l2tp_rcvdata_lower+0x946 Jul 12 04:18:05 mybox kernel: #7 0xffffffff81c370ca at ng_apply_item+0x21a Jul 12 04:18:05 mybox kernel: #8 0xffffffff81c36d1a at ng_snd_item+0x38a Jul 12 04:18:05 mybox kernel: #9 0xffffffff81c4d3e2 at ng_ksocket_incoming2+0x2f2 Jul 12 04:18:05 mybox kernel: #10 0xffffffff81c36f62 at ng_apply_item+0xb2 Jul 12 04:18:05 mybox kernel: #11 0xffffffff81c38d39 at ngthread+0x1b9 Jul 12 04:18:05 mybox kernel: #12 0xffffffff8091a4ea at fork_exit+0x9a Jul 12 04:18:05 mybox kernel: #13 0xffffffff80d3be0e at fork_trampoline+0xe Jul 12 04:18:05 mybox kernel: Uptime: 1d19h53m59s Jul 12 04:18:05 mybox kernel: Dumping 560 out of 6119 MB:..3%..12%..23%..32%..43%..52%..63%..72%..83%..92% Jul 12 04:18:05 mybox kernel: Dump complete Jul 12 04:18:05 mybox kernel: [...] From owner-freebsd-net@freebsd.org Wed Jul 13 13:35:30 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA9F9B978ED for ; Wed, 13 Jul 2016 13:35:30 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD7091170 for ; Wed, 13 Jul 2016 13:35:30 +0000 (UTC) (envelope-from patfbsd@davenulle.org) Received: from mr185083 (mr185083.univ-rennes1.fr [129.20.185.83]) by mx.zohomail.com with SMTPS id 1468416928157671.7441868731358; Wed, 13 Jul 2016 06:35:28 -0700 (PDT) Date: Wed, 13 Jul 2016 15:35:23 +0200 From: Patrick Lamaiziere To: freebsd-net@freebsd.org Subject: 10/stable pfsync bulk fail Message-ID: <20160713153523.1640e0e0@mr185083> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd10.3) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 13:35:30 -0000 Hello, 10/stable rev 302560 I'm building a pair of firewalls with pf and carp and the states are well synchronized between the firewalls. But at startup or using "service pfsync restart" pfsync fails the bulk update. In rare situations the bulk is successful but I don't know why. Jul 13 15:01:31 fucop1 kernel: carp: demoted by 240 to 3240 (pfsync bulk start) Jul 13 15:02:32 fucop1 kernel: carp: demoted by -240 to 3000 (pfsync bulk done) Jul 13 15:03:07 fucop1 kernel: carp: demoted by 240 to 3240 (pfsync bulk start) Jul 13 15:04:12 fucop1 kernel: carp: demoted by -240 to 3000 (pfsync bulk fail) /etc/rc.conf pfsync_enable="YES" pfsync_syncdev="ix1" # ix1 : pfsync ifconfig_ix1="inet 192.168.255.253/24 -tso -lro -vlanhwtso description PF_SYNC" and on the second FW # ix1 : pfsync ifconfig_ix1="inet 192.168.255.254/24 -tso -lro -vlanhwtso description PF_SYNC" and ix1 is skip in pf.conf I've tried user the syncpeer option too. Does it work for you? Or any idea Thanks, regards From owner-freebsd-net@freebsd.org Wed Jul 13 17:46:28 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B314B989FA for ; Wed, 13 Jul 2016 17:46:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3A2571C31 for ; Wed, 13 Jul 2016 17:46:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DHkRd7091633 for ; Wed, 13 Jul 2016 17:46:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211031] [panic] in ng_uncallout when argument is NULL Date: Wed, 13 Jul 2016 17:46:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-BETA1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 17:46:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211031 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 17:52:24 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FE5DB98D8F for ; Wed, 13 Jul 2016 17:52:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F6321465 for ; Wed, 13 Jul 2016 17:52:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DHqNsb006246 for ; Wed, 13 Jul 2016 17:52:24 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211003] net.inet.tcp.delayed_ack=0 delaying local tcp requests Date: Wed, 13 Jul 2016 17:52:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.2-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 17:52:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211003 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Keywords| |regression Summary|net.inet.tcp.delayed_ack=3D0 |net.inet.tcp.delayed_ack= =3D0 |and local TCP traffic |delaying local tcp requests --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 17:57:50 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27124B970B0 for ; Wed, 13 Jul 2016 17:57:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13CFB19AD for ; Wed, 13 Jul 2016 17:57:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DHvnB7013273 for ; Wed, 13 Jul 2016 17:57:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210943] Page fault in ip6_setpktopts when syncthing is started with pflog loaded Date: Wed, 13 Jul 2016 17:57:50 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 17:57:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210943 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 17:57:58 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C31D5B97120 for ; Wed, 13 Jul 2016 17:57:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1CFF1A71 for ; Wed, 13 Jul 2016 17:57:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DHvwow013464 for ; Wed, 13 Jul 2016 17:57:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210943] Page fault in ip6_setpktopts when syncthing is started with pflog loaded Date: Wed, 13 Jul 2016 17:57:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 17:57:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210943 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 19:42:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 525A9B989DE for ; Wed, 13 Jul 2016 19:42:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 421371D55 for ; Wed, 13 Jul 2016 19:42:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DJg8Dn071942 for ; Wed, 13 Jul 2016 19:42:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210943] Page fault in ip6_setpktopts when syncthing is started with pflog loaded Date: Wed, 13 Jul 2016 19:42:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 19:42:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210943 --- Comment #5 from commit-hook@freebsd.org --- A commit references this bug: Author: dim Date: Wed Jul 13 19:41:19 UTC 2016 New revision: 302784 URL: https://svnweb.freebsd.org/changeset/base/302784 Log: Fix a page fault in ip6_setpktopt(), occurring when the pflog module is loaded, and syncthing is started, which uses setsockopt(IPV6_PKGINFO). This is because pflog interfaces do not normally have an IPv6 address, causing the ND_IFINFO() macro to dereference a NULL pointer. Reviewed by: ae PR: 210943 MFC after: 3 days Changes: head/sys/netinet6/ip6_output.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 19:45:00 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79378B98B58 for ; Wed, 13 Jul 2016 19:45:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 620081E7F for ; Wed, 13 Jul 2016 19:45:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DJj0T2077475 for ; Wed, 13 Jul 2016 19:45:00 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210943] Page fault in ip6_setpktopts when syncthing is started with pflog loaded Date: Wed, 13 Jul 2016 19:45:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: dim@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 19:45:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210943 Dimitry Andric changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|New |Closed --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Jul 13 20:32:04 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A579B986E6 for ; Wed, 13 Jul 2016 20:32:04 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6794B18CB for ; Wed, 13 Jul 2016 20:32:04 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-it0-x234.google.com with SMTP id u186so31099513ita.0 for ; Wed, 13 Jul 2016 13:32:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:from:date:message-id:subject:to:cc; bh=/sKzQy+BldOPKqgVzPmN7UUSa0ZxpmVt09ljmA8jsbc=; b=qiHZhcPeZorvXDhbuZYGcv7Hy3aP1Teb2j7mgGRuUlzG1UrfrKeuT32bCmt4tnv1Z6 us455THN13dXEkO3xeD4chfwur94gX55Io6pw/bKY7sj1sKWVKHgOOIo0Nt4RrnxvzlG AL6PvarP3vy2oK5MngAfYkQlX2hUj6DxfgBoqAu0USSRVInCAJgo8RYrkm7qs8wsZFz3 VA0jIXDIDTI2SdkeC67d0YSEfqtt2PQBAwHgv24Euy7SP3O9JpedwN8X7KggLTn2v/cu 3d/UEBaB5LBh7v+QxBUyzxL2gepI58jgl9Wfo+Kshlpvk1JQw132cKYRmFNrctke2QJ+ kYJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to:cc; bh=/sKzQy+BldOPKqgVzPmN7UUSa0ZxpmVt09ljmA8jsbc=; b=YqwVkgwTm27hV2rxGxzi8AxsHx65Hc44ky+8MX1KODHKqMQhuZC66zNTfQkuO9/vnS ZwmzRIqtAVpoceQMAltIWSom2fAt+cU/U14loAkRjBv4N4vxCgMEL4BUJs+CPBHm682u ViQulL8/Ye0rpU/innUlZguafm1ItIcOCFeMM4JNXl7kXMxtu4b1/hSg5f6tEbUOJdL8 DrhI5QtFFlkk6FvfLs08QbxY9zBQiRXVXBbkERmb2mbX4pRNIXIZj6M+JzH/HBaT6cpm 2czusx+8V8RauU2cP2CSRkMquc+Gc3lOdMTrOB2iulE59z+RLRIUP9lvg7nwPWHjNdcD pAFQ== X-Gm-Message-State: ALyK8tLAHIq8up9s9YjH5p6zRmS3trIq1aRFnOE0vJ8YWYfG5eAyhDU2Lgfs3kPH6Dw+T2DvXY3sjprLBf9DSbGW X-Received: by 10.36.122.129 with SMTP id a123mr23172099itc.44.1468441923827; Wed, 13 Jul 2016 13:32:03 -0700 (PDT) MIME-Version: 1.0 Sender: sobomax@sippysoft.com Received: by 10.36.59.193 with HTTP; Wed, 13 Jul 2016 13:32:03 -0700 (PDT) From: Maxim Sobolev Date: Wed, 13 Jul 2016 13:32:03 -0700 X-Google-Sender-Auth: gkshRdIOj6qWI8WubTmfOAaD0ag Message-ID: Subject: 11.0-BETA1 'ifconfig igb0 media auto' causes panic in llentry_free() To: stable@freebsd.org, FreeBSD Release Engineering Team Cc: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 20:32:04 -0000 Hi, we are seeing consistent crash doing 'ifconfig igb0 media auto' after interface has been provisioned by the dhcpclient. This is stable/11 sources from svn revision 302593. That problem did not happen to us before the upgrade from from 11.0-ALPHA3, svn revision 301898 from head. Sreenshot of the backtrace is here: http://sobomax.sippysoft.com/DSC00012.JPG -Maxim From owner-freebsd-net@freebsd.org Wed Jul 13 20:46:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F3BDB98CA4; Wed, 13 Jul 2016 20:46:36 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CF5C13DF; Wed, 13 Jul 2016 20:46:36 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=pJu3ND5N+6vGJYl/iH06zOAK/7S9xYL1PbJWB533mZQ=; b=FkKAiEOJxx45CqXf1FsKQM9VNQ A3iBuAS4CGkB10tSDEEkd6dJDB/TZuJiiMslpyZ5ftRShdzhFgu+Kt4hMdGMHnYSWSsfjtpj5P4VM CzxcvW+2DJuHyqILig/bwdpCA8ayQhYJ+IRY9V40/ZKkoBYJkEoCH+SXaV2ifGwBMcIU=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:19112 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bNR3D-000NiH-Na; Wed, 13 Jul 2016 15:46:35 -0500 Received: from proxy.na.alcatel-lucent.com ([135.245.48.82]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 13 Jul 2016 15:46:35 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 13 Jul 2016 15:46:35 -0500 From: Larry Rosenman To: Maxim Sobolev Cc: stable@freebsd.org, FreeBSD Release Engineering Team , FreeBSD Net , owner-freebsd-stable@freebsd.org Subject: Re: 11.0-BETA1 'ifconfig igb0 media auto' causes panic in llentry_free() In-Reply-To: References: Message-ID: <94f16de461db3e04d5f2f882cce34aec@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 20:46:36 -0000 On 2016-07-13 15:32, Maxim Sobolev wrote: > Hi, we are seeing consistent crash doing 'ifconfig igb0 media auto' > after > interface has been provisioned by the dhcpclient. This is stable/11 > sources > from svn revision 302593. > > That problem did not happen to us before the upgrade from from > 11.0-ALPHA3, > svn revision 301898 from head. > > Sreenshot of the backtrace is here: > http://sobomax.sippysoft.com/DSC00012.JPG > > -Maxim see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210884 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-net@freebsd.org Wed Jul 13 20:57:48 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25D5AB98F9D for ; Wed, 13 Jul 2016 20:57:48 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1CD01C43 for ; Wed, 13 Jul 2016 20:57:47 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-it0-x22a.google.com with SMTP id f6so31176830ith.1 for ; Wed, 13 Jul 2016 13:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=DxXdf3AO7WUS9Zx7GL6BRIaL2ThFthVX8N26+Z4TnJI=; b=dTM8hthKKej5q4gRkhnKXoAhMUFji5l0oUvi0rdiWZtcgNITymJ+8WiVJnM3/qN6oa wyYdwaHbE3LspBJ50JOIroTNXvP0qOuMrWiUB+Nl/G8UG8RiYGVKl8kNatqaur9JULnu ZvxZaJhL1r9JGE5EiOYACEUJW5mt9tAHQJa8j0GeWfyYMYEow2jC8qDTiFz8e+6OF8ye hfpqsYMwDHaSUm5ZVV+7AxSGGz1FyMRaQ3FN0avbax5gbSOQOesu9rDv+8CyxNp7TEQC qep0t7/WfuNSYkvAHFOWCfJPeYbTk6EwuEBJPkf8z1ZoEfgdlBhATdIec4ALqB30ha+A Zu8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=DxXdf3AO7WUS9Zx7GL6BRIaL2ThFthVX8N26+Z4TnJI=; b=lusB1GSpem8MaH+Ws2dcchakm8bK2IHyymfTACkgEAm4U8/ZiiOYKuiP1PFnC6eTPY hcW0BzF1/aXbtcM15HcRlclIvBpv0qmbVohECi2J99A3qa2FqHrG/IoYBqZdRjY3OqI6 BCe0nFic/VzwSsGTKCLx8cbdAB/l6zzYPTCwjoMrZf+CxGI2159D41HG8nquxLW9hI/i AwBHekQG2b8s/xLk1mNjlEv/sJE1pt8sioL38Kh0pub4urifkzkDt41WARYhtHls87oz CENbnFZ+Cs0qxbUm5Is3YHzreFLNmOWTuuUjhhgXDoR45dZKu6uDMC5nDH9NC/KTJ/Lm S+rw== X-Gm-Message-State: ALyK8tLcuvLaC6xLlMS0BgtfUdygWEG0g9hmaM4Q885lwdTruVlv+w7WRyMgs84Bx1Iu41cHZolEx48OpZ3nrK8c X-Received: by 10.36.90.79 with SMTP id v76mr10668336ita.16.1468443467374; Wed, 13 Jul 2016 13:57:47 -0700 (PDT) MIME-Version: 1.0 Sender: sobomax@sippysoft.com Received: by 10.36.59.193 with HTTP; Wed, 13 Jul 2016 13:57:46 -0700 (PDT) In-Reply-To: <94f16de461db3e04d5f2f882cce34aec@thebighonker.lerctr.org> References: <94f16de461db3e04d5f2f882cce34aec@thebighonker.lerctr.org> From: Maxim Sobolev Date: Wed, 13 Jul 2016 13:57:46 -0700 X-Google-Sender-Auth: sqeBDKxoRDW6eHIX6JwXhZGnjZM Message-ID: Subject: Re: 11.0-BETA1 'ifconfig igb0 media auto' causes panic in llentry_free() To: Larry Rosenman Cc: stable@freebsd.org, FreeBSD Release Engineering Team , FreeBSD Net , owner-freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 20:57:48 -0000 Thanks, looks like the same issue. I'll try the patch from ticket. -Max On Wed, Jul 13, 2016 at 1:46 PM, Larry Rosenman wrote: > On 2016-07-13 15:32, Maxim Sobolev wrote: > >> Hi, we are seeing consistent crash doing 'ifconfig igb0 media auto' after >> interface has been provisioned by the dhcpclient. This is stable/11 >> sources >> from svn revision 302593. >> >> That problem did not happen to us before the upgrade from from >> 11.0-ALPHA3, >> svn revision 301898 from head. >> >> Sreenshot of the backtrace is here: >> http://sobomax.sippysoft.com/DSC00012.JPG >> >> -Maxim >> > see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210884 > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > From owner-freebsd-net@freebsd.org Wed Jul 13 21:00:57 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D406B9716C; Wed, 13 Jul 2016 21:00:57 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22F841EE9; Wed, 13 Jul 2016 21:00:57 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=GfCdEvo8V43al0EiYXZ/CQkXSHFNBUOeDTHLzVQkNM0=; b=bzqGDxvshjw8ONH+p9sf/k9E0u o7ry41zlFoCeIEwoU5BHV8cvKQ/kEirMNv8eDN3tavm5wp8+VAXVuvAGZKXOkj27UhayvJ3DwX3g6 idlKNqyBRHKlneYVBx1ou3RQcxAX+9cYzmg7h+nRgIK2t2VBrojJXy+XloIVUUTSsiBU=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:59000 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bNRH6-000OEU-Hh; Wed, 13 Jul 2016 16:00:56 -0500 Received: from proxy.na.alcatel-lucent.com ([135.245.48.82]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 13 Jul 2016 16:00:56 -0500 MIME-Version: 1.0 Date: Wed, 13 Jul 2016 16:00:56 -0500 From: Larry Rosenman To: Maxim Sobolev Cc: stable@freebsd.org, FreeBSD Release Engineering Team , FreeBSD Net , owner-freebsd-stable@freebsd.org, sobomax@sippysoft.com Subject: Re: 11.0-BETA1 'ifconfig igb0 media auto' causes panic in llentry_free() In-Reply-To: References: <94f16de461db3e04d5f2f882cce34aec@thebighonker.lerctr.org> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.2.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 21:00:57 -0000 NOTE: I get an insta-panic on boot :( I'm waiting for Gleb to respond. On 2016-07-13 15:57, Maxim Sobolev wrote: > Thanks, looks like the same issue. I'll try the patch from ticket. > > -Max > > On Wed, Jul 13, 2016 at 1:46 PM, Larry Rosenman wrote: > > On 2016-07-13 15:32, Maxim Sobolev wrote: > Hi, we are seeing consistent crash doing 'ifconfig igb0 media auto' after > interface has been provisioned by the dhcpclient. This is stable/11 sources > from svn revision 302593. > > That problem did not happen to us before the upgrade from from 11.0-ALPHA3, > svn revision 301898 from head. > > Sreenshot of the backtrace is here: > http://sobomax.sippysoft.com/DSC00012.JPG > > -Maxim see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210884 > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 [1] E-Mail: ler@lerctr.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] tel:%2B1%20214-642-9640 From owner-freebsd-net@freebsd.org Wed Jul 13 23:18:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F8A1B98917 for ; Wed, 13 Jul 2016 23:18:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 616F614EE for ; Wed, 13 Jul 2016 23:18:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6DNI5Xt015607 for ; Wed, 13 Jul 2016 23:18:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211031] [panic] in ng_uncallout when argument is NULL Date: Wed, 13 Jul 2016 23:18:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-BETA1 X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mizhka@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 23:18:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211031 --- Comment #1 from Michael Zhilin --- Patch: https://reviews.freebsd.org/D7209 At least it works for me. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 14 05:04:27 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62E93B97052 for ; Thu, 14 Jul 2016 05:04:27 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2920E1163 for ; Thu, 14 Jul 2016 05:04:27 +0000 (UTC) (envelope-from sobomax@sippysoft.com) Received: by mail-it0-x231.google.com with SMTP id f6so38404379ith.1 for ; Wed, 13 Jul 2016 22:04:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sippysoft-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=/sRVhxILLUy/G7kLiaZzamiOBP0e7fCAmW/7f6hg1NM=; b=DF4F3BguTVBKdPqz9VBi7jGRhX3x092H6b6US7A9a6LuJkOtrNEAvB/ELl8RXHwMPl b2RaRABFxQRT3K+APBv1jALtjLX7DAHuH3EHxxRvc24l6lbRhGy5VkgWS+VKKUMEo1gs MRutY4Oa7lDr0a+Ea+FiTE/Z8I7DFPeqIoCQMDTOu7unWYklkff+4kLfJ0jeplgrcv8W j3kwFy5AgwrseJNKXizAXYLxL5CPx+E0K+nr0sc8HNiWIIe1Tt4b8kzdrAjBwsIMIk1K m1EIGzrUPLUEoDPSstZMVZPz1RA73XdhmoOQ3zLuxqWpgJdl0jY2iH3Er2kumY6ADp3l iDKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=/sRVhxILLUy/G7kLiaZzamiOBP0e7fCAmW/7f6hg1NM=; b=UvpeMUV98G3v3Z9RnBkRNVTpaJ9k+OVlLhsuHaqOYQYvcLTuXR4D5uqIj7D/f9cwSj Q86A73KrUO9dDBozUOs2f7SPQaeGDjBetXLsZ4w80ZvuYUMaYIh7/38YSdAA1NutXpyo hZyMQzfYD34EEfPvITbuLVf9/gyV2a8MH9q+O/63A168wdtDjNdMNDkerky6sYXipjhs Eu0fHOQFlc4XDlRfFWAev8XLeElP+mX4mNZp4kKkBOKAFxUJ1skFffPBogPxuytadS4E n0bOmZupqtrG1p6j6fQda2N/xnfGWFS7+WjWXYAPnlCU9ywuPYhRjhSAJQH+u+ysavMi xtWA== X-Gm-Message-State: ALyK8tITpqiKddMMK0mlsrwoav8BL455oOENEByr9T1yAiJ8t52HUThv3IUUmfdqu5IM0c4MQbfd/kq9unL+jj/q X-Received: by 10.36.188.65 with SMTP id n62mr26735385ite.61.1468472666497; Wed, 13 Jul 2016 22:04:26 -0700 (PDT) MIME-Version: 1.0 Sender: sobomax@sippysoft.com Received: by 10.36.59.193 with HTTP; Wed, 13 Jul 2016 22:04:26 -0700 (PDT) In-Reply-To: References: <94f16de461db3e04d5f2f882cce34aec@thebighonker.lerctr.org> From: Maxim Sobolev Date: Wed, 13 Jul 2016 22:04:26 -0700 X-Google-Sender-Auth: wBdr6ttK4IHn-WlpMJE1NFe1upw Message-ID: Subject: Re: 11.0-BETA1 'ifconfig igb0 media auto' causes panic in llentry_free() To: Larry Rosenman Cc: stable@freebsd.org, FreeBSD Release Engineering Team , FreeBSD Net , owner-freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 05:04:27 -0000 Larry, thanks for the pointer. Patch in the PR fixed the issue for me. Added comment in there. -Max On Wed, Jul 13, 2016 at 2:00 PM, Larry Rosenman wrote: > NOTE: I get an insta-panic on boot :( > > I'm waiting for Gleb to respond. > > > > On 2016-07-13 15:57, Maxim Sobolev wrote: > > Thanks, looks like the same issue. I'll try the patch from ticket. > > -Max > > > > On Wed, Jul 13, 2016 at 1:46 PM, Larry Rosenman wrote: > >> On 2016-07-13 15:32, Maxim Sobolev wrote: >> >>> Hi, we are seeing consistent crash doing 'ifconfig igb0 media auto' after >>> interface has been provisioned by the dhcpclient. This is stable/11 >>> sources >>> from svn revision 302593. >>> >>> That problem did not happen to us before the upgrade from from >>> 11.0-ALPHA3, >>> svn revision 301898 from head. >>> >>> Sreenshot of the backtrace is here: >>> http://sobomax.sippysoft.com/DSC00012.JPG >>> >>> -Maxim >> >> see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210884 >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 >> >> > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > From owner-freebsd-net@freebsd.org Thu Jul 14 07:23:52 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8609EB98DB8 for ; Thu, 14 Jul 2016 07:23:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7525C1F10 for ; Thu, 14 Jul 2016 07:23:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6E7Np2h062904 for ; Thu, 14 Jul 2016 07:23:52 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211031] [panic] in ng_uncallout when argument is NULL Date: Thu, 14 Jul 2016 07:23:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: crash, needs-qa, patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: cc version keywords bug_status bug_file_loc flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 07:23:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211031 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kib@FreeBSD.org, | |re@FreeBSD.org Version|11.0-BETA1 |CURRENT Keywords| |crash, needs-qa, patch Status|New |Open URL| |https://reviews.freebsd.org | |/D7209 Flags| |mfc-stable11? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 14 07:24:03 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB104B98DDF for ; Thu, 14 Jul 2016 07:24:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA6D61F9B for ; Thu, 14 Jul 2016 07:24:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6E7O3RF063216 for ; Thu, 14 Jul 2016 07:24:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211031] [panic] in ng_uncallout when argument is NULL Date: Thu, 14 Jul 2016 07:24:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: crash, needs-qa, patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 07:24:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211031 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable10? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 14 07:25:36 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 488A3B98EFE for ; Thu, 14 Jul 2016 07:25:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 373AF137C for ; Thu, 14 Jul 2016 07:25:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6E7Pa6T065312 for ; Thu, 14 Jul 2016 07:25:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 210943] Page fault in ip6_setpktopts when syncthing is started with pflog loaded Date: Thu, 14 Jul 2016 07:25:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: dim@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: assigned_to flagtypes.name resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 07:25:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210943 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |dim@FreeBSD.org Flags| |mfc-stable10?, | |mfc-stable11? Resolution|FIXED |--- Status|Closed |In Progress --- Comment #6 from Kubilay Kocak --- Assign to committer that resolved. Re-open for MFC to stable/11, stable/10 Please set flag mfc-stable* to + if/when committed, or - if not appropriate with comment --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 14 16:00:09 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BBB6B98607 for ; Thu, 14 Jul 2016 16:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A75015E1 for ; Thu, 14 Jul 2016 16:00:09 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6EG09E4009930 for ; Thu, 14 Jul 2016 16:00:09 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 199096] Kernel panic after some time using mpd (netgraph) and ipfw Date: Thu, 14 Jul 2016 16:00:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: donaldbaud@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 16:00:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D199096 Donald Baud changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |donaldbaud@yahoo.com --- Comment #2 from Donald Baud --- We started using mpd5.8 (netgraph l2tp) with FreeBSD RELENG-10.3=20 This is a replacement to a cisco7206 npeg (LNS)=20 Terminates about 300 l2tp tunnels, 800sessions and 800Mbit/s I noticed several unresolved bug reports relating to crash with netgraph: Bug 199096 Kernel panic after some time using mpd (netgraph) and ipfw=20 Bug 176401 [netgraph] page fault in netgraph=20 Bug 154286 [netgraph] [panic] 8.2-PRERELEASE panic in netgraph=20 Bug 154091 - [netgraph] [panic] netgraph, unaligned mbuf?=20 Bug 153497 - [netgraph] netgraph panic due to race conditions=20 I don't want to start an new PR but I'm noticing many panic's, once a day mostly: I've experimented with net.graph values but did't do any difference: # grep net.graph /etc/sysctl.conf ; grep net.graph /boot/loader.conf #net.graph.maxdgram=3D524288 #default 20480 #net.graph.recvspace=3D524288 #default 20480 #net.graph.maxdata=3D65536 #default 4096 #net.graph.maxalloc=3D65536 #default 4096 # uname -a FreeBSD mybox.example.com 10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat M= ay 28 12:23:44 UTC 2016=20=20=20=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 # sysctl -a | grep net.graph net.graph.mppe.max_rekey: 1000 net.graph.mppe.log_max_rekey: 1 net.graph.mppe.block_on_max_rekey: 0 net.graph.control.proto: 2 net.graph.data.proto: 1 net.graph.family: 32 net.graph.recvspace: 20480 net.graph.maxdgram: 20480 net.graph.msg_version: 8 net.graph.abi_version: 12 net.graph.maxdata: 4096 net.graph.maxalloc: 4096 net.graph.threads: 4 # kldstat Id Refs Address Size Name 1 34 0xffffffff80200000 17bc6a8 kernel 2 2 0xffffffff81c11000 114db ipfw.ko 3 1 0xffffffff81c23000 d32f dummynet.ko 4 1 0xffffffff81c31000 3831 ng_socket.ko 5 9 0xffffffff81c35000 ba02 netgraph.ko 6 1 0xffffffff81c41000 2b99 ng_mppc.ko 7 1 0xffffffff81c44000 80c rc4.ko 8 1 0xffffffff81c45000 23dc vmmemctl.ko 9 1 0xffffffff81c48000 397d ng_l2tp.ko 10 1 0xffffffff81c4c000 4b04 ng_ksocket.ko 11 1 0xffffffff81c51000 17d6 ng_tee.ko 12 1 0xffffffff81c53000 40d2 ng_iface.ko 13 1 0xffffffff81c58000 5829 ng_ppp.ko 14 1 0xffffffff81c5e000 18b1 ng_tcpmss.ko 15 1 0xffffffff81c60000 2df7 ng_vjc.ko # vmstat -z | head -1 ; vmstat -z | grep NetGraph ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP NetGraph items: 72, 4123, 0, 1271,96158617, 0, 0 NetGraph data items: 72, 4123, 4, 1639,244942156, 0, 0 # /etc/rc.conf ipv6_network_interfaces=3D"none" # Default is auto=20 ip6addrctl_enable=3D"NO" # New way to disable IPv6 support=20 devd_enable=3D"NO"=20 mpd_enable=3D"YES" quagga_enable=3D"YES" quagga_daemons=3D"zebra ospfd" =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Crash log: Jul 14 03:20:16 mybox kernel: Fatal trap 12: page fault while in kernel mode Jul 14 03:20:16 mybox kernel: cpuid =3D 3; apic id =3D 03 Jul 14 03:20:16 mybox kernel: fault virtual address =3D 0x60 Jul 14 03:20:16 mybox kernel: fault code =3D supervisor read= data, page not present Jul 14 03:20:16 mybox kernel: instruction pointer =3D 0x20:0xffffffff80a27d7a Jul 14 03:20:16 mybox kernel: stack pointer =3D 0x28:0xfffffe0174dcb600 Jul 14 03:20:16 mybox kernel: frame pointer =3D 0x28:0xfffffe0174dcb620 Jul 14 03:20:16 mybox kernel: code segment =3D base 0x0, limit 0xfffff, type 0x1b Jul 14 03:20:16 mybox kernel: =3D DPL 0, pres 1, long 1, def32 0, gran 1 Jul 14 03:20:16 mybox kernel: processor eflags =3D interrupt enabled, resu= me, IOPL =3D 0 Jul 14 03:20:16 mybox kernel: current process =3D 656 (mpd5) Jul 14 03:20:16 mybox kernel: trap number =3D 12 Jul 14 03:20:16 mybox kernel: panic: page fault Jul 14 03:20:16 mybox kernel: cpuid =3D 3 Jul 14 03:20:16 mybox kernel: KDB: stack backtrace: Jul 14 03:20:16 mybox kernel: #0 0xffffffff8098e390 at kdb_backtrace+0x60 Jul 14 03:20:16 mybox kernel: #1 0xffffffff80951066 at vpanic+0x126 Jul 14 03:20:16 mybox kernel: #2 0xffffffff80950f33 at panic+0x43 Jul 14 03:20:16 mybox kernel: #3 0xffffffff80d55f7b at trap_fatal+0x36b Jul 14 03:20:16 mybox kernel: #4 0xffffffff80d5627d at trap_pfault+0x2ed Jul 14 03:20:16 mybox kernel: #5 0xffffffff80d558fa at trap+0x47a Jul 14 03:20:16 mybox kernel: #6 0xffffffff80d3b8d2 at calltrap+0x8 Jul 14 03:20:16 mybox kernel: #7 0xffffffff80a8161d at in_ifadownkill+0x8d Jul 14 03:20:16 mybox kernel: #8 0xffffffff80a26e70 at rn_walktree+0x80 Jul 14 03:20:16 mybox kernel: #9 0xffffffff80a8151f at in_ifadown+0x9f Jul 14 03:20:16 mybox kernel: #10 0xffffffff80a76a2c at in_control+0x76c Jul 14 03:20:16 mybox kernel: #11 0xffffffff80a17d7b at ifioctl+0x15eb Jul 14 03:20:16 mybox kernel: #12 0xffffffff809a8c35 at kern_ioctl+0x255 Jul 14 03:20:16 mybox kernel: #13 0xffffffff809a8930 at sys_ioctl+0x140 Jul 14 03:20:16 mybox kernel: #14 0xffffffff80d5694f at amd64_syscall+0x40f Jul 14 03:20:16 mybox kernel: #15 0xffffffff80d3bbbb at Xfast_syscall+0xfb Jul 14 03:20:16 mybox kernel: Uptime: 1d8h24m23s Jul 14 03:20:16 mybox kernel: Dumping 557 out of 6119 MB:..3%..12%..21%..32%..41%..52%..61%..72%..81%..92% Jul 14 03:20:16 mybox kernel: Dump complete =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D other crash logs: /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Fatal trap 9: general protection fault while in kernel mode /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: cpuid =3D 0; apic id = =3D 00 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: instruction pointer= =20=20=20=20=20=20 =3D 0x20:0xffffffff81c3828d /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: stack pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174da8380 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: frame pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174da83c0 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: code segment=20=20=20= =20=20=20=20=20=20=20=20=20=20 =3D base 0x0, limit 0xfffff, type 0x1b /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: =3D DPL 0, pres 1, lo= ng 1, def32 0, gran 1 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: current process=20=20= =20=20=20=20=20=20=20=20 =3D 659 (ng_queue3) /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: trap number=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 =3D 9 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: panic: general protec= tion fault /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: cpuid =3D 0 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: KDB: stack backtrace: /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #0 0xffffffff8098e390= at kdb_backtrace+0x60 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #1 0xffffffff80951066= at vpanic+0x126 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #2 0xffffffff80950f33= at panic+0x43 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #3 0xffffffff80d55f7b= at trap_fatal+0x36b /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #4 0xffffffff80d55bfd= at trap+0x77d /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #5 0xffffffff80d3b8d2= at calltrap+0x8 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #6 0xffffffff81c49606= at ng_l2tp_rcvdata_lower+0x946 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #7 0xffffffff81c370ca= at ng_apply_item+0x21a /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #8 0xffffffff81c36d1a= at ng_snd_item+0x38a /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #9 0xffffffff81c4d3e2= at ng_ksocket_incoming2+0x2f2 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #10 0xffffffff81c36f6= 2 at ng_apply_item+0xb2 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #11 0xffffffff81c38d3= 9 at ngthread+0x1b9 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #12 0xffffffff8091a4e= a at fork_exit+0x9a /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: #13 0xffffffff80d3be0= e at fork_trampoline+0xe /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Uptime: 1d19h53m59s /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Dumping 560 out of 61= 19 MB:..3%..12%..23%..32%..43%..52%..63%..72%..83%..92% /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Dump complete [...] /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Hypervisor: Origin =3D "VMwareVMware" /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: real memory =3D 6442= 450944 (6144 MB) /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: avail memory =3D 6183= 518208 (5897 MB) /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: Event timer "LAPIC" quality 600 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: ACPI APIC Table: /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: FreeBSD/SMP: 1 packag= e(s) x 4 core(s) [...] /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: em0: port 0x2000-0x203f mem 0xfd5c0000-0xfd5dffff,0xfdff0000-0xfdffffff irq 18 at device 0.0 on pci2 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: em0: Ethernet address: 00:50:56:8d:d7:57 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: pcib3: at device 21.0 on pci0 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: pci3: = on pcib3 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: vmx0: port 0x4000-0x400f mem 0xfd4fc000-0xfd4fcfff,0xfd4fd000-0xfd4fdfff,0xfd4fe000-0xfd4fffff irq 18 at device 0.0 on pci3 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: vmx0: Ethernet addres= s: 00:50:56:8d:4c:68 [...] /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: vmx1: port 0x5000-0x500f mem 0xfd3fc000-0xfd3fcfff,0xfd3fd000-0xfd3fdfff,0xfd3fe000-0xfd3fffff irq 19 at device 0.0 on pci11 /var/log/messages.1.bz2:Jul 12 04:18:05 mybox kernel: vmx1: Ethernet addres= s: 00:50:56:8d:85:29 [...] /var/log/messages.1.bz2:Jul 12 04:18:32 mybox kernel: VMware memory control driver initialized /var/log/messages.1.bz2:Jul 12 04:18:32 mybox kernel: done -------- /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: Fatal trap 9: general protection fault while in kernel mode /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: cpuid =3D 0; apic id = =3D 01 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: instruction pointer= =20=20=20=20=20=20 =3D 0x20:0xffffffff81c31f40 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: stack pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe017359a510 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: frame pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe017359a560 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: code segment=20=20=20= =20=20=20=20=20=20=20=20=20=20 =3D base 0x0, limit 0xfffff, type 0x1b /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: =3D DPL 0, pres 1, lo= ng 1, def32 0, gran 1 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: current process=20=20= =20=20=20=20=20=20=20=20 =3D 656 (mpd5) /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: trap number=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 =3D 9 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: panic: general protec= tion fault /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: cpuid =3D 0 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: KDB: stack backtrace: /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #0 0xffffffff8098e390= at kdb_backtrace+0x60 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #1 0xffffffff80951066= at vpanic+0x126 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #2 0xffffffff80950f33= at panic+0x43 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #3 0xffffffff80d55f7b= at trap_fatal+0x36b /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #4 0xffffffff80d55bfd= at trap+0x77d /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #5 0xffffffff80d3b8d2= at calltrap+0x8 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #6 0xffffffff81c395b6= at ng_add_hook+0x106 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #7 0xffffffff81c390bb= at ng_mkpeer+0x3b /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #8 0xffffffff81c37275= at ng_apply_item+0x3c5 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #9 0xffffffff81c36d1a= at ng_snd_item+0x38a /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #10 0xffffffff81c319f= 1 at ngc_send+0x221 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #11 0xffffffff809cc2d= 6 at sosend_generic+0x476 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #12 0xffffffff809d263= 5 at kern_sendit+0x245 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #13 0xffffffff809d295= 9 at sendit+0x129 /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #14 0xffffffff809d281= d at sys_sendto+0x4d /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #15 0xffffffff80d5694= f at amd64_syscall+0x40f /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: #16 0xffffffff80d3bbb= b at Xfast_syscall+0xfb /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: Uptime: 14h36m50s /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: Dumping 327 out of 61= 19 MB:..5%..15%..25%..35%..44%..54%..64%..74%..84%..93% /var/log/messages.1.bz2:Jul 12 18:55:22 mybox kernel: Dump complete ------ /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: Fatal trap 9: general protection fault while in kernel mode /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: cpuid =3D 0; apic id = =3D 00 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: instruction pointer= =20=20=20=20=20=20 =3D 0x20:0xffffffff80b8f26b /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: stack pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d12730 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: frame pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d127d0 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: code segment=20=20=20= =20=20=20=20=20=20=20=20=20=20 =3D base 0x0, limit 0xfffff, type 0x1b /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: =3D DPL 0, pres 1, lo= ng 1, def32 0, gran 1 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: current process=20=20= =20=20=20=20=20=20=20=20 =3D 16 (syncer) /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: trap number=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 =3D 9 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: panic: general protec= tion fault /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: cpuid =3D 3 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: KDB: stack backtrace: /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #0 0xffffffff8098e390= at kdb_backtrace+0x60 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #1 0xffffffff80951066= at vpanic+0x126 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #2 0xffffffff80950f33= at panic+0x43 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #3 0xffffffff80d55f7b= at trap_fatal+0x36b /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #4 0xffffffff80d55bfd= at trap+0x77d /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #5 0xffffffff80d3b8d2= at calltrap+0x8 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #6 0xffffffff80baf5be= at ffs_geom_strategy+0x15e /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #7 0xffffffff80bc0fc0= at ufs_strategy+0x80 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #8 0xffffffff80e82a5b= at VOP_STRATEGY_APV+0xab /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #9 0xffffffff809dc23c= at bufstrategy+0x2c /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #10 0xffffffff809dc16= 5 at bufwrite+0x185 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #11 0xffffffff80bb139= c at ffs_syncvnode+0x2dc /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #12 0xffffffff80bb01e= 3 at ffs_fsync+0x23 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #13 0xffffffff80e81af= 7 at VOP_FSYNC_APV+0xa7 /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #14 0xffffffff809fbe1= b at sched_sync+0x3ab /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #15 0xffffffff8091a4e= a at fork_exit+0x9a /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: #16 0xffffffff80d3be0= e at fork_trampoline+0xe /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: Uptime: 1d12h20m38s /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: Dumping 563 out of 61= 19 MB:..3%..12%..23%..32%..43%..52%..63%..71%..83%..91% /var/log/messages.3.bz2:Jul 10 08:23:37 mybox kernel: Dump complete -------- /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: Fatal trap 12: page f= ault while in kernel mode /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: cpuid =3D 1; apic id = =3D 01 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: fault virtual address= =20=20=20=20 =3D 0x28 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: fault code=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 =3D supervisor read data, page not present /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: instruction pointer= =20=20=20=20=20=20 =3D 0x20:0xffffffff81c38283 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: stack pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d85540 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: frame pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d85580 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: code segment=20=20=20= =20=20=20=20=20=20=20=20=20=20 =3D base 0x0, limit 0xfffff, type 0x1b /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: =3D DPL 0, pres 1, lo= ng 1, def32 0, gran 1 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: current process=20=20= =20=20=20=20=20=20=20=20 =3D 628 (ng_queue2) /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: trap number=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 =3D 12 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: panic: page fault /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: cpuid =3D 1 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: KDB: stack backtrace: /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #0 0xffffffff8098e390= at kdb_backtrace+0x60 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #1 0xffffffff80951066= at vpanic+0x126 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #2 0xffffffff80950f33= at panic+0x43 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #3 0xffffffff80d55f7b= at trap_fatal+0x36b /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #4 0xffffffff80d5627d= at trap_pfault+0x2ed /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #5 0xffffffff80d558fa= at trap+0x47a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #6 0xffffffff80d3b8d2= at calltrap+0x8 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #7 0xffffffff81c5e509= at ng_tcpmss_rcvdata+0x2d9 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #8 0xffffffff81c370ca= at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #9 0xffffffff81c36d1a= at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #10 0xffffffff81c5a1c= 8 at ng_ppp_comp_recv+0x148 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #11 0xffffffff81c370c= a at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #12 0xffffffff81c36d1= a at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #13 0xffffffff81c370c= a at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #14 0xffffffff81c36d1= a at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #15 0xffffffff81c370c= a at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #16 0xffffffff81c36d1= a at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: #17 0xffffffff81c4d3e= 2 at ng_ksocket_incoming2+0x2f2 /var/log/messages.5.bz2:Jul 8 08:09:04 mybox kernel: Uptime: 5d17h47m38s ------- /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: Fatal trap 12: page f= ault while in kernel mode /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: cpuid =3D 2; apic id = =3D 02 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: fault virtual address= =20=20=20=20 =3D 0x28 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: fault code=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 =3D supervisor read data, page not present /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: instruction pointer= =20=20=20=20=20=20 =3D 0x20:0xffffffff81c38283 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: stack pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d99660 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: frame pointer=20=20= =20=20=20=20=20=20=20=20=20=20 =3D 0x28:0xfffffe0174d996a0 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: code segment=20=20=20= =20=20=20=20=20=20=20=20=20=20 =3D base 0x0, limit 0xfffff, type 0x1b /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: =3D DPL 0, pres 1, lo= ng 1, def32 0, gran 1 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: processor eflags =3D interrupt enabled, resume, IOPL =3D 0 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: current process=20=20= =20=20=20=20=20=20=20=20 =3D 627 (ng_queue3) /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: trap number=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 =3D 12 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: panic: page fault /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: cpuid =3D 2 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: KDB: stack backtrace: /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #0 0xffffffff8098e390= at kdb_backtrace+0x60 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #1 0xffffffff80951066= at vpanic+0x126 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #2 0xffffffff80950f33= at panic+0x43 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #3 0xffffffff80d55f7b= at trap_fatal+0x36b /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #4 0xffffffff80d5627d= at trap_pfault+0x2ed /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #5 0xffffffff80d558fa= at trap+0x47a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #6 0xffffffff80d3b8d2= at calltrap+0x8 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #7 0xffffffff81c5a042= at ng_ppp_bypass+0x62 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #8 0xffffffff81c370ca= at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #9 0xffffffff81c36d1a= at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #10 0xffffffff81c370c= a at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #11 0xffffffff81c36d1= a at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #12 0xffffffff81c370c= a at ng_apply_item+0x21a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #13 0xffffffff81c36d1= a at ng_snd_item+0x38a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #14 0xffffffff81c4d3e= 2 at ng_ksocket_incoming2+0x2f2 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #15 0xffffffff81c36f6= 2 at ng_apply_item+0xb2 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #16 0xffffffff81c38d3= 9 at ngthread+0x1b9 /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: #17 0xffffffff8091a4e= a at fork_exit+0x9a /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: Uptime: 11h52m56s /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: Dumping 591 out of 61= 19 MB:..3%..11%..22%..33%..41%..52%..63%..71%..82%..92% /var/log/messages.5.bz2:Jul 8 20:02:30 mybox kernel: Dump complete --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Jul 14 17:45:05 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E62F0B99988 for ; Thu, 14 Jul 2016 17:45:05 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CBA91D5A; Thu, 14 Jul 2016 17:45:05 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f43.google.com with SMTP id o80so122593890wme.1; Thu, 14 Jul 2016 10:45:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=4DSWyi2MamA+UrkDmskBhbY5VpHbzKDKJ7X/FafWky8=; b=WX4VQP01xxgpSFdxVpG/r3xIIpRDExB43EHXeOzM/h9VlUAWgZKT95z3p81lg+kwny 1RQc6sKNK9aF85Jvc1a8kaNBKUfzisVM9oN5kMY4oimHwHsq4Fv0WaEnNBD09k0ufamT DD/iN/Jc7dDCK0aydXBZRCDqzBtlAAUi0ALfaMSwQGWFe/8zuR0w+7bu0oh6anrUB8ZF rv0sJNuUpiT5IV2TwozuETYpEXaMZzGZhswKFxK6qJpdmMbUVL0BLp1DlrlTI7O3oeSX gFl7N8tvraFfMQfiVnCNY+hvveDAlXlAnNXusECE0i19W5RRHhUpjnRbfLZJnRPSqe/b GngA== X-Gm-Message-State: ALyK8tL7EkAF7UHqrvFExRqFq8WIgruXTQdQeEIeqenfQZfRGZi3JysyObQJuCesWKtvUg== X-Received: by 10.28.141.4 with SMTP id p4mr36807644wmd.46.1468517935762; Thu, 14 Jul 2016 10:38:55 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id d64sm5216745wmc.22.2016.07.14.10.38.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Jul 2016 10:38:54 -0700 (PDT) Subject: Re: TCP stack lock contention with short-lived connections To: freebsd-net@freebsd.org, pkelsey@freebsd.org, np@FreeBSD.org References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <53861209.2000306@verisign.com> <53880525.6000203@gmail.com> <5665A78E.3090401@freebsd.org> <5635309e-3235-e574-9bd2-a7a484ac0825@freebsd.org> Cc: k simon , hiren panchasara From: Julien Charbon Message-ID: <508b340a-2e55-43e1-99d7-e4b53ca4687d@freebsd.org> Date: Thu, 14 Jul 2016 19:38:47 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <5635309e-3235-e574-9bd2-a7a484ac0825@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2HhO8O9uRaD1aso3m468VOXiFxkG6HDBx" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 17:45:06 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --2HhO8O9uRaD1aso3m468VOXiFxkG6HDBx Content-Type: multipart/mixed; boundary="voOVPM01S0Wwp9kEsjiph80hoCQ7VjDrf" From: Julien Charbon To: freebsd-net@freebsd.org, pkelsey@freebsd.org, np@FreeBSD.org Cc: k simon , hiren panchasara Message-ID: <508b340a-2e55-43e1-99d7-e4b53ca4687d@freebsd.org> Subject: Re: TCP stack lock contention with short-lived connections References: <537F39DF.1090900@verisign.com> <537FB51D.2060401@verisign.com> <53861209.2000306@verisign.com> <53880525.6000203@gmail.com> <5665A78E.3090401@freebsd.org> <5635309e-3235-e574-9bd2-a7a484ac0825@freebsd.org> In-Reply-To: <5635309e-3235-e574-9bd2-a7a484ac0825@freebsd.org> --voOVPM01S0Wwp9kEsjiph80hoCQ7VjDrf Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, On 6/28/16 12:06 PM, Julien Charbon wrote: > On 12/7/15 4:36 PM, Julien Charbon wrote: >> On 30/05/14 06:12, k simon wrote: >>> Does any plan commit and MFC to the 10-stable ? >> >> I got a bit of interest of having the performance improvements for >> short-lived TCP connections in 10-stable. Just to share the current >> status to a wider audience: >> >> - I maintain a stack of our TCP performance related patches for >> 10.2-RELENG here: >> >> https://github.com/verisign/freebsd/commits/10.2/tcp-scale >=20 > Got more request to MFC TCP stack short-lived connection changes (see > below) in 10: >=20 > #1 Decrease lock contention within the TCP accept case by removing > the INP_INFO lock from tcp_usr_accept > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D271119 >=20 > #2 In tcp_input(), don't acquire the pcbinfo global write lock for SYN > packets targeting a listening socket. > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D271119 >=20 > #3 A connection in TIME_WAIT state before calling close() actually did > not received any RST packet. > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D273014 >=20 > #4 Decompose TCP INP_INFO lock to increase short-lived TCP connections > scalability > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D286227 >=20 > Fix a kernel assertion issue introduced with r286227 > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D286443 >=20 > #5 Make clear that TIME_WAIT timeout expiration is managed solely by > tcp_tw_2msl_scan() > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D286873 >=20 > If nobody complains, I plan to MFC them in stable/10. After actually > quite a bunch of tests as I see a lot of changes that might impact thes= e > MFCs (like TFO support, etc.). MFC over stable/10 of above commits done and tested here: https://github.com/verisign/freebsd/commits/10/tcp-scale Nothing particular, the result make stable/10 much closer to -CURRENT TCP locking-wise. Patrick and Navdeep, you might want to look (or not) at the last commit and glance if everything looks ok TFO-wise (Patrick) and Chelsio-wise (Navdeep): https://github.com/verisign/freebsd/commit/787ff2dec1ad8a7343f86c0f6e7591= 47fc64dac8 I did the diff with -CURRENT, and it looked quite the same than -CURRENT state. Will push these commits slowly in stable/10 if all good.= Thanks. -- Julien --voOVPM01S0Wwp9kEsjiph80hoCQ7VjDrf-- --2HhO8O9uRaD1aso3m468VOXiFxkG6HDBx Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXh84sAAoJEKVlQ5Je6dhxwfwIAI9IqC9latFLvTV9LJ+Y+r3L vAWWs9Qhzp7sNVXOcgs5ZPoT2PSOa36hoZuq4GCBlKnw72r82jOv7vIzY/USdkTR qhZRZpFPidbcdl18SArWkLKaVF/6D17xf9NRCekz1BpxQccZz3dlCpLJNSxGo2Xx DBb69UCEAmNjjuXEVcEm/AYa7r22rG0pm2yquN7Gcai6bJKPbOFqH7gw/2r6awJg O/qAx4AXbTVcJlhABax0tnToWCYN5fvxlxh8WZToz3C1HcXXVIAGRKhQSo93FH2C 4/d0wLLelVG/gPtqw2U9eayb5UIJCD9SVzhxmJj2RE/u3FuFpnsTMyaA6Fjcj4M= =W+ck -----END PGP SIGNATURE----- --2HhO8O9uRaD1aso3m468VOXiFxkG6HDBx-- From owner-freebsd-net@freebsd.org Thu Jul 14 20:06:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0243CB9936F for ; Thu, 14 Jul 2016 20:06:32 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C35B11738 for ; Thu, 14 Jul 2016 20:06:31 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-io0-x234.google.com with SMTP id q83so86153583iod.1 for ; Thu, 14 Jul 2016 13:06:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to; bh=svxFh8vxmNvDIzJUNx6EgtmugJzaJCarPvSA0xEJi8o=; b=foLdpNn+cojKTfY1LhJgtM6+kpbM/MbHNbeLSqi7JxTtqutnaJAmHrcp8xojpkojos 28U2qDVJ91TSQgcfxzCtNmr2Fs1j8gs4RDneMIbagPNoHbPQNgB4PHNO7MF8tgiGZwSD evRtVPQ8cCEJrTgRXddwt3lMo3vwxsBOvkshI8qbJ/d55zlxJEVzzlenYvA43+1rWydn w5PnOfYt3KsZLHAbhHmpnz4ZqVDHDA/P/yfxyRraQf5v+3QUxbgw2ErGDVEnX/sgcnk0 gikhwAi9yBBvAupzT4Bx3khmKUTh1H8Gwn2EwroyqWmgvTsxj63vyIUPol+EbEm9+YoA vQ9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:from:date:message-id:subject :to; bh=svxFh8vxmNvDIzJUNx6EgtmugJzaJCarPvSA0xEJi8o=; b=axM1tsTaJxtvmkjU211HGRzk9Vcvs82Kp42wUPvnFQtQoR/1ot//dhtFmBjb0gAzKA E1aLz80+qED4RoCocAPH9/KiZ8xIOuOWDrnuZ+EyEe3sGwgz/oVm6HDrg8NDz4zoOHkC gT/6DxXpAPm57QOIcshBigXCpRfT+2hOW1cuKI+94P6xjFbjOxNXVIlxuTt1zvOnyev7 pJeLsjE/9PbZDtniNcUZo+H7f+kTI2uyajZHhjaL2v1s7V83vBnSELO44trmAB6y8Mlf 20BFrpdV6D1FdPBOmtKmirhD8WekxG+01tLHmxyja6UWJviXPk3I9mYyNKMC69Uq2h0w YNPQ== X-Gm-Message-State: ALyK8tLK3awoKCLiZyxpypEGoQ1SqsE0TTejsDM1Xeu33Win6UM/G13Kpg4sfzilhb5CIW5aLk07H+j/74T1iw== X-Received: by 10.107.144.10 with SMTP id s10mr15999908iod.165.1468526790963; Thu, 14 Jul 2016 13:06:30 -0700 (PDT) MIME-Version: 1.0 Sender: adrian.chadd@gmail.com Received: by 10.36.141.129 with HTTP; Thu, 14 Jul 2016 13:06:30 -0700 (PDT) From: Adrian Chadd Date: Thu, 14 Jul 2016 13:06:30 -0700 X-Google-Sender-Auth: BdAIcqL5aRzOoOJ2L57YtpijLck Message-ID: Subject: proposal: splitting NIC RSS up from stack RSS To: FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 20:06:32 -0000 Hi, now that 11 is branched and marching on, I'd like to start pushing some more improvements/evolution into the RSS side of things. The short list feedback from people is: * it'd be nice to be able to configure per-device RSS keys on the fly; * it'd be nice to be able to configure per-device RSS bucket mappings on the fly; * it'd be nice to be able to configure per-device RSS hash configurations on the fly; * it'd be nice to be able to configure per-bucket CPU set mappings on the fly; * it'd be nice to split the RSS driver side, the RSS packet input side and the RSS stack side of things up into separate options; * UDP IPv6 RSS support would be nice (it works, but i need to test/integrate bz's v6 udp locking changes for it to really matter); * it'd be nice to scale linearly on incoming /and/ outgoing connections. Right now incoming connections are easy, but outgoing connections aren't so easy. The other big thing, mostly to be expected, is: * it'd be nice if this were better documented; * it'd be nice if we had easy examples of this stuff working, complete with library bits in base. I'm going to tidy up the NetworkRSS bits in the wiki soon and map out a roadmap for 12 with some other bits and pieces. The "can we have RSS for NICs but not for the stack, and have keys/mapping/bucket configurable" is actually a biggish thing, as that ties into people wanting to abuse things with netmap. They don't care about the rest of the stack being RSS aware; they just want to be able to control the NIC configurations from userspace and then get it completely out of the way. I'd appreciate any other feedback/comments/suggestions. If you're using RSS and you haven't told me then please let me know! thanks, -adrian From owner-freebsd-net@freebsd.org Thu Jul 14 20:57:38 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93A92B9921B for ; Thu, 14 Jul 2016 20:57:38 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5FF891C19 for ; Thu, 14 Jul 2016 20:57:38 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 58DBAB99219; Thu, 14 Jul 2016 20:57:38 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 581F0B99214; Thu, 14 Jul 2016 20:57:38 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f53.google.com (mail-wm0-f53.google.com [74.125.82.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E60E51C17; Thu, 14 Jul 2016 20:57:37 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f53.google.com with SMTP id i5so4089534wmg.0; Thu, 14 Jul 2016 13:57:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=sjTxoo/eCUL2oM5uNLExyCOqJi5HCTpFlzfpjZNQZl4=; b=dAccgpvybsUDA6DzK1us1Prxcpwju9dVXnFXHUFBje4Cz1Y8bpJ+LTlKsuLv+XvuBW jFiL37bvRdRUNgIoQ5JDHiscl+zMoXvhkhf0HJMkS8pGYuTYl9JIJ5zB3so4ZMTieLYN p3SWEK3/5sJHpFNAQKnvaRBzej9rRdvv+ok2+A/+YvGYqTn8kTghMwKdDhxGPxx6BsFy Uc2x/3XSmXGAops5l2+yvVC+Z9uKblczUcB5/d+acvHLRnqGYE6UKNDeU2ce5NMLLDYC xt5GoAFnQR6WZiDcoMgId1Mb7hdMsIgq9ZNOS+m/kYal0xuvztkXctoi0Ml/6d0rzRS5 Klzw== X-Gm-Message-State: ALyK8tIvFFEYSwuhdIrNv6PYmbcQCQcuZv8FNclyR7Eg9PqWQi1fcU9u3BsCcQeBsKJ4jA== X-Received: by 10.28.199.205 with SMTP id x196mr16683033wmf.96.1468515679633; Thu, 14 Jul 2016 10:01:19 -0700 (PDT) Received: from [172.20.10.4] (4.232.197.178.dynamic.wless.lssmb00p-cgnat.res.cust.swisscom.ch. [178.197.232.4]) by smtp.gmail.com with ESMTPSA id b186sm4968236wmg.23.2016.07.14.10.01.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Jul 2016 10:01:18 -0700 (PDT) Subject: Re: panic with tcp timers To: Gleb Smirnoff , rrs@FreeBSD.org References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> Cc: hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org From: Julien Charbon Message-ID: Date: Thu, 14 Jul 2016 19:01:11 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="3bJTeB9odClQb2ngDjv5fox53UGeQumpb" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 20:57:38 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3bJTeB9odClQb2ngDjv5fox53UGeQumpb Content-Type: multipart/mixed; boundary="wOpjdVBETEfMPxFEjbCdXKObs3doI97cF" From: Julien Charbon To: Gleb Smirnoff , rrs@FreeBSD.org Cc: hselasky@FreeBSD.org, net@FreeBSD.org, current@FreeBSD.org Message-ID: Subject: Re: panic with tcp timers References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> In-Reply-To: <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> --wOpjdVBETEfMPxFEjbCdXKObs3doI97cF Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 6/20/16 11:55 AM, Julien Charbon wrote: > On 6/20/16 9:39 AM, Gleb Smirnoff wrote: >> On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: >> J> > Comparing stable/10 and head, I see two changes that could >> J> > affect that: >> J> >=20 >> J> > - callout_async_drain >> J> > - switch to READ lock for inp info in tcp timers >> J> >=20 >> J> > That's why you are in To, Julien and Hans :) >> J> >=20 >> J> > We continue investigating, and I will keep you updated. >> J> > However, any help is welcome. I can share cores. >> >> Now, spending some time with cores and adding a bunch of >> extra CTRs, I have a sequence of events that lead to the >> panic. In short, the bug is in the callout system. It seems >> to be not relevant to the callout_async_drain, at least for >> now. The transition to READ lock unmasked the problem, that's >> why NetflixBSD 10 doesn't panic. >> >> The panic requires heavy contention on the TCP info lock. >> >> [CPU 1] the callout fires, tcp_timer_keep entered >> [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); >> [CPU 2] schedules the callout >> [CPU 2] tcp_discardcb called >> [CPU 2] callout successfully canceled >> [CPU 2] tcpcb freed >> [CPU 1] unblocks... panic >> >> When the lock was WLOCK, all contenders were resumed in a >> sequence they came to the lock. Now, that they are readers, >> once the lock is released, readers are resumed in a "random" >> order, and this allows tcp_discardcb to go before the old >> running callout, and this unmasks the panic. >=20 > Highly interesting. I should be able to reproduce that (will be usefu= l > for testing the corresponding fix). Finally, I was able to reproduce it (without glebius fix). The trick was to really lower TCP keep timer expiration: $ sysctl -a | grep tcp.keep net.inet.tcp.keepidle: 7200000 net.inet.tcp.keepintvl: 75000 net.inet.tcp.keepinit: 75000 net.inet.tcp.keepcnt: 8 $ sudo bash -c "sysctl net.inet.tcp.keepidle=3D10 && sysctl net.inet.tcp.keepintvl=3D50 && sysctl net.inet.tcp.keepinit=3D10" Password: net.inet.tcp.keepidle: 7200000 -> 10 net.inet.tcp.keepintvl: 75000 -> 50 net.inet.tcp.keepinit: 75000 -> 10 Note: It will certainly close all your ssh connections to the tested server. Now I will test in order: #1. glebius fix https://svnweb.freebsd.org/base?view=3Drevision&revision=3D302350 #2. rss extra fix https://reviews.freebsd.org/D7135 #3. rrs TCP Timer cleanup https://reviews.freebsd.org/D7136 My panic for reference: Fatal trap 9: general protection fault while in kernel mode cpuid =3D 10; apic id =3D 28 [root@atlas-dl360-4 ~]# instruction pointer =3D 0x20:0xffffffff80c346= f1 stack pointer =3D 0x28:0xfffffe1f29b848b0 frame pointer =3D 0x28:0xfffffe1f29b848e0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 12 (swi4: clock (4)) trap number =3D 9 panic: general protection fault cpuid =3D 10 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe1f29b844a0 vpanic() at vpanic+0x182/frame 0xfffffe1f29b84520 panic() at panic+0x43/frame 0xfffffe1f29b84580 trap_fatal() at trap_fatal+0x351/frame 0xfffffe1f29b845e0 trap() at trap+0x820/frame 0xfffffe1f29b847f0 calltrap() at calltrap+0x8/frame 0xfffffe1f29b847f0 --- trap 0x9, rip =3D 0xffffffff80c346f1, rsp =3D 0xfffffe1f29b848c0, rbp= =3D 0xfffffe1f29b848e0 --- tcp_timer_keep() at tcp_timer_keep+0x51/frame 0xfffffe1f29b848e0 softclock_call_cc() at softclock_call_cc+0x19c/frame 0xfffffe1f29b849c0 softclock() at softclock+0x47/frame 0xfffffe1f29b849e0 intr_event_execute_handlers() at intr_event_execute_handlers+0x96/frame 0xfffffe1f29b84a20 ithread_loop() at ithread_loop+0xa6/frame 0xfffffe1f29b84a70 fork_exit() at fork_exit+0x84/frame 0xfffffe1f29b84ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f29b84ab0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- -- Julien --wOpjdVBETEfMPxFEjbCdXKObs3doI97cF-- --3bJTeB9odClQb2ngDjv5fox53UGeQumpb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJXh8VdAAoJEKVlQ5Je6dhxhccH/R7BIEReY5MtXw8l37IDBIB2 pK2uuSS+mvscTnIUzJcaCMPfXLsH/b5gmFpaqFGhouVsl0Z/pBl45br2jMXggFph Z9ApSUFhEdfkTeM0tVp2VHOnMnIn8+L/gdSY4S2dKyPk/rEq/5DzIf0Ys2q34XJ1 WTltD3IsDjS1baOpy4O6iwSgoZnNTuZerOQqsJXmZ+ZayLM9OF/TGS8w+ztqewQL 9eKfZM7EoYKVdMsYjD/ECZOGy1pw9lFflHQkNaSdUMCePFPLy29DoTXSfALzl5+P 4JLnkRxKzoLoy8ep3LzVm91lwGZIigrkWGobGqAo+YYR9Np6Aq0680ZggPn50Ac= =rq23 -----END PGP SIGNATURE----- --3bJTeB9odClQb2ngDjv5fox53UGeQumpb-- From owner-freebsd-net@freebsd.org Thu Jul 14 21:02:26 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BA37B994A0 for ; Thu, 14 Jul 2016 21:02:26 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 15FEC12B1 for ; Thu, 14 Jul 2016 21:02:26 +0000 (UTC) (envelope-from ler@lerctr.org) Received: by mailman.ysv.freebsd.org (Postfix) id 11B32B9949A; Thu, 14 Jul 2016 21:02:26 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10E56B99498; Thu, 14 Jul 2016 21:02:26 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C86CA12AD; Thu, 14 Jul 2016 21:02:25 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=0bUhy+HwcKimMN98IUmKSSIcpZtLF9apKdNSGskKiaQ=; b=KUrqvK4oH79Df8KvoqdREXT3se 5bKHrKpOv3+Q1yM4MaF9lVu5k4aGicSC2/9qgPGkP7xP5HEeWD44o3pEjTgmpQISlmrdCuBhvUCbm Hx9UNecJ08tS7W+cFqUaYyonX0eyJ7Bauq2buYYOh1tEbRWE1oQVfhIOQV287FVyVRQk=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:12473 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bNnm5-000HmE-51; Thu, 14 Jul 2016 16:02:25 -0500 Received: from proxy.na.alcatel-lucent.com ([135.245.48.74]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 14 Jul 2016 16:02:25 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 14 Jul 2016 16:02:25 -0500 From: Larry Rosenman To: Julien Charbon Cc: Gleb Smirnoff , rrs@freebsd.org, hselasky@freebsd.org, net@freebsd.org, current@freebsd.org, owner-freebsd-current@freebsd.org Subject: Re: panic with tcp timers In-Reply-To: References: <20160617045319.GE1076@FreeBSD.org> <1f28844b-b4ea-b544-3892-811f2be327b9@freebsd.org> <20160620073917.GI1076@FreeBSD.org> <1d18d0e2-3e42-cb26-928c-2989d0751884@freebsd.org> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 21:02:26 -0000 On 2016-07-14 12:01, Julien Charbon wrote: > Hi, > > On 6/20/16 11:55 AM, Julien Charbon wrote: >> On 6/20/16 9:39 AM, Gleb Smirnoff wrote: >>> On Fri, Jun 17, 2016 at 11:27:39AM +0200, Julien Charbon wrote: >>> J> > Comparing stable/10 and head, I see two changes that could >>> J> > affect that: >>> J> > >>> J> > - callout_async_drain >>> J> > - switch to READ lock for inp info in tcp timers >>> J> > >>> J> > That's why you are in To, Julien and Hans :) >>> J> > >>> J> > We continue investigating, and I will keep you updated. >>> J> > However, any help is welcome. I can share cores. >>> >>> Now, spending some time with cores and adding a bunch of >>> extra CTRs, I have a sequence of events that lead to the >>> panic. In short, the bug is in the callout system. It seems >>> to be not relevant to the callout_async_drain, at least for >>> now. The transition to READ lock unmasked the problem, that's >>> why NetflixBSD 10 doesn't panic. >>> >>> The panic requires heavy contention on the TCP info lock. >>> >>> [CPU 1] the callout fires, tcp_timer_keep entered >>> [CPU 1] blocks on INP_INFO_RLOCK(&V_tcbinfo); >>> [CPU 2] schedules the callout >>> [CPU 2] tcp_discardcb called >>> [CPU 2] callout successfully canceled >>> [CPU 2] tcpcb freed >>> [CPU 1] unblocks... panic >>> >>> When the lock was WLOCK, all contenders were resumed in a >>> sequence they came to the lock. Now, that they are readers, >>> once the lock is released, readers are resumed in a "random" >>> order, and this allows tcp_discardcb to go before the old >>> running callout, and this unmasks the panic. >> >> Highly interesting. I should be able to reproduce that (will be >> useful >> for testing the corresponding fix). > > Finally, I was able to reproduce it (without glebius fix). The trick > was to really lower TCP keep timer expiration: > > $ sysctl -a | grep tcp.keep > net.inet.tcp.keepidle: 7200000 > net.inet.tcp.keepintvl: 75000 > net.inet.tcp.keepinit: 75000 > net.inet.tcp.keepcnt: 8 > $ sudo bash -c "sysctl net.inet.tcp.keepidle=10 && sysctl > net.inet.tcp.keepintvl=50 && sysctl net.inet.tcp.keepinit=10" > Password: > net.inet.tcp.keepidle: 7200000 -> 10 > net.inet.tcp.keepintvl: 75000 -> 50 > net.inet.tcp.keepinit: 75000 -> 10 > > Note: It will certainly close all your ssh connections to the tested > server. > > Now I will test in order: > > #1. glebius fix > https://svnweb.freebsd.org/base?view=revision&revision=302350 > > #2. rss extra fix > https://reviews.freebsd.org/D7135 > > #3. rrs TCP Timer cleanup > https://reviews.freebsd.org/D7136 > > My panic for reference: > > Fatal trap 9: general protection fault while in kernel mode > cpuid = 10; apic id = 28 > [root@atlas-dl360-4 ~]# instruction pointer = > 0x20:0xffffffff80c346f1 > stack pointer = 0x28:0xfffffe1f29b848b0 > frame pointer = 0x28:0xfffffe1f29b848e0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (swi4: clock (4)) > trap number = 9 > panic: general protection fault > cpuid = 10 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe1f29b844a0 > vpanic() at vpanic+0x182/frame 0xfffffe1f29b84520 > panic() at panic+0x43/frame 0xfffffe1f29b84580 > trap_fatal() at trap_fatal+0x351/frame 0xfffffe1f29b845e0 > trap() at trap+0x820/frame 0xfffffe1f29b847f0 > calltrap() at calltrap+0x8/frame 0xfffffe1f29b847f0 > --- trap 0x9, rip = 0xffffffff80c346f1, rsp = 0xfffffe1f29b848c0, rbp = > 0xfffffe1f29b848e0 --- > tcp_timer_keep() at tcp_timer_keep+0x51/frame 0xfffffe1f29b848e0 > softclock_call_cc() at softclock_call_cc+0x19c/frame 0xfffffe1f29b849c0 > softclock() at softclock+0x47/frame 0xfffffe1f29b849e0 > intr_event_execute_handlers() at intr_event_execute_handlers+0x96/frame > 0xfffffe1f29b84a20 > ithread_loop() at ithread_loop+0xa6/frame 0xfffffe1f29b84a70 > fork_exit() at fork_exit+0x84/frame 0xfffffe1f29b84ab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f29b84ab0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > > -- > Julien please see also https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210884 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-net@freebsd.org Thu Jul 14 21:19:22 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06993B99837 for ; Thu, 14 Jul 2016 21:19:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA7BC194F for ; Thu, 14 Jul 2016 21:19:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6ELJL3q021619 for ; Thu, 14 Jul 2016 21:19:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194109] [lor] if_lagg rmlock <-> if_addr_lock Date: Thu, 14 Jul 2016 21:19:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: np@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: version cc bug_severity Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jul 2016 21:19:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D194109 Navdeep Parhar changed: What |Removed |Added ---------------------------------------------------------------------------- Version|CURRENT |11.0-STABLE CC| |melifaro@FreeBSD.org, | |np@FreeBSD.org, | |re@FreeBSD.org, | |sbruno@FreeBSD.org Severity|Affects Only Me |Affects Many People --- Comment #2 from Navdeep Parhar --- Deadlocks due to this LOR are easy to reproduce.=20 https://reviews.freebsd.org/D6845 has a proposed fix (not reviewed yet).=20 r272211 is mentioned as the possible culprit in that review. This still occurs on 11.0-BETA1 so I've updated the version and added re@ to the CC list. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 15 02:22:50 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1940FB97047 for ; Fri, 15 Jul 2016 02:22:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0685B1A32 for ; Fri, 15 Jul 2016 02:22:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6F2Mnrw019651 for ; Fri, 15 Jul 2016 02:22:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 211106] [net][fib][loopback][route] loopback route added only to interface fib, even if net.add_addr_allfibs = 1 Date: Fri, 15 Jul 2016 02:22:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 02:22:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211106 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 15 03:45:31 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA40DB982BA; Fri, 15 Jul 2016 03:45:31 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC4651306; Fri, 15 Jul 2016 03:45:31 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1468554328859242.3635984150202; Thu, 14 Jul 2016 20:45:28 -0700 (PDT) Date: Thu, 14 Jul 2016 20:45:28 -0700 From: Matthew Macy To: "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" Message-ID: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> Subject: callout_drain either broken or man page needs updating MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 03:45:31 -0000 Upon updating my drm-next branch to the latest -CURRENT callout_drain returning no longer means that the function was in fact pending when it was called. This little bit of code will panic because dwork->wq is NULL, because the callout was _not_ in fact enqueued. So either it's no longer possible to reliably query if a callout was pending while clearing it and we're ok with that or glebius last commit needs some further re-work. #define del_timer_sync(timer) (callout_drain(&(timer)->timer_callout) == 1) static inline bool flush_delayed_work(struct delayed_work *dwork) { if (del_timer_sync(&dwork->timer)) linux_queue_work(dwork->cpu, dwork->wq, &dwork->work); return (flush_work(&dwork->work)); } From owner-freebsd-net@freebsd.org Fri Jul 15 04:21:50 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F472B98CE7; Fri, 15 Jul 2016 04:21:50 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AD76812EB; Fri, 15 Jul 2016 04:21:49 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 45F861FE024; Fri, 15 Jul 2016 06:21:46 +0200 (CEST) Subject: Re: callout_drain either broken or man page needs updating To: Matthew Macy , "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" , Gleb Smirnoff References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> From: Hans Petter Selasky Message-ID: <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> Date: Fri, 15 Jul 2016 06:25:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 04:21:50 -0000 On 07/15/16 05:45, Matthew Macy wrote: > glebius last commit needs some further re-work. Hi, Glebius commit needs to be backed out, at least the API change that changes the return value when calling callout_stop() when the callout is scheduled and being serviced. Simply because there is code out there, like Mattew and others have discovered that is "refcounting" on the callout_reset() and expecting that a subsequent callout_stop() will return 1 to "unref". If you consider this impossible, maybe a fourth return value is needed for CANCELLED and DRAINING . Further, getting the callouts straight in the TCP stack is a matter of doing the locking correctly, which some has called "my magic bullet" and not the return values. I've proposed in the following revision https://svnweb.freebsd.org/changeset/base/302768 to add a new callout API that accepts a locking function so that the callout code can run its cancelled checks at the right place for situations where more than one lock is needed. Consider this case: > void > tcp_timer_2msl(void *xtp) > { > struct tcpcb *tp = xtp; > struct inpcb *inp; > CURVNET_SET(tp->t_vnet); > #ifdef TCPDEBUG > int ostate; > > ostate = tp->t_state; > #endif > INP_INFO_RLOCK(&V_tcbinfo); > inp = tp->t_inpcb; > KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); > INP_WLOCK(inp); > tcp_free_sackholes(tp); > if (callout_pending(&tp->t_timers->tt_2msl) || > !callout_active(&tp->t_timers->tt_2msl)) { Here we have custom in-house race check that doesn't affect the return value of callout_reset() nor callout_stop(). > INP_WUNLOCK(tp->t_inpcb); > INP_INFO_RUNLOCK(&V_tcbinfo); > CURVNET_RESTORE(); > return; I propose the following solution: > > static void > tcp_timer_2msl_lock(void *xtp, int do_lock) > { > struct tcpcb *tp = xtp; > struct inpcb *inp; > > inp = tp->t_inpcb; > > if (do_lock) { > CURVNET_SET(tp->t_vnet); > INP_INFO_RLOCK(&V_tcbinfo); > INP_WLOCK(inp); > } else { > INP_WUNLOCK(inp); > INP_INFO_RUNLOCK(&V_tcbinfo); > CURVNET_RESTORE(); > } > } > callout_init_lock_function(&callout, &tcp_timer_2msl_lock, CALLOUT_RETURNUNLOCKED); Then in softclock_call_cc() it will look like this: > > CC_UNLOCK(cc); > if (c_lock != NULL) { > if (have locking function) > tcp_timer_2msl_lock(c_arg, 1); > else > class->lc_lock(c_lock, lock_status); > /* > * The callout may have been cancelled > * while we switched locks. > */ Actually "CC_LOCK(cc)" should be in-front of cc_exec_cancel() to avoid races testing, setting and clearing this variable, like done in hps_head. > if (cc_exec_cancel(cc, direct)) { > if (have locking function) > tcp_timer_2msl_lock(c_arg, 0); > else > class->lc_unlock(c_lock); > goto skip; > } > cc_exec_cancel(cc, direct) = true; > > .... > > skip: > if ((c_iflags & CALLOUT_RETURNUNLOCKED) == 0) { > if (have locking function) > ... > else > class->lc_unlock(c_lock); > } The whole point about this is to make the the cancelled check atomic. 1) Lock TCP 2) Lock CC_LOCK() 3) change callout state --HPS From owner-freebsd-net@freebsd.org Fri Jul 15 04:38:21 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 721E4B970AA; Fri, 15 Jul 2016 04:38:21 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E48D1ADF; Fri, 15 Jul 2016 04:38:21 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1468557492950338.6732714114978; Thu, 14 Jul 2016 21:38:12 -0700 (PDT) Date: Thu, 14 Jul 2016 21:38:12 -0700 From: Matthew Macy To: "Larry Rosenman" Cc: =?UTF-8?Q?=22Edward_Tomasz_Napiera=C5=82a=22?= , "" , "" , "freebsd-net@freebsd.org" Message-ID: <155ecd901bd.10421acf2569401.8259879667024383313@nextbsd.org> In-Reply-To: <90e04710a003b900f9d16d853636f767@thebighonker.lerctr.org> References: <20160706231707.GA2352@borg.lerctr.org> <9f51ccaf5e8f15f269fa865a3adfdc7f@thebighonker.lerctr.org> <20160707132814.GA77070@brick> <90e04710a003b900f9d16d853636f767@thebighonker.lerctr.org> Subject: Re: refcnt 0 on LLE at boot.... MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 04:38:21 -0000 ---- On Thu, 07 Jul 2016 06:36:19 -0700 Larry Rosenman wr= ote ----=20 > Thanks for that. I've added myself to the cc list, and a comment about= =20 > having 2 vmcore's. >=20 This was introduced by 302350. It broke the return value of callout_{stop= ,drain}. returning 1 even if the callout system did not hold a reference. T= hat in turn broke the following code in lltable_free: =09LIST_FOREACH_SAFE(lle, &dchain, lle_chain, next) { =09=09if (callout_stop(&lle->lle_timer) > 0) =09=09=09LLE_REMREF(lle); =09=09llentry_free(lle); =09} >=20 > On 2016-07-07 08:28, Edward Tomasz Napiera=C5=82a wrote: > > FWIW, I'm seeing this too. I've filed a PR: > >=20 > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210884 > >=20 > > On 0707T0813, Larry Rosenman wrote: > >> and now it's been up for 13+ hours. I do have both VMCORE's from the= =20 > >> 2 > >> crashes. > >>=20 > >>=20 > >>=20 > >> On 2016-07-06 18:22, Larry Rosenman wrote: > >> > Got a similar crash a few minutes later. > >> > > >> > > >> > On 2016-07-06 18:17, Larry Rosenman wrote: > >> >> First boot, and I got the following panic. 2nd boot ran just fine= . > >> >> > >> >> > >> >> borg.lerctr.org dumped core - see /var/crash/vmcore.0 > >> >> > >> >> Wed Jul 6 18:13:34 CDT 2016 > >> >> > >> >> FreeBSD borg.lerctr.org 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #5 r302379= : > >> >> Wed Jul 6 16:59:11 CDT 2016 > >> >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > >> >> > >> >> panic: bogus refcnt 0 on lle 0xfffff800aa941200 > >> >> > >> >> GNU gdb 6.1.1 [FreeBSD] > >> >> Copyright 2004 Free Software Foundation, Inc. > >> >> GDB is free software, covered by the GNU General Public License, a= nd > >> >> you are > >> >> welcome to change it and/or distribute copies of it under certain > >> >> conditions. > >> >> Type "show copying" to see the conditions. > >> >> There is absolutely no warranty for GDB. Type "show warranty" for > >> >> details. > >> >> This GDB was configured as "amd64-marcel-freebsd"... > >> >> > >> >> Unread portion of the kernel message buffer: > >> >> Copyright (c) 1992-2016 The FreeBSD Project. > >> >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993= , > >> >> 1994 > >> >> The Regents of the University of California. All rights reserv= ed. > >> >> FreeBSD is a registered trademark of The FreeBSD Foundation. > >> >> FreeBSD 11.0-ALPHA6 #5 r302379: Wed Jul 6 16:59:11 CDT 2016 > >> >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > >> >> FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based= on > >> >> LLVM 3.8.0) > >> >> can't re-use a leaf (ixl_rx_miss_bufs)! > >> >> MEMGUARD DEBUGGING ALLOCATOR INITIALIZED: > >> >> MEMGUARD map base: 0xfffffe0000400000 > >> >> MEMGUARD map size: 128604256 KBytes > >> >> VT(vga): resolution 640x480 > >> >> CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz > >> >> K8-class CPU) > >> >> Origin=3D"GenuineIntel" Id=3D0x10676 Family=3D0x6 Model=3D0x1= 7 > >> >> Stepping=3D6 > >> >> > >> >> Features=3D0xbfebfbff > >> >> > >> >> Features2=3D0xce3bd > >> >> AMD Features=3D0x20100800 > >> >> AMD Features2=3D0x1 > >> >> VT-x: HLT,PAUSE > >> >> TSC: P-state invariant, performance statistics > >> >> real memory =3D 68719476736 (65536 MB) > >> >> avail memory =3D 65382842368 (62353 MB) > >> >> Event timer "LAPIC" quality 400 > >> >> ACPI APIC Table: > >> >> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs > >> >> FreeBSD/SMP: 2 package(s) x 4 core(s) > >> >> random: unblocking device. > >> >> ioapic0 irqs 0-23 on motherboard > >> >> ioapic1 irqs 24-47 on motherboard > >> >> random: entropy device external interface > >> >> netmap: loaded module > >> >> module_register_init: MOD_LOAD (vesa, 0xffffffff80f2cb40, 0) error= 19 > >> >> kbd1 at kbdmux0 > >> >> vtvga0: on motherboard > >> >> cryptosoft0: on motherboard > >> >> acpi0: on motherboard > >> >> acpi0: Power Button (fixed) > >> >> unknown: I/O range not supported > >> >> cpu0: on acpi0 > >> >> cpu1: on acpi0 > >> >> cpu2: on acpi0 > >> >> cpu3: on acpi0 > >> >> cpu4: on acpi0 > >> >> cpu5: on acpi0 > >> >> cpu6: on acpi0 > >> >> cpu7: on acpi0 > >> >> hpet0: iomem 0xfed00000-0xfed003ff ir= q > >> >> 0,8 on acpi0 > >> >> Timecounter "HPET" frequency 14318180 Hz quality 950 > >> >> Event timer "HPET" frequency 14318180 Hz quality 350 > >> >> Event timer "HPET1" frequency 14318180 Hz quality 340 > >> >> Event timer "HPET2" frequency 14318180 Hz quality 340 > >> >> atrtc0: port 0x70-0x71 on acpi0 > >> >> Event timer "RTC" frequency 32768 Hz quality 0 > >> >> attimer0: port 0x40-0x43,0x50-0x53 on acpi0 > >> >> Timecounter "i8254" frequency 1193182 Hz quality 0 > >> >> Event timer "i8254" frequency 1193182 Hz quality 100 > >> >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > >> >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on a= cpi0 > >> >> pcib0: port 0xcf8-0xcff on acpi0 > >> >> pci0: on pcib0 > >> >> pcib1: at device 2.0 on pci0 > >> >> pci1: on pcib1 > >> >> pcib2: irq 16 at device 0.0 on pci1 > >> >> pci2: on pcib2 > >> >> pcib3: irq 16 at device 0.0 on pci2 > >> >> pci3: on pcib3 > >> >> pcib4: at device 0.0 on pci3 > >> >> pci4: on pcib4 > >> >> pcib5: at device 0.2 on pci3 > >> >> pci5: on pcib5 > >> >> pcib6: irq 18 at device 2.0 on pci2 > >> >> pci6: on pcib6 > >> >> em0: port 0x2000-0x= 201f > >> >> mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0= .0 > >> >> on pci6 > >> >> em0: Using an MSI interrupt > >> >> em0: Ethernet address: 00:30:48:f2:29:9c > >> >> em0: netmap queues/slots: TX 1/1024, RX 1/1024 > >> >> em1: port 0x2020-0x= 203f > >> >> mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0= .1 > >> >> on pci6 > >> >> em1: Using an MSI interrupt > >> >> em1: Ethernet address: 00:30:48:f2:29:9d > >> >> em1: netmap queues/slots: TX 1/1024, RX 1/1024 > >> >> pcib7: at device 0.3 on pci1 > >> >> pci7: on pcib7 > >> >> pcib8: at device 4.0 on pci0 > >> >> pci8: on pcib8 > >> >> vgapci0: port 0x3000-0x307f mem > >> >> 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff = irq > >> >> 36 at device 0.0 on pci8 > >> >> hdac0: mem 0xd9000000-0xd9003fff irq= 37 > >> >> at device 0.1 on pci8 > >> >> pcib9: at device 6.0 on pci0 > >> >> pci9: on pcib9 > >> >> pcib10: irq 17 at device 28.0 on pci0 > >> >> pcib10: [GIANT-LOCKED] > >> >> pci10: on pcib10 > >> >> pcib11: irq 32 at device 0.0 on pci10 > >> >> pci11: on pcib11 > >> >> pcm0: port 0x4080-0x409f,0x4000-0x407f = irq > >> >> 32 at device 0.0 on pci11 > >> >> pcm0: system configuration > >> >> SubVendorID: 0x1412, SubDeviceID: 0x2403 > >> >> XIN2 Clock Source: 24.576MHz(96kHz*256) > >> >> MPU-401 UART(s) #: not implemented > >> >> ADC #: 1 and SPDIF receiver connected > >> >> DAC #: 4 > >> >> Multi-track converter type: AC'97(SDATA_OUT:packed) > >> >> S/PDIF(IN/OUT): 1/1 ID# 0x00 > >> >> GPIO(mask/dir/state): 0xff/0xff/0xff > >> >> uhci0: port > >> >> 0x1800-0x181f irq 17 at device 29.0 on pci0 > >> >> usbus0 on uhci0 > >> >> uhci1: port > >> >> 0x1820-0x183f irq 19 at device 29.1 on pci0 > >> >> usbus1 on uhci1 > >> >> uhci2: port > >> >> 0x1840-0x185f irq 18 at device 29.2 on pci0 > >> >> usbus2 on uhci2 > >> >> ehci0: mem 0xd9600400-0xd96007f= f > >> >> irq 17 at device 29.7 on pci0 > >> >> usbus3: EHCI version 1.0 > >> >> usbus3 on ehci0 > >> >> pcib12: at device 30.0 on pci0 > >> >> pci12: on pcib12 > >> >> vgapci1: port 0x5000-0x50ff mem > >> >> 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 o= n > >> >> pci12 > >> >> vgapci1: Boot video device > >> >> isab0: at device 31.0 on pci0 > >> >> isa0: on isab0 > >> >> atapci0: port > >> >> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 o= n > >> >> pci0 > >> >> ata0: at channel 0 on atapci0 > >> >> ahci0: port > >> >> 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x1= 89f > >> >> mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 > >> >> ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported > >> >> ahcich0: at channel 0 on ahci0 > >> >> ahcich1: at channel 1 on ahci0 > >> >> ahcich2: at channel 2 on ahci0 > >> >> ahcich3: at channel 3 on ahci0 > >> >> ahcich4: at channel 4 on ahci0 > >> >> ahcich5: at channel 5 on ahci0 > >> >> ichsmb0: port > >> >> 0x1100-0x111f irq 19 at device 31.3 on pci0 > >> >> smbus0: on ichsmb0 > >> >> acpi_button0: on acpi0 > >> >> ipmi0: port 0xca2-0xca3 on acpi0 > >> >> ipmi0: KCS mode found at io 0xca2 on acpi > >> >> atkbdc0: port 0x60,0x64 irq 1 on acp= i0 > >> >> atkbd0: irq 1 on atkbdc0 > >> >> kbd0 at atkbd0 > >> >> atkbd0: [GIANT-LOCKED] > >> >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on > >> >> acpi0 > >> >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > >> >> fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2= on > >> >> acpi0 > >> >> fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > >> >> ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on > >> >> acpi0 > >> >> ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode > >> >> ppc0: FIFO with 16/16/9 bytes threshold > >> >> ppbus0: on ppc0 > >> >> lpt0: on ppbus0 > >> >> lpt0: Interrupt-driven port > >> >> ppi0: on ppbus0 > >> >> ichwd0: on isa0 > >> >> orm0: at iomem 0xc0000-0xcafff on isa0 > >> >> coretemp0: on cpu0 > >> >> est0: on cpu0 > >> >> coretemp1: on cpu1 > >> >> est1: on cpu1 > >> >> coretemp2: on cpu2 > >> >> est2: on cpu2 > >> >> coretemp3: on cpu3 > >> >> est3: on cpu3 > >> >> coretemp4: on cpu4 > >> >> est4: on cpu4 > >> >> coretemp5: on cpu5 > >> >> est5: on cpu5 > >> >> coretemp6: on cpu6 > >> >> est6: on cpu6 > >> >> coretemp7: on cpu7 > >> >> est7: on cpu7 > >> >> usbus0: 12Mbps Full Speed USB v1.0 > >> >> fuse-freebsd: version 0.4.4, FUSE ABI 7.8 > >> >> ZFS filesystem version: 5 > >> >> ZFS storage pool version: features support (5000) > >> >> Timecounters tick every 1.000 msec > >> >> nvme cam probe device init > >> >> hdacc0: at cad 0 on hdac0 > >> >> hdaa0: at nid 1 on hdacc0 > >> >> pcm1: at nid 4 on hdaa0 > >> >> pcm2: at nid 5 on hdaa0 > >> >> ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 > >> >> ipmi0: Number of channels 8 > >> >> ipmi0: Attached watchdog > >> >> usbus1: 12Mbps Full Speed USB v1.0 > >> >> usbus2: 12Mbps Full Speed USB v1.0 > >> >> ugen0.1: at usbus0 > >> >> uhub0: on > >> >> usbus0 > >> >> ugen1.1: at usbus1 > >> >> uhub1: on > >> >> usbus1 > >> >> ugen2.1: at usbus2 > >> >> uhub2: on > >> >> usbus2 > >> >> usbus3: 480Mbps High Speed USB v2.0 > >> >> ugen3.1: at usbus3 > >> >> uhub3: on > >> >> usbus3 > >> >> ata0: DMA limited to UDMA33, controller found non-ATA66 cable > >> >> uhub0: 2 ports with 2 removable, self powered > >> >> uhub1: 2 ports with 2 removable, self powered > >> >> uhub2: 2 ports with 2 removable, self powered > >> >> ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 > >> >> ada0: ACS-2 ATA SATA 3.x device > >> >> cd0 at ata0 bus 0 scbus0 target 0 lun 0 > >> >> cd0: Removable CD-ROM SCSI device > >> >> cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) > >> >> cd0: Attempt to query device size failed: NOT READY, Medium not > >> >> present > >> >> ada0: Serial Number Z3032MSL > >> >> ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada0: 3815447MB (7814037168 512 byte sectors) > >> >> ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 > >> >> ada1: ATA8-ACS SATA 3.x device > >> >> ada1: Serial Number 5YD6FPLG > >> >> ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada1: 1907729MB (3907029168 512 byte sectors) > >> >> ada1: quirks=3D0x1<4K> > >> >> ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 > >> >> ada2: ATA8-ACS SATA 3.x device > >> >> ada2: Serial Number 5YDA3PC5 > >> >> ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada2: 1907729MB (3907029168 512 byte sectors) > >> >> ada2: quirks=3D0x1<4K> > >> >> ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 > >> >> ada3: ATA8-ACS SATA 3.x device > >> >> ada3: Serial Number Y4ODK8VHFSBA > >> >> ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > >> >> ada3: 3815447MB (7814037168 512 byte sectors) > >> >> ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 > >> >> ada4: ACS-2 ATA SATA 3.x device > >> >> ada4: Serial Number S30144ZV > >> >> ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada4: 3815447MB (7814037168 512 byte sectors) > >> >> ada4: quirks=3D0x1<4K> > >> >> ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 > >> >> ada5: ATA8-ACS SATA 3.x device > >> >> ada5: Serial Number 5YD5RBS8 > >> >> ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada5: 1907729MB (3907029168 512 byte sectors) > >> >> ada5: quirks=3D0x1<4K> > >> >> SMP: AP CPU #6 Launched! > >> >> SMP: AP CPU #4 Launched! > >> >> SMP: AP CPU #7 Launched! > >> >> SMP: AP CPU #2 Launched! > >> >> SMP: AP CPU #5 Launched! > >> >> SMP: AP CPU #1 Launched! > >> >> SMP: AP CPU #3 Launched! > >> >> Timecounter "TSC-low" frequency 1163772834 Hz quality 1000 > >> >> hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 > >> >> IAP/2/40/0x3ff > >> >> IAF/3/40/0x67 > >> >> Trying to mount root from zfs:zroot/ROOT/oldroot []... > >> >> Root mount waiting for: usbus3 > >> >> uhub3: 6 ports with 6 removable, self powered > >> >> Root mount waiting for: usbus3 > >> >> Root mount waiting for: usbus3 > >> >> ugen3.2: at usbus3 > >> >> ukbd0: on > >> >> usbus3 > >> >> kbd2 at ukbd0 > >> >> ugen0.2: at usbus0 > >> >> <118>Setting hostuuid: 53d19f64-d663-a017-8922-003048f2299c. > >> >> <118>Setting hostid: 0xbf916f25. > >> >> <118>Starting file system checks: > >> >> <118>Mounting local filesystems:. > >> >> <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/l= ib > >> >> /usr/local/lib/compat /usr/local/lib/gcc48 /usr/local/lib/mysql > >> >> /usr/local/lib/perl5/5.20/mach/CORE /usr/local/lib/qt4 > >> >> /usr/local/lib/virtualbox /usr/local/llvm36/lib /usr/local/llvm37/= lib > >> >> <118>32-bit compatibility ldconfig path: /usr/lib32 > >> >> /usr/local/lib32/compat > >> >> <118>/etc/rc: WARNING: $kldxref_enable is not set properly - see > >> >> rc.conf(5). > >> >> <118>Setting hostname: borg.lerctr.org. > >> >> <118>Setting up harvesting: > >> >> [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYB= OARD,ATTACH,CACHED > >> >> <118>Feeding entropy: . > >> >> <5>lagg0: link state changed to DOWN > >> >> <118>Created clone interfaces: lagg0. > >> >> <118>Starting Network: lo0 em0 em1 lagg0. > >> >> <118>lo0: flags=3D8049 metric 0 mtu= 16384 > >> >> <118> options=3D600003 > >> >> <118> inet6 ::1 prefixlen 128 > >> >> <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > >> >> <118> inet 127.0.0.1 netmask 0xff000000 > >> >> <118> nd6 options=3D21 > >> >> <118> groups: lo > >> >> <118>em0: flags=3D8843 met= ric 0 > >> >> mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> nd6 options=3D29 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118>em1: flags=3D8843 met= ric 0 > >> >> mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> nd6 options=3D29 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118>lagg0: flags=3D8843 m= etric > >> >> 0 mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> inet 192.168.200.4 netmask 0xfffffc00 broadcast 192.168.2= 03.255 > >> >> <118> inet6 fe80::230:48ff:fef2:299c%lagg0 prefixlen 64 scopeid= 0x4 > >> >> <118> nd6 options=3D23 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118> groups: lagg > >> >> <118> laggproto loadbalance lagghash l2,l3,l4 > >> >> <118> laggport: em0 flags=3D0<> > >> >> <118> laggport: em1 flags=3D0<> > >> >> <118>Starting devd. > >> >> uhid0: on > >> >> usbus3 > >> >> <118>add host 127.0.0.1: gateway lo0 fib 0: route already in table > >> >> <118>add net default: gateway 192.168.200.11 > >> >> <118>add host ::1: gateway lo0 fib 0: route already in table > >> >> <118>add net fe80::: gateway ::1 > >> >> <118>add net ff02::: gateway ::1 > >> >> <118>add net ::ffff:0.0.0.0: gateway ::1 > >> >> <118>add net ::0.0.0.0: gateway ::1 > >> >> <118>Starting local_unbound. > >> >> <5>em0: link state changed to UP > >> >> <5>lagg0: link state changed to UP > >> >> <118>Waiting for nameserver to start... good > >> >> <118>Starting rtsold. > >> >> <5>em1: link state changed to UP > >> >> panic: bogus refcnt 0 on lle 0xfffff800aa941200 > >> >> cpuid =3D 4 > >> >> KDB: stack backtrace: > >> >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > >> >> 0xfffffe2dfe57b1a0 > >> >> vpanic() at vpanic+0x182/frame 0xfffffe2dfe57b220 > >> >> kassert_panic() at kassert_panic+0x126/frame 0xfffffe2dfe57b290 > >> >> llentry_free() at llentry_free+0x136/frame 0xfffffe2dfe57b2c0 > >> >> nd6_cache_lladdr() at nd6_cache_lladdr+0x2ff/frame 0xfffffe2dfe57b= 390 > >> >> nd6_ra_input() at nd6_ra_input+0x1347/frame 0xfffffe2dfe57b670 > >> >> icmp6_input() at icmp6_input+0xa93/frame 0xfffffe2dfe57b810 > >> >> ip6_input() at ip6_input+0xc70/frame 0xfffffe2dfe57b8f0 > >> >> netisr_dispatch_src() at netisr_dispatch_src+0x80/frame > >> >> 0xfffffe2dfe57b950 > >> >> ether_demux() at ether_demux+0x14a/frame 0xfffffe2dfe57b980 > >> >> ether_nh_input() at ether_nh_input+0x340/frame 0xfffffe2dfe57b9c0 > >> >> netisr_dispatch_src() at netisr_dispatch_src+0x80/frame > >> >> 0xfffffe2dfe57ba20 > >> >> ether_input() at ether_input+0x62/frame 0xfffffe2dfe57ba50 > >> >> if_input() at if_input+0xa/frame 0xfffffe2dfe57ba60 > >> >> em_rxeof() at em_rxeof+0x28c/frame 0xfffffe2dfe57bae0 > >> >> em_handle_que() at em_handle_que+0x40/frame 0xfffffe2dfe57bb20 > >> >> taskqueue_run_locked() at taskqueue_run_locked+0x13c/frame > >> >> 0xfffffe2dfe57bb80 > >> >> taskqueue_thread_loop() at taskqueue_thread_loop+0x88/frame > >> >> 0xfffffe2dfe57bbb0 > >> >> fork_exit() at fork_exit+0x84/frame 0xfffffe2dfe57bbf0 > >> >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe2dfe57bbf0 > >> >> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > >> >> Uptime: 22s > >> >> Dumping 3120 out of 64465 > >> >> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > >> >> > >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/zfs.ko > >> >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols > >> >> from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/opensolaris.ko > >> >> Reading symbols from /boot/kernel/linux.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/linux.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/linux.ko > >> >> Reading symbols from /boot/kernel/linux_common.ko...Reading symbol= s > >> >> from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/linux_common.ko > >> >> Reading symbols from /boot/kernel/if_lagg.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/if_lagg.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/if_lagg.ko > >> >> Reading symbols from /boot/kernel/snd_envy24ht.ko...Reading symbol= s > >> >> from /usr/lib/debug//boot/kernel/snd_envy24ht.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/snd_envy24ht.ko > >> >> Reading symbols from /boot/kernel/snd_spicds.ko...Reading symbols = from > >> >> /usr/lib/debug//boot/kernel/snd_spicds.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/snd_spicds.ko > >> >> Reading symbols from /boot/kernel/coretemp.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/coretemp.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/coretemp.ko > >> >> Reading symbols from /boot/kernel/fuse.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/fuse.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fuse.ko > >> >> Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ichsmb.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ichsmb.ko > >> >> Reading symbols from /boot/kernel/smbus.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/smbus.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/smbus.ko > >> >> Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ichwd.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ichwd.ko > >> >> Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/cpuctl.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/cpuctl.ko > >> >> Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols f= rom > >> >> /usr/lib/debug//boot/kernel/cryptodev.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/cryptodev.ko > >> >> Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols f= rom > >> >> /usr/lib/debug//boot/kernel/dtraceall.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtraceall.ko > >> >> Reading symbols from /boot/kernel/profile.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/profile.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/profile.ko > >> >> Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/dtrace.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtrace.ko > >> >> Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading > >> >> symbols from > >> >> /usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/systrace_freebsd32.ko > >> >> Reading symbols from /boot/kernel/systrace.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/systrace.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/systrace.ko > >> >> Reading symbols from /boot/kernel/sdt.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/sdt.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/sdt.ko > >> >> Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/fasttrap.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fasttrap.ko > >> >> Reading symbols from /boot/kernel/fbt.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/fbt.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fbt.ko > >> >> Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/dtnfscl.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtnfscl.ko > >> >> Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/dtmalloc.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtmalloc.ko > >> >> Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ipmi.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ipmi.ko > >> >> Reading symbols from /boot/kernel/ipmi_linux.ko...Reading symbols = from > >> >> /usr/lib/debug//boot/kernel/ipmi_linux.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ipmi_linux.ko > >> >> Reading symbols from /boot/kernel/hwpmc.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/hwpmc.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/hwpmc.ko > >> >> Reading symbols from /boot/kernel/filemon.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/filemon.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/filemon.ko > >> >> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/uhid.ko > >> >> #0 doadump (textdump=3D1) at pcpu.h:221 > >> >> 221 pcpu.h: No such file or directory. > >> >> in pcpu.h > >> >> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 > >> >> #1 0xffffffff80a48945 in kern_reboot (howto=3D) > >> >> at /usr/src/sys/kern/kern_shutdown.c:366 > >> >> #2 0xffffffff80a48f1b in vpanic (fmt=3D, > >> >> ap=3D) at /usr/src/sys/kern/kern_shutdown= .c:759 > >> >> #3 0xffffffff80a48d56 in kassert_panic (fmt=3D) > >> >> at /usr/src/sys/kern/kern_shutdown.c:649 > >> >> #4 0xffffffff80b351d6 in llentry_free (lle=3D) > >> >> at /usr/src/sys/net/if_llatbl.c:432 > >> >> #5 0xffffffff80c5c90f in nd6_cache_lladdr (ifp=3D0xfffff80068d658= 00, > >> >> from=3D, lladdr=3D, > >> >> lladdrlen=3D, type=3D134, code=3D0) > >> >> at /usr/src/sys/netinet6/nd6.c:1972 > >> >> #6 0xffffffff80c62f57 in nd6_ra_input (m=3D, > >> >> off=3D, icmp6len=3D) > >> >> at /usr/src/sys/netinet6/nd6_rtr.c:437 > >> >> #7 0xffffffff80c3ab03 in icmp6_input (mp=3D, > >> >> offp=3D, proto=3D) > >> >> at /usr/src/sys/netinet6/icmp6.c:798 > >> >> #8 0xffffffff80c50e90 in ip6_input (m=3DCannot access memory at a= ddress > >> >> 0x0 > >> >> ) > >> >> at /usr/src/sys/netinet6/ip6_input.c:921 > >> >> #9 0xffffffff80b47470 in netisr_dispatch_src (proto=3D6, source= =3D0, > >> >> m=3D0xfffff80068dfc600) at /usr/src/sys/net/netisr.c:1121 > >> >> #10 0xffffffff80b3209a in ether_demux (ifp=3D= , > >> >> m=3D0x0) > >> >> at /usr/src/sys/net/if_ethersubr.c:850 > >> >> #11 0xffffffff80b32e90 in ether_nh_input (m=3D) > >> >> at /usr/src/sys/net/if_ethersubr.c:639 > >> >> #12 0xffffffff80b47470 in netisr_dispatch_src (proto=3D5, source= =3D0, > >> >> m=3D0xfffff80068dfc600) at /usr/src/sys/net/netisr.c:1121 > >> >> #13 0xffffffff80b32402 in ether_input (ifp=3D= , > >> >> m=3D0x0) > >> >> at /usr/src/sys/net/if_ethersubr.c:759 > >> >> #14 0xffffffff80b2ee8a in if_input (ifp=3D0x0, sendmp=3D0x0) > >> >> at /usr/src/sys/net/if.c:3956 > >> >> #15 0xffffffff80523a6c in em_rxeof (count=3D99) > >> >> at /usr/src/sys/dev/e1000/if_em.c:4872 > >> >> #16 0xffffffff805250b0 in em_handle_que (context=3D0xfffffe1eaa699= 000, > >> >> pending=3D) at > >> >> /usr/src/sys/dev/e1000/if_em.c:1598 > >> >> #17 0xffffffff80a9a23c in taskqueue_run_locked (queue=3D >> >> out>) > >> >> at /usr/src/sys/kern/subr_taskqueue.c:465 > >> >> #18 0xffffffff80a9ad38 in taskqueue_thread_loop (arg=3D >> >> out>) > >> >> at /usr/src/sys/kern/subr_taskqueue.c:719 > >> >> #19 0xffffffff80a0ba54 in fork_exit ( > >> >> callout=3D0xffffffff80a9acb0 , > >> >> arg=3D0xfffffe1eaa69b730, frame=3D0xfffffe2dfe57bc00) > >> >> at /usr/src/sys/kern/kern_fork.c:1038 > >> >> #20 0xffffffff80e943fe in fork_trampoline () > >> >> at /usr/src/sys/amd64/amd64/exception.S:611 > >> >> #21 0x0000000000000000 in ?? () > >> >> Current language: auto; currently minimal > >> >> (kgdb) > >> >> > >> >> vmcore IS available. > >>=20 > >> -- > >> Larry Rosenman http://www.lerctr.org/~ler > >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > >> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to=20 > >> "freebsd-current-unsubscribe@freebsd.org" >=20 > --=20 > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g" From owner-freebsd-net@freebsd.org Fri Jul 15 05:09:51 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8F74B97D2A for ; Fri, 15 Jul 2016 05:09:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCCB01734 for ; Fri, 15 Jul 2016 05:09:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u6F59pnE052681 for ; Fri, 15 Jul 2016 05:09:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 194109] [lor] if_lagg rmlock <-> if_addr_lock Date: Fri, 15 Jul 2016 05:09:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: needs-qa, patch, regression X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: keywords flagtypes.name bug_status bug_file_loc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 05:09:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D194109 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |needs-qa, patch Flags| |mfc-stable11? Status|New |Open URL| |https://reviews.freebsd.org | |/D6845 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Jul 15 05:14:49 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF571B97F3A; Fri, 15 Jul 2016 05:14:49 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C12681BC8; Fri, 15 Jul 2016 05:14:49 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1468559686827214.80335229340017; Thu, 14 Jul 2016 22:14:46 -0700 (PDT) Date: Thu, 14 Jul 2016 22:14:46 -0700 From: Matthew Macy To: "Hans Petter Selasky" Cc: "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" , "Gleb Smirnoff" Message-ID: <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> In-Reply-To: <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> Subject: Re: callout_drain either broken or man page needs updating MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 05:14:49 -0000 ---- On Thu, 14 Jul 2016 21:21:57 -0700 Hans Petter Selasky wrote ---- > On 07/15/16 05:45, Matthew Macy wrote: > > glebius last commit needs some further re-work. > > Hi, > > Glebius commit needs to be backed out, at least the API change that > changes the return value when calling callout_stop() when the callout is > scheduled and being serviced. Simply because there is code out there, > like Mattew and others have discovered that is "refcounting" on the > callout_reset() and expecting that a subsequent callout_stop() will > return 1 to "unref". Yes. This is the cause of the "refcnt 0 on LLE at boot..." regression. -M > > If you consider this impossible, maybe a fourth return value is needed > for CANCELLED and DRAINING . > > Further, getting the callouts straight in the TCP stack is a matter of > doing the locking correctly, which some has called "my magic bullet" and > not the return values. I've proposed in the following revision > https://svnweb.freebsd.org/changeset/base/302768 to add a new callout > API that accepts a locking function so that the callout code can run its > cancelled checks at the right place for situations where more than one > lock is needed. > > Consider this case: > > > void > > tcp_timer_2msl(void *xtp) > > { > > struct tcpcb *tp = xtp; > > struct inpcb *inp; > > CURVNET_SET(tp->t_vnet); > > #ifdef TCPDEBUG > > int ostate; > > > > ostate = tp->t_state; > > #endif > > INP_INFO_RLOCK(&V_tcbinfo); > > inp = tp->t_inpcb; > > KASSERT(inp != NULL, ("%s: tp %p tp->t_inpcb == NULL", __func__, tp)); > > INP_WLOCK(inp); > > tcp_free_sackholes(tp); > > if (callout_pending(&tp->t_timers->tt_2msl) || > > !callout_active(&tp->t_timers->tt_2msl)) { > > Here we have custom in-house race check that doesn't affect the return > value of callout_reset() nor callout_stop(). > > > INP_WUNLOCK(tp->t_inpcb); > > INP_INFO_RUNLOCK(&V_tcbinfo); > > CURVNET_RESTORE(); > > return; > > > I propose the following solution: > > > > > static void > > tcp_timer_2msl_lock(void *xtp, int do_lock) > > { > > struct tcpcb *tp = xtp; > > struct inpcb *inp; > > > > inp = tp->t_inpcb; > > > > if (do_lock) { > > CURVNET_SET(tp->t_vnet); > > INP_INFO_RLOCK(&V_tcbinfo); > > INP_WLOCK(inp); > > } else { > > INP_WUNLOCK(inp); > > INP_INFO_RUNLOCK(&V_tcbinfo); > > CURVNET_RESTORE(); > > } > > } > > > > callout_init_lock_function(&callout, &tcp_timer_2msl_lock, > CALLOUT_RETURNUNLOCKED); > > Then in softclock_call_cc() it will look like this: > > > > > CC_UNLOCK(cc); > > if (c_lock != NULL) { > > if (have locking function) > > tcp_timer_2msl_lock(c_arg, 1); > > else > > class->lc_lock(c_lock, lock_status); > > /* > > * The callout may have been cancelled > > * while we switched locks. > > */ > > Actually "CC_LOCK(cc)" should be in-front of cc_exec_cancel() to avoid > races testing, setting and clearing this variable, like done in hps_head. > > > if (cc_exec_cancel(cc, direct)) { > > if (have locking function) > > tcp_timer_2msl_lock(c_arg, 0); > > else > > class->lc_unlock(c_lock); > > goto skip; > > } > > cc_exec_cancel(cc, direct) = true; > > > > .... > > > > skip: > > if ((c_iflags & CALLOUT_RETURNUNLOCKED) == 0) { > > if (have locking function) > > ... > > else > > class->lc_unlock(c_lock); > > } > > The whole point about this is to make the the cancelled check atomic. > > 1) Lock TCP > 2) Lock CC_LOCK() > 3) change callout state > > --HPS > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-net@freebsd.org Fri Jul 15 08:43:10 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2C93B9A190; Fri, 15 Jul 2016 08:43:10 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (glebi.us [96.95.210.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CC1D01AD8; Fri, 15 Jul 2016 08:43:10 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id u6F8h3fu070160 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 15 Jul 2016 01:43:04 -0700 (PDT) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id u6F8h3UA070159; Fri, 15 Jul 2016 01:43:03 -0700 (PDT) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 15 Jul 2016 01:43:03 -0700 From: Gleb Smirnoff To: Matthew Macy Cc: Hans Petter Selasky , "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" Subject: Re: callout_drain either broken or man page needs updating Message-ID: <20160715084303.GD1076@FreeBSD.org> References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 08:43:11 -0000 On Thu, Jul 14, 2016 at 10:14:46PM -0700, Matthew Macy wrote: M> > On 07/15/16 05:45, Matthew Macy wrote: M> > > glebius last commit needs some further re-work. M> > M> > Glebius commit needs to be backed out, at least the API change that M> > changes the return value when calling callout_stop() when the callout is M> > scheduled and being serviced. Simply because there is code out there, M> > like Mattew and others have discovered that is "refcounting" on the M> > callout_reset() and expecting that a subsequent callout_stop() will M> > return 1 to "unref". M> M> Yes. This is the cause of the "refcnt 0 on LLE at boot..." regression. No it isn't. The regression is caused by unintentional change of return value for never scheduled callout. The fix is now being tested, see PR 210884. The piece of ND6 code that Hans quotes isn't affected by change of return value for scheduled+running callout, since ND6 doesn't create callouts in this tricky state. -- Totus tuus, Glebius. From owner-freebsd-net@freebsd.org Fri Jul 15 13:29:48 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A94FCB98AC5; Fri, 15 Jul 2016 13:29:48 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6BC621B7A; Fri, 15 Jul 2016 13:29:47 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 8FB591FE024; Fri, 15 Jul 2016 15:29:45 +0200 (CEST) Subject: Re: callout_drain either broken or man page needs updating To: Gleb Smirnoff , Matthew Macy References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> <20160715084303.GD1076@FreeBSD.org> Cc: "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" From: Hans Petter Selasky Message-ID: Date: Fri, 15 Jul 2016 15:33:40 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.0 MIME-Version: 1.0 In-Reply-To: <20160715084303.GD1076@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 13:29:48 -0000 On 07/15/16 10:43, Gleb Smirnoff wrote: > On Thu, Jul 14, 2016 at 10:14:46PM -0700, Matthew Macy wrote: > M> > On 07/15/16 05:45, Matthew Macy wrote: > M> > > glebius last commit needs some further re-work. > M> > > M> > Glebius commit needs to be backed out, at least the API change that > M> > changes the return value when calling callout_stop() when the callout is > M> > scheduled and being serviced. Simply because there is code out there, > M> > like Mattew and others have discovered that is "refcounting" on the > M> > callout_reset() and expecting that a subsequent callout_stop() will > M> > return 1 to "unref". > M> > M> Yes. This is the cause of the "refcnt 0 on LLE at boot..." regression. > > No it isn't. The regression is caused by unintentional change of return > value for never scheduled callout. The fix is now being tested, see PR 210884. > > The piece of ND6 code that Hans quotes isn't affected by change of return > value for scheduled+running callout, since ND6 doesn't create callouts in > this tricky state. > Hi, Can you explain this a bit more Gleb? Can't user-space tools like "route" delete LLE entries at _any_ time? From what I can see, there is nothing preventing "nd6_llinfo_settimer_locked()" running concurrently with "nd6_llinfo_timer()". Even though the delay is in the hz range, this doesn't prevent the race I'm pointing at. And what about the pending check in "kern/subr_taskqueue.c"? Won't it be off-by one in case the callout is scheduled when it is being serviced? pending = !!(callout_stop(&timeout_task->c) > 0); How this cannot happen? --HPS From owner-freebsd-net@freebsd.org Fri Jul 15 18:34:52 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C2E0B99D3E; Fri, 15 Jul 2016 18:34:52 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D0301ED1; Fri, 15 Jul 2016 18:34:52 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1468607684344383.69427869629476; Fri, 15 Jul 2016 11:34:44 -0700 (PDT) Date: Fri, 15 Jul 2016 11:34:44 -0700 From: Matthew Macy To: "Gleb Smirnoff" Cc: "Hans Petter Selasky" , "freebsd-current@freebsd.org" , "freebsd-net@freebsd.org" Message-ID: <155efd6dec6.1184b7a2546701.470842708059763905@nextbsd.org> In-Reply-To: <20160715084303.GD1076@FreeBSD.org> References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> <20160715084303.GD1076@FreeBSD.org> Subject: Re: callout_drain either broken or man page needs updating MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 18:34:52 -0000 ---- On Fri, 15 Jul 2016 01:43:03 -0700 Gleb Smirnoff wrote ---- > On Thu, Jul 14, 2016 at 10:14:46PM -0700, Matthew Macy wrote: > M> > On 07/15/16 05:45, Matthew Macy wrote: > M> > > glebius last commit needs some further re-work. > M> > > M> > Glebius commit needs to be backed out, at least the API change that > M> > changes the return value when calling callout_stop() when the callout is > M> > scheduled and being serviced. Simply because there is code out there, > M> > like Mattew and others have discovered that is "refcounting" on the > M> > callout_reset() and expecting that a subsequent callout_stop() will > M> > return 1 to "unref". > M> > M> Yes. This is the cause of the "refcnt 0 on LLE at boot..." regression. I misread his comment on the reason for the failure. But, the failure is caused by a regression in callout_stop. > No it isn't. The regression is caused by unintentional change of return > value for never scheduled callout. The fix is now being tested, see PR 210884. Thanks. Let me know when I can update. -M From owner-freebsd-net@freebsd.org Fri Jul 15 18:38:54 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE61BB99FF0; Fri, 15 Jul 2016 18:38:54 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E62AF1596; Fri, 15 Jul 2016 18:38:53 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=7FS+t8K3bl7Ag5b5C1H/SfbBjM2qksyrPNgjyeFJqt0=; b=b1k1s3a3ZACqHAxTOE/kfupedf AZ2aKwh2kLEBeYNJVrG6Rii6RKQR2/iB3sF4fEL4PwgGWBnOEYsPzz3kjLJHgdwxyBdF3H4E88st8 2JXbeK0euLHrnWM3rwl1iu7pWjULq7DWhz9YR7rld0CdS2H5OIhoOrpOGd9wyPgPW2Ho=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:61910 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bO80i-00068z-La; Fri, 15 Jul 2016 13:38:52 -0500 Received: from proxy.na.alcatel-lucent.com ([135.245.48.73]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 15 Jul 2016 13:38:52 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 15 Jul 2016 13:38:52 -0500 From: Larry Rosenman To: Matthew Macy Cc: Gleb Smirnoff , Hans Petter Selasky , freebsd-current@freebsd.org, freebsd-net@freebsd.org, owner-freebsd-current@freebsd.org Subject: Re: callout_drain either broken or man page needs updating In-Reply-To: <155efd6dec6.1184b7a2546701.470842708059763905@nextbsd.org> References: <155eca8bae0.d811ff9b567670.7363072028299444677@nextbsd.org> <087a20a1-2b8b-2050-c75f-78aac964b457@selasky.org> <155ecfa7c59.b552d7c5570767.4742594321655958557@nextbsd.org> <20160715084303.GD1076@FreeBSD.org> <155efd6dec6.1184b7a2546701.470842708059763905@nextbsd.org> Message-ID: <385925d91a2967587dacd2fe29d5a075@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 18:38:55 -0000 On 2016-07-15 13:34, Matthew Macy wrote: > ---- On Fri, 15 Jul 2016 01:43:03 -0700 Gleb Smirnoff > wrote ---- > > On Thu, Jul 14, 2016 at 10:14:46PM -0700, Matthew Macy wrote: > > M> > On 07/15/16 05:45, Matthew Macy wrote: > > M> > > glebius last commit needs some further re-work. > > M> > > > M> > Glebius commit needs to be backed out, at least the API change > that > > M> > changes the return value when calling callout_stop() when the > callout is > > M> > scheduled and being serviced. Simply because there is code > out there, > > M> > like Mattew and others have discovered that is "refcounting" > on the > > M> > callout_reset() and expecting that a subsequent callout_stop() > will > > M> > return 1 to "unref". > > M> > > M> Yes. This is the cause of the "refcnt 0 on LLE at boot..." > regression. > > I misread his comment on the reason for the failure. But, the failure > is caused by a regression in callout_stop. > > > No it isn't. The regression is caused by unintentional change of > return > > value for never scheduled callout. The fix is now being tested, see > PR 210884. > > Thanks. Let me know when I can update. > -M > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" URL: https://svnweb.freebsd.org/changeset/base/302894 has the fix in HEAD. (It's a one-liner). -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281