From owner-freebsd-pf@freebsd.org Wed Feb 14 12:47:08 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16CA2F1F83F for ; Wed, 14 Feb 2018 12:47:08 +0000 (UTC) (envelope-from Joe@stream-technologies.com) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0066.outbound.protection.outlook.com [104.47.1.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 319BA75659 for ; Wed, 14 Feb 2018 12:47:06 +0000 (UTC) (envelope-from Joe@stream-technologies.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=streamtechnologiesuk.onmicrosoft.com; s=selector1-streamtechnologies-com01e; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=h8ayCqsKVtP7w+jFGo8FDyCrYnL9qWRd6dAC+CDf3k8=; b=esLH1GTKvRxuu6c3RKDfrH1OaMulnZlq9CCBx2L9g/F3A0qy2XNP0V0+OJfOPS9qNIeJV0ATg5UbIce/BXJCu6q09YYDOKwgsdJWzHnZlBxfetQmejgh4OSfca/BKsmaI/zgZVN5R0O+5ppzyzAujmkOvtVbwNRnRYLdc1yAkiQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Joe@stream-technologies.com; Received: from [192.168.6.128] (212.250.79.109) by AM4PR07MB3411.eurprd07.prod.outlook.com (2603:10a6:205:a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.506.7; Wed, 14 Feb 2018 12:47:04 +0000 To: freebsd-pf@freebsd.org From: Joe Jones Subject: Kernel Panic Message-ID: <5A842FC6.7020806@stream-technologies.com> Date: Wed, 14 Feb 2018 12:47:02 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [212.250.79.109] X-ClientProxiedBy: LNXP265CA0066.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5d::30) To AM4PR07MB3411.eurprd07.prod.outlook.com (2603:10a6:205:a::32) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f0aea2c7-b0a7-419f-050f-08d573a90cce X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:AM4PR07MB3411; X-Microsoft-Exchange-Diagnostics: 1; AM4PR07MB3411; 3:BtHpk/TFC/uuTZs+84z3yCFP9w7tUBEaKfmSCPL9Ez9dXJ48s2FuKryR3z11FlxVhDR4LdGFI63udnJE9pnG3gGlwhNZv+0EGxyved6ndcFHLc1/bwGA9uD163A7zBlaxWdZjmWBM9c9i+gM5l1JbSc7TL0wrsB8pVhDZY8g6MXqqJkJsHBo46tyjYPWvaIWG1vODlLQ1nz8pNe0tDHO98ipTLmNJz53ReI4yI+BdIxyZ8i8zLtaDZ0Raj+yWcOX; 25:HiW4yeYi1u0NxslxZHCRhwz8G+VlZ2TF4FEWtaJIifc/JcE8ARGsrShZQTXRisCKiORwm9ca4BjuCPimNBcn/v0bxQ+iAI3Szc0B8oJr1zds/i1PlLmAjGv861OLQkqt1P1n1/P3H8BRgWNNoY/2sC8yo6W0aY1dafwCJE5oX6Jwq69S2lM/lsn055zB/32aBjPtk0y5tq5BD8Wdm5BZdVTsTlEtvju1vSxOZYYDaAnBeH6pWVHc/Q+5osRYKcCYMwksS3kVfNAYpTNYz9Kf1U3lRZm2HIqyixTM4ukuKkXPrpqnJomSfKNeHdSCspSM8i9inPo35+pCsMbnfqiAoA==; 31:dKUNemtaqYUxukpoRykrDeEIfFIPirQ+MsRlvYOd3sOFGYkLKeop5oqQvKzf2YlV6+eOKfGnFqK1GmvKZzEbZvxXSHWrwV3XOBsfiTmjAMlG4G3lE0KgeExCg6OX+KkX7YNrU+7ne0LXOROKZCeTQvNghaP2JE3dEr6q5scmYw/YtJ1XqKY2bDrJw2wtnnyUnjdonUljSR9jWIPewI6aSnAwNWfds0+HMvjNaj+ufCI= X-MS-TrafficTypeDiagnostic: AM4PR07MB3411: X-Microsoft-Exchange-Diagnostics: 1; AM4PR07MB3411; 20:781rHomiC88a9p66xgG5+KvwDt/3Ddav4nstd994hyDpOHWzewVYM8yj0rw7fkdoFqlFzyZdcK2HuXi8z/Re+Q8vlNL6ml92uRoLx1kobhgpVeTp8jGDqrFlyq1ounBsWiP2nLGhOTgxR13DPRKkc4t1KzrHcH0diL198Tj9wQk=; 4:zoWEJkRvDXZ4Pz5KQmxBLaoOoz45oVB1FwFDQyUlpY7CrYlvneqn+mZbP5GZNXiyTTzAljy6Qr/XbS+ZjBKbRYYHDJ1Gzn8KT8APcQPFn+pTa21sz8VXAyraRsB25IsJ2vZbUPGYA9eEjxzQqw964Wtg3xnV7VNK+oW45MwUx2kLUBGly9SojZBuXuYOuJx6ZTt4jg6o3xi2jf9jdlYZ35eYK1831O6271NPu3v3bZG+ehI/W25BTrA5LiYOOSe6Gwo7RFc+Yqt4iatDHTqKUQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(8121501046)(5005006)(3231101)(2400082)(944501161)(10201501046)(3002001)(93006095)(93001095)(6041288)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(6072148)(201708071742011); SRVR:AM4PR07MB3411; BCL:0; PCL:0; RULEID:; SRVR:AM4PR07MB3411; X-Forefront-PRVS: 0583A86C08 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(376002)(346002)(39380400002)(39840400004)(366004)(396003)(199004)(189003)(33656002)(6486002)(65806001)(47776003)(65956001)(72206003)(66066001)(221733001)(53936002)(3480700004)(478600001)(64126003)(230700001)(2361001)(8936002)(81166006)(117156002)(81156014)(50466002)(25786009)(186003)(77096007)(26005)(2351001)(8676002)(106356001)(65816011)(59450400001)(86362001)(386003)(5660300001)(105586002)(575784001)(16526019)(87266011)(83506002)(16576012)(80792005)(58126008)(97736004)(316002)(305945005)(7736002)(68736007)(2906002)(36756003)(52116002)(59896002)(23676004)(2486003)(67846002)(80316001)(52146003)(6916009)(7116003)(3846002)(6116002)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR07MB3411; H:[192.168.6.128]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: stream-technologies.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtBTTRQUjA3TUIzNDExOzIzOnp3SFNiN1M2Tlo2QTJMUU8yS2hubFk4enJB?= =?utf-8?B?WnZ4S1NPY2w1dFJWMDdic0lncTg3Sy9WYUViMVFHcmtwb0ZRVjI1cGVpZ3NY?= =?utf-8?B?cDRQT2xUaHdjaDlMWllSK2tPUXVIbnE3K3lKbkRTTnVGYys1cUdBYytpVllD?= =?utf-8?B?elJ5RnBkSE9DSnpOb0VBeU5LNVkxbUNZclMrTWJHOVJRUUgrdmVKOXQwUWM1?= =?utf-8?B?QTVnMU0yZW1hRXBPUnp5ZzA2YTg5WEJ2SmpVb2g2WDVNY2ZXMytkbkdkd1VQ?= =?utf-8?B?eXprRUwxZUlNNHphdnFUUU5GUHM0TXhuOHgzeDVhbE5XeVlCcmRNeUc4UG1X?= =?utf-8?B?S1JkRnNpbCs4OE1SaHMxSERUQlBYemw4YVhXZ0dKTlJVSzRjWmlmR1I4TWlu?= =?utf-8?B?Nm9pNXI1b2NocU5lUlZzblhzTCttbmlySDFSdU5LaTR0d2lGSkZVb2NHbytG?= =?utf-8?B?WW1TeFd6QzBoSlp5VXdwaFFXSFRmOVdnS1EyeGJQSy9QelpEcW82UDhpbmFK?= =?utf-8?B?eFQwSXJPWGdET0IrNEd0aDdtZHdlRklpaGhmTnoyRE4vamN3bzd1N21OeFQ2?= =?utf-8?B?UklQWmErMXgxQ2FBbzBsWmZIYm5FQkJwSmU0am1lOGMzMTJmNDJRd0JTWGdQ?= =?utf-8?B?YXZWdk5RVzBwVkxua1puOER4ZjQyWnRLMms5NDBMRmQ2RVJwdlBKYVZ6SW5I?= =?utf-8?B?YnNaODRGUWhvYXNEenl5RDdXSkxaY3pDcjRYQmtCVVN6VSsvSXNoVzRrcUJN?= =?utf-8?B?Z3FnWXVXUXFLdzRRenNJbUhoR2JQUGMrdEtXL1NWd2VLZXJJRnRyQjFiY041?= =?utf-8?B?ZVhmZ2p3Y3JGdVU4Tk9JMmF2Um90MmRkcnJCS0phNHJqV3REdTdoQXY4MUxZ?= =?utf-8?B?T2dMdWZYM0Q0cjZZTXM0SHpQaFB2UXlTSnNPbjFQalc3SDJnZ05tRnlvNXoz?= =?utf-8?B?VElkVi9QRWVvOHh1MmpuUWgyemtWUkdYTjVodUJMY3JEZjFJWWRYNG05M2NF?= =?utf-8?B?N2ZHRlU5VHZFcXZkK3BNeGY0azVkWVNKOFlYdSt1R3R0UDY4d1ErK2FHc1NX?= =?utf-8?B?ckdwOUJHR3FKSkZ3eno1N2VhY0p3THpaYW13a1NVbVRhZm41OFlock14K0Vo?= =?utf-8?B?bGlxL3NDK3RpRmhzbDdjU0pnR0xYWThublJ6MmhQL3p4elZrbjdsWUlWckZM?= =?utf-8?B?S205UXlDV0VKeEd1NjVHYXVpcWxnblNMR3VDK2pxOGxWNVc5emtVaDk1Nkto?= =?utf-8?B?K25HSDFhVitZRkdZQUY2Y3dJVWpEMWUzZFhzbEdUYlQ3RnJvbklwTEp1a2NX?= =?utf-8?B?aTNQMVBLQWEyVnFvTktOeWQwRzl3eU1JMXVpby9HRk1HNWlPUTZNL1dibXQx?= =?utf-8?B?K3RKMGZkNTQxVWsxaE9zQUZVb3lLVVp0NVFxMExOMVl3dmlybHlNT054NGky?= =?utf-8?B?YzZkSFllMktFQ0g2a2FaWWZrNHpkOHhJWjFJV3cvaUlMTDZzYm5tZkVMbFov?= =?utf-8?B?WUdZOVBVeHZVMDAyNWkyMDhBbEFOZDM4VGQxY244dURXRk8xWE10SjZiN1NE?= =?utf-8?B?UjBEek1LdXBDdkE0NXY4UkxjcWZzY25zem82YWc1djU3OVhTVkFybnZnNzhI?= =?utf-8?B?NlFoNGE1ckovM2hpS1lpTy9Ya042dGdtbEFSL0licVNpVzNTV1R0dmI2OFZK?= =?utf-8?B?K0hWUG1TSUNrTUlwTUJRUHY5QUJBWENGanBGaVA5WFV2Q1k1WEFpZXJTZlZH?= =?utf-8?B?Zm9ZbHE3enVYdGIrcTE4Y2hYZWlBbDRHVmR2cERmdzJkYi9pNEMvT0kweU9m?= =?utf-8?B?Z2NDdXNOL2xTVTg3a3UxVzNDVUNSRG1maWplWk5MUFN1MmV2ZXNWYW4wS2E5?= =?utf-8?B?blhXZmZuU3AwckxOdnIyL3M1RkRBRTRqeHhISUV5aFZ0VHVQcXN6TGtJNmsz?= =?utf-8?B?Vmw3SjlraERiLzlqKzhnamFIcHpVdDlvVS9VRk9kWkpXTUFMTVl2d3JtK2pp?= =?utf-8?B?Z2RTZnVnM0N2WDFmV2o0dGdiS2Z6UnZ2QXk3TVNiNUJHMzFaUVFaWFVYOVBF?= =?utf-8?Q?mBpo=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM4PR07MB3411; 6:xhxyBheXvqbPy1eyjfQJ/3RNNkTl8mbFyladzkn+hsusV905oly2nm6sMbP+CJJABB+Tks0I53ZcGSxWUqPgi9huEKrSu8WPbiX17WhdQIA73irLSy7bOmTMeTPq8q67xdMBKDP5E0G3DMFsUbj0NXenyRRIGg/gIaUL8mZaaf1i3TAPfT2c1yP/+m9IFlOgkauIcoUXO1002gXKPbcaPFnMoJSPGxClNmaElufU6dVCORaNgGhEOj5zGU5+f2MihFCHP3dB05Lr7J19Rl3EN+yJ8vis0fpPlTW10gMGi0de6fyfjh3jnx860zTfRWkNtd6TcIiQ9Tyr7VeaquX76VYEqoyyNdLv4/6257rool4=; 5:TkRFLf8R5LIPeaG58+0nqx4A2FHNLQWLaecI6XPMdmfgIGZJMpCF8kdduoGqZYI9vmSxkFTVypQkutd+mvnm0y5yQIxIVgQZHWbhcZuxCXACM5i++xdatNjeEDS7Qjjf/roUbSUBlfnoXmtnLZFSMbj8SPf0eIHgo5+iFrIyZlI=; 24:mmp6cXV+hWw0p2zUJDgjf8Gc9NozLwJqpHB08up5deWp9RAXvMimXKSC5o8ulvcw3JVDl+GkI8AkC0cuiMCrskyxx7rMuldN1aYkzU9AFd4=; 7:yVC8BO14lt4Es6n6ENyOhKqI0KY+csQf2GqkkfuCSVPlsBIhQIL6zOvoMyhNB+6z32QMesfGYIxGntgZV3UuoY90mXpoZIAgspel7FnnxNkZo9GQIMGDtwHJGkjNlSvSdRQah1joAlhKJceJ/n9KyhBzspYmE3HVlHIDrzu05WCxn3vL07Xjw0Q2nBmb57/1Wx+1xbQhSXp12W5eXQbCdpQ6ULsTqAovSq/m4Lg6WLs23KjS746mS7yCGnYWssd2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: stream-technologies.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2018 12:47:04.3558 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f0aea2c7-b0a7-419f-050f-08d573a90cce X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 720fa073-5781-43bf-bc14-7bef2603ed21 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR07MB3411 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Feb 2018 12:47:08 -0000 Hi, we are running test traffic through our system, after between 1 and 12 hours we get a kernel panic, always in the pfr_pool_get function in /usr/src/sys/netpfil/pf/pf_table.c line 2140. After a bit of investigation I confirmed that ke2 is set to null on line 2122. We have tried a kernel compiled from /base/releng/11.1 and the equivalent branch of pfsense and get the same thing. The network card is a 4 port Intel I350 (igb). The test uses 2 of the 4 ports, msix with 2 queues per port. net.link.ifqmaxlen="2048" hw.igb.rx_process_limit="-1" hw.igb.num_queues="2" Next I'm going to start stripping parts out of our setup to try and get to a system that is stable under the test load. Can anyone shed any light on what is happening here. Thanks Joe Jones GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: current process = 12 (irq272: igb1:que 1) trap number = 12 panic: page fault cpuid = 3 KDB: stack backtrace: #0 0xffffffff80aadb97 at kdb_backtrace+0x67 #1 0xffffffff80a6bbf6 at vpanic+0x186 #2 0xffffffff80a6ba63 at panic+0x43 #3 0xffffffff80ee0832 at trap_fatal+0x322 #4 0xffffffff80ee0889 at trap_pfault+0x49 #5 0xffffffff80ee00c6 at trap+0x286 #6 0xffffffff80ec4d41 at calltrap+0x8 #7 0xffffffff824c65df at pf_map_addr+0x58f #8 0xffffffff824c6a11 at pf_get_translation+0x1e1 #9 0xffffffff824bade1 at pf_test_rule+0x241 #10 0xffffffff824b78bb at pf_test+0x174b #11 0xffffffff824c961d at pf_check_out+0x1d #12 0xffffffff80b71f4b at pfil_run_hooks+0x7b #13 0xffffffff80bcdb71 at ip_tryforward+0x1f1 #14 0xffffffff80bcfb88 at ip_input+0x368 #15 0xffffffff80b71140 at netisr_dispatch_src+0xa0 #16 0xffffffff80b5ac0d at ether_demux+0x16d #17 0xffffffff80b5b89b at ether_nh_input+0x31b Uptime: 4h17m34s Dumping 2778 out of 65374 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/carp.ko...Reading symbols from /usr/lib/debug//boot/kernel/carp.ko.debug...done. done. Loaded symbols for /boot/kernel/carp.ko Reading symbols from /boot/kernel/pfsync.ko...Reading symbols from /usr/lib/debug//boot/kernel/pfsync.ko.debug...done. done. Loaded symbols for /boot/kernel/pfsync.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /usr/lib/debug//boot/kernel/pf.ko.debug...done. done. Loaded symbols for /boot/kernel/pf.ko Reading symbols from /boot/modules/mellynat.ko...done. Loaded symbols for /boot/modules/mellynat.ko Reading symbols from /boot/modules/mellycount.ko...done. Loaded symbols for /boot/modules/mellycount.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /usr/lib/debug//boot/kernel/pflog.ko.debug...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko #0 doadump (textdump=) at pcpu.h:222 222 pcpu.h: No such file or directory. in pcpu.h (kgdb) backtrace #0 doadump (textdump=) at pcpu.h:222 #1 0xffffffff80a6b771 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80a6bc30 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff80a6ba63 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0xffffffff80ee0832 in trap_fatal (frame=0xfffffe000031ac40, eva=153) at /usr/src/sys/amd64/amd64/trap.c:801 #5 0xffffffff80ee0889 in trap_pfault (frame=0xfffffe000031ac40, usermode=0) at pcpu.h:222 #6 0xffffffff80ee00c6 in trap (frame=0xfffffe000031ac40) at /usr/src/sys/amd64/amd64/trap.c:421 #7 0xffffffff80ec4d41 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff824dba80 in pfr_pool_get (kt=0xfffff80010e17000, pidx=, counter=0xfffff8000e925298, af=) at /usr/src/sys/netpfil/pf/pf_table.c:2141 #9 0xffffffff824c65df in pf_map_addr (af=, r=, saddr=, naddr=, init_addr=, sn=) at uma.h:363 #10 0xffffffff824c6a11 in pf_get_translation (pd=, m=, off=, direction=, kif=, sn=0xfffffe000031b248, skp=, nkp=, saddr=, daddr=, sport=, dport=) at /usr/src/sys/netpfil/pf/pf_lb.c:223 #11 0xffffffff824bade1 in pf_test_rule () at /usr/src/sys/netpfil/pf/pf.c:3310 #12 0xffffffff824b78bb in pf_test (dir=, ifp=, m0=, inp=) at /usr/src/sys/netpfil/pf/pf.c:6280 #13 0xffffffff824c961d in pf_check_out (arg=, m=0xfffffe000031b6a0, ifp=, dir=, inp=) at /usr/src/sys/netpfil/pf/pf_ioctl.c:3627 #14 0xffffffff80b71f4b in pfil_run_hooks (ph=0xffffffff81dca688, mp=0xfffffe000031b700, ifp=0xfffff8000e7fe000, dir=2, inp=0x0) at /usr/src/sys/net/pfil.c:109 #15 0xffffffff80bcdb71 in ip_tryforward (m=0xfffff805f1da2800) at /usr/src/sys/netinet/ip_fastfwd.c:306 #16 0xffffffff80bcfb88 in ip_input (m=0xfffff805f1da2800) at /usr/src/sys/netinet/ip_input.c:573 #17 0xffffffff80b71140 in netisr_dispatch_src (proto=1, source=, m=) at /usr/src/sys/net/netisr.c:1120 #18 0xffffffff80b5ac0d in ether_demux (ifp=0xfffff8000e2f8800, m=) at /usr/src/sys/net/if_ethersubr.c:860 #19 0xffffffff80b5b89b in ether_nh_input (m=) at /usr/src/sys/net/if_ethersubr.c:641 #20 0xffffffff80b71140 in netisr_dispatch_src (proto=5, source=, m=) at /usr/src/sys/net/netisr.c:1120 #21 0xffffffff80b5aea6 in ether_input (ifp=, m=0x0) at /usr/src/sys/net/if_ethersubr.c:764 #22 0xffffffff80541558 in igb_rxeof (count=-29544262) at /usr/src/sys/dev/e1000/if_igb.c:4957 #23 0xffffffff8054091f in igb_msix_que (arg=0xfffff8000e6b2c68) at /usr/src/sys/dev/e1000/if_igb.c:1612 #24 0xffffffff80a3223c in intr_event_execute_handlers (p=, ie=0xfffff8000e6b2e00) at /usr/src/sys/kern/kern_intr.c:1262 #25 0xffffffff80a32526 in ithread_loop (arg=0xfffff8000e7881c0) at /usr/src/sys/kern/kern_intr.c:1275 #26 0xffffffff80a2f895 in fork_exit (callout=0xffffffff80a32450 , arg=0xfffff8000e7881c0, frame=0xfffffe000031bac0) at /usr/src/sys/kern/kern_fork.c:1042 #27 0xffffffff80ec527e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #28 0x0000000000000000 in ?? () Current language: auto; currently minimal