From owner-freebsd-net@freebsd.org Fri Jan 12 22:19:41 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA9BFE7FEAD for ; Fri, 12 Jan 2018 22:19:41 +0000 (UTC) (envelope-from 01000160ec74ed80-b5c71214-1727-4769-ab34-4fd4fe118f5a-000000@amazonses.com) Received: from a8-60.smtp-out.amazonses.com (a8-60.smtp-out.amazonses.com [54.240.8.60]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6F10C83D4B for ; Fri, 12 Jan 2018 22:19:40 +0000 (UTC) (envelope-from 01000160ec74ed80-b5c71214-1727-4769-ab34-4fd4fe118f5a-000000@amazonses.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=ae7m2yrxjw65l2cqdpjxuucyrvy564tn; d=tarsnap.com; t=1515795574; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding; bh=d2FriLlhqd5YfajjPU09n+miRpyovmpCRGMWKiyIhbA=; b=Y6Q02BS7I/sOPs6aaqro1U9nQmWU+4SCGrjCHBypKxYRCsT6zZxn5NMWH3aDcKHY LsDG+UaFrF5I/Isrd5/lJbQo+7PG7vg0OhRfOsrTgqS4kh0rtKFpaqvYrSBPpcQ/ht5 iciW7oTnv9YtbRkrdJX6FHl4PtE/TLlxdilT8L94= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=224i4yxa5dv7c2xz3womw6peuasteono; d=amazonses.com; t=1515795574; h=Subject:To:References:Cc:From:Message-ID:Date:MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:Feedback-ID; bh=d2FriLlhqd5YfajjPU09n+miRpyovmpCRGMWKiyIhbA=; b=Ec9gr+CwZZGlXituzaiJ2VW2ScLySTpb5MrXBY1J01a5E1fIPjDsPulWasdE5mmk 2p1O6gXJzNB9EfRlkhG8nc9VFJG30AGu5RX0aN/VvZ1HtdDKn1cmQFC5e4dLzPJ92mX 7uC+ESA0ZrlaaxspbDULw++b6WM6Q4M/m1+FQXz0= Subject: Re: Panic with recent -CURRENT kernel in EC2 To: "Pieper, Jeffrey E" , "Bradley T. Hughes" , "freebsd-cloud@freebsd.org" References: <01000160e9f7ae62-92937548-d46c-4687-87c0-2205119d5132-000000@email.amazonses.com> <191060CB-F1F8-4E34-8191-99EB7B28597C@intel.com> Cc: "freebsd-net@freebsd.org" , Eric Joyner From: Colin Percival Message-ID: <01000160ec74ed80-b5c71214-1727-4769-ab34-4fd4fe118f5a-000000@email.amazonses.com> Date: Fri, 12 Jan 2018 22:19:34 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <191060CB-F1F8-4E34-8191-99EB7B28597C@intel.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-SES-Outgoing: 2018.01.12-54.240.8.60 Feedback-ID: 1.us-east-1.Lv9FVjaNvvR5llaqfLoOVbo2VxOELl7cjN0AOyXnPlk=:AmazonSES X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jan 2018 22:19:41 -0000 On 01/12/18 08:16, Pieper, Jeffrey E wrote: > Hrm… > >> taskqgroup_attach_cpu: setaffinity failed: 22 >> taskqgroup_attach_cpu: setaffinity failed: 22 > > Is this a fresh snapshot? That should have been fixed in > https://svnweb.freebsd.org/base?view=revision&revision=327013 . Can you > please file a bug on this? This looks like an iflib issue. I can confirm that this happens with the 12.0-CURRENT r327524 snasphot (on a c4.8xlarge EC2 instance, in case it matters). Colin Percival > On 1/12/18, 2:43 AM, "owner-freebsd-net@freebsd.org on behalf of Colin Percival" wrote: > > This sounds like it may be related to r327031, which make ixgbe (and thus ixv) > use iflib... maybe erj@ will have some ideas here? > > In case it helps, dhclient sets the MTU, which in the past has caused > problems due to drivers resetting -- the fact that sys_ioctl is ending > up in iflib_init_locked makes me wonder if this is a similar issue. (If > it is, there's a locking problem somewhere...) > > Colin Percival > > On 01/12/18 01:23, Bradley T. Hughes wrote: > > (Cross-posting to -net, but I'm not subscribed there... yet) > > > > I was wondering if anyone else has experienced this panic when dhclient > tries to get an IP address after booting a recent 12.0-CURRENT kernel? Using > the 12.0-CURRENT AMI from early November works, but with the latest snapshot > or when using a kernel built from source I get this: > > > > ixv0: mem 0xf3000000-0xf3003fff,0xf3004000-0xf3007fff at device 3.0 on pci0 > > ixv0: using 2048 tx descriptors and 2048 rx descriptors > > ixv0: msix_init qsets capped at 1 > > ixv0: pxm cpus: 2 queue msgs: 2 admincnt: 1 > > ixv0: using 1 rx queues 1 tx queues > > ixv0: Using MSIX interrupts with 2 vectors > > ixv0: allocated for 1 queues > > ixv0: allocated for 1 rx queues > > taskqgroup_attach_cpu: setaffinity failed: 22 > > taskqgroup_attach_cpu: setaffinity failed: 22 > > ixv0: Ethernet address: 0a:ad:68:70:ea:78 > > ixv0: netmap queues/slots: TX 1/2048, RX 1/2048 > > ... > > panic: Assertion if_getdrvflags(ifp) == i failed at /usr/src/sys/net/iflib.c:2199 > > cpuid = 3 > > time = 1515673176 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe008a8ee600 > > vpanic() at vpanic+0x19c/frame 0xfffffe008a8ee680 > > kassert_panic() at kassert_panic+0x126/frame 0xfffffe008a8ee6f0 > > iflib_init_locked() at iflib_init_locked+0x75f/frame 0xfffffe008a8ee750 > > iflib_if_ioctl() at iflib_if_ioctl+0x744/frame 0xfffffe008a8ee7b0 > > ifioctl() at ifioctl+0x1903/frame 0xfffffe008a8ee850 > > kern_ioctl() at kern_ioctl+0x2c4/frame 0xfffffe008a8ee8b0 > > sys_ioctl() at sys_ioctl+0x15c/frame 0xfffffe008a8ee980 > > amd64_syscall() at amd64_syscall+0x79b/frame 0xfffffe008a8eeab0 > > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe008a8eeab0 > > --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x80100a4fa, rsp = 0x7fffffffe408, rbp = 0x7fffffffe460 --- > > > > Thanks in advance :) -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid