From owner-freebsd-net@FreeBSD.ORG Mon Jul 11 12:12:27 2011 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DFBF21065670 for ; Mon, 11 Jul 2011 12:12:26 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 780608FC12 for ; Mon, 11 Jul 2011 12:12:26 +0000 (UTC) Received: by bwa20 with SMTP id 20so4308354bwa.13 for ; Mon, 11 Jul 2011 05:12:25 -0700 (PDT) Received: by 10.204.42.68 with SMTP id r4mr2742736bke.28.1310384755518; Mon, 11 Jul 2011 04:45:55 -0700 (PDT) Received: from [192.168.10.3] ([82.76.253.74]) by mx.google.com with ESMTPS id af13sm9915305bkc.7.2011.07.11.04.45.53 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 11 Jul 2011 04:45:54 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Vlad Galu In-Reply-To: <4E1AE1AD.4020907@rdtc.ru> Date: Mon, 11 Jul 2011 13:45:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <833EECF8-245B-4085-B853-AC753DBE0D19@dudu.ro> References: <4E1AE1AD.4020907@rdtc.ru> To: Eugene Grosbein X-Mailer: Apple Mail (2.1084) Cc: "net@freebsd.org" Subject: Re: Repeating kernel panic within dummynet X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Jul 2011 12:12:27 -0000 On Jul 11, 2011, at 1:42 PM, Eugene Grosbein wrote: > Hi! >=20 > My FreeBSD 8.2/amd64 routers use dummynet heavily > and keep panic with the *same* KDB backtrace: >=20 > dummynet: bad switch -256! >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0x0 > fault code =3D supervisor read instruction, page not = present > instruction pointer =3D 0x20:0x0 > stack pointer =3D 0x28:0xffffff81229d9a10 > frame pointer =3D 0x28:0xffffff81229d9a40 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 0 (dummynet) > trap number =3D 12 > panic: page fault > cpuid =3D 0 > KDB: stack backtrace: > db_trace_self_wrapper() at 0xffffffff801aaaca =3D = db_trace_self_wrapper+0x2a > kdb_backtrace() at 0xffffffff80329667 =3D kdb_backtrace+0x37 > panic() at 0xffffffff802f6cb7 =3D panic+0x187 > trap_fatal() at 0xffffffff804d8b50 =3D trap_fatal+0x290 > trap_pfault() at 0xffffffff804d8f2f =3D trap_pfault+0x28f > trap() at 0xffffffff804d940f =3D trap+0x3df > calltrap() at 0xffffffff804c0b44 =3D calltrap+0x8 > --- trap 0xc, rip =3D 0, rsp =3D 0xffffff81229d9a10, rbp =3D = 0xffffff81229d9a40 --- > uart_z8530_class() at 0 > mb_dtor_pack() at 0xffffffff802e4787 =3D mb_dtor_pack+0x37 > uma_zfree_arg() at 0xffffffff8049ba5a =3D uma_zfree_arg+0x3a > m_freem() at 0xffffffff803556a7 =3D m_freem+0x37 > dummynet_send() at 0xffffffff803e909d =3D dummynet_send+0x2d > dummynet_task() at 0xffffffff803e93c6 =3D dummynet_task+0x1c6 > taskqueue_run_locked() at 0xffffffff80335a65 =3D = taskqueue_run_locked+0x85 > taskqueue_thread_loop() at 0xffffffff80335bfe =3D = taskqueue_thread_loop+0x4e > fork_exit() at 0xffffffff802ca4bf =3D fork_exit+0x11f > fork_trampoline() at 0xffffffff804c108e =3D fork_trampoline+0xe > --- trap 0, rip =3D 0, rsp =3D 0xffffff81229d9d00, rbp =3D 0 --- > Uptime: 2d5h17m39s > Dumping 4087 MB (4 chunks) > chunk 0: 1MB (150 pages) ... ok > chunk 1: 3575MB (915072 pages) 3559 3543 3527 3511 3495 3479 >=20 >=20 > It does not finish writing dump and hangs until IPMI watchdog reboots = the box. > I've tried to use debug.minidump=3D1 but it still hangs while = crashdumps is generating > and stops responding to Ctrl-Alt-ESC meantime. >=20 > Sadly, I cannot add options INVARIANTS to the kernel because it makes = my mpd-based > routers to panic very often (every 2-3 hours) due to famous 'dangling = pointer' > problem - PPPoE user disconnects, its ngXXX interface got removed, = then its traffic > goes out various system queues (netisr, dummynet etc.) and another = kind of panic > occurs due to INVARIANTS' references to non-existent ifp. Hi Eugene, If your ISR threads aren't already bound to CPUs, you can bind them and = try using INVARIANTS.