From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 1 20:33:43 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94CB41065670 for ; Mon, 1 Jun 2009 20:33:43 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id 1AAF48FC17 for ; Mon, 1 Jun 2009 20:33:42 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: by fg-out-1718.google.com with SMTP id e12so558589fga.12 for ; Mon, 01 Jun 2009 13:33:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=KXsIpRxNg0jxjunboT1BkdfbPtZ+FSVI4+Q0SoTETDM=; b=MpbdD/oXXXqPlbZjydP2LLDJohFt8JRIoGGhwOf73DSmjgIUJs6a53IiY5UeqlAZTO TBzJLiO1H5EWUjZDvzTezUMg/891DONKmKi+snvW9zG/6p92X0glAeAoldBWo96o5HQ4 Zy9jaaAO5mVFH71a+jeJA+hT98jTZvBUZM4dk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=bvhQkGwLTnrDhnQw5Is2afLCo1+qBnrec/YRF8BCyWrCkS3F/Zn9htBayrKgILDbyA dYgfC1wGpKy+SadPQISAeT6sz4fzUZRgGRh0CLvoBJCrI4WHnunGqtq47V9ZFkEOejjf lOAdpFb7/G//JMwdSrFmiVCLEaGz6QYdJ01Xc= Received: by 10.86.81.15 with SMTP id e15mr7060431fgb.35.1243888422181; Mon, 01 Jun 2009 13:33:42 -0700 (PDT) Received: from localhost ([95.69.172.207]) by mx.google.com with ESMTPS id l19sm9812680fgb.22.2009.06.01.13.33.40 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 01 Jun 2009 13:33:41 -0700 (PDT) To: Asmodean Dark References: Organization: TOA Ukraine From: Mikolaj Golub Date: Mon, 01 Jun 2009 23:33:38 +0300 In-Reply-To: (Asmodean Dark's message of "Mon\, 01 Jun 2009 10\:05\:46 +0400") Message-ID: <86oct7r8wt.fsf@kopusha.onet> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-hackers@freebsd.org Subject: Re: 7.1-STABLE crash X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2009 20:33:43 -0000 On Mon, 01 Jun 2009 10:05:46 +0400 Asmodean Dark wrote: AD> Hello, AD> We have cluster of FreeBSD VPN servers with running mpd3.18 and sometimes it crash: Do you really need a such outdated version of mpd while mpd5 is available in ports? AD> # uname -a AD> FreeBSD vpn 7.1-STABLE FreeBSD 7.1-STABLE #13: Wed Mar 18 14:53:13 YEKT 2009 root@vpn:/usr/src/sys/i386/compile/kconf i386 AD> # dmesg AD> Fatal trap 12: page fault while in kernel mode AD> cpuid = 1; apic id = 01 AD> fault virtual address = 0x8 AD> fault code = supervisor read, page not present AD> instruction pointer = 0x20:0x806dcb88 AD> stack pointer = 0x28:0xe70775c4 AD> frame pointer = 0x28:0xe70775e8 AD> code segment = base 0x0, limit 0xfffff, type 0x1b AD> = DPL 0, pres 1, def32 1, gran 1 AD> processor eflags = interrupt enabled, resume, IOPL = 0 AD> current process = 22 (em0 taskq) AD> trap number = 12 AD> (kgdb) bt AD> #0 0x8063d6b0 in doadump () AD> #1 0x8063dc44 in boot () AD> #2 0x8063e0ca in panic () AD> #3 0x807dab3d in trap_fatal () AD> #4 0x807daeba in trap_pfault () AD> #5 0x807db7bd in trap () AD> #6 0x807c2a3b in calltrap () AD> #7 0x806dcb88 in rn_match () AD> #8 0x806ddc8a in rn_lookup () AD> #9 0x8070e460 in ipfw_chk (args=0xe70175fc) at ../../../netinet/ip_fw2.c:1894 AD> #10 0x80710c3d in ipfw_check_in (arg=0x0, m0=0xe7017700, ifp=0x91c5a800, dir=1, inp=0x0) at ../../../netinet/ip_fw_pfil.c:125 AD> #11 0x806dc20f in pfil_run_hooks () AD> #12 0x80713984 in ip_input (m=0x91954c00) at ../../../netinet/ip_input.c:416 AD> #13 0x806ec0d9 in ng_iface_rcvdata () AD> #14 0x806e9570 in ng_apply_item () AD> #15 0x806e8569 in ng_snd_item () AD> #16 0x806e9570 in ng_apply_item () AD> #17 0x806e8569 in ng_snd_item () AD> #18 0x806e9570 in ng_apply_item () AD> #19 0x806e8569 in ng_snd_item () AD> #20 0x806f16a7 in ng_ppp_proto_recv () AD> #21 0x806f3ed2 in ng_ppp_rcvdata () AD> #22 0x806e9570 in ng_apply_item () AD> #23 0x806e8569 in ng_snd_item () AD> #24 0x806e9570 in ng_apply_item () AD> #25 0x806e8569 in ng_snd_item () AD> #26 0x806ee3c3 in ng_ksocket_incoming2 () AD> #27 0x806e969d in ng_apply_item () AD> #28 0x806ea8aa in ngintr () AD> #29 0x806dab72 in swi_net () AD> #30 0x8061e265 in ithread_loop () AD> #31 0x8061adf5 in fork_exit () AD> #32 0x807c2ab0 in fork_trampoline () It looks like the kernel crashed when ipfw was looking for the packet's src/dst in a table. The table might have been corrupted (being modified?) at that time. Do you have some automatic blocker or some other script that periodically add/remove IPs in a ipfw table? AD> What can I do with it? Are additional info needed? It is a bit strange for me that in the bt output the file source information is displayed only for several functions. Do you have the kernel and all modules built with the debugging symbols? If you don't I would recommend to rebuild the kernel so I would be able to provide bt (from the next crash :-) with all necessary info available. Do the bt output for other crashes looks the same? -- Mikolaj Golub