From owner-freebsd-hackers Wed Oct 23 20:50:37 1996 Return-Path: owner-hackers Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA05905 for hackers-outgoing; Wed, 23 Oct 1996 20:50:37 -0700 (PDT) Received: from dg-rtp.dg.com (dg-rtp.rtp.dg.com [128.222.1.2]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id UAA05895 for ; Wed, 23 Oct 1996 20:50:33 -0700 (PDT) Received: by dg-rtp.dg.com (5.4R3.10/dg-rtp-v02) id AA21519; Wed, 23 Oct 1996 23:50:02 -0400 Received: from ponds by dg-rtp.dg.com.rtp.dg.com; Wed, 23 Oct 1996 23:50 EDT Received: from lakes.water.net (lakes [10.0.0.3]) by ponds.water.net (8.7.5/8.7.3) with ESMTP id XAA00521 for ; Wed, 23 Oct 1996 23:32:25 -0400 (EDT) Received: (from rivers@localhost) by lakes.water.net (8.7.5/8.6.9) id XAA11460 for freebsd-hackers@freefall.cdrom.com; Wed, 23 Oct 1996 23:32:58 -0400 (EDT) Date: Wed, 23 Oct 1996 23:32:58 -0400 (EDT) From: Thomas David Rivers Message-Id: <199610240332.XAA11460@lakes.water.net> To: ponds!freefall.cdrom.com!freebsd-hackers Subject: Re: Daily panic's with 2.1.5R. Content-Type: text Sender: owner-hackers@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > >One question before I get to that. I recall that the kernel > >boot messages used to say something about the previous panic when > >it discovered the savecore image. Did this go away, or am I > >simply mis-remembering? > > You're not mis-remembering, but I don't know why you didn't see it. It > comes out from syslogd, so perhaps there is something wrong with that on > your machine. Hmmm... I haven't touched the syslogd configuration - this is straight off of the net 2.1.5R distribution, except for a recompiled kernel. Here's my syslog.conf: *.err;kern.debug;auth.notice;mail.crit /dev/console *.notice;kern.debug;lpr,auth.info;mail.crit /var/log/messages mail.info /var/log/maillog lpr.info /var/log/lpd-errs cron.* /var/cron/log *.err root *.notice;auth.debug root *.alert root *.emerg * !startslip *.* /var/log/slip.log > > >panic: ifree: freeing free inode > >#0 0xf0193abb in boot () > >(kgdb) where > >#0 0xf0193abb in boot () > >#1 0xf0112b73 in panic () > >#2 0xf0176db7 in ffs_vfree () > >#3 0xf017c042 in ufs_inactive () > >#4 0xf0128ca5 in vrele () > >#5 0xf0128c07 in vput () > >#6 0xf017f574 in ufs_remove () > >#7 0xf012acae in unlink () > >#8 0xf019be36 in syscall () > >#9 0xf01912fb in Xsyscall () > >#10 0x2d9a in ?? () > >#11 0x2b2a in ?? () > >#12 0x2507 in ?? () > >#13 0x19b9 in ?? () > >#14 0x10d3 in ?? () > > > > > >The traceback is pretty straightforward - an unlink() has > >caused an inode to be duplicately free'd. > > > >Has anyone seen this before? > > > >I've seen similar problems when installing - from version 2.1.0 > >on. I've been able to work around them in the past by adjusting > >the inode values on newfs parms on the file system. I'm guessing > >that adjustment simply covers up the problem. > > > > I see in ffs_vfree() that I should have seen a message detailing > >some information - but that's not in the /var/log/messages (not likely > >it could be, eh?) and the screen has long since scrolled on... > > > > I have the kernel & vmcore files, I can put them on freefall > >if anyone wants to take a gander... > > Do a: > > tail -100 vmcore.0 | strings | more > > ...and you should find the original kernel message buffer. > Sure enough - along with the a lot of the contents of my Cnews active file, I got: dev = 0x20004, ino = 640, fs = /usr panic: ifree: freeing free inode syncing disks... 36 36 33 27 21 9 1 1 1 1 1 1 1 1 1 1 1 1 1 1 giving up dumping to dev 20001, offset 290304 dump 7 6 5 4 3 2 1 0 pretty handy tidbit to know. This just means that inode #640 on /dev/wd0s1e was redudantly free'd, most likely when a C-news expire was running. I did a quick scan of the mail logs I have to discover that Terry had a "hack" (his words, not mine :-) ) to vfs_subr.c to fix an off-by-one error in the free list management there. Apparently, that didn't make it into 2.1.5. I scanned the RCS log for vfs_subr.c, nothing obvious jumped out. Also, I diff'd the 2.1.5 and -current vfs_subr.c. But, there's been a few other changes that make it difficult to determine just what Terry's fix would have been... Anyway, the mail logs hint that there is in 2.1.5 a problem with free inode management... I tried reading through vfs_alloc.c - I didn't see any obvious problem there. I'm betting there's nothing wrong there, vfs_vfree() is probably getting called twice with the same inode for some reason... Since this is a daily occurrence, I could easily take steps to try and diagnose this (extra kernel printf()'s, etc...) but I'm not a VNOP wizard. Anyone have any ideas of where to start? - Dave R. -