Date: Sat, 30 May 2009 20:51:29 -0500 From: "Larry Rosenman" <ler@lerctr.org> To: "Randy Bush" <randy@psg.com> Cc: current <current@freebsd.org> Subject: Re: kern/134011: [hang] swap_pager_getswapspace(4): failed Message-ID: <d842742ebb1142510448fa01d71ec3ac.squirrel@webmail.lerctr.org> In-Reply-To: <m2eiu6xcvo.wl%randy@psg.com> References: <m27i00btdf.wl%randy@psg.com> <m2eiu6xcvo.wl%randy@psg.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, May 30, 2009 8:46 pm, Randy Bush wrote: >> this time it actually said something interesting on console! >> >> for some values of 'interesting' :) >> >> swap_pager_getswapspace(3): failed >> swap_pager_getswapspace(3): failed >> swap_pager_getswapspace(3): failed >> swap_pager_getswapspace(3): failed >> swap_pager_getswapsp >> a >> lcatale t(ra1p 612): : pafgae ifaullte wdhi >> e sin wkearnepl m_odpe >> acpugied =r 1_; agpiec itd s= w01a >> afasulpt vairtcuael (ad3dre)ss := 0x0f >> odfalulet dc >> e s =w saupeprv_isopr awgrieter _dagtae, ptagse nowta >> ppressepnat >> 0inest(ru3ct)i:on pfoinateir l= e0dx2 >> :0sxfwfaffpff_ffp80a47gc25e6 >> rst_acgk eptoinsterw a p s =p 0ax2c8:0exf(3ff)ff:f80 7f9fd1a680i >> poieadme >> ntserw a p _ =p 0xag28e:0rxf_fffgff8e07t9fsd16we0 >> 0xdep ssegpmenta c= beas(e 30x0), :lim itf 0xaffifflf,e >> tydpe >> 1bs >> w =a DpPL _0, ppraesg 1,e lorng_ 1g, deef3t2 s0,w >> garapn s1 >> ledaocceseso(r 3ef)la:gs =f ianteirrulpt eendab >> , rseswumae, pIOP_L p= a0 >> gcuerrren_t pgroecests s = w789a (psysslpogad) >> ctreap( n9umb)er: = f12 >> epainilc:e pdag >> fsauwlta >> pcp_uipd =a 1g >> eUptime: 9h50m49s >> Physical memory: 4083 MB >> Dumping 1958 MB: >> >> <end. required power cycle> > > a bit better in last night's syslog, possibly during backup > > randy > > > May 30 00:40:14 work0 kernel: lock order reversal: > May 30 00:40:14 work0 kernel: 1st 0xffffff0057d019d0 ufs (ufs) @ > /usr/src/sys/ufs/ffs/ffs_snapshot.c:423 > May 30 00:40:14 work0 kernel: 2nd 0xffffff8052c01aa0 bufwait (bufwait) @ > /usr/src/sys/kern/vfs_bio.c:2556 > May 30 00:40:14 work0 kernel: 3rd 0xffffff0004b8d098 ufs (ufs) @ > /usr/src/sys/ufs/ffs/ffs_snapshot.c:544 > May 30 00:40:16 work0 kernel: lock order reversal: > May 30 00:40:16 work0 kernel: 1st 0xffffff8052c01aa0 bufwait (bufwait) @ > /usr/src/sys/kern/vfs_bio.c:2556 > May 30 00:40:16 work0 kernel: 2nd 0xffffff00d35c7d30 snaplk (snaplk) @ > /usr/src/sys/ufs/ffs/ffs_snapshot.c:793 > May 30 00:50:14 work0 kernel: lock order reversal: > May 30 00:50:14 work0 kernel: 1st 0xffffff00d35c7d30 snaplk (snaplk) @ > /usr/src/sys/kern/vfs_vnops.c:297 > May 30 00:50:14 work0 kernel: 2nd 0xffffff0057d019d > May 30 00:50:14 work0 kernel: 0 ufs (ufs) @ /u > May 30 00:50:14 work0 kernel: s > May 30 00:50:15 work0 kernel: r/src/sys/ > May 30 00:50:15 work0 kernel: ufs/ffs/ > May 30 00:50:15 work0 kernel: ffs_snap > May 30 00:50:15 work0 kernel: shot.c: > May 30 00:50:15 work0 kernel: 1587 > May 30 01:45:21 work0 kernel: > May 30 01:45:21 work0 kernel: > May 30 01:45:21 work0 kernel: Fatal trap 12: page fault while in kernel > mode > May 30 01:45:21 work0 kernel: cpuid = 0; apic id = 00 > May 30 01:45:21 work0 kernel: fault virtual address = 0x0 > May 30 01:45:21 work0 kernel: fault code = supervisor write data, page > not present > May 30 01:45:21 work0 kernel: instruction pointer = > 0x20:0xffffffff8047c256 > May 30 01:45:21 work0 kernel: sta > May 30 01:45:21 work0 kernel: c > May 30 01:45:21 work0 kernel: k pointer = 0x28:0xffffff807a057680 > May 30 01:45:21 work0 kernel: frame pointer = > 0x28:0xffffff807a0576e0 > May 30 01:45:21 work0 kernel: code segment = base 0x0, limit 0xfffff, > type 0x1b > May 30 01:45:21 work0 kernel: = DPL 0, pres 1, long 1, def32 0, gran > May 30 01:45:21 work0 kernel: 1 > May 30 01:45:21 work0 kernel: processor eflags = interrup > May 30 01:45:21 work0 kernel: t enabled, resume, > May 30 01:45:21 work0 kernel: IOPL = 0 > May 30 01:45:21 work0 kernel: current process > May 30 01:45:21 work0 kernel: = 9181 (nfcapd) > May 30 02:10:04 work0 syslogd: kernel boot file is /boot/kernel/kernel See Kip's replies in my ZFS Crash thread. I suspect you have compression turned on in ZFS, and there is an unbounded allocation of memory in the ZFS (de)compression code. Kip is working on a patch to put some bounds on it. I've changed my tunables to not allow the ARC to grab ALL of physmem (vfs.zfs.arc_max) and testing a full backup now. Waiting, patiently, for code patch from Kip. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d842742ebb1142510448fa01d71ec3ac.squirrel>