From owner-freebsd-current@FreeBSD.ORG Thu Jan 6 19:36:13 2005 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5854516A4CE; Thu, 6 Jan 2005 19:36:13 +0000 (GMT) Received: from obsecurity.dyndns.org (CPE0050040655c8-CM00111ae02aac.cpe.net.cable.rogers.com [69.199.47.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAF0443D2F; Thu, 6 Jan 2005 19:36:12 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 9C75A519EB; Thu, 6 Jan 2005 11:36:51 -0800 (PST) Date: Thu, 6 Jan 2005 11:36:51 -0800 From: Kris Kennaway To: Kris Kennaway Message-ID: <20050106193651.GA95873@xor.obsecurity.org> References: <20050106025042.GA26670@xor.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZGiS0Q5IWpPtfppv" Content-Disposition: inline In-Reply-To: <20050106025042.GA26670@xor.obsecurity.org> User-Agent: Mutt/1.4.2.1i cc: alc@FreeBSD.org cc: current@FreeBSD.org Subject: Re: Deadlock with 32GB swap-backed md X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2005 19:36:13 -0000 --ZGiS0Q5IWpPtfppv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jan 05, 2005 at 06:50:42PM -0800, Kris Kennaway wrote: > In my quest for ever-more-insane things to do to panic package build > machines, I switched them over to use a 32GB swap partition as a > swap-backed md, in the hopes of improving build times by cutting back > on disk accesses. This seemed to work (I wasn't sure you could use > this much on i386, but the system let me configure as much as 32GB), > except two of them have locked up. ps shows Another one: 45649 c2ca5bd0 0 370 45649 4000100 [SLPQ vmwait 0xc07e9018][SLP] sshd 594 c1ba21f8 0 1 594 0000000 [SLPQ vmwait 0xc07e9018][SLP] inetd 414 c1f63bd0 0 0 0 0000204 [SLPQ vmwait 0xc07e9018][SLP] md3 280 c19ffbd0 0 0 0 0000204 [SLPQ mdwait 0xc1f01000][SLP] md2 115 c1ba27e0 0 0 0 0000204 [SLPQ mdwait 0xc1cfd800][SLP] md1 73 c19ff9d8 0 0 0 0000204 [SLPQ mdwait 0xc1a6f800][SLP] md0 7 c19fcdc8 0 0 0 0000204 [SLPQ VMWait 0xc07e9728][SLP] paged= aemon 0 c0786ec0 0 0 0 0000200 [SLPQ vmwait 0xc07e9018][SLP] swapp= er =20 db> tr 45649 Tracing pid 45649 tid 100115 td 0xc2be9730 sched_switch(c2be9730,0,1,11e,445fba21) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,1) at mi_switch+0x1d3 sleepq_switch(c07282e2,1,c2be9730,1,d67fa8b4) at sleepq_switch+0x10d sleepq_wait(c07e9018,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c07e9018,c07e9280,244,c073f19e,0) at msleep+0x523 vm_wait(c27e918c,0,c073dad5,6fc,5) at vm_wait+0x71 swp_pager_meta_build(4,0,5,0,4) at swp_pager_meta_build+0x174 swap_pager_copy(c1f6a8c4,c27e918c,0,0,1) at swap_pager_copy+0x1b7 vm_object_collapse(c27e918c,0,c073ebec,204,c2904dec) at vm_object_collapse+= 0x398 vm_object_deallocate(c2904dec,0,c073e2ab,89a,c1046780) at vm_object_dealloc= ate+0x330 vm_map_delete(c2bbc000,0,bfc00000,c2bbc000,0) at vm_map_delete+0x2b8 vm_map_remove(c2bbc000,0,bfc00000,35e,bfbfeed8) at vm_map_remove+0x57 exec_new_vmspace(d67fab88,c077b2a0,c0723d62,291,c050d9f6) at exec_new_vmspa= ce+0x26a exec_elf32_imgact(d67fab88,8,c0724e62,15e,c0725bab) at exec_elf32_imgact+0x= 244 kern_execve(c2be9730,8078050,807c5a0,bfbfeed0,0) at kern_execve+0x444 execve(c2be9730,d67fad14,c,c2be9730,3) at execve+0x2e syscall(bfbf002f,2f,bfbf002f,5,0) at syscall+0x137 Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (59, FreeBSD ELF32, execve), eip =3D 0x282ef2cf, esp =3D 0xbfbf= debc, ebp =3D 0xbfbfded8 --- db> tr 594 Tracing pid 594 tid 100036 td 0xc199b8a0 sched_switch(c199b8a0,0,1,11e,7883439f) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,0) at mi_switch+0x1d3 sleepq_switch(c07282e2,0,c199b8a0,0,d5307c30) at sleepq_switch+0x10d sleepq_wait(c07e9018,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c07e9018,c07e9280,244,c073f19e,0) at msleep+0x523 vm_wait(c1a33e00,0,d5307c78,c05908c6,c2871450) at vm_wait+0x71 vm_forkproc(c199b8a0,c28707e0,c2871450,14,12b) at vm_forkproc+0x65 fork1(c199b8a0,14,0,d5307cd8,0) at fork1+0xd44 fork(c199b8a0,d5307d14,279,c199b8a0,0) at fork+0x28 syscall(2f,2f,bfbf002f,7,bfbfdc04) at syscall+0x137 Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (2, FreeBSD ELF32, fork), eip =3D 0x280e980f, esp =3D 0xbfbfdb8= c, ebp =3D 0xbfbfee8c --- db> tr 414 Tracing pid 414 tid 100068 td 0xc1ba4b80 sched_switch(c1ba4b80,0,1,11e,8be00bb9) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,0) at mi_switch+0x1d3 sleepq_switch(c07282e2,0,c1ba4b80,0,d5591bf4) at sleepq_switch+0x10d sleepq_wait(c07e9018,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c07e9018,c07e9280,244,c073f19e,0) at msleep+0x523 vm_wait(c1f0b528,0,c073f0ba,593,0) at vm_wait+0x71 vm_page_grab(c1f0b528,348e59,0,80,0) at vm_page_grab+0x161 md_kthread(c1efa800,d5591d48,c0724fc7,30e,c1ba4b80) at md_kthread+0x24b fork_exit(c04a0800,c1efa800,d5591d48) at fork_exit+0xa9 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip =3D 0, esp =3D 0xd5591d7c, ebp =3D 0 --- db> tr 280 Tracing pid 280 tid 100039 td 0xc199bcf0 sched_switch(c199bcf0,0,1,11e,203b0089) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,0) at mi_switch+0x1d3 sleepq_switch(c07282e2,0,c199bcf0,0,d5310c38) at sleepq_switch+0x10d sleepq_wait(c1f01000,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c1f01000,c1f01024,24c,c071da28,0) at msleep+0x523 md_kthread(c1f01000,d5310d48,c0724fc7,30e,c199bcf0) at md_kthread+0x74a fork_exit(c04a0800,c1f01000,d5310d48) at fork_exit+0xa9 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip =3D 0, esp =3D 0xd5310d7c, ebp =3D 0 --- db> db> tr 115 Tracing pid 115 tid 100062 td 0xc1ba42e0 sched_switch(c1ba42e0,0,1,11e,13c8acf5) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,0) at mi_switch+0x1d3 sleepq_switch(c07282e2,0,c1ba42e0,0,d557fc38) at sleepq_switch+0x10d sleepq_wait(c1cfd800,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c1cfd800,c1cfd824,24c,c071da28,0) at msleep+0x523 md_kthread(c1cfd800,d557fd48,c0724fc7,30e,c1ba42e0) at md_kthread+0x74a fork_exit(c04a0800,c1cfd800,d557fd48) at fork_exit+0xa9 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip =3D 0, esp =3D 0xd557fd7c, ebp =3D 0 --- db> tr 73 Tracing pid 73 tid 100040 td 0xc19fd000 sched_switch(c19fd000,0,1,11e,c5bdaa59) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,0) at mi_switch+0x1d3 sleepq_switch(c07282e2,0,c19fd000,0,d5313c38) at sleepq_switch+0x10d sleepq_wait(c1a6f800,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c1a6f800,c1a6f824,24c,c071da28,0) at msleep+0x523 md_kthread(c1a6f800,d5313d48,c0724fc7,30e,c19fd000) at md_kthread+0x74a fork_exit(c04a0800,c1a6f800,d5313d48) at fork_exit+0xa9 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip =3D 0, esp =3D 0xd5313d7c, ebp =3D 0 --- db> tr 7 Tracing pid 7 tid 100046 td 0xc19fd8a0 sched_switch(c19fd8a0,0,1,11e,75b47f0d) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,1) at mi_switch+0x1d3 sleepq_switch(c07282e2,1,c19fd8a0,1,d5325a90) at sleepq_switch+0x10d sleepq_wait(c07e9728,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c07e9728,c07e9280,240,c073f1a5,0) at msleep+0x523 vm_wait(c1f0b528,0,c073dad5,6fc,1e1f10) at vm_wait+0xc7 swp_pager_meta_build(341ee0,0,1e1f10,0,1e1f10) at swp_pager_meta_build+0x174 swap_pager_putpages(c1f0b528,d5325be0,10,0,d5325b40) at swap_pager_putpages= +0x2ae vm_pageout_flush(d5325be0,10,0,0,0) at vm_pageout_flush+0x160 vm_pageout(0,d5325d48,c0724fc7,30e,bb5b602c) at vm_pageout+0x1814 fork_exit(c06a37f0,0,d5325d48) at fork_exit+0xa9 fork_trampoline() at fork_trampoline+0x8 --- trap 0x1, eip =3D 0, esp =3D 0xd5325d7c, ebp =3D 0 --- db> tr 0 Tracing pid 0 tid 0 td 0xc07870c0 sched_switch(c07870c0,0,1,11e,465022cd) at sched_switch+0x115 mi_switch(1,0,c072a51b,1ab,1) at mi_switch+0x1d3 sleepq_switch(c07282e2,1,c07870c0,1,c0c20d34) at sleepq_switch+0x10d sleepq_wait(c07e9018,0,c07282e2,da,0) at sleepq_wait+0x1a msleep(c07e9018,c07e9280,244,c073f19e,0) at msleep+0x523 vm_wait(c0786ec0,0,44,c0728f4a,2710) at vm_wait+0x71 scheduler(0,0,c0748b8c,c1ec00,c1e000) at scheduler+0x1ce mi_startup() at mi_startup+0xd6 begin() at begin+0x2c db> Kris --ZGiS0Q5IWpPtfppv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFB3ZNTWry0BWjoQKURAmzCAJwPBFDmdMgyidJhKkopQUkwmLITBQCggY5y RtVP9+6fqv49Z4hDWUT3ZWo= =1fpm -----END PGP SIGNATURE----- --ZGiS0Q5IWpPtfppv--