Date: 01 Nov 2007 21:20:42 +0100 From: "Arno J. Klaassen" <arno@heho.snv.jussieu.fr> To: freebsd-hackers@freebsd.org Subject: "indefinite" wait buffer patch Message-ID: <wpsl3p1zvp.fsf@heho.snv.jussieu.fr>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
while slowly testing releng_7, I remembered I have since about
two years the attached diff in my releng_6 sources (patch
recreated against releng_7 with low timeouts for debugging) :
it addresses the situation when one creates a huge swap-space on
a (relatively) slow disk-subsystem : e.g. for scientific computing
it sometimes makes sense to have, e.g. 8G swap for 2G main memory
if you know you're treating N less then 2G matrices and process
is CPU-bound for quite a while for 1 matrix before switching to
the other.
But then, when switching from one matrix to another, dmesg gets
flooded by :
"indefinite wait buffer"
messages.
The attached patch shows in fact that the wait buffer is never
"indefinite" (unless a real HW-problem probably) and linearly
increases timeout to match with reality.
The last chunk is just to prevent for a panic at reboot when
there is so much data swapped out that is doesn't get treated
before 'reboot-finish-time-out'.
With this patch, dmesg lookes like following (with low timeout
values, on production systems I use
TIMO_CHUNK 20
TIMO_START 1O
) :
Nov 1 20:09:52 install kernel: swap_pager: wait buffer timeout 1 (1 secs): bufobj: 0, blkno: 1649, size: 28672
Nov 1 20:09:52 install kernel: swap_pager: wait buffer timeout 2 (2 secs): bufobj: 0, blkno: 1649, size: 28672
Nov 1 20:09:52 install kernel: swap_pager: wait buffer completed (2 retry): bufobj: 0, blkno: 1649, size: 28672
Nov 1 20:37:09 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 329073, size: 32768
Nov 1 20:37:10 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 329073, size: 32768
Nov 1 20:39:06 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 381137, size: 32768
Nov 1 20:39:07 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 381137, size: 32768
Nov 1 20:39:11 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 381161, size: 32768
Nov 1 20:39:14 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 381161, size: 32768
Nov 1 20:39:19 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 381209, size: 32768
Nov 1 20:39:20 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 381209, size: 32768
Nov 1 20:43:18 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 300329, size: 32768
Nov 1 20:43:18 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 300329, size: 32768
Nov 1 20:44:23 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 330617, size: 32768
Nov 1 20:44:24 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 330617, size: 32768
Nov 1 20:44:28 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 330649, size: 32768
Nov 1 20:44:28 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 330649, size: 32768
Nov 1 20:44:33 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 330665, size: 32768
Nov 1 20:44:36 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 330665, size: 32768
Nov 1 20:45:18 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 356481, size: 32768
Nov 1 20:45:18 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 356481, size: 32768
Nov 1 20:45:23 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 356521, size: 32768
Nov 1 20:45:27 install kernel: swap_pager: wait buffer timeout 2 (4 secs): bufobj: 0, blkno: 356521, size: 32768
Nov 1 20:45:31 install kernel: swap_pager: wait buffer completed (2 retry): bufobj: 0, blkno: 356521, size: 32768
Nov 1 20:46:37 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 391113, size: 32768
Nov 1 20:46:37 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 391113, size: 32768
Nov 1 20:46:42 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 391129, size: 32768
Nov 1 20:46:45 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 391129, size: 32768
Nov 1 20:48:18 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 432249, size: 32768
Nov 1 20:48:18 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 432249, size: 32768
Nov 1 20:48:25 install kernel: swap_pager: wait buffer timeout 1 (4 secs): bufobj: 0, blkno: 432273, size: 32768
Nov 1 20:48:25 install kernel: swap_pager: wait buffer completed (1 retry): bufobj: 0, blkno: 432273, size: 32768
(And then the timeout of 8 secs apperently is enough for this test setup).
Any thoughts?
Regards,
Arno
[-- Attachment #2 --]
Index: sys/vm/swap_pager.c
===================================================================
RCS file: /home/ncvs/src/sys/vm/swap_pager.c,v
retrieving revision 1.295
diff -u -r1.295 swap_pager.c
--- sys/vm/swap_pager.c 5 Aug 2007 21:04:32 -0000 1.295
+++ sys/vm/swap_pager.c 1 Nov 2007 18:59:18 -0000
@@ -941,6 +941,10 @@
vm_page_t mreq;
int i;
int j;
+ int retry = 0;
+#define TIMO_CHUNK 2
+#define TIMO_START 1 /* set low to force quick first timeout */
+ static int timo_secs = TIMO_START;
daddr_t blk;
mreq = m[reqpage];
@@ -1066,16 +1070,28 @@
*/
VM_OBJECT_LOCK(object);
while ((mreq->oflags & VPO_SWAPINPROG) != 0) {
+ if (retry == 0) {
mreq->oflags |= VPO_WANTED;
vm_page_lock_queues();
vm_page_flag_set(mreq, PG_REFERENCED);
vm_page_unlock_queues();
PCPU_INC(cnt.v_intrans);
- if (msleep(mreq, VM_OBJECT_MTX(object), PSWP, "swread", hz*20)) {
+ }
+ if (msleep(mreq, VM_OBJECT_MTX(object), PSWP, "swread", hz*timo_secs)) {
printf(
-"swap_pager: indefinite wait buffer: bufobj: %p, blkno: %jd, size: %ld\n",
- bp->b_bufobj, (intmax_t)bp->b_blkno, bp->b_bcount);
+"swap_pager: wait buffer timeout %d (%d secs): bufobj: %p, blkno: %jd, size: %ld\n",
+ ++retry, timo_secs, bp->b_bufobj, (intmax_t)bp->b_blkno, bp->b_bcount);
+ if (retry*TIMO_CHUNK > timo_secs) {
+ timo_secs = retry*TIMO_CHUNK;
+ }
+ } else {
+ if (retry > 0) {
+ printf(
+"swap_pager: wait buffer completed (%d retry): bufobj: %p, blkno: %jd, size: %ld\n",
+ retry, bp->b_bufobj, (intmax_t)bp->b_blkno, bp->b_bcount);
+ }
}
+
}
/*
@@ -1553,6 +1569,7 @@
swp_pager_force_pagein(vm_object_t object, vm_pindex_t pindex)
{
vm_page_t m;
+ int ret;
vm_object_pip_add(object, 1);
m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL|VM_ALLOC_RETRY);
@@ -1567,8 +1584,18 @@
return;
}
- if (swap_pager_getpages(object, &m, 1, 0) != VM_PAGER_OK)
- panic("swap_pager_force_pagein: read from swap failed");/*XXX*/
+ if ((ret=swap_pager_getpages(object, &m, 1, 0)) != VM_PAGER_OK) {
+ if (ret == VM_PAGER_FAIL) {
+ printf("swp_pager_force_pagein: VM_PAGER_FAIL\n");
+ }
+ else {
+ if (ret == VM_PAGER_ERROR) {
+ printf("swp_pager_force_pagein: VM_PAGER_ERROR\n");
+ }
+ else
+ panic("swap_pager_force_pagein: read from swap failed");/*XXX*/
+ }
+ }
vm_object_pip_subtract(object, 1);
vm_page_lock_queues();
vm_page_dirty(m);
[-- Attachment #3 --]
Arno J. Klaassen
SCITO S.A.
8 rue des Haies
F-75020 Paris, France
http://scito.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?wpsl3p1zvp.fsf>