Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 31 Oct 2018 04:48:22 +0000
From:      bugzilla-noreply@freebsd.org
To:        bugs@FreeBSD.org
Subject:   [Bug 232800] Processes killed out of swap space
Message-ID:  <bug-232800-227-pztJw1B3GQ@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-232800-227@https.bugs.freebsd.org/bugzilla/>
References:  <bug-232800-227@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D232800

--- Comment #2 from teksimian@gmail.com ---
(In reply to Mark Millard from comment #1)
Thank you for your feedback. I've added vm.pageout_oom_seq=3D120 to sysctl.=
conf.

Both of the following messages have been logged;

Oct 30 23:08:20 host kernel: swap_pager: indefinite wait buffer: bufobj: 0,
blkno: 1143, size: 16384
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
477,size 69632, error 12

swap is filebacked;

[user@host /var/log]$ cat /etc/fstab
# Device        Mountpoint      FStype  Options Dump    Pass#
...
md99    none    swap    sw,late,file=3D/usr/swap0 0       0

the swap file is in /usr which is the same fs as / (it was provisioned this=
 way
by the cloud provider.)  I can roll back the vm to a 10.4 snapshot. I'm
confident the issue wouldnt be present.  The 10.4 install has been solid, a=
nd
this began /immediately/ after the upgrade.



Reboot 1

[user@host /var/log]$ sudo less all.log

...

Oct 30 23:05:58 host kernel: Starting jails:
Oct 30 23:08:20 host kernel:
Oct 30 23:08:20 host kernel: swap_pager: indefinite wait buffer: bufobj: 0,
blkno: 1143, size: 16384
Oct 30 23:08:20 host last message repeated 2 times
Oct 30 23:08:20 host kernel: pid 2532 (mysqld), uid 88, was killed: out of =
swap
space
Oct 30 23:08:20 host kernel: swap_pager: indefinite wait buffer: bufobj: 0,
blkno: 1143, size: 16384
Oct 30 23:08:20 host last message repeated 2 times
Oct 30 23:08:20 host kernel: pid 791 (ntpd), uid 0, was killed: out of swap
space
Oct 30 23:08:20 host kernel: Oct 30 23:08:20 host kernel: pid 2532 (mysqld),
uid 88, was killed: out of swap space
Oct 30 23:08:20 host kernel: Oct 30 23:08:20 host kernel: pid 791 (ntpd), u=
id
0, was killed: out of swap space

...

[user@host /var/log]$ uname -a
FreeBSD host.domain.com 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4 #0: Thu Sep=
 27
08:16:24 UTC 2018=20=20=20=20
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
[user@host /var/log]$ swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/md99          524288     5296   518992     1%
[user@host /var/log]$ uptime
11:13PM  up 8 mins, 2 users, load averages: 0.06, 0.11, 0.08
[user@host /var/log]$ sudo service ntpd start
Starting ntpd.
[user@host /var/log]$ sudo jail -c sql
sql: created
[user@host /var/log]$ swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/md99          524288    59912   464376    11%
[user@host /var/log]$ jls
   JID  IP Address      Hostname                      Path
     2  127.0.0.253     php.domain.com                 /usr/local/jails/php
     3  127.0.0.254     www.domain.com                 /usr/local/jails/www
     4  127.0.0.252     sql.domain.com                 /usr/local/jails/sql



Reboot 2

[user@host /var/log]$ sudo less all.log

Oct 30 23:16:50 host kernel: Starting jails:
Oct 30 23:16:52 host kernel:
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
445,size 131072, error 12
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
477,size 69632, error 12
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
494,size 8192, error 12
...
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
946,size 69632, error 12
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
963,size 36864, error 12
Oct 30 23:16:52 host kernel: swap_pager: I/O error - pageout failed; blkno
972,size 12288, error 12
Oct 30 23:18:12 host kernel: pid 2532 (mysqld), uid 88, was killed: out of =
swap
space
Oct 30 23:18:12 host kernel: Oct 30 23:18:12 host kernel: pid 2532 (mysqld),
uid 88, was killed: out of swap space

[user@host ~]$ uptime
11:18PM  up 2 mins, 1 user, load averages: 0.23, 0.25, 0.11
[user@host ~]$ jls
   JID  IP Address      Hostname                      Path
     2  127.0.0.253     php.domain.com                 /usr/local/jails/php
     3  127.0.0.254     www.domain.com                 /usr/local/jails/www
[user@host ~]$ swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/md99          524288     5752   518536     1%
[user@host ~]$ sudo jail -c sql
sql: created
[user@host ~]$ swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/md99          524288    54272   470016    10%

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-232800-227-pztJw1B3GQ>