Date: Tue, 27 Mar 2012 02:32:26 +0000 From: "Philip M. Gollucci" <pgollucci@gmail.com> To: "Philip M. Gollucci" <pgollucci@gmail.com> Cc: questions@freebsd.org, current@freebsd.org Subject: Re: freebsd 9.0-release + zfs + mysqld(percona) = kernel: swap zone exhausted, increase kern.maxswzone Message-ID: <4F7126BA.1000202@p6m7g8.com> In-Reply-To: <4F708223.4060803@p6m7g8.com> References: <4F708223.4060803@p6m7g8.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigC256B7493D0030294755CCCD Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Some other tuning updates $ zfs set zfs:zfs_nocacheflush =3D 1 $ sysctl vfs.zfs.prefetch_disable=3D1 $ cat /etc/my.cnf skip-innodb-doublewrite innodb_flush_log_at_trx_commit=3D2 $ zfs set primarycache=3Dmetadata zmysqlD $ zfs set atime=3Doff zmysqlD $ zfs set recordsize=3D16k zmysqlD but not on zmysqlL my next plan is to turn off tmpfs and use ZVOL swaps then to simply use just zroot/tmp as a normal dir. after that I'll drastically increase maxswzone. still hoping someone has already done this. On 03/26/12 14:50, Philip M. Gollucci wrote: > /var/log/messages > Mar 23 22:21:50 sabertooth kernel: swap zone exhausted, increase > kern.maxswzone > Mar 23 22:21:50 sabertooth kernel: pid 86697 (mysqld), uid 88, was > killed: out of swap space >=20 > how to repeat: > $ mysql -ux < file.sql (~150GB) worth >=20 > basically, it slows down continually until it dies. IF you (suspend) > the process in time it recovers some, but eventually you have to suspen= d > it every 1s for ~3 minutes. The load is ~10 at this point. >=20 > I've looked at top, ps, iostat, zpool iostat, vmstat -z, vmstat -m > and I don't see anything wonky. I can provide more info on request. >=20 > system description: >=20 > $ df > zmysqlD 801G 658G 142G 82% /var/db/mysql/data > zmysqlL 133G 26G 107G 20% /var/db/mysql/log >=20 > its a 600GB innodb space, mysql has > innodb_buffer_pool_size =3D 80GB > about 1GB of data is MyISAM the rest is InnoDB >=20 > The machine has 96GB of RAM >=20 > $ cat /etc/fstab > /dev/gpt/swap0 none swap sw 0 0 > /dev/gpt/swap1 none swap sw 0 0 >=20 > tmpfs /tmp tmpfs rw 2 0 >=20 > swapinfo -h will show %6 and %6 usage on the swap devices > /tmp remains < 5% used >=20 > $ grep maxswzone /boot/loader.conf > kern.maxswzone=3D"67108864" ## double the default >=20 > $ gpart show > =3D> 34 286749421 da3 GPT (136G) > 34 128 1 freebsd-boot (64k) > 162 201326592 2 freebsd-swap (96G) > 201326754 85422701 3 freebsd-zfs (40G) >=20 > =3D> 34 286749421 da4 GPT (136G) > 34 128 1 freebsd-boot (64k) > 162 201326592 2 freebsd-swap (96G) > 201326754 85422701 3 freebsd-zfs (40G) >=20 > da[012] are SSDs, the rest are 15krpm >=20 > $ zpool status > pool: zmysqlD > state: ONLINE > scan: none requested > config: >=20 > NAME STATE READ WRITE CKSUM > zmysqlD ONLINE 0 0 0 > raidz2-0 ONLINE 0 0 0 > da7 ONLINE 0 0 0 > da8 ONLINE 0 0 0 > da9 ONLINE 0 0 0 > da10 ONLINE 0 0 0 > da11 ONLINE 0 0 0 > da12 ONLINE 0 0 0 > da13 ONLINE 0 0 0 > da14 ONLINE 0 0 0 > logs > da0 ONLINE 0 0 0 > cache > da2 ONLINE 0 0 0 >=20 > errors: No known data errors >=20 > pool: zmysqlL > state: ONLINE > scan: none requested > config: >=20 > NAME STATE READ WRITE CKSUM > zmysqlL ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > da6 ONLINE 0 0 0 > cache > da1 ONLINE 0 0 0 >=20 > errors: No known data errors >=20 > pool: zroot > state: ONLINE > scan: none requested > config: >=20 > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > da3p3 ONLINE 0 0 0 > da4p3 ONLINE 0 0 0 >=20 >=20 >=20 --=20 ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci@p6m7g8.com) c: 703.336.9354 Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Director Operations, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. --------------enigC256B7493D0030294755CCCD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFPcSa8dbiP+9ubjBwRAgn+AJ467lTaFaaQs9OaV3fJnb1i25+1BACcCVh4 b8MyZzmS7/aGS8Zdh9+sHA8= =knaH -----END PGP SIGNATURE----- --------------enigC256B7493D0030294755CCCD--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F7126BA.1000202>