Date: Sun, 24 Jun 2018 12:03:29 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: current@freebsd.org Subject: numa involved in instability and swap usage despite RAM free? Message-ID: <20180624120329.Horde.HWORumQ7Ng1KAUeviJNtoc3@webmail.leidinger.net>
next in thread | raw e-mail | index | archive | help
This message is in MIME format and has been PGP signed. --=_-oBR2oHEOll7in8gWqzLcvA Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I don't have hard evidence, but there is enough "smell" to open up a=20=20 discussion... Short: Can=20it be that enabling numa in the kernel is the reason why some=20=20 people=20see instability with zfs and usage of swap while a lot of free=20= =20 RAM=20is available? Long: I have a dual-socket Xeon system (E5620 + L5630... yes, not the same,=20=20 but=20compatible enough to be able to run together) with 64 GB RAM. I=20=20 run=20-current on it (currently it's at r333966 and it was for all the=20= =20 tests=20below). What I see with numa enabled and no zfs patches is, that at some point=20= =20 I=20have half the RAM free, swap is in use, and after a lot of compiling=20= =20 ports=20in different jails ZFS comes to a halt (sometimes I can unblock=20= =20 by=20killing a compile, sometimes I can't even kill, only way out is=20=20 power-cycle).=20I've seen this around twice a week. When I keep numa enabled and have applied this ZFS patch=20=20 https://reviews.freebsd.org/D7538=20the bahavior changes. AFter a while=20= =20 half=20of the RAM is free, swap is in use, and after enough compiling=20=20 ports=20in jails I get a panic (unfortunately not enough debug info in=20= =20 the=20textdump to know exactly what he problem is). Since 2 weeks I have numa compiled out of the kernel (and still the=20=20 ZFS=20patch inside). The system is down to 17 GB free and NO swap in=20=20 use.=20I'm compiling ports in 16 jails (one of them with parts of KDE5 =3D= =20=20 currently=20about 700 ports compiled) and not a single issue like the=20=20 above. For=20everyone with swap issues or ZFS issues similar to the ones I=20=20 see...=20do you have numa enabled and can you please try without and=20=20 report=20back? Can it be that if memory request can not be fulfilled from one numa=20=20 domain,=20there is no fallback to another numa domain for all the=20=20 various=20kinds of memory allocation we have in the kernel=20=20 (contigmem/no-sleep/...)? Bye, Alexander. --=20 http://www.Leidinger.net=20Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_-oBR2oHEOll7in8gWqzLcvA Content-Type: application/pgp-signature Content-Description: Digitale PGP-Signatur Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJbL2xxAAoJEKrxQhqFIICEVcsP/jVTtyEtb92DPBitIacgUk1F 3xzRc+0SfWYqyU8PAHsIRkJQFdvQ7rvLqwNznqvkLdi3FvmT+6PKJcORhHrNBMEo KhJGByGFPAdpnAoA7ZVJY7r6RWJsNU61AvPdgn0ZORPX73cbuo5zeCfwpIiYPGEH oja29zNA7XLnh9AZ5NJMl3zmvCEUOOONkENoUE/tDi4YMVy1m8EVmtSFAsdtBYHU G9incSchNzRx3tkK8kojknLjZCFQn44yOKYhANKwgMSECR5sZuvAcQaSLUNUvdcr 6OpjCFcHWsTFIr5vJvEqml+rSSnvt89q5JZBsP9T8/6dXs1rUuBKKLkVWR2A/bT9 d7c5WX4xFlk6vRRfcZ6pQ6GQQOs06IaxxPvRXDAdJcFCMj1gKatKumg55+IXxbwQ ovxuHay+YfmQxi/DuO0HAeh5JNd1NL3u2Dpb9Xs+xey0yKQXorFTU4ADiqcWXp/v kAiz0Cyf3gnaYuRA6bRZdUdTThj/3NDKKWztcX6pe+VGepSBBRsskEcfIns6oNgG n+CnkUIp5sjos+kG0iXD7d+ZqVPiHsdiWdvLzOuJD1bjagDbMD23eoA+m3uYLcKn buf/DJoW8VthcvlKPsh9Lxh2ChfiT3C1MVS0BTGeN6tfOWsPpg9E5c0e1mouWV9+ nMgw+XE0WfA2UDrP0X8k =ckNr -----END PGP SIGNATURE----- --=_-oBR2oHEOll7in8gWqzLcvA--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180624120329.Horde.HWORumQ7Ng1KAUeviJNtoc3>