Date: Sun, 24 Jun 2018 12:03:29 +0200 From: Alexander Leidinger <Alexander@leidinger.net> To: current@freebsd.org Subject: numa involved in instability and swap usage despite RAM free? Message-ID: <20180624120329.Horde.HWORumQ7Ng1KAUeviJNtoc3@webmail.leidinger.net>
index | next in thread | raw e-mail
[-- Attachment #1 --] Hi, I don't have hard evidence, but there is enough "smell" to open up a discussion... Short: Can it be that enabling numa in the kernel is the reason why some people see instability with zfs and usage of swap while a lot of free RAM is available? Long: I have a dual-socket Xeon system (E5620 + L5630... yes, not the same, but compatible enough to be able to run together) with 64 GB RAM. I run -current on it (currently it's at r333966 and it was for all the tests below). What I see with numa enabled and no zfs patches is, that at some point I have half the RAM free, swap is in use, and after a lot of compiling ports in different jails ZFS comes to a halt (sometimes I can unblock by killing a compile, sometimes I can't even kill, only way out is power-cycle). I've seen this around twice a week. When I keep numa enabled and have applied this ZFS patch https://reviews.freebsd.org/D7538 the bahavior changes. AFter a while half of the RAM is free, swap is in use, and after enough compiling ports in jails I get a panic (unfortunately not enough debug info in the textdump to know exactly what he problem is). Since 2 weeks I have numa compiled out of the kernel (and still the ZFS patch inside). The system is down to 17 GB free and NO swap in use. I'm compiling ports in 16 jails (one of them with parts of KDE5 = currently about 700 ports compiled) and not a single issue like the above. For everyone with swap issues or ZFS issues similar to the ones I see... do you have numa enabled and can you please try without and report back? Can it be that if memory request can not be fulfilled from one numa domain, there is no fallback to another numa domain for all the various kinds of memory allocation we have in the kernel (contigmem/no-sleep/...)? Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJbL2xxAAoJEKrxQhqFIICEVcsP/jVTtyEtb92DPBitIacgUk1F 3xzRc+0SfWYqyU8PAHsIRkJQFdvQ7rvLqwNznqvkLdi3FvmT+6PKJcORhHrNBMEo KhJGByGFPAdpnAoA7ZVJY7r6RWJsNU61AvPdgn0ZORPX73cbuo5zeCfwpIiYPGEH oja29zNA7XLnh9AZ5NJMl3zmvCEUOOONkENoUE/tDi4YMVy1m8EVmtSFAsdtBYHU G9incSchNzRx3tkK8kojknLjZCFQn44yOKYhANKwgMSECR5sZuvAcQaSLUNUvdcr 6OpjCFcHWsTFIr5vJvEqml+rSSnvt89q5JZBsP9T8/6dXs1rUuBKKLkVWR2A/bT9 d7c5WX4xFlk6vRRfcZ6pQ6GQQOs06IaxxPvRXDAdJcFCMj1gKatKumg55+IXxbwQ ovxuHay+YfmQxi/DuO0HAeh5JNd1NL3u2Dpb9Xs+xey0yKQXorFTU4ADiqcWXp/v kAiz0Cyf3gnaYuRA6bRZdUdTThj/3NDKKWztcX6pe+VGepSBBRsskEcfIns6oNgG n+CnkUIp5sjos+kG0iXD7d+ZqVPiHsdiWdvLzOuJD1bjagDbMD23eoA+m3uYLcKn buf/DJoW8VthcvlKPsh9Lxh2ChfiT3C1MVS0BTGeN6tfOWsPpg9E5c0e1mouWV9+ nMgw+XE0WfA2UDrP0X8k =ckNr -----END PGP SIGNATURE-----help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180624120329.Horde.HWORumQ7Ng1KAUeviJNtoc3>