Date: Sat, 18 Oct 2014 20:45:20 +0000 From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 194455] New: [PATCH] Preserve lowmem for devices on large memory machines Message-ID: <bug-194455-8@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194455 Bug ID: 194455 Summary: [PATCH] Preserve lowmem for devices on large memory machines Product: Base System Version: 11.0-CURRENT Hardware: Any OS: Any Status: Needs Triage Severity: Affects Only Me Priority: --- Component: kern Assignee: freebsd-bugs@FreeBSD.org Reporter: conrad.meyer@isilon.com Created attachment 148440 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=148440&action=edit ( applies with -p1 on CURRENT ~r273263 ) Log: ==================================8<================================== Add an intermediate VM_FREELIST "DMA32" between DEFAULT and ISADMA on AMD64. Physical allocations try freelists ascending, so the effect is to prefer DEFAULT, then DMA32, then ISADMA. This leaves low memory available for things that need it (DMA) on large-memory systems which statically allocate a lot (e.g. bufs). Sponsored by: EMC / Isilon storage division ==================================8<================================== Historical note/context: So, we have 96-256 GB systems, and scale nbufs up with memory. On these systems we tune for much more than 4GB in bufs. It turns out the 'buf' static array is allocated from physmem fairly early (along with some other things that scale with memory), and without this change, devices starved for 32-bit DMA space. Internally, we had just bumped the ISADMA region from 16MB (24-bit) to 4GB. I think the attached patch with a seperate 32-bit DMA region is slightly cleaner (and avoids other devices starving out ISA BUS DMA devices, if those still exist). But, either approach fixes the problem for us. So, if people object to adding another freelist, a compromise might be renaming the ISADMA freelist to "DMA" and bumping the region up to 32 bits. Testing on an AMD64 VM: # sysctl vm.phys_free vm.phys_free: DOMAIN 0: FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 257 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 2 | 1 | 0 9 ( 2048K) | 2 | 0 | 0 8 ( 1024K) | 1 | 0 | 0 7 ( 512K) | 1 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 0 | 2 | 0 4 ( 64K) | 0 | 1 | 0 3 ( 32K) | 4 | 1 | 0 2 ( 16K) | 3 | 3 | 0 1 ( 8K) | 0 | 9 | 0 0 ( 4K) | 1 | 1 | 0 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 221 | 0 | 0 11 ( 8192K) | 1 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 1 | 0 | 0 8 ( 1024K) | 0 | 0 | 0 7 ( 512K) | 1 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 1 | 0 | 0 4 ( 64K) | 1 | 0 | 0 3 ( 32K) | 0 | 0 | 0 2 ( 16K) | 0 | 0 | 0 1 ( 8K) | 0 | 0 | 0 0 ( 4K) | 1 | 0 | 0 FREE LIST 2: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 0 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 0 | 0 | 0 8 ( 1024K) | 1 | 0 | 0 7 ( 512K) | 0 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 1 | 0 | 0 4 ( 64K) | 2 | 0 | 0 3 ( 32K) | 2 | 0 | 0 2 ( 16K) | 1 | 0 | 0 1 ( 8K) | 0 | 0 | 0 0 ( 4K) | 1 | 0 | 0 -- You are receiving this mail because: You are the assignee for the bug.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-194455-8>