Date: Sun, 15 Jul 2012 09:39:03 -0700 From: mdf@FreeBSD.org To: Justin Hibbits <chmeeedalf@gmail.com> Cc: freebsd-current <freebsd-current@freebsd.org>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: panic with DEBUG_MEMGUARD on PowerPC Message-ID: <CAMBSHm99OWq4sK-Wamnfy%2B_W5rJ-2WCQ24kGLFTK%2BQqfQzDs-Q@mail.gmail.com> In-Reply-To: <E7611D3D-806F-4BA1-9B83-6C903D23D6EB@gmail.com> References: <A3CD63CD-694A-48F5-B0F7-9C8923AFCB90@gmail.com> <CAMBSHm-5Xix46MaYAwBek6hWvcOHZ7%2BR_4cpdG5SH_5RD7difQ@mail.gmail.com> <307005B6-C8E5-4DCF-BD10-6BC79D8C2FE3@gmail.com> <CAMBSHm-CM0hc2Cu=C-zf7ArENqVz9iOHCjb0wMSPCnYQbXKqdA@mail.gmail.com> <E7611D3D-806F-4BA1-9B83-6C903D23D6EB@gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Sat, Jul 14, 2012 at 8:39 AM, Justin Hibbits <chmeeedalf@gmail.com> wrote: > On Jul 13, 2012, at 12:20 AM, mdf@freebsd.org wrote: > >> On Thu, Jul 12, 2012 at 6:33 PM, Justin Hibbits <chmeeedalf@gmail.com> >> wrote: >>> >>> On Jul 12, 2012, at 9:11 PM, mdf@freebsd.org wrote: >>> >>>> On Thu, Jul 12, 2012 at 4:43 PM, Justin Hibbits <chmeeedalf@gmail.com> >>>> wrote: >>>>> >>>>> >>>>> When tracking down a panic exposed by INVARIANTS, I tried setting >>>>> DEBUG_MEMGUARD, so I could find the culprit that's trashing freed >>>>> memory. >>>>> However, this causes a panic at bootup. It shows up right after the >>>>> first >>>>> WARNING: WITNESS message, with the following: >>>>> >>>>> Tracing, and printf() debugging, I see arguments to vm_map_findspace(): >>>>> start: 0xD0000000, length: 4246446080, and map->max_offset = >>>>> 4026531839. >>>>> >>>>> Beyond that, I'm lost with tracking this down. Machine is a dual >>>>> processor >>>>> PowerPC G4, with 2GB RAM. >>>> >>>> >>>> >>>> The length is 0xFD1BA000 which is almost 4GB. Asking for 4GB of >>>> virtual space for 2GB of RAM sounds about right (it's been a while >>>> since I was in this code), unless this is a 32-bit kernel, in which >>>> case it'd be too much since there isn't that much virtual space >>>> available. >>>> >>>> So, is the kernel 32-bit? What are the values used and returned by >>>> memguard_fudge()? The intent of that routine is to get kmeminit() to >>>> allocate a larger map so memguard can use part of it for private >>>> virtual addresses. But it shouldn't be asking for "too much"; i.e. >>>> the intent was to check both physical and virtual space available and >>>> be greedy, but not too greedy. >>>> >>>> There were some issues with that code for some platforms that e.g. >>>> didn't define a VM_KMEM_SIZE_MAX, but alc@ fixed that in r216425. >>> >>> >>> It is a 32-bit kernel, on 32-bit hardware. The values for memguard_fudge >>> are (defaults): >>> >>> tmp: 4246446080, vm_kmem_size: 117440512, vm_kmem_size_max: 0 >>> >>> When setting vm.kmem_size/vm.kmem_size_max to 2GB they are: >>> >>> tmp: 2147483648, vm_kmem_size: 214793648, vm_kmem_sizee_max: 2147483648 >>> (all >>> 2GB). >>> >>> But the start and map->max_offset remain the same on all runs I make. >> >> >> memguard_fudge is still broken for 32-bit architectures with no >> vm_kmem_max. In the absence of a km_max to limit the value, we >> essentially use twice the physical memory for the virtual limit. But >> with 2GB on a 32-bit machine, this requires 4GB of virtual space. >> >> Setting vm_kmem_size_max to 2GB should work; I'd expect to see >> tmp=about 200MB, which is much larger than the input 112MB but the >> allocation should work. But I don't really know what else PowerPC has >> need of for virtual space, so that still could be too large. >> >> You can try smaller values of vm_kmem_size_max, like 1GB or 512MB. >> You shouldn't need to set vm_kmem_size at all. At some point the >> added space for the memguard_map will be small enough that the >> kmem_suballoc will work. >> >> Hmm, what is the min_offset and max_offset of kernel_map when the call >> to memguard_fudge is made? >> >> Thanks, >> matthew > > > > Without setting vm.kmem_size/vm.kmem_size_max, I see the following: > > map: 0x1000000, min_offset: 0xD0000000, max_offset: 0xEFFFFFFF > > It does boot when I set vm.kmem_size=256M/vm.kmem_size_max=512M. > > When I tried 512M/1024M, it panicked at the same place -- kmem_suballoc from > kmeminit. So it looks like I have to set vm.kmem_size/vm.kmem_size_max way > back in order for it to boot with memguard(9). Please try the attached patch (or at http://people.freebsd.org/~mdf/memguard.diff). Thanks, matthew [-- Attachment #2 --] Index: kern/kern_malloc.c =================================================================== --- kern/kern_malloc.c (revision 238451) +++ kern/kern_malloc.c (working copy) @@ -739,17 +739,17 @@ kmeminit(void *dummy) * This allows for kmem map sparseness, but limits the size * to something sane. Be careful to not overflow the 32bit * ints while doing the check or the adjustment. */ if (vm_kmem_size / 2 / PAGE_SIZE > mem_size) vm_kmem_size = 2 * mem_size * PAGE_SIZE; #ifdef DEBUG_MEMGUARD - tmp = memguard_fudge(vm_kmem_size, vm_kmem_size_max); + tmp = memguard_fudge(vm_kmem_size, kernel_map); #else tmp = vm_kmem_size; #endif kmem_map = kmem_suballoc(kernel_map, &kmembase, &kmemlimit, tmp, TRUE); kmem_map->system_map = 1; #ifdef DEBUG_MEMGUARD Index: vm/memguard.c =================================================================== --- vm/memguard.c (revision 238451) +++ vm/memguard.c (working copy) @@ -154,49 +154,49 @@ SYSCTL_ULONG(_vm_memguard, OID_AUTO, fre &memguard_frequency_hits, 0, "# times MemGuard randomly chose"); /* * Return a fudged value to be used for vm_kmem_size for allocating * the kmem_map. The memguard memory will be a submap. */ unsigned long -memguard_fudge(unsigned long km_size, unsigned long km_max) +memguard_fudge(unsigned long km_size, const struct vm_map *parent_map) { - u_long mem_pgs = cnt.v_page_count; + u_long mem_pgs, parent_size; vm_memguard_divisor = 10; TUNABLE_INT_FETCH("vm.memguard.divisor", &vm_memguard_divisor); + parent_size = vm_map_max(parent_map) - vm_map_min(parent_map) + + PAGE_SIZE; /* Pick a conservative value if provided value sucks. */ if ((vm_memguard_divisor <= 0) || - ((km_size / vm_memguard_divisor) == 0)) + ((parent_size / vm_memguard_divisor) == 0)) vm_memguard_divisor = 10; /* * Limit consumption of physical pages to * 1/vm_memguard_divisor of system memory. If the KVA is * smaller than this then the KVA limit comes into play first. * This prevents memguard's page promotions from completely * using up memory, since most malloc(9) calls are sub-page. */ + mem_pgs = cnt.v_page_count; memguard_physlimit = (mem_pgs / vm_memguard_divisor) * PAGE_SIZE; /* * We want as much KVA as we can take safely. Use at most our - * allotted fraction of kmem_max. Limit this to twice the - * physical memory to avoid using too much memory as pagetable - * pages. - */ - memguard_mapsize = km_max / vm_memguard_divisor; - /* size must be multiple of PAGE_SIZE */ - memguard_mapsize = round_page(memguard_mapsize); - if (memguard_mapsize == 0 || - memguard_mapsize / (2 * PAGE_SIZE) > mem_pgs) + * allotted fraction of the parent map's size. Limit this to + * twice the physical memory to avoid using too much memory as + * pagetable pages (size must be multiple of PAGE_SIZE). + */ + memguard_mapsize = round_page(parent_size / vm_memguard_divisor); + if (memguard_mapsize / (2 * PAGE_SIZE) > mem_pgs) memguard_mapsize = mem_pgs * 2 * PAGE_SIZE; - if (km_max > 0 && km_size + memguard_mapsize > km_max) - return (km_max); + if (km_size + memguard_mapsize > parent_size) + memguard_mapsize = 0; return (km_size + memguard_mapsize); } /* * Initialize the MemGuard mock allocator. All objects from MemGuard come * out of a single VM map (contiguous chunk of address space). */ void Index: vm/memguard.h =================================================================== --- vm/memguard.h (revision 238451) +++ vm/memguard.h (working copy) @@ -30,17 +30,17 @@ #define _VM_MEMGUARD_H_ #include "opt_vm.h" struct malloc_type; struct vm_map; #ifdef DEBUG_MEMGUARD -unsigned long memguard_fudge(unsigned long, unsigned long); +unsigned long memguard_fudge(unsigned long, const struct vm_map *); void memguard_init(struct vm_map *); void *memguard_alloc(unsigned long, int); void *memguard_realloc(void *, unsigned long, struct malloc_type *, int); void memguard_free(void *); int memguard_cmp_mtp(struct malloc_type *, unsigned long); int memguard_cmp_zone(uma_zone_t); int is_memguard_addr(void *); #else Index: vm/vm_map.h =================================================================== --- vm/vm_map.h (revision 238451) +++ vm/vm_map.h (working copy) @@ -195,23 +195,23 @@ struct vm_map { /* * vm_flags_t values */ #define MAP_WIREFUTURE 0x01 /* wire all future pages */ #define MAP_BUSY_WAKEUP 0x02 #ifdef _KERNEL static __inline vm_offset_t -vm_map_max(vm_map_t map) +vm_map_max(const struct vm_map *map) { return (map->max_offset); } static __inline vm_offset_t -vm_map_min(vm_map_t map) +vm_map_min(const struct vm_map *map) { return (map->min_offset); } static __inline pmap_t vm_map_pmap(vm_map_t map) { return (map->pmap);help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMBSHm99OWq4sK-Wamnfy%2B_W5rJ-2WCQ24kGLFTK%2BQqfQzDs-Q>