From owner-freebsd-hackers@freebsd.org Wed Jul 13 10:39:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C3F6B97393; Wed, 13 Jul 2016 10:39:00 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from mail-oi0-x243.google.com (mail-oi0-x243.google.com [IPv6:2607:f8b0:4003:c06::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24DEA1A22; Wed, 13 Jul 2016 10:39:00 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: by mail-oi0-x243.google.com with SMTP id c199so3944315oig.1; Wed, 13 Jul 2016 03:39:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=bWChwR9es9dvNxIfy7Tttb3oZYqpc0oDXEkeFA0cpHY=; b=nDMp41pqTWVlQKt3tekoeWUFHesZT3vjHb/EkmiCuGy7javiB7VuCh42wZ68Tgt74Y aq7npJbavp4+06fimamJx4y98/jJBF49VqRnbQqpmwhZbjIR+x1F8R+gAXmGVB4+hOip 5wIAgE1sx/epSMexF4aRJ4+vcQ3GJofcVgn7wS2iC8GyUzgdTXJpuVO+P7LE1kSobOmO HtZh0pT3Iv+mKwXrSykH9HJ6VfUK3K7977iaOlmoCD448fACvokKyAYxiX1F6oviUaqH A3BlRnKgajTdTbvfN0fRKRPdNtXIzkfro5jkGXZq2sAae8Jmwk7/peurdVHGM2Id+iqa yHcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=bWChwR9es9dvNxIfy7Tttb3oZYqpc0oDXEkeFA0cpHY=; b=PlattCT+DZdjO2RMQ2G7KOcRrBp0cSwNWdUIaXQXoh/FDirBvUFvXARUs0xVxpzMUd 1cQuadBgFq0tCIJ5sMmtgB5c8s4ox73L9psb3tlftA5TpWm6D8ZDo0jHLDO35p5Fac8p 9HGDKp5INQCwZ/7GQA/bC4ZawY3UWe0u+8VC3kZiyEdtyHtXsxmGLGkZ2PWJ4bnhdQsA kRifp4LcH5Ixd74sD3hx/NCXkvl2tjwHRyGaUTJEBleuRxKm9hWBbjfZLbmIwTUvr/nv GUvGAuO8DorxVKW9XwbI6ltsc0+kuEecjK7egMFJUSf5GHZzs8dirIgd+qz1gqTc1YZL NCmw== X-Gm-Message-State: ALyK8tICIJO8X22LjMUhWoIWFB8ZFTc4rUL8t8DZoOcia5e3hG33lv/ruHiTT31lkY0Gw11EiTCHGOQbAKvtrw== X-Received: by 10.202.74.9 with SMTP id x9mr732298oia.165.1468406339097; Wed, 13 Jul 2016 03:38:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.32.17 with HTTP; Wed, 13 Jul 2016 03:38:58 -0700 (PDT) From: Shrikanth Kamath Date: Wed, 13 Jul 2016 03:38:58 -0700 Message-ID: Subject: rogue module allocating 4G wired memory / pmap_growkernel does vm_page_alloc to get 4G wired To: freebsd-hackers@freebsd.org, freebsd-amd64@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jul 2016 10:39:00 -0000 I have this broadcom sdk module while getting loaded by a rc script calling kldload triggers a 4G jump in the wired memory (as shown by output of top) # top ... ... Mem: 77M Active, 644M Inact, 4473M Wired, 1600K Cache, 606M Buf, 2734M Free ... Upon using dtrace I get the responsible stack trace as this one which does most vm_page_alloc 0 10310 vm_page_alloc:entry kernel`pmap_growkernel+0x1cc kernel`vm_map_insert+0x3ee kernel`vm_map_find+0x14d kernel`link_elf_load_file+0x845 kernel`linker_load_module+0x6ec kernel`kern_kldload+0xab kernel`sys_kldload+0x84 kernel`amd64_syscall+0x3b6 kernel`0xffffffff805413a7 And the number of page allocations that happen in this context does also sum up to around 4G. It showed the total page allocations that happened in this context was vm_page_alloc 1045039 which is approx 3.99G. On probing a little deeper, the link_elf_load_file calls for vm_map_find with mapspace of 20MB. This results in a call to vm_map_findspace which I believe first checks if it can do a contiguous allocation within the kernel_map else it goes trying to map it beyond the kernel_map. Probing further we see that before the call to vm_map_find the kernel_vm_end is at the default value of kernbase at 0xffffffff80000000 The vm_map_findspace then picks the new start address to be 0xffffffff83d68000. The call to pmap_growkernel does grow the number of kernel page table entries to match the new end address i.e 0xffffffff85136000 and thereby kernel_vm_end is also adjusted to 0xffffffff85200000 from the initial default value of 0xffffffff80000000. Is my understanding correct here that vm_map_find is unable to allocate the 20MB mapspace requested in the kernel_map and hence calls pmap_growkernel thereby resulting in the wired memory increase? -- Shrikanth R K