From owner-freebsd-arch@FreeBSD.ORG Thu Jul 29 17:01:40 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6699B1065676 for ; Thu, 29 Jul 2010 17:01:40 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id 365D38FC25 for ; Thu, 29 Jul 2010 17:01:39 +0000 (UTC) Received: by pxi8 with SMTP id 8so216812pxi.13 for ; Thu, 29 Jul 2010 10:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=KIrV2jo9MaNakAL1H67VVIzBfeBmPDIuDwku9dc8UnM=; b=ZEDzGXc3J6LzSxkUbYNwTirN0R/W7tzREOYl7NtHPRaSVkjKlT0c/3BVntN+NOSKYf RDznAeXARCZAoeBQSdWRrBpruD6pj07lVR7vwqW26irI2p56F0ZJEKH+rZ3YgJ8z26Tf aPxsJwXCQkMUucB6tathCYWAN2BwHH5I7LpY8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=Vjs3wmxJcHfQmQyw1yw+5mjkVNUTA5qJ4G6ViZrRPm+yU56kvYHOZvLWrkME+Dg5gQ rjBLNeYujBan59JNAb1bqRx5YH3arxqzCvM9jCjicn/2/o24OWdIXPvkmtJTN63SSt/E nKGTefgHBDlbZva1rWbZkXp/eyXNSMo2hFCzY= MIME-Version: 1.0 Received: by 10.142.153.8 with SMTP id a8mr381059wfe.272.1280422899466; Thu, 29 Jul 2010 10:01:39 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.42.6.85 with HTTP; Thu, 29 Jul 2010 10:01:39 -0700 (PDT) Date: Thu, 29 Jul 2010 10:01:39 -0700 X-Google-Sender-Auth: QdJZyDriMqxt9m0PI0t2dIVklQs Message-ID: From: mdf@FreeBSD.org To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: dwmalone@maths.tcd.ie, alc@freebsd.org, iedowse@freebsd.org Subject: memguard(9) rewrite, part 2 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Jul 2010 17:01:40 -0000 Back in March I asked about interest in a memguard(9) redo. I've had the time to get the code to a place I'm pretty happy with, and we've successfully used it at work without running into some of the resource limitations that the original memguard(9) gave. http://people.freebsd.org/~mdf/bsd-memguard.diff The gist of the new implementation is to reserve a lot of KVA for memguard(9) to use, and then to avoid re-using KVA as long as possible. Rather than keep the physical pages around, though, on free(9) the pages are returned to the system. The KVA is allocated using vm_map_findspace() from a current pointer into the memguard_map, which is incremented until the end of the map is encountered, at which time it wraps. This is a "free" way to avoid re-use of KVA as long as possible; any other scheme requires more than O(1) data to track what has been used. I've limited the KVA to 2x ram size, and also limited the physical memory that memguard(9) can take to vm_memguard_divisor fraction of physical memory (instead of limiting both KVA and physical to vm_memguard_divisor as the original code did). This patch also allows for tweaking which malloc type is guarded at run time, will randomly guard allocations of any type if requested, has a knob to always guard allocations of PAGE_SIZE or larger since it won't waste any memory, will optionally add guard pages of unmapped KVA at the beginning and end of the allocation to catch overruns more easily, and also can impose minimum allocation sizes on guarded memory so that the page promotions don't waste too much space. Assuming alc@ is happy with the VM changes and no one has any further suggestions, I'd like to commit this some time next week. I'd also like to MFC to stable/8 and stable/7 since this patch doesn't introduce any KBI/ABI/KPI/API changes. Apart from the general desire to have production systems run as fast as possible, I'd really like more tools like memguard(9) to be always-on, to help catch bugs the first time instead of requiring multiple recreates. Thanks, matthew