From owner-freebsd-hackers@freebsd.org Thu Nov 17 19:42:13 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A64A4C47B34 for ; Thu, 17 Nov 2016 19:42:13 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69B6E1FE3 for ; Thu, 17 Nov 2016 19:42:13 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x22f.google.com with SMTP id y23so73749633itc.0 for ; Thu, 17 Nov 2016 11:42:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=roqciExN4tsxiaXO0Z/d9ytQgxcqa0z7Opdl1Cw3U+0=; b=jIsJHUWrGq06Oehtl+KsjjBkHi/6F0VAi6EeHb6psgIC2w5nI2RLV5gW7333yGmlLo YRv7ih/F9VtOgVyrI+fenTvA4APdCWNU/iJH7PyaEm8i+ZAeCxgr2nucPvTvNfKW45QH LdJNhl7OmC2NdJGZ1ymBNbQyHwB+HRYPhCp06PVgeyHf69KqFhSdthwyFFAh9C8VfJI4 jH9te+I3/IBeKDYLPkxc4l8jnK2kJEunGOpOV9gICtW/gxp0lJB6s07BS9DPwxLqYKKQ sfiMljXdOPS9Sn3UgvoqdYBOUvVmWuSZrEmaHsuwjU8PTHzaFK4yPRAa5bMe16BX2ZDU cWUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=roqciExN4tsxiaXO0Z/d9ytQgxcqa0z7Opdl1Cw3U+0=; b=k+zNUnP0gRERpdZ7h4M1zstZrd9xwC9HT5IJ7RJeUpXwwKYN8LsSPa6dGdca+E3yDv 8yAXtAqCi+rXvpOuYOJdVDAHsjSaJdUDd8PfO56JPUB1/piGcuVW8mU0KfZhR95q0Wx7 CW1dG0ljdoV54h072+wGvW3J+ytQ7c58LnHG1qQc9HtqUfWToslIG72ORig7DAn0gSd1 UXxKVjUmk1qNZh9X++X1kTOnOELB8v+Ix8cuYqSsJ/YgDO+jXQQw9Rr9scChx4+5hPcP ymG0NAmDsxalZTTZroIH4yL5CPik2Fl2bR86559MoYayaAiFnoqy8mj3R7a33w9OGTsc kNzg== X-Gm-Message-State: ABUngveEVmizehMlSzffk2PhlsTULkPiNUeW2TII7pCNj30Pzwl9SoekFCAdvQr5SZK3Uqea0cHU+e4/+arP+A== X-Received: by 10.107.162.204 with SMTP id l195mr4799724ioe.169.1479411732518; Thu, 17 Nov 2016 11:42:12 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.27.212 with HTTP; Thu, 17 Nov 2016 11:41:52 -0800 (PST) In-Reply-To: References: <20161110012624.GA23701@lonesome.com> <20161110215549.GL91607@kduck.kaduk.org> From: Ed Maste Date: Thu, 17 Nov 2016 14:41:52 -0500 X-Google-Sender-Auth: 6xQKyFOe7mTUcCZb2PkvbQDTKZw Message-ID: Subject: Re: sbrk(0) replacement for memory resource tracking? To: Matthias Andree Cc: Benjamin Kaduk , "freebsd-hackers@FreeBSD.org" , Mark Linimon Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Nov 2016 19:42:13 -0000 I seem to be missing the beginning of this thread, so pardon the lack of context in quotes below. >>> To me it looks like the sbrk() function is going away from our base >>> system underneath a stable 11-* branch. If that is true, I'll have to >>> object to that and request sbrk() be put back, we add a deprecation >>> notice now (if necessary via errata notice) and pull it only from FreeBSD12. Stable/11 never provided sbrk on arm64, and sbrk is not in the first FreeBSD release to support arm64, 11.0. Right now nothing changes for existing architectures, which continue to provide sbrk. >>> OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and >>> only to track its resource usage. I'll be happy to help porting to >> Same for zephyr. >> >>> something else that serves the same purpose, aka "how much memory am I >>> using" - but what would that be? >> I think there isn't really a drop-in replacement. N.B. that the number >> from sbrk(0) has been meaningless for quite some time, since jemalloc >> uses mmap to get more space. > OK. So the quick and dirty way to re-enable e2fsprogs on those > architectures whilst scrapping any memory statistics would be to #define > sbrk(a) (a) which would just invalidate stats, providing the application > handles bogus data. Indeed, and that's no worse than today: applications calling sbrk(0) to measure memory use are already handling bogus data. > Other than that, it would seem that mallctl("epoch", ...) to synch up > stats, and mallctl("stats.active", ...) or perhaps or "stats.mapped" > gets me close to what comparing sbrk(0) over process lifetime would have > achieved, wouldn't it? This is assuming sbrk() had page granularity > anyhow and stats.active provides exactly that (gross memory allocated). > Possibly this also wants mallctlnametomib and mallctlbymib for > optimization if called often. Right? That seems broadly sensible to me, and I think it will be useful to have a few worked examples to demonstrate collection of memory stats with jemalloc.