From owner-freebsd-hackers@freebsd.org Thu Nov 10 22:27:03 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26B64C3B91E for ; Thu, 10 Nov 2016 22:27:03 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9549CE73 for ; Thu, 10 Nov 2016 22:27:01 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mandree.no-ip.org ([77.182.233.153]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M6ioC-1cypiX3g16-00wXgf; Thu, 10 Nov 2016 23:26:42 +0100 Received: from [IPv6:::1] (localhost6.localdomain6 [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id D42BC23D87C; Thu, 10 Nov 2016 23:26:37 +0100 (CET) Subject: Re: sbrk(0) replacement for memory resource tracking? To: Benjamin Kaduk References: <20161110012624.GA23701@lonesome.com> <20161110215549.GL91607@kduck.kaduk.org> Cc: "freebsd-hackers@FreeBSD.org" , Mark Linimon From: Matthias Andree Message-ID: Date: Thu, 10 Nov 2016 23:26:37 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161110215549.GL91607@kduck.kaduk.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K0:wZOcRZzytzSMrDqUaNAFgPfZYaSz7LrmAGN+KqmQQX4aM136Z6r v6pBPxybkDnijjjunQp6CD1+JfqdACKboV+x5aQDxxS9avXJOfdyvGRVu+kWUBsO7irCEic ZJK4KhkHy8bEAadqYFM8e9yFRy/haMsCTYYM55bkxKkDqWKV6sQjHUQ9ZmV4RTVucC6u11A TDtCseQ6xMGIZAF0BkPHw== X-UI-Out-Filterresults: notjunk:1;V01:K0:u9HThrfUORk=:MavIeAZMCotgofVMXg5GB6 +lfL61R2s+I1Qla6M75YkIBQC3U2IsLfP2nqrT97ZQOtNdfxk2hx+beZ84Bh6vI7JiodEwlBr 3okox19mJJ1GWxzwZJckV7vtNKT34pQNVFCB1J9xXjEf86Un9NjN7cRvSthJuprNckNdvt4z5 fPy9tLQF4DMKro95Dzz8J5QvFOyVXu7b/fVoG2w0rVD3urFEG74UPYioCfQLRC0DnOMEsGQEw zC1G8YcfF2JEQatSCzVge/xDabEtFilG8HJUZhZqnDfwqrgrUzojE6bp6jjtCJb5GxqoU49kM QfUZ8P4x7czEIydJ5k1gcB+e8f8/a06MApgV9h4+LdxVXCt0NfTqzxbE8mfZQTf1QHstCN5l6 WwGTM5QKZXnB1C+G5HNDqj4/csYXlIa3CCPFJ3PV5S5MtlBnJFSGyIQZW1nJXd9TW3sSMOeLI q163x+rTdNpK2l6ZjViqDBSuZTzdw+fDjBHF2ubpNGL1kcrOS6KsU086dNu3gj1q48H/n9WsA PdAtAitR+dzbgEkuPZ3C8JOHrk2oZ3KbG3H+61VPRxNiqaf4TxHmEWuktBtVBjDrqo6B+ojkX WB6qEk/isFAAQL2lQRf36lS9FEZAP4UZweqy0V+FOL/IByOrWPD+uQp1IZMn0f/oQvZR0jw2e AO/oFnjoHJl70qcLnKWS3TpcfmObT0z71P233WHdz4nL/CROVWERiv4X4eBG6YR9BYLW3AsAx ANnXvQho2kJloFJ+qCK37LvFsU67wCLL+v7DgYPdwR1hIf5Fi3NFU0wtMfEjI6/i+KXasj/Tr 6rvvOeR X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 22:27:03 -0000 Am 10.11.2016 um 22:55 schrieb Benjamin Kaduk: > On Thu, Nov 10, 2016 at 10:21:18PM +0100, Matthias Andree wrote: >> Am 10.11.2016 um 02:26 schrieb Mark Linimon: >>> FYI. Unfortunately I do not know what the generic fix is yet. But a= t >>> least this will prevent the package builders from wasting time right = now. >>> >>> I will try to keep the following page updated as I learn more: >>> >>> https://wiki.freebsd.org/PortsBrokenWithSbrk >>> >>> (oops, I forgot I have not put in the proper logfile URLs yet. Let m= e >>> get started on that.) >>> >>> mcl >> Please help me understand the issue, and if by adding one or two >> introductory paragraphs to the Wiki. > > Looks like r300303 is the relevant one for aarch64 (RISC-V got a simila= r treatment > later?). > >> To me it looks like the sbrk() function is going away from our base >> system underneath a stable 11-* branch. If that is true, I'll have to= >> object to that and request sbrk() be put back, we add a deprecation >> notice now (if necessary via errata notice) and pull it only from Free= BSD12. > At the time I somehow convinced myself that it had never been in a stab= le > release and was thus okay, but maybe I'm misremembering. Hmm, or maybe= it > is okay for a tier-2 architecture [in the mind of the committer, not ne= cessarly > me, to be clear]. > >> OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and= >> only to track its resource usage. I'll be happy to help porting to > Same for zephyr. > >> something else that serves the same purpose, aka "how much memory am I= >> using" - but what would that be? > I think there isn't really a drop-in replacement. N.B. that the number= > from sbrk(0) has been meaningless for quite some time, since jemalloc > uses mmap to get more space. OK. So the quick and dirty way to re-enable e2fsprogs on those architectures whilst scrapping any memory statistics would be to #define sbrk(a) (a) which would just invalidate stats, providing the application handles bogus data. Other than that, it would seem that mallctl("epoch", ...) to synch up stats, and mallctl("stats.active", ...) or perhaps or "stats.mapped" gets me close to what comparing sbrk(0) over process lifetime would have achieved, wouldn't it? This is assuming sbrk() had page granularity anyhow and stats.active provides exactly that (gross memory allocated). Possibly this also wants mallctlnametomib and mallctlbymib for optimization if called often. Right?