From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 10 18:05:22 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DF54832E; Thu, 10 Jan 2013 18:05:22 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4DF7AD9F; Thu, 10 Jan 2013 18:05:22 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r0AI5ETu042504; Thu, 10 Jan 2013 20:05:14 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r0AI5ETu042504 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r0AI5EUH042503; Thu, 10 Jan 2013 20:05:14 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 10 Jan 2013 20:05:14 +0200 From: Konstantin Belousov To: Alfred Perlstein Subject: Re: malloc+utrace, tracking memory leaks in a running program. Message-ID: <20130110180514.GS2561@kib.kiev.ua> References: <50D52B10.1060205@mu.org> <50EE6281.7030602@mu.org> <50EE6630.2010902@mu.org> <20130110073854.GQ2561@kib.kiev.ua> <50EEDB5E.2010906@mu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fqIB0bRxfTYxTb/F" Content-Disposition: inline In-Reply-To: <50EEDB5E.2010906@mu.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: Daniel Eischen , hackers@freebsd.org, Jason Evans X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jan 2013 18:05:22 -0000 --fqIB0bRxfTYxTb/F Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 10, 2013 at 10:16:46AM -0500, Alfred Perlstein wrote: > On 1/10/13 2:38 AM, Konstantin Belousov wrote: > > On Thu, Jan 10, 2013 at 01:56:48AM -0500, Alfred Perlstein wrote: > >> Here are more convenient links that give diffs against FreeBSD and > >> jemalloc for the proposed changes: > >> > >> FreeBSD: > >> https://github.com/alfredperlstein/freebsd/compare/13e7228d5b83c8fcfc6= 3a0803a374212018f6b68~1...utrace2 > >> > > Why do you need to expedite the records through the ktrace at all ? > > Wouldn't direct write(2)s to a file allow for better performance > > due to not stressing kernel memory allocator and single writing thread ? > > Also, the malloc coupling to the single-system interface would be > > prevented. > > > > I believe that other usermode tracers also behave in the similar way, > > using writes and not private kernel interface. > > > > Also, what /proc issues did you mentioned ? There is > > sysctl kern.proc.vmmap which is much more convenient than /proc/pid/map > > and does not require /proc mounted. > > > >> jemalloc: > >> https://github.com/alfredperlstein/jemalloc/compare/master...utrace2 > >> >=20 > Konstantin, you are right, it is a strange thing this utrace. I am not= =20 > sure why it was done this way. >=20 > You are correct in that much more efficient system could be made using=20 > writes gathered into a single write(2). Even without writes gathering, non-coalesced writes should be faster than utrace. >=20 > Do you think there is any reason they may have re-used the kernel paths= =20 > for ktrace even at the cost of efficiency? I can only speculate. The utracing of the malloc calls in the context of the ktrace stream is useful for the human reading the trace. Instead of seeing the sequence of unexplanaible calls allocating and freeing memory, you would see something more penetrable. For example, you would see accept/malloc/read/write/free, which could be usefully interpreted as network server serving the client. This context is not needed for a leak detector. >=20 > About kern.proc.vmmap I will look into that. >=20 > -Alfred --fqIB0bRxfTYxTb/F Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQ7wLZAAoJEJDCuSvBvK1Be24P/3rsEBB5K2R1rGoy+N+GYuHa QFmlyug+gRwHk9B/kiu2T4oVvRU8c0TqQtOacD1Y3lNIPDkTQ0P4zjMGf8zSfg6M QEnuNVBFo/29Tc/Eg5walNGogTEyZU1p7fKPNavaSHYoPwuoY8YaPEJY3h6De7rh gg2dAMbzpGK0jMdWM/3x1pKr+Adaz3/QDLZde2OTrJOYlNH5+jC7JjE5MFAJYIsj gVVsNDksETctHWcRgA4eQ7E9dN5gMyR2+sbGYAF/Xiw2ywjvhxCG5HvEPlPE5hK6 aAtGyy7+e/rlTGQoixdkgIp9cGyq+pJJ3+bqyQ84RIIKuvfNO+jR5ipMhVSsP3Tj 5SiDqGg7/ME68fCWd7RtOWu4PImRF/jBkzkLl7msEeOonED076hDpEu2sqNRECae vucD+eJj86sZxQ7jovmJWRrTUibWnd1EbkX9iKoN2atRebYKvW49jB32Uyv2ziXm w8wVRg8km2dC7qwnlbOgdzBkV4Vh9QXF2Ay0xwtfojhyvhd86mH2y7j/PLfAHlQH WcSer8FgJ/xR/VAMPVThYrUJmZfp5y9sKPAHqQMHqOU+ZQTwYBdDRtpjY/j3cTBK QwfUaaFf3t5Fo4M48Cymh8phsUoe/WTvjcwVUKAsjAws3Hj9KLM8OgxDaGQhM48f pQieI/nVrd12o6AaE2fm =B9Xf -----END PGP SIGNATURE----- --fqIB0bRxfTYxTb/F--