From owner-freebsd-arch@freebsd.org Sun Nov 22 04:41:16 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4DCEA32BDF for ; Sun, 22 Nov 2015 04:41:16 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-lb0-x235.google.com (mail-lb0-x235.google.com [IPv6:2a00:1450:4010:c04::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E3AD12EA; Sun, 22 Nov 2015 04:41:16 +0000 (UTC) (envelope-from artemb@gmail.com) Received: by lbbkw15 with SMTP id kw15so80739460lbb.0; Sat, 21 Nov 2015 20:41:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EEVJy4aG/EYy3oWYchSf1yIpzFZx38Vqnzn09IUxH0I=; b=KKCoc0W51olntd6bGBE8BMqTeXgZnIOginrFg+rQFsCKC59H+29GxjqWoqAhNEjlkE Vnf6OBCxLfcfPqmMm0329tP9z1i6IRpMoC9gwqs465falbKxvuftsh+AaNH4FP3b1SHv 9S7w8iWxE8nWcHoSagYKimyMn5MzcYbIz49VMpSVZj1V6QvdDvdDXRnsjUlK6wDQoudF C1R/jaA/7pVNyYCOkiVDJYntChnP6hPFHyVfTH43GGCExNRNSdRfTs80ECQlsZ0uLsyk kAkQgrnf/6U1SnmUX19Gq/jVDh/kYJyPqC7NNtn/qthFOjzwDBJ2jzf3iBPkdtUuW/kS f7hw== MIME-Version: 1.0 X-Received: by 10.112.198.106 with SMTP id jb10mr8659142lbc.111.1448167273712; Sat, 21 Nov 2015 20:41:13 -0800 (PST) Sender: artemb@gmail.com Received: by 10.25.207.1 with HTTP; Sat, 21 Nov 2015 20:41:13 -0800 (PST) In-Reply-To: <20151122024542.GA44664@wkstn-mjohnston.west.isilon.com> References: <20151122024542.GA44664@wkstn-mjohnston.west.isilon.com> Date: Sat, 21 Nov 2015 20:41:13 -0800 X-Google-Sender-Auth: JwsxpNg6fjtWZSIxnI96fXyTOug Message-ID: Subject: Re: zero-cost SDT probes From: Artem Belevich To: Mark Johnston Cc: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2015 04:41:16 -0000 On Sat, Nov 21, 2015 at 6:45 PM, Mark Johnston wrote: > Hi, > > For the past while I've been experimenting with various ways to > implement "zero-cost" SDT DTrace probes. Basically, at the moment an SDT > probe site expands to this: > > if (func_ptr != NULL) > func_ptr(); > > I wonder how much of an overhead that currently adds. Do you have any benchmark numbers comparing performance of no SDT, current SDT implementation and "zero-cost" one. --Artem When the probe is enabled, func_ptr is set to dtrace_probe(); otherwise > it's NULL. With zero-cost probes, the SDT_PROBE macros expand to > > func(); > > When the kernel is running, each probe site has been overwritten with > NOPs. When a probe is enabled, one of the NOPs is overwritten with a > breakpoint, and the handler uses the PC to figure out which probe fired. > This approach has the benefit of incurring less overhead when the probe > is not enabled; it's more complicated to implement though, which is why > this hasn't already been done. > > I have a working implementation of this for amd64 and i386[1]. Before > adding support for the other arches, I'd like to get some idea as to > whether the approach described below is sound and acceptable. > > The main difficulty is in figuring out where the probe sites actually > are once the kernel is running. In my patch, a probe site is a call to > an externally-defined function which is defined in an > automatically-generated C file. At link time, we first perform a partial > link of all the kernel's object files. Then, a script uses the relocations > against the still-undefined probe functions to generate > 1) stub functions for the probes, so that the kernel can actually be > linked, and > 2) a linker set containing the offsets of each probe site relative to > the beginning of the text section. > The result is linked with the partially-linked kernel to generate the > final kernel file. > > During boot, we iterate over the linker set, using the offsets plus the > address of btext to overwrite probe sites with NOPs. SDT probes in kernel > modules are handled differently (and more simply): the kernel linker just > has special handling for relocations against symbols named __dtrace_sdt_*; > this is how illumos/Solaris implements all of this. > > My uncertainty revolves around the use of relocations in the > partially-linked kernel to determine the address of probe sites in the > running kernel. With the GNU ld in base, this happens to work because > the final link doesn't modify the text section. Is this something I can > rely upon? Will this assumption be false with the advent of lld and LTO? > Are there other, cleaner ways to implement what I described above? > > Thanks, > -Mark > > [1] https://people.freebsd.org/~markj/patches/sdt-zerocost/ > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >