From owner-freebsd-arch@freebsd.org Mon Nov 23 00:15:13 2015 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0367A3571E for ; Mon, 23 Nov 2015 00:15:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 929E51F79 for ; Mon, 23 Nov 2015 00:15:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by pacej9 with SMTP id ej9so173043751pac.2 for ; Sun, 22 Nov 2015 16:15:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=E8GGbVh+L7CfH4pLlm38YTFAVMzBfSyrvvhKk4GN2Hk=; b=cYjufNk3O4wJ/nNulORVqbAGyHxEk6676a5I9sFjg99sGZtk3pfkc1chUJr+0BXrOW XYS9pWwgVz1qPYPIoGY4ASMbd3cpXj0x+EwpD+UDfl2vWVWPRZ+aMHv0notN6K4Gd37s EQm2JtNwvig4X1glQhNcnnc1IITukLYEncs0D6aMx3/wTXCRDg2csTKn5UzljpBBR2fY +9LStEmMvicKHxX0kl1QTNnSlPqD8D/YfUiGVqNWNGLQUVcCukpOnUPSHKqdABrtQslU 669Nm3aNqKpGofdxwFQCcvlggnj98vwRSttN3b1JmxUldfem9LWfMuSFdVf8GFyDqNaE TAew== X-Received: by 10.68.224.106 with SMTP id rb10mr22513812pbc.17.1448237713226; Sun, 22 Nov 2015 16:15:13 -0800 (PST) Received: from raichu ([104.232.114.184]) by smtp.gmail.com with ESMTPSA id c1sm7921012pas.1.2015.11.22.16.15.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Nov 2015 16:15:12 -0800 (PST) Sender: Mark Johnston Date: Sun, 22 Nov 2015 16:15:11 -0800 From: Mark Johnston To: "Simon J. Gerraty" Cc: freebsd-arch@freebsd.org Subject: Re: zero-cost SDT probes Message-ID: <20151123001511.GB5647@raichu> References: <20151122024542.GA44664@wkstn-mjohnston.west.isilon.com> <2753.1448173777@chaos> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2753.1448173777@chaos> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Nov 2015 00:15:13 -0000 On Sat, Nov 21, 2015 at 10:29:37PM -0800, Simon J. Gerraty wrote: > Mark Johnston wrote: > > For the past while I've been experimenting with various ways to > > implement "zero-cost" SDT DTrace probes. Basically, at the moment an SDT > > probe site expands to this: > > Would it be feasible to compile the probes into the kernel > as active calls to a registrar function? > That would eliminate all the complexity of finding PC's > though you'd probably need to pass extra args to convey the point of the > probe? > > It would hurt boot time a little too - each probe point would make a > call to register itself (and get overwritten with nops as a reward) but > very simple? I considered such an approach but didn't pursue it for a few reasons: - We'd have to pass a unique probe site identifier as an argument, which requires at least one extra instruction at the probe site. - If the probe site is a tail call, how can the registrar find the correct caller? - If a probe site isn't patched until multiple CPUs have started, how do we safely overwrite the call site in the face of the possibility that another thread is executing the call at the same time? When it comes to enabling or disabling a probe, we only need to write a single byte, but overwriting multiple bytes seems unsafe. I think the last point could possibly be addressed by overwriting the first byte of the call with a breakpoint before overwriting the rest of the call site with NOPs, using the breakpoint handler to fix up any threads that reached the probe site as it was being modified. But this detracts a bit from the simplicity of the approach.