From owner-freebsd-arch@freebsd.org Mon Dec 19 19:39:39 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 24535C87524; Mon, 19 Dec 2016 19:39:39 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73A50187B; Mon, 19 Dec 2016 19:39:38 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail-d.allbsd.org (p2027-ipbf1605funabasi.chiba.ocn.ne.jp [123.225.191.27]) (authenticated bits=56) by mail.allbsd.org (8.15.2/8.15.2) with ESMTPSA id uBJJdEtQ006323 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) (Client CN "/OU=GT07882699/OU=See+20www.rapidssl.com/resources/cps+20+28c+2915/OU=Domain+20Control+20Validated+20-+20RapidSSL+28R+29/CN=*.allbsd.org", Issuer "/C=US/O=GeoTrust+20Inc./CN=RapidSSL+20SHA256+20CA+20-+20G3"); Tue, 20 Dec 2016 04:39:34 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from alph.allbsd.org (alph.allbsd.org [192.168.0.10]) by mail-d.allbsd.org (8.15.2/8.15.2) with ESMTPS id uBJJbxEg049627 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Dec 2016 04:37:59 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.15.2/8.15.2) with ESMTPA id uBJJbuSp049616; Tue, 20 Dec 2016 04:37:59 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Tue, 20 Dec 2016 04:36:46 +0900 (JST) Message-Id: <20161220.043646.1181938468712455328.hrs@allbsd.org> To: freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org Reply-To: freebsd-arch@FreeBSD.org Subject: RFC: DTrace probes for debugging or testing in userland programs From: Hiroki Sato X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.7 on Emacs 25.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Tue_Dec_20_04_36_46_2016_787)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.99 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.4.3 (mail.allbsd.org [133.31.130.32]); Tue, 20 Dec 2016 04:39:35 +0900 (JST) X-Spam-Status: No, score=-99.4 required=13.0 tests=CONTENT_TYPE_PRESENT, FAKEDWORD_BACKQUOTE,QENCPTR1,USER_IN_WHITELIST autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on gatekeeper.allbsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 19:39:39 -0000 ----Security_Multipart(Tue_Dec_20_04_36_46_2016_787)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi, I am trying to rewrite userland programs (especially daemons) to support userland DTrace probes to make it possible to trace the behavior by using dtrace(1). The purpose is to provide a consistent interface to enable/collect debug log and show internal states. A lot of daemons define their own debug output function like dprintf() and there is no consistency of where to be recorded. I would like your comment about this change because I want to know whether this change (or direction) is acceptable or not. I put an example by using syslogd at the following URL: (diff) https://people.freebsd.org/~hrs/syslogd_usdt.20161220-1.diff (tarball) https://people.freebsd.org/~hrs/syslogd_usdt.20161220-1.tar.gz You can try to compile a new syslogd, run it, and then attach dtrace(1) to the syslogd process by "dtrace -q -CI./ -s ./syslogd_trace.d -p `pgrep syslogd`" in the same directory. Basically this change is invisible for normal users. This includes some rough edges but I think it is sufficient to understand the concept. I do not intend to commit this soon. Questions from me are the following: 1. Where should foo_probes.d and foo_trace.d be installed? And if they depend on foo.h, where should foo.h be? 2. Is documenting probes into foo.8 reasonable? The details are as follows. * Rewrite example This rewrite is twofold. First, simply replace a call of the logging function into DTrace USDT like this: (old) | dprintf("an error occurred, errno=%d\n", errno); (new) | FOO_LOG(LOG_ERR, "an error occurred, errno=%d\n", errno); And then if it involves a result of a specific operation, replace trace probes for the simple logging with dedicated ones: (old) | error = bind(s, res->ai_addr, res->ar_addrlen); | if (error) { | dprintf("bind failed, errno=%d\n", errno); | close(s); | return (-1); | } (new) | error = bind(s, res->ai_addr, res->ar_addrlen); | if (error) { | FOO_SOCK_BIND_FAILED(errno, s, res->ai_addr, res->ar_addrlen); | close(s); | return (-1); | } | FOO_SOCK_BIND_SUCCESS(s, res->ai_addr, res->ar_addrlen); and implement the functionality of the original log message in D script: | foo$target:::sock-bind-failed | { | printf("bind failed, errno=%d", arg0); | /* The other argN can be used to report in more detail. */ | } * Additional files in the existing directory layout After this rewrite, the directory layout for a daemon "foo" will look like the following: | Makefile | foo.8 | foo.c | foo.h | foo_probes.d | foo_trace.d foo_probes.d and foo_trace.d are added. The former is the definition of USDT probes, and the latter is a D script example to reproduce the original debug log by dprintf() or something like that. A section to describe what probes are available is added into foo.8. One can trace the foo daemon using "dtrace -Cs foo_trace.d -p `pgrep foo`" on runtime, and also can create own script. foo.h may be added because foo_probes.d and foo_trace.d often require information of data structure used in foo.c. * Possible incompatible change A debug flag to activate additional logging is no longer necessary after this rewrite, so we can remove it (-d flag in the case of syslogd). And dump of the internal state can be implemented as a SIGINFO handler. In the syslogd example, SIGINFO dumps syslogd configuration and access control list. -- Hiroki ----Security_Multipart(Tue_Dec_20_04_36_46_2016_787)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlhYNs4ACgkQTyzT2CeTzy3utQCfRQ7RboCA4kENZfoHYr7xdiYL HNAAoJJC1PpDrIS27+quyzUKD9FdwIDO =OeQe -----END PGP SIGNATURE----- ----Security_Multipart(Tue_Dec_20_04_36_46_2016_787)---- From owner-freebsd-arch@freebsd.org Mon Dec 19 20:27:33 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 978F1C8891A; Mon, 19 Dec 2016 20:27:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wm0-x22d.google.com (mail-wm0-x22d.google.com [IPv6:2a00:1450:400c:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3047819A8; Mon, 19 Dec 2016 20:27:33 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wm0-x22d.google.com with SMTP id a197so103305637wmd.0; Mon, 19 Dec 2016 12:27:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HOjDF+E0nEZuBfG11fvtIUGM/mFXeuCvjZH+TDYxm9w=; b=nx1/Jt8dq/WTJY5w3AL6sbngXI0CoTkB3VyHUJC5bXTfDcnRyhsxvic1RxlggfakI3 LVyxY46lotAiLZFqWHkkd9P8f+vK6MAg02jyPaOYIDQx1NPedKGBFUlaB4iwHrgvY8MQ sHsTuOd1A73nwxVSpNkeiwrhjgMyjxKx+U1OwrQFpTwDEa+JS6MIE8So9z//GvBE/3sk tTO0TRsjdx+191+/GTe06y/js5tyA+MVYBEFt6fcQCS1ecjUH21lTYX3MO/LkddrWZm5 BsxaXRXJ2mL0haperf7AqzXPKE4z+rGnFsy2qx3XhAYtXH7rRxSvLl3FUKIylsE64Zk2 Gt0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HOjDF+E0nEZuBfG11fvtIUGM/mFXeuCvjZH+TDYxm9w=; b=CGs2OUMQFgFObHJZbCjM7t3rf5+MGgcepypBI6qhzIF0OzlJmRZU6ONtN+DSEacEIw 9XHe8b8vTTZNC3tP64PP7win8wDU1Boc3E8wHemB2IRrHebfjmeAZcbQdSDh69BTsLmf +yG5Pq9rer2NLn57cHL3oTTS2sJQlpfHrK60+WBJSZDFLRU+h9vS5tVyJQCSh45EH+6e pzlFL5v6hDyyIpwlM/plAKTP+MY9f2V0Plgtzd4jX82NooYCmB6S7kA1OoMveIxSppr1 2pU1Leaj2a2wnAswo8dI67i5S8XbVfQZ9q3ug77oNzCiu1eYZ4jhtSU+Y1a+Cyw34gRR rXBQ== X-Gm-Message-State: AIkVDXKvtUbtnD2MkA+jxFQpghz2iXbzWftU6yKiEaBE6wSGwJOzjs03TUGX4Rrl+2LtGlorPCbA8kUmmk2Bng== X-Received: by 10.28.228.5 with SMTP id b5mr14157568wmh.129.1482179251399; Mon, 19 Dec 2016 12:27:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.44.1 with HTTP; Mon, 19 Dec 2016 12:27:30 -0800 (PST) In-Reply-To: <20161220.043646.1181938468712455328.hrs@allbsd.org> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> From: Adrian Chadd Date: Mon, 19 Dec 2016 12:27:30 -0800 Message-ID: Subject: Re: RFC: DTrace probes for debugging or testing in userland programs To: "freebsd-arch@freebsd.org" Cc: freebsd-current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 20:27:33 -0000 Hi, I'd love to see a unified-ish logging API for FreeBSD applications. I always end up reusing some C code I have here that I based on some Squid style logging API in ages past. I could always polish it up and put it up for review. I'm not a big fan of requiring dtrace to use it though. On a lot of the embedded systems dtrace varies from "it's very big" through to "we don't have enough RAM/flash to do this". So although I like the sentiment, I don't think using dtrace for program logging is the right answer. I like what apple did to wrap the program logging stuff so people didn't just write their own libraries (hi!) and so there's a unified-ish way to interact with apple programs. I think we could do with that. Thanks, -adrian From owner-freebsd-arch@freebsd.org Mon Dec 19 20:30:42 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80B7DC88BB7; Mon, 19 Dec 2016 20:30:42 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-wj0-x244.google.com (mail-wj0-x244.google.com [IPv6:2a00:1450:400c:c01::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 141341F56; Mon, 19 Dec 2016 20:30:42 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: by mail-wj0-x244.google.com with SMTP id j10so25119260wjb.3; Mon, 19 Dec 2016 12:30:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=DlgS4UstoJniGSuFeA1MOkMtJUMJJHo0Q+oSE8v5j/E=; b=Oc9lK4thwvzKvYQREZ1TfvdrC876d/TQCFwDoc2n01h9WxbZMnG+AXU25Pr29ggoPS UuE8zuNTi/ZtgQrjLIPI9tJ/sYaOwPnPADmiEGYNsG62f6wzNmoiC7n/qJ8MAgNCMGNC 1duH94scQupT5w+4C6kTHMGZJGj6OJM+hbS/Wm9dL5y0z7I8USmfANhRv5goG49PdQnK yiL++4uFLPGeUDsgSuKL9q+03EcPPCWz8uiM7ZEwz0SkLHE8iQse7S1Xqcx8KU6gyn+2 XexkpR+XFZ9oS1ZAh9aTHGSXsV9URlbjnHd2h03rmrvBF2UuL0KatxAJm22/4ck9wzNc V8gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=DlgS4UstoJniGSuFeA1MOkMtJUMJJHo0Q+oSE8v5j/E=; b=HDkZ2J7QRwgBa74aiMi0M6iKQq2AaTIkvELFV4xrHV3Y0Y6FOw5smi84HdX+ADZxfc wGuqhBWMr8zHhppq6jBOl4P6e748jp+JEoDVrjZwtUqdi360b86pHRv6nBCR85hPcba9 hQ+43rKLX8uz340NPVmhmlJojTYCsmjKI7mzgSz5D7qJN3s24qW4Il8lTevsKqke0OpV +g2qaIkb2Ubo1ElOSD3qeKRrEsSCclPWRHQk1N1ZWcUs3ZvaH92fZCo1wdf0WBCBme0P GgTi8sZ+mwjSoe5NnVCRvuMO943bWbYXZJkvcRinfRapgIrEmLgEaW725sHjV1Y5cvJD d0Lg== X-Gm-Message-State: AKaTC03x/p5aenMrj2duk1AIf33lsBMQkOQHhV0fv104OEym0grPi4cW04hxQvKFd9HgHQ== X-Received: by 10.194.7.233 with SMTP id m9mr15095233wja.108.1482179440282; Mon, 19 Dec 2016 12:30:40 -0800 (PST) Received: from freebsd-laptop (213-202-67-199.dsl.iskon.hr. [213.202.67.199]) by smtp.gmail.com with ESMTPSA id l67sm18701625wmf.20.2016.12.19.12.30.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Dec 2016 12:30:39 -0800 (PST) Date: Mon, 19 Dec 2016 21:30:36 +0100 From: Domagoj Stolfa To: freebsd-arch@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: RFC: DTrace probes for debugging or testing in userland programs Message-ID: <20161219203036.GD65993@freebsd-laptop> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/QKKmeG/X/bPShih" Content-Disposition: inline In-Reply-To: <20161220.043646.1181938468712455328.hrs@allbsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 20:30:42 -0000 --/QKKmeG/X/bPShih Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, > You can try to compile a new syslogd, run it, and then attach > dtrace(1) to the syslogd process by "dtrace -q -CI./ > -s ./syslogd_trace.d -p `pgrep syslogd`" in the same directory. one thing that comes to mind is the lack of a way to actually fire these probes without running plenty of DTrace scripts. The SDT provider dynamically links onto the binary using the linker set, so that probes can be called. This is accomplished using the DTrace command that you've issued, would we need to issue one for each daemon that operates in such a way? > Questions from me are the following: >=20 > 1. Where should foo_probes.d and foo_trace.d be installed? And if > they depend on foo.h, where should foo.h be? >=20 > 2. Is documenting probes into foo.8 reasonable? This would again depend on the way that they're implemented. If USDTs are used, documenting them might be beneficial, as the user might at one point want to turn them off dynamically or write their own script. > foo_probes.d and foo_trace.d are added. The former is the > definition of USDT probes, and the latter is a D script example to > reproduce the original debug log by dprintf() or something like > that. A section to describe what probes are available is added into > foo.8. One can trace the foo daemon using "dtrace -Cs foo_trace.d -p > `pgrep foo`" on runtime, and also can create own script. >=20 > foo.h may be added because foo_probes.d and foo_trace.d often require > information of data structure used in foo.c. >=20 > * Possible incompatible change >=20 > A debug flag to activate additional logging is no longer necessary > after this rewrite, so we can remove it (-d flag in the case of > syslogd). And dump of the internal state can be implemented as a > SIGINFO handler. In the syslogd example, SIGINFO dumps syslogd > configuration and access control list. One thing that could be done as well is instead of using the SDT provider, a new provider could be written, which could in turn be controlled by the additional flags, or pehaps even a sysctl integer that would specify the level of logging that one would want. This provider could also be entirely disabled, causing it to disable all the existing probes and, similarly to SDT, use the linker set to patch a couple of nops back in. The user could also attach to this provider with their own scripts, should they wish to perform some other form of monitoring as well. This might be redundant with the SDT provider though, so perhaps a more generic, backwards-compatible way can be thought of that would allow this sort of behaviour? --=20 Best regards, Domagoj Stolfa. --/QKKmeG/X/bPShih Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEHQB+y96lmmv+IXofwxT+ikb0YU4FAlhYQ2wACgkQwxT+ikb0 YU5tEAf/XK4zQ0LKyP5OsCoEX6Q2zmAoej+Dbgpc9Fu6dTZs32Y9B+0tel4x/LVE ew5MO7wMGLoLPpHAQbjRADtRoFeoiyw6AdueXXnNIHVO+9fFfeSAI0RoDakdg27D 1l3yox+v+DlODfX/30vtG4Nl99eSgopn9HB71GgOwoPEBzC86cfsjhgvuKyPevma tgPHs4FQQiu86MGyN2PKcdy0BqAN6a/1Fhqt64Sa4r9SY7DwDevzvRjRO3PemAq6 9YXaP6JM7yg6Bd4V5AKLt7ROBZb0bNv68LFpcvkwbi4nVPZHyhILf1b5l/1RErwb 3OyerJc+XAyccp5FBgslxL5S4ThZkA== =UV+u -----END PGP SIGNATURE----- --/QKKmeG/X/bPShih-- From owner-freebsd-arch@freebsd.org Mon Dec 19 20:47:26 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3443CC883B9; Mon, 19 Dec 2016 20:47:26 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B871A1E8B; Mon, 19 Dec 2016 20:47:25 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id u144so20726171wmu.0; Mon, 19 Dec 2016 12:47:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=4dU+jL/x/Vq3LLr6s2F9LlEkuOY0CzvJtWJTvyrH+j4=; b=X5185aUuzKLoUcCdsibFgK9s1GJVFfnJTDFFYJ/dj+mjf8MxiCIaCrHFwBqjDLcYrz 2oDWWEV98wqWFI8oUUTq+xqQ4tDoxJwN5dWKn8HUCVVGVh8GqMrmeHPMVB8HTOrOh2ON ddY4KoPOTGat/YhqM1p9qAF4KTg0gTAbGlBvkJ3tUUn4Cgte83AOy3o7g1rAfjEqi4kb KDH0AV3YWZ/hgpabOLM3aPmheVKvTYabLRLtFhZVoeLOJBOOA4NnOFmn3MZtrTRdBfh6 VEnK+vwdCc9Y0dwquij7zjBWdGOO9SFC3L9ZNHOW0Xjx1L7vtRx0Wtxz3Ao/RnpL/dPC jSKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=4dU+jL/x/Vq3LLr6s2F9LlEkuOY0CzvJtWJTvyrH+j4=; b=h8nGhcz8WikvwWRzJyFoTThE8s2drBvC3xKBTjnVPfqiSMgI0Wd12nO27mqz+BYvpt k9Qw4BhA0o60mdtvCqP0OmEDi54mmgnLbAGTwRvlQHb2rxVC6/AwYCf9Kg92tRILSZSs vJDKgHlzgyNdbXAyavHwEThxI7lUeHeKpkG3HczUPbcEr2LkpaS6rHotcKUbuXpHTea0 D39/PICTWtJ8vMWztdhzQmZzlSh6Snzwaop4wnXKWjcnN5UYDUiifbb4oYWYVbWW8wmq ErWlGLfD4347G8plDdaucpSO/iZSNVVr1DbPP+E8BSxSdo5lVlTFMpLIuDXDNT4FKuNJ RAXw== X-Gm-Message-State: AIkVDXLiVOLrkpxKNyEfaR1PZaA1DeSUheQLztpz6PFP+RrkuKoq1r4HTaN7gcXRBlTwFg== X-Received: by 10.28.180.214 with SMTP id d205mr15994284wmf.131.1482180443066; Mon, 19 Dec 2016 12:47:23 -0800 (PST) Received: from freebsd-laptop (213-202-67-199.dsl.iskon.hr. [213.202.67.199]) by smtp.gmail.com with ESMTPSA id 14sm18898985wmk.1.2016.12.19.12.47.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Dec 2016 12:47:22 -0800 (PST) Date: Mon, 19 Dec 2016 21:47:19 +0100 From: Domagoj Stolfa To: Adrian Chadd Cc: "freebsd-arch@freebsd.org" , freebsd-current Subject: RFC: DTrace probes for debugging or testing in userland programs Message-ID: <20161219204719.GE65993@freebsd-laptop> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/Zw+/jwnNHcBRYYu" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 20:47:26 -0000 --/Zw+/jwnNHcBRYYu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, > I'd love to see a unified-ish logging API for FreeBSD applications. I > always end up reusing some C code I have here that I based on some > Squid style logging API in ages past. I could always polish it up and > put it up for review. > > I'm not a big fan of requiring dtrace to use it though. On a lot of > the embedded systems dtrace varies from "it's very big" through to "we > don't have enough RAM/flash to do this". DTrace indeed is very heavyweight, this could be an opt-in kind of thing compile time, hidden somewhere in the logging system employed. Personally, I think that keeping the diffs in the actual daemons to the bare minimum(1-2 LoC) should be one of the priorities. Additionally, the logging system should by default be lightweight, with compile time options to change the actual logging method(a simple log, DTrace, ...). > So although I like the sentiment, I don't think using dtrace for > program logging is the right answer. I like what apple did to wrap > the program logging stuff so people didn't just write their own > libraries (hi!) and so there's a unified-ish way to interact with > apple programs. I think we could do with that. This sounds like a pretty clean solution, and the logging method could be hid somewhere deep in there. I would personally like to see an option where I could pick DTrace for logging, as it allows for some interesting scripts to be written, however I tend to agree that this should not be the default. --=20 Best regards, Domagoj Stolfa. --/Zw+/jwnNHcBRYYu Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEHQB+y96lmmv+IXofwxT+ikb0YU4FAlhYR1cACgkQwxT+ikb0 YU79Ugf+OUJw1oqxjYJ4pO3NO4KIhe85NSfyheTv7J98Z149v4TMmoirWjIJbcS1 3too/UvBwD3EmROzngD1dLS2kD8k3r8gCmIDh2eIA4S8FC/CtHIYJ1V9AETbxAhq vDWub7ndrTw0vjckoDDWLx8jQJmY17KmkZzUrM4RDizoTBQBZYcE2seBggM2mP7P 33vRsoguy2yABmbCTzg1q/LGQrYpGj5VrdbwyoeloxthQugqDb54yf2/fS2BfeS8 6xfZv11yh4/OhH7BIPMLSvWIT4Xw6rD6ng0XSJdjqmbkZsWNW3n22xTEgxjgd4sn U1rlTuRL7vnDFOD0KUhmVmmghrgvDg== =2mVa -----END PGP SIGNATURE----- --/Zw+/jwnNHcBRYYu-- From owner-freebsd-arch@freebsd.org Mon Dec 19 21:08:44 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58948C88B10 for ; Mon, 19 Dec 2016 21:08:44 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD8081AA0 for ; Mon, 19 Dec 2016 21:08:43 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail-d.allbsd.org (p2027-ipbf1605funabasi.chiba.ocn.ne.jp [123.225.191.27]) (authenticated bits=56) by mail.allbsd.org (8.15.2/8.15.2) with ESMTPSA id uBJL8Jci087432 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) (Client CN "/OU=GT07882699/OU=See+20www.rapidssl.com/resources/cps+20+28c+2915/OU=Domain+20Control+20Validated+20-+20RapidSSL+28R+29/CN=*.allbsd.org", Issuer "/C=US/O=GeoTrust+20Inc./CN=RapidSSL+20SHA256+20CA+20-+20G3"); Tue, 20 Dec 2016 06:08:39 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from alph.allbsd.org (alph.allbsd.org [192.168.0.10]) by mail-d.allbsd.org (8.15.2/8.15.2) with ESMTPS id uBJL74ZV050421 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Dec 2016 06:07:04 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.15.2/8.15.2) with ESMTPA id uBJL72Ub050418; Tue, 20 Dec 2016 06:07:03 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Tue, 20 Dec 2016 06:05:42 +0900 (JST) Message-Id: <20161220.060542.16356944875657261.hrs@allbsd.org> To: domagoj.stolfa@gmail.com, adrian.chadd@gmail.com Cc: freebsd-arch@freebsd.org Reply-To: freebsd-arch@FreeBSD.org Subject: Re: RFC: DTrace probes for debugging or testing in userland programs From: Hiroki Sato In-Reply-To: <20161219204719.GE65993@freebsd-laptop> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> <20161219204719.GE65993@freebsd-laptop> X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.7 on Emacs 25.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Tue_Dec_20_06_05_42_2016_527)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.99 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.4.3 (mail.allbsd.org [133.31.130.32]); Tue, 20 Dec 2016 06:08:40 +0900 (JST) X-Spam-Status: No, score=-99.9 required=13.0 tests=CONTENT_TYPE_PRESENT, QENCPTR1,USER_IN_WHITELIST autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on gatekeeper.allbsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 21:08:44 -0000 ----Security_Multipart(Tue_Dec_20_06_05_42_2016_527)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Domagoj Stolfa wrote in <20161219204719.GE65993@freebsd-laptop>: do> Hello, do> do> > I'd love to see a unified-ish logging API for FreeBSD applications. I do> > always end up reusing some C code I have here that I based on some do> > Squid style logging API in ages past. I could always polish it up and do> > put it up for review. do> > do> > I'm not a big fan of requiring dtrace to use it though. On a lot of do> > the embedded systems dtrace varies from "it's very big" through to "we do> > don't have enough RAM/flash to do this". do> do> DTrace indeed is very heavyweight, this could be an opt-in kind of thing do> compile time, hidden somewhere in the logging system employed. do> Personally, I think that keeping the diffs in the actual daemons to the do> bare minimum(1-2 LoC) should be one of the priorities. Additionally, the do> logging system should by default be lightweight, with compile time do> options to change the actual logging method(a simple log, DTrace, ...). do> do> > So although I like the sentiment, I don't think using dtrace for do> > program logging is the right answer. I like what apple did to wrap do> > the program logging stuff so people didn't just write their own do> > libraries (hi!) and so there's a unified-ish way to interact with do> > apple programs. I think we could do with that. do> do> This sounds like a pretty clean solution, and the logging method could do> be hid somewhere deep in there. I would personally like to see an option do> where I could pick DTrace for logging, as it allows for some interesting do> scripts to be written, however I tend to agree that this should not be do> the default. To be clear: my proposal is to replace only debug logging (i.e. for developers), not the other logging in general, as the subject line says. Although I agree that DTrace is not lightweight, I think impact of just adding tracing probes is small. -- Hiroki ----Security_Multipart(Tue_Dec_20_06_05_42_2016_527)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlhYS6YACgkQTyzT2CeTzy3KLgCfXsd4GL/nxJKYODu14ZVjwnZ4 HwMAn1jx2MUxDq24hN0jJRoGOPu2j/3o =ducG -----END PGP SIGNATURE----- ----Security_Multipart(Tue_Dec_20_06_05_42_2016_527)---- From owner-freebsd-arch@freebsd.org Mon Dec 19 21:54:31 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78360C88D3F; Mon, 19 Dec 2016 21:54:31 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1DC76195D; Mon, 19 Dec 2016 21:54:31 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id g23so21041085wme.1; Mon, 19 Dec 2016 13:54:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=QcfkL9GjGPXKzrWRPR8r7rGuYkhimYKRmAWp2lChL9M=; b=cZHDlYtRg/Jh01UnAqJf783RA5l667obIvIExYPbaAy7idk9VMSWsy/iALk4DW0ePh jitk3FQX9HeAKD0JaAJNqBrleWpTLqQ2dM6JSd49hnluLjIgbPIP70YgVYk5W+L829CT Kucypf7LDDCD8jLJBSwKxjMFksrTEHUdDWhm96Jd04lOJn1tsrMJhAQjfohK+A8B3I6v bBU7cLjD33ez5zCrR2Q+2QkBG1K3ZGfdIEnpBYY2BpmDzPtE24YUscWHRMfO3F/vzw/I z+HKTbaw6Lj4PWId80/jCYbuhuNw9hnq6RirVGiElJCxVeNpCVipIvE+sQ5gbZRpYkDP BGHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=QcfkL9GjGPXKzrWRPR8r7rGuYkhimYKRmAWp2lChL9M=; b=Sye98hc6Yfk0U9XQ9ZA/cO6CCQDR+5L+vjUjJuDKz1YCxET5A1fW9DdciQqZnphDSh amMakrngb7SDo4JzIct1itYqfnblVz9eY4/xImrUJqpbeUPSJTV/9PjoOUBJAtzqRE9u 3+77uX0i4Eq4gBANe9wD0po+ttFXmSe8sWjM+g7QilfFHOMsJEbUrQwuLGG8mUpVnMgU Lm1WoLjMHL7ukTlqHwiCbUGocbOq5jRqZAu0AfXzfJHl8ntiguckeT1A7KEo9paa2HSa SFzpwe7NIGVzdP22y6bSGcwhddbRNEdweLmU+OHU56XbDT+SAPvc/qacRk/NMrQGBSVI ik2w== X-Gm-Message-State: AIkVDXIkBE6QKefXn/y0PKJU/ydF4/37EPf1EnDWoukZHUp0P0+ksaPMH2zzohV4J7GECw== X-Received: by 10.28.148.81 with SMTP id w78mr14825291wmd.42.1482184468290; Mon, 19 Dec 2016 13:54:28 -0800 (PST) Received: from freebsd-laptop (213-202-67-199.dsl.iskon.hr. [213.202.67.199]) by smtp.gmail.com with ESMTPSA id 6sm22456064wjt.5.2016.12.19.13.54.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Dec 2016 13:54:27 -0800 (PST) Date: Mon, 19 Dec 2016 22:54:25 +0100 From: Domagoj Stolfa To: freebsd-arch@FreeBSD.org Cc: adrian.chadd@gmail.com, freebsd-current@freebsd.org Subject: RFC: DTrace probes for debugging or testing in userland programs Message-ID: <20161219215425.GB42198@freebsd-laptop> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> <20161219204719.GE65993@freebsd-laptop> <20161220.060542.16356944875657261.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="+xNpyl7Qekk2NvDX" Content-Disposition: inline In-Reply-To: <20161220.060542.16356944875657261.hrs@allbsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 21:54:31 -0000 --+xNpyl7Qekk2NvDX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, > To be clear: my proposal is to replace only debug logging (i.e. for > developers), not the other logging in general, as the subject line > says. Although I agree that DTrace is not lightweight, I think > impact of just adding tracing probes is small. >=20 > -- Hiroki I believe this would indeed be beneficial for debugging. My main concern is the way that we would have to attach to probes should we use SDT provider as it works now. If we were to create probes for each of the daemons, which we would then run individually, and if I'm not mistaken, that would spawn 2 times the amount of processes than are currently=20 spawned due to each ``dtrace'' call from the command line spawns a process. Currently, I'm personally leaning towards a different provider, or perhaps a generalization of some operating principles in the SDT provider, so that code duplication can be avoided. Implementing a different provider would allow for a way to automatically log and fire the probes(this can be done quite easily in the provider). Additionally, the user could attach to the probes and perform some different operations in them as well. If we say that the logging level is 0, we could use a linker set and patch up the daemons to nops, so that there is no overhead. If we want to log something, we could simply patch them back in(this could be a sysctl). If the user wants to log something, one could attach to the probe using ``dtrace'', which would cause the provider to then patch up only that probe in the daemons, resulting in no overhead in other daemons. The sole purpose of this is to have a way to toggle logging and avoid spawning many processes of dtrace in order to log. The functionality could also be implemented in the way that Adrian was talking about, we could just say that the log is a debug log, and we only want to trigger it when a certain flag is set. --=20 Best regards, Domagoj Stolfa. --+xNpyl7Qekk2NvDX Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEHQB+y96lmmv+IXofwxT+ikb0YU4FAlhYVxAACgkQwxT+ikb0 YU6lKwf+PhS9G+GvTsY3TcXq42B2XlCGl2crE4KCjkd18LJXyJ6ENvK6PsD1U7w4 DqifvCCfWy7IxJY1vYuxPVJiZEDbgecEMqnUgXj/BTGjCYJ85pwKm52q7D7HPTir 3yNhnWQTR3INg1EU+kpKV5S/1OxzCBKS7Sd9asAOxuXIY0ofGFg+Yu38xOAFLism YwcVr4qXlJB9dVp/GyVh3CxT2T0E9NIu1YpgUiiHZLj1yxyBhudzvEUd6ccvsoUU jVyQizC3fTe1uDgd/U0E+O5mou+oAvyGNnEsnJAt3wMYttNCaKibp0UQmbrrYZ45 UeapOVWu7Kgzs0P6oqI2us9Hu7U+UQ== =tTaz -----END PGP SIGNATURE----- --+xNpyl7Qekk2NvDX-- From owner-freebsd-arch@freebsd.org Mon Dec 19 22:34:01 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 324C9C88A1E for ; Mon, 19 Dec 2016 22:34:01 +0000 (UTC) (envelope-from aminruhulamin19@gmail.com) Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E567D137C for ; Mon, 19 Dec 2016 22:34:00 +0000 (UTC) (envelope-from aminruhulamin19@gmail.com) Received: by mail-io0-x241.google.com with SMTP id b194so20592735ioa.3 for ; Mon, 19 Dec 2016 14:34:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=+gUSqbQRXLvibXkJfsjWiCpeLLAdu1a6bJ2cK031Jkk=; b=Yec5w3sDVnSmt8Fiw9SSeFk/ImfrJLnsMehkR4oP9anWlmO728Mbj3DJa/xuAp/4ea BblY/iH/yeb5BBACAIhWJGMn2CHm9B4JKtb93+l1mdnFs+ZihbU59DUJsg4cZ754G7fq 5jTwB6YIE0ezg9IW/n8t4TuW6AgmoYbNL4k8VGe4lfYxIOg7PcjiyLWIdP+GLoOb/ac0 AWvDjG7DTRqAulmxDvKRij2UGIRcEa3i4Ds+jB12T1zuEjPPu/oRW0Mwx9iIl9RAd4EF sRSDCfKSdZbVgga/agchum7Ff8bnbfq04609v3tLlyUfWJzSZJXLqH+jxmx+mRHdZ3k9 lPUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+gUSqbQRXLvibXkJfsjWiCpeLLAdu1a6bJ2cK031Jkk=; b=nd8IcJDVmxVYLiL3gADtJl87J8xpQeBIG/jT1ZLyE18oXkACuKv21YV4lsAKU1NJU/ eYt7e+gZ/539tQXu+p0YogmDhAVdKfbR+qtowLXpLhcfSi7TjscBQZyP5pK8c+FiLaOA JEc/BcRqVze8yeq9LMzFlC27pZQphJeSWSsaFlF8FM5AxQclnzFhcW2ju0JtlU0I5Zhs F1FbMFORBksdMYgfKNVOiwfv9CrvXWgYiNSTcY2FXQZt4UrZxAjI/AlEhr9PCaKodSZa c9H5fKxUMMbszn2DZqeA9ZzrClrMEntzY1S/yQDjwlVC2w5cdqTKn4gG3sONUZhWfxZO /CYA== X-Gm-Message-State: AIkVDXKN+f2u3nSII7Nq9eG/X30MNDIEjr17oCIsxrrRVPPMMdxG6xlOymdF+aT9zKKQz71ZcpfPOr6PCy0JVw== X-Received: by 10.107.48.134 with SMTP id w128mr22051439iow.226.1482186837272; Mon, 19 Dec 2016 14:33:57 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.166.140 with HTTP; Mon, 19 Dec 2016 14:33:56 -0800 (PST) Received: by 10.107.166.140 with HTTP; Mon, 19 Dec 2016 14:33:56 -0800 (PST) From: Ruhulamin Amin Date: Mon, 19 Dec 2016 14:33:56 -0800 Message-ID: Subject: To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 22:34:01 -0000 Aminr8860@gmail.combd From owner-freebsd-arch@freebsd.org Mon Dec 19 23:08:20 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DCA9C88439 for ; Mon, 19 Dec 2016 23:08:20 +0000 (UTC) (envelope-from hrs@allbsd.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AACE0125F for ; Mon, 19 Dec 2016 23:08:19 +0000 (UTC) (envelope-from hrs@allbsd.org) Received: from [10.155.117.172] ([1.66.104.193]) (authenticated bits=0) by mail.allbsd.org (8.15.2/8.15.2) with ESMTPA id uBJN859H096380; Tue, 20 Dec 2016 08:08:16 +0900 (JST) (envelope-from hrs@allbsd.org) Content-Type: text/plain; charset=iso-2022-jp Mime-Version: 1.0 (1.0) Subject: Re: RFC: DTrace probes for debugging or testing in userland programs From: Hiroki Sato X-Mailer: iPhone Mail (14C92) In-Reply-To: <20161219215425.GB42198@freebsd-laptop> Date: Tue, 20 Dec 2016 08:08:05 +0900 Cc: freebsd-arch@FreeBSD.org, adrian.chadd@gmail.com Content-Transfer-Encoding: quoted-printable Message-Id: <9887B4D9-A336-498D-ADDE-E683B0DEC2E8@allbsd.org> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> <20161219204719.GE65993@freebsd-laptop> <20161220.060542.16356944875657261.hrs@allbsd.org> <20161219215425.GB42198@freebsd-laptop> To: Domagoj Stolfa X-Virus-Scanned: clamav-milter 0.99 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.allbsd.org [133.31.130.32]); Tue, 20 Dec 2016 08:08:16 +0900 (JST) X-Spam-Status: No, score=1.3 required=13.0 tests=CONTENT_TYPE_PRESENT, FORGED_RCVD_IP,ISO2022JP_BODY,ISO2022JP_CHARSET,MIMEQENC,QENCPTR1,QENCPTR2 autolearn=no autolearn_force=no version=3.4.1 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on gatekeeper.allbsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2016 23:08:20 -0000 2016/12/20 6:54=1B$B!"=1B(BDomagoj Stolfa =1B$B$N= %a%C%;!<%8=1B(B: > Hello, >=20 >> To be clear: my proposal is to replace only debug logging (i.e. for >> developers), not the other logging in general, as the subject line >> says. Although I agree that DTrace is not lightweight, I think >> impact of just adding tracing probes is small. >>=20 >> -- Hiroki >=20 > I believe this would indeed be beneficial for debugging. My main concern > is the way that we would have to attach to probes should we use SDT > provider as it works now. If we were to create probes for each of the > daemons, which we would then run individually, and if I'm not mistaken, > that would spawn 2 times the amount of processes than are currently=20 > spawned due to each ``dtrace'' call from the command line spawns a process= . No. A single dtrace process can catch fired probes of multiple processes if you want. Filtering probes or the results from probes is things which should be done in the D script level, not in the target binary, in the case of DTrace. My proposal does not cover how to use the probes. It just provides probes and a sample script to reproduce the (almost) same results with printf() debug logging. The big difference is that a developer can use these probes for a more complex debug script and/or regression test script because the probes can carry data structures in question, not just a string. -- Hiroki= From owner-freebsd-arch@freebsd.org Tue Dec 20 00:04:26 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDF8BC885F0; Tue, 20 Dec 2016 00:04:26 +0000 (UTC) (envelope-from SRS0+xyJv=YD=mail.turbofuzz.com=jkh@ixsystems.com) Received: from mx.ixsystems.com (mx.ixsystems.com [12.229.62.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN ".", Issuer "." (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CFB691CCE; Tue, 20 Dec 2016 00:04:26 +0000 (UTC) (envelope-from SRS0+xyJv=YD=mail.turbofuzz.com=jkh@ixsystems.com) Received: from localhost (localhost.localdomain [127.0.0.1]) by mx.ixsystems.com (Postfix) with ESMTP id 3tjJ1Q4QJSzCqPb; Mon, 19 Dec 2016 16:04:26 -0800 (PST) X-Virus-Scanned: Scrollout F1 at ixsystems.com Received: from mx.ixsystems.com ([127.0.0.1]) by localhost (mx.ixsystems.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id wpm3bPjxWHTW; Mon, 19 Dec 2016 16:04:15 -0800 (PST) Received: from zimbra.ixsystems.com (mail.ixsystems.com [10.246.0.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.ixsystems.com (Postfix) with ESMTPS id 3tjJ1C14X1zCrgh; Mon, 19 Dec 2016 16:04:15 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.ixsystems.com (Postfix) with ESMTP id 0A0859F115E; Mon, 19 Dec 2016 16:04:15 -0800 (PST) Received: from zimbra.ixsystems.com ([127.0.0.1]) by localhost (zimbra.ixsystems.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id lElWI0lZsH8h; Mon, 19 Dec 2016 16:04:12 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by zimbra.ixsystems.com (Postfix) with ESMTP id 8F5BD9F115D; Mon, 19 Dec 2016 16:04:12 -0800 (PST) X-Virus-Scanned: amavisd-new at ixsystems.com Received: from zimbra.ixsystems.com ([127.0.0.1]) by localhost (zimbra.ixsystems.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id cFI05c1gSq1V; Mon, 19 Dec 2016 16:04:12 -0800 (PST) Received: from eyeball.ixsystems.com (eyeball.ixsystems.com [10.250.0.175]) by zimbra.ixsystems.com (Postfix) with ESMTPSA id 332CE9F115E; Mon, 19 Dec 2016 16:04:12 -0800 (PST) From: Jordan Hubbard Message-Id: Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: RFC: DTrace probes for debugging or testing in userland programs Date: Mon, 19 Dec 2016 16:04:11 -0800 In-Reply-To: Cc: "freebsd-arch@freebsd.org" , freebsd-current To: Adrian Chadd References: <20161220.043646.1181938468712455328.hrs@allbsd.org> X-Mailer: Apple Mail (2.3259) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 00:04:27 -0000 > On Dec 19, 2016, at 12:27 PM, Adrian Chadd = wrote: >=20 > So although I like the sentiment, I don't think using dtrace for > program logging is the right answer. I like what apple did to wrap > the program logging stuff so people didn't just write their own > libraries (hi!) and so there's a unified-ish way to interact with > apple programs. I think we could do with that. Thanks! We did a number of other things with ASL (Apple System Logger) which I = miss very much today and would hope to see in any FreeBSD equivalent: 1. We structured all log data into dictionaries, so every application = and/or subsystem within that application can add its own =E2=80=9Ctags=E2=80= =9D without squashing other key information. This also unified the = character encoding format, so some applications were no longer logging = in ISO-Latin1, others in UTF-8 and yet others in SHIFT-JIS. 2. There=E2=80=99s also a logging database, as one of the many possible = =E2=80=9Coutput sinks=E2=80=9D, so searches / queries are fast (and = there=E2=80=99s an API for querying and managing its contents). 3. We added client-side and server side logging filters, so you can = =E2=80=9Ccrank an application up=E2=80=9D or shut its mouth without = having to make any code changes. 4. It=E2=80=99s all thread-safe. - Jordan From owner-freebsd-arch@freebsd.org Tue Dec 20 00:38:08 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07568C88D10; Tue, 20 Dec 2016 00:38:08 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wj0-x242.google.com (mail-wj0-x242.google.com [IPv6:2a00:1450:400c:c01::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1C02AAF; Tue, 20 Dec 2016 00:38:07 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wj0-x242.google.com with SMTP id kp2so25660540wjc.0; Mon, 19 Dec 2016 16:38:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=461xUVmZnZeRUuaRZ+A78xEad/2VWl1/tWse/wGpI0I=; b=VzrmGK4C+Sv0fbfFI3ibV9pqYuNFv1Sp2/mtOqwbc2NJU50pRMp5jP8ti5XXGTjTq4 BYs97DiZOZ8rlq80bUAZ5HXFyKtw7a2oxqFer2b5gNXvtyVTUNW6mVi9Zcz7Cc5637z7 ZOGRfkNHbZNBQqzfggv22q9S8lXkGSehSmCyiCzHTcBSow9wahNf3ZoPqMGffFI7bQWz MYXZ5gQ7QvwkZBDiZsCrUMdux4GCZSVdjapInKBdaUvFfaL4+pwd09fEevvrNju7Hniz y6OXdDLGoa+qac577E4lInwzL4Yw3DsEabIpMv2+V9SvsGE2Yf3YQgyS4tFtI1sZBHVw /Ygw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=461xUVmZnZeRUuaRZ+A78xEad/2VWl1/tWse/wGpI0I=; b=GHV4R0Gf5RnuGchbNLL9GUP4brps7HUYGkK0iTENoGo1O4Rs80mXT7oJR/ajVmcXd3 M/US4HpAGCMSCjIhzHlplHe3JQRG77SCaFd2U3d4NGSws+zrw0zemMqWdlc96yrKAMpA a63jHH8cSqVIIVBSvSI0LQx5aM9ttNT55aH2jnIs6pYssBmCE3x+7rE9htEE5gHcKd0G WhKQcJCXo01QcFBTU+bVgQi/KYsCUJCDNdnBY+JM6Vv5WTvBx63ViNjPZu1VnGdeM6rh 461VwTdWG7W54H49bxpZQy5E6liaJnEs/sEiiIHBYOCHu6YxwzjZMksOu7tPjiHJCXJC i7Kg== X-Gm-Message-State: AIkVDXLsGl899ogUjc4tG4pyZGpMwDhLLfGFxvtn4K5FBIIY1powabcdD49j3jM4d0gjajxeAhHmuBrbXqH9Rg== X-Received: by 10.194.187.103 with SMTP id fr7mr15204315wjc.99.1482194286007; Mon, 19 Dec 2016 16:38:06 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.44.1 with HTTP; Mon, 19 Dec 2016 16:38:05 -0800 (PST) In-Reply-To: References: <20161220.043646.1181938468712455328.hrs@allbsd.org> From: Adrian Chadd Date: Mon, 19 Dec 2016 16:38:05 -0800 Message-ID: Subject: Re: RFC: DTrace probes for debugging or testing in userland programs To: Jordan Hubbard Cc: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 00:38:08 -0000 On 19 December 2016 at 16:04, Jordan Hubbard wrote= : > > On Dec 19, 2016, at 12:27 PM, Adrian Chadd wrote= : > > So although I like the sentiment, I don't think using dtrace for > program logging is the right answer. I like what apple did to wrap > the program logging stuff so people didn't just write their own > libraries (hi!) and so there's a unified-ish way to interact with > apple programs. I think we could do with that. > > > Thanks! > > We did a number of other things with ASL (Apple System Logger) which I mi= ss > very much today and would hope to see in any FreeBSD equivalent: > > 1. We structured all log data into dictionaries, so every application and= /or > subsystem within that application can add its own =E2=80=9Ctags=E2=80=9D = without squashing > other key information. This also unified the character encoding format, = so > some applications were no longer logging in ISO-Latin1, others in UTF-8 a= nd > yet others in SHIFT-JIS. > > 2. There=E2=80=99s also a logging database, as one of the many possible = =E2=80=9Coutput > sinks=E2=80=9D, so searches / queries are fast (and there=E2=80=99s an AP= I for querying and > managing its contents). > > 3. We added client-side and server side logging filters, so you can =E2= =80=9Ccrank > an application up=E2=80=9D or shut its mouth without having to make any c= ode > changes. > > 4. It=E2=80=99s all thread-safe. Hm. Where's the ASL source hiding? I'm kinda hoping it's light weight enough to port over without porting a lot of other stuff, but I am also afraid it's Apple.. :) -adrian From owner-freebsd-arch@freebsd.org Tue Dec 20 03:35:02 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4AE0C89F31; Tue, 20 Dec 2016 03:35:02 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.allbsd.org", Issuer "RapidSSL SHA256 CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32D9F18F3; Tue, 20 Dec 2016 03:35:02 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail-d.allbsd.org (p2027-ipbf1605funabasi.chiba.ocn.ne.jp [123.225.191.27]) (authenticated bits=56) by mail.allbsd.org (8.15.2/8.15.2) with ESMTPSA id uBK3YaFN038795 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) (Client CN "/OU=GT07882699/OU=See+20www.rapidssl.com/resources/cps+20+28c+2915/OU=Domain+20Control+20Validated+20-+20RapidSSL+28R+29/CN=*.allbsd.org", Issuer "/C=US/O=GeoTrust+20Inc./CN=RapidSSL+20SHA256+20CA+20-+20G3"); Tue, 20 Dec 2016 12:34:56 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from alph.allbsd.org (alph.allbsd.org [192.168.0.10]) by mail-d.allbsd.org (8.15.2/8.15.2) with ESMTPS id uBK3XLg4054650 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 20 Dec 2016 12:33:21 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:::1]) (authenticated bits=0) by alph.allbsd.org (8.15.2/8.15.2) with ESMTPA id uBK3XKZW054647; Tue, 20 Dec 2016 12:33:20 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Tue, 20 Dec 2016 12:25:35 +0900 (JST) Message-Id: <20161220.122535.1576963544479470984.hrs@allbsd.org> To: adrian.chadd@gmail.com Cc: jkh@mail.turbofuzz.com, freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: RFC: DTrace probes for debugging or testing in userland programs From: Hiroki Sato In-Reply-To: References: X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.7 on Emacs 25.1 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart(Tue_Dec_20_12_25_35_2016_644)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.99 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.4.3 (mail.allbsd.org [133.31.130.32]); Tue, 20 Dec 2016 12:34:58 +0900 (JST) X-Spam-Status: No, score=-100.0 required=13.0 tests=CONTENT_TYPE_PRESENT, ISO2022JP_BODY,QENCPTR1,USER_IN_WHITELIST autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on gatekeeper.allbsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 03:35:02 -0000 ----Security_Multipart(Tue_Dec_20_12_25_35_2016_644)-- Content-Type: Text/Plain; charset=iso-2022-jp Content-Transfer-Encoding: 7bit Adrian Chadd wrote in : ad> On 19 December 2016 at 16:04, Jordan Hubbard wrote: ad> > ad> > On Dec 19, 2016, at 12:27 PM, Adrian Chadd wrote: ad> > ad> > So although I like the sentiment, I don't think using dtrace for ad> > program logging is the right answer. I like what apple did to wrap ad> > the program logging stuff so people didn't just write their own ad> > libraries (hi!) and so there's a unified-ish way to interact with ad> > apple programs. I think we could do with that. ad> > ad> > ad> > Thanks! ad> > ad> > We did a number of other things with ASL (Apple System Logger) which I miss ad> > very much today and would hope to see in any FreeBSD equivalent: ad> > ad> > 1. We structured all log data into dictionaries, so every application and/or ad> > subsystem within that application can add its own “tags” without squashing ad> > other key information. This also unified the character encoding format, so ad> > some applications were no longer logging in ISO-Latin1, others in UTF-8 and ad> > yet others in SHIFT-JIS. ad> > ad> > 2. There’s also a logging database, as one of the many possible “output ad> > sinks”, so searches / queries are fast (and there’s an API for querying and ad> > managing its contents). ad> > ad> > 3. We added client-side and server side logging filters, so you can “crank ad> > an application up” or shut its mouth without having to make any code ad> > changes. ad> > ad> > 4. It’s all thread-safe. ad> ad> Hm. Where's the ASL source hiding? I'm kinda hoping it's light weight ad> enough to port over without porting a lot of other stuff, but I am ad> also afraid it's Apple.. :) libsystem_asl at https://opensource.apple.com/source/syslog/syslog-349.1.1/? -- Hiroki ----Security_Multipart(Tue_Dec_20_12_25_35_2016_644)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEABECAAYFAlhYpK8ACgkQTyzT2CeTzy2kJwCfYkyJnpYZOf0GTVbZ8+Xh5KVl lTEAoNYNo1b/5iU4qq5urKLk7QoYBtQh =fl23 -----END PGP SIGNATURE----- ----Security_Multipart(Tue_Dec_20_12_25_35_2016_644)---- From owner-freebsd-arch@freebsd.org Tue Dec 20 05:18:59 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7203CC897AA; Tue, 20 Dec 2016 05:18:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wj0-x229.google.com (mail-wj0-x229.google.com [IPv6:2a00:1450:400c:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0829F1275; Tue, 20 Dec 2016 05:18:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mail-wj0-x229.google.com with SMTP id tg4so168147812wjb.1; Mon, 19 Dec 2016 21:18:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KyWQ+FIKxnphmJVUumPZIfuSuF9NPJ2vGpoVOZ9lM0Q=; b=gLBQycbW/HU4YdMFPPWm8+weLj3TZmB/nLSJxwIQqRBgYAsjfnwto279lQfCuxmt8o waSYiCBglRHqWgaSJ/8IvpWdDKWg7wf7e7O/pg0pB2XTvjwkRctpzmtn44RonjDLqlkW qwHnd6rcJ3WImd0CbTG5TTwnkosaP29eR+uwdMowk+Aq/Pu7zS/2PvPE5XfcMwpBv44h 0yr7RzbEwENrfTkOaC8A6bRpE1CdfRSHdZ8fVDgyAFCne7P8Hd/ucwpShgWJYE6+r5D1 jaVGWADbERkAbb2JN1mxbC/8trVOhnLh2IZceO1LIuqoXVlwqtjN3rNb2sKTcsEmbKpD m9Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KyWQ+FIKxnphmJVUumPZIfuSuF9NPJ2vGpoVOZ9lM0Q=; b=cPOxNBEtfL8R7yrG9I650ncptOcUGJlgmyv+2QmTJydUsN7DbCrUM5eyF5RWhQ73Wt HuO7Ry1ZZ/QApCD3YeAhjqp0jJQpI+M5O7h+Z//Bxgj+ir7Z6Nudgs5885cfo3S+KYha McQeiKahXfW19+/GCXJzqa5pOtGfprZpoMq6QqQKSgfBtB9gwAL2Vi8L8i0so9cbZ7Td +EtpioW95werFNWHYxAW0s1MZp7Z6g/jAehFAxZjDV5945Ewu8U5Ne7d7szYoxyICaPC Udg1Qp1GPdAi2MyXKibUef97Sm43rCJpoxSbBh6NXfI3Rx2yrAAN09mfaV8SIoHzgyfa nevQ== X-Gm-Message-State: AIkVDXLRgEZX/f+k6p8tWwlqf8DV7dDoXcxhRKdZprR3slZ4fwn/vekBb1+7c2kAldP6S4ai0MW9vdwF7ZCUJA== X-Received: by 10.194.187.103 with SMTP id fr7mr15812136wjc.99.1482211136864; Mon, 19 Dec 2016 21:18:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.44.1 with HTTP; Mon, 19 Dec 2016 21:18:56 -0800 (PST) In-Reply-To: <20161220.122535.1576963544479470984.hrs@allbsd.org> References: <20161220.122535.1576963544479470984.hrs@allbsd.org> From: Adrian Chadd Date: Mon, 19 Dec 2016 21:18:56 -0800 Message-ID: Subject: Re: RFC: DTrace probes for debugging or testing in userland programs To: Hiroki Sato Cc: Jordan Hubbard , freebsd-current , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 05:18:59 -0000 *mumble* damnit jordan this requires libdispatch *mumble* -a From owner-freebsd-arch@freebsd.org Tue Dec 20 12:44:12 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54555C88259 for ; Tue, 20 Dec 2016 12:44:12 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3E56671 for ; Tue, 20 Dec 2016 12:44:11 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id m203so24076431wma.3 for ; Tue, 20 Dec 2016 04:44:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=MLGiRNqGmyy+v8GTIWhRcdUb75eAE0/wFr6xezh8Xoc=; b=qaiBDziigfnb15NFs8clSU4HADO0sL7Tf4ZyaWv4Qd+1YjboRMjvVeZ2ZQqUq8lBGy LZKk+On/RHowpvC0FIs65yTZaoahvdT/GuBOpVsItVOHMX8ynhn/flH7OZUzlgZ26caF aF6gnoGoYSZGWJjz6WDN5X1uexSivA0cEf5Ue847dm8lWp61FfSuYn4cf4pkrW5Y+mOg zioeMpzBI8OQ3wD9GfS4P6gjVmZuZxp+wE6Awx5ABdOlerVRkVi0GR+f1IbOd6CiENd5 D04BOA8QGw5vsIrmpAw76M00ITlnWvrVwfnYmmZbihSIh0FZZu54xIfGV5m7huFWR5LJ vTxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=MLGiRNqGmyy+v8GTIWhRcdUb75eAE0/wFr6xezh8Xoc=; b=k5sS//vbCVPsq5IDMSaUcn6c2VJ9Y542u8QL6p0wIAVBT0gfbP9/jGqt87//70esd4 WcqSU15y7GO0m7ZIxTRWwmLVgA0CtWjnhhi77v7DS3bQKCEvwsFO0H/xMd0jI97Sc9VG bUC1xmiclhbaWBVodfEAApcWmmEuwef2i3uO7miQ5t5cg1xlAvd5B4w71dXTG1h9i5fz 40bDlPzbTy81Y5gY02gHMCzPKZjDelWty15+1L4t7s2Fmjj/4GDumLTKt/U58Z4Yb8hC mWlmo0p4mj00jESxXlg0ZcG50LyvFfltsuz/HtPNhVkv+2/o0p+Cnjyklk8eCo6GqRYL e7Kw== X-Gm-Message-State: AIkVDXI8/VetPx5QNPwwomX4jDqoKx3onxZAWUXbUGp0jnEp/PNf+gWe7/gFMq0clCDD7g== X-Received: by 10.28.4.199 with SMTP id 190mr1727720wme.11.1482237850366; Tue, 20 Dec 2016 04:44:10 -0800 (PST) Received: from freebsd-laptop ([31.217.98.238]) by smtp.gmail.com with ESMTPSA id w197sm21939773wmd.11.2016.12.20.04.44.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Dec 2016 04:44:09 -0800 (PST) Date: Tue, 20 Dec 2016 13:44:06 +0100 From: Domagoj Stolfa To: Hiroki Sato Cc: freebsd-arch@FreeBSD.org, adrian.chadd@gmail.com Subject: RFC: DTrace probes for debugging or testing in userland programs Message-ID: <20161220124406.GA1713@freebsd-laptop> References: <20161220.043646.1181938468712455328.hrs@allbsd.org> <20161219204719.GE65993@freebsd-laptop> <20161220.060542.16356944875657261.hrs@allbsd.org> <20161219215425.GB42198@freebsd-laptop> <9887B4D9-A336-498D-ADDE-E683B0DEC2E8@allbsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="liOOAslEiF7prFVr" Content-Disposition: inline In-Reply-To: <9887B4D9-A336-498D-ADDE-E683B0DEC2E8@allbsd.org> User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Dec 2016 12:44:12 -0000 --liOOAslEiF7prFVr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, > No. A single dtrace process can catch > fired probes of multiple processes if you want. >=20 > Filtering probes or the results from probes > is things which should be done in the > D script level, not in the target binary, > in the case of DTrace. My proposal does > not cover how to use the probes. It just > provides probes and a sample script to > reproduce the (almost) same results with > printf() debug logging. The big difference > is that a developer can use these probes > for a more complex debug script and/or > regression test script because the > probes can carry data structures in > question, not just a string. I suppose this comes down to a way that you want to implement this. Using the SDT provider would require you to attach to the probes using a D script to perform simple logging, and additional scripts in order to do more complex operations(such as quantize certain arguments that you're getting in during that log and what not). Eventhough your original proposal doesn't cover using those probes, I'm still puzzled as to how you would actually attach to the probes, i.e. would you have a script for each daemon separately or would you create 1 big script, causing it to attach to all the probes created in the separate daemons? There doesn't seem to be a way around having atleast one script in order to make the probes actually fire(that is, call dtrace_probe()). This is one of the reasons why I've argued for a different provider as opposed to SDT. Another thing that could perhaps be done is implement the probes in the daemons and simply attach to them whenever the user needs to attach while leaving the actual logging functionality as is. That way the SDT provider would dynamically link the function call into the daemon whenever one would want to trace for more complex logging(there could be example scripts). The arguments could be documented, allowing for an easy way to write the scripts and have context of what the arguments are. In any case, I would very much like to have access to SDT probes in daemons in order to get full disclosure on what's happening in real time, instead of getting a pre-defined log that I would have to post-process, and would like to thank you for your efforts in doing this. --=20 Best regards, Domagoj Stolfa. --liOOAslEiF7prFVr Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEHQB+y96lmmv+IXofwxT+ikb0YU4FAlhZJ5UACgkQwxT+ikb0 YU4KPwf/WQaTWF+RctK3X2dK3cVjhWT/v+qUgRxQZRB6tA0EG/u8aTrcYZdV7jjy aHRQBWWUbXCv5jRqLu6iztpNsiwchgFRfW39bVLyg5LsJihCT9QghTskvKUt0tvO lp8b9CvrbqQW5HdFAfHm18Zhu4xR+5BD0SFu7qMRwl7Hg1hVGwt34KAPTwl9mj8e bCvn+xCtQWolSCIfJEk1TMMUV90hkGX4iTNGTyusZPstzTcc2tpSORGWRnb8PJpZ 2y4R8xQt9Rv9X+u01FEbmIvofQaLpVSFya1QTJ0OyYuJ0x9xg2cWPRYoEwCn+uOb zNugFLwXx4rkR1JBgocZ3tRWuUxtGA== =pOaZ -----END PGP SIGNATURE----- --liOOAslEiF7prFVr-- From owner-freebsd-arch@freebsd.org Thu Dec 22 19:37:06 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5951C8D28A for ; Thu, 22 Dec 2016 19:37:06 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-qk0-x243.google.com (mail-qk0-x243.google.com [IPv6:2607:f8b0:400d:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D696B98 for ; Thu, 22 Dec 2016 19:37:06 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-qk0-x243.google.com with SMTP id t184so14472684qkd.1 for ; Thu, 22 Dec 2016 11:37:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=cc:message-id:from:to:in-reply-to:content-transfer-encoding :mime-version:subject:date:references; bh=hnXU+Xm2Ahl8K9IgddzZko2da8VOBJRVCLveVqFusxw=; b=Q1rx/eIWdBl+3aWFRA4pn9eynKqFdWixj72zm42lHTGh+fowLai6mhHu8XdWM7QPCm cQswiVJgfe+k5l8UhETlsORY0HIDE9y/Jc3Fb6KJ9hYvsTbjpOF1fPYrw5ABTM/DTd5/ wngFoDUDEmSvgDF8KNjZV4z4To3Y+S2vdlnns9+y6XkhpPP/2b2r8froAc/6nWy+SOuE WLJsGN0DIxhiRRHJUc5KGnh5hItcxT1ulMuT08afeHeUdaPCDd7EFCryFyu2xRHM+LLy 4+RwjWmXypx/u7tx2Z7L94HTS7ucnuWO40qu9R/2P+S5BW5a1r4savpQf7yS0MuHnsWX E/FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:cc:message-id:from:to:in-reply-to :content-transfer-encoding:mime-version:subject:date:references; bh=hnXU+Xm2Ahl8K9IgddzZko2da8VOBJRVCLveVqFusxw=; b=r9kVqKIyHKm72zWkj6RJ/QykPAmpi6SyLDynCLg8jlGRnLwjFRNhOoo09tV6CpbfQy 1fd4wGbypTeMnrGufNbjazJsVOHvwowyGLFZqHG3/d3BvP7ymSatkXuSsKDKCCUGvycu 0YUCZgQE6NP9TPeDJb7M8bKaaExcw2p1OpkGu2vlxr6LIrqFtcuBtoewTlxgs6ZJRnAh uCZQTgLn7VD50ubU3+yXmR6XPy6LL5h0YZt5WvpeFdWn3Bp7aVAEsc+czBX/0UvWnsRT 71MgwayfTJmcKsi86AiU5drPXG8NxqGnNZlAPiWx1JVl5eYQ81wU+jEZgeUf9ZCm2SPc nD8A== X-Gm-Message-State: AIkVDXJLYmXcK6+Sz6/tILcVCBhqfvz8QZBrmjj1gSUjvwu/H0TsJCS9ubONyieLrwms9A== X-Received: by 10.55.160.18 with SMTP id j18mr12897220qke.239.1482435425769; Thu, 22 Dec 2016 11:37:05 -0800 (PST) Received: from blackstar.home (pool-173-79-29-94.washdc.fios.verizon.net. [173.79.29.94]) by smtp.gmail.com with ESMTPSA id p196sm18607695qke.47.2016.12.22.11.37.04 (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 22 Dec 2016 11:37:05 -0800 (PST) Cc: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= , FreeBSD Arch Message-Id: <6C1FBD30-8301-4C6D-8C8B-653C6C096A93@gmail.com> From: Justin Hibbits To: Warner Losh In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v936) Subject: Re: Order of device suspend/resume Date: Thu, 22 Dec 2016 14:37:04 -0500 References: <20161215114033.r33nt3fqhnfi7hqw@dhcp-3-221.uk.xensource.com> <7469755.xT5lfhErkd@ralph.baldwin.cx> X-Mailer: Apple Mail (2.936) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2016 19:37:07 -0000 On Dec 16, 2016, at 12:25 AM, Warner Losh wrote: > On Thu, Dec 15, 2016 at 8:34 PM, Justin Hibbits =20 > wrote: >> >> On Dec 15, 2016, at 3:38 PM, John Baldwin wrote: >> >>> On Thursday, December 15, 2016 11:40:33 AM Roger Pau Monn=E9 wrote: >>>> >>>> Hello, >>>> >>>> I'm currently dealing with a bug in the Xen suspend/resume =20 >>>> sequence, and >>>> I've >>>> found that lacking a way to order device priority during suspend/=20= >>>> resume >>>> is >>>> proving quite harmful for Xen (and maybe other systems too). The =20= >>>> current >>>> suspend/resume code simply scans the root bus, and suspends/=20 >>>> resumes every >>>> device >>>> based on the order they are attached to their parents. The =20 >>>> problem here >>>> is that >>>> there's no way to tell that some devices should be resumed before =20= >>>> others, >>>> for >>>> example the event timers/time counters/uarts should definitely be =20= >>>> resume >>>> before >>>> other devices, but that's seems to happens mostly out of chance. >>>> >>>> Currently most time related devices are attached directly to the =20= >>>> nexus, >>>> which >>>> means they will get resumed first, but for example the uart is =20 >>>> currently >>>> attached to the pci bus IIRC, which means it gets resumed quite =20 >>>> late. On >>>> Xen >>>> systems, this is even worse. The Xen PV bus (that contains all >>>> Xen-related >>>> devices) is attached the last one (because it tends to pick up =20 >>>> unused >>>> memory >>>> regions for it's own usage) and this bus also contains the PV =20 >>>> timecounter >>>> which >>>> should be resumed _before_ other devices, or else timecounting =20 >>>> will be >>>> completely screwed and things can get stuck in indefinitely long =20= >>>> loops >>>> (due to >>>> the fact that the timecounter is implemented based on the uptime =20= >>>> of the >>>> host, >>>> and that changes from host-to-host). >>>> >>>> In order to solve this I could add a hack to the Xen resume process >>>> (which is >>>> already different from the ACPI one), but this looks gross. I =20 >>>> could also >>>> attach >>>> the Xen PV timer to the nexus directly (as it was done before), =20 >>>> but I >>>> also >>>> prefer to keep all Xen-related devices in the same bus for =20 >>>> coherency. >>>> Last >>>> option would be to add some kind of suspend/resume priorities to =20= >>>> the >>>> devices, >>>> and do more than one suspend/resume pass. This is more complex and >>>> requires more >>>> changes, so I would like to know if it would be helpful for other >>>> systems, or if >>>> someone has already attempted to do it. >>> >>> >>> I think Justin Hibbits had some patches to make use of the boot-time >>> new-bus >>> passes for suspend and resume which I think would help with this. =20= >>> You >>> suspend >>> things in the reverse order of boot and resume operates in the =20 >>> same order >>> as >>> boot. >>> >>> -- >>> John Baldwin >> >> >> John is right. I have a (somewhat abandoned due to time and focus) =20= >> branch, >> https://svnweb.freebsd.org/base/projects/pmac_pmu/ which has the =20 >> necessary >> code working mostly on PowerPC. The diff can be found at >> https://reviews.freebsd.org/D203 too. > > Cool. Does it have a mechanism similar to the attach code that lets > you run again at each pass? > > Warner Not exactly. The code will call the BUS_SUSPEND_CHILD() as it rolls =20 back the pass levels, and stop on errors. The meat is in a rewrite of =20= bus_generic_suspend() in that review. - Justin= From owner-freebsd-arch@freebsd.org Sat Dec 24 00:14:37 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26569C8ECAF for ; Sat, 24 Dec 2016 00:14:37 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 02768B7B for ; Sat, 24 Dec 2016 00:14:37 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 01C0FC8ECAE; Sat, 24 Dec 2016 00:14:37 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3D39C8ECAD for ; Sat, 24 Dec 2016 00:14:36 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3064B7A; Sat, 24 Dec 2016 00:14:36 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pg0-x22a.google.com with SMTP id y62so61851850pgy.1; Fri, 23 Dec 2016 16:14:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:date:message-id:cc:to:mime-version; bh=LO4XPNlQAv8r87LkIIgnHDfRgtIotAf7XDR8bnOTlA0=; b=Wq5RGWSUsFWt4Azz8AEAyIsticTTeoRvEUuonp49kSRfnvutKfZpc9wsbbcpimiuQ5 ywnKwBO7w3yVBslnZ7cu4xeF1KDtlDaD6T/ScDjJFRiIrnbI+JGI0I9hmp3dIZTP9Ya0 8lcaZMINwMgNXs6OccCVGdQZryQSAAKlNDkAjjZeSRXAJGZ2dfaT+A6ALpQPO7Z18wNS s8W1FyVu+8KkYDvliM8UY6Gu6VjmTqbtE9VUz749cR5cnXjOOQmNp38xURjj94BbOW4m 2Lx+Fqwb2jUzDRNbp6BPIWufaES9TvmOvYKGxp4+ltD01HvDwLGe/JKLKjsKKbSB5JTw KeeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:date:message-id:cc:to:mime-version; bh=LO4XPNlQAv8r87LkIIgnHDfRgtIotAf7XDR8bnOTlA0=; b=tiymCMmeuhcZUj5TymNipg105rYGuesqvFvokB+ZJwybe2GLoTvcIF4FNZuwEXjkDV GpVClx2LJxzx8w2QICv5rQywrWEq8LAU+4znPUSatJJqf/wrTZ4xz4qbwAhJjeWFtQDm lDRPXavNpreuCdaI4nUr9HribWUjx3rd0wb+uD/Q7LBefjd6D1u7zi0jQaf1wkXkPCzO /ccDilna0/l3ROzhJVUQBOuG3UuERiJ1+9iWNMX1O06nbZxAaznGfcqeqXi7tCjVoUIy R1xLvPtut2U+A2/xNLGZZpOGryP3Wu0gdV1ymH9tyj6ZoGFFv2ui1Zl+sviI4FfhVYHr ayIg== X-Gm-Message-State: AIkVDXL0gixSJtLmfPatLmw5atHL372bk8skf+ouQdyN+GtJQ0UZkaF9kVkIvJKf5e6TMg== X-Received: by 10.84.215.2 with SMTP id k2mr33821500pli.58.1482538476204; Fri, 23 Dec 2016 16:14:36 -0800 (PST) Received: from [192.168.20.12] (c-73-19-52-228.hsd1.wa.comcast.net. [73.19.52.228]) by smtp.gmail.com with ESMTPSA id a11sm64658530pfe.96.2016.12.23.16.14.34 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 23 Dec 2016 16:14:35 -0800 (PST) From: "Ngie Cooper (yaneurabeya)" X-Pgp-Agent: GPGMail Content-Type: multipart/signed; boundary="Apple-Mail=_E2060269-FEC6-425C-9441-8376FC9A1DCF"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: [RFC] Change program prefix for bsnmpd for many files from snmpd to bsnmpd to avoid namespace pollution with net-snmp's snmpd daemon Date: Fri, 23 Dec 2016 16:14:33 -0800 Message-Id: Cc: harti@FreeBSD.org, syrinx@FreeBSD.org To: "freebsd-arch@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2016 00:14:37 -0000 --Apple-Mail=_E2060269-FEC6-425C-9441-8376FC9A1DCF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, One of the things I noticed when trying to add uniform logging = for bsnmpd (syslog.conf files, newsyslog.conf files) is that it clashes = with net-snmp (which uses /var/log/snmpd.conf, program name exposed to = syslog(9) is =E2=80=9Csnmpd"). Unfortunately, because the prefix is = uniform between many files, changing just the log file prefix will = change a number of other items. I=E2=80=99m asking for input on the = following change: - /etc/snmpd.conf -> /etc/bsnmpd.conf - /var/run/snmpd.pid -> /var/run/bsnmpd.conf I would also, create bsnmpd configs for newsyslog and syslogd = when making this change. The pidfile change is rather straightforward, but the config = file might catch some folks. Are there any major concerns with renaming = the config file? Thanks, -Ngie --Apple-Mail=_E2060269-FEC6-425C-9441-8376FC9A1DCF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJYXb3qAAoJEPWDqSZpMIYVmyEP/RFyA7sR6y0VPd8uaWkfSQSC pGEL8KwFPxpzVKErGytfW0pTXwlEDW4/YSWKwRN44LQYlD4cmMAqElo3x8G5xmeW 6QSWyBRVhQlGaPHMITpVFNNDsm6H423TMcUdcPCqmujan1+Rq7qeNqD/tyxYDJ6n 49YotknFmRL8FKBuj4CcxaTKnVUYvZvG0hzoFAP0aYJR3O3GAy6XE3wMqgFwhFKp QPWCyPMyoIBKmz78Dcot7Yosv/GltEf8SHy2MEJV15kzDEVoVvqdSR6Pvd+MUnKr QNtFx67ZXBiHbVLlvO7ffrNBTBYJdotfbxA13DsKf43BhDRr9TMqFuVrePUOGz0g WCdIcM4A2BsV038cLZIO7yllWiYl/TipYFFHiaQIzfG9SforLtbOTiAxjPMyQF/t wOa7wtXhheafgfq+VhCZoPnNPsJIH9luBTbGdSnaH0Tl8BZqL1JHDowwEDW0PTGV eIJ9xk87pFUwPQ0J4aiomXfQeBprjFY6IXvTygeFaLlemd9bpVL/yOwI7sMmMTkh uerWBeSxthV3+GQ+8KofZENcdixBVwXSQNCMpKbdflvga+wHzU/tmTwd03qpKY+5 5xN10KnRfB/EKp5hQSiVRBj6Pz173jWxXMOhBMY7lZ1BBZ3dv60CmV+aJX6Nvjty vYkBUNZ7iBcu3Gpb+PWo =CAgy -----END PGP SIGNATURE----- --Apple-Mail=_E2060269-FEC6-425C-9441-8376FC9A1DCF-- From owner-freebsd-arch@freebsd.org Sat Dec 24 22:23:17 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A0BBC8F5EE for ; Sat, 24 Dec 2016 22:23:17 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BD4481122; Sat, 24 Dec 2016 22:23:16 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA26003; Sun, 25 Dec 2016 00:23:14 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1cKuig-000HQr-Hg; Sun, 25 Dec 2016 00:23:14 +0200 Subject: Re: superio driver To: "Constantine A. Murenin" References: <0d588744-c83f-1f04-3c79-89cba63949a3@FreeBSD.org> Cc: "freebsd-arch@freebsd.org" From: Andriy Gapon Message-ID: Date: Sun, 25 Dec 2016 00:22:18 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Dec 2016 22:23:17 -0000 On 04/11/2016 00:30, Constantine A. Murenin wrote: > On 3 November 2016 at 05:16, Andriy Gapon wrote: >> On 17/10/2016 11:19, Andriy Gapon wrote: >>> >>> I would like to add a driver for Super I/O chips: >>> https://reviews.freebsd.org/D8175 >>> In the review request you can find my rationale and description for the driver >>> as well as some question about its design. >>> At this point I am looking mostly for some help with the design, but any reviews >>> of the current code are also appreciated. >> >> Still looking for comments and suggestions. >> Thanks! > > I think it's an interesting idea; I've had it a few years ago in the > context of OpenBSD, but didn't pursue it further due to a lack of much > uniformity around the Super I/O chips from different vendors. > > LPC Super I/O chips from Winbond/Nuvoton and ITE do use a separate ISA > port for its Hardware Monitor functionality (often 0x290/8), and the > exact port number itself is still best be discovered through the Super > I/O driver (0x2e/2). On the other hand, the watchdog timer is > nonetheless usually implemented within the confines of the main Super > I/O logic itself (e.g., 0x2e/2). > > http://mail-index.netbsd.org/tech-kern/2010/02/17/msg007343.html > http://mdoc.su/f110/wbwd.4 Nod. > In the case of Winbond, as per above, it did result in two drivers > being there in {Net,Open,DragonFly}BSD, one for the Super I/O that > does the discovery (and which might potentially have support for the > watchdog functionality), and another for the Hardware Monitor (the > lm-compatible one, which can also attach directly on I2C in > {Net,Open}BSD): > > http://bxr.su/DragonFly/sys/dev/powermng/wbsio/wbsio.c > > http://mdoc.su/n,o,d/wbsio.4 > http://mdoc.su/n,o,d/lm.4 Nod. > However, in the case of http://mdoc.su/o/viasio.4 and > http://mdoc.su/n/itesio.4, the logic of Super I/O and Hardware Monitor > may be stuffed together into a single driver, even though both > functions are still performed at separate IO ports (look for the > multiple bus_space_map(9) calls, in fact, viasio(4) has separate ports > not only for the Hardware Monitor function, but for the Watchdog Timer > as well, something which is unique to it and is not shared by ITE and > Winbond/Nuvoton chips; nonetheless, all of these ports are claimed by > a single driver): > > http://bxr.su/OpenBSD/sys/dev/isa/viasio.c#viasio_hm_init > http://bxr.su/NetBSD/sys/dev/isa/itesio_isa.c#itesio_isa_attach I see. > If support for all of these Super IO chips would be implemented in a > single driver That's not a goal, it may or may not happen. > then FreeBSD might as well end up with its own version > of what the I2C bus scan is on OpenBSD: > > http://bxr.su/OpenBSD/sys/dev/i2c/i2c_scan.c#iic_probe_sensor I think that we would never have anything like this in FreeBSD. Poking random I2C / SMBus devices is considered too dangerous. But that's not important for this discussion. > Which is not necessarily a bad thing, and, in fact, provides an I2C > discovery interface which is very similar to what the situation is on > the more well designed platforms compared to x86, e.g., those that > have a smart property-based discovery mechanism via Open Firmware. > > However, realistically, I am not sure what will we gain in this case > of probing for all of these Super I/O chips from a single bus driver; > in fact, as you've already noticed yourself, even the probing > mechanisms are already different between ITE and Winbond/Nuvoton, so, > what exactly do we gain by having it all together in a new bus? My idea is very simple. A Super I/O chip usually implements many functions. There can be multiple drivers for the same Super I/O, each of them handling a subset of the functions. Each of the drivers may need some extra information about the Super I/O like its revision or its pin mapping configuration. So, the idea is that the code and logic that can be shared between those multiple drivers is contained in the superio driver. The drivers for individual functions attach to the superio driver and use its services. Some drivers may need those services only at the attach time to discover the version of the Super I/O and required resources (somewhat like wbsio and lm in other BSDs). Other drivers (like watchdog drivers) may continuously use those services to access the Super I/O's registers. Also, the superio driver would act as an arbiter for selecting the current LDN in this scheme. Finally, the individual drivers may have multiple attachments. E.g. a driver for a HWM function may have an isa attachment and a superio attachment (and possibly an I2C attachment). So, it would be up to a user to specify how the driver should attach. In some configurations the superio driver could be excluded at all. In others it could be used to auto-discover the Super I/O device and its configuration for the HWM component. Hope that this makes some sense. -- Andriy Gapon