From nobody Wed Aug 16 02:46:02 2023 X-Original-To: dtrace@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQXZp2QHvz4q4Nc for ; Wed, 16 Aug 2023 02:46:26 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE Root Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQXZm1mG6z4jHb; Wed, 16 Aug 2023 02:46:24 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 2a01:4f8:13b:39f::9f:25 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net; dmarc=none Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id F01638D4A178; Wed, 16 Aug 2023 02:46:19 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id F3E5D5C3A830; Wed, 16 Aug 2023 02:46:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id c4n6AaVI7AaS; Wed, 16 Aug 2023 02:46:03 +0000 (UTC) Received: from strong-iwl0.sbone.de (strong-iwl0.sbone.de [IPv6:fde9:577b:c1a9:4902:b66b:fcff:fef3:e3d2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 0D2A95C3A82F; Wed, 16 Aug 2023 02:46:02 +0000 (UTC) Date: Wed, 16 Aug 2023 02:46:02 +0000 (UTC) From: "Bjoern A. Zeeb" To: Mark Johnston cc: dtrace@freebsd.org Subject: Re: DTrace, kernel loader, unknown probes, enable on load? In-Reply-To: Message-ID: <5p24r5s1-9s7n-0644-4n27-0o82876pn107@yvfgf.mnoonqbm.arg> References: X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: A discussion list for developers working on DTrace in FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-dtrace List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-dtrace@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Spamd-Result: default: False [-2.21 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.91)[-0.913]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f8:13b:39f::9f:25]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[dtrace@freebsd.org]; R_DKIM_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[zabbadoz.net]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQXZm1mG6z4jHb On Thu, 24 Mar 2022, Bjoern A. Zeeb wrote: Hi, > On Mon, 14 Feb 2022, Mark Johnston wrote: > > Hi, > > sorry I had lost track .. > >> On Sat, Feb 12, 2022 at 12:16:45AM +0000, Bjoern A. Zeeb wrote: >>> On Fri, 11 Feb 2022, Mark Johnston wrote: >>> >>>> It appears to be sufficient to simply move the kld_load hook to before >>>> module registration, patch below. In the case of a subsequent error, >>>> the unload hook is called so DTrace gets a chance to clean up. I can't >>>> see any reasons not to move it, though there's at least one non-dtrace >>>> consumer that needs a look. >>> >>> HWPMC? >> >> Yes. >> >>> >>> It does work for my case with -Z which will ease work massively. >>> I can't wait for the "morning" and more time then to look at things :-) >>> >>> Please put me on subscribers should you put up a review. >> >> So there's one wrinkle I haven't thought through: when FBT probes are >> enabled in a KLD, we use the kld_unload_try eventhandler to block >> unloading of the module. Now, if we permit FBT probes to be enabled in >> a KLD before its sysinits run, then the kldload might fail, and the >> kernel linker will try to unload the module. But then FBT will block >> the unload. What's the right thing to do there? > > I dunno. > > I can only say that the local change has helped quite a few times in the > last month in my dev tree. > > Shall we add a PR or review and track progress there with a possibly > wider audience? I just got bitten by this again running some older scripts missing output. Thankfully remembered the adjustment. Is there any way we can solve this proper? /bz >>> Thanks a lot Mark and a happy weekend! >>> Bjoern >>> >>> >>>> diff --git a/sys/kern/kern_linker.c b/sys/kern/kern_linker.c >>>> index 2e4c95f16c8f..55661b9f9aa2 100644 >>>> --- a/sys/kern/kern_linker.c >>>> +++ b/sys/kern/kern_linker.c >>>> @@ -452,6 +452,7 @@ linker_load_file(const char *filename, linker_file_t >>>> *result) >>>> if (error != ENOENT) >>>> foundfile = 1; >>>> if (lf) { >>>> + EVENTHANDLER_INVOKE(kld_load, lf); >>>> error = linker_file_register_modules(lf); >>>> if (error == EEXIST) { >>>> linker_file_unload(lf, LINKER_UNLOAD_FORCE); >>>> @@ -472,7 +473,6 @@ linker_load_file(const char *filename, linker_file_t >>>> *result) >>>> return (ENOEXEC); >>>> } >>>> linker_file_enable_sysctls(lf); >>>> - EVENTHANDLER_INVOKE(kld_load, lf); >>>> *result = lf; >>>> return (0); >>>> } >>>> >>>> >>> >>> -- >>> Bjoern A. Zeeb r15:7 >> >> > > -- Bjoern A. Zeeb r15:7 From nobody Wed Sep 6 19:24:32 2023 X-Original-To: freebsd-dtrace@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rgskr70rVz4s6QR; Wed, 6 Sep 2023 19:24:36 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rgskq3jNtz4HHT; Wed, 6 Sep 2023 19:24:35 +0000 (UTC) (envelope-from domagoj.stolfa@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=Noq59pHl; spf=pass (mx1.freebsd.org: domain of domagoj.stolfa@gmail.com designates 2a00:1450:4864:20::52d as permitted sender) smtp.mailfrom=domagoj.stolfa@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-5230a22cfd1so139597a12.1; Wed, 06 Sep 2023 12:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694028274; x=1694633074; darn=freebsd.org; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=h03ubsRKFXkbSODWtBQsKIockKIYbyFEyezMTY12Qj4=; b=Noq59pHluT9v3Ltnq+r6uPzDp4ZFMRLh4qOvgmobvoS86ksBdvCGjuG+BjHNneyjxJ m89LAFf7Jt31bUJlofBMjC6KnjawreOp09IcIcs4d/EgKQToeexcsKrxfv/l64IzY+dm LnXuLVaOk16WHxzJ0c3goYtCnDTt4xf9FrtVqxqO6FoDVCc4shC8ltmM/xWTKNNYfV49 OUgtI/3naNzghXDufIwfNvsnEmkQbxvouZ8kgmnSti3hx41QJc4aPkZz82dPGwLEhTSW La4BYI8SGavJl78cuKoe5Q6BbNWXr5R9ZGrB93CmfeDk3NIiezu6YcAReubj06fdEJ6V svQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694028274; x=1694633074; h=content-transfer-encoding:subject:from:to:content-language :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=h03ubsRKFXkbSODWtBQsKIockKIYbyFEyezMTY12Qj4=; b=Xe4EdkCV0GOkNGMmGIHig0BOFTCcTaPfPyTpxPDTR1ExBYpsnn28mGqjh5o3Ffd4Fo fu21NR0Y/tjsvBst18/H5+DIjwNfvVDD87rMSL8I3rzsY0d8l6z6W8x6QDt8buw4Top2 EZNq5HaMf1/1DWs+ik8rX87qkjCLhknw0Eoc30LPhBMlQ4t25aM+wU/Ibzz/vlVxjTBP EyCj4pvqeV9+GPHnEGAWl4U+VZbERqnh7037xcmdMK6u7LpYd+zJPCXHRVnltcQpgjJU v52d93ebKEjut3t6xfM1TxYuC6uMHJoXxKCBOiWHqMYKdCpxUz2OOH1h8XP8/943Axnu 22DQ== X-Gm-Message-State: AOJu0Yzj620SKzLmD9/w5kVqgOJixg9S51DtQ2N5fHZesgdeODM1Ba1k iTfCLg9Gre7sK09gtxXGvsCcx8seg8cUTA== X-Google-Smtp-Source: AGHT+IF44CxPP+GSBPYJHqv/u2CaGBya0FO/lh1JtpaXGwm+Jg8/u9vIvkulEjLGjkcapucgOwxJ2Q== X-Received: by 2002:a05:6402:70c:b0:52d:212d:78e8 with SMTP id w12-20020a056402070c00b0052d212d78e8mr3037913edx.34.1694028273343; Wed, 06 Sep 2023 12:24:33 -0700 (PDT) Received: from [192.168.10.99] (226-31.dsl.iskon.hr. [89.164.226.31]) by smtp.gmail.com with ESMTPSA id r14-20020a056402034e00b00523d2a1626esm8844254edw.6.2023.09.06.12.24.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Sep 2023 12:24:33 -0700 (PDT) Message-ID: <8670bde3-58bf-66db-b228-a7d7bc94a48b@gmail.com> Date: Wed, 6 Sep 2023 21:24:32 +0200 List-Id: A discussion list for developers working on DTrace in FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-dtrace List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-dtrace@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US To: freebsd-hackers@freebsd.org, freebsd-dtrace@freebsd.org From: Domagoj Stolfa Subject: Structured/machine-readable output for DTrace Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.85 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.85)[-0.848]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52d:from]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org,freebsd-dtrace@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TAGGED_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Rgskq3jNtz4HHT Hi: I've implemented machine-readable output in the form of JSON, XML and HTML into libdtrace, and by extension dtrace(1) using libxo. The goal is not to replace linking to libdtrace in any way when one wishes to build a custom application around it, but rather to supplement it (as it would now also support this form of output by default) and act as a middle-ground between linking directly to libdtrace and parsing the pretty-printed dtrace(1) output that is available today (but not replace it). In order to get it to work, you can simply run dtrace(1) as normal, but add -x oformat=json|xml|html|plain. I've posted the patch up for review if anyone wants to take a look or try it out [1] and posted a hacked up Python program (by no means good Python code) as an example of how this might be used [2]. I would appreciate any feedback you might have. As this patch is still in its early stages however, there isn't much documentation available for it. I aim to address this soon. [1]: https://reviews.freebsd.org/D41745 [2]: https://reviews.freebsd.org/D41745 -- Domagoj