From nobody Mon Aug 16 13:57:36 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B49831768585; Mon, 16 Aug 2021 13:57:47 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward100p.mail.yandex.net (forward100p.mail.yandex.net [77.88.28.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpG2L37yDz4W5g; Mon, 16 Aug 2021 13:57:46 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from sas1-f446987054ad.qloud-c.yandex.net (sas1-f446987054ad.qloud-c.yandex.net [IPv6:2a02:6b8:c08:bf8a:0:640:f446:9870]) by forward100p.mail.yandex.net (Yandex) with ESMTP id D6EEB5980EEC; Mon, 16 Aug 2021 16:57:37 +0300 (MSK) Received: from sas2-1cbd504aaa99.qloud-c.yandex.net (sas2-1cbd504aaa99.qloud-c.yandex.net [2a02:6b8:c14:7101:0:640:1cbd:504a]) by sas1-f446987054ad.qloud-c.yandex.net (mxback/Yandex) with ESMTP id 2aMQ2cidaJ-vbIOOCIs; Mon, 16 Aug 2021 16:57:37 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1629122257; bh=1DfMwVSzFZAD4za0t5B4qRHFNB/wSjp0j6OsBzif21w=; h=In-Reply-To:References:Date:Message-ID:From:To:Subject; b=UYDEW1S/ypMwoNqyIZE8GchX+e9e+8qWJeA2lHvNbghjkmEek6DWzJ+fecqTgr3uL MCBpnz7JbgpU189ybXXj4S8RvIVminpDk6gLaSj+AMtW+NvuvPGnMNZx0Sek0WnjEd 7Vnbwtl1A5Hg+zsU7vXqJsBhd7/5jXpHrQKUMdNU= Received: by sas2-1cbd504aaa99.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id wViyCcSAFE-vbUmNhc8; Mon, 16 Aug 2021 16:57:37 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated To: Eugene Grosbein , alfadev , "freebsd-hackers@FreeBSD.org" , "freebsd-ipfw@FreeBSD.org" , "Alexander V. Chernikov" References: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> From: "Andrey V. Elsukov" Message-ID: <2f5f9f6e-27da-1d4a-e462-93ef12f84d33@yandex.ru> Date: Mon, 16 Aug 2021 16:57:36 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="wbct9JzOgqnVweG2vEHvgb1NQueBvX6kw" X-Rspamd-Queue-Id: 4GpG2L37yDz4W5g X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b="UYDEW1S/"; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of bu7cher@yandex.ru designates 77.88.28.100 as permitted sender) smtp.mailfrom=bu7cher@yandex.ru X-Spamd-Result: default: False [-6.07 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip4:77.88.0.0/18]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[yandex.ru:+]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; NEURAL_HAM_SHORT(-0.97)[-0.968]; SIGNED_PGP(-2.00)[]; FREEMAIL_TO(0.00)[grosbein.net,protonmail.com,FreeBSD.org,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:77.88.0.0/18, country:RU]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[77.88.28.100:from] X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --wbct9JzOgqnVweG2vEHvgb1NQueBvX6kw Content-Type: multipart/mixed; boundary="nrBYf2gus7xPntRhO5VGfCPpYsJREMn9z"; protected-headers="v1" From: "Andrey V. Elsukov" To: Eugene Grosbein , alfadev , "freebsd-hackers@FreeBSD.org" , "freebsd-ipfw@FreeBSD.org" , "Alexander V. Chernikov" Message-ID: <2f5f9f6e-27da-1d4a-e462-93ef12f84d33@yandex.ru> Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated References: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> In-Reply-To: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> --nrBYf2gus7xPntRhO5VGfCPpYsJREMn9z Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable 10.08.2021 08:08, Eugene Grosbein =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > Such ruleset could decrease filtering overhead several times but I'm > afraid that ipfw is not right tool for this task at the moment. >=20 > ipfw has "tables" to optimize large list matching and they perform > great but for layer3 IP matching, not for layer2 MAC matching. We have a patch that adds ability to keep MAC addresses in the tables. I hope we will push it into upstreem soon. --=20 WBR, Andrey V. Elsukov --nrBYf2gus7xPntRhO5VGfCPpYsJREMn9z-- --wbct9JzOgqnVweG2vEHvgb1NQueBvX6kw Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAmEabtAFAwAAAAAACgkQAcXqBBDIoXq2 TQgAsQK2y8hlwBojnQ25ibtxtbhFcaXOZRsCVJ9mgMNfsHyjG0LlSLR38wF851I7fPUf7gI+3tWt j9UbNVdxnSwFRvdhu9leXS7RUfDo4GhwCoFt8KSbtSvi9AlOMqiXgMolE1CY/0lD9JfHvMa2VnFN eiTX+0hPbt8t+T6lnXg+Az0h0h/OtDEodDNIy1PPKteGAfng+kG8cOVulLeLsk+KLPlM6SMfGw3p OO+F2aM8A0sl+KIrRdc86vUbQ54lrmeNBG0OS51k9Vx+QVWxUcPN04EeRFgU7Q2JWRwsgFLhZrHL GVqTar55rFf+2IB4h6MrJXOE6UxMj90Ie/e8veCDNw== =EX5L -----END PGP SIGNATURE----- --wbct9JzOgqnVweG2vEHvgb1NQueBvX6kw-- From nobody Mon Aug 16 15:19:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 66C93176E9CD; Mon, 16 Aug 2021 15:20:22 +0000 (UTC) (envelope-from christos@freebsd.org) Received: from christos (mail.margiolis.net [95.179.159.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpHsd5QSmz4gLG; Mon, 16 Aug 2021 15:20:21 +0000 (UTC) (envelope-from christos@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=default; bh=lfTwti2lb0qR Vxj3+Gu8M4nrxi3Wr1J7BDsDdKIGV1k=; h=subject:cc:to:from:date; d=margiolis.net; b=pRjBjDfSnwwNe53e6TJ1lxQUfjHc3tzm2Pr7d5Qz5qfzvu+m+Wd 1lwjzuwG8/uhb0v2YD2cYYQ3thkL2txqI+gobb3iv1EiE4Y2DAo2YYO9f3uIvQ4H67KqL6 1JgV0bRZYlHR7qp/godWONGGs0yuYdyEIO4iAaGuhi2wKLaXaA= Received: from pleb (ppp-94-64-49-138.home.otenet.gr [94.64.49.138]) by christos (OpenSMTPD) with ESMTPSA id 7f220712 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Mon, 16 Aug 2021 15:20:12 +0000 (UTC) Date: Mon, 16 Aug 2021 18:19:56 +0300 From: Christos Margiolis To: soc-status@freebsd.org Cc: hselasky@freebsd.org, freebsd-hackers@freebsd.org Subject: [GSoC'21 Weekly Update #10] Sound mixer improvements Message-ID: <20210816151956.zonyz5hcrq245qr5@pleb> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 4GpHsd5QSmz4gLG X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:20473, ipnet:95.179.144.0/20, country:US] X-ThisMailContainsUnwantedMimeParts: N Hello, This was the final week of GSoC 2021. Luckily, the project was already finished, so I could focus more on my country's wildfire situation I talked about in my last email. This week I'll be creating my final evaluation and the GSoC project will be officially done! If anyone reading this is interested in taking part in next year's GSoC, go for it without a second thought -- the whole thing has been a great learning experience and lots of fun. :-) Thanks to hselasky@ for mentoring me and the rest of the FreeBSD team for being responsive and helpful! The project's Wiki article can be found at: https://wiki.freebsd.org/SummerOfCode2021Projects/SoundMixerImprovements -- Christos From nobody Mon Aug 16 17:53:55 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A4A98174F5BC for ; Mon, 16 Aug 2021 17:53:58 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpMGt4HH2z4s2l for ; Mon, 16 Aug 2021 17:53:58 +0000 (UTC) (envelope-from debdrup@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1629136438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iLDhxVVVyzCDSF/9j/tUFONzQfyiIlAXTK8ajcXm280=; b=p6vSnGbMuveYGoc+kG7fr3uo4WJg2xIJhRsGy0DXSLs2y7CkDHe80pNE9JDRRvntNMdExZ rZzUmiFJWV3ukGYIcS3npMW7WINqdSep1+flga19HpBXmUYP/qvN2ndpp5O1j6wN+QRZfC k1qQJFVaRAQ5rlX781zpCTO6am9F3jQgEbH7/ksLRmxAdJ8hdbJrcK00mD3+b+GdXQ3UbH tYWE3n8/59T57Ibw+HnQBGLZyIc2LDUaxWBjygFJqWQS2CrLsQAEtCUIxRKad/zzV5OczY 2AoVVDVKsx4TNJUTClOGnHX7tc1vgbRTrIvikJDzricIsXdBHjzVi8BW3oX0oQ== Received: by freefall.freebsd.org (Postfix, from userid 1471) id 79B65842B; Mon, 16 Aug 2021 17:53:58 +0000 (UTC) Date: Mon, 16 Aug 2021 19:53:55 +0200 From: Daniel Ebdrup Jensen To: freebsd-hackers@freebsd.org Subject: Re: [GSoC'21 Weekly Update #10] Sound mixer improvements Message-ID: <20210816175355.uyeixeqfik4fbq6b@nerd-thinkpad.local> Mail-Followup-To: Daniel Ebdrup Jensen , freebsd-hackers@freebsd.org References: <20210816151956.zonyz5hcrq245qr5@pleb> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="623yevxoqqbpp37o" Content-Disposition: inline In-Reply-To: <20210816151956.zonyz5hcrq245qr5@pleb> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1629136438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iLDhxVVVyzCDSF/9j/tUFONzQfyiIlAXTK8ajcXm280=; b=IX6AC//yvrnOyCyxHtvf0qMEAbSW9BB2C7mGmQxzuQ0vDczpAyDx0dpwhb7Fh+rUxu6B4/ xl/Siyz0tAVdoH/+9SeZ0REj03CvzyjL/XHq8zpb7UOTEcT4fDjUcRU20fH9v5dEXbx9JN 5IO5Ky7fHPA6iGx4moxWw1SQpwooB76w+5yOeA8mCB1gAfMh8JGTD8lfV/Lz03yt4XY7w6 jRVbt6AK3cSl3b5VpViTCf7cLaEAfHAkcvrkG/TgyjG8KIwxogemkKJZbWwtRa1A6EBHHE 1nR63ErO1hu+q2vA7vHoRrWhz/EzMVAFOCuFfsjhwDZNn6ZoItZyjJeGzr5fGA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1629136438; a=rsa-sha256; cv=none; b=osa4fng7nmmn7LwejvILezl6LFqBpMdL5q7nAvtDmzHz140BhjxEDW/cSEkIicjqva+A7o hPvma4DhGD6sgVeg88PLTkZ6QOIK6KUTLZ7WumsZgjY3j3vMjKSOqZmAOxmaWckpb9lG2P EmHwyoqnuQKwSNaAFJcVjqtzn0U4VBsqIwMl9TmEkr/Zqq6N4aNDJvDsqefmYw10XpwTPv ovRCFDWKHUDVefaTaDjl+KhAaGbYF+LztI/qWqvAUJjlBjjeRVf7AtSu5YCO5/EWHXB2ix R5Zby66J3h6Zd6WwIVCo4w9Y+LXsF0JhMr/hl/tYt4i8h1u1idmiL9YbH8cRkQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --623yevxoqqbpp37o Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline On Mon, Aug 16, 2021 at 06:19:56PM +0300, Christos Margiolis wrote: >Hello, > >This was the final week of GSoC 2021. Luckily, the project was already >finished, so I could focus more on my country's wildfire situation I >talked about in my last email. This week I'll be creating my final >evaluation and the GSoC project will be officially done! > >If anyone reading this is interested in taking part in next year's GSoC, >go for it without a second thought -- the whole thing has been a great >learning experience and lots of fun. :-) > >Thanks to hselasky@ for mentoring me and the rest of the FreeBSD team >for being responsive and helpful! > >The project's Wiki article can be found at: >https://wiki.freebsd.org/SummerOfCode2021Projects/SoundMixerImprovements > >-- >Christos > Hi Christos, I'm glad you ended up finding a mentor for this project, following our several conversations about it, after I happened to mention it in passing. :) You've really run with it and done a great job! I hope to see you continue to work on things, and eventually get punished appropriately. ;) Yours appreciatively, Daniel Ebdrup Jensen --623yevxoqqbpp37o Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAmEapjNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN 87poPggAp5oHFLGZtzWvBSQ/hXTAxCyzszK69g/ajbF92OxPdwyTTie5T/hkI4zc hKR/MG9JG+m+yDmXVilRuZXr47W8kS4MwlDdDguSmj0EBN/tvVwDbUwQm97HfZYS FIJZlBhG3k2EOWElcNfwcqyLjQnPrUe2Q7F+UUKHd5NvWwSQfU4CYX+r/Dl9jQrk oX5+3/sTveHKJY2Zdf5ib3vDjmcaHXL26lOfv+uSxrwzrrjuR0WeaD0Dr1sKKIE2 MMhHIERxqz7biF88sIB/uSXb2uH/aF3/+nOBr21K22oAYutaUx/WFMPBnwIWF5lF TS7e6WvhKNTD4IqEXYzFwyRoT689Tg== =vMue -----END PGP SIGNATURE----- --623yevxoqqbpp37o-- From nobody Mon Aug 16 18:50:46 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 60F49175B29F for ; Mon, 16 Aug 2021 18:51:04 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpNXl4w3cz3D9b for ; Mon, 16 Aug 2021 18:51:03 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f51.google.com with SMTP id h63-20020a9d14450000b02904ce97efee36so22105859oth.7 for ; Mon, 16 Aug 2021 11:51:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=2facFIvElFB0X9KHlE2/Q+0ZVfqvIiLt+4hKQ6mW0oQ=; b=SIb6IshVvjoPTZcBpl+en84mu4Ek4bematYJ5oOGdGQLoANmZh0hqUdL1ZEG9qFGk5 TiC3qf+Zp4DJKxAWpM0zn6xVXYpOS8iiLiuYKoeK509RJISm+B+s0JHgVLKZSRImCzK7 NoJQcLdDoc4fwkb67lNusbba/hwDdpoURoolJBCRPmEytc+efUnBZLVy7zTh2QPpqTzc I2zRlIlc3iSO15uDch7sgYciesNxf2nmz8Bcs3ODzh2jKYdMHKvC4LawuRTEJPBpAzYx Qv/cHj6YKf1R0+6OQ2aXqKUVwP8WDlbJS9rWRUUWSRQ8CnV4HBBANY3UM6rUpXaY/Qa3 e0yw== X-Gm-Message-State: AOAM530ZtSopy1Uj4WHDJhOAdwd1LPffsC3wAAx5+wwmbXVBfJukTr2v /ZeCbfZSpFSrW/KAhB9eiXwYvloKMPlkDCZBH17crfsc X-Google-Smtp-Source: ABdhPJyxClNYRwfMZQlMiaOonFNJkb3HQrQQJXjz71PYclRxxDuJ9DXU3vLfc7pICrLg3lVWtGHIXJh5zI0sPo5m6Eg= X-Received: by 2002:a9d:450c:: with SMTP id w12mr158546ote.18.1629139857183; Mon, 16 Aug 2021 11:50:57 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Mon, 16 Aug 2021 12:50:46 -0600 Message-ID: Subject: sysctl is too slow To: FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000007a17d605c9b1aeb7" X-Rspamd-Queue-Id: 4GpNXl4w3cz3D9b X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.51 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-0.25 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.971]; RCVD_COUNT_TWO(0.00)[2]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[209.85.210.51:from]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.28)[-0.275]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.51:from]; NEURAL_SPAM_LONG(1.00)[1.000]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000007a17d605c9b1aeb7 Content-Type: text/plain; charset="UTF-8" ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace shows that almost all of its time is spent in sys_sysctl. ktrace shows that both ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl they care about: 1) To get the next oid 2) To get the sysctl's name 3) To get the oidfmt 4) To get the size of the value 5) To get the value itself. Each of these steps takes about equal time, and together all five take about 100us. If the time per call is mostly syscall overhead, then the process could be sped up by 80% by combining all of these things into a single syscall: return the next oid, its name, its format, the size of its value, and optimistically the value itself, assuming the user passed a sufficiently large buffer. Am I missing something? Is there any other reason why sysctl is so slow? Or should I forget about it, and try to export ZFS's dataset stats through devstat instead? -Alan --0000000000007a17d605c9b1aeb7-- From nobody Mon Aug 16 19:30:51 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 45D34175DEBB for ; Mon, 16 Aug 2021 19:30:53 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpPQj0yy5z3HLR; Mon, 16 Aug 2021 19:30:53 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x12a.google.com with SMTP id c24so36561660lfi.11; Mon, 16 Aug 2021 12:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=aUshE31Bb+tTsBJe5u3K1ZBUpPGGDq+j2q8w7MIEHmY=; b=aYHDUYPVrUEE4DKecaBPAeCP/Vg3wGeQKaWFAbJyMkOxWhGXffY9n7kibOTnlEUWsA 7IfJVvgvi129zz8vmuUiFiXbqCsRe/kHpyACR8WzqhW0CqHOSaUqbXGsJtBIo7ICivXT RChYE+7gD3ibaUwAOXUlD2H8J0EPYLoOPQIPwaWrVfPsx2d1n8pDS1JZsAPykFwzxECQ ckOR3GFypQv39VbO8JNFqdHx4oLUSyUTMaP5SvzCX1smjklaUhMmtz8TfgxU7ip5c+LV TV6jtQg9iZzS9dsRSYT0JRSrBzAVDpYwIXqWHbR8njn1htk/p5nGS1rGUXbtZooY83UI nsIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=aUshE31Bb+tTsBJe5u3K1ZBUpPGGDq+j2q8w7MIEHmY=; b=cuLQsgxvUD7JotjtLt6OVl5Ng9yKeBI1sD7G0Br7j1UmrXUcNkWF2rDHmaygeJxHhc yYqfJEcj6GxzPdIpQoRWLZeqJTekBgiKMANa6BBPagTgI8iV3PpFDNcf1gmmARawaanQ z5Pw0qZJ9OHwkNg6yy2u1oJfVh+6dEbIN3UeKZjev6xGKapefMWDGHgkSncb8+kyT5Ik 9PYu6812dAFqnvzL9q14d8emPs42f53Pkh/uAdoR1vsr0DIzSggrw0eRFw/ZtOj49pNM AvQV48dDvVbsihsPWHOnqdoX/FJT2VRAitv1z+VdjI6ner/NfWjnDlG7kgdfU/OBN7bp DiBg== X-Gm-Message-State: AOAM531jLp6iuPmRB99uS04GMKIZWjh1kN7MRCEK7ReX+XOiwezqDTJx HbzTiep3oK0GgR03oQBv2z+wKLUkiBAMsPIF2FoF9w9a X-Google-Smtp-Source: ABdhPJzhJbYeDo3QCQpkWMChkHT+4tt405keXnp7QglNtXgp6bmb+3UK6k2VPJzTJBBujaDaAHdM+y+1DLICKWWzvvs= X-Received: by 2002:ac2:4e8c:: with SMTP id o12mr122210lfr.374.1629142251856; Mon, 16 Aug 2021 12:30:51 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a2e:6f04:0:0:0:0:0 with HTTP; Mon, 16 Aug 2021 12:30:51 -0700 (PDT) In-Reply-To: References: From: Mateusz Guzik Date: Mon, 16 Aug 2021 21:30:51 +0200 Message-ID: Subject: Re: sysctl is too slow To: Alan Somers Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GpPQj0yy5z3HLR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Last time I checked lookup of a sysctl was very bad with linear scans all over. Short of complete revamp of the entire thing I would start with replacing the scans with a RB tree at each level. As is if you indeed have 5000 datasets, you are doing increasingly longer walks. On 8/16/21, Alan Somers wrote: > ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace shows > that almost all of its time is spent in sys_sysctl. ktrace shows that both > ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl > they care about: > > 1) To get the next oid > 2) To get the sysctl's name > 3) To get the oidfmt > 4) To get the size of the value > 5) To get the value itself. > > Each of these steps takes about equal time, and together all five take > about 100us. If the time per call is mostly syscall overhead, then the > process could be sped up by 80% by combining all of these things into a > single syscall: return the next oid, its name, its format, the size of its > value, and optimistically the value itself, assuming the user passed a > sufficiently large buffer. > > Am I missing something? Is there any other reason why sysctl is so slow? > Or should I forget about it, and try to export ZFS's dataset stats through > devstat instead? > -Alan > -- Mateusz Guzik From nobody Mon Aug 16 21:13:11 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9E53817654A4 for ; Mon, 16 Aug 2021 21:13:29 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oo1-f48.google.com (mail-oo1-f48.google.com [209.85.161.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpRj53x4Fz3h8x for ; Mon, 16 Aug 2021 21:13:29 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oo1-f48.google.com with SMTP id m11-20020a056820034b00b0028bb60b551fso1176262ooe.5 for ; Mon, 16 Aug 2021 14:13:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Jqo4SKACkIl/X3UniObK5Nd1ZXiN055iAWP4MonNqP0=; b=BBF/5OoC19Qwz/tt7CeM9CWKFK9OX2raSt9nImRB62aICtDbNDXzO024XXnefAyKKL 5zc42Gqr3IJZxEFqhaqcN7tl+fkmNBlwAnk3rj4CZClr1Y0h3Tpn8tA0YW06XAUzJ9To uaGLqHq1MWEjbICm3gid11skxW6tUwfd3JgvZ2KC8TpTKEfR6c7kmXWpLoeErTkS5ydD Jzw12nDP4cjuQSgzeHRst8OSQ++cZIzmsC1N+UUhkT8LfjOoroKg5oiQ7lB7crFdSFoc qsRMXs4lS8UPKJIYaP8bWy352KtnawwTacGcFgaL03YNjkDYVevTh5lyg+6QIpOp+bOO wNYg== X-Gm-Message-State: AOAM533+GcAmJTWmL42QqRYXUf202qcv3oXAm93EX9bx8Ij8R88gbLfN V/wEiaWQv1xguk4npnBvVjwJptCvCzTG4y8OYuo= X-Google-Smtp-Source: ABdhPJwwB3BtRvswaTP0tBp93lGfyt6kPrNhzLM16kZj4z457vEK8102fP0nXj3j3Dc9dyH8rxkD70jCr5q6lqtKe30= X-Received: by 2002:a4a:b481:: with SMTP id b1mr193595ooo.79.1629148402810; Mon, 16 Aug 2021 14:13:22 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Mon, 16 Aug 2021 15:13:11 -0600 Message-ID: Subject: Re: sysctl is too slow To: Mateusz Guzik Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000d6013605c9b3ab76" X-Rspamd-Queue-Id: 4GpRj53x4Fz3h8x X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000d6013605c9b3ab76 Content-Type: text/plain; charset="UTF-8" Yes, I see what you're talking about now. There are a bunch of linked lists in sysctl_find_oid etc. Good point. -Alan On Mon, Aug 16, 2021 at 1:30 PM Mateusz Guzik wrote: > Last time I checked lookup of a sysctl was very bad with linear scans all > over. > > Short of complete revamp of the entire thing I would start with > replacing the scans with a RB tree at each level. As is if you indeed > have 5000 datasets, you are doing increasingly longer walks. > > On 8/16/21, Alan Somers wrote: > > ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace > shows > > that almost all of its time is spent in sys_sysctl. ktrace shows that > both > > ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl > > they care about: > > > > 1) To get the next oid > > 2) To get the sysctl's name > > 3) To get the oidfmt > > 4) To get the size of the value > > 5) To get the value itself. > > > > Each of these steps takes about equal time, and together all five take > > about 100us. If the time per call is mostly syscall overhead, then the > > process could be sped up by 80% by combining all of these things into a > > single syscall: return the next oid, its name, its format, the size of > its > > value, and optimistically the value itself, assuming the user passed a > > sufficiently large buffer. > > > > Am I missing something? Is there any other reason why sysctl is so slow? > > Or should I forget about it, and try to export ZFS's dataset stats > through > > devstat instead? > > -Alan > > > > > -- > Mateusz Guzik > --000000000000d6013605c9b3ab76-- From nobody Mon Aug 16 22:03:45 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D391017687C6 for ; Mon, 16 Aug 2021 22:03:55 +0000 (UTC) (envelope-from joerg@bec.de) Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpSqG4tFlz3mLk for ; Mon, 16 Aug 2021 22:03:54 +0000 (UTC) (envelope-from joerg@bec.de) Received: (Authenticated sender: joerg@bec.de) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 5D127200002 for ; Mon, 16 Aug 2021 22:03:47 +0000 (UTC) Date: Tue, 17 Aug 2021 00:03:45 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Subject: Re: sysctl is too slow Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4GpSqG4tFlz3mLk X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of joerg@bec.de has no SPF policy when checking 217.70.178.232) smtp.mailfrom=joerg@bec.de X-Spamd-Result: default: False [-0.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[joerg]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_LOW(-0.10)[217.70.178.232:from]; DMARC_NA(0.00)[bec.de]; NEURAL_SPAM_SHORT(1.00)[0.997]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:29169, ipnet:217.70.176.0/20, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[217.70.178.232:from] X-ThisMailContainsUnwantedMimeParts: N On Mon, Aug 16, 2021 at 09:30:51PM +0200, Mateusz Guzik wrote: > Last time I checked lookup of a sysctl was very bad with linear scans all over. > > Short of complete revamp of the entire thing I would start with > replacing the scans with a RB tree at each level. As is if you indeed > have 5000 datasets, you are doing increasingly longer walks. The RB tree is what NetBSD is doing. The alternative that might be simpler is to have a flag bit in the node "child nodes are sorted" and lazily apply a sort whenever the node is scanned first, dropping the flag when new children are attached or detached. That would allow a binary search without extra memory cost? Joerg From nobody Mon Aug 16 22:05:25 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 878041769395 for ; Mon, 16 Aug 2021 22:05:29 +0000 (UTC) (envelope-from joerg@bec.de) Received: from relay12.mail.gandi.net (relay12.mail.gandi.net [217.70.178.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpSs41r0nz3n6L for ; Mon, 16 Aug 2021 22:05:28 +0000 (UTC) (envelope-from joerg@bec.de) Received: (Authenticated sender: joerg@bec.de) by relay12.mail.gandi.net (Postfix) with ESMTPSA id 45CBA200003 for ; Mon, 16 Aug 2021 22:05:27 +0000 (UTC) Date: Tue, 17 Aug 2021 00:05:25 +0200 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Subject: Re: sysctl is too slow Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4GpSs41r0nz3n6L X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of joerg@bec.de has no SPF policy when checking 217.70.178.232) smtp.mailfrom=joerg@bec.de X-Spamd-Result: default: False [-0.20 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[joerg]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RWL_MAILSPIKE_POSSIBLE(0.00)[217.70.178.232:from]; DMARC_NA(0.00)[bec.de]; NEURAL_SPAM_SHORT(1.00)[1.000]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:29169, ipnet:217.70.176.0/20, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[217.70.178.232:from] X-ThisMailContainsUnwantedMimeParts: N On Tue, Aug 17, 2021 at 12:03:45AM +0200, Joerg Sonnenberger wrote: > On Mon, Aug 16, 2021 at 09:30:51PM +0200, Mateusz Guzik wrote: > > Last time I checked lookup of a sysctl was very bad with linear scans all over. > > > > Short of complete revamp of the entire thing I would start with > > replacing the scans with a RB tree at each level. As is if you indeed > > have 5000 datasets, you are doing increasingly longer walks. > > The RB tree is what NetBSD is doing. Sorry, it is what it should have been doing. I wonder if I am bored enough to implement the shuffling now... Joerg From nobody Tue Aug 17 01:19:35 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5FC17174E035 for ; Tue, 17 Aug 2021 01:19:44 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lj1-x22b.google.com (mail-lj1-x22b.google.com [IPv6:2a00:1450:4864:20::22b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpY9D1yjVz4YwQ; Tue, 17 Aug 2021 01:19:44 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lj1-x22b.google.com with SMTP id f2so8484953ljn.1; Mon, 16 Aug 2021 18:19:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=6BulJIAFN2yoT2qc3P1XkMyuZiDq6zHdDPP7d7x0BXM=; b=MJy9XX6/O5736N/NNN/vjczqvDuKSSyCxqr0Ej0crwPZuDAZAum546d1ZMflhVjf1S lVnXa7CuFHfB88/1DFDqQVK5V/cirhf24KMrOKlPnkWB7XDsGy0bAq1wYdzU9l4I8NVK zjMLSNCLixiHuC/0CAv6/KBKtNufTkIb6x1b/QYx+Jcn8CkLMFcYY3jpe9vLzCe5PpG1 kDMIz9iyG3k0sRl892Imt2W/GcU/SBrp2S5j/wY8WSKa/edZ7bORh/F9AC4P0BN5IydP b7Qt+sS7RMD0Cc43UVNzvWnTvkH92IdKsmVTyV30bTnlb3MVY8LXtUCJP4hBJkfTDvaK oesQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=6BulJIAFN2yoT2qc3P1XkMyuZiDq6zHdDPP7d7x0BXM=; b=mPQrFvpfXsFGshqNfJ7cBYJy+6xH4BxnywDQPtiF4fYw6B93t2Z5mXjIIw1yEPBkVd 4h/q/LfNTFVi60g/zKclR5kDgcGUYpLLDDOyz9cGNN+COVoEvcNRc1voBksjn+aEr222 wmg3Dwd21MKruVXW+pJJOThxphZD+4wn2+enUZM802RSSqi+pgp0OJ1REI04N0wM5cVY rEjp+Zg824/wA5kxjRQBqh8AChm/WocQdETJuo5k1dluQQ5/HNYwQ+tI/k562DXsepJ1 FPFgF6aClbFIAxi4KOKW6URFmNbz9uj95/cUuhMxNmb6slzBENdMsso8rwCV5Bpx2DVW SvaA== X-Gm-Message-State: AOAM533rmvfs6AeavfTc5co+NBexc7BppnFcNA4K2yjfHwmEQec7ixpY 31YyBFtV0XoPM/MMZNkz01hHe2fImwvub1F8BgoYiuoo X-Google-Smtp-Source: ABdhPJziD5IxXeocbz8X4NdQwWyzU1gB07v2eaQAJmV2twbRrPoQO4bflJN0WZibHSsq7B438drSP9XARTTePVf7jGs= X-Received: by 2002:a2e:bd85:: with SMTP id o5mr811474ljq.313.1629163176649; Mon, 16 Aug 2021 18:19:36 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a2e:6f04:0:0:0:0:0 with HTTP; Mon, 16 Aug 2021 18:19:35 -0700 (PDT) In-Reply-To: References: From: Mateusz Guzik Date: Tue, 17 Aug 2021 03:19:35 +0200 Message-ID: Subject: Re: sysctl is too slow To: Alan Somers Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GpY9D1yjVz4YwQ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 8/16/21, Alan Somers wrote: > Yes, I see what you're talking about now. There are a bunch of linked > lists in sysctl_find_oid etc. Good point. > -Alan > You still want to get a flamegraph, chances are most of the problem is in zfs. > On Mon, Aug 16, 2021 at 1:30 PM Mateusz Guzik wrote: > >> Last time I checked lookup of a sysctl was very bad with linear scans all >> over. >> >> Short of complete revamp of the entire thing I would start with >> replacing the scans with a RB tree at each level. As is if you indeed >> have 5000 datasets, you are doing increasingly longer walks. >> >> On 8/16/21, Alan Somers wrote: >> > ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace >> shows >> > that almost all of its time is spent in sys_sysctl. ktrace shows that >> both >> > ztop and sysctl(8) call sys_sysctl a total of five times for each >> > sysctl >> > they care about: >> > >> > 1) To get the next oid >> > 2) To get the sysctl's name >> > 3) To get the oidfmt >> > 4) To get the size of the value >> > 5) To get the value itself. >> > >> > Each of these steps takes about equal time, and together all five take >> > about 100us. If the time per call is mostly syscall overhead, then the >> > process could be sped up by 80% by combining all of these things into a >> > single syscall: return the next oid, its name, its format, the size of >> its >> > value, and optimistically the value itself, assuming the user passed a >> > sufficiently large buffer. >> > >> > Am I missing something? Is there any other reason why sysctl is so >> > slow? >> > Or should I forget about it, and try to export ZFS's dataset stats >> through >> > devstat instead? >> > -Alan >> > >> >> >> -- >> Mateusz Guzik >> > -- Mateusz Guzik From nobody Tue Aug 17 01:38:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 22CFC174FE66 for ; Tue, 17 Aug 2021 01:39:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f173.google.com (mail-oi1-f173.google.com [209.85.167.173]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpYbl0B0xz4cBx for ; Tue, 17 Aug 2021 01:39:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f173.google.com with SMTP id bj40so29641057oib.6 for ; Mon, 16 Aug 2021 18:39:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/Owrbu3+A1HIxibpSVCP6+UcCzcSvLQWV+GEUeOeACo=; b=qfI7X0j8tcA1+k0TZhBcPpXo0OhCssmNI26NlhYAoghRAtjg0DXgEBGYEPavWAEZ46 bOLLwkn/Qh3tQea1P7aHXnXhNYY/gUTNsx8ogjNs7pEovX6cznJBrYPwz//wiFTC/pGN ARx1htzehKxUWtAQDk24ksKpvACmktF7KRyDvgILx+mJ1jyvZEBR9j2YZ76W1xzmaXhx QoiHg/uwu7kGmOqNcsUp0vBRHAc1w9q65bCQP2bnOpa+b4jQNz8UO3EpsGU6Qe8kCOJA SXB/zQ7egp05ITyhn6PA3L+H3B6zJ69rHbcMrouHCXgp9Hxl1AW8YPkiD2IxgEGEPnCO m6xg== X-Gm-Message-State: AOAM531s4n0GYa6m5VWz4rNwehUxGRLn1I8nifcrukmtlS6UvWDIfVk0 CYbnBi5L8v6TMoawzk5fHCsSSstbA+ZRYSINCyo= X-Google-Smtp-Source: ABdhPJwQR8Bea/fK+Y3rGapLWxpsPWE7uthZxsWPM4yd63jR7xhwTDqZw56Xagl/P2YvlMRNzRcLaSpfcvLOhFU/qsk= X-Received: by 2002:a54:4812:: with SMTP id j18mr584378oij.55.1629164347920; Mon, 16 Aug 2021 18:39:07 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Mon, 16 Aug 2021 19:38:56 -0600 Message-ID: Subject: Re: sysctl is too slow To: Mateusz Guzik Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000003d160c05c9b762f8" X-Rspamd-Queue-Id: 4GpYbl0B0xz4cBx X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000003d160c05c9b762f8 Content-Type: text/plain; charset="UTF-8" Actually, I did get a flamegraph, and only 0.77% of samples were in ZFS. On Mon, Aug 16, 2021 at 7:19 PM Mateusz Guzik wrote: > On 8/16/21, Alan Somers wrote: > > Yes, I see what you're talking about now. There are a bunch of linked > > lists in sysctl_find_oid etc. Good point. > > -Alan > > > > You still want to get a flamegraph, chances are most of the problem is in > zfs. > > > On Mon, Aug 16, 2021 at 1:30 PM Mateusz Guzik wrote: > > > >> Last time I checked lookup of a sysctl was very bad with linear scans > all > >> over. > >> > >> Short of complete revamp of the entire thing I would start with > >> replacing the scans with a RB tree at each level. As is if you indeed > >> have 5000 datasets, you are doing increasingly longer walks. > >> > >> On 8/16/21, Alan Somers wrote: > >> > ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace > >> shows > >> > that almost all of its time is spent in sys_sysctl. ktrace shows that > >> both > >> > ztop and sysctl(8) call sys_sysctl a total of five times for each > >> > sysctl > >> > they care about: > >> > > >> > 1) To get the next oid > >> > 2) To get the sysctl's name > >> > 3) To get the oidfmt > >> > 4) To get the size of the value > >> > 5) To get the value itself. > >> > > >> > Each of these steps takes about equal time, and together all five take > >> > about 100us. If the time per call is mostly syscall overhead, then > the > >> > process could be sped up by 80% by combining all of these things into > a > >> > single syscall: return the next oid, its name, its format, the size of > >> its > >> > value, and optimistically the value itself, assuming the user passed a > >> > sufficiently large buffer. > >> > > >> > Am I missing something? Is there any other reason why sysctl is so > >> > slow? > >> > Or should I forget about it, and try to export ZFS's dataset stats > >> through > >> > devstat instead? > >> > -Alan > >> > > >> > >> > >> -- > >> Mateusz Guzik > >> > > > > > -- > Mateusz Guzik > --0000000000003d160c05c9b762f8-- From nobody Wed Aug 18 16:21:28 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2224E174F56C for ; Wed, 18 Aug 2021 16:21:48 +0000 (UTC) (envelope-from dan@langille.org) Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GqY7b1JhXz4gqQ for ; Wed, 18 Aug 2021 16:21:47 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 79075580B30 for ; Wed, 18 Aug 2021 12:21:36 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 18 Aug 2021 12:21:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= to:from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=fm2; bh=/Gt2Zzx8K6YTzc8UwoQlUaM2Jf ApC54CTk+tvP95UNo=; b=uDlRCrtemtsgfoX7s/XLP50xjv6LX12ZAkyQ8jqJeP TArOxp7kDM0hST01AFgTKak4zC4idb0+1xH22tiSauasJq7xiiCz5GFEUuzY5/wF jxDkdUrX+TU5ZqT4twzs9XkIsSe9LTTKS+e+Uxm3Okw089VhinunO3rKWsE/Pz3f ttZaTixwGo6KJfKSnen3aoaZRWgsze7S9Vmzlg4rEbOW+2W7PQHyzVuMz7JN0kZ4 CrIbkN8VpKy/9u5Vx18e75kYap4NglaZHmptCwqa+8by/CxOqcThvAXBhYxdWZGi eeoScTymV0cAuGgOZhTKOOUMGxt5wfhD2K5TQBcjYO9A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=/Gt2Zz x8K6YTzc8UwoQlUaM2JfApC54CTk+tvP95UNo=; b=MAgYjxV4M0SZIYXaEiiP14 uCp0qzl/uOkGTpHTD0QDncUag5Tni2VisQNntd9MWAd9GdWP43PWF2aPPlhKJql2 P/7J3MTpqJmNliGsPXnsEjxG/P0i0CV8KhMbuhxgN+gsQpfy3SbeVnzsKle0mpj6 gitQpBxvsLuVWSKD/a1W+iYXx1XkA9Q6OQawNP7ZzZQMJv0rtiIXkdUXHQveSqB4 ZxzgbGlplMiY75XrJlreF/GsrZxs9FUhm+GVeBiRI9Sk+LTZj4v7zYIV3ylFvWgh aeAzp5wA1EEvU3Vu2qxRpI4Z9oAqhoyaLH7D2TIDOMxKFqhIf+O7RRgDhkdfDHAg == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrleehgdelkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepvffhuffkffgfgggtgfesthekredttd efjeenucfhrhhomhepffgrnhcunfgrnhhgihhllhgvuceouggrnheslhgrnhhgihhllhgv rdhorhhgqeenucggtffrrghtthgvrhhnpeefveffhfduveevudfhjeetgfeukeeuvedvle ekhefgfeffueefueeltefhheevjeenucffohhmrghinheplhgrnhhgihhllhgvrdhorhhg necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepuggrnh eslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 18 Aug 2021 12:21:30 -0400 (EDT) To: freebsd-hackers@freebsd.org From: Dan Langille Subject: verifying a given jail is running Message-ID: <329b32d4-8f8b-d672-21b1-80820d5b318f@langille.org> Date: Wed, 18 Aug 2021 12:21:28 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:52.0) Gecko/20100101 PostboxApp/7.0.48 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4GqY7b1JhXz4gqQ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=uDlRCrte; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=MAgYjxV4; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.221 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-5.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.221:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.221]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.221:from] X-ThisMailContainsUnwantedMimeParts: N Hello, I want a script to verify a given jail is running. I think I have a rather simple solution but I'm presenting what I tried as I went along. I have a Nagios script for monitoring that a given jail is running: https://git.langille.org/dvl/nagios/src/branch/master/check_freebsd_jails I found that via a search, made some improvements, and I'm now using it in production. It seems to be low-overhead: $ time /usr/local/libexec/nagios-custom/check_freebsd_jails freshports Jail freshports (10.55.0.37) is running under JID 156 in /jails/freshports real    0m0.025s user    0m0.011s sys    0m0.017s I could easily invoke that from within my scripts. I then thought: why not jexec? $ time sudo jexec freshports date Wed Aug 18 15:23:25 UTC 2021 real    0m0.029s user    0m0.013s sys    0m0.003s Roughly the same time. Next to try was: $ time jls -j freshports    JID  IP Address      Hostname                      Path    156  10.55.0.37      freshports                    /jails/freshports real    0m0.003s user    0m0.000s sys    0m0.005s That's much faster, not that speed is really relevant here. I could use it like this in a shell script. jls -j freshports > /dev/null 2>&1 if [ $? = 0 ] ; then   echo jail is running else   echo jail is not running fi The main reason I need this: Verifying the jail is running before continuing with a script. Thank you. -- Dan Langille dan@langille.org From nobody Wed Aug 18 17:19:22 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D4A1B175C159 for ; Wed, 18 Aug 2021 17:19:25 +0000 (UTC) (envelope-from SRS0=eyDw=NJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GqZQ54wWtz4nQl for ; Wed, 18 Aug 2021 17:19:25 +0000 (UTC) (envelope-from SRS0=eyDw=NJ=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 143EF28417; Wed, 18 Aug 2021 19:19:24 +0200 (CEST) Received: from illbsd.quip.test (ip-94-113-69-69.net.upcbroadband.cz [94.113.69.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id AA27F28416; Wed, 18 Aug 2021 19:19:22 +0200 (CEST) Subject: Re: verifying a given jail is running To: Dan Langille , freebsd-hackers@freebsd.org References: <329b32d4-8f8b-d672-21b1-80820d5b318f@langille.org> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <71fb0edd-2b8a-e9ec-ed7e-ca8d6cdfff17@quip.cz> Date: Wed, 18 Aug 2021 19:19:22 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <329b32d4-8f8b-d672-21b1-80820d5b318f@langille.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4GqZQ54wWtz4nQl X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 18/08/2021 18:21, Dan Langille wrote: > I could use it like this in a shell script. > > jls -j freshports > /dev/null 2>&1 > if [ $? = 0 ] ; then >   echo jail is running > else >   echo jail is not running > fi > > The main reason I need this: Verifying the jail is running before > continuing with a script. You can save one line with this: if jls -j freshports > /dev/null 2>&1 ; then echo "jail is running" else echo "jail is not running" fi Or if you don't need the echoed output and just need to stop script execution: jls -j freshports > /dev/null 2>&1 || exit 5 Miroslav Lachman PS: Use jls -d if dying jail is also accepted as running jail (see man jls) From nobody Wed Aug 18 18:57:26 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A9AE51761CA1 for ; Wed, 18 Aug 2021 18:57:31 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GqcbG4lK9z4v3l for ; Wed, 18 Aug 2021 18:57:30 +0000 (UTC) (envelope-from rollingbits@gmail.com) Received: by mail-pj1-x1032.google.com with SMTP id n5so3231119pjt.4 for ; Wed, 18 Aug 2021 11:57:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=content-transfer-encoding:from:mime-version:date:subject:message-id :to; bh=jakOPsFiTg+5X1K8XtJrDiqBr049wtSVMnGH2Aifc84=; b=PiVQzsmW1EYh+7eVZlbyIqmbNqRdiUSCQEn1GNvFAZgTi5M8xP5dnwWm2dkf9cpL3G x9mqLqyTJ7ynsdloriMS0eKejWfdp2AgeAgaGeE86/tQbDZ9HtNI+mS6z+39B3kkneAD Zr/bcCRaoT2KRVKpWM+DUPfWbFNN7NdYQkeDW5g8YF+flMN+IFtI9WEfQKIVtq0V/p6i RkbchIFiU51Y22n4ljwJgfTdPPR9RLzspXZMMSp6B+NlbMw+rMvOr/dBOV20u5Y6Z88x 1L9UTauHsUy+nM3t3a2iZcRro0L6g7WAbhqQZJiVVt7MUgOfkgp8I6ZSHYh9MzNEnQtj rp/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version:date :subject:message-id:to; bh=jakOPsFiTg+5X1K8XtJrDiqBr049wtSVMnGH2Aifc84=; b=UfXha8g7vGr7MBVkKpWf2aQJAFXhMsLHPj42l/0QiJqRPe7Gv6OZcbf/ZekF2Bs1hS mF9nyzV7nPW05+q8Lk80HJJNIK1SYakEqtGL639MKAoUCIO5ynKHUo/0wIn62JJxg/E0 XL7P23SR1YKnPKPcuZG+tj/u9f00tm57IG+IcFozwtz+B+2lMrfzHfs6l42WLTLvZpF3 jbV7lxALEX3lycJATV4+ZZmV89ds2Zkp7Ao1qvJTcNVXLpz8pnujPBYGahl9FkrlO7bz XB9FnMEgvKxdUJ1YrxU4442MRfJtQl1yjoNnFeorBABH1goz+yUJc2gecwRQzj+vlmj3 vY1w== X-Gm-Message-State: AOAM533naCfGikM/ohaR6w6gnlofEsnEHKChIE8tbjAkMQrc1jPEpPTd ue99EwnhAY6w0NO9CGVwBPYRr7ZyRJvaeQ== X-Google-Smtp-Source: ABdhPJy7xToV9E+yD81YLUL0ZybCMgA8XgZgaecH9eqLvTAgacMMpfFfScpSPh4u7UNmToJxIKibDQ== X-Received: by 2002:a17:90b:4504:: with SMTP id iu4mr10950452pjb.209.1629313049369; Wed, 18 Aug 2021 11:57:29 -0700 (PDT) Received: from smtpclient.apple ([2804:389:201b:f1a1:b009:62ab:c7bf:10a2]) by smtp.gmail.com with ESMTPSA id q3sm584924pgl.23.2021.08.18.11.57.28 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Aug 2021 11:57:28 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-3D56742C-7081-419D-BD22-D84A54FA9625 Content-Transfer-Encoding: 7bit From: =?utf-8?Q?Lucas_Nali_de_Magalh=C3=A3es?= List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (1.0) Date: Wed, 18 Aug 2021 15:57:26 -0300 Subject: Thank you Message-Id: To: FreeBSD Hackers X-Mailer: iPhone Mail (18G82) X-Rspamd-Queue-Id: 4GqcbG4lK9z4v3l X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=PiVQzsmW; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rollingbits@gmail.com designates 2607:f8b0:4864:20::1032 as permitted sender) smtp.mailfrom=rollingbits@gmail.com X-Spamd-Result: default: False [-2.67 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; R_MIXED_CHARSET(0.83)[subject]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1032:from]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail-3D56742C-7081-419D-BD22-D84A54FA9625 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi. I really have no words to describe how I feel but I think it's the moment to= say thank you all for the hard work done on FreeBSD. After more than a year= in social distancing Sars-Cov-2 with remote-work boosting like nothing else= , the last few days of almost empty inbox made a strong impression in me.=20= I started using this OS almost 20 years ago while looking for a more light O= S and found an environment that lets me create and work on my projects due t= o it's permissive license. I remember to think "This time I'll learn and con= tribute with something." When other boxes started to crash, I also switched t= hem to FreeBSD. They remained in use as long as possible. Most of them are t= oo broken to use but I made a few nice experiments with one of them in this p= andemic. I plotted a few graphics in M$Excel to put the thing I see in numbers. They a= re in https://imgur.com/a/You5huO and aren't my best work. The last one came= from a "table" I found in the FreeBSD site and it's the best image I got. T= he others are from a compilation of the data of a third party archive of the= se messages, this morning, that I thought be easier to parse. --=20 rollingbits =E2=80=94 =F0=9F=93=A7 rollingbits@icloud.com =F0=9F=93=A7 rolli= ngbits@gmail.com =F0=9F=93=A7 rollingbits@yahoo.com =F0=9F=93=A7 rollingbits= @terra.com.br =F0=9F=93=A7 rollingbits@globo.com= From nobody Thu Aug 19 14:12:24 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9D4E8177B825 for ; Thu, 19 Aug 2021 14:14:06 +0000 (UTC) (envelope-from freebsd-hackers@phoe.frmug.org) Received: from frmug.org (enterprise.frmug.org [213.36.253.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gr6Fn4L9Nz4nbT for ; Thu, 19 Aug 2021 14:14:05 +0000 (UTC) (envelope-from freebsd-hackers@phoe.frmug.org) Received: by frmug.org (Postfix, from userid 66) id 3296E12B29F; Thu, 19 Aug 2021 16:14:04 +0200 (CEST) Received: by memo2.memo.frmug.org (Postfix, from userid 1001) id 6738317A40; Thu, 19 Aug 2021 16:12:24 +0200 (CEST) Date: Thu, 19 Aug 2021 16:12:24 +0200 From: Bertrand Petit To: freebsd-hackers@freebsd.org Subject: Is snp(4) in working order? Message-ID: <20210819141224.GA93701@memo2.memo.frmug.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 4Gr6Fn4L9Nz4nbT X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=frmug.org (policy=none); spf=neutral (mx1.freebsd.org: 213.36.253.97 is neither permitted nor denied by domain of freebsd-hackers@phoe.frmug.org) smtp.mailfrom=freebsd-hackers@phoe.frmug.org X-Spamd-Result: default: False [0.73 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.27)[-0.271]; DMARC_POLICY_SOFTFAIL(0.10)[frmug.org : No valid SPF, No valid DKIM,none]; R_SPF_NEUTRAL(0.00)[?all:c]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROMTLD(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:12322, ipnet:213.36.0.0/16, country:FR]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N I'm wondering if, on a 12.2 host, the snp(4) device is out of order or if I'm improperly using it. Here is the situation: a GPS receiver connected to the host through USB (umodem driver) feeding ntpd and keeping it happy. On this setup I perform the following calls as root in a dedicated process: int a=open("/dev/cuaU0", O_RDONLY|O_NONBLOCK); isatty(a); /* Yes it is */ int b=open("/dev/snp", O_RDONLY); ioctl(b, SNPSTTY, &a); int pending; ioctl(b, FIONREAD, &pending); /* Empty */ char buf; read(b, 1, &buf); I ommited error handling for brevity, all calls are successfull except read() which never return. I expected to be able to read the same bytes stream as ntpd does but I get nothing. Am I misusing the snp(4) interface or is it non-functioning? [As a side note: I could also had used gpsd but hooking ntpd directly to the serial device yields a far better jitter.] -- %!PS -- Bertrand Petit /D{def}def/E{exch}D/G{get}D/I{2 div}D/U{dup}D/L{roll}D/Y{setgray}D/N{newpath}D /O{N 0 0 moveto}D/P{pop}D/T{translate}D currentpagedevice/PageSize G U 0 G/w E D 1 G /h E D w I h I T 0 Y 1 setlinewidth 0 1 2 { P 120 rotate 2 4 w U mul h U mul add sqrt I 50 add {N 50 0 3 2 L 0 360 arc stroke}for}for/s{O true charpath pathbbox exch 4 -1 L E sub I 3 1 L sub I} D /l(bp)D 0.94 Y /Helvetica findfont 22 scalefont setfont l s P(x)s exch P T O l show showpage From nobody Thu Aug 19 17:14:49 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CAA4C12DE4AD for ; Thu, 19 Aug 2021 17:15:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GrBGb3VlDz3GDS for ; Thu, 19 Aug 2021 17:15:03 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 17JHEokr036365 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 19 Aug 2021 20:14:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 17JHEokr036365 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 17JHEndD036364; Thu, 19 Aug 2021 20:14:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 19 Aug 2021 20:14:49 +0300 From: Konstantin Belousov To: Bertrand Petit Cc: freebsd-hackers@freebsd.org Subject: Re: Is snp(4) in working order? Message-ID: References: <20210819141224.GA93701@memo2.memo.frmug.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210819141224.GA93701@memo2.memo.frmug.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GrBGb3VlDz3GDS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Thu, Aug 19, 2021 at 04:12:24PM +0200, Bertrand Petit wrote: > > I'm wondering if, on a 12.2 host, the snp(4) device is out of order or > if I'm improperly using it. > > Here is the situation: a GPS receiver connected to the host through > USB (umodem driver) feeding ntpd and keeping it happy. On this setup I perform > the following calls as root in a dedicated process: > > int a=open("/dev/cuaU0", O_RDONLY|O_NONBLOCK); > isatty(a); /* Yes it is */ > int b=open("/dev/snp", O_RDONLY); > ioctl(b, SNPSTTY, &a); > int pending; ioctl(b, FIONREAD, &pending); /* Empty */ > char buf; read(b, 1, &buf); > > I ommited error handling for brevity, all calls are successfull except read() > which never return. > > I expected to be able to read the same bytes stream as ntpd does but I > get nothing. Am I misusing the snp(4) interface or is it non-functioning? > > [As a side note: I could also had used gpsd but hooking ntpd directly > to the serial device yields a far better jitter.] Did you tried watch(8) first? I just used watch successfully over /dev/pts/N pseudoterminal. From nobody Thu Aug 19 18:33:44 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B2FE4175ACD6 for ; Thu, 19 Aug 2021 18:33:49 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GrD1T0W9Cz3Mj4 for ; Thu, 19 Aug 2021 18:33:49 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ed1-x533.google.com with SMTP id g21so10212137edw.4 for ; Thu, 19 Aug 2021 11:33:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=YulLD1KtuKkPjIcSaRkF/0NIWYeTkbTrQqpoQ49Ghhw=; b=HVLn7LMPR8qM25h6ZS3gXLHiDXIZ/7udJmXuWnn1wEcl05QS5aVYC8APsz4o8AciY0 SviTkubBETfYbwR7YOprwYTr3bcm5oksW+Rdu3dn4PK2ZtyNjWQsaawvVfFRGF6ylBQX +VRnQiybkE0g4kYclaQinjurSCncRxJwp0Y1HLoTnUBD4jDKacWbG/Lr8wJ0QoqChsh7 4bvB3caXV9cys4/I6Cb7frQcmv8+8cT6CMm00w9GqZzGWT/MY0WhM+iqVExvJ1paoci3 QxsxV42m9Q+/NMuGKhn3rNGm6FuF3Ny++uD6wHHOmLFNQ4RalYkrSBOtTGfe0XfVO2kp zQhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=YulLD1KtuKkPjIcSaRkF/0NIWYeTkbTrQqpoQ49Ghhw=; b=dCbASAt4/X3ocGyuonCm37PteLslt+daSBfwjMjbjzRP+RtUeOxR50GAQS4RxBxEZK 1tJFOk2KLTFsG+V/HKN/R9+fvcZN12mXWYDV9mjkTWmAWG5DnTtU+AeO9R9bpNpvpPum xtAVP8oX6VSOajdze7AwB4/ZyBaehoL/8cNPvmEUn1tzv+/KqYQulu9aX2UIgtp6Or+6 biMo/pdIe9a89n21HfOWwrC3cyejEGeeDWTBn7aajOxtALpAKr3KK3077uR7HNY7obD3 g0xm0NAdMx4aF7v8TSgjJEspktLu/1duH/8KNl/Jq5ougfMlFhXKLNa2SclyCZqHBCwW 23Jw== X-Gm-Message-State: AOAM532H0mos2kpIIHVUsoxq5y5dlohggFChOqk2eDjD7+9q4GWhkzij s2dPRYnXIoUQerLNXPD0crgbb1ofoME= X-Google-Smtp-Source: ABdhPJzvPG8a8azngDls/J2l1xPmTUAj7S8s4Az2AstYWLC1P5ed/NvlEzw2q8fbZtPOODwzILeL+w== X-Received: by 2002:a05:6402:546:: with SMTP id i6mr18223284edx.80.1629398028255; Thu, 19 Aug 2021 11:33:48 -0700 (PDT) Received: from ernst.home (p5b023517.dip0.t-ipconnect.de. [91.2.53.23]) by smtp.gmail.com with ESMTPSA id a4sm2251801edv.53.2021.08.19.11.33.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Aug 2021 11:33:47 -0700 (PDT) Date: Thu, 19 Aug 2021 20:33:44 +0200 From: Gary Jennejohn To: Bertrand Petit Cc: freebsd-hackers@freebsd.org Subject: Re: Is snp(4) in working order? Message-ID: <20210819183344.358e1f55@ernst.home> In-Reply-To: References: <20210819141224.GA93701@memo2.memo.frmug.org> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GrD1T0W9Cz3Mj4 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=HVLn7LMP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::533 as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[91.2.53.23:received]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[0.999]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::533:from]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Thu, 19 Aug 2021 20:14:49 +0300 Konstantin Belousov wrote: > On Thu, Aug 19, 2021 at 04:12:24PM +0200, Bertrand Petit wrote: > > > > I'm wondering if, on a 12.2 host, the snp(4) device is out of order or > > if I'm improperly using it. > > > > Here is the situation: a GPS receiver connected to the host through > > USB (umodem driver) feeding ntpd and keeping it happy. On this setup I perform > > the following calls as root in a dedicated process: > > > > int a=open("/dev/cuaU0", O_RDONLY|O_NONBLOCK); > > isatty(a); /* Yes it is */ > > int b=open("/dev/snp", O_RDONLY); > > ioctl(b, SNPSTTY, &a); > > int pending; ioctl(b, FIONREAD, &pending); /* Empty */ > > char buf; read(b, 1, &buf); > > > > I ommited error handling for brevity, all calls are successfull except read() > > which never return. > > > > I expected to be able to read the same bytes stream as ntpd does but I > > get nothing. Am I misusing the snp(4) interface or is it non-functioning? > > > > [As a side note: I could also had used gpsd but hooking ntpd directly > > to the serial device yields a far better jitter.] > Did you tried watch(8) first? > > I just used watch successfully over /dev/pts/N pseudoterminal. > I thought about using watch(8) too. If nothing else, reading the source of /usr/src/usr.sbin/watch/watch.c will give you a good idea about how to correctly use snp(4) in your C-code. Pay particuar attention to how FIONREAD is used in main(). -- Gary Jennejohn From nobody Fri Aug 20 08:54:10 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 602781775400 for ; Fri, 20 Aug 2021 08:59:08 +0000 (UTC) (envelope-from freebsd-hackers@phoe.frmug.org) Received: from frmug.org (enterprise.frmug.org [213.36.253.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GrbCv4hclz531N for ; Fri, 20 Aug 2021 08:59:07 +0000 (UTC) (envelope-from freebsd-hackers@phoe.frmug.org) Received: by frmug.org (Postfix, from userid 66) id 50E7812B2E3; Fri, 20 Aug 2021 10:59:06 +0200 (CEST) Received: by memo2.memo.frmug.org (Postfix, from userid 1001) id A48D417A40; Fri, 20 Aug 2021 10:54:10 +0200 (CEST) Date: Fri, 20 Aug 2021 10:54:10 +0200 From: Bertrand Petit To: Konstantin Belousov , Gary Jennejohn Cc: freebsd-hackers@freebsd.org Subject: Re: Is snp(4) in working order? Message-ID: <20210820085410.GA97799@memo2.memo.frmug.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <20210819183344.358e1f55@ernst.home> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: 4GrbCv4hclz531N X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=frmug.org (policy=none); spf=neutral (mx1.freebsd.org: 213.36.253.97 is neither permitted nor denied by domain of freebsd-hackers@phoe.frmug.org) smtp.mailfrom=freebsd-hackers@phoe.frmug.org X-Spamd-Result: default: False [-0.27 / 15.00]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[frmug.org : No valid SPF, No valid DKIM,none]; R_SPF_NEUTRAL(0.00)[?all:c]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_SPAM_MEDIUM(0.72)[0.721]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.990]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:12322, ipnet:213.36.0.0/16, country:FR]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Thu, Aug 19, 2021 at 08:14:49PM +0300, Konstantin Belousov wrote: > Did you tried watch(8) first? No, I didn't; yes, now I did and observed the same behavior, which is silence from the device. After reading sys/dev/snp/snp.c, I understand that the sighted nothingness is normal and should be expected because the snp(4) device can only snoop on egress tty streams. The manual page lacks on this point. Since I need access to ingress trafic, I suppose I need to go through the bpf/usbdump way. (I'm not looking forward to that.) On Thu, Aug 19, 2021 at 08:33:44PM +0200, Gary Jennejohn wrote: > C-code. Pay particuar attention to how FIONREAD is used in main(). I must mention that FIONREAD on snp(4) devices no longer has special return status codes. The snp(4) manual page and the watch(8) command are both way out of sync with the actual implementation of the device driver. Disappearance, closure and detachement of the snooped device are now signaled by an EIO return on read() and by an EIO or ENXIO return on write(). I will issue a PR for this manual page. -- %!PS -- Bertrand Petit /D{def}def/E{exch}D/G{get}D/I{2 div}D/U{dup}D/L{roll}D/Y{setgray}D/N{newpath}D /O{N 0 0 moveto}D/P{pop}D/T{translate}D currentpagedevice/PageSize G U 0 G/w E D 1 G /h E D w I h I T 0 Y 1 setlinewidth 0 1 2 { P 120 rotate 2 4 w U mul h U mul add sqrt I 50 add {N 50 0 3 2 L 0 360 arc stroke}for}for/s{O true charpath pathbbox exch 4 -1 L E sub I 3 1 L sub I} D /l(bp)D 0.94 Y /Helvetica findfont 22 scalefont setfont l s P(x)s exch P T O l show showpage From nobody Fri Aug 20 21:25:05 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6452E12746C6; Fri, 20 Aug 2021 21:25:07 +0000 (UTC) (envelope-from dmilith@gmail.com) Received: from mail-qv1-xf2b.google.com (mail-qv1-xf2b.google.com [IPv6:2607:f8b0:4864:20::f2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Grvmf3ySwz3GPl; Fri, 20 Aug 2021 21:25:06 +0000 (UTC) (envelope-from dmilith@gmail.com) Received: by mail-qv1-xf2b.google.com with SMTP id jz1so6251353qvb.13; Fri, 20 Aug 2021 14:25:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=ICtinW9A+Aedqf1o3pr2sZdosyLmBYyQEm5Tc2Yd7YQ=; b=LZd24zLaRHQvJrnYj7ITTQDuYkV8a5QyYtd7jov7gdjt771aJwJxlukkQcG5KP3qzM NB/YHVAD6cWEb88LtBdFnHpxLi80gKi46ka8UViP5Gh1R3wLZCCBjnMBHLNw9QWsD/H7 BMSWfLWc7dHpUeYsH3Po5F1X23ePYul5qIv9OMzFYfg2C9IoyzDfV+MPTU7+0Twjb3zC FI5aofe1aFkRFJPvhktVya7PAIo6mOs0X7rbrEfqCdoChtmLeM6SFrzEzoC2jF/VyNGC fKZn3foICPz6B5wBw1POP6gMNoJdkCETHq/99vC9HLqZHtWeQwwKTM+xVknVTu2IARxC RoWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=ICtinW9A+Aedqf1o3pr2sZdosyLmBYyQEm5Tc2Yd7YQ=; b=XRaeXtsxrW3B84DX94RYdlYggvAy8z7Eh9l7qi9ulHHxhGyh26/CmclWnXorH3asyl Knwu7twUwT9EpZJaqsvY2UyQcq6j2/nwpPZkLOP+85AIkW3sFwwPW2TjEthB74OjnW3j 6S5ekgq8gu+iKmHHW9mmHbbCUtSbJ62XKbIw8RvCifRfJDn1e/bzSvzwoH21xHgviMiv ddbCCh5punt4kFZVs9+Obov7FVw21mvEqi6dBhC5MqeXAWSI9Mvbt9jCnk5UJrv1///j Xs4IY+BBuRWK6tKAvves9CaUNp+UvHFE4CRztqdAxnxjMlo0Ve2i/KYwjH7nPoWodo/v 6ISg== X-Gm-Message-State: AOAM531H/Cyi37GzESnuZnMFmjxe6QYzOw77qJC6TEEzuyLapMH7EFF/ f3/j7cS8wvB2oUzePprVxosQOKHFVjhz/fzIact0Vu84jFo= X-Google-Smtp-Source: ABdhPJw15R3sOY7MoVXQQ78lHczqS4InQ/1wYNtAqzzY7Zq9cgCUzOg816K4LGPWKFr6GKN+mmU8hsKioGgZEi/bt+o= X-Received: by 2002:a0c:e885:: with SMTP id b5mr22500874qvo.48.1629494705935; Fri, 20 Aug 2021 14:25:05 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:ac8:5b81:0:0:0:0:0 with HTTP; Fri, 20 Aug 2021 14:25:05 -0700 (PDT) From: "dmilith ." Date: Fri, 20 Aug 2021 23:25:05 +0200 Message-ID: Subject: Problem with USB serial connection to UPS ETA-1600-sinus To: freebsd-hackers , freebsd-stable Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Grvmf3ySwz3GPl X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=LZd24zLa; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of dmilith@gmail.com designates 2607:f8b0:4864:20::f2b as permitted sender) smtp.mailfrom=dmilith@gmail.com X-Spamd-Result: default: False [-3.88 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2b:from]; NEURAL_HAM_SHORT(-0.88)[-0.885]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N Hello! I struggle with a peculiar device that "just works" under Linux - my ETA 1600-sinus UPS. I've tried hacking it directly and via "NUT" UPS software - but no luck. On Linux it's enough to set blazer_usb driver in upsc.conf and software connects normally to the device (using Megatec protocol). Under FreeBSD I experience crazy issues with a similar setup - after a whole day of hacking, I start to guess it's about libusb provided with FreeBSD 13.0 on arm64 (one I've been using). I explained some technical details here: https://github.com/networkupstools/nut/issues/1077 If anyone has a glimpse of an idea what I may also try to get it working=E2=80=A6 Please help kind regards --=20 Daniel Dettlaff Versatile Knowledge Systems verknowsys.com From nobody Sun Aug 22 15:29:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9973E1782905 for ; Sun, 22 Aug 2021 15:30:06 +0000 (UTC) (envelope-from alfadev@protonmail.com) Received: from mail-40141.protonmail.ch (mail-40141.protonmail.ch [185.70.40.141]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gszp54Q1Jz3w7J for ; Sun, 22 Aug 2021 15:30:05 +0000 (UTC) (envelope-from alfadev@protonmail.com) Date: Sun, 22 Aug 2021 15:29:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1629646197; bh=SeLqjxl6mNxMVsMJal7QKKSJphed8V1q7e3ZNVPqX9s=; h=Date:To:From:Reply-To:Subject:From; b=JrVLX8sOIJpmfD/lHB0acXysR2WsnMCCT+6hUOl1CsRmaojPaqUFuOIxhR0EugJLn HLTS+JxnPw6ih3dkFYFwnDyK+T872EE7FnSkcc8cZSBlVqWj/yvFK01O8dDztq4Zaz XNkFYDlaQHTBrMLa+qgAIbA/7NpuwCpk8648DYNE= To: "freebsd-hackers@FreeBSD.org" , "freebsd-net@FreeBSD.org" Reply-To: alfadev Subject: FreeBSD 12.2 traffic not occurs onVXLAN Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_DkuqNngXHOvWSV4nVW5YDK4ud7qr2l6BI7J3T6Y" X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4Gszp54Q1Jz3w7J X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protonmail.com header.s=protonmail header.b=JrVLX8sO; dmarc=pass (policy=quarantine) header.from=protonmail.com; spf=pass (mx1.freebsd.org: domain of alfadev@protonmail.com designates 185.70.40.141 as permitted sender) smtp.mailfrom=alfadev@protonmail.com X-Spamd-Result: default: False [1.10 / 15.00]; HAS_REPLYTO(0.00)[alfadev@protonmail.com]; FREEMAIL_FROM(0.00)[protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24]; MIME_BASE64_TEXT_BOGUS(1.00)[]; DKIM_TRACE(0.00)[protonmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; MIME_BASE64_TEXT(0.10)[]; NEURAL_HAM_SHORT(-1.00)[-0.995]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; HAS_PHPMAILER_SIG(0.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(1.00)[1.000]; TO_DN_EQ_ADDR_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.141:from] Reply-To: alfadev@protonmail.com From: alfadev via freebsd-hackers X-Original-From: alfadev X-ThisMailContainsUnwantedMimeParts: Y This is a multi-part message in MIME format. --b1_DkuqNngXHOvWSV4nVW5YDK4ud7qr2l6BI7J3T6Y Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGksIEkgc3VjY2Vzc2Z1bGx5IGNvbmZpZ3VyZWQgVlhMQU4gdHVubmVsIGJldHdlZW4gYW1kNjQg RnJlZUJTRCAxMS4yIHRvIHg2NCBMaW51eApCdXQgaW4gRnJlZUJTRCAxMi4yIHdpdGggYmVsb3cg c2FtZSBjb25maWd1cmF0aW9uIG5vdCB3b3Jrcy4KU28gV2hhdCBpcyB0SGUgcHJvYmxlbSB3aXRo IEZyZWVCU0QgMTIuMiBpcyBpdCBidWcgb3IgYW55IG90aGVyIHRoaW5nPwpBbnkgaGVscCB3b3Vs ZCBiZSBhb29yZWNpYXRlZC4uCgpNeSBmdWxseSB3b3JraW5nIHRlc3RlZCBjb25maWd1cmF0aW9u IGlzOgoKRnJlZUJTRCAxMS4yIHNpZGU6CnBoeXNpY2FsIGludGVyZmFjZTogaWdiMAppZmNvbmZp ZyB2eGxhbjQwOTUgY3JlYXRlIHZ4bGFuaWQgNDA5NSB2eGxhbmxvY2FsIDE5Mi4xNjguOTkuMSB2 eGxhbnJlbW90ZSAxOTIuMTY4Ljk5Ljk5IGluZXQgMTkyLjE2OC4xNTcuMS8yNAoKTGludXggc2lk ZToKcGh5c2ljYWwgaW50ZXJmYWNlczogZXRoMCxldGgxCgppcCBsaW5rIGFkZCBuYW1lIHZ4bGFu NDA5NSB0eXBlIHZ4bGFuIGlkIDQwOTUgcmVtb3RlIDE5Mi4xNjguOTkuMSBsb2NhbCAxOTIuMTY4 Ljk5Ljk5CmlwIGxpbmsgYWRkIG5hbWUgdmJyMCB0eXBlIGJyaWRnZQppcCBsaW5rIHNldCBldGgx IG1hc3RlciB2YnIwCmlwIGxpbmsgc2V0IHZ4bGFuNDA5NSBtYXN0ZXIgdmJyMAppcCBsaW5rIHNl dCB2YnIwIHVwCgp0aGVyZSBpcyBhIGNsaWVudCBjb25uZWN0ZWQgb24gZXRoMSBhbmQgaGF2ZSBJ UCA6IDE5Mi4xNjguMTU3LjEwMApodHRwIGh0dHBzICwgaWNtcCAuLiB0cmFmZmljIHBhc3NlcyB0 aHJvdWdoIGJldHdlZW4gY2xpZW50IGFuZCB0dW5uZWwKZXZleXRoaW5nIHdvcmtzIHdlbGwu --b1_DkuqNngXHOvWSV4nVW5YDK4ud7qr2l6BI7J3T6Y--