From nobody Mon Sep 13 01:00:23 2021 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 26A1317BF718 for ; Mon, 13 Sep 2021 01:00:37 +0000 (UTC) (envelope-from jo@bruelltuete.com) Received: from email.jo-t.de (seppel.jo-t.de [45.132.244.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4H77Sg6S1hz4crp for ; Mon, 13 Sep 2021 01:00:35 +0000 (UTC) (envelope-from jo@bruelltuete.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bruelltuete.com; s=bruelltuete18a; t=1631494828; bh=gjqCJVbC2GDdAkyc1OCkn2jN8PtnxkP7di2LsQ2D8Uw=; h=Message-ID:Date:MIME-Version:Subject:To:References:From:From; b=buPZmVqMjQycsBG1jMNHnBqU07dY1RMvl2CsUR+1aryiyZAJ7oHp0MsUq44+vUf9r 3Itxi68NVIDb2bFjVAdtsvLBj3SYghBVAyNL1YdCzDOnC/7eWWpbEEbml1eFM8TfeD LtPy69MpzDYIgUfJS7eDH9VzBNMPgF6vv807yrQpW85wTMOs+9sq3Ee2pnfQVbnsTh 7N3xSpJtXcZbgUV/fGG6N9gycAy3BVTSdJ01/0ZhHb48uPG49J4P80svRRrEYbbanV wYaN4Oarb2Sx5kNhPvm9/QB4ykSY/n1O1GK66HwJ2fASRgzDsZ4ioHbKHIB/0cMYue L0SDlBAZ02bpw== Message-ID: <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> Date: Mon, 13 Sep 2021 02:00:23 +0100 List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 Subject: Re: RFC reviews for ggate and hastd Content-Language: en-GB To: jmg@funkthat.com References: <20210909223355.GE5928@funkthat.com> Cc: freebsd-geom@FreeBSD.org In-Reply-To: <20210909223355.GE5928@funkthat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4H77Sg6S1hz4crp X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bruelltuete.com header.s=bruelltuete18a header.b=buPZmVqM; dmarc=pass (policy=reject) header.from=bruelltuete.com; spf=pass (mx1.freebsd.org: domain of jo@bruelltuete.com designates 45.132.244.126 as permitted sender) smtp.mailfrom=jo@bruelltuete.com X-Spamd-Result: default: False [-3.96 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[bruelltuete.com:s=bruelltuete18a]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bruelltuete.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bruelltuete.com,reject]; NEURAL_HAM_SHORT(-0.96)[-0.964]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:197540, ipnet:45.132.244.0/22, country:DE]; MID_RHS_MATCH_FROM(0.00)[] Reply-To: jo@bruelltuete.com From: Johannes Totz via freebsd-geom X-Original-From: Johannes Totz X-ThisMailContainsUnwantedMimeParts: N On 09/09/2021 23:33, John-Mark Gurney wrote: > Johannes Totz via freebsd-geom wrote this message on Thu, Sep 02, 2021 at 21:55 +0100: >> (looks like gmane swallowed my 1st message, trying again) >> >> Hey folks, >> >> any ggate or hastd users here? I've got some code reviews for you. >> Please take a look if you get a chance: >> >> https://reviews.freebsd.org/D31727 >> Fix potential out-of-bounds read in the geom-gate kernel module. >> >> https://reviews.freebsd.org/D31722 >> Dynamically alloc buffers in ggatec, instead of assuming a fixed size on >> the stack. >> >> https://reviews.freebsd.org/D31709 >> Simple rc script to start ggated. > > I'll try to look at them. > > I've broken out the ggate code to: https://www.funkthat.com/gitea/jmg/ggate Nice, thanks! I noticed the http branch. One weekend toy project idea I wanted to get around to was to write a ggated impl that talks to Backblaze. > and have a number of fixes in that tree, one of them being having the > server (ggated) be multi-threaded, to improve IOPs. Another is dropping > the setting of tcp buffer sizes to allow the dynamic buffer sizing code > to work, allowing it to grow larger than 128kB for better perf on high > latency links.. > > In another branch, I have a version that runs the client over ssh using > sftp (requires a modified sftp-server for best performance, and security) > allowing for better security.. > > The ssh branch does have an rc.d script for that. I'll look at which one > should be used, and adapting the man page that I've written for that one > as well. > > Thanks for the help/work! > From nobody Tue Sep 14 21:21:37 2021 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4F94917CFACC for ; Tue, 14 Sep 2021 21:21:47 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H8GWG1Nt5z3hkm for ; Tue, 14 Sep 2021 21:21:45 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 18ELLbeH060515 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 14 Sep 2021 14:21:37 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 18ELLbcD060514; Tue, 14 Sep 2021 14:21:37 -0700 (PDT) (envelope-from jmg) Date: Tue, 14 Sep 2021 14:21:37 -0700 From: John-Mark Gurney To: Johannes Totz Cc: freebsd-geom@FreeBSD.org Subject: Re: RFC reviews for ggate and hastd Message-ID: <20210914212137.GI5928@funkthat.com> Mail-Followup-To: Johannes Totz , freebsd-geom@FreeBSD.org References: <20210909223355.GE5928@funkthat.com> <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Tue, 14 Sep 2021 14:21:37 -0700 (PDT) X-Rspamd-Queue-Id: 4H8GWG1Nt5z3hkm X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com] X-ThisMailContainsUnwantedMimeParts: N Johannes Totz wrote this message on Mon, Sep 13, 2021 at 02:00 +0100: > On 09/09/2021 23:33, John-Mark Gurney wrote: > > Johannes Totz via freebsd-geom wrote this message on Thu, Sep 02, 2021 at 21:55 +0100: > >> (looks like gmane swallowed my 1st message, trying again) > >> > >> Hey folks, > >> > >> any ggate or hastd users here? I've got some code reviews for you. > >> Please take a look if you get a chance: > >> > >> https://reviews.freebsd.org/D31727 > >> Fix potential out-of-bounds read in the geom-gate kernel module. > >> > >> https://reviews.freebsd.org/D31722 > >> Dynamically alloc buffers in ggatec, instead of assuming a fixed size on > >> the stack. > >> > >> https://reviews.freebsd.org/D31709 > >> Simple rc script to start ggated. > > > > I'll try to look at them. > > > > I've broken out the ggate code to: https://www.funkthat.com/gitea/jmg/ggate > > Nice, thanks! > > I noticed the http branch. One weekend toy project idea I wanted to get > around to was to write a ggated impl that talks to Backblaze. I looked at the Backblaze B2 API, and I don't see a way to do partial updates of a file. All the API that I see require you to upload the entire file, so I don't think that it'll work. I abanded http as a solution, because of issues w/ WebDAV and partial updates and the IETF not being very sane about it: https://blog.sphere.chronosempire.org.uk/2012/11/21/webdav-and-the-http-patch-nightmare Also, the fact that apache would randomly truncate a file didn't help either... > > and have a number of fixes in that tree, one of them being having the > > server (ggated) be multi-threaded, to improve IOPs. Another is dropping > > the setting of tcp buffer sizes to allow the dynamic buffer sizing code > > to work, allowing it to grow larger than 128kB for better perf on high > > latency links.. > > > > In another branch, I have a version that runs the client over ssh using > > sftp (requires a modified sftp-server for best performance, and security) > > allowing for better security.. > > > > The ssh branch does have an rc.d script for that. I'll look at which one > > should be used, and adapting the man page that I've written for that one > > as well. > > > > Thanks for the help/work! -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From nobody Sun Sep 19 16:27:40 2021 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 89B8217CEBC2 for ; Sun, 19 Sep 2021 16:27:51 +0000 (UTC) (envelope-from jo@bruelltuete.com) Received: from email.jo-t.de (seppel.jo-t.de [45.132.244.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HCClp34Ttz4vcj for ; Sun, 19 Sep 2021 16:27:50 +0000 (UTC) (envelope-from jo@bruelltuete.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bruelltuete.com; s=bruelltuete18a; t=1632068792; bh=AO4uyn+04wrtc/9vdBah77g/Q2WeCLV8qVw8UBdOgBU=; h=Message-ID:Date:MIME-Version:Subject:To:References:From:From; b=QbFoySuqyZI31NTNKn0v59cuuFzGVttcSe/IoIsY3+rWyd7v3qJwemS0d9OxQMmBS ZDCV3QrKjR+73uIqjYiFLk1JnXQFv766jS6w1BxosBCvvedTCdVxQ4Qhj1bJ9A7qNz 3qFSKw1ts6TZuJ/Geiu3iks12tgYfDxMt4NSDbAZr6mgTGSdUTCgEFTKlmqtjVyJ9V RB8QN2qdEIxA+XjnAYDZjjEl7Egi0e4gYZf9g57UVssil9K+T3kV6WVu/OodU51z6v aWybnUhn/2lG/NRBXXESa11m8ijSQ4vjtjsWuN/ZlvQuStgGlPjvc1uiA1M7jkiQRL 50hCZYGZjAkqw== Message-ID: <4f79999f-9a07-531b-c8db-eb78b273b71f@bruelltuete.com> Date: Sun, 19 Sep 2021 17:27:40 +0100 List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 Subject: Re: RFC reviews for ggate and hastd Content-Language: en-GB To: freebsd-geom@FreeBSD.org References: <20210909223355.GE5928@funkthat.com> <31ac6be7-e438-00d0-b63a-b27b09ed5d58@bruelltuete.com> <20210914212137.GI5928@funkthat.com> Cc: jmg@funkthat.com In-Reply-To: <20210914212137.GI5928@funkthat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4HCClp34Ttz4vcj X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bruelltuete.com header.s=bruelltuete18a header.b=QbFoySuq; dmarc=pass (policy=reject) header.from=bruelltuete.com; spf=pass (mx1.freebsd.org: domain of jo@bruelltuete.com designates 45.132.244.126 as permitted sender) smtp.mailfrom=jo@bruelltuete.com X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[bruelltuete.com:s=bruelltuete18a]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bruelltuete.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[bruelltuete.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.997]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:197540, ipnet:45.132.244.0/22, country:DE]; MID_RHS_MATCH_FROM(0.00)[] Reply-To: jo@bruelltuete.com From: Johannes Totz via freebsd-geom X-Original-From: Johannes Totz X-ThisMailContainsUnwantedMimeParts: N On 14/09/2021 22:21, John-Mark Gurney wrote: > Johannes Totz wrote this message on Mon, Sep 13, 2021 at 02:00 +0100: >> On 09/09/2021 23:33, John-Mark Gurney wrote: >>> Johannes Totz via freebsd-geom wrote this message on Thu, Sep 02, 2021 at 21:55 +0100: >>>> (looks like gmane swallowed my 1st message, trying again) >>>> >>>> Hey folks, >>>> >>>> any ggate or hastd users here? I've got some code reviews for you. >>>> Please take a look if you get a chance: >>>> >>>> https://reviews.freebsd.org/D31727 >>>> Fix potential out-of-bounds read in the geom-gate kernel module. >>>> >>>> https://reviews.freebsd.org/D31722 >>>> Dynamically alloc buffers in ggatec, instead of assuming a fixed size on >>>> the stack. >>>> >>>> https://reviews.freebsd.org/D31709 >>>> Simple rc script to start ggated. >>> >>> I'll try to look at them. >>> >>> I've broken out the ggate code to: https://www.funkthat.com/gitea/jmg/ggate >> >> Nice, thanks! >> >> I noticed the http branch. One weekend toy project idea I wanted to get >> around to was to write a ggated impl that talks to Backblaze. > > I looked at the Backblaze B2 API, and I don't see a way to do partial > updates of a file. All the API that I see require you to upload the entire > file, so I don't think that it'll work. > > I abanded http as a solution, because of issues w/ WebDAV and partial > updates and the IETF not being very sane about it: > https://blog.sphere.chronosempire.org.uk/2012/11/21/webdav-and-the-http-patch-nightmare Oh I was just gonna store each block as a separate file on the backend. No need to keep it all together as one huge image. Sure we'd end up with literally billions of small files. But I'd ignore that until it becomes an actual problem. > Also, the fact that apache would randomly truncate a file didn't help > either... > >>> and have a number of fixes in that tree, one of them being having the >>> server (ggated) be multi-threaded, to improve IOPs. Another is dropping >>> the setting of tcp buffer sizes to allow the dynamic buffer sizing code >>> to work, allowing it to grow larger than 128kB for better perf on high >>> latency links.. >>> >>> In another branch, I have a version that runs the client over ssh using >>> sftp (requires a modified sftp-server for best performance, and security) >>> allowing for better security.. >>> >>> The ssh branch does have an rc.d script for that. I'll look at which one >>> should be used, and adapting the man page that I've written for that one >>> as well. >>> >>> Thanks for the help/work! > From nobody Sun Sep 19 21:00:40 2021 X-Original-To: geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 17AC317D9167 for ; Sun, 19 Sep 2021 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HCKpc3vsPz3kw0 for ; Sun, 19 Sep 2021 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3332623EDE for ; Sun, 19 Sep 2021 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 18JL0evE012547 for ; Sun, 19 Sep 2021 21:00:40 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 18JL0e8A012546 for geom@FreeBSD.org; Sun, 19 Sep 2021 21:00:40 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202109192100.18JL0e8A012546@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: geom@FreeBSD.org Subject: Problem reports for geom@FreeBSD.org that need special attention Date: Sun, 19 Sep 2021 21:00:40 +0000 List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16320852400.3ba41.11480" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: Y --16320852400.3ba41.11480 Date: Sun, 19 Sep 2021 21:00:40 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 218679 | [geli] add a verify command Open | 237269 | panic in glabel (g_label_destroy) stop after resi Open | 238814 | geom: topology lock being dropped in dumpconf of Open | 242747 | geli: AMD Epyc+GELI not using Hardware AES 4 problems total for which you should take action. --16320852400.3ba41.11480--