From nobody Tue Nov 21 04:17:44 2023 X-Original-To: freebsd-hubs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SZB271RmNz51jn1 for ; Tue, 21 Nov 2023 04:18:23 +0000 (UTC) (envelope-from junho.choi@gmail.com) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SZB261XK2z3YmX; Tue, 21 Nov 2023 04:18:22 +0000 (UTC) (envelope-from junho.choi@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=HJ7i05Ak; spf=pass (mx1.freebsd.org: domain of junho.choi@gmail.com designates 2607:f8b0:4864:20::829 as permitted sender) smtp.mailfrom=junho.choi@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-41cc0e9d92aso31297121cf.3; Mon, 20 Nov 2023 20:18:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700540301; x=1701145101; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=iglL7zCk9KoUIVpp5wc0AOoXGmJX4J99KZARMn5Jjrw=; b=HJ7i05AkS4oGWiNVDBFa7+nZanqGiShaHZOluX7/y9KB5wSHrixf3WWJ01DQtL835n ZJv55/+Dz9/WcGBSyOlL7sPxgU4byk5pVbnMHWpdY915k/dDMzDW88TpTG6PnjUxUTYf IGlguaCVjsEzo6BOF8O9jjYeucwPHo2ZFZ9IcmuBMFPpisuJCwqaedhPMMdmvQFRU4SB q7CajtFLF93KiGkPJYwP3MgstDLpTL53PB7qAbawhby6/wr8zjdt1oKNz9c1q7vBqNjw XaKspiwZIgVNqaIu0Ziq95Eo53N+AcCadHYX7vwd40Vq25FouwYBRSuEoTjxvpcSU52z SVvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700540301; x=1701145101; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iglL7zCk9KoUIVpp5wc0AOoXGmJX4J99KZARMn5Jjrw=; b=NO3N1MhzWAhNXHI94qvRyMJvpmM3XHYdECZxd0XMeML10FLNBRgQd6Pl3p8ZwvljM3 btJqy/YUj0wb9aFMUeoU43OyOoWI9Eaml89sT5bryOnFNg/kYH87czluWtT2m8Xy/u35 SCIZXshNROO85gcrOhbIpA+l5ExOuP2SFA/3FCg0XHjR3mua4kubvU/slBPNyl6sz60M N2+pBit7Rn9681aPmw5qBuBS/oVCcoQ/IZuvdt43bvCc1WMy81wguDugDZjLYBYxvMmW Uty5R6+R6X6+7r82IUDoW6R4eFc869c5FoakWUz9zJM6JyCjz+5yga1maUV6JmXAfGA3 rNhg== X-Gm-Message-State: AOJu0YxsSHRd/iQdUWN1iYVgaH2fiiLeFWhUX9fSOm1QsGSp42yYtgE1 muPuuhmsRRT0YQJ9yNOQq94N6L5p/+H7EvdK0ieKcTm9U/z1tA== X-Google-Smtp-Source: AGHT+IHf15xjrrsWVQA/S6JWRRi6NBBypa7VTL4cSybFpUO0u6ggWRXEmP2c/4XamB8BpS4Oa1ELFbXq5C8dhk6lGLg= X-Received: by 2002:a05:622a:429a:b0:423:6d42:dbc8 with SMTP id cr26-20020a05622a429a00b004236d42dbc8mr1721288qtb.50.1700540301089; Mon, 20 Nov 2023 20:18:21 -0800 (PST) List-Id: FreeBSD Distributions Hubs: mail sup ftp List-Archive: https://lists.freebsd.org/archives/freebsd-hubs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hubs@freebsd.org MIME-Version: 1.0 References: <9793F40D-BA47-4F12-9F03-70C4AB0D75C7@freebsd.org> <327523F3-5122-455F-952D-65DC36A7C1B1@freebsd.org> <816C7FD5-A139-47DC-A45E-0D53FC3DB8A5@freebsd.org> In-Reply-To: From: Junho Choi Date: Tue, 21 Nov 2023 13:17:44 +0900 Message-ID: Subject: Re: ftp-master rsync client issue? To: Philip Paeps Cc: admin@kr.freebsd.org, freebsd-hubs@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.00 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_DN_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::829:from]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hubs@freebsd.org] X-Rspamd-Queue-Id: 4SZB261XK2z3YmX X-Spamd-Bar: -- This problem (transfer timeout) was solved after I switched back to 'freebsd' tcp functions. net.inet.tcp.functions_default=3Dfreebsd Previously this server was set to net.inet.tcp.functions_default=3Drack Looks like 'rack' or 'bbr' new tcp functions are not stable yet? Best, On Wed, Nov 8, 2023 at 10:13=E2=80=AFPM Junho Choi w= rote: > > Philip, > > Thanks for your help. Now rsync is connecting but getting slow and > almost stopped after a few minutes: > > $ /usr/local/bin/rsync -vaHz --delete --progress --stats > ftp-master.freebsd.org::FreeBSD/ /FreeBSD/ > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > This is the FreeBSD master archive for the ftp area /pub/FreeBSD > It is available for primary mirrors in order to ensure rapid archive > updates and distribution. > > rsync -vaHz --delete rsync://ftp-master.freebsd.org/FreeBSD/ ~ftp/pub/Fre= eBSD/ > > Contact information etc: http://www.FreeBSD.org/doc/en/articles/hubs/ > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > ******** MINIMUM RSYNC VERSION REQUIRED =3D 3.00 (prefer 3.1.0+) **= ******* > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > receiving incremental file list > ./ > TIMESTAMP > 35 100% 34.18kB/s 0:00:00 (xfr#1, ir-chk=3D1006/1013) > dir.sizes > 3,110 100% 337.46kB/s 0:00:00 (xfr#2, ir-chk=3D1005/1013) > development/ > development/TIMESTAMP > 35 100% 3.42kB/s 0:00:00 (xfr#3, ir-chk=3D1027/1041) > development/tarballs/ > development/tarballs/doc_main.tar.gz > 851,895 0% 392.42kB/s 0:04:33 ^C^C > > Latency is high (expected) and no obvious loss. > > > ping -c 10 ftp-master.freebsd.org > PING ftp-master.freebsd.org (96.47.72.39): 56 data bytes > 64 bytes from 96.47.72.39: icmp_seq=3D0 ttl=3D50 time=3D206.468 ms > 64 bytes from 96.47.72.39: icmp_seq=3D1 ttl=3D50 time=3D205.864 ms > 64 bytes from 96.47.72.39: icmp_seq=3D2 ttl=3D50 time=3D205.773 ms > 64 bytes from 96.47.72.39: icmp_seq=3D3 ttl=3D50 time=3D205.814 ms > 64 bytes from 96.47.72.39: icmp_seq=3D4 ttl=3D50 time=3D205.780 ms > 64 bytes from 96.47.72.39: icmp_seq=3D5 ttl=3D50 time=3D205.958 ms > 64 bytes from 96.47.72.39: icmp_seq=3D6 ttl=3D50 time=3D205.775 ms > 64 bytes from 96.47.72.39: icmp_seq=3D7 ttl=3D50 time=3D205.817 ms > 64 bytes from 96.47.72.39: icmp_seq=3D8 ttl=3D50 time=3D206.643 ms > 64 bytes from 96.47.72.39: icmp_seq=3D9 ttl=3D50 time=3D205.772 ms > > --- ftp-master.freebsd.org ping statistics --- > 10 packets transmitted, 10 packets received, 0.0% packet loss > round-trip min/avg/max/stddev =3D 205.772/205.966/206.643/0.302 ms > > I used to sync with ftp-master all the time before this connection issue. > I can sync with other mirrors (e.g. ftp.jp) so probably not a network > issue here. > > Any help appreciated. > > Best, > > On Wed, Nov 8, 2023 at 9:49=E2=80=AFAM Philip Paeps = wrote: > > > > On 2023-11-08 05:44:22 (+0800), Junho Choi wrote: > > > On Tue, Nov 7, 2023 at 4:05=E2=80=AFPM Philip Paeps > > > wrote: > > >> On 2023-11-07 05:40:20 (+0800), Junho Choi wrote: > > >>> On Tue, Nov 7, 2023 at 1:58=E2=80=AFAM Philip Paeps wrote: > > >>>> On 2023-11-06 19:34:59 (+0800), Junho Choi wrote: > > >>>>> I am running ftp/rsync.kr.freebsd.org and recently (not sure when > > >>>>> it > > >>>>> starts, but at least a few weeks) starting to see it's always > > >>>>> full: > > >>>> > > >>>> Could you let me know a src address I can grep for in the > > >>>> ftp-master > > >>>> logs? > > >>> > > >>> Source address should be 182.252.182.30 (ftp.kr.freebsd.org) > > >> > > >> I believe I've fixed your problem. You were hitting the wrong > > >> section > > >> of the rsyncd.conf with tighter limits. Please let me know if you > > >> still > > >> run into limits. > > > > > > Yes it started working again. Thanks! > > > > Hooray! > > > > > For a full mirror, what's the requirement? Last time I checked it's > > > too big for me. (I have one server running, not easy to expand) > > > > Still too big for you in that case, I think. ;-) > > > > We can do single-machine installations, but the machine has to be very > > beefy. pkg alone currently needs about 10TB on disk on average, and a > > bit more during times we have to support three stable/X branches plus > > main. The workload also wants 100+GB memory for ZFS to keep up. > > > > But no worries. As I wrote earlier: you're well connected to Japan, > > where we have two full multi-machine installations (AS9597 and AS7530). > > > > Thanks again for keeping ftp.kr.freebsd.org up! > > > > Best wishes. > > Philip > > > > -- > > Philip Paeps > > Senior Reality Engineer > > Alternative Enterprises > > > > -- > Junho Choi | https://saturnsoft.net --=20 Junho Choi | https://saturnsoft.net