From nobody Wed Nov 8 13:13:18 2023 X-Original-To: freebsd-hubs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQQX50JdKz50Ttk for ; Wed, 8 Nov 2023 13:13:57 +0000 (UTC) (envelope-from junho.choi@gmail.com) Received: from mail-qt1-x82a.google.com (mail-qt1-x82a.google.com [IPv6:2607:f8b0:4864:20::82a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQQX45D74z3fWH; Wed, 8 Nov 2023 13:13:56 +0000 (UTC) (envelope-from junho.choi@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x82a.google.com with SMTP id d75a77b69052e-41cd1fe4645so40799691cf.0; Wed, 08 Nov 2023 05:13:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699449235; x=1700054035; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=p2fqOdpegli7RS3QTo1pBC3asBKRmuG7c1799q4ccR8=; b=WuHoUFG9gmEX1WIBS3MDAOElhGDqt175omHih22DsP55ECpRoVwOA1249++BLV0wg0 1vLEFD3OBf2dlw+JDkjEQnDhcc/uoqdgBAGjQ0pvx9pFFYrsoFMe04CfkLQGevzES2IB anTVp+nC7tOY/SjrY5NUjh8A8QoMbp4/Nkaj2gKObdfYFPiOKCLJuwUQD++yZNJ0InEN /a64Zz+7PIPOMpTr+u89ueG2VxeYuxzeT1i3l+FPZVtUClAOuBSOqmGjNbeSuTfph7gr A140/S6pcnIWryjHK6fUmAn7KEyyyXazAZoI3fvlNHBUX0GKW62lyAGoRl+YszbO3Moz OOjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699449235; x=1700054035; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p2fqOdpegli7RS3QTo1pBC3asBKRmuG7c1799q4ccR8=; b=oajQLohUENA2od3OBdKRJExckn9ES3cV3w2wZ08OZGNaO4lhanvEnhyBhJLsgv8/AM /8jzPO07Mn4bozf/mo3ErFLW0AJyePUd8EH8LeA/vZw8NcKgNXsWoDJfk6D4j4PYcefI dqXYolf35LkKmRqCqH1R2TAE9gmKX3ZDk4Tx2nNalgEVYMYn2TgNcOaY13D54qBZfYoh Q9vm2G/H2swlb01BhofeFAoEk9Nkke5BcbroMaSh98SbVH05h094v76jJV5lJiyn3jCY 5WVwoLpQiuxN7hXh0jIApgNdhXcOvHo5ymTb2BRgx+t5xZ0DUN4p43TciQg1bQ9H8INL uoHw== X-Gm-Message-State: AOJu0YyCNkdq2h1LmXCZ0MxuCAHD0NIDibpb4BwdR9N2hlBUtap9OqN3 p6DoSKILr5MTecxrv4jWAncQOvhsy9kLq107Uy/P36fu9ik8iA== X-Google-Smtp-Source: AGHT+IGRKVMtK62d5ySMIyDb6oBIWa49E7YWomR8JGPhmcsdboAmRS+ovpID8chMiop9uf9EWV9RxV+MQVHmMVRUkf8= X-Received: by 2002:a05:622a:244:b0:41e:24aa:81a0 with SMTP id c4-20020a05622a024400b0041e24aa81a0mr2037233qtx.62.1699449234962; Wed, 08 Nov 2023 05:13:54 -0800 (PST) List-Id: FreeBSD Distributions Hubs: mail sup ftp List-Archive: https://lists.freebsd.org/archives/freebsd-hubs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hubs@freebsd.org MIME-Version: 1.0 References: <9793F40D-BA47-4F12-9F03-70C4AB0D75C7@freebsd.org> <327523F3-5122-455F-952D-65DC36A7C1B1@freebsd.org> <816C7FD5-A139-47DC-A45E-0D53FC3DB8A5@freebsd.org> In-Reply-To: <816C7FD5-A139-47DC-A45E-0D53FC3DB8A5@freebsd.org> From: Junho Choi Date: Wed, 8 Nov 2023 22:13:18 +0900 Message-ID: Subject: Re: ftp-master rsync client issue? To: Philip Paeps Cc: admin@kr.freebsd.org, freebsd-hubs@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Queue-Id: 4SQQX45D74z3fWH Philip, Thanks for your help. Now rsync is connecting but getting slow and almost stopped after a few minutes: $ /usr/local/bin/rsync -vaHz --delete --progress --stats ftp-master.freebsd.org::FreeBSD/ /FreeBSD/ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D This is the FreeBSD master archive for the ftp area /pub/FreeBSD It is available for primary mirrors in order to ensure rapid archive updates and distribution. rsync -vaHz --delete rsync://ftp-master.freebsd.org/FreeBSD/ ~ftp/pub/FreeB= SD/ Contact information etc: http://www.FreeBSD.org/doc/en/articles/hubs/ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D ******** MINIMUM RSYNC VERSION REQUIRED =3D 3.00 (prefer 3.1.0+) ****= ***** =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D receiving incremental file list ./ TIMESTAMP 35 100% 34.18kB/s 0:00:00 (xfr#1, ir-chk=3D1006/1013) dir.sizes 3,110 100% 337.46kB/s 0:00:00 (xfr#2, ir-chk=3D1005/1013) development/ development/TIMESTAMP 35 100% 3.42kB/s 0:00:00 (xfr#3, ir-chk=3D1027/1041) development/tarballs/ development/tarballs/doc_main.tar.gz 851,895 0% 392.42kB/s 0:04:33 ^C^C Latency is high (expected) and no obvious loss. > ping -c 10 ftp-master.freebsd.org PING ftp-master.freebsd.org (96.47.72.39): 56 data bytes 64 bytes from 96.47.72.39: icmp_seq=3D0 ttl=3D50 time=3D206.468 ms 64 bytes from 96.47.72.39: icmp_seq=3D1 ttl=3D50 time=3D205.864 ms 64 bytes from 96.47.72.39: icmp_seq=3D2 ttl=3D50 time=3D205.773 ms 64 bytes from 96.47.72.39: icmp_seq=3D3 ttl=3D50 time=3D205.814 ms 64 bytes from 96.47.72.39: icmp_seq=3D4 ttl=3D50 time=3D205.780 ms 64 bytes from 96.47.72.39: icmp_seq=3D5 ttl=3D50 time=3D205.958 ms 64 bytes from 96.47.72.39: icmp_seq=3D6 ttl=3D50 time=3D205.775 ms 64 bytes from 96.47.72.39: icmp_seq=3D7 ttl=3D50 time=3D205.817 ms 64 bytes from 96.47.72.39: icmp_seq=3D8 ttl=3D50 time=3D206.643 ms 64 bytes from 96.47.72.39: icmp_seq=3D9 ttl=3D50 time=3D205.772 ms --- ftp-master.freebsd.org ping statistics --- 10 packets transmitted, 10 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 205.772/205.966/206.643/0.302 ms I used to sync with ftp-master all the time before this connection issue. I can sync with other mirrors (e.g. ftp.jp) so probably not a network issue here. Any help appreciated. Best, On Wed, Nov 8, 2023 at 9:49=E2=80=AFAM Philip Paeps wr= ote: > > On 2023-11-08 05:44:22 (+0800), Junho Choi wrote: > > On Tue, Nov 7, 2023 at 4:05=E2=80=AFPM Philip Paeps > > wrote: > >> On 2023-11-07 05:40:20 (+0800), Junho Choi wrote: > >>> On Tue, Nov 7, 2023 at 1:58=E2=80=AFAM Philip Paeps wrote: > >>>> On 2023-11-06 19:34:59 (+0800), Junho Choi wrote: > >>>>> I am running ftp/rsync.kr.freebsd.org and recently (not sure when > >>>>> it > >>>>> starts, but at least a few weeks) starting to see it's always > >>>>> full: > >>>> > >>>> Could you let me know a src address I can grep for in the > >>>> ftp-master > >>>> logs? > >>> > >>> Source address should be 182.252.182.30 (ftp.kr.freebsd.org) > >> > >> I believe I've fixed your problem. You were hitting the wrong > >> section > >> of the rsyncd.conf with tighter limits. Please let me know if you > >> still > >> run into limits. > > > > Yes it started working again. Thanks! > > Hooray! > > > For a full mirror, what's the requirement? Last time I checked it's > > too big for me. (I have one server running, not easy to expand) > > Still too big for you in that case, I think. ;-) > > We can do single-machine installations, but the machine has to be very > beefy. pkg alone currently needs about 10TB on disk on average, and a > bit more during times we have to support three stable/X branches plus > main. The workload also wants 100+GB memory for ZFS to keep up. > > But no worries. As I wrote earlier: you're well connected to Japan, > where we have two full multi-machine installations (AS9597 and AS7530). > > Thanks again for keeping ftp.kr.freebsd.org up! > > Best wishes. > Philip > > -- > Philip Paeps > Senior Reality Engineer > Alternative Enterprises --=20 Junho Choi | https://saturnsoft.net