Date: Wed, 8 Nov 2023 22:13:18 +0900 From: Junho Choi <junho.choi@gmail.com> To: Philip Paeps <philip@freebsd.org> Cc: admin@kr.freebsd.org, freebsd-hubs@freebsd.org Subject: Re: ftp-master rsync client issue? Message-ID: <CAJ5e%2BHAwaUtYMP5vfUo1kryAHSYa=dxA6eY4jn9SDaJkBC4GtA@mail.gmail.com> In-Reply-To: <816C7FD5-A139-47DC-A45E-0D53FC3DB8A5@freebsd.org> References: <CAJ5e%2BHD5SaFYwEW7hs4E_LVwQokA-f3P8xxT-%2Beb4%2BnHgc4_OQ@mail.gmail.com> <9793F40D-BA47-4F12-9F03-70C4AB0D75C7@freebsd.org> <CAJ5e%2BHC3PnEy93EpQQ=XpbxEn02N5=5rG8Wwe-JF-D%2Bp09SY%2BA@mail.gmail.com> <327523F3-5122-455F-952D-65DC36A7C1B1@freebsd.org> <CAJ5e%2BHAm4DhZ83K91hhGSpEeRyiKwC4fJ_5XVmCJz=Ko5wHOig@mail.gmail.com> <816C7FD5-A139-47DC-A45E-0D53FC3DB8A5@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Philip, Thanks for your help. Now rsync is connecting but getting slow and almost stopped after a few minutes: $ /usr/local/bin/rsync -vaHz --delete --progress --stats ftp-master.freebsd.org::FreeBSD/ <ftp-pub>/FreeBSD/ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D This is the FreeBSD master archive for the ftp area /pub/FreeBSD It is available for primary mirrors in order to ensure rapid archive updates and distribution. rsync -vaHz --delete rsync://ftp-master.freebsd.org/FreeBSD/ ~ftp/pub/FreeB= SD/ Contact information etc: http://www.FreeBSD.org/doc/en/articles/hubs/ =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D ******** MINIMUM RSYNC VERSION REQUIRED =3D 3.00 (prefer 3.1.0+) ****= ***** =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D receiving incremental file list ./ TIMESTAMP 35 100% 34.18kB/s 0:00:00 (xfr#1, ir-chk=3D1006/1013) dir.sizes 3,110 100% 337.46kB/s 0:00:00 (xfr#2, ir-chk=3D1005/1013) development/ development/TIMESTAMP 35 100% 3.42kB/s 0:00:00 (xfr#3, ir-chk=3D1027/1041) development/tarballs/ development/tarballs/doc_main.tar.gz 851,895 0% 392.42kB/s 0:04:33 ^C^C Latency is high (expected) and no obvious loss. > ping -c 10 ftp-master.freebsd.org PING ftp-master.freebsd.org (96.47.72.39): 56 data bytes 64 bytes from 96.47.72.39: icmp_seq=3D0 ttl=3D50 time=3D206.468 ms 64 bytes from 96.47.72.39: icmp_seq=3D1 ttl=3D50 time=3D205.864 ms 64 bytes from 96.47.72.39: icmp_seq=3D2 ttl=3D50 time=3D205.773 ms 64 bytes from 96.47.72.39: icmp_seq=3D3 ttl=3D50 time=3D205.814 ms 64 bytes from 96.47.72.39: icmp_seq=3D4 ttl=3D50 time=3D205.780 ms 64 bytes from 96.47.72.39: icmp_seq=3D5 ttl=3D50 time=3D205.958 ms 64 bytes from 96.47.72.39: icmp_seq=3D6 ttl=3D50 time=3D205.775 ms 64 bytes from 96.47.72.39: icmp_seq=3D7 ttl=3D50 time=3D205.817 ms 64 bytes from 96.47.72.39: icmp_seq=3D8 ttl=3D50 time=3D206.643 ms 64 bytes from 96.47.72.39: icmp_seq=3D9 ttl=3D50 time=3D205.772 ms --- ftp-master.freebsd.org ping statistics --- 10 packets transmitted, 10 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 205.772/205.966/206.643/0.302 ms I used to sync with ftp-master all the time before this connection issue. I can sync with other mirrors (e.g. ftp.jp) so probably not a network issue here. Any help appreciated. Best, On Wed, Nov 8, 2023 at 9:49=E2=80=AFAM Philip Paeps <philip@freebsd.org> wr= ote: > > On 2023-11-08 05:44:22 (+0800), Junho Choi wrote: > > On Tue, Nov 7, 2023 at 4:05=E2=80=AFPM Philip Paeps <philip@freebsd.org= > > > wrote: > >> On 2023-11-07 05:40:20 (+0800), Junho Choi wrote: > >>> On Tue, Nov 7, 2023 at 1:58=E2=80=AFAM Philip Paeps wrote: > >>>> On 2023-11-06 19:34:59 (+0800), Junho Choi wrote: > >>>>> I am running ftp/rsync.kr.freebsd.org and recently (not sure when > >>>>> it > >>>>> starts, but at least a few weeks) starting to see it's always > >>>>> full: > >>>> > >>>> Could you let me know a src address I can grep for in the > >>>> ftp-master > >>>> logs? > >>> > >>> Source address should be 182.252.182.30 (ftp.kr.freebsd.org) > >> > >> I believe I've fixed your problem. You were hitting the wrong > >> section > >> of the rsyncd.conf with tighter limits. Please let me know if you > >> still > >> run into limits. > > > > Yes it started working again. Thanks! > > Hooray! > > > For a full mirror, what's the requirement? Last time I checked it's > > too big for me. (I have one server running, not easy to expand) > > Still too big for you in that case, I think. ;-) > > We can do single-machine installations, but the machine has to be very > beefy. pkg alone currently needs about 10TB on disk on average, and a > bit more during times we have to support three stable/X branches plus > main. The workload also wants 100+GB memory for ZFS to keep up. > > But no worries. As I wrote earlier: you're well connected to Japan, > where we have two full multi-machine installations (AS9597 and AS7530). > > Thanks again for keeping ftp.kr.freebsd.org up! > > Best wishes. > Philip > > -- > Philip Paeps > Senior Reality Engineer > Alternative Enterprises --=20 Junho Choi <junho dot choi at gmail.com> | https://saturnsoft.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ5e%2BHAwaUtYMP5vfUo1kryAHSYa=dxA6eY4jn9SDaJkBC4GtA>