From owner-freebsd-current@freebsd.org Sun Sep 6 12:47:15 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 535E03D0498 for ; Sun, 6 Sep 2020 12:47:15 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (Client CN "mail.evolve.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bkrlk2JMxz3ZjP for ; Sun, 6 Sep 2020 12:47:13 +0000 (UTC) (envelope-from freebsd@grem.de) Received: by mail.evolve.de (OpenSMTPD) with ESMTP id 0960bb03; Sun, 6 Sep 2020 12:47:05 +0000 (UTC) Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id 70de7362 (TLSv1.2:ECDHE-RSA-CHACHA20-POLY1305:256:NO); Sun, 6 Sep 2020 12:47:02 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Subject: Re: Is pkg site forbidden by brower? From: Michael Gmelin In-Reply-To: Date: Sun, 6 Sep 2020 14:47:00 +0200 Cc: grarpamp , freebsd-current@freebsd.org Message-Id: References: To: Niclas Zeising X-Mailer: iPhone Mail (17H35) X-Rspamd-Queue-Id: 4Bkrlk2JMxz3ZjP X-Spamd-Bar: - X-Spamd-Result: default: False [-1.74 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[grem.de:s=20180501]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:213.239.217.29/32]; TAGGED_RCPT(0.00)[freebsd]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[grem.de]; NEURAL_HAM_LONG(-0.99)[-0.990]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[grem.de:+]; NEURAL_HAM_SHORT(-0.70)[-0.703]; NEURAL_HAM_MEDIUM(-1.04)[-1.043]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Sep 2020 12:47:15 -0000 > On 6. Sep 2020, at 12:00, Niclas Zeising wro= te: >=20 > =EF=BB=BFOn 2020-09-06 09:00, grarpamp wrote: >>> On 9/6/20, Kevin Oberman wrote: >>> On Sat, Sep 5, 2020 at 8:04 PM Yoshihiro Ota wrote: >>>> Is "403 Forbidden" an intended response for a brower access to >>>> http://pkg.freebsd.org/FreeBSD:12:i386/ nowdays? >>>>=20 >>>> I used to see available packages with a brower and decided which one to= >>>> use. >> Some more people have noted this change >> as breaking tool scripts, etc. >> And useful meta files are unfortunately now invisible: >> packagesite.txz, meta.txz, pkg.txz, pkg.txz.sig >> If someone want to block the '/.../All/' dir full of pkgs, >> maybe, but do not block any other part of the hier. >=20 > The reason that folder listing was disabled on the package download sites i= s that it used too much resources. For every hit on those URLs, the web ser= ver had to dynamically generate the folder listing, and send it. This cause= d DDoS-like scenarios, where these were hit repeatedly, which caused problem= s for legitimate traffic. Since the relevant information is available in th= e txz files above, and also on freshports, and since pkg have no need for di= rectory listing, it was disabled. >=20 Is this part of why pkg repos are performing so much better recently? I=E2=80= =98m quite happy about that :) If there=E2=80=99s a use case for having access to this information, we coul= d simply provide it through a static index.html that=E2=80=99s recreated eve= ry time the directory changes. Cheers, Michael