From owner-freebsd-current@freebsd.org Sun Sep 6 19:09:22 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 916A33DAE80 for ; Sun, 6 Sep 2020 19:09:22 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from mail03.asahi-net.or.jp (mail03.asahi-net.or.jp [202.224.55.15]) by mx1.freebsd.org (Postfix) with ESMTP id 4Bl1Dd02C0z4MJD for ; Sun, 6 Sep 2020 19:09:20 +0000 (UTC) (envelope-from ota@j.email.ne.jp) Received: from vmware.advok.com (cpe-184-152-96-96.nj.res.rr.com [184.152.96.96]) (Authenticated sender: NR2Y-OOT) by mail03.asahi-net.or.jp (Postfix) with ESMTPSA id DCF3A2D8B7; Mon, 7 Sep 2020 04:09:16 +0900 (JST) Date: Sun, 6 Sep 2020 15:05:49 -0400 From: Yoshihiro Ota To: Michael Gmelin Cc: Niclas Zeising , grarpamp , freebsd-current@freebsd.org Subject: Re: Is pkg site forbidden by brower? Message-Id: <20200906150549.d3dc2c85698a1ff604a6bcaf@j.email.ne.jp> In-Reply-To: References: X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; i386-portbld-freebsd12.0) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Bl1Dd02C0z4MJD X-Spamd-Bar: / X-Spamd-Result: default: False [0.03 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:202.224.55.0/24]; NEURAL_HAM_SHORT(-0.73)[-0.731]; RCVD_NO_TLS_LAST(0.10)[]; RCVD_IN_DNSWL_LOW(-0.10)[202.224.55.15:from]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:4685, ipnet:202.224.32.0/19, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.95)[-0.946]; RWL_MAILSPIKE_NEUTRAL(0.00)[202.224.55.15:from]; RECEIVED_SPAMHAUS_PBL(0.00)[184.152.96.96:received]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.99)[-0.990]; TAGGED_RCPT(0.00)[freebsd]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[email.ne.jp]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FREEMAIL_CC(0.00)[daemonic.se,gmail.com,freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Sep 2020 19:09:22 -0000 On Sun, 6 Sep 2020 14:47:00 +0200 Michael Gmelin wrote: > > > > On 6. Sep 2020, at 12:00, Niclas Zeising wrote: > > > > 〓On 2020-09-06 09:00, grarpamp wrote: > >>> On 9/6/20, Kevin Oberman wrote: > >>> On Sat, Sep 5, 2020 at 8:04 PM Yoshihiro Ota wrote: > >>>> Is "403 Forbidden" an intended response for a brower access to > >>>> http://pkg.freebsd.org/FreeBSD:12:i386/ nowdays? > >>>> > >>>> I used to see available packages with a brower and decided which one to > >>>> use. > >> Some more people have noted this change > >> as breaking tool scripts, etc. > >> And useful meta files are unfortunately now invisible: > >> packagesite.txz, meta.txz, pkg.txz, pkg.txz.sig > >> If someone want to block the '/.../All/' dir full of pkgs, > >> maybe, but do not block any other part of the hier. > > > > The reason that folder listing was disabled on the package download sites is that it used too > > much resources. For every hit on those URLs, the web server had to dynamically generate the > > folder listing, and send it. This caused DDoS-like scenarios, where these were hit repeatedly, > > which caused problems for legitimate traffic. Since the relevant information is available in > > the txz files above, and also on freshports, and since pkg have no need for directory listing, > > it was disabled. > > > > Is this part of why pkg repos are performing so much better recently? I‘m quite happy about > that :) > > If there’s a use case for having access to this information, we could simply provide it through a > static index.html that’s recreated every time the directory changes. > > Cheers, > Michael Michael, Do you own pkg sites? I like the idea of creating static index.html file as there are modules with version numbers and also modules with multiple version numbers. Brower helps to find which versions are available, for example. Regards, Hiro