From owner-freebsd-ports@freebsd.org Mon Aug 10 13:28:45 2020 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 14CD13A34FF for ; Mon, 10 Aug 2020 13:28:45 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4BQGy46CkLz4JMt for ; Mon, 10 Aug 2020 13:28:44 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: by mailman.nyi.freebsd.org (Postfix) id D38453A3A50; Mon, 10 Aug 2020 13:28:44 +0000 (UTC) Delivered-To: ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D34ED3A3A4F for ; Mon, 10 Aug 2020 13:28:44 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mail.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BQGy400pcz4JLK; Mon, 10 Aug 2020 13:28:43 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from e.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (3096 bits) server-digest SHA256) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id 0ABAC152E88; Mon, 10 Aug 2020 15:28:36 +0200 (CEST) Received: (from lars@localhost) by e.0x20.net (8.15.2/8.15.2/Submit) id 07ADSZMa055642; Mon, 10 Aug 2020 15:28:35 +0200 (CEST) (envelope-from lars) Date: Mon, 10 Aug 2020 15:28:35 +0200 From: Lars Engels To: Steve Wills Cc: "ports@FreeBSD.org" Subject: Re: [HEADS UP] Planned deprecation of portsnap Message-ID: <20200810132835.GN24022@e.0x20.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Editor: VIM - Vi IMproved 8.0 User-Agent: Mutt/1.14.2 (2020-05-25) X-Rspamd-Queue-Id: 4BQGy400pcz4JLK X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of lars@e.0x20.net has no SPF policy when checking 46.251.251.56) smtp.mailfrom=lars@e.0x20.net X-Spamd-Result: default: False [0.97 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.16)[-0.163]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[0x20.net]; AUTH_NA(1.00)[]; NEURAL_HAM_SHORT(-0.10)[-0.105]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.04)[0.039]; RCVD_COUNT_ONE(0.00)[1]; FORGED_SENDER(0.30)[lars.engels@0x20.net,lars@e.0x20.net]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:31400, ipnet:46.251.251.0/24, country:DE]; FROM_NEQ_ENVFROM(0.00)[lars.engels@0x20.net,lars@e.0x20.net]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2020 13:28:45 -0000 On Tue, Aug 04, 2020 at 02:43:20PM -0400, Steve Wills wrote: > > We are planning to deprecate use of portsnap in ports. > > The reasons are as follows (in no particular order): > > * Portsnap doesn't support quarterly branches, even years after > quarterly branches were created and changed to the default for non-HEAD > packages. > > * Portsnap doesn't seem to save disk space compared to svn or git, if > you count the metadata (stored in /var/db/portsnap by default) and you > do an apples-to-apples comparison of svn or git without history and > ignoring possible ZFS compression. That is, you use "svn export" or git > "clone --depth 1", you see this disk usage: > > 342M svnexport > 426M git > 477M portsnap > > * Portsnap also doesn't work offline which git does. With git, you can > also easily add the history by running "git pull --unshallow" > > * This migration away from portsnap fits well with the planned migration > to git. > > * Also based on the patches we've seen in Bugzilla for some time, usage > of portsnap causes folks to too easily accidentally submit patches to > Bugzilla which don't apply easily. > > * Since portsnap doesn't support quarterly branches, it often causes > users to build on the wrong branch or end up with mismatched packages. > That is, they install packages from quarterly via pkg, then want to > customize so run portsnap and build from head, which can cause problems, > as we often see. Even when this doesn't happen, it adds to > troubleshooting to verify that it didn't. > > We are aware people have gotten used to portsnap, but believe: > > * People should be able to easily use svnlite in base or git from pkgs. > (Very few people seem to actually use WITHOUT_SVNLITE). > > * There is also the possibility of falling back to fetching a tar or zip > from https://cgit-beta.freebsd.org/ports/ although this does make > updating harder. > > How it will be done, in order: > > * Update poudriere to use svn by default. This is already done: > > https://github.com/freebsd/poudriere/pull/764 > > https://github.com/freebsd/poudriere/commit/bd68f30654e2a8e965fbdc09aad238c8bf5cdc10 > > * Update docs not to mention portsnap. This is already in progress: > > https://reviews.freebsd.org/D25800 > https://reviews.freebsd.org/D25801 > https://reviews.freebsd.org/D25803 > https://reviews.freebsd.org/D25805 > https://reviews.freebsd.org/D25808 > https://svnweb.freebsd.org/changeset/base/363798 > > Many thanks to the folks who have worked and are working on this! > > * Make WITHOUT_PORTSNAP default in base. Currently not certain when this > will happen. May not happen before 13.0, but hopefully it will. > > * Eventually, portsnap servers will see low enough usage they can be > disabled. > > We welcome any constructive feedback. All input would be heard, and if > the plans need to be amended, we will come back to you with the amended > plan in a couple of weeks. This process will take some time and > hopefully won't be too disruptive to anyone's usual workflow. > I'm probably fine with this and I think that all of the (now) supported methods have pros and cons. To leverage the UX flaws of git and svn(lite) compared to portsnap having a wrapper script around the two tools would be very appreciated. Something like # portsnap-ng --mode git --branch 2020Q2 --destination /ports/2020Q2 fetch extract The package devel/git does not seem to be installed, do you want to install it? (Y/n) _ With sane defaults, so you can just run portsnap fetch extract like you're used to and this downloads the latest ports tree to /usr/ports using base svnlite(1). While we're here: Can we please have a separate user that is used to checkout the tree? Lars P.S.: Please DO NOT name the wrapper portsnap-ng. :-)