From nobody Mon Nov 22 08:18:33 2021 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D3222189586A; Mon, 22 Nov 2021 08:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HyKsj4pVwz4c47; Mon, 22 Nov 2021 08:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 80BB51F613; Mon, 22 Nov 2021 08:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1AM8IXaV098649; Mon, 22 Nov 2021 08:18:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1AM8IXEi098648; Mon, 22 Nov 2021 08:18:33 GMT (envelope-from git) Date: Mon, 22 Nov 2021 08:18:33 GMT Message-Id: <202111220818.1AM8IXEi098648@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Baptiste Daroussin Subject: git: bcb098e384dc - main - fetchindex: make compression format configurable List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-all@freebsd.org X-BeenThere: dev-commits-ports-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bapt X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: bcb098e384dc2d5694d098975161328a2699ed07 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by bapt: URL: https://cgit.FreeBSD.org/ports/commit/?id=bcb098e384dc2d5694d098975161328a2699ed07 commit bcb098e384dc2d5694d098975161328a2699ed07 Author: Baptiste Daroussin AuthorDate: 2021-11-22 08:12:07 +0000 Commit: Baptiste Daroussin CommitDate: 2021-11-22 08:18:15 +0000 fetchindex: make compression format configurable Add INDEX_COMPRESSION_FORMAT as a variable a user can overwrite via /etc/make.conf to allow the compression format used for INDEX to be changed. Available compression formats are: bz2, xz and zst for now. While here change the default compression format to be .xz (the smallest), for environement where performance of unxz matters it can switched to zst. bz2 format is kept for compatibility with existing tooling. In order to simplify the code, bsdcat(1) is now used for uncompressing instead detecting the compression format manually and switch the decompression tool manually Note the bzip2 compression format is now considered as deprecated --- CHANGES | 9 +++++++++ Makefile | 18 +++++++++++++----- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/CHANGES b/CHANGES index 7f86abd2e4a5..8fb6b7d6b0a3 100644 --- a/CHANGES +++ b/CHANGES @@ -10,6 +10,15 @@ in the release notes and/or placed into UPDATING. All ports committers are allowed to commit to this file. +20211122: +AUTHOR: bapt@FreeBSD.org + + The default compression for INDEX file as fetch via make fetchindex has + switched from .bz2 to .xz (.zst is also available). + + The .bz2 format is still available but is considered deprecated and may + be removed in the futur. + 20211010: AUTHOR: bapt@FreeBSD.org diff --git a/Makefile b/Makefile index b2e17255f8ae..bb5edc8f6f86 100644 --- a/Makefile +++ b/Makefile @@ -66,17 +66,25 @@ PORTSTOP= yes index: ${INDEXDIR}/${INDEXFILE} -fetchindex: ${INDEXDIR}/${INDEXFILE}.bz2 - @if bunzip2 < ${INDEXDIR}/${INDEXFILE}.bz2 > ${INDEXDIR}/${INDEXFILE}.tmp ; then \ +INDEX_COMPRESSION_FORMAT?= xz + +.if ${INDEX_COMPRESSION_FORMAT} != xz && \ + ${INDEX_COMPRESSION_FORMAT} != bz2 && ${INDEX_COMPRESSION_FORMAT} != zst +.error "Invalid compression format: ${INDEX_COMPRESSION_FORMAT}, expecting xz, bz2 or zst" +.endif + +fetchindex: ${INDEXDIR}/${INDEXFILE}.${INDEX_COMPRESSION_FORMAT} + @if bsdcat < ${INDEXDIR}/${INDEXFILE}.${INDEX_COMPRESSION_FORMAT} > ${INDEXDIR}/${INDEXFILE}.tmp ; then \ chmod a+r ${INDEXDIR}/${INDEXFILE}.tmp; \ ${MV} ${INDEXDIR}/${INDEXFILE}.tmp ${INDEXDIR}/${INDEXFILE}; \ - ${RM} ${INDEXDIR}/${INDEXFILE}.bz2 \ + ${RM} ${INDEXDIR}/${INDEXFILE}.${INDEX_COMPRESSION_FORMAT} \ else ; \ ${RM} ${INDEXDIR}/${INDEXFILE}.tmp ; \ fi -${INDEXDIR}/${INDEXFILE}.bz2: .PHONY - ${FETCHINDEX} ${INDEXDIR}/${INDEXFILE}.bz2 ${MASTER_SITE_INDEX}${INDEXFILE}.bz2 +${INDEXDIR}/${INDEXFILE}.${INDEX_COMPRESSION_FORMAT}: .PHONY + ${FETCHINDEX} ${INDEXDIR}/${INDEXFILE}.${INDEX_COMPRESSION_FORMAT} \ + ${MASTER_SITE_INDEX}${INDEXFILE}.${INDEX_COMPRESSION_FORMAT} MASTER_SITE_INDEX?= https://www.FreeBSD.org/ports/ SETENV?= /usr/bin/env