Date: Thu, 8 May 2025 16:25:58 GMT From: Xin LI <delphij@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: 9679eedea94c - releng/14.3 - MFV: xz 5.8.1. Message-ID: <202505081625.548GPwvL011825@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch releng/14.3 has been updated by delphij: URL: https://cgit.FreeBSD.org/src/commit/?id=9679eedea94c9d60c372c67350242acfe18e2b22 commit 9679eedea94c9d60c372c67350242acfe18e2b22 Author: Xin LI <delphij@FreeBSD.org> AuthorDate: 2025-05-04 07:06:22 +0000 Commit: Xin LI <delphij@FreeBSD.org> CommitDate: 2025-05-08 16:24:51 +0000 MFV: xz 5.8.1. PR: bin/286252 Approved by: re (cperciva) (cherry picked from commit 128836d304d93f2d00eb14069c27089ab46c38d4) (cherry picked from commit 5cf27a49a2de91ae1f369912a7bf3859fbc79355) --- contrib/xz/AUTHORS | 2 +- contrib/xz/COPYING | 25 +- contrib/xz/ChangeLog | 6031 +++++++++++++++----- contrib/xz/README | 107 +- contrib/xz/THANKS | 37 + contrib/xz/TODO | 25 +- contrib/xz/src/common/my_landlock.h | 141 + contrib/xz/src/common/sysdefs.h | 58 +- contrib/xz/src/common/tuklib_common.h | 11 +- contrib/xz/src/common/tuklib_gettext.h | 11 + contrib/xz/src/common/tuklib_mbstr.h | 25 +- contrib/xz/src/common/tuklib_mbstr_nonprint.c | 162 + contrib/xz/src/common/tuklib_mbstr_nonprint.h | 71 + contrib/xz/src/common/tuklib_mbstr_width.c | 34 +- contrib/xz/src/common/tuklib_mbstr_wrap.c | 294 + contrib/xz/src/common/tuklib_mbstr_wrap.h | 204 + contrib/xz/src/common/tuklib_physmem.c | 11 +- contrib/xz/src/liblzma/api/lzma/bcj.h | 97 + contrib/xz/src/liblzma/api/lzma/container.h | 2 +- contrib/xz/src/liblzma/api/lzma/lzma12.h | 2 +- contrib/xz/src/liblzma/api/lzma/version.h | 4 +- contrib/xz/src/liblzma/check/check.h | 18 - contrib/xz/src/liblzma/check/crc32_arm64.h | 55 +- contrib/xz/src/liblzma/check/crc32_fast.c | 46 +- contrib/xz/src/liblzma/check/crc32_loongarch.h | 65 + contrib/xz/src/liblzma/check/crc32_small.c | 3 + contrib/xz/src/liblzma/check/crc32_table.c | 42 - contrib/xz/src/liblzma/check/crc32_x86.S | 14 +- contrib/xz/src/liblzma/check/crc64_fast.c | 39 +- contrib/xz/src/liblzma/check/crc64_table.c | 37 - contrib/xz/src/liblzma/check/crc64_x86.S | 14 +- .../xz/src/liblzma/check/crc_clmul_consts_gen.c | 160 + contrib/xz/src/liblzma/check/crc_common.h | 111 +- contrib/xz/src/liblzma/check/crc_x86_clmul.h | 519 +- contrib/xz/src/liblzma/common/alone_decoder.c | 3 +- contrib/xz/src/liblzma/common/auto_decoder.c | 5 +- contrib/xz/src/liblzma/common/block_decoder.c | 6 +- contrib/xz/src/liblzma/common/block_encoder.c | 6 +- contrib/xz/src/liblzma/common/common.c | 8 +- contrib/xz/src/liblzma/common/file_info.c | 22 +- contrib/xz/src/liblzma/common/index_decoder.c | 9 +- contrib/xz/src/liblzma/common/index_encoder.c | 6 +- contrib/xz/src/liblzma/common/index_hash.c | 7 +- contrib/xz/src/liblzma/common/lzip_decoder.c | 14 +- contrib/xz/src/liblzma/common/memcmplen.h | 12 +- contrib/xz/src/liblzma/common/stream_decoder.c | 16 +- contrib/xz/src/liblzma/common/stream_decoder_mt.c | 140 +- contrib/xz/src/liblzma/common/stream_encoder_mt.c | 10 +- contrib/xz/src/liblzma/common/string_conversion.c | 99 +- contrib/xz/src/liblzma/liblzma_generic.map | 10 + contrib/xz/src/liblzma/liblzma_linux.map | 10 + contrib/xz/src/liblzma/lz/lz_decoder.c | 21 +- contrib/xz/src/liblzma/lz/lz_decoder.h | 103 +- contrib/xz/src/liblzma/lz/lz_encoder.c | 2 +- contrib/xz/src/liblzma/lz/lz_encoder_hash.h | 30 +- contrib/xz/src/liblzma/lzma/lzma2_encoder.c | 9 +- contrib/xz/src/liblzma/lzma/lzma_decoder.c | 2 +- contrib/xz/src/liblzma/simple/arm.c | 4 +- contrib/xz/src/liblzma/simple/arm64.c | 22 +- contrib/xz/src/liblzma/simple/armthumb.c | 7 +- contrib/xz/src/liblzma/simple/ia64.c | 4 +- contrib/xz/src/liblzma/simple/powerpc.c | 4 +- contrib/xz/src/liblzma/simple/riscv.c | 18 + contrib/xz/src/liblzma/simple/sparc.c | 5 +- contrib/xz/src/liblzma/simple/x86.c | 24 + contrib/xz/src/lzmainfo/lzmainfo.c | 52 +- contrib/xz/src/xz/args.c | 41 +- contrib/xz/src/xz/args.h | 2 +- contrib/xz/src/xz/coder.c | 28 +- contrib/xz/src/xz/file_io.c | 260 +- contrib/xz/src/xz/file_io.h | 10 +- contrib/xz/src/xz/list.c | 39 +- contrib/xz/src/xz/main.c | 10 +- contrib/xz/src/xz/message.c | 506 +- contrib/xz/src/xz/options.c | 15 +- contrib/xz/src/xz/private.h | 1 + contrib/xz/src/xz/sandbox.c | 78 +- contrib/xz/src/xz/suffix.c | 12 +- contrib/xz/src/xz/util.c | 10 +- contrib/xz/src/xz/xz.1 | 94 +- contrib/xz/src/xzdec/xzdec.c | 75 +- lib/liblzma/Makefile | 9 +- lib/liblzma/Symbol.map | 9 + lib/liblzma/Versions.def | 5 +- lib/liblzma/config.h | 18 +- usr.bin/lzmainfo/Makefile | 5 +- usr.bin/xz/Makefile | 4 +- usr.bin/xzdec/Makefile | 3 +- 88 files changed, 7849 insertions(+), 2552 deletions(-) diff --git a/contrib/xz/AUTHORS b/contrib/xz/AUTHORS index 5eff238ae413..f805a204ecb7 100644 --- a/contrib/xz/AUTHORS +++ b/contrib/xz/AUTHORS @@ -24,7 +24,7 @@ Authors of XZ Utils by Michał Górny. Architecture-specific CRC optimizations were contributed by - Ilya Kurdyukov, Hans Jansen, and Chenxi Mao. + Ilya Kurdyukov, Chenxi Mao, and Xi Ruoyao. Other authors: - Jonathan Nieder diff --git a/contrib/xz/COPYING b/contrib/xz/COPYING index aed21531497c..ef3371389d7d 100644 --- a/contrib/xz/COPYING +++ b/contrib/xz/COPYING @@ -40,6 +40,12 @@ XZ Utils Licensing free software licenses. These aren't built or installed as part of XZ Utils. + The following command may be helpful in finding per-file license + information. It works on xz.git and on a clean file tree extracted + from a release tarball. + + sh build-aux/license-check.sh -v + For the files under the BSD Zero Clause License (0BSD), if a copyright notice is needed, the following is sufficient: @@ -59,25 +65,6 @@ XZ Utils Licensing - COPYING.GPLv2: GNU General Public License version 2 - COPYING.GPLv3: GNU General Public License version 3 - A note about old XZ Utils releases: - - XZ Utils releases 5.4.6 and older and 5.5.1alpha have a - significant amount of code put into the public domain and - that obviously remains so. The switch from public domain to - 0BSD for newer releases was made in Febrary 2024 because - public domain has (real or perceived) legal ambiguities in - some jurisdictions. - - There is very little *practical* difference between public - domain and 0BSD. The main difference likely is that one - shouldn't claim that 0BSD-licensed code is in the public - domain; 0BSD-licensed code is copyrighted but available under - an extremely permissive license. Neither 0BSD nor public domain - require retaining or reproducing author, copyright holder, or - license notices when distributing the software. (Compare to, - for example, BSD 2-Clause "Simplified" License which does have - such requirements.) - If you have questions, don't hesitate to ask for more information. The contact information is in the README file. diff --git a/contrib/xz/ChangeLog b/contrib/xz/ChangeLog index 2d36d7bb1043..577dce5e12a2 100644 --- a/contrib/xz/ChangeLog +++ b/contrib/xz/ChangeLog @@ -1,834 +1,4468 @@ -commit 9331ce4009ddc839f5191d234cc41b2d4797376d +commit a522a226545730551f7e7c2685fab27cf567746c Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-10-01 12:21:22 +0300 +Date: 2025-04-03 14:34:43 +0300 - Bump version and soname for 5.6.3 + Bump version and soname for 5.8.1 src/liblzma/Makefile.am | 2 +- src/liblzma/api/lzma/version.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -commit f52857ffde768058db0e0e13f68a2660ca9f1330 +commit 1c462c2ad86ff85766928638431029cd0b0dc995 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-10-01 12:17:39 +0300 +Date: 2025-04-03 14:34:43 +0300 - Add NEWS for 5.6.3 + Add NEWS for 5.8.1 - NEWS | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - 1 file changed, 125 insertions(+) + NEWS | 30 ++++++++++++++++++++++++++++++ + 1 file changed, 30 insertions(+) -commit b8f52990b5d47a50902bf33cd2305ce985457bac +commit 513cabcf7f5ce1c3ed0619e791393fc53d1dbbd0 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-10-01 12:10:23 +0300 +Date: 2025-04-03 14:34:43 +0300 - Update THANKS + Tests: Call lzma_code() in smaller chunks in fuzz_common.h - (cherry picked from commit 1ebbe915d4e0d877154261b5f8103719a6722975) + This makes it easy to crash fuzz_decode_stream_mt when tested + against the code from 5.8.0. + + Obviously this might make it harder to reach some other code path now. + The previous code has been in use since 2018 when fuzzing was added + in 106d1a663d4b ("Tests: Add a fuzz test program and a config file + for OSS-Fuzz."). - THANKS | 2 ++ - 1 file changed, 2 insertions(+) + tests/ossfuzz/fuzz_common.h | 31 ++++++++++++++++++++++++------- + 1 file changed, 24 insertions(+), 7 deletions(-) -commit 51f6f455873911894f155e6997bc23a9be8f42ba +commit 48440e24a25911ae59e8518b67a1e0f6f1c293bf Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-10-01 12:10:23 +0300 +Date: 2025-04-03 14:34:43 +0300 - Tests/Windows: Add the application manifest to the test programs - - This ensures that the test programs get executed the same way as - the binaries that are installed. + Tests: Add a fuzzing target for the multithreaded .xz decoder - (cherry picked from commit 74702ee00ecfd080d8ab11118cd25dbe6c437ec0) + It doesn't seem possible to trigger the CVE-2025-31115 bug with this + fuzzing target at the moment. It's because the code in fuzz_common.h + passes the whole input buffer to lzma_code() at once. - CMakeLists.txt | 14 ++++++++++---- - tests/Makefile.am | 10 ++++++++++ - tests/tests.cmake | 33 ++++++++++++++++++++++++++++++++- - tests/tests_w32res.rc | 18 ++++++++++++++++++ - 4 files changed, 70 insertions(+), 5 deletions(-) + tests/ossfuzz/fuzz_decode_stream_mt.c | 47 +++++++++++++++++++++++++++++++++++ + 1 file changed, 47 insertions(+) -commit bf518b9ba446327a062ddfe67e7e0a5baed2394f +commit 0c80045ab82c406858d9d5bcea9f48ebc3d0a81d Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-10-01 12:10:23 +0300 +Date: 2025-04-03 14:34:42 +0300 - Windows: Embed an application manifest in the EXE files - - IMPORTANT: This includes a security fix to command line tool - argument handling. - - Some toolchains embed an application manifest by default to declare - UAC-compliance. Some also declare compatibility with Vista/8/8.1/10/11 - to let the app access features newer than those of Vista. - - We want all the above but also two more things: - - - Declare that the app is long path aware to support paths longer - than 259 characters (this may also require a registry change). - - - Force the code page to UTF-8. This allows the command line tools - to access files whose names contain characters that don't exist - in the current legacy code page (except unpaired surrogates). - The UTF-8 code page also fixes security issues in command line - argument handling which can be exploited with malicious filenames. - See the new file w32_application.manifest.comments.txt. + liblzma: mt dec: Fix lack of parallelization in single-shot decoding - Thanks to Orange Tsai and splitline from DEVCORE Research Team - for discovering this issue. - - Thanks to Vijay Sarvepalli for reporting the issue to me. + Single-shot decoding means calling lzma_code() by giving it the whole + input at once and enough output buffer space to store the uncompressed + data, and combining this with LZMA_FINISH and no timeout + (lzma_mt.timeout = 0). This way the file is decoded with a single + lzma_code() call if possible. - Thanks to Kelvin Lee for testing with MSVC and helping with - the required build system fixes. + The bug prevented the decoder from starting more than one worker thread + in single-shot mode. The issue was noticed when reviewing the code; + there are no bug reports. Thus maybe few have tried this mode. - (cherry picked from commit 46ee0061629fb075d61d83839e14dd193337af59) + Fixes: 64b6d496dc81 ("liblzma: Threaded decoder: Always wait for output if LZMA_FINISH is used.") - CMakeLists.txt | 18 +++ - src/Makefile.am | 4 +- - src/common/common_w32res.rc | 5 + - src/common/w32_application.manifest | 28 ++++ - src/common/w32_application.manifest.comments.txt | 178 +++++++++++++++++++++++ - 5 files changed, 232 insertions(+), 1 deletion(-) + src/liblzma/common/stream_decoder_mt.c | 11 +++++++++-- + 1 file changed, 9 insertions(+), 2 deletions(-) -commit 5718ce932e6ad4262d5fffc9e2a7a838f963d7e5 +commit 8188048854e8d11071b8a50d093c74f4c030acc9 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-29 14:46:52 +0300 +Date: 2025-04-03 14:34:42 +0300 - Windows: Set DLL name accurately in StringFileInfo on Cygwin and MSYS2 + liblzma: mt dec: Don't modify thr->in_size in the worker thread - Now the information in the "Details" tab in the file properties - dialog matches the naming convention of Cygwin and MSYS2. This - is only a cosmetic change. + Don't set thr->in_size = 0 when returning the thread to the stack of + available threads. Not only is it useless, but the main thread may + read the value in SEQ_BLOCK_THR_RUN. With valid inputs, it made + no difference if the main thread saw the original value or 0. With + invalid inputs (when worker thread stops early), thr->in_size was + no longer modified after the previous commit with the security fix + ("Don't free the input buffer too early"). - (cherry picked from commit dad153091552b52a41b95ec4981c6951f1cae487) + So while the bug appears harmless now, it's important to fix it because + the variable was being modified without proper locking. It's trivial + to fix because there is no need to change the value. Only main thread + needs to set the value in (in SEQ_BLOCK_THR_INIT) when starting a new + Block before the worker thread is activated. + + Fixes: 4cce3e27f529 ("liblzma: Add threaded .xz decompressor.") + Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc> + Thanks-to: Sam James <sam@gentoo.org> - src/liblzma/liblzma_w32res.rc | 10 +++++++++- - 1 file changed, 9 insertions(+), 1 deletion(-) + src/liblzma/common/stream_decoder_mt.c | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) -commit e77c0ca61d12ebac433b7661840cb18d7031700a +commit d5a2ffe41bb77b918a8c96084885d4dbe4bf6480 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-25 15:47:55 +0300 +Date: 2025-04-03 14:34:42 +0300 - common_w32res.rc: White space edits + liblzma: mt dec: Don't free the input buffer too early (CVE-2025-31115) - LANGUAGE and VS_VERSION_INFO begin new statements so put an empty line - between them. + The input buffer must be valid as long as the main thread is writing + to the worker-specific input buffer. Fix it by making the worker + thread not free the buffer on errors and not return the worker thread to + the pool. The input buffer will be freed when threads_end() is called. - (cherry picked from commit 8940ecb96fe9f0f2a9cfb8b66fe9ed31ffbea904) - - src/common/common_w32res.rc | 15 ++++++++------- - 1 file changed, 8 insertions(+), 7 deletions(-) - -commit e0ba0f26d9f3f53cedc92fb13303924c39d00392 -Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-28 20:09:50 +0300 - - CMake: Add the resource files to the Cygwin and MSYS2 builds + With invalid input, the bug could at least result in a crash. The + effects include heap use after free and writing to an address based + on the null pointer plus an offset. - Autotools-based build has always done this so this is for consistency. + The bug has been there since the first committed version of the threaded + decoder and thus affects versions from 5.3.3alpha to 5.8.0. - However, the CMake build won't create the DEF file when building - for Cygwin or MSYS2 because in that context it should be useless. - (If Cygwin or MSYS2 is used to host building of normal Windows - binaries then the DEF file is still created.) + As the commit message in 4cce3e27f529 says, I had made significant + changes on top of Sebastian's patch. This bug was indeed introduced + by my changes; it wasn't in Sebastian's version. + + Thanks to Harri K. Koskinen for discovering and reporting this issue. - (cherry picked from commit c3b9dad07d3fd9319f88386b7095019bcea45ce1) + Fixes: 4cce3e27f529 ("liblzma: Add threaded .xz decompressor.") + Reported-by: Harri K. Koskinen <x64nop@nannu.org> + Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc> + Thanks-to: Sam James <sam@gentoo.org> - CMakeLists.txt | 16 ++++++++++------ - 1 file changed, 10 insertions(+), 6 deletions(-) + src/liblzma/common/stream_decoder_mt.c | 31 ++++++++++++++++++++++--------- + 1 file changed, 22 insertions(+), 9 deletions(-) -commit 69637d0c323c0d7d9619cff637c7ce97dabc4f02 +commit c0c835964dfaeb2513a3c0bdb642105152fe9f34 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-28 15:19:14 +0300 +Date: 2025-04-03 14:34:42 +0300 - CMake: Fix Windows resource file dependencies + liblzma: mt dec: Simplify by removing the THR_STOP state - If common_w32res.rc is modified, the resource files need to be rebuilt. - In contrast, the liblzma*.map files truly are link dependencies. + The main thread can directly set THR_IDLE in threads_stop() which is + called when errors are detected. threads_stop() won't return the stopped + threads to the pool or free the memory pointed by thr->in anymore, but + it doesn't matter because the existing workers won't be reused after + an error. The resources will be cleaned up when threads_end() is + called (reinitializing the decoder always calls threads_end()). - (cherry picked from commit da4f275bd1c18b897e5c2dd0043546de3accce0a) + Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc> + Thanks-to: Sam James <sam@gentoo.org> - CMakeLists.txt | 17 +++++++++-------- - 1 file changed, 9 insertions(+), 8 deletions(-) + src/liblzma/common/stream_decoder_mt.c | 75 +++++++++++++--------------------- + 1 file changed, 29 insertions(+), 46 deletions(-) -commit af8533459c60d7bc5b55f2f516251af4572169e4 +commit 831b55b971cf579ee16a854f177c36b20d3c6999 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-29 01:20:03 +0300 +Date: 2025-04-03 14:34:42 +0300 - CMake: Checking for CYGWIN covers MSYS2 too - - On MSYS2, both CYGWIN and MSYS are set. + liblzma: mt dec: Fix a comment - (cherry picked from commit 1c673c0aac7f7dee8dda2c1140351c8417a71e47) + Reviewed-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc> + Thanks-to: Sam James <sam@gentoo.org> - CMakeLists.txt | 2 +- + src/liblzma/common/stream_decoder_mt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -commit eca08e4c204db404911e513f95110dcb0fb919bd +commit b9d168eee4fb6393b4fe207c0aeb5faee316ca1a Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-28 09:37:30 +0300 +Date: 2025-04-03 14:34:30 +0300 - Translations: Add the SPDX license identifier to pt_BR.po - - (cherry picked from commit 6aaa0173b839e28429d43a8b62d257ad2f3b4521) + liblzma: Add assertions to lzma_bufcpy() - po/pt_BR.po | 2 ++ - 1 file changed, 2 insertions(+) + src/liblzma/common/common.c | 6 ++++++ + 1 file changed, 6 insertions(+) -commit 85801c96c32456300177fbbad1506b07f5dd0a47 +commit c8e0a4897b4d0f906966f5d4d4f662221d64f3ae Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-25 16:41:37 +0300 +Date: 2025-04-02 16:40:22 +0300 - Windows/CMake: Use the correct resource file for lzmadec.exe - - CMakeLists.txt was using xzdec_w32res.rc for both xzdec and lzmadec. - - Fixes: 998d0b29536094a89cf385a3b894e157db1ccefe - (cherry picked from commit dc7b9f24b737e4e55bcbbdde6754883f991c2cfb) + DOS: Update Makefile to fix the build - CMakeLists.txt | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + dos/Makefile | 2 ++ + 1 file changed, 2 insertions(+) -commit a341d19c835a8c10fcf561b00b548c53af43381e +commit 307c02ed698a69763ef1c9c0df4ff24727442118 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-25 21:29:59 +0300 +Date: 2025-03-29 12:41:32 +0200 - Translations: Update the Brazilian Portuguese translation + sysdefs.h: Avoid <stdalign.h> even with C11 compilers + + Oracle Developer Studio 12.6 on Solaris 10 claims C11 support in + __STDC_VERSION__ and supports _Alignas. However, <stdalign.h> is missing. + We only need alignas, so define it to _Alignas with C11/C17 compilers. + If something included <stdalign.h> later, it shouldn't cause problems. - (cherry picked from commit b834ae5f80911a3819d6cdb484f61b257174c544) + Thanks to Ihsan Dogan for reporting the issue and testing the fix. + + Fixes: c0e7eaae8d6eef1e313c9d0da20ccf126ec61f38 - po/pt_BR.po | 144 ++++++++++++++++++++++-------------------------------------- - 1 file changed, 53 insertions(+), 91 deletions(-) + src/common/sysdefs.h | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) -commit e69c0b9b2e00ade984393ef9cabac57342072328 +commit 7ce38b318339d6c01378a77585e08169ca3a604e Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-17 01:21:15 +0300 +Date: 2025-03-29 12:32:05 +0200 Update THANKS - - (cherry picked from commit eceb023d4c129fd63ee881a2d8696eaf52ad1532) THANKS | 1 + 1 file changed, 1 insertion(+) -commit aef9a25b3200457c16846b046222fb2c7967afe0 -Author: Tobias Stoeckmann <tobias@stoeckmann.org> -Date: 2024-09-16 23:19:46 +0200 +commit 688e51bde4c987589717b2be1a1fde9576c604fc +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-29 12:21:51 +0200 - lzmainfo: Avoid integer overflow - - The MB output can overflow with huge numbers. Most likely these are - invalid .lzma files anyway, but let's avoid garbage output. - - lzmadec was adapted from LZMA Utils. The original code with this bug - was written in 2005, over 19 years ago. - - Co-authored-by: Lasse Collin <lasse.collin@tukaani.org> - Closes: https://github.com/tukaani-project/xz/pull/144 - (cherry picked from commit 76cfd0a9bb33ae8e534b1f73f6359dc825589f2f) + Translations: Update the Croatian translation - src/lzmainfo/lzmainfo.c | 5 ++--- - 1 file changed, 2 insertions(+), 3 deletions(-) + po/hr.po | 14 +++++++------- + 1 file changed, 7 insertions(+), 7 deletions(-) -commit 40a7f163f56aca6b3c8b83e9382f5e5cb4f8e93b -Author: Tobias Stoeckmann <tobias@stoeckmann.org> -Date: 2024-09-16 22:04:40 +0200 +commit 173fb5c68b08a8c1369550267be258132b7760c6 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-25 18:23:57 +0200 - xzdec: Remove unused short option -M - - "xzdec -M123" exited with exit status 1 without printing - any messages. The "M:" entry should have been removed when - the memory usage limiter support was removed from xzdec. - - Fixes: 792331bdee706aa852a78b171040ebf814c6f3ae - Closes: https://github.com/tukaani-project/xz/pull/143 - [ Lasse: Commit message edits ] - - (cherry picked from commit 78355aebb7fb654302e5e33692ba109909dacaff) + doc/SHA256SUMS: Add 5.8.0 - src/xzdec/xzdec.c | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + doc/SHA256SUMS | 6 ++++++ + 1 file changed, 6 insertions(+) -commit c98714a57058ac381365c2ff1e1d1cd63a5742c4 +commit db9258e828bc2cd96e3954f1ddcc9d3530589025 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-10 13:54:47 +0300 +Date: 2025-03-25 15:18:32 +0200 - Update THANKS + Bump version and soname for 5.8.0 - (cherry picked from commit e5758db7bd75587a2499e0771907521a4aa86908) + Also remove the LZMA_UNSTABLE macro. - THANKS | 1 + - 1 file changed, 1 insertion(+) + src/liblzma/Makefile.am | 2 +- + src/liblzma/api/lzma/bcj.h | 2 -- + src/liblzma/api/lzma/version.h | 6 +++--- + src/liblzma/common/common.h | 2 -- + src/liblzma/liblzma_generic.map | 2 +- + src/liblzma/liblzma_linux.map | 2 +- + 6 files changed, 6 insertions(+), 10 deletions(-) -commit 4ed449517817b3659b35d19f39703e3c460f46c2 -Author: Firas Khalil Khana <firasuke@gmail.com> -Date: 2024-09-10 12:30:32 +0300 +commit bfb752a38f89ed03fc93d54f11c09f43fda64bc2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-25 15:18:32 +0200 - Build: Fix a typo in autogen.sh - - Fixes: e9be74f5b129fe8a5388d588e68b1b7f5168a310 - Closes: https://github.com/tukaani-project/xz/pull/141 - (cherry picked from commit 80ffa38f56657257ed4d90d76f6bd2f2bcb8163c) + Add NEWS for 5.8.0 - autogen.sh | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + NEWS | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 62 insertions(+) -commit 3b83577a1547e72cb78a905ad3d308a799ded485 +commit 6ccbb904da851eb0c174c8dbd43e84da31739720 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-02 20:08:40 +0300 +Date: 2025-03-25 15:18:31 +0200 - Translations: Update Chinese (simplified) translation - - Differences to the zh_CN.po file from the Translation Project: - - - Two uses of \v were fixed. + Translations: Run "make -C po update-po" - - Missing "OPTS" translation in --riscv[=OPTS] was copied from - previous lines. + POT-Creation-Date is set to match the timestamp in 5.7.2beta which + in the Translation Project is known as 5.8.0-pre1. The strings + haven't changed since 5.7.1alpha but a few comments have. + + This is a very noisy commit, but this helps keeping the PO files + similar between the Git repository and stable release tarballs. + + po/ca.po | 964 ++++++++++++++++++++++++++++++++++++++++++++--------------- + po/cs.po | 935 ++++++++++++++++++++++++++++++++++++++++++---------------- + po/da.po | 663 ++++++++++++++++++++++++++++++----------- + po/de.po | 7 +- + po/eo.po | 966 +++++++++++++++++++++++++++++++++++++++++++++--------------- + po/es.po | 7 +- + po/fi.po | 2 +- + po/fr.po | 916 +++++++++++++++++++++++++++++++++++++++++--------------- + po/hu.po | 966 +++++++++++++++++++++++++++++++++++++++++++++--------------- + po/ka.po | 7 +- + po/ko.po | 7 +- + po/nl.po | 7 +- + po/pl.po | 7 +- + po/pt_BR.po | 962 ++++++++++++++++++++++++++++++++++++++++++++--------------- + po/sr.po | 2 +- + po/sv.po | 7 +- + po/tr.po | 7 +- + po/uk.po | 7 +- + po/vi.po | 948 +++++++++++++++++++++++++++++++++++++++++++--------------- + po/zh_CN.po | 940 ++++++++++++++++++++++++++++++++++++++++++++-------------- + po/zh_TW.po | 2 +- + 21 files changed, 6209 insertions(+), 2120 deletions(-) + +commit 891a5f057a6bb2dd2e3ce5e3bdd7a1f1ee03b800 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-25 15:18:31 +0200 + + Translations: Run po4a/update-po - - "make update-po" was run to remove line numbers from comments. + Also remove the trivial obsolete messages like man page dates. - (cherry picked from commit 68c54e45d042add64a4cb44bfc87ca74d29b87e2) + This is a noisy commit, but this helps keeping the PO files similar + between the Git repository and stable release tarballs. - po/zh_CN.po | 102 ++++++++++++++++++++++++------------------------------------ - 1 file changed, 40 insertions(+), 62 deletions(-) + po4a/fr.po | 82 +++++++++++++++++++++++++++++++++++++------------------ + po4a/pt_BR.po | 88 +++++++++++++++++++++++++++++++++++++++++------------------ + po4a/sr.po | 79 ++++++++++++++++++++++++++++++++++------------------- + 3 files changed, 167 insertions(+), 82 deletions(-) -commit 06f4c7edda0387eb6a2d6303804b59dcf4d3db1f +commit 4f52e7387012cb3510b01c937dd9b3a0c6a3ac6c Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-09-02 19:40:50 +0300 +Date: 2025-03-25 15:18:31 +0200 - Translations: Update the Catalan translation - - Differences to the ca.po file from the Translation Project: - - - An overlong line translating --filters-help was wrapped. - - - "make update-po" was used to remove line numbers from the comments - to match the changes in fccebe2b4fd513488fc920e4dac32562ed3c7637 - and 093490b58271e9424ce38a7b1b38bcf61b9c86c6. xz.pot in the TP - is older than these commits. + Translations: Partially fix overtranslation in Serbian man pages - (cherry picked from commit 2230692aa1bcebb586100183831e3daf1714d60a) + Names of environment variables and some other strings must be present + in the original form. The translator couldn't be reached so I'm + changing some of the strings myself. In the "Robot mode" section, + occurrences in the middle of sentences weren't changed to reduce + the chance of grammar breakage, but I kept the translated strings in + parenthesis in the headings. It's not ideal, but now people shouldn't + need to look at the English man page to find the English strings. - po/ca.po | 171 ++++++++++++++++++++++++++------------------------------------- - 1 file changed, 69 insertions(+), 102 deletions(-) + po4a/sr.po | 66 ++++++++++++++++++++++++++++++++++++++++++-------------------- + 1 file changed, 45 insertions(+), 21 deletions(-) -commit 406cb5b669e47c0e45c98f1afb7be998084a93d0 +commit ff5d944749b99eb5ab35e2ebaf01d05a59e7169b Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-08-22 11:01:07 +0300 +Date: 2025-03-25 15:18:31 +0200 - Update THANKS - - (cherry picked from commit 5e375987509fab484b7bef0b90be92f241c58c91) + liblzma: Count the extra bytes in LZMA/LZMA2 decoder memory usage - THANKS | 1 + - 1 file changed, 1 insertion(+) + src/liblzma/lz/lz_decoder.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) -commit 3a4a05d75eb41ddc41899324df0511670ceaaf1e -Author: Yifeng Li <tomli@tomli.me> -Date: 2024-08-22 02:18:49 +0000 +commit 943b012d09f717f7b44284c4e4976ea41264c731 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-25 15:18:31 +0200 - liblzma: Fix x86-64 movzw compatibility in range_decoder.h + liblzma: Use SSE2 intrinsics instead of memcpy() in dict_repeat() - Support for instruction "movzw" without suffix in "GNU as" was - added in commit [1] and stabilized in binutils 2.27, released - in August 2016. Earlier systems don't accept this instruction - without a suffix, making range_decoder.h's inline assembly - unable to build on old systems such as Ubuntu 16.04, creating - error messages like: + SSE2 is supported on every x86-64 processor. The SSE2 code is used on + 32-bit x86 if compiler options permit unconditional use of SSE2. - lzma_decoder.c: Assembler messages: - lzma_decoder.c:371: Error: no such instruction: `movzw 2(%r11),%esi' - lzma_decoder.c:373: Error: no such instruction: `movzw 4(%r11),%edi' - lzma_decoder.c:388: Error: no such instruction: `movzw 6(%r11),%edx' - lzma_decoder.c:398: Error: no such instruction: `movzw (%r11,%r14,4),%esi' + dict_repeat() copies short random-sized unaligned buffers. At least + on glibc, FreeBSD, and Windows (MSYS2, UCRT, MSVCRT), memcpy() is + clearly faster than byte-by-byte copying in this use case. Compared + to the memcpy() version, the new SSE2 version reduces decompression + time by 0-5 % depending on the machine and libc. It should never be + slower than the memcpy() version. - Change "movzw" to "movzwl" for compatibility. + However, on musl 1.2.5 on x86-64, the memcpy() version is the slowest. + Compared to the memcpy() version: - [1] https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c07315e0c610e0e3317b4c02266f81793df253d2 + - The byte-by-version takes 6-7 % less time to decompress. + - The SSE2 version takes 16-18 % less time to decompress. - Suggested-by: Lasse Collin <lasse.collin@tukaani.org> - Tested-by: Yifeng Li <tomli@tomli.me> - Signed-off-by: Yifeng Li <tomli@tomli.me> - Fixes: 3182a330c1512cc1f5c87b5c5a272578e60a5158 - Fixes: https://github.com/tukaani-project/xz/issues/121 - Closes: https://github.com/tukaani-project/xz/pull/136 - (cherry picked from commit 6cd7c8607843c337edfe2c472aa316602a393754) + The numbers are from decompressing a Linux kernel source tarball in + single-threaded mode on older AMD and Intel systems. The tarball + compresses well, and thus dict_repeat() performance matters more + than with some other files. - src/liblzma/rangecoder/range_decoder.h | 24 ++++++++++++------------ - 1 file changed, 12 insertions(+), 12 deletions(-) + src/liblzma/lz/lz_decoder.c | 14 ++++++-- + src/liblzma/lz/lz_decoder.h | 87 ++++++++++++++++++++++++++++++++++++++++----- + 2 files changed, 90 insertions(+), 11 deletions(-) -commit 4669f06d1a8d31de4b8b5861b5e8afd82cacd721 +commit bc14e4c94e788d42eeab984298391fc0ca46f969 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-19 20:02:43 +0300 +Date: 2025-03-25 15:18:31 +0200 - Build: Comment that elf_aux_info(3) will be available on OpenBSD >= 7.6 + liblzma: Add "restrict" to a few functions in lz_decoder.h + + This doesn't make any difference in practice because compilers can + already see that writing through the dict->buf pointer cannot modify + the contents of *dict itself: The LZMA decoder makes a local copy of + the lzma_dict structure, and even if it didn't, the pointer to + lzma_dict in the LZMA decoder is already "restrict". - (cherry picked from commit bf901dee5d4c46609645e50311c0cb2dfdcf9738) + It's nice to add "restrict" anyway. uint8_t is typically unsigned char + which can alias anything. Without the above conditions or "restrict", + compilers could need to assume that writing through dict->buf might + modify *dict. This would matter in dict_repeat() because the loops + refer to dict->buf and dict->pos instead of making local copies of + those members for the duration of the loops. If compilers had to + assume that writing through dict->buf can affect *dict, then compilers + would need to emit code that reloads dict->buf and dict->pos after + every write through dict->buf. - CMakeLists.txt | 2 +- - configure.ac | 17 +++++++++++------ - 2 files changed, 12 insertions(+), 7 deletions(-) + src/liblzma/lz/lz_decoder.h | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) -commit 9edddda5636d7b3504a033c31e8ea763e293fd35 +commit e82ee090c567e560f51a056775a17f534d159d65 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-13 22:10:37 +0300 +Date: 2025-03-25 15:18:30 +0200 - liblzma: Tweak a comment + liblzma: Define LZ_DICT_INIT_POS for initial dictionary position - (cherry picked from commit 7c292dd0bf23cefcdf4b1509f3666322e08a7ede) + It's more readable. - src/liblzma/simple/arm64.c | 4 ++-- - 1 file changed, 2 insertions(+), 2 deletions(-) + src/liblzma/lz/lz_decoder.c | 4 ++-- + src/liblzma/lz/lz_decoder.h | 9 ++++++--- + 2 files changed, 8 insertions(+), 5 deletions(-) -commit 1a93ab55d1563f5eb9b2c1b8240384046fe4bb97 +commit 8e7cd0091e5239334437decbe1989662d45a2f47 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-11 22:17:56 +0300 +Date: 2025-03-25 15:18:30 +0200 - CMake: Bump maximum policy version to 3.30 + Windows: Update README-Windows.txt about UCRT - CMakeLists.txt | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + windows/README-Windows.txt | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) -commit cfe4465742ad2963fb0d9795e258615d7c1cf32d +commit 2c24292d341e505e5579fccac3bce5bc71d839ef Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-09 14:27:51 +0300 +Date: 2025-03-25 15:18:15 +0200 Update THANKS - - (cherry picked from commit 028185dd4889e3d6235ff13560160ebca6985021) THANKS | 1 + 1 file changed, 1 insertion(+) -commit 0f47db18d04434203b350bde4909a5e468f197cc +commit 48053c90898fa191a216aefca01626520a7413f4 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-06 14:04:48 +0300 +Date: 2025-03-17 15:33:25 +0200 - xz: Remove the TODO comment about --recursive - - It won't be implemented. find + xargs is more flexible, for example, - it allows compressing small files in parallel. An example for that - has been included in the xz man page since 2010. - - (cherry picked from commit baecfa142644eb5f5c6dd6f8e2f531c362fa3747) + Translations: Update the Italian translation - src/xz/args.c | 1 - - 1 file changed, 1 deletion(-) + po/it.po | 32 ++++++++++++++++---------------- + 1 file changed, 16 insertions(+), 16 deletions(-) -commit 07f52c3528e43c4a925a3fc59a933c89f5604d92 +commit 8d6f06a65f50358fad13567f5dd8af41ef1d2b58 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-03 20:45:48 +0300 +Date: 2025-03-17 15:28:56 +0200 - CMake: Link xz against Threads::Threads if using pthreads + Translations: Update the Portuguese translation - The liblzma target was recently changed to link against Threads::Threads - with the PRIVATE keyword. I had forgotten that xz itself depends on - pthreads too due to pthread_sigmask(). Thus, the build broke when - building shared liblzma and pthread_sigmask() wasn't in libc. - - Thanks to Peter Seiderer for the bug report. - - Fixes: ac05f1b0d7cda1e7ae79775a8dfecc54601d7f1c - Fixes: https://github.com/tukaani-project/xz/issues/129#issuecomment-2204522994 - (cherry picked from commit b3e53122f42796aaebd767bab920cf7bedf69966) + The language tag in the Translation Project is pt, not pt_PT, + thus I changed the "Language:" line to pt. - CMakeLists.txt | 13 +++++++++++++ - 1 file changed, 13 insertions(+) + po/pt.po | 1045 +++++++++++++++++++++++++++++++------------------------------- + 1 file changed, 526 insertions(+), 519 deletions(-) -commit eccb4d258b01651d06a2a31b8b68be9b04b7998c +commit c3439b039f46fe547ad603e16dc3bd63c1ca9b0c Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-07-02 22:49:33 +0300 +Date: 2025-03-14 13:02:21 +0200 - Update THANKS - - (cherry picked from commit 5742ec1fc7f2cf1c82cfe3477bb90594a4658374) + Translations: Update the Italian translation - THANKS | 1 + - 1 file changed, 1 insertion(+) + po/it.po | 1020 +++++++++++++++++++++++++++++++------------------------------- + 1 file changed, 516 insertions(+), 504 deletions(-) -commit c9bd00327f064778babb014302718a18d65cf7d3 -Author: Sam James <sam@gentoo.org> -Date: 2024-06-28 14:18:35 +0300 +commit 79b4ab8d79528dd633a84df2d29e63f5d13ccbdf +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2025-03-12 20:48:39 +0200 - CI: Speed up Valgrind job by using --trace-children-skip-by-arg=... - - This addresses the issue I mentioned in - 6c095a98fbec70b790253a663173ecdb669108c4 and speeds up the Valgrind - job a bit, because non-xz tools aren't run unnecessarily with - Valgrind by the script tests. + Translations: Update the Italian man page translations - (cherry picked from commit 7e99856f66c07852c4e0de7aa01951e9147d86b0) + Only trivial additions but this keeps the file in sync with the TP. - .github/workflows/ci.yml | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + po4a/it.po | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) -commit 495de6ec9d7834c4ef4d5286844ef7b784eb951b +commit 515b6fc8557825e1335012b3b1c8cf71e2c38775 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-06-25 16:00:22 +0300 +Date: 2025-03-12 19:38:54 +0200 - Build: Prepend, not append, PTHREAD_CFLAGS to LIBS - - It shouldn't make any difference because LIBS should be empty - at that point in configure. But prepending is the correct way - because in general the libraries being added might require other - libraries that come later on the command line. - - (cherry picked from commit 2402e8a1ae92676fa0d4cb1b761d7f62f005c098) + Translations: Update the Italian man page translations - configure.ac | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) + po4a/it.po | 129 ++++++++++++++++++++++++++++++++++++------------------------- + 1 file changed, 77 insertions(+), 52 deletions(-) -commit 55bf3f49a812e20a21e42323e39526bb31d9341a +commit 333b7c0b776295f0941269b4e6cdb1a0ba5f6218 Author: Lasse Collin <lasse.collin@tukaani.org> -Date: 2024-06-25 14:24:29 +0300 +Date: 2025-03-10 21:00:31 +0200 - Build: Use AC_LINK_IFELSE to handle implicit function declarations - - It's more robust in case the compiler allows pre-C99 implicit function - declarations. If an x86 intrinsic is missing and gets treated as - implicit function, the linking step will very probably fail. This - isn't the only way to workaround implicit function declarations but - it might be the simplest and cleanest. - - The problem hasn't been observed in the wild. - - There are a couple more AC_COMPILE_IFELSE uses in configure.ac. - Of these, Landlock check calls prctl() and in theory could have - the same problem. In practice it doesn't as the check program - looks for several other things too. However, it was changed to - AC_LINK_IFELSE still to look more correct. - - Similarly, m4/tuklib_cpucores.m4 and m4/tuklib_physmem.m4 were - updated although they haven't given any trouble either. They - have worked all these years because those check programs rely - on specific headers and types: if headers or types are missing, - compilation will fail. Using the linker makes these checks more - similar to the ones in cmake/tuklib_*.cmake which always link. - - (cherry picked from commit 7bb46f2b7b3989c1b589a247a251470f65e91cda) + Translations: Update the Korean man page translations - configure.ac | 8 ++++++-- *** 14820 LINES SKIPPED ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202505081625.548GPwvL011825>