From nobody Sun Dec 24 04:22:52 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SySZ95rCLz53l5g for ; Sun, 24 Dec 2023 04:22:57 +0000 (UTC) (envelope-from madis555@hot.ee) Received: from SMTPOUT09.DKA.mailcore.net (smtpout09.dka.mailcore.net [194.19.134.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtpout09.dka.mailcore.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SySZ93MH3z4Cw9 for ; Sun, 24 Dec 2023 04:22:57 +0000 (UTC) (envelope-from madis555@hot.ee) Authentication-Results: mx1.freebsd.org; none Received: from SMTP.DKA.mailcore.net (unknown [10.1.0.53]) by SMTPOUT01.DKA.mailcore.net (Postfix) with ESMTP id 3C89AE0A5D; Sun, 24 Dec 2023 05:22:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=online.ee; s=mailcore; t=1703391774; bh=0RaXSS3nqtQnqUroYv9y0dpzHbbTWBPYdr3Y0z5WkXg=; h=Date:From:To:Subject:In-Reply-To:References:From; b=mbhv45sFQ8tpE/6JcAUxle6KUsjyPfQo8x6li19/unhXZ5JRGhGMLkX34960O+ODj G5bdecU3ZY6kPqPzoPDH1FlvIOhrREsZ7pAF816VB5Nta01kypDak9pyGpCFpHYfGd 7Vlai+JSFxd/8w+7vwdacpPPGH66/nyfUXHO6y1dmeop+BH5y+HSLJAUQ7QTL5ysuY 8mP5VZuICIYvY1qtQt6pvjuciDtrIyDqZQoTi7FZ7u2saovHBnqyBTmQ+G35KkUGbF gjxNO3eJvYV6SW/HJGAx0zrFyZzM4kwRsTwEUEHrt6/g76o7GcJbZlfIM0uW9Kzf/F NCuFMuojpE0bA== Received: from [127.0.0.1] (169-122-46-176.dyn.estpak.ee [176.46.122.169]) by SMTP.DKA.mailcore.net (Postfix) with ESMTPSA id 12FC2400AC; Sun, 24 Dec 2023 05:22:54 +0100 (CET) Date: Sun, 24 Dec 2023 06:22:52 +0200 From: Sulev-Madis Silber To: d@delphij.net, Xin Li , freebsd-current@freebsd.org Subject: Re: Proposal: Disable compression of newsyslog by default User-Agent: K-9 Mail for Android In-Reply-To: References: Message-ID: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3308, ipnet:194.19.128.0/18, country:SE] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4SySZ93MH3z4Cw9 On 23 December 2023 09:18:23 EET, Xin Li wrote: >Hi, > >Inspired by D42961, I propose that we move forward with disabling the com= pression by default in newsyslog, as implemented in https://reviews=2Efreeb= sd=2Eorg/D43169 > >Historically, newsyslog has compressed rotated log files to save disk spa= ce=2E This approach was valuable in the early days where storage space was = limited=2E it's still limited >However, the landscape has changed significantly=2E Modern file systems,= such as ZFS, now offer native compression capabilities=2E not everyone uses them >Additionally, the widespread availability of larger hard drives has dimin= ished the necessity for additional compression=2E but data sizes also have increased massively >Notably, the need to decompress log files for pattern searches poses a si= gnificant inconvenience, further questioning the utility of this legacy fea= ture=2E should be up to each admin to cba decompression vs=2E plain speed/size/etc >In commit 906748d208d3, flags J, X, Y, Z can now indicate that a log file= is eligible for compression rather than directly enforcing it=2E It allows= for a more flexible approach, wherein the actual compression method can be= set to "none" or specified as one among bzip2, gzip, xz, or zstd=2E that's good approach >Therefore I would propose that we change the default compression setting = to "none" in FreeBSD 15=2E0=2E This change reflects our adaptation to the = evolving technological environment and user needs=2E It also aligns with t= he broader initiative to modernize our systems while maintaining flexibilit= y and efficiency=2E unsure about this=2E generic zroot install would be fine with this i guess= , and usual log sizes? other custom installs need tuning anyway >I look forward to your thoughts and feedback on this proposal=2E > >Cheers, indeed=2E we have large disks now=2E but we fill them all=2E i started wit= h 1=2E2g one=2E was too small=2E needed to compress for space=2E now i have= 12t=2E it's still too small=2E i compress for space=2E they make them up t= o 22t nowadays=2E this is about 20000 times larger but still feels small=2E= how did this happen? we just did this to ourselves=2E data sizes have kept= up with storage and bandwidth=2E gamer might get 1gbit/s connection at hom= e so (s)he only needs to wait for one hour to download new game=2E just as = dialup user once did, wait for hours=2E or it could be photographer, graphi= cs designer or architect who works=2E both cases still use compressed data = as cpu and ram permits it and it saves a lot of time and space=2E now, it's= also related to servers as those things don't disappear to or appear from = just thin air=2E they come from machines, some of them hopefully running fb= sd, where admins wonder how to deal with large log sizes=2E they need them = for audit purposes=2E or statistics=2E hardware allows, so they compress it= =2E "write-only-read-never" data benefits from, eg, xz a lot=2E as others a= lready have told so yeah, from (only!) 25+ years of experience, i can confirm that humankin= d has developed AND used everything at max=2E internet, first for military = and educational uses, now for connecting washing machines=2E oh and, first = hdd, state of art device then, can only store *part* of *compressed* photo = now now, this might not be related to default fbsd installs in common usage wh= ere default base syslog creates tiny amount of data per week but one of reasons was given how everything fits uncompressed nowdays=2E t= o our disks and pipes=2E which it really doesn't