Date: Sun, 24 Dec 2023 06:22:52 +0200 From: Sulev-Madis Silber <madis555@hot.ee> To: d@delphij.net, Xin Li <delphij@delphij.net>, freebsd-current@freebsd.org Subject: Re: Proposal: Disable compression of newsyslog by default Message-ID: <E124EE62-15CA-48C7-BED1-BA21B5587EDF@hot.ee> In-Reply-To: <ace7d170-acd1-4dc8-8016-a08c261d91f0@delphij.net> References: <ace7d170-acd1-4dc8-8016-a08c261d91f0@delphij.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 23 December 2023 09:18:23 EET, Xin Li <delphij@delphij=2Enet> wrote: >Hi, > >Inspired by D42961, I propose that we move forward with disabling the com= pression by default in newsyslog, as implemented in https://reviews=2Efreeb= sd=2Eorg/D43169 > >Historically, newsyslog has compressed rotated log files to save disk spa= ce=2E This approach was valuable in the early days where storage space was = limited=2E it's still limited >However, the landscape has changed significantly=2E Modern file systems,= such as ZFS, now offer native compression capabilities=2E not everyone uses them >Additionally, the widespread availability of larger hard drives has dimin= ished the necessity for additional compression=2E but data sizes also have increased massively >Notably, the need to decompress log files for pattern searches poses a si= gnificant inconvenience, further questioning the utility of this legacy fea= ture=2E should be up to each admin to cba decompression vs=2E plain speed/size/etc >In commit 906748d208d3, flags J, X, Y, Z can now indicate that a log file= is eligible for compression rather than directly enforcing it=2E It allows= for a more flexible approach, wherein the actual compression method can be= set to "none" or specified as one among bzip2, gzip, xz, or zstd=2E that's good approach >Therefore I would propose that we change the default compression setting = to "none" in FreeBSD 15=2E0=2E This change reflects our adaptation to the = evolving technological environment and user needs=2E It also aligns with t= he broader initiative to modernize our systems while maintaining flexibilit= y and efficiency=2E unsure about this=2E generic zroot install would be fine with this i guess= , and usual log sizes? other custom installs need tuning anyway >I look forward to your thoughts and feedback on this proposal=2E > >Cheers, indeed=2E we have large disks now=2E but we fill them all=2E i started wit= h 1=2E2g one=2E was too small=2E needed to compress for space=2E now i have= 12t=2E it's still too small=2E i compress for space=2E they make them up t= o 22t nowadays=2E this is about 20000 times larger but still feels small=2E= how did this happen? we just did this to ourselves=2E data sizes have kept= up with storage and bandwidth=2E gamer might get 1gbit/s connection at hom= e so (s)he only needs to wait for one hour to download new game=2E just as = dialup user once did, wait for hours=2E or it could be photographer, graphi= cs designer or architect who works=2E both cases still use compressed data = as cpu and ram permits it and it saves a lot of time and space=2E now, it's= also related to servers as those things don't disappear to or appear from = just thin air=2E they come from machines, some of them hopefully running fb= sd, where admins wonder how to deal with large log sizes=2E they need them = for audit purposes=2E or statistics=2E hardware allows, so they compress it= =2E "write-only-read-never" data benefits from, eg, xz a lot=2E as others a= lready have told so yeah, from (only!) 25+ years of experience, i can confirm that humankin= d has developed AND used everything at max=2E internet, first for military = and educational uses, now for connecting washing machines=2E oh and, first = hdd, state of art device then, can only store *part* of *compressed* photo = now now, this might not be related to default fbsd installs in common usage wh= ere default base syslog creates tiny amount of data per week but one of reasons was given how everything fits uncompressed nowdays=2E t= o our disks and pipes=2E which it really doesn't
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E124EE62-15CA-48C7-BED1-BA21B5587EDF>