Date: Thu, 2 Dec 2021 10:00:24 -0700 From: Alan Somers <asomers@freebsd.org> To: Stefan Blachmann <sblachmann@gmail.com> Cc: Johannes Totz <jo@bruelltuete.com>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: dd performance [was Re: Call for Foundation-supported Project Ideas] Message-ID: <CAOtMX2jkUthc0nMdas71QV81cf-1r0bQSgShY-RVnLAhyPjsQQ@mail.gmail.com> In-Reply-To: <CACc-My1qsby=t%2BNMvBH6hDQ7cBNdFvbWoxs91JDLZmz3hQd79Q@mail.gmail.com> References: <861r36xzpe.fsf@phe.ftfl.ca> <20211128220732.GA81140@troutmask.apl.washington.edu> <d0c77bfe-6a37-e177-f64d-2e1d3fc23dc2@gmail.com> <20211129003635.GA81568@troutmask.apl.washington.edu> <CAPyFy2DaDsn9oq1-pXysJ702wjO3kGAhmtKVHz9AnJYAtjwdhA@mail.gmail.com> <01e739f7-ccb2-c59f-9843-9d5214032b77@bruelltuete.com> <CACc-My1qsby=t%2BNMvBH6hDQ7cBNdFvbWoxs91JDLZmz3hQd79Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is very surprising to me. I never see dd take significant CPU consumption until the speed gets up into the GB/s range. What are you using for the bs= option? If you set that too low, or use the default, it will needlessly consume extra CPU and IOPs. I usually set it to 1m for this kind of usage. And what kind of HDDs are these, connected to what kind of controller? On Thu, Dec 2, 2021 at 9:54 AM Stefan Blachmann <sblachmann@gmail.com> wrote: > > Regarding the suggestions to either improve or replace the ULE > scheduler, I would like to share another observation. > > Usually when I need to zero out HDDs using dd, I use a live Linux. > This time I did that on FreeBSD (13). > My observations: > - On the same hardware, the data transfer rate is a small fraction > (about 1/4th) of which is achieved by Linux. > - The first dd process, which erases the first HDD, gets almost all > CPU and I/O time. The second process which does the second HDD is > getting starved. It actually really starts only after the first one > finished. > > To me it was *very* surprising to find out that, while erasing two > similar HDDs concurrently takes about one day on Linux, on FreeBSD, > the first HDD was finished after three days, and only after that the > remaining second dd process got the same CPU time, making it proceed > fast instead of creepingly slowly. > > So I guess this might be a scheduler issue. > I certainly will do some tests using the old scheduler when I got time. > And, I ask myself: > Could it be a good idea to sponsor porting the Dragonfly scheduler to FreeBSD? > > On 12/2/21, Johannes Totz <jo@bruelltuete.com> wrote: > > On 29/11/2021 03:17, Ed Maste wrote: > >> On Sun, 28 Nov 2021 at 19:37, Steve Kargl > >> <sgk@troutmask.apl.washington.edu> wrote: > >>> > >>> It's certainly not the latest and greatest, > >>> CPU: Intel(R) Core(TM)2 Duo CPU T7250 @ 2.00GHz (1995.04-MHz > >>> K8-class CPU) > >> > >> If you're content to use a compiler from a package you can save a lot > >> of time by building with `CROSS_TOOLCHAIN=llvm13` and > >> `WITHOUT_TOOLCHAIN=yes`. Or, instead of WITHOUT_TOOLCHAIN perhaps > >> `WITHOUT_CLANG=yes`, `WITHOUT_LLD=yes` and `WITHOUT_LLDB=yes`. > > > > (re-send to list, sorry) > > Can we disconnect the compiler optimisation flag for base and clang? I > > don't need the compiler to be build with -O2 but I want the resulting > > base system to have optimisations enabled. > > Right now, looks like both get -O2 and a lot of time is spent on > > optimising the compiler (for no good reason). > > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2jkUthc0nMdas71QV81cf-1r0bQSgShY-RVnLAhyPjsQQ>