Date: Mon, 20 Aug 2018 11:57:29 -0700 From: Conrad Meyer <cem@freebsd.org> To: Ali Abdallah <aliovx@gmail.com> Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org> Subject: Re: rand_harvestq high cpu usage when /dev/urandom is used Message-ID: <CAG6CVpUEA0X7bK=BG8CiLyD_9SJbkJ%2B8=pOWfWwCL3mFpiiuBQ@mail.gmail.com> In-Reply-To: <CAO3OKx_3nKDEKP_eii0QoC7DG57Y_KLrHxwJrV7r7UMRL%2BX5_A@mail.gmail.com> References: <CAO3OKx_3nKDEKP_eii0QoC7DG57Y_KLrHxwJrV7r7UMRL%2BX5_A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Ali, On Mon, Aug 20, 2018 at 4:27 AM, Ali Abdallah <aliovx@gmail.com> wrote: > Hello, > > I was just sorting randomly some jpg image files using: > > ls *.jpg | sort -R --random-source=/dev/urandom > > The above command never exited. Later I noticed that > one of my CPU is always running 100%. top -S reveals that it is > rand_harvestq kernel service. > > Is this is a bug? This occurs on 12-ALPHA1 and 11.2 There is probably at least one bug in sort(1). sort has special behavior if the --random-source matches its default (/dev/random), but otherwise doesn't understand device files or pipes very well. Since urandom isn't exactly the same path as /dev/random, sort fails pretty hard. sort attempts to seed its internal RNG by MD5ing the provided random source path. For its default path, /dev/random, it grabs at most MAX_DEFAULT_RANDOM_SEED_DATA_SIZE (or 1024) bytes. This is hugely excessive and MD5ing it is totally unnecessary, but still mostly harmless. For non-default files, it just passes the path to MD5File, which will read() until EOF. Since /dev/urandom will never return EOF, sort --random-source=/dev/urandom will get stuck in MD5File forever. This is totally stupid. I'm not sure why rand_harvestq would take 100% of a CPU core even as a result of excessive consumption of /dev/urandom, but it is certainly possible that sort(1) is consuming 100% of a CPU core reading from urandom and MD5ing the result. All the best, Conrad
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAG6CVpUEA0X7bK=BG8CiLyD_9SJbkJ%2B8=pOWfWwCL3mFpiiuBQ>