Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 23 Jan 2020 14:35:13 +0100
From:      Miroslav Lachman <000.fbsd@quip.cz>
To:        Wojciech Puchar <wojtek@puchar.net>
Cc:        freebsd-hackers@freebsd.org
Subject:   Re: slow directory operation on huge dirs
Message-ID:  <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz>
In-Reply-To: <alpine.BSF.2.20.2001231411400.63433@puchar.net>
References:  <alpine.BSF.2.20.2001191930040.17538@puchar.net> <alpine.BSF.2.20.2001231245190.98419@puchar.net> <cf4d3d5f-6c9e-81cf-f6a2-ade177b9f8ff@quip.cz> <alpine.BSF.2.20.2001231411400.63433@puchar.net>

next in thread | previous in thread | raw e-mail | index | archive | help
Wojciech Puchar wrote on 2020/01/23 14:12:
>>>>
>>> nobody have idea what's the problem?
>>
>> Is it "just slow" or is your disk too busy? (iostat)
> 
> no. disk is idle, system CPU load takes half of system (8 cores)!
> 
>>
>> Can it be that pre-cacheing on huge directory makes disk too busy 
>> because there is too much to read?
> this part works fine. The problem begins where it's already cached.
> 
> To get the problem you need BOTH of:
> 
> - large maxvnodes like half million
> - large directory (in order of 100000 files)

I remember very similar problem with slow access of directories with 80 
000+ files. It was always slow like frozen hell. There is some 
recommendation to avoid more than 10 000 of file entries in a directory. 
That is why filesystem caches or sessions in PHP must create more levels 
of subdirectories.
I know you cannot control users of maildirs.
I am not sure if this situation can be improved or not.

Miroslav Lachman



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9b426949-be87-106a-46c3-f1b6a2e5bb83>