From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:13:02 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15AB71F1AA5 for ; Thu, 23 Jan 2020 13:13:02 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483N4D6bcGz4cM8 for ; Thu, 23 Jan 2020 13:13:00 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00NDCsN1064241 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 14:12:55 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579785175; bh=MqJmYb+pCgIjfOqx0M1U6sdkldv6xmsCGSXDFPetwTQ=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=Z0/+7egQlQX0OLNrUDHlOmIwbMFuLFmFNZrK70fIwS0sKxIDKCBN2rvldFhEvDUmK 9MayCjBljmr0UW6bwT5m1y/6Q5PmBAbuNlo51BI/kL5y16xGv6ghtZMho7hXzCANP+ 18Vl55JNcEBtC4y+zKCGBViB7RpwMC6nIQQMLTuo= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00NDCsnD064238; Thu, 23 Jan 2020 14:12:54 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 23 Jan 2020 14:12:54 +0100 (CET) From: Wojciech Puchar To: Miroslav Lachman <000.fbsd@quip.cz> cc: Wojciech Puchar , freebsd-hackers@freebsd.org Subject: Re: slow directory operation on huge dirs In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 483N4D6bcGz4cM8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=Z0/+7egQ; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.64)[ip: (-6.97), ipnet: 194.1.144.0/24(-3.49), asn: 43476(-2.79), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:13:02 -0000 >>> >> nobody have idea what's the problem? > > Is it "just slow" or is your disk too busy? (iostat) no. disk is idle, system CPU load takes half of system (8 cores)! > > Can it be that pre-cacheing on huge directory makes disk too busy because > there is too much to read? this part works fine. The problem begins where it's already cached. To get the problem you need BOTH of: - large maxvnodes like half million - large directory (in order of 100000 files)