From owner-freebsd-current@freebsd.org Tue Jul 5 04:16:45 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6BB0B84A03 for ; Tue, 5 Jul 2016 04:16:45 +0000 (UTC) (envelope-from huanghwh@gmail.com) Received: from mail-vk0-x244.google.com (mail-vk0-x244.google.com [IPv6:2607:f8b0:400c:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CD531E84; Tue, 5 Jul 2016 04:16:45 +0000 (UTC) (envelope-from huanghwh@gmail.com) Received: by mail-vk0-x244.google.com with SMTP id k68so7170462vkb.2; Mon, 04 Jul 2016 21:16:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc; bh=IRXX+RXnq7wXMUb2GrWmpOyOaszzlb0XBNZTZJ8n03U=; b=qU0MCrCtzKBQrdLF21mAJrsp9mpKHFgGVeYogKRTfJmv8FM+nVdgtmRjfK8phs/02t jHJApNvHXbv2LrynQGv98A69wp9VNHEE9ngfA3+TJ7mUxL8Lw8Pb1TnjTK+9XkWqMIm2 jd18I2Fxe4uIapSUWJ59AJve5qVb2DsxinHlCcGOxP/onqWdG4mkyntKaMic7LOpVCbe 0BzFGCf7ON8AbAvWkbeNz3g+JUtGMPggUDkTMWDjUmz5h3Gnn5pl3Z7b8R4PP5Zn4V/G BvQ+5qFnLxkKv4Q2OENYxJOtkGUf319talZGXW3eOco5Bl79mvWGaxwQ72sNcKDAoqKz wrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=IRXX+RXnq7wXMUb2GrWmpOyOaszzlb0XBNZTZJ8n03U=; b=OZTf6CVKJE9Gd9wQJZtpdKFrTylHERoUFYFQESydzBuN4QAnHH5rCrnOg79/9gNJQ7 uJ35eGu5B6ZGKpJlLDV0NmCYjs2Z92dTaN5TEnKzPiZ/w5dkeZScSaVZXSCjLraGhLQk 0oovqzT4ZXbhRZH9v33vVnHz9I1JbB88fqFKURiqhRfm+Q/UX3hbV0IH8WxMEAYvvMlw BeYLN+oimM0sPzr25iyfg7XaPqQqwMycsaT5Ku8QkSFYG4BxpeZuBZLhIpiv7r3ivdBA h49WtdVWIQ7O974ltc4xP7eyy+xOR784uAL5aWdvEs8KO68yiMn3jeGVS//Rqf27ZTmt M/2g== X-Gm-Message-State: ALyK8tJC6FG9zh0MupbDhQoMPrGO7G9dBoiYGkv3ch9ypigjGATzwcwUDjvhepcOzi6fMSSklWwSuUtPcZqL+w== X-Received: by 10.176.1.17 with SMTP id 17mr5243246uak.154.1467692203409; Mon, 04 Jul 2016 21:16:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.129.65 with HTTP; Mon, 4 Jul 2016 21:16:42 -0700 (PDT) Reply-To: huanghwh@gmail.com In-Reply-To: <20160704070401.uavtrarwtkreolck@ivaldir.etoilebsd.net> References: <20160704062026.oqml7ehc6ka7gg37@ivaldir.etoilebsd.net> <20160704064119.xm7tcw53vejchgmm@ivaldir.etoilebsd.net> <20160704070401.uavtrarwtkreolck@ivaldir.etoilebsd.net> From: Huang Wen Hui Date: Tue, 5 Jul 2016 12:16:42 +0800 Message-ID: Subject: Re: ls eat high CPU time when LANG=zh_CN.UTF-8 and LC_ALL=zh_CN.UTF-8 To: Baptiste Daroussin Cc: freebsd-current Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jul 2016 04:16:46 -0000 These 2 files can make ls suck: touch =E7=81=AB=E7=81=BE1 touch =E7=81=AB=E7=81=BE2 2 files start with 2 same Chinese chars. % lldb /bin/ls (lldb) target create "/bin/ls" Current executable set to '/bin/ls' (x86_64). (lldb) run Process 2185 launching Process 2185 launched: '/bin/ls' (x86_64) Enter Control+C: libc.so.7 was compiled with optimization - stepping may behave oddly; variables may not be available. Process 2185 stopped * thread #1: tid =3D 100261, 0x0000000800ff5aa7 libc.so.7`_collate_lookup [inlined] largesearch(table=3D) + 38 at collate.c:276, stop reason =3D signal SIGSTOP frame #0: 0x0000000800ff5aa7 libc.so.7`_collate_lookup [inlined] largesearch(table=3D) + 38 at collate.c:276 [opt] 273 next =3D (low + high) / 2; 274 p =3D tab + next; 275 compar =3D key - p->val; -> 276 if (compar =3D=3D 0) 277 return (p); 278 if (compar > 0) 279 low =3D next + 1; (lldb) bt * thread #1: tid =3D 100261, 0x0000000800ff5aa7 libc.so.7`_collate_lookup [inlined] largesearch(table=3D) + 38 at collate.c:276, stop reason =3D signal SIGSTOP * frame #0: 0x0000000800ff5aa7 libc.so.7`_collate_lookup [inlined] largesearch(table=3D) + 38 at collate.c:276 [opt] frame #1: 0x0000000800ff5a81 libc.so.7`_collate_lookup(table=3D, t=3D, len=3D, pri=3D, which=3D, state=3D) + 465 at collate.c:343 [opt] frame #2: 0x0000000800fd80a9 libc.so.7`wcscoll_l(ws1=3D, ws2=3D, locale=3D) + 985 at wcscoll.c:171 [opt] frame #3: 0x0000000800fd4d19 libc.so.7`strcoll_l(s=3D"=E7=81=AB=E7=81= =BE1", s2=3D"=E7=81=AB=E7=81=BE2", locale=3D0x000000080124a338) + 393 at strcoll.c:101 [opt] frame #4: 0x0000000800fe9313 libc.so.7`qsort(a=3D, n=3D, es=3D, cmp=3D(libc.so.7`fts_compar at fts.c= :966)) + 13763 at qsort.c:130 [opt] frame #5: 0x0000000800f25297 libc.so.7`fts_sort(sp=3D, head=3D, nitems=3D) + 135 at fts.c:995 [opt] frame #6: 0x0000000800f2638e libc.so.7`fts_children(sp=3D, instr=3D2) + 254 at fts.c:570 [opt] frame #7: 0x00000000004030df ls`traverse(argc=3D, argv=3D, options=3D) + 463 at ls.c:576 [opt] frame #8: 0x0000000000402eeb ls`main(argc=3D, argv=3D) + 2299 at ls.c:498 [opt] frame #9: 0x00000000004020cf ls`_start + 383 2016-07-04 15:04 GMT+08:00 Baptiste Daroussin : > On Mon, Jul 04, 2016 at 02:51:46PM +0800, Huang Wen Hui wrote: > > 2016-07-04 14:41 GMT+08:00 Baptiste Daroussin : > > > > > On Mon, Jul 04, 2016 at 02:36:11PM +0800, Huang Wen Hui wrote: > > > > 2016-07-04 14:20 GMT+08:00 Baptiste Daroussin : > > > > > > > > > On Mon, Jul 04, 2016 at 11:56:36AM +0800, Huang Wen Hui wrote: > > > > > > Hi, > > > > > > On very recent CURRENT, ls can eat high CPU time when > > > LANG=3Dzh_CN.UTF-8 > > > > > and > > > > > > LC_ALL=3Dzh_CN.UTF-8: > > > > > > > > > > > > % uname -a > > > > > > FreeBSD mbp.gddsn.org.cn 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #121 > > > r302331M: > > > > > Mon > > > > > > Jul 4 10:47:27 CST 2016 root@mbp.gddsn.org.cn: > > > > > /usr/obj/usr/src/sys/MACBOOK > > > > > > amd64 > > > > > > > > > > > > top show: > > > > > > 4457 hwh 1 100 0 16784K 4416K CPU4 4 0:22 > 98.86% > > > ls > > > > > > > > > > > > any ideas? > > > > > > > > > > > Is it in all directories or only in directories with files in > chinese > > > > > characters? > > > > > > > > > Yes, the directory contain Chinese characters. > > > > > > > > > > > > > > Is it only happening when you run ls with some arguments (in > particular > > > > > -l) or > > > > > with any arguments? > > > > > > > > > I use ls -wGl > > > > > > > > > > > > > > Do you see the same if you force any other locale like en_US.UTF-= 8? > > > > > > > > > There is no problem if set en_US.UTF-8. > > > > > > > > > > > > > Best regards, > > > > > Bapt > > > > > > > > > > > Can you try: > > > env -i LANG=3Dzh_CN.UTF-8 LC_COLLATE=3DC ls -l > > > > > > And tell me if it still happen? > > > > > No problem with this command. > > > > Ok so there might be an very inefficient code in the new chinese collatio= n > code > I will look into it thanks a lot for reporting. > > Best regards, > Bapt >