From nobody Tue Jan 18 23:07:15 2022 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 43D1019688E7 for ; Tue, 18 Jan 2022 23:07:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jdktq6kQWz3QFK for ; Tue, 18 Jan 2022 23:07:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C19881CEC for ; Tue, 18 Jan 2022 23:07:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 20IN7F58023497 for ; Tue, 18 Jan 2022 23:07:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 20IN7FHo023496 for bugs@FreeBSD.org; Tue, 18 Jan 2022 23:07:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 254763] grep very slow with 13.0-RC4 Date: Tue, 18 Jan 2022 23:07:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 13.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: Mark.Martinec@ijs.si X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642547236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SPseVnoLwH1b+USbBmfYjDnJimG6o9qB7I4tjmGlzK4=; b=Q7lQugVB6DY24lUEmm6JbeEbBB/qEG0HxMnTfk5Z5D6TePYjlf3yeIVGpeHnsQptG6U0+B hhGJ4boh3cHg1Sx9Mrau8yLkS5KFEpr+PfweoFOX2GbRzh1qoDXg0madZVReqzhq076+o1 VkRsUdJdKD0Efll8tEtySanGGZON/fWNBMNKWzXlC96kQsr29cRoYYijPcYh3+u7Yao6Se grnS4vI+qotE/JbPGcOMoCI9vsLj1Ky/3vSRNyKIIASJDQll0ODelgl0bk3w9vJqRo1Fw3 otwnvb7pVaUhwv/QGihjPFn+KqWRJdrd+bBKKQwG6y8Z3JE4zfbA7Al6rrIgeg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642547236; a=rsa-sha256; cv=none; b=rq/OO/2qatWosZSAMEmR5gpA+lJa5tleI29u8ikaLzWZjZOc3GxRcf4NSBvWbLk80Lp0zT r/DxKlJPxm9hEidhyTRM6hglOCl/8aLAyPMu3jPAiyn1GxRcVt74Egwb42Yh0I6Euj6amo rhi7W9fiONczRbftWBqQpR6lj4nkaAIo7A2OEhYKUELYptN2veFvZfyhXAl+0f47MqRy5T FBCwelZqDqjDXVRXEVDTEKjGTKdh5rHnH1bOv46MUtqCNWiETBYZFnBL9OVYoTnTS408R7 x19hBGdAKQ3NOihZcwKCls9+xXvl9mt5JGP1cnPothDDiant94F8BWTpa0JUiQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D254763 Mark.Martinec@ijs.si changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |Mark.Martinec@ijs.si --- Comment #11 from Mark.Martinec@ijs.si --- I was about to open a new PR, but I realized the problem has already been reported, so I'm commenting here. My particular issue: fgrep -i is terribly slow on 13.0. More in detail: Searching for a string though a one day's worth of mail log (100 MB, postfix) takes unreasonably long time with BSD grep when option -i (case-insensitive search) is specified: $ time /usr/bin/fgrep -i fwzhkqwfoherbfqo mail.log real 59.664s, user 58.907s, sys 0.222s $ time /usr/bin/fgrep fwzhkqwfoherbfqo mail.log real 0.276s, user 0.097s, sys 0.173s With option -i it takes 200-times longer than without -i. Compared to pcregrep or GNU grep, it takes 500-times longer. fgrep and grep and egrep are the same in this respect. An interesting observation: as the length of a search string increases, so does the run time. It would be worthwhile investing some time in finding and fixing this particular hotspot in BSD grep. (the machine above was a slightly older 13.0-RELEASE-p6 amd64, with SSD disks and ZFS, the file is cached (cache warmed up)) ---------------------- The rest below is some more benchmarking, this time on a faster machine with nvme disks, ZFS, same log file. Absolute times are shorter, but the ratio is about the same. $ uname -a FreeBSD xxx 13.0-RELEASE-p6 FreeBSD ... amd64/sys/GENERIC amd64 $ ls -l mail.log xxx 99999998 Jan 18 22:49 mail.log BSD grep: =3D=3D=3D=3D=3D=3D=3D=3D=3D $ time /usr/bin/fgrep -i fwzh mail.log real 0m8.733s, user 0m8.664s, sys 0m0.061s $ time /usr/bin/fgrep -i fwzhkqwf mail.log real 0m12.759s, user 0m12.666s, sys 0m0.057s $ time /usr/bin/fgrep -i fwzhkqwfoherbfqo mail.log real 0m18.922s, user 0m18.813s, sys 0m0.064s $ time /usr/bin/fgrep -i fwzhkqwfoherbfqojhkqnsazmzlwknhg mail.log real 0m32.593s, user 0m32.438s, sys 0m0.056s BSD grep without -i: $ time /usr/bin/fgrep fwzhkqwfoherbfqojhkqnsazmzlwknhg mail.log real 0m0.112s, user 0m0.033s, sys 0m0.079s GNU grep (textutils/gnugrep): =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D $ time /usr/local/bin/fgrep -i fwzh mail.log real 0m0.204s, user 0m0.073s, sys 0m0.022s $ time /usr/local/bin/fgrep -i fwzhkqwf mail.log real 0m0.203s, user 0m0.060s, sys 0m0.015s $ time /usr/local/bin/fgrep -i fwzhkqwfoherbfqo mail.log real 0m0.175s, user 0m0.036s, sys 0m0.017s $ time /usr/local/bin/fgrep -i fwzhkqwfoherbfqojhkqnsazmzlwknhg mail.log real 0m0.217s, user 0m0.083s, sys 0m0.006s pcregrep: =3D=3D=3D=3D=3D=3D=3D=3D=3D $ time pcregrep -i fwzh mail.log real 0m0.249s, user 0m0.114s, sys 0m0.026s $ time pcregrep -i fwzhkqwf mail.log real 0m0.227s, user 0m0.094s, sys 0m0.013s $ time pcregrep -i fwzhkqwfoherbfqo mail.log real 0m0.128s, user 0m0.073s, sys 0m0.055s $ time pcregrep -i fwzhkqwfoherbfqojhkqnsazmzlwknhg mail.log real 0m0.126s, user 0m0.079s, sys 0m0.047s --=20 You are receiving this mail because: You are the assignee for the bug.=