From owner-svn-src-stable-12@freebsd.org Sun Oct 6 04:10:29 2019 Return-Path: Delivered-To: svn-src-stable-12@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 76459FA02D; Sun, 6 Oct 2019 04:10:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46m9BY24Cmz4Xh4; Sun, 6 Oct 2019 04:10:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2013A1F8B6; Sun, 6 Oct 2019 04:10:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x964ASJF041942; Sun, 6 Oct 2019 04:10:28 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x964ASp5041939; Sun, 6 Oct 2019 04:10:28 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201910060410.x964ASp5041939@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sun, 6 Oct 2019 04:10:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r353138 - in stable/12: contrib/netbsd-tests/usr.bin/grep usr.bin/grep X-SVN-Group: stable-12 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in stable/12: contrib/netbsd-tests/usr.bin/grep usr.bin/grep X-SVN-Commit-Revision: 353138 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Oct 2019 04:10:29 -0000 Author: kevans Date: Sun Oct 6 04:10:28 2019 New Revision: 353138 URL: https://svnweb.freebsd.org/changeset/base/353138 Log: MFC r352691: bsdgrep(1): fixes of empty pattern/exit code/-c behavior When an empty pattern is encountered in the pattern list, I had previously broken bsdgrep to count that as a "match all" and ignore any other patterns in the list. This commit rectifies that mistake, among others: - The -v flag semantics were not quite right; lines matched should have been counted differently based on whether the -v flag was set or not. procline now definitively returns whether it's matched or not, and interpreting that result has been kicked up a level. - Empty patterns with the -x flag was broken similarly to empty patterns with the -w flag. The former is a whole-line match and should be more strict, only matching blank lines. No -x and no -w will will match the empty string at the beginning of each line. - The exit code with -L was broken, w.r.t. modern grep. Modern grap will exit(0) if any file that didn't match was output, so our interpretation was simply backwards. The new interpretation makes sense to me. Tests updated and added to try and catch some of this. This misbehavior was found by autoconf while fixing ports found in PR 229925 expecting either a more sane or a more GNU-like sed. Modified: stable/12/contrib/netbsd-tests/usr.bin/grep/t_grep.sh stable/12/usr.bin/grep/grep.c stable/12/usr.bin/grep/util.c Directory Properties: stable/12/ (props changed) Modified: stable/12/contrib/netbsd-tests/usr.bin/grep/t_grep.sh ============================================================================== --- stable/12/contrib/netbsd-tests/usr.bin/grep/t_grep.sh Sun Oct 6 04:02:29 2019 (r353137) +++ stable/12/contrib/netbsd-tests/usr.bin/grep/t_grep.sh Sun Oct 6 04:10:28 2019 (r353138) @@ -413,6 +413,60 @@ wflag_emptypat_body() atf_check -o file:test4 grep -w -e "" test4 } +atf_test_case xflag_emptypat +xflag_emptypat_body() +{ + printf "" > test1 + printf "\n" > test2 + printf "qaz" > test3 + printf " qaz\n" > test4 + + # -x is whole-line, more strict than -w. + atf_check -s exit:1 -o empty grep -x -e "" test1 + + atf_check -o file:test2 grep -x -e "" test2 + + atf_check -s exit:1 -o empty grep -x -e "" test3 + + atf_check -s exit:1 -o empty grep -x -e "" test4 + + total=$(wc -l /COPYRIGHT | sed 's/[^0-9]//g') + + # Simple checks that grep -x with an empty pattern isn't matching every + # line. The exact counts aren't important, as long as they don't + # match the total line count and as long as they don't match each other. + atf_check -o save:xpositive.count grep -Fxc '' /COPYRIGHT + atf_check -o save:xnegative.count grep -Fvxc '' /COPYRIGHT + + atf_check -o not-inline:"${total}" cat xpositive.count + atf_check -o not-inline:"${total}" cat xnegative.count + + atf_check -o not-file:xnegative.count cat xpositive.count +} + +atf_test_case xflag_emptypat_plus +xflag_emptypat_plus_body() +{ + printf "foo\n\nbar\n\nbaz\n" > target + printf "foo\n \nbar\n \nbaz\n" > target_spacelines + printf "foo\nbar\nbaz\n" > matches + printf " \n \n" > spacelines + + printf "foo\n\nbar\n\nbaz\n" > patlist1 + printf "foo\n\nba\n\nbaz\n" > patlist2 + + sed -e '/bar/d' target > matches_not2 + + # Normal handling first + atf_check -o file:target grep -Fxf patlist1 target + atf_check -o file:matches grep -Fxf patlist1 target_spacelines + atf_check -o file:matches_not2 grep -Fxf patlist2 target + + # -v handling + atf_check -s exit:1 -o empty grep -Fvxf patlist1 target + atf_check -o file:spacelines grep -Fxvf patlist1 target_spacelines +} + atf_test_case excessive_matches excessive_matches_head() { @@ -551,6 +605,12 @@ grep_nomatch_flags_head() grep_nomatch_flags_body() { + grep_type + + if [ $? -eq $GREP_TYPE_GNU_FREEBSD ]; then + atf_expect_fail "this test does not pass with GNU grep in base" + fi + printf "A\nB\nC\n" > test1 atf_check -o inline:"1\n" grep -c -C 1 -e "B" test1 @@ -563,7 +623,7 @@ grep_nomatch_flags_body() atf_check -o inline:"test1\n" grep -l -A 1 -e "B" test1 atf_check -o inline:"test1\n" grep -l -C 1 -e "B" test1 - atf_check -s exit:1 -o inline:"test1\n" grep -L -e "D" test1 + atf_check -o inline:"test1\n" grep -L -e "D" test1 atf_check -o empty grep -q -e "B" test1 atf_check -o empty grep -q -B 1 -e "B" test1 @@ -777,6 +837,8 @@ atf_init_test_cases() atf_add_test_case egrep_empty_invalid atf_add_test_case zerolen atf_add_test_case wflag_emptypat + atf_add_test_case xflag_emptypat + atf_add_test_case xflag_emptypat_plus atf_add_test_case excessive_matches atf_add_test_case wv_combo_break atf_add_test_case fgrep_sanity Modified: stable/12/usr.bin/grep/grep.c ============================================================================== --- stable/12/usr.bin/grep/grep.c Sun Oct 6 04:02:29 2019 (r353137) +++ stable/12/usr.bin/grep/grep.c Sun Oct 6 04:10:28 2019 (r353138) @@ -218,20 +218,9 @@ static void add_pattern(char *pat, size_t len) { - /* Do not add further pattern is we already match everything */ - if (matchall) - return; - /* Check if we can do a shortcut */ if (len == 0) { matchall = true; - for (unsigned int i = 0; i < patterns; i++) { - free(pattern[i].pat); - } - pattern = grep_realloc(pattern, sizeof(struct pat)); - pattern[0].pat = NULL; - pattern[0].len = 0; - patterns = 1; return; } /* Increase size if necessary */ @@ -652,7 +641,7 @@ main(int argc, char *argv[]) aargv += optind; /* Empty pattern file matches nothing */ - if (!needpattern && (patterns == 0)) + if (!needpattern && (patterns == 0) && !matchall) exit(1); /* Fail if we don't have any pattern */ @@ -699,11 +688,10 @@ main(int argc, char *argv[]) r_pattern = grep_calloc(patterns, sizeof(*r_pattern)); - /* Don't process any patterns if we have a blank one */ #ifdef WITH_INTERNAL_NOSPEC - if (!matchall && grepbehave != GREP_FIXED) { + if (grepbehave != GREP_FIXED) { #else - if (!matchall) { + { #endif /* Check if cheating is allowed (always is for fgrep). */ for (i = 0; i < patterns; ++i) { @@ -735,7 +723,12 @@ main(int argc, char *argv[]) matched = true; } - /* Find out the correct return value according to the - results and the command line option. */ + if (Lflag) + matched = !matched; + + /* + * Calculate the correct return value according to the + * results and the command line option. + */ exit(matched ? (file_err ? (qflag ? 0 : 2) : 0) : (file_err ? 2 : 1)); } Modified: stable/12/usr.bin/grep/util.c ============================================================================== --- stable/12/usr.bin/grep/util.c Sun Oct 6 04:02:29 2019 (r353137) +++ stable/12/usr.bin/grep/util.c Sun Oct 6 04:10:28 2019 (r353138) @@ -210,7 +210,7 @@ procmatch_match(struct mprintc *mc, struct parsec *pc) while (pc->matchidx >= MAX_MATCHES) { /* Reset matchidx and try again */ pc->matchidx = 0; - if (procline(pc)) + if (procline(pc) == !vflag) printline(pc, ':'); else break; @@ -355,7 +355,7 @@ procfile(const char *fn) return (0); } - line_matched = procline(&pc); + line_matched = procline(&pc) == !vflag; if (line_matched) ++lines; @@ -469,18 +469,33 @@ procline(struct parsec *pc) matchidx = pc->matchidx; - /* Special case: empty pattern with -w flag, check first character */ - if (matchall && wflag) { + /* + * With matchall (empty pattern), we can try to take some shortcuts. + * Emtpy patterns trivially match every line except in the -w and -x + * cases. For -w (whole-word) cases, we only match if the first + * character isn't a word-character. For -x (whole-line) cases, we only + * match if the line is empty. + */ + if (matchall) { if (pc->ln.len == 0) return (true); - wend = L' '; - if (sscanf(&pc->ln.dat[0], "%lc", &wend) != 1 || iswword(wend)) - return (false); - else + if (wflag) { + wend = L' '; + if (sscanf(&pc->ln.dat[0], "%lc", &wend) == 1 && + !iswword(wend)) + return (true); + } else if (!xflag) return (true); - } else if (matchall) - return (true); + /* + * If we don't have any other patterns, we really don't match. + * If we do have other patterns, we must fall through and check + * them. + */ + if (patterns == 0) + return (false); + } + matched = false; st = pc->lnstart; nst = 0; @@ -609,8 +624,6 @@ procline(struct parsec *pc) /* Reflect the new matchidx in the context */ pc->matchidx = matchidx; - if (vflag) - matched = !matched; return matched; }