From owner-svn-src-head@freebsd.org Tue Dec 10 19:16:01 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5EDC01E2335; Tue, 10 Dec 2019 19:16:01 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47XVCP1pxRz481H; Tue, 10 Dec 2019 19:16:01 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 395622F5E3; Tue, 10 Dec 2019 19:16:01 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xBAJG1Xb080841; Tue, 10 Dec 2019 19:16:01 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xBAJG0Lf080839; Tue, 10 Dec 2019 19:16:00 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201912101916.xBAJG0Lf080839@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Tue, 10 Dec 2019 19:16:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r355590 - in head/usr.bin/sed: . tests tests/regress.multitest.out X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in head/usr.bin/sed: . tests tests/regress.multitest.out X-SVN-Commit-Revision: 355590 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Dec 2019 19:16:01 -0000 Author: kevans Date: Tue Dec 10 19:16:00 2019 New Revision: 355590 URL: https://svnweb.freebsd.org/changeset/base/355590 Log: sed: process \r, \n, and \t This is both reasonable and a common GNUism that a lot of ported software expects. Universally process \r, \n, and \t into carriage return, newline, and tab respectively. Newline still doesn't function in contexts where it can't (e.g. BRE), but we process it anyways rather than passing UB \n (escaped ordinary) through to the underlying regex engine. Adding a --posix flag to disable these was considered, but sed.1 already declares this version of sed a super-set of POSIX specification and this behavior is the most likely expected when one attempts to use one of these escape sequences in pattern space. This differs from pre-r197362 behavior in that we now honor the three arguably most common escape sequences used with sed(1) and we do so outside of character classes, too. Other escape sequences, like \s and \S, will come later when GNU extensions are added to libregex; sed will likely link against libregex by default, since the GNU extensions tend to be fairly un-intrusive. PR: 229925 Reviewed by: bapt, emaste, pfg Differential Revision: https://reviews.freebsd.org/D22750 Modified: head/usr.bin/sed/compile.c head/usr.bin/sed/tests/regress.multitest.out/8.22 head/usr.bin/sed/tests/sed2_test.sh Modified: head/usr.bin/sed/compile.c ============================================================================== --- head/usr.bin/sed/compile.c Tue Dec 10 18:57:39 2019 (r355589) +++ head/usr.bin/sed/compile.c Tue Dec 10 19:16:00 2019 (r355590) @@ -395,10 +395,21 @@ compile_delimited(char *p, char *d, int is_tr) continue; } else if (*p == '\\' && p[1] == '[') { *d++ = *p++; - } else if (*p == '\\' && p[1] == c) + } else if (*p == '\\' && p[1] == c) { p++; - else if (*p == '\\' && p[1] == 'n') { - *d++ = '\n'; + } else if (*p == '\\' && + (p[1] == 'n' || p[1] == 'r' || p[1] == 't')) { + switch (p[1]) { + case 'n': + *d++ = '\n'; + break; + case 'r': + *d++ = '\r'; + break; + case 't': + *d++ = '\t'; + break; + } p += 2; continue; } else if (*p == '\\' && p[1] == '\\') { @@ -428,13 +439,29 @@ compile_ccl(char **sp, char *t) *t++ = *s++; if (*s == ']') *t++ = *s++; - for (; *s && (*t = *s) != ']'; s++, t++) + for (; *s && (*t = *s) != ']'; s++, t++) { if (*s == '[' && ((d = *(s+1)) == '.' || d == ':' || d == '=')) { *++t = *++s, t++, s++; for (c = *s; (*t = *s) != ']' || c != d; s++, t++) if ((c = *s) == '\0') return NULL; + } else if (*s == '\\') { + switch (s[1]) { + case 'n': + *t = '\n'; + s++; + break; + case 'r': + *t = '\r'; + s++; + break; + case 't': + *t = '\t'; + s++; + break; + } } + } return (*s == ']') ? *sp = ++s, ++t : NULL; } @@ -521,8 +548,23 @@ compile_subst(char *p, struct s_subst *s) linenum, fname, *p); if (s->maxbref < ref) s->maxbref = ref; - } else if (*p == '&' || *p == '\\') - *sp++ = '\\'; + } else { + switch (*p) { + case '&': + case '\\': + *sp++ = '\\'; + break; + case 'n': + *p = '\n'; + break; + case 'r': + *p = '\r'; + break; + case 't': + *p = '\t'; + break; + } + } } else if (*p == c) { if (*++p == '\0' && more) { if (cu_fgets(lbuf, sizeof(lbuf), &more)) Modified: head/usr.bin/sed/tests/regress.multitest.out/8.22 ============================================================================== --- head/usr.bin/sed/tests/regress.multitest.out/8.22 Tue Dec 10 18:57:39 2019 (r355589) +++ head/usr.bin/sed/tests/regress.multitest.out/8.22 Tue Dec 10 19:16:00 2019 (r355590) @@ -1,2 +1 @@ -1 -2 +1X2 Modified: head/usr.bin/sed/tests/sed2_test.sh ============================================================================== --- head/usr.bin/sed/tests/sed2_test.sh Tue Dec 10 18:57:39 2019 (r355589) +++ head/usr.bin/sed/tests/sed2_test.sh Tue Dec 10 19:16:00 2019 (r355590) @@ -69,9 +69,29 @@ inplace_command_q_body() atf_check -s not-exit:0 stat -q '.!'* } +atf_test_case escape_subst +escape_subst_head() +{ + atf_set "descr" "Verify functional escaping of \\n, \\r, and \\t" +} +escape_subst_body() +{ + printf "a\nt\\\t\n\tb\n\t\tc\r\n" > a + tr -d '\r' < a > b + printf "a\tb c\rx\n" > c + + atf_check -o 'inline:a\nt\\t\n' sed '/\t/d' a + atf_check -o 'inline:a\nt\\t\n b\n c\r\n' sed 's/\t/ /g' a + atf_check -o 'inline:a\nt\\t\n\t\tb\n\t\t\t\tc\r\n' sed 's/\t/\t\t/g' a + atf_check -o 'inline:a\nt\n\tb\n\t\tc\r\n' sed 's/\\t//g' a + atf_check -o file:b sed 's/\r//' a + atf_check -o 'inline:abcx\n' sed 's/[ \r\t]//g' c +} + atf_init_test_cases() { atf_add_test_case inplace_command_q atf_add_test_case inplace_hardlink_src atf_add_test_case inplace_symlink_src + atf_add_test_case escape_subst }