From owner-svn-src-head@freebsd.org Sat Feb 13 16:10:54 2021 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 39581530DF5; Sat, 13 Feb 2021 16:10:54 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DdFht0xcLz4gpr; Sat, 13 Feb 2021 16:10:54 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-il1-x134.google.com with SMTP id z18so1988182ile.9; Sat, 13 Feb 2021 08:10:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=JqLgxL1EQrNpJTrgu04Qo7hpngFDjihduUHrDWtuQwM=; b=uRs3JNnMeY3RPHewb8EEE4wzmQ3P5G91Ta6oBagDgG7JFy462KKunvQvgwcn3PjWMT 9n7yi5BwtyfnyVHxmresQMi8VKqG86VB1rvNSLSoli9rnYEZiMFbEIjmmPiGmqCSdsiK +bXN/GHB78ngoQG6yD0q+XAORj5krMJVRyBlXwdtB20O8tdJOdmPHmCVj0vuysSDcnnm pVCgQzNMALqiQdzwVM6Ncn7md3KoyzRzfsUBl874+gdG8kAWpaRqKj6QG0hssTshf0m+ ALXeZnTONdmAFIQmxB8Rq+7E2dTZF07zsCu2/XAGYfTIhW6E6LUY+yQiH6lbiOm2pEgZ FxYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=JqLgxL1EQrNpJTrgu04Qo7hpngFDjihduUHrDWtuQwM=; b=rhSBPNp090EapOwVn6Ifj0iazMb7oGUIHgGiLyVzIkED3zvt/8X6CJtsBBPzXT7ZW8 RjeyW1UpGXNEWCsx0yQwYSh6YpjtGcU8iDlCL33kokc9jAr+c2VTTBJi15ogTUDnakj6 9XKTBq2TA/N6uO6ZOEWgw01egVWxm8HhRbh7Uir4hr6GivH8HGaQKwCB1J+R8E8lIjPJ 8xKEFNNsI1E6KLJslcMUGngGF3KefJsmdpqJN2Y750xj60yJKyaVDSQ9eopydlCQ15FH pIZ3pbC9thy8utyw+w+4es4VZpnE72eRr9m2TQXqd9jhzEdbBKcsJf8qbxnN+ioKnm55 oYUQ== X-Gm-Message-State: AOAM530vUmxKu9Zi+BG+itnrCCecfZY0i5j38LLzX4cghbtUgSUaaTrq KVWRGMr0o/mHWD3mPQPPsloVXeIa+Ko= X-Google-Smtp-Source: ABdhPJzhiYXIipfhbpYcMMz9nIhHuwafyVnik/z29382DiQpkfDkVaa3I906g0SGVI4l3MffotEExA== X-Received: by 2002:a92:c60d:: with SMTP id p13mr6760523ilm.118.1613232652626; Sat, 13 Feb 2021 08:10:52 -0800 (PST) Received: from raichu ([142.126.164.150]) by smtp.gmail.com with ESMTPSA id d135sm5920451iog.35.2021.02.13.08.10.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 13 Feb 2021 08:10:51 -0800 (PST) Sender: Mark Johnston Date: Sat, 13 Feb 2021 11:10:49 -0500 From: Mark Johnston To: Jung-uk Kim Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r362333 - in head: contrib/flex contrib/flex/src usr.bin/lex usr.bin/lex/lib Message-ID: References: <202006181809.05II9G8p054025@repo.freebsd.org> <0a0ac901-c1d2-54d7-d585-2856ae4fe6ea@FreeBSD.org> <68440f27-7b70-cc1f-d646-9669c4093838@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <68440f27-7b70-cc1f-d646-9669c4093838@FreeBSD.org> X-Rspamd-Queue-Id: 4DdFht0xcLz4gpr X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Feb 2021 16:10:54 -0000 On Sat, Feb 13, 2021 at 02:39:56AM -0500, Jung-uk Kim wrote: > On 21. 2. 13., Jung-uk Kim wrote: > > On 21. 2. 12., Mark Johnston wrote: > >> On Thu, Jun 18, 2020 at 06:09:16PM +0000, Jung-uk Kim wrote: > >>> Author: jkim > >>> Date: Thu Jun 18 18:09:16 2020 > >>> New Revision: 362333 > >>> URL: https://svnweb.freebsd.org/changeset/base/362333 > >>> > >>> Log: > >>> MFV: r362286 > >>> > >>> Merge flex 2.6.4. > >> > >> This seems to have introduced a regression: input() now returns 0 > >> instead of EOF to indicate that the end of input was reached. This has > >> been reported in a few places: > >> > >> https://github.com/westes/flex/issues/448 > >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911415 > >> > >> It breaks the scanner used by libdtrace, and as a result dtrace is > >> unable to resolve some probe argument types: > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253440 > >> > >> I have a small patch to libdtrace which works around this. The upstream > >> commit that introduced the change is fairly old at this point, and > >> upstream seems somewhat unresponsive, so I'm not sure it'll get > >> reverted. Some other scanners in the tree look like they'd be affected, > >> like crypto/heimdal/lib/asn1/lex.l or > >> sys/contrib/dev/acpica/compiler/aslsupport.l, so I'm wondering if we > >> should revert this particular change for 13.0. Certainly we should fix > >> the DTrace bug one way or another for 13.0. > > > > I think it does not make much sense and we should revert the upstream > > change. Does the attached patch work for you? The original patch fixes it, the updated one does not since libdtrace sets -l in LFLAGS. :) Removing it causes a build failure but it is trivial to resolve. I am running some tests now to see if anything regressed. > Now I believe the author wanted Lex compatibility. Maybe we should do > the attached patch instead? Is it common to use the base system's flex for ports and out-of-tree code generally? I have no real idea. I'd be worried about configure scripts that use the flex version to determine whether or not to apply a workaround for this change. I also note that Debian does not seem to have patched this. Looking through consumers in the tree (all *.l files), the following scanners could be affected: - libdtrace - contrib/com_err/lex.l - contrib/ipfilter/iplang/iplang_l.l - does not appear to be built - crypto/heimdal/lib/asn1/lex.l - crypto/heimdal/lib/com_err/lex.l - crypto/heimdal/lib/hx509/sel-lex.l - crypto/heimdal/lib/sl/slc-lex.l - sys/contrib/dev/acpica/compiler/aslsupport.l - sys/contrib/dev/acpica/compiler/prparser.l It would not be too difficult to patch them, and I think in most cases the problem is not very severe and is only triggered by invalid input. For instance, asn1_compile enters an infinite loop if the input file contains an unterminated comment. So for 13.0 I am kind of inclined to just patch consumers. If you think we should revert the change in flex, I'm ok with that too and will remove the use of -l in libdtrace. > diff --git a/contrib/flex/src/flex.skl b/contrib/flex/src/flex.skl > index 242645f53245..c23b944ea473 100644 > --- a/contrib/flex/src/flex.skl > +++ b/contrib/flex/src/flex.skl > @@ -1863,7 +1863,11 @@ m4_ifdef( [[M4_YY_USE_LINENO]], > case EOB_ACT_END_OF_FILE: > { > if ( yywrap( M4_YY_CALL_ONLY_ARG ) ) > +#ifdef YY_FLEX_LEX_COMPAT > return 0; > +#else > + return EOF; > +#endif > > if ( ! YY_G(yy_did_buffer_switch_on_eof) ) > YY_NEW_FILE;