From owner-freebsd-current@freebsd.org Sat Jul 11 00:14:14 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 981C93778C1 for ; Sat, 11 Jul 2020 00:14:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B3Vl935TNz47WQ for ; Sat, 11 Jul 2020 00:14:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: JrDRBAAVM1lc4NZUvLudrv3QSNMbTQPTidouGTYmMnwrpciMuBxLc99GWrgOWfI mp1Y4gateDc4FL5qcwFqZm9DOsmD9gx.5CDdm0iVRb_zH0Y7QeSsKuDfBqi0pRdk1mFJI.VxHLWH I3QqQWdf8fWPePeyLAvrAQ9Lt_FYL_Igytc8ThSjVLwLJkSNfakBuEYcTO.3BGO738g56wne8zs9 HsJ7CmY65f1zfexR_rfZx7Jhe.KI7awQtEuihcEey59ruKns9pPeLu903KqWJBde9njHjIuPevlg o8XaOn.q7MwXCSvaB2UBM.BxaEukiWhSVOzHCk6nUZisrGDSVbODzC6QVM4QPgk_S0t3iST9JAK2 AKwS9ycp50L.ZE5I6a1UoUm5ZnN2v5U9XyX8TDcC_csXXJw8JhoGOVP1NXRsQ1xRhm8DUpMKZDlO uEX8_cQ6FxQV5ddlaLVAZeRnBkKSkxaw0vyBA6pfQaQIakAEO0KvSI14AjQPne_fDUWYna1WxkuQ ayw4mPXS5MwHgtCk_bKI6xKODzWAhr92WA8gVd3aHDEQ.SNl.aHRoY.lYOth1h1GBU04tpoFR4w4 jyuFebeMEsoeW3oMYIC2HC7BSGDpO23EDdgbpdU0W_Cgh_Xc.elO6fePNa.P8JxPeiyELPWRAfIK 68G5VrZa4ZEXVsAaLCn4o2FPGSbDm1xBP.A4mNiV4kF3AdMfKWxdGXg.TBLDmi.JUTzxRWF86NGp fc.Sj_qxYM0eQbFhpXqV3eoFIKm68g9mlmWebHA2ViB8zHxamK7gbYH9trzZVE3hhz_6bh2zArmw i40XfLZDJyAioSAWdMxAwmvMMDv3FvI6V1LtkqehJurnefniUZqQbQ_pbVNOIB8S6anWRnMDMKpq EauoATXXl2vpFuLOrxxQr53n2Tmmi26xi1D2ZNEG_NH7mfTHOvbLLU.KAyA975UyHDqbCFY.fWfF gTb5XYR02tR_rnZJAY.hXUAoqGfDG29C8PzS6HlJNlAQdD2mGWPHH4SfV2kPx8J.nS2.LyD4tZl_ 8_cLCFkJa8xr4JLWXrqgbMtxgnlVxCIoGIZjC78gma5bAlHGNLR9Hxu4b7_PdMDHETwgrNzVRNdN YndHr88z.Xm1PRTgujyvp4IrIHnEhT0r6Uo_cdLPmvuPd9K2jUKNAVJmqq2EY3vAmdrdnWDCy2VE 9SuPAsVAnk.5EGmGrCnw1QWTwMP.s4ykO9NhmxczVIB8kxVj9TNvKlh_3WjJKiZd4eFbo1vYrefn eiEK4NB5fyNPHxGjXXGJeqgnHeSQixGbjmN5xug1OPlQ92v1deE1Z6c4p4q_Bk_oB9IbpzlRJalD m.XnSgX0Kfp6rqNMfgVqLZ57JxFjcjFY_9oo_XMBMkpf_Tpmx.R_RgB8wYgQjBab_23vytQhT_ng eDhZH.Wet_.ww9nIcs8Gm.RlKYtVcIRnNZ00ZMZvVacnoV1fJERzjWCVQew-- Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Sat, 11 Jul 2020 00:14:11 +0000 Received: by smtp411.mail.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 4ef9987ea87d0498253f8a536517d6a2; Sat, 11 Jul 2020 00:14:10 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: svn commit: r352558 - head/usr.bin/top From: Mark Millard In-Reply-To: Date: Fri, 10 Jul 2020 17:14:10 -0700 Cc: Steve Wills , "daichi@freebsd.org" , FreeBSD Current , Hiroki Sato Content-Transfer-Encoding: quoted-printable Message-Id: References: <1BDFB387-930D-4F4D-8729-A5850F1C15B9.ref@yahoo.com> <1BDFB387-930D-4F4D-8729-A5850F1C15B9@yahoo.com> <61107ecc-6f9b-a4db-7b1e-ec75f73939ee@FreeBSD.org> To: Yuri Pankov X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 4B3Vl935TNz47WQ X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.34 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.84)[-0.838]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.011]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Jul 2020 00:14:14 -0000 On 2020-Jul-10, at 16:12, Yuri Pankov wrote: > Mark Millard wrote: >> On 2020-Jul-10, at 11:05, Yuri Pankov wrote: >>> Steve Wills wrote: >>>> On 11/28/19 4:08 PM, Mark Millard via svn-src-head wrote: >>>>>> Author: daichi >>>>>> Date: Fri Sep 20 17:37:23 2019 >>>>>> New Revision: 352558 >>>>>> URL: >>>>>> https://svnweb.freebsd.org/changeset/base/352558 >>>>>>=20 >>>>>>=20 >>>>>> Log: >>>>>> top(1): support multibyte characters in command names (ARGV = array) >>>>>> depending on locale. >>>>>> - add setlocale() >>>>>> - remove printable() function >>>>>> - add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to = display >>>>>> non-printable characters that do not use C-style backslash = sequences >>>>>> in three digit octal sequence, or remove it >>>>>> This change allows multibyte characters to be displayed = according to >>>>>> locale. If it is recognized as a non-display character = according to the >>>>>> locale, it is displayed in three digit octal sequence. >>>>>>=20 >>>>>=20 >>>>> Initially picking on tab characters as an example of what is >>>>> probably a somewhat broader issue . . . >>>>>=20 >>>>> Ever since this change, characters like tabs that do not fit >>>>> in the next character cell when output, but for which they >>>>> are !isprintable(...), now mess up the top display. Again >>>>> using tab as an example: line wrapping from the text having >>>>> been shifted over by more than one character cell. top does >>>>> not track the line wrapping result in how it decides what >>>>> to output for the following display updates. >>>>>=20 >>>> Apologies for the way late reply here, but I just now bothered = tracking this down. This commit seems to be the cause of some corruption = I'm seeing in long running top(1) as well. As Mark mentions, if I use = "hh" it clears up. Should I open a bugzilla bug? I can share screenshots = of the corruption, such as: >>>> https://i.imgur.com/Xqlwf9h.png >>>> https://i.imgur.com/Jv0d5NU.png >>>=20 >>> Does removing VIS_SAFE fixes the issue for you? >>>=20 >>> As for original Mark's report (which I missed), removing = isprintable() doesn't look wrong as vis(3) should take of its = functionality (and in multibyte-aware way). >> vis (as used) and the old isprintable logic are not >> equivalent when multi-byte is not needed/involved. >> Otherwise I'd not have had anything to ever report. >> If vis can do what is needed, more work needed to >> be done when the change was made in order to avoid >> msesed up displays in single-byte contexts. >>> Also, is there an easy way to reproduce this? >> The following sort of command (the empty space inside quoted >> text are tab characters): >> # tr '0\n 1\n 2\n 3\n 4\n 5\n 6\n 7\n = 8\n' '\t0 \t1 \t2 \t3 \t4 \t5 \t6 \t7 = \t8' < /dev/zero > /dev/null >> causes my 200 character wide window running top to show: >> 32920 root 100 0 12764Ki 2420Ki CPU3 3 2:22 = 99.87% tr 0\\n 1\\n 2\\n 3\\n 4\\n 5\\n 6\\n 7\\n = 8\\n \\t0 \\t1 \\t2 \\t3 \\t4 \\t5 \\t6 \\t733 = \\t8 20 7172 5448Ki CPU23 23 0:00 0.04% top = -HiSCazopid >> But that does not show where the lines wrap at the edges of the = window, >> so breaking it up explicitly after the first "\" in \\7: >> 32920 root 100 0 12764Ki 2420Ki CPU3 3 2:22 = 99.87% tr 0\\n 1\\n 2\\n 3\\n 4\\n 5\\n 6\\n 7\\n = 8\\n \\t0 \\t1 \\t2 \\t3 \\t4 \\t5 \\t6 \ >> \t733 \\t8 20 7172 5448Ki CPU23 23 0:00 = 0.04% top -HiSCazopid >> Note how \n turned into \\n , taking an extra character for >> each \n . Similarly for \t vs. \\t . (Other examples do >> similarly.) >> The tab characters really do use more than one character cell >> on the display (sometimes). >> The text from the tr command ends up spread across 2 lines >> as things look like in the window where top is running. >> I ran top in another ssh session first and then the tr command. >> Before running the tr command, top showed as: >> 33019 root 20 0 17172Ki 5448Ki CPU24 24 0:00 = 0.05% top -HiSCazopid >> If you do not end up with top listed just after tr in top's output, >> then it will not be top's line that ends up partially overwritten. >> If you have wider windows, you may need more text in the tr quoted >> strings. >> In another experiment I inserted a large number of backspace = characters >> (control-H's) at the front of the first quoted string in the tr = command. >> The top output displayed: >> 0\\n5 ro1\\n 2\\93 3\\n12764\\n 25\\ni CP6\\n 97\\n:12 = 100.00\t0r \nHiS\\t1pid \\t2 \\t3 \\t4 \\t5 \\t6 \\t >> 33094 root 20 0 17172Ki 5488Ki CPU21 21 0:00 = 0.06% top -HiSCazopid >> In other words, backspace moved the cursor position back over prior >> fields on the line and then the later line content overwrote those >> fields instead of being after "tr" someplace (or truncated off). >> Note that part of "-HiSCazopid" shows up on both lines. The extra >> is from when top was running but tr had not started yet. top is >> not managing text replacement correctly for output characters that >> end up not being just "in" the next character-cell on the terminal. >> The same sort of result happens when instead adding just one >> carriage return (control-M) in front of that first quuoted >> string instead: >> 0\\n8 ro1\\n 2\\92 3\\n12764\\n 25\\ni CP6\\n 117\\n:11 = 100.00\t0r \nHiS\\t1pid \\t2 \\t3 \\t4 \\t5 \\t6 \\t >> 33094 root 20 0 17172Ki 5488Ki CPU23 23 0:00 = 0.04% top -HiSCazopid >> I do not intend to try to find all examples of characters that >> cause problems but used to not cause problems. >> =46rom what I've seen, cursor positioning escape character sequences >> seem to be sent through and cause overwrites at arbitrary places >> on screen, based on the escape sequence content. There are command >> lines around that contain such sequences. So I sometimes see the >> first few lines of top's output have garbage text from commands >> that were listed below at some point overwriting the top text. >> Part of what is going on is top avoiding rewriting characters >> that its tracking indicates have not been updated. When the >> actual display and that supposed-tracking mismatch, the >> display ends up wrong when updated (bad text continues to >> display). >> The text in commands should not make "top -a" output mess up >> the display of other lines in top's output, nor of other >> top output fields on the same line. In my view, if some usage >> contexts need otherwise, it should take an extra command line >> option to put top in a mode that might do such things. The >> default behavior should strictly avoid having such things >> happen. >=20 > Thanks. >=20 > The attached diff seems to take care of the issue for me, adding = VIS_TAB and removing VIS_SAFE, which can be blamed for passing through = the following: >=20 > VIS_SAFE Currently this form allows space, tab, newline, backspace, > bell, and return =E2=80=94 in addition to all graphic = characters =E2=80=94 > unencoded. > A quick test suggests agreement. We will see how it looks for on-going use. But I'll note that top's man page should document the translations that are being used: it is not the same text that top produced before -r352558 and one should be able to read the man page to find out how to interpret what top reports for the likes of top -a . (It does not appear that escape sequences or vertical tab would have gone through unencoded. So I'm still unclear how I ever had the top few lines of top's output messed up by command text. So it is also unclear that this change would make a difference for such. We will see over time if that text is ever messed up.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)