From owner-freebsd-current@freebsd.org Fri Jul 10 22:25:14 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D93B37562E for ; Fri, 10 Jul 2020 22:25:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B3SKP0HRtz42Qr for ; Fri, 10 Jul 2020 22:25:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: ZXfl.Z0VM1kI2kNKVla1OuNYbW.yYnFI.ZgMDIN3xIHk1g8vSjdBHFLuzus4ysG kGS5v_82E9hdTATkXvq4Cr3bT.My_RAaNIYwlbgFfv7_.eTJ1iJp3CAOKPnpXWmEvxdfr5YFouM2 U8hcN1Hvd0xTOX.Q5nrXRpkY0Z4cR6lN5dYrFigmxLDrqLnBp4_XxpoW8OgrZmXysDd3ux9.p8C6 amuUD4vc8ykJSiiFlFanLeR6CEP1cqThTGB8tjJnGLlZoXJH8f03whQYWwZ9q8W.mQuyrj7k1xVI 4zaUMe9jKosZSC7wa3N0pjS8vw4dQNNm7lcIvOIlZIdXaZUNewX9jg0k0EWCd0.bUI8wMEq8Kz.t y3olNnvVnrHvZaarJnjyiPmsvLbugoETF2dRwW0htQiei68tZrkYbJDJsmE02FKLXKt6iFtQdmyL z5RJrGhBBmGLi.eGuIwYtfI79MJLib_ekabHmOM2WlhN51X7K67WgxNtuUDjmG07.KTbiFrh6fwd GMo0ha_iF8zmqgkdBVP2fpkxFGSHjg5dRoHYmx6NNyrDxTSTAZh6Xz7hmYN6qnUsGhGmronc8KUj LYADrimI315T3TSdz4C_CDgaZqeMukw36NE0gVIEM3DI69glEBLBRbz1EHqDP9wMFuNTvCA6aUuc rmdEmUj8tqTVXs7JtuEdYoxNL8_jvMPo5U.R4F8ca0McIxlFu8OoapirPOnWCvoyr9pTmG50tsy. M3UAt7tAOv_ZC_VOre.0IRLSssD2YFlj2zNpsGqm3l5zB.J8L6M8HF.F3Fevep_CrT_6iIVWxlW6 uGC2yPjloeslXqCX68xf9xrvFYdz8cEzf_zHaHWRG9rF10rHho5WCGEVv4Tbu2D1aCroYd.Mr.1A qp9B3TK3lzu7qepGhjmcn4_EzmpAdDBiP4uS5bu_GOlaferOIduAcxAQSZi9rgDMoHpJb64IizVw 3YBS1GJtz55FhyieJI4mEopT2e5U.32E1rWTwxS8TSuFfg3y37BKLvayE2jfHTuYJyTBgYJMcw0c uMGfw42KnZFNZbZOzN.avqKO_f3YnWlSATJhA5Vk7ty8D1Vc.qcAxXAx.DB9txi27NpqCvmYkqV3 7WwSI1MYnlq.ej3eXuNhVhpsol0JQqUHi0p8P2nKqP.M_BE9WnIAjo5ryNexDFGeB1I9sk09zJwZ OzFxtNHyiORud_xm55c5jQ_LUARExO3vU6P5n1e.cponPI5MKcVuE7Ben0.epS5D6EDYqOcJOn0i FU7R5OBQh4iPmFvJ5YOBdO_PW5UgX1r.yOhz5vFu8DnSvEP3wP05gEuMvnoAkIz5PUCr4R4zPI45 3ZZbsLYs9MtHnYc5VnuY6biw30C5dBzBcrPiWJztWYRbyOgqtzB3mm4fnPcv9UYF0Js0JMeIpJCk lq1O2YV0sLtjxdtZqyIGrcsb9hOUZyAtHG_UiSwjiFmdr_XPNMV_rhEA. Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Fri, 10 Jul 2020 22:25:11 +0000 Received: by smtp413.mail.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 0d4950d434f3f0c3d51e5f8df7815078; Fri, 10 Jul 2020 22:25:07 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: svn commit: r352558 - head/usr.bin/top From: Mark Millard In-Reply-To: Date: Fri, 10 Jul 2020 15:25:06 -0700 Cc: Steve Wills , "daichi@freebsd.org" , FreeBSD Current , Hiroki Sato Content-Transfer-Encoding: quoted-printable Message-Id: References: <1BDFB387-930D-4F4D-8729-A5850F1C15B9.ref@yahoo.com> <1BDFB387-930D-4F4D-8729-A5850F1C15B9@yahoo.com> <61107ecc-6f9b-a4db-7b1e-ec75f73939ee@FreeBSD.org> To: Yuri Pankov X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Rspamd-Queue-Id: 4B3SKP0HRtz42Qr X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.76 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.26)[-1.255]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.011]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jul 2020 22:25:14 -0000 On 2020-Jul-10, at 11:05, Yuri Pankov wrote: > Steve Wills wrote: >> On 11/28/19 4:08 PM, Mark Millard via svn-src-head wrote: >>>> Author: daichi >>>> Date: Fri Sep 20 17:37:23 2019 >>>> New Revision: 352558 >>>> URL: >>>> https://svnweb.freebsd.org/changeset/base/352558 >>>>=20 >>>>=20 >>>> Log: >>>> top(1): support multibyte characters in command names (ARGV = array) >>>> depending on locale. >>>> - add setlocale() >>>> - remove printable() function >>>> - add VIS_OCTAL and VIS_SAFE to the flag of strvisx() to = display >>>> non-printable characters that do not use C-style backslash = sequences >>>> in three digit octal sequence, or remove it >>>> This change allows multibyte characters to be displayed = according to >>>> locale. If it is recognized as a non-display character according = to the >>>> locale, it is displayed in three digit octal sequence. >>>>=20 >>>=20 >>> Initially picking on tab characters as an example of what is >>> probably a somewhat broader issue . . . >>>=20 >>> Ever since this change, characters like tabs that do not fit >>> in the next character cell when output, but for which they >>> are !isprintable(...), now mess up the top display. Again >>> using tab as an example: line wrapping from the text having >>> been shifted over by more than one character cell. top does >>> not track the line wrapping result in how it decides what >>> to output for the following display updates. >>>=20 >> Apologies for the way late reply here, but I just now bothered = tracking this down. This commit seems to be the cause of some corruption = I'm seeing in long running top(1) as well. As Mark mentions, if I use = "hh" it clears up. Should I open a bugzilla bug? I can share screenshots = of the corruption, such as: >> https://i.imgur.com/Xqlwf9h.png >> https://i.imgur.com/Jv0d5NU.png >=20 > Does removing VIS_SAFE fixes the issue for you? >=20 > As for original Mark's report (which I missed), removing isprintable() = doesn't look wrong as vis(3) should take of its functionality (and in = multibyte-aware way). vis (as used) and the old isprintable logic are not equivalent when multi-byte is not needed/involved. Otherwise I'd not have had anything to ever report. If vis can do what is needed, more work needed to be done when the change was made in order to avoid msesed up displays in single-byte contexts. > Also, is there an easy way to reproduce this? The following sort of command (the empty space inside quoted text are tab characters): # tr '0\n 1\n 2\n 3\n 4\n 5\n 6\n 7\n = 8\n' '\t0 \t1 \t2 \t3 \t4 \t5 \t6 \t7 = \t8' < /dev/zero > /dev/null causes my 200 character wide window running top to show: 32920 root 100 0 12764Ki 2420Ki CPU3 3 2:22 99.87% = tr 0\\n 1\\n 2\\n 3\\n 4\\n 5\\n 6\\n 7\\n 8\\n = \\t0 \\t1 \\t2 \\t3 \\t4 \\t5 \\t6 \\t733 \\t8 = 20 7172 5448Ki CPU23 23 0:00 0.04% top -HiSCazopid But that does not show where the lines wrap at the edges of the window, so breaking it up explicitly after the first "\" in \\7: 32920 root 100 0 12764Ki 2420Ki CPU3 3 2:22 99.87% = tr 0\\n 1\\n 2\\n 3\\n 4\\n 5\\n 6\\n 7\\n 8\\n = \\t0 \\t1 \\t2 \\t3 \\t4 \\t5 \\t6 \ \t733 \\t8 20 7172 5448Ki CPU23 23 0:00 0.04% = top -HiSCazopid Note how \n turned into \\n , taking an extra character for each \n . Similarly for \t vs. \\t . (Other examples do similarly.) The tab characters really do use more than one character cell on the display (sometimes). The text from the tr command ends up spread across 2 lines as things look like in the window where top is running. I ran top in another ssh session first and then the tr command. Before running the tr command, top showed as: 33019 root 20 0 17172Ki 5448Ki CPU24 24 0:00 0.05% = top -HiSCazopid If you do not end up with top listed just after tr in top's output, then it will not be top's line that ends up partially overwritten. If you have wider windows, you may need more text in the tr quoted strings. In another experiment I inserted a large number of backspace characters (control-H's) at the front of the first quoted string in the tr command. The top output displayed: 0\\n5 ro1\\n 2\\93 3\\n12764\\n 25\\ni CP6\\n 97\\n:12 = 100.00\t0r \nHiS\\t1pid \\t2 \\t3 \\t4 \\t5 \\t6 \\t 33094 root 20 0 17172Ki 5488Ki CPU21 21 0:00 0.06% = top -HiSCazopid In other words, backspace moved the cursor position back over prior fields on the line and then the later line content overwrote those fields instead of being after "tr" someplace (or truncated off). Note that part of "-HiSCazopid" shows up on both lines. The extra is from when top was running but tr had not started yet. top is not managing text replacement correctly for output characters that end up not being just "in" the next character-cell on the terminal. The same sort of result happens when instead adding just one carriage return (control-M) in front of that first quuoted string instead: 0\\n8 ro1\\n 2\\92 3\\n12764\\n 25\\ni CP6\\n 117\\n:11 = 100.00\t0r \nHiS\\t1pid \\t2 \\t3 \\t4 \\t5 \\t6 \\t 33094 root 20 0 17172Ki 5488Ki CPU23 23 0:00 0.04% = top -HiSCazopid I do not intend to try to find all examples of characters that cause problems but used to not cause problems. =46rom what I've seen, cursor positioning escape character sequences seem to be sent through and cause overwrites at arbitrary places on screen, based on the escape sequence content. There are command lines around that contain such sequences. So I sometimes see the first few lines of top's output have garbage text from commands that were listed below at some point overwriting the top text. Part of what is going on is top avoiding rewriting characters that its tracking indicates have not been updated. When the actual display and that supposed-tracking mismatch, the display ends up wrong when updated (bad text continues to display). The text in commands should not make "top -a" output mess up the display of other lines in top's output, nor of other top output fields on the same line. In my view, if some usage contexts need otherwise, it should take an extra command line option to put top in a mode that might do such things. The default behavior should strictly avoid having such things happen. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)