From owner-freebsd-arch@freebsd.org Sun Jul 7 13:04:52 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04E6715E4825 for ; Sun, 7 Jul 2019 13:04:52 +0000 (UTC) (envelope-from mike@karels.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 79C0D860E7 for ; Sun, 7 Jul 2019 13:04:51 +0000 (UTC) (envelope-from mike@karels.net) Received: by mailman.ysv.freebsd.org (Postfix) id 349A815E4823; Sun, 7 Jul 2019 13:04:51 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 120B215E4822 for ; Sun, 7 Jul 2019 13:04:51 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (mail.karels.net [216.160.39.52]) by mx1.freebsd.org (Postfix) with ESMTP id 4252C860E1 for ; Sun, 7 Jul 2019 13:04:42 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.15.2/8.15.2) with ESMTP id x67D4fm4060233; Sun, 7 Jul 2019 08:04:41 -0500 (CDT) (envelope-from mike@karels.net) Message-Id: <201907071304.x67D4fm4060233@mail.karels.net> To: arch@freebsd.org cc: brde@optusnet.com.au, alc@feeebsd.org From: Mike Karels Reply-to: mike@karels.net Subject: btoc()/ctob() (was Re: svn commit: r349791 - head/sys/vm) In-reply-to: Your message of Sun, 07 Jul 2019 03:48:54 +1000. <20190707023441.B2047@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <60189.1562504486.1@mail.karels.net> Content-Transfer-Encoding: quoted-printable Date: Sun, 07 Jul 2019 08:04:41 -0500 X-Rspamd-Queue-Id: 4252C860E1 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.51 / 15.00]; ARC_NA(0.00)[]; HAS_REPLYTO(0.00)[mike@karels.net]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.99)[0.993,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[karels.net]; AUTH_NA(1.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; NEURAL_SPAM_MEDIUM(0.80)[0.805,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mail.karels.net]; NEURAL_SPAM_LONG(0.74)[0.738,0]; R_SPF_NA(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:209, ipnet:216.160.0.0/15, country:US]; FREEMAIL_CC(0.00)[optusnet.com.au]; IP_SCORE(-0.01)[asn: 209(-0.02), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jul 2019 13:04:52 -0000 [I moved my reply/new question to arch@, and bcc'd src-committers@ et al, where the interaction started. Excuse the top post, but the previous message is included just for reference; it need not be included in any replies. Followups to arch@, please.] [Sorry if this topic has been covered before; just let me know. Neither my freebsd.org or Google searches located anything, although I didn't try all that hard.] I am astounded to hear that the btoc() macro still lives on, along with ctob() (bytes to clicks, and clicks to bytes). They don't seem to have suitable MI replacements (at least not in ). For those who don't know/remember: these macros were used to convert physical addresses to values for the segment base registers (PARs) on the PDP-11, where there was no actual "page size" analogous to newer hardware. (A click, the 'c' in btoc/ctob, was 64 bytes, the unit of scaling to keep physical addresses in the hardware registers fitting in 16 bits.) Since then, the macros hav= e been subverted to operate in pages, starting with the VAX in 32/V, rather than coming up with a new, more appropriate name. I am wondering whether it is finally time to deprecate PDP-11-based nomenclature. Clicks have not been a "thing" for many years; but pages are nearly universal (modulo superpages, etc). I see that many architectures have architecture-dependent xxx_btop() and xxx_ptob() macros, which are similar but annoyingly different in details like casts. Unlike btoc(), the xxx_btop() versions do not round. I wonde= r, though, whether it is time to hoist *_btop/*_ptob into a new architecture- independent replacement for btoc/ctob. btop() and ptob() come to mind, although I don't know what Linux (etc) does along these lines. I see just over 100 uses of btoc()/ctob() overall, but of course we would need to keep deprecated versions for a while. Thoughts on changing the KPI here? We'd have to agree on details like rounding and casting, but my first reaction would be no to both. Mike > Date: Sun, 7 Jul 2019 03:48:54 +1000 (EST) > From: Bruce Evans > X-X-Sender: bde@besplex.bde.org > To: Doug Moore > cc: src-committers@freebsd.org, svn-src-all@freebsd.org, > svn-src-head@freebsd.org > Subject: Re: svn commit: r349791 - head/sys/vm > In-Reply-To: <201907061555.x66FtGsg025314@repo.freebsd.org> > Message-ID: <20190707023441.B2047@besplex.bde.org> > References: <201907061555.x66FtGsg025314@repo.freebsd.org> > MIME-Version: 1.0 > Content-Type: TEXT/PLAIN; charset=3DUS-ASCII; format=3Dflowed > X-Optus-CM-Score: 0 > X-Optus-CM-Analysis: v=3D2.2 cv=3DP6RKvmIu c=3D1 sm=3D1 tr=3D0 cx=3Da_id= p_d > a=3DPalzARQSbocsUSjMRkwAPg=3D=3D:117 a=3DPalzARQSbocsUSjMRkwAPg=3D=3D:1= 7 > a=3DjpOVt7BSZ2e4Z31A5e1TngXxSK0=3D:19 a=3Dkj9zAlcOel0A:10 > a=3Dk1N1_3MYVckZGDfFxwYA:9 a=3DCjuIK1q_8ugA:10 > Precedence: bulk > X-Loop: FreeBSD.org > Sender: owner-src-committers@freebsd.org > List-Id: FreeBSD mail > X-Rspamd-Queue-Id: 158A46ECA9 > X-Spamd-Bar: ------ > Authentication-Results: mx1.freebsd.org > X-Spamd-Result: default: False [-6.98 / 15.00]; > NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; > NEURAL_HAM_SHORT(-0.98)[-0.982,0]; > REPLY(-4.00)[]; > NEURAL_HAM_LONG(-1.00)[-1.000,0] > X-UID: 364675 > Status: RO > X-Keywords: $NotJunk NonJunk = > Content-Length: 5846 > On Sat, 6 Jul 2019, Doug Moore wrote: > > Log: > > Fix style(9) violations involving division by PAGE_SIZE. > It is style violation to even use an explicit division by PAGE_SIZE > instead of the btoc() conversion macro (*). > > Modified: head/sys/vm/swap_pager.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > --- head/sys/vm/swap_pager.c Sat Jul 6 15:34:23 2019 (r349790) > > +++ head/sys/vm/swap_pager.c Sat Jul 6 15:55:16 2019 (r349791) > > @@ -523,7 +523,7 @@ swap_pager_swap_init(void) > > * but it isn't very efficient). > > * > > * The nsw_cluster_max is constrained by the bp->b_pages[] > > - * array (MAXPHYS/PAGE_SIZE) and our locally defined > > + * array MAXPHYS / PAGE_SIZE and our locally defined > > * MAX_PAGEOUT_CLUSTER. Also be aware that swap ops are > > * constrained by the swap device interleave stripe size. > > * > The macro is less readable in comments. > > @@ -538,7 +538,7 @@ swap_pager_swap_init(void) > > * have one NFS swap device due to the command/ack latency over NFS. > > * So it all works out pretty well. > > */ > > - nsw_cluster_max =3D min((MAXPHYS/PAGE_SIZE), MAX_PAGEOUT_CLUSTER); > > + nsw_cluster_max =3D min(MAXPHYS / PAGE_SIZE, MAX_PAGEOUT_CLUSTER); > > > > nsw_wcount_async =3D 4; > > nsw_wcount_async_max =3D nsw_wcount_async; > > > > Modified: head/sys/vm/vnode_pager.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > > --- head/sys/vm/vnode_pager.c Sat Jul 6 15:34:23 2019 (r349790) > > +++ head/sys/vm/vnode_pager.c Sat Jul 6 15:55:16 2019 (r349791) > > @@ -544,8 +544,8 @@ vnode_pager_addr(struct vnode *vp, vm_ooffset_t ad= dres > > *rtaddress +=3D voffset / DEV_BSIZE; > > ... > Using explicit division by DEV_BSIZE instead of the btodb() conversion > macro is another style bug. > (*) The macros use shifts while the divisions use division. The hard-co= ded > versions could use shifts too, so there is another set of style bugs fro= m > only using shifts sometimes. Shifts are faster if the type of the divid= end > is signed. > Oops. The macro also rounds up. So hard-coding the division is not jus= t > a style bug. It is wrong unless the dividend is a multiple of the page > size. This shouldn't be assumed for values like MAXBSIZE. > There are many more style bugs involving btoc(): > - 'c' in it means 'click', which might mean a virtual page size while > PAGE_SIZE might mean the physical page size. Or vice versa. dyson > retired from BSD not long after I asked him to clean this up 20+ > years ago. > - btoc() is the only clearly MI macro for this. The better-named macros > amd64_ptob() and i386_ptob() are of course MD. amd64_ptob() is never > used in sys/amd64. i386_ptob() is used 8 times in sys/i386 (all in > pmap.c), and 1 of these uses is in a macro which is used 22 times. > These uses give another set of style bugs. They are just obfuscation= s > if clicks are the same as pages, and probably incomplete otherwise. > However, it is correct for MD code to use physical pages unless it is > implementing virtual pages. > These macros don't round up, so they are not equivalent to btoc() eve= n > if the click size is PAGE_SIZE. > - there is also the better-named macro atop(). This doesn't round up. > I think it is supposed to be MI. It is replicated > for all arches. 'a' in it means 'address', which is less general tha= n > 'b' for 'byte, so it is a worse name than btop(). > - the macro with the best name btop() doesn't exist. > In the opposite direction, there are ctob(), {amd64,i386}_ptob(), > ptoa(), and direct multiplications by PAGE_SIZE and direct shifts by > PAGE_SHIFT. {amd64,i386}_ptob() is not used even on i386. This directi= on > is trickier since the (npages << PAGE_SHIFT) overflows if npages has the > correct type (int). The caller should convert npages to something like > vm_offset_t before using the macro and the need for this is less obvious > than for a direct expression. This is of course undocumented except by > the source code which shows: > - ctob(), ptoa() and i386_ptob() need conversion in the caller, but > amd64_ptob() converts to unsigned long in the macro. Since the last > macro is unused, the caller should convert in all used cases. Covers= ion > is most important on 64-bit arches. On i386, overflow occurs at 2G > starting with int npages but u_int npages works up to 4G which is > enough for i386 addresses but not for amd64 addresses or i386-PAE > byte counts. > I once sprinkled conversions in the macros. btodb() still uses my old > code for this, where the code is sophisticated so as to avoid using > the long long abomination, but which became mostly nonsense when daddr_t > was expanded from 32 bits to 64. Since this macro shifts right, > conversion is not really necessary, and conversion to daddr_t gives > much the same pessimations as conversion to the abomination. dbtodb() > simply converts to daddr_t. I forget if I wrote that. btoc() converts > to vm_offset_t, but since it shifts right conversion is not really > necessary. > Conversions in macros are a wrong way to do this. jake taught me this > in connection with sparc64 i386-PAE work. Conversion in the caller > allows the caller to control the type of the result and to not pessimize > by expanding the type. I think jake intentionally left out conversions > in the sparc64 macros but didn't change the x86 macros much. Other arch= es > use a random mixture. E.g., ptoa() converts to unsigned long on amd64, > arm64, riscv and sparc64 (so jake didn't change this?), to unsigned > on arm, and doesn't convert on i386, mips or powerpc. Always converting > to vm_offset_t would be reasonable, but gives namespace problems. It > is stupid that the MI uses vm_offset_t for btoc() where > conversion is not really needed, while is more careful > about namespace problems so it avoids using even u_long for ptoa(). > Bruce From owner-freebsd-arch@freebsd.org Wed Jul 10 18:14:42 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E667015DF260 for ; Wed, 10 Jul 2019 18:14:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 668466A7EF for ; Wed, 10 Jul 2019 18:14:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 21BB315DF252; Wed, 10 Jul 2019 18:14:41 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09E4B15DF251 for ; Wed, 10 Jul 2019 18:14:40 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EABE06A7EA for ; Wed, 10 Jul 2019 18:14:39 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-4.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 88018E453 for ; Wed, 10 Jul 2019 18:14:39 +0000 (UTC) (envelope-from jhb@FreeBSD.org) To: arch@FreeBSD.org From: John Baldwin Subject: expand_number(3) Openpgp: preference=signencrypt Autocrypt: addr=jhb@FreeBSD.org; keydata= mQGiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg7QeSm9obiBCYWxk d2luIDxqaGJARnJlZUJTRC5vcmc+iGAEExECACAFAkTQ+awCGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRBy3lIGd+N/BI6RAJ9S97fvbME+3hxzE3JUyUZ6vTewDACdE1stFuSfqMvM jomvZdYxIYyTUpC5Ag0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Dsgnr3 1AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh+Goj XlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cMSOrH YUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOFQVHO EVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq1tqz hltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZTwtX sNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m7Z16 4yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioIAjjH aIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbUKWwx Q4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjHuW+I SQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZNwwCf afMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: Date: Wed, 10 Jul 2019 11:14:35 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: EABE06A7EA X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.96)[-0.964,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2019 18:14:42 -0000 While working on some bhyve changes I happened to look at the source to expand_number(3) and noticed what I think are several bugs. I wrote some tests to validate that these bugs are present, but in some cases I'm not sure what the correct / desired behavior is? I posted a review with the tests and questions about what some of the behavior should be with folks who have worked on expand_number(3) previously, but haven't gotten any feedback. The URL to the review is below if other folks would like to offer feedback on some of my open questions such as should we support negative numbers, hex and octal bases, trailing garbage, etc.? https://reviews.freebsd.org/D20796 -- John Baldwin From owner-freebsd-arch@freebsd.org Wed Jul 10 21:29:31 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 860D015E3E42 for ; Wed, 10 Jul 2019 21:29:31 +0000 (UTC) (envelope-from phil@juniper.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A2FA672D62 for ; Wed, 10 Jul 2019 21:29:30 +0000 (UTC) (envelope-from phil@juniper.net) Received: by mailman.ysv.freebsd.org (Postfix) id 6648C15E3E40; Wed, 10 Jul 2019 21:29:30 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40FED15E3E3F for ; Wed, 10 Jul 2019 21:29:30 +0000 (UTC) (envelope-from phil@juniper.net) Received: from mx0b-00273201.pphosted.com (mx0b-00273201.pphosted.com [67.231.152.164]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A78D372D59; Wed, 10 Jul 2019 21:29:26 +0000 (UTC) (envelope-from phil@juniper.net) Received: from pps.filterd (m0108163.ppops.net [127.0.0.1]) by mx0b-00273201.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x6ALTOx5024142; Wed, 10 Jul 2019 14:29:24 -0700 Received: from nam04-sn1-obe.outbound.protection.outlook.com (mail-sn1nam04lp2057.outbound.protection.outlook.com [104.47.44.57]) by mx0b-00273201.pphosted.com with ESMTP id 2tnghdgxcg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 Jul 2019 14:29:24 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cfsnoSv9mxX37O/Chi/M0VCVEoyy4Q99k2CV2t8zvhD6jdLQAtsXnkEX2nWSbzl9FU0Oz6pQHLEfbM9xElJHAjMJ0TQCfHHxHIUCOZVJuZYNwabXdLM26dzxdhZXH7pT58ToJIbhtP1b3QnEN8nNicJq3W0L3Jfn1B04baUNIa8yD2Z8tHuuNSZ8taUgTSbTOvC5thzYG+25EWJ/K5NgaZA8PTrBSmxZe+ihwkPgZ1RBDelaNEItQVIasM5caQjj8EW3Vo8Atu9VmPQw8nCSHPz0AM53J90Fr3AxaSB4ohnkDxVrG38azwOeFOqQIAGEKZO5ptEmp/u7NBIX5TJdRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o+I/9DrbBZ4kaQHQWvt0g6QDK4FJycsGWHoKX7L7OeI=; b=lVreLNCgiGXvbKPWUBrV0bbQ7ydNAM9vxBeG7mt/g28JeQ26uP28XjgGK146Gl2F2NmyTvTqn8hs6VBPdn4B7jBUfrapuxsV0knUBomxggEUiraU8fMCm9AiCVNRZPBpQfUC7+HB/Hk5g1ti+OHz+H8o3m1Ag0Bq4lSnzJ5z35Cqz03/Wnt6jvYkecHyXGqwtB1BMMqW5F0yTwyDTo7jOvnckdeZKRe6mC1WR/iC/7WTsr3qHQcqKy4OiaV5tqgJFsDRZ0P5HO04arhz0Qnf3oHRD7CBLZBVWWBXsUxHH2rh6ozSe5m43KH5K6K0lIiWq7tCpcjAa0ACIuwbGI9JNw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=softfail (sender ip is 66.129.239.12) smtp.rcpttodomain=freebsd.org smtp.mailfrom=juniper.net;dmarc=fail (p=reject sp=quarantine pct=100) action=oreject header.from=juniper.net;dkim=none (message not signed);arc=none Received: from BYAPR05CA0014.namprd05.prod.outlook.com (2603:10b6:a03:c0::27) by BL0PR05MB5570.namprd05.prod.outlook.com (2603:10b6:208:6c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.7; Wed, 10 Jul 2019 21:29:21 +0000 Received: from BY2NAM05FT003.eop-nam05.prod.protection.outlook.com (2a01:111:f400:7e52::207) by BYAPR05CA0014.outlook.office365.com (2603:10b6:a03:c0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2073.6 via Frontend Transport; Wed, 10 Jul 2019 21:29:21 +0000 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.12 as permitted sender) Received: from P-EXFEND-EQX-01.jnpr.net (66.129.239.12) by BY2NAM05FT003.mail.protection.outlook.com (10.152.100.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2052.8 via Frontend Transport; Wed, 10 Jul 2019 21:29:20 +0000 Received: from P-EXBEND-EQX-01.jnpr.net (10.104.8.52) by P-EXFEND-EQX-01.jnpr.net (10.104.8.54) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 10 Jul 2019 14:29:20 -0700 Received: from p-mailhub01.juniper.net (10.104.20.6) by P-EXBEND-EQX-01.jnpr.net (10.104.8.52) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Wed, 10 Jul 2019 14:29:20 -0700 Received: from idle.juniper.net (idleski.juniper.net [172.25.5.10]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id x6ALTJYH031875; Wed, 10 Jul 2019 14:29:19 -0700 (envelope-from phil@juniper.net) Received: from [10.0.0.3] (localhost [127.0.0.1]) by idle.juniper.net (8.15.2/8.15.2) with ESMTP id x6ALSDSW009645; Wed, 10 Jul 2019 17:28:13 -0400 (EDT) (envelope-from phil@juniper.net) From: Phil Shafer To: John Baldwin CC: Subject: Re: expand_number(3) Date: Wed, 10 Jul 2019 17:29:18 -0400 X-Mailer: MailMate (1.12.5r5635) Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; format=flowed Content-Transfer-Encoding: quoted-printable X-EXCLAIMER-MD-CONFIG: e3cb0ff2-54e7-4646-8a04-0dae4ac7b136 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:66.129.239.12; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10019020)(4636009)(376002)(396003)(346002)(39860400002)(136003)(2980300002)(199004)(189003)(70206006)(70586007)(336012)(229853002)(14444005)(36756003)(6916009)(6306002)(53936002)(2906002)(50226002)(8746002)(26005)(6246003)(8936002)(33656002)(316002)(186003)(426003)(305945005)(8676002)(486006)(81166006)(86362001)(53546011)(966005)(81156014)(68736007)(446003)(67846002)(11346002)(126002)(47776003)(476003)(76506006)(2616005)(4326008)(7126003)(450100002)(5660300002)(356004)(51416003)(76176011)(50466002)(478600001)(69596002); DIR:OUT; SFP:1102; SCL:1; SRVR:BL0PR05MB5570; H:P-EXFEND-EQX-01.jnpr.net; FPR:; SPF:SoftFail; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 67689a77-3d40-4d73-0e35-08d7057dabdc X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(4710121)(4711121)(1401327)(2017052603328); SRVR:BL0PR05MB5570; X-MS-TrafficTypeDiagnostic: BL0PR05MB5570: X-MS-Exchange-PUrlCount: 2 X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-Forefront-PRVS: 0094E3478A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info: nCnlTOdwkte6U7XW263oZSTaxzSCzmntX4Pf/onRbQqKyj8H6lQsK8b36Q9mpHsXUdx3XwxHZwqiuDT+1+ENsLWrO512EXeT9DymXfFrF0u6RwHGngj4IFB8mI8Qx/t3tRuf9IT1h87TYenUxQ1m4L4i9JRaln0Fq2BfeyhvD2TCE2madCGvirkgEc7oajdx9i4l8u0rc20806p08aiP5eKjWKc0kxipIgrP8BsilIHDYmrJ3o6kBNtoKMWSz4yzltNiY1IWFR5C2PcTNUWjHsfRqSxniMcBKM+mgRKj6MhP2ht+x6ag9k7m1TPmkZxKKEgYGhECd6wuzNSP2ZVb+w7DLdG77xK19d8iBd5yjfcgioSaBHCqJHm2jv2eDTGztekG5/xaN9uajeBXM9ksQI/D6jdN/PBm5rqx+fXe7sw= X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2019 21:29:20.8370 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 67689a77-3d40-4d73-0e35-08d7057dabdc X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.12]; Helo=[P-EXFEND-EQX-01.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR05MB5570 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-07-10_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1907100248 X-Rspamd-Queue-Id: A78D372D59 X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.977,0] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jul 2019 21:29:31 -0000 Would a reasonable goal be to make expand_number and humanize_number = (with HN_IEC_PREFIXES) should be as compatible as possible? The API doesn't allow many humanize-style features, but having the = number->string->number work would be ideal so support for = HN_DECIMAL-style input should be added, so "1.1K" does the right thing = (thought it rounds to 1126). Lacking support for "suffix", trailing garbage should not be an error = (ala atoi), though I wish there was a parameter like strtol()'s endptr = to pass the next point to the character back in. If there was a suffix = parameter, the endptr would be skipped over it, if present. The expand_number API uses uint64_t where humanize_number uses signed = int64_t so the answer for negative numbers is DOA. Is it worth making a new expandize_number() function = (dehumanize_number?) that brings the API more into sync with = humanize_number? Thanks, Phil On 10 Jul 2019, at 14:14, John Baldwin wrote: > While working on some bhyve changes I happened to look at the source > to expand_number(3) and noticed what I think are several bugs. I > wrote some tests to validate that these bugs are present, but in some > cases I'm not sure what the correct / desired behavior is? I posted > a review with the tests and questions about what some of the behavior > should be with folks who have worked on expand_number(3) previously, > but haven't gotten any feedback. The URL to the review is below if > other folks would like to offer feedback on some of my open questions > such as should we support negative numbers, hex and octal bases, > trailing garbage, etc.? > > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__reviews.freebsd.= org_D20796&d=3DDwICAg&c=3DHAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=3D= And7spKEXmRNIrq8pYCiSg&m=3DEHG6EmocNCDS6VoOZVxzZoCq-BqbJjBpsC9kXhGFTd0&s=3D= bUQ8xvhg6vv95W5fHxswOrzDIeVxFsFJIa2582tjHgA&e=3D > > -- = > John Baldwin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__lists.freebsd.or= g_mailman_listinfo_freebsd-2Darch&d=3DDwICAg&c=3DHAkYuh63rsuhr6Scbfh0UjBX= eMK-ndb3voDTXcWzoCI&r=3DAnd7spKEXmRNIrq8pYCiSg&m=3DEHG6EmocNCDS6VoOZVxzZo= Cq-BqbJjBpsC9kXhGFTd0&s=3D_ChOm4nn2d9pTlDuA6I2EdYXc73LOiS_N4c0k0DOWPo&e=3D= > To unsubscribe, send any mail to = > "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@freebsd.org Thu Jul 11 17:34:07 2019 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48A5015D9933 for ; Thu, 11 Jul 2019 17:34:07 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CE46F896AE for ; Thu, 11 Jul 2019 17:34:06 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8ECBF15D9932; Thu, 11 Jul 2019 17:34:06 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 696BE15D9931 for ; Thu, 11 Jul 2019 17:34:06 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 090DD896AD; Thu, 11 Jul 2019 17:34:06 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-4.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 7D50B1891E; Thu, 11 Jul 2019 17:34:05 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: expand_number(3) To: Phil Shafer Cc: arch@FreeBSD.org References: From: John Baldwin Openpgp: preference=signencrypt Autocrypt: addr=jhb@FreeBSD.org; keydata= mQGiBETQ+XcRBADMFybiq69u+fJRy/0wzqTNS8jFfWaBTs5/OfcV7wWezVmf9sgwn8TW0Dk0 c9MBl0pz+H01dA2ZSGZ5fXlmFIsee1WEzqeJzpiwd/pejPgSzXB9ijbLHZ2/E0jhGBcVy5Yo /Tw5+U/+laeYKu2xb0XPvM0zMNls1ah5OnP9a6Ql6wCgupaoMySb7DXm2LHD1Z9jTsHcAQMD /1jzh2BoHriy/Q2s4KzzjVp/mQO5DSm2z14BvbQRcXU48oAosHA1u3Wrov6LfPY+0U1tG47X 1BGfnQH+rNAaH0livoSBQ0IPI/8WfIW7ub4qV6HYwWKVqkDkqwcpmGNDbz3gfaDht6nsie5Z pcuCcul4M9CW7Md6zzyvktjnbz61BADGDCopfZC4of0Z3Ka0u8Wik6UJOuqShBt1WcFS8ya1 oB4rc4tXfSHyMF63aPUBMxHR5DXeH+EO2edoSwViDMqWk1jTnYza51rbGY+pebLQOVOxAY7k do5Ordl3wklBPMVEPWoZ61SdbcjhHVwaC5zfiskcxj5wwXd2E9qYlBqRg7QeSm9obiBCYWxk d2luIDxqaGJARnJlZUJTRC5vcmc+iGAEExECACAFAkTQ+awCGwMGCwkIBwMCBBUCCAMEFgID AQIeAQIXgAAKCRBy3lIGd+N/BI6RAJ9S97fvbME+3hxzE3JUyUZ6vTewDACdE1stFuSfqMvM jomvZdYxIYyTUpC5Ag0ERND5ghAIAPwsO0B7BL+bz8sLlLoQktGxXwXQfS5cInvL17Dsgnr3 1AKa94j9EnXQyPEj7u0d+LmEe6CGEGDh1OcGFTMVrof2ZzkSy4+FkZwMKJpTiqeaShMh+Goj XlwIMDxyADYvBIg3eN5YdFKaPQpfgSqhT+7El7w+wSZZD8pPQuLAnie5iz9C8iKy4/cMSOrH YUK/tO+Nhw8Jjlw94Ik0T80iEhI2t+XBVjwdfjbq3HrJ0ehqdBwukyeJRYKmbn298KOFQVHO EVbHA4rF/37jzaMadK43FgJ0SAhPPF5l4l89z5oPu0b/+5e2inA3b8J3iGZxywjM+Csq1tqz hltEc7Q+E08AAwUIAL+15XH8bPbjNJdVyg2CMl10JNW2wWg2Q6qdljeaRqeR6zFus7EZTwtX sNzs5bP8y51PSUDJbeiy2RNCNKWFMndM22TZnk3GNG45nQd4OwYK0RZVrikalmJY5Q6m7Z16 4yrZgIXFdKj2t8F+x613/SJW1lIr9/bDp4U9tw0V1g3l2dFtD3p3ZrQ3hpoDtoK70ioIAjjH aIXIAcm3FGZFXy503DOA0KaTWwvOVdYCFLm3zWuSOmrX/GsEc7ovasOWwjPn878qVjbUKWwx Q4QkF4OhUV9zPtf9tDSAZ3x7QSwoKbCoRCZ/xbyTUPyQ1VvNy/mYrBcYlzHodsaqUDjHuW+I SQQYEQIACQUCRND5ggIbDAAKCRBy3lIGd+N/BCO8AJ9j1dWVQWxw/YdTbEyrRKOY8YZNwwCf afMAg8QvmOWnHx3wl8WslCaXaE8= Message-ID: <79c3ce4d-0f4b-37f3-9e66-58a71d67962c@FreeBSD.org> Date: Thu, 11 Jul 2019 10:34:01 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 090DD896AD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.94 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.94)[-0.939,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2019 17:34:07 -0000 On 7/10/19 2:29 PM, Phil Shafer wrote: > Would a reasonable goal be to make expand_number and humanize_number > (with HN_IEC_PREFIXES) should be as compatible as possible? In regards to IEC, I think mapping 'K' to 1000 by default would not be great, but we should support '2MiB' in addition to '2MB' probably. > The API doesn't allow many humanize-style features, but having the > number->string->number work would be ideal so support for > HN_DECIMAL-style input should be added, so "1.1K" does the right thing > (thought it rounds to 1126). Hmm, that seems harder to reason about, or maybe it requires parsing the original value as a float or double. I'm tempted to only support whole numbers to give more deterministic behavior such that if expand_number does support a value you will always get what you passed to humanize_number. > Lacking support for "suffix", trailing garbage should not be an error > (ala atoi), though I wish there was a parameter like strtol()'s endptr > to pass the next point to the character back in. If there was a suffix > parameter, the endptr would be skipped over it, if present. Yes, the API doesn't really make it easy to deal with trailing garbage as in some cases (e.g. bhyve's memory size) trailing garbage is an error and unless expand_number detects it I have no way of enforcing that. I don't really like atoi() for that reason even though it is convenient. I'm not sure of a good answer here. > The expand_number API uses uint64_t where humanize_number uses signed > int64_t so the answer for negative numbers is DOA. Except you can cast the result and it works (see the tests in the review). That said, I'm fine with that being the case, we should just pick one and document it as such. > Is it worth making a new expandize_number() function > (dehumanize_number?) that brings the API more into sync with > humanize_number? The main thing I wish was some way to reason about trailing garbage, but for now I'd just like to more clearly document how the API should work. > Thanks, > Phil > > > > On 10 Jul 2019, at 14:14, John Baldwin wrote: > >> While working on some bhyve changes I happened to look at the source >> to expand_number(3) and noticed what I think are several bugs. I >> wrote some tests to validate that these bugs are present, but in some >> cases I'm not sure what the correct / desired behavior is? I posted >> a review with the tests and questions about what some of the behavior >> should be with folks who have worked on expand_number(3) previously, >> but haven't gotten any feedback. The URL to the review is below if >> other folks would like to offer feedback on some of my open questions >> such as should we support negative numbers, hex and octal bases, >> trailing garbage, etc.? >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.freebsd.org_D20796&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=And7spKEXmRNIrq8pYCiSg&m=EHG6EmocNCDS6VoOZVxzZoCq-BqbJjBpsC9kXhGFTd0&s=bUQ8xvhg6vv95W5fHxswOrzDIeVxFsFJIa2582tjHgA&e= >> >> -- >> John Baldwin >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freebsd.org_mailman_listinfo_freebsd-2Darch&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=And7spKEXmRNIrq8pYCiSg&m=EHG6EmocNCDS6VoOZVxzZoCq-BqbJjBpsC9kXhGFTd0&s=_ChOm4nn2d9pTlDuA6I2EdYXc73LOiS_N4c0k0DOWPo&e= >> To unsubscribe, send any mail to >> "freebsd-arch-unsubscribe@freebsd.org" -- John Baldwin