From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 00:44:35 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 17C044AC for ; Sun, 3 Feb 2013 00:44:35 +0000 (UTC) (envelope-from mjacob@freebsd.org) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 738B6D7B for ; Sun, 3 Feb 2013 00:44:24 +0000 (UTC) Received: from [192.168.135.7] (quaver.net [76.14.49.207]) (authenticated bits=0) by ns1.feral.com (8.14.5/8.14.4) with ESMTP id r130i3g6018401 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Sat, 2 Feb 2013 16:44:03 -0800 (PST) (envelope-from mjacob@freebsd.org) Message-ID: <510DB2CF.30605@freebsd.org> Date: Sat, 02 Feb 2013 16:43:59 -0800 From: Matthew Jacob Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: Physbio changes final call for tests and reviews References: <20130202163322.GA2522@kib.kiev.ua> <20130202214709.GA99418@alchemy.franken.de> In-Reply-To: <20130202214709.GA99418@alchemy.franken.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (ns1.feral.com [192.67.166.1]); Sat, 02 Feb 2013 16:44:03 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: mjacob@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 00:44:35 -0000 On 2/2/2013 1:47 PM, Marius Strobl wrote: > > One note: mjacob@ probably will be annoyed if you don't wrap the > changes to isp(4) in __FreeBSD_version so the same source still > compiles on older ones. > Thanks Marius, but I actually don't care about this at this point. From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 10:04:48 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id AEF696FD; Sun, 3 Feb 2013 10:04:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BDE6EFD9; Sun, 3 Feb 2013 10:04:47 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA06990; Sun, 03 Feb 2013 12:04:44 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U1wRI-0006Aj-15; Sun, 03 Feb 2013 12:04:44 +0200 Message-ID: <510E363B.9050207@FreeBSD.org> Date: Sun, 03 Feb 2013 12:04:43 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Alan Cox Subject: Re: kva size on amd64 References: <507E7E59.8060201@FreeBSD.org> <51098743.2050603@FreeBSD.org> <510A2C09.6030709@FreeBSD.org> <510AB848.3010806@rice.edu> <510B8F2B.5070609@FreeBSD.org> <510D5C37.6000507@rice.edu> In-Reply-To: <510D5C37.6000507@rice.edu> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: alc@FreeBSD.org, Alan Cox , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 10:04:48 -0000 on 02/02/2013 20:34 Alan Cox said the following: > On 02/01/2013 03:47, Andriy Gapon wrote: >> Actually, I have been obsessed quite for some time with an idea of confining ZFS >> to its own submap. But ZFS does its allocations through malloc(9) and uma(9) >> (depending on configuration). It seemed like a bit of work to provide support >> for per-zone or per-tag submaps in uma and malloc. >> What is your opinion of this approach? > > I'm skeptical that it would accomplish anything. Specifically, I don't > think that it would have any impact on the fragmentation problem that we > have with ZFS. On amd64, with its direct map, all small allocations are > implemented by uma_small_alloc() and accessed through the direct map, > rather than coming from the kmem map. Outside of ZFS, large, multipage > allocations from the kmem map aren't that common. So, for all practical > purposes, ZFS has the kmem map to itself. I agree. My line of thinking was this. Allocate smaller kmem_map bu default (e.g. 1/3 of memory as it was before). If ZFS is not used, then that's it. If ZFS is used then it allocated an additional submap out of kernel_map. The submap would be sized based on configured or autoconfigured maximum size of ARC, e.g. 1.5 * arc_max_size. But that's probably more wasteful than needed for those who use ZFS. > While I'm here, I'll offer some other food for thought. In HEAD, we > have a new-ish function, vm_page_alloc_contig(), that can allocate > contiguous, unmapped physical pages either to an arbitrary vm object or > VM_ALLOC_NOOBJ, just like vm_page_alloc(). Moreover, just like > vm_page_alloc(), it honors the VM_ALLOC_{NORMAL,SYSTEM,INTERRUPT} > thresholds and wakes the page daemon when appropriate. > > Using this function, you could rewrite the multipage allocation code to > first attempt allocation through vm_page_alloc_contig() and then fall > back to the kmem map only if vm_page_alloc_contig() fails. This is a very interesting idea for systems with direct map! P.S. I hope I don't get an indigestion with all the food :-) -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 10:10:58 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1228E7F0 for ; Sun, 3 Feb 2013 10:10:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 38F4569 for ; Sun, 3 Feb 2013 10:10:56 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA07034; Sun, 03 Feb 2013 12:10:54 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U1wXG-0006BR-FM; Sun, 03 Feb 2013 12:10:54 +0200 Message-ID: <510E37AE.8060206@FreeBSD.org> Date: Sun, 03 Feb 2013 12:10:54 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: axe vm.max_wired References: <20120517055425.GA802@infradead.org> <4FC762DD.90101@FreeBSD.org> <4FC81D9C.2080801@FreeBSD.org> <4FC8E29F.2010806@shatow.net> <4FC95A10.7000806@freebsd.org> <4FC9F94B.8060708@FreeBSD.org> <51098977.4000603@FreeBSD.org> <20130131091853.GI2522@kib.kiev.ua> <510B7B92.4030804@FreeBSD.org> <20130202162509.GZ2522@kib.kiev.ua> In-Reply-To: <20130202162509.GZ2522@kib.kiev.ua> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 10:10:58 -0000 on 02/02/2013 18:25 Konstantin Belousov said the following: > On Fri, Feb 01, 2013 at 10:23:46AM +0200, Andriy Gapon wrote: >> So, I still think that vm.max_wired as it is used now is too arbitrary and too >> indiscriminate to be useful. > It is sized well to the default size of the buffer map, which takes 10% > of the physical RAM of the machine. Since buffers wiring the pages, be > it VMIO or malloc buffer, this leaves 20% for other things, like mbufs, > page tables and user wires. That's the attitude that I don't agree with. "We leave 20% for user wirings, that should be enough for everyone." If it's not enough then you must tune. I rather prefer an approach where we say this is the least amount of memory that should stay unwired. The rest is up to a user. >> There are other tools to limit page wiring by userland e.g. memlocked limit. > The memlock limit is per-process. It is completely useless as the safety > measure. There is also a limit on number of processes, etc. >> But, as I've said in the original email, I can agree with vm.max_wired >> usefulness if it is set to something more reasonable by default. >> IMO, it should not be a fixed percentage of available memory, it should be >> derived from other VM thresholds related to paging. > > Might be. Please provide a suggestion or better, a change. On my to do list. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 15:57:21 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BB57B35D; Sun, 3 Feb 2013 15:57:21 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id 4704ED9A; Sun, 3 Feb 2013 15:57:20 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.5/8.14.5/ALCHEMY.FRANKEN.DE) with ESMTP id r13FvIAr006234; Sun, 3 Feb 2013 16:57:18 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.5/8.14.5/Submit) id r13FvIvL006233; Sun, 3 Feb 2013 16:57:18 +0100 (CET) (envelope-from marius) Date: Sun, 3 Feb 2013 16:57:18 +0100 From: Marius Strobl To: Konstantin Belousov Subject: Re: Physbio changes final call for tests and reviews Message-ID: <20130203155718.GA6204@alchemy.franken.de> References: <20130202163322.GA2522@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130202163322.GA2522@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: powerpc@freebsd.org, mips@freebsd.org, current@freebsd.org, jeff@freebsd.org, ia64@freebsd.org, arch@freebsd.org, sparc64@freebsd.org, arm@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 15:57:21 -0000 On Sat, Feb 02, 2013 at 06:33:22PM +0200, Konstantin Belousov wrote: > Hi, > I finished the last (insignificant) missed bits in the Jeff' physbio > work. Now I am asking for the last round of testing and review, esp. for > the !x86 architectures. Another testing focus are the SCSI HBAs and RAID > controllers which drivers are changed by the patchset. Please do test > this before the patchset is committed into HEAD ! > > The plan is to commit the patch somewhere in two weeks from this moment. > The patch is required for the finalizing of the unmapped I/O work for UFS > I did in parallel, which I hope to finish shortly after the commit. > > Patch is available at http://people.freebsd.org/~kib/misc/physbio.5.diff > Once you bring in said UFS changes, will the use of bus_dmamap_load_ccb(9) be a requirement for disk controller drivers? Marius From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 16:11:56 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 94E856BB; Sun, 3 Feb 2013 16:11:56 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0CB75E0A; Sun, 3 Feb 2013 16:11:55 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r13GBjmC051486; Sun, 3 Feb 2013 18:11:45 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r13GBjmC051486 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r13GBjRR051485; Sun, 3 Feb 2013 18:11:45 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 3 Feb 2013 18:11:45 +0200 From: Konstantin Belousov To: Marius Strobl Subject: Re: Physbio changes final call for tests and reviews Message-ID: <20130203161145.GQ2522@kib.kiev.ua> References: <20130202163322.GA2522@kib.kiev.ua> <20130203155718.GA6204@alchemy.franken.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mi2hulEOATkNQ14l" Content-Disposition: inline In-Reply-To: <20130203155718.GA6204@alchemy.franken.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, jeff@freebsd.org, current@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 16:11:56 -0000 --mi2hulEOATkNQ14l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Feb 03, 2013 at 04:57:18PM +0100, Marius Strobl wrote: > On Sat, Feb 02, 2013 at 06:33:22PM +0200, Konstantin Belousov wrote: > > Hi, > > I finished the last (insignificant) missed bits in the Jeff' physbio > > work. Now I am asking for the last round of testing and review, esp. for > > the !x86 architectures. Another testing focus are the SCSI HBAs and RAID > > controllers which drivers are changed by the patchset. Please do test > > this before the patchset is committed into HEAD ! > >=20 > > The plan is to commit the patch somewhere in two weeks from this moment. > > The patch is required for the finalizing of the unmapped I/O work for U= FS > > I did in parallel, which I hope to finish shortly after the commit. > >=20 > > Patch is available at http://people.freebsd.org/~kib/misc/physbio.5.diff > >=20 >=20 > Once you bring in said UFS changes, will the use of bus_dmamap_load_ccb(9) > be a requirement for disk controller drivers? Generally speaking, no. I plan to do the gradual migration of the drivers, definitely not forcing the unmapped bios down to the drivers which are not tested yet. In the patch, driver indicates the support for unmapped bios by a DISKFLAG. If flag is set, driver could receive both mapped and unmapped bios, and use of the bus_dmamap_load_ccb(), while formally is only convenience, is essentially the requirement. If driver does not set the flag, it receives the same i/o requests as it does now. Geom performs transient compat mapping for the unmapped requests on its own for such drivers. As result, driver does not need a change. My plan is to convert ahci(4) and then some often used high-profile drivers like mfi(4) and mps(4). I can also hope for isci(4) help. Everything else, IMO, could be done on the best efforts basis, when both developers time and testing facilities are available. Jeff wanted to do all driver conversion in one pass, but IMO this is unrealistic. Still, I started write some helpers which should provide the transient one-page mappings for PIO modes. You can look at some previous version of the unmapped patch at http://people.freebsd.org/~kib/misc/unmapped.8.patch. It only contain a hack for ahci(4), which should be fixed properly after physbio is committed. --mi2hulEOATkNQ14l Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRDoxBAAoJEJDCuSvBvK1BqJQP/RpIphfsDzjo53QLX3+xnClm OB8WWZvZpxc3qOXPd5pKeOYggeE/6hkrUIL7AMlmg0qNKU7RXQmVJkcX+Yiuke5h oL1LSyIEBFuz8j7ks04H8ZOTjZpQQDeXYJPTrkNXcHqKWWZMMfsCZU1YNmzc0Shi m2ot3p7sL5gqUFaodC0qRejP5OOiMkdplwuSmyi3atoobrdMjqeadpDYCj6Ia9RU QBmWtr22Pj3AmVczK6L5jLk+uq7Al+ezwGEN4Bt5oT5B6pQMc8la6PyPNHVNdflA pW4qG0Pw4l6Gn/9ccpC001T8XJYq18j7U6yJIp1azoongTMmS1PFUssohSBtCwl0 q1xu3FTsh5U+M8z2HuxOyLiL1Eduf2uqFxnqPFuX0q0m9Wb3mCVauRd2sKkL8kvi WmW2x+PTuYiiFxU9MvtABZPWa8UVSpLJl30u2ptDheNb9vWmPBhfXps3bO8J0nN1 hIsdPJ+A0yevdqeLIDy6mdB7fUuEogHdCL9u9KtoOptpUPeiXpmUK7PsSyTz6G89 4Oz7jbSfmYGUY0rqiR+gfFZDa+4L8jYFYpxuneyGpTY77FvjCFrZ3efrJbbFj2r5 siu7GASJna3KgjQpj1G3UAU7j3dTuybRvBGd6/tKCbARSBGOcq2nYic3DpLVwJq9 kLGLoOEyKX9j+oJlWRtl =idda -----END PGP SIGNATURE----- --mi2hulEOATkNQ14l-- From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 16:46:26 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 75935D90; Sun, 3 Feb 2013 16:46:26 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id E129CFBB; Sun, 3 Feb 2013 16:46:25 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.5/8.14.5/ALCHEMY.FRANKEN.DE) with ESMTP id r13GkOjY006468; Sun, 3 Feb 2013 17:46:24 +0100 (CET) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.5/8.14.5/Submit) id r13GkOFw006467; Sun, 3 Feb 2013 17:46:24 +0100 (CET) (envelope-from marius) Date: Sun, 3 Feb 2013 17:46:24 +0100 From: Marius Strobl To: Konstantin Belousov Subject: Re: Physbio changes final call for tests and reviews Message-ID: <20130203164624.GL80850@alchemy.franken.de> References: <20130202163322.GA2522@kib.kiev.ua> <20130203155718.GA6204@alchemy.franken.de> <20130203161145.GQ2522@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130203161145.GQ2522@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org, jeff@freebsd.org, current@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 16:46:26 -0000 On Sun, Feb 03, 2013 at 06:11:45PM +0200, Konstantin Belousov wrote: > On Sun, Feb 03, 2013 at 04:57:18PM +0100, Marius Strobl wrote: > > On Sat, Feb 02, 2013 at 06:33:22PM +0200, Konstantin Belousov wrote: > > > Hi, > > > I finished the last (insignificant) missed bits in the Jeff' physbio > > > work. Now I am asking for the last round of testing and review, esp. for > > > the !x86 architectures. Another testing focus are the SCSI HBAs and RAID > > > controllers which drivers are changed by the patchset. Please do test > > > this before the patchset is committed into HEAD ! > > > > > > The plan is to commit the patch somewhere in two weeks from this moment. > > > The patch is required for the finalizing of the unmapped I/O work for UFS > > > I did in parallel, which I hope to finish shortly after the commit. > > > > > > Patch is available at http://people.freebsd.org/~kib/misc/physbio.5.diff > > > > > > > Once you bring in said UFS changes, will the use of bus_dmamap_load_ccb(9) > > be a requirement for disk controller drivers? > > Generally speaking, no. I plan to do the gradual migration of the drivers, > definitely not forcing the unmapped bios down to the drivers which are > not tested yet. In the patch, driver indicates the support for unmapped > bios by a DISKFLAG. If flag is set, driver could receive both mapped > and unmapped bios, and use of the bus_dmamap_load_ccb(), while formally > is only convenience, is essentially the requirement. > > If driver does not set the flag, it receives the same i/o requests as > it does now. Geom performs transient compat mapping for the unmapped > requests on its own for such drivers. As result, driver does not need > a change. > > My plan is to convert ahci(4) and then some often used high-profile drivers > like mfi(4) and mps(4). I can also hope for isci(4) help. > > Everything else, IMO, could be done on the best efforts basis, when both > developers time and testing facilities are available. Jeff wanted to do > all driver conversion in one pass, but IMO this is unrealistic. Still, I > started write some helpers which should provide the transient one-page > mappings for PIO modes. Okay > > You can look at some previous version of the unmapped patch at > http://people.freebsd.org/~kib/misc/unmapped.8.patch. It only contain a > hack for ahci(4), which should be fixed properly after physbio is committed. Hrm, the changes to the sparc64 pmap code in the latter patch might need some more attention as some of the functions used for copying pages there IIRC have constraints on the aligment of source and destination as well as on the count. Can you say something about these properties when pmap_copy_page_offs() is called via pmap_copy_pages()? Marius From owner-freebsd-arch@FreeBSD.ORG Sun Feb 3 17:17:59 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 25E009F6; Sun, 3 Feb 2013 17:17:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 863CA1A4; Sun, 3 Feb 2013 17:17:58 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r13HHhjL058355; Sun, 3 Feb 2013 19:17:43 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.4 kib.kiev.ua r13HHhjL058355 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r13HHh0O058354; Sun, 3 Feb 2013 19:17:43 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 3 Feb 2013 19:17:43 +0200 From: Konstantin Belousov To: Marius Strobl Subject: Re: Physbio changes final call for tests and reviews Message-ID: <20130203171743.GS2522@kib.kiev.ua> References: <20130202163322.GA2522@kib.kiev.ua> <20130203155718.GA6204@alchemy.franken.de> <20130203161145.GQ2522@kib.kiev.ua> <20130203164624.GL80850@alchemy.franken.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YyxSosoRaUW6PdRh" Content-Disposition: inline In-Reply-To: <20130203164624.GL80850@alchemy.franken.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org, jeff@freebsd.org, current@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2013 17:17:59 -0000 --YyxSosoRaUW6PdRh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Feb 03, 2013 at 05:46:24PM +0100, Marius Strobl wrote: > On Sun, Feb 03, 2013 at 06:11:45PM +0200, Konstantin Belousov wrote: > > On Sun, Feb 03, 2013 at 04:57:18PM +0100, Marius Strobl wrote: > > > On Sat, Feb 02, 2013 at 06:33:22PM +0200, Konstantin Belousov wrote: > > > > Hi, > > > > I finished the last (insignificant) missed bits in the Jeff' physbio > > > > work. Now I am asking for the last round of testing and review, esp= =2E for > > > > the !x86 architectures. Another testing focus are the SCSI HBAs and= RAID > > > > controllers which drivers are changed by the patchset. Please do te= st > > > > this before the patchset is committed into HEAD ! > > > >=20 > > > > The plan is to commit the patch somewhere in two weeks from this mo= ment. > > > > The patch is required for the finalizing of the unmapped I/O work f= or UFS > > > > I did in parallel, which I hope to finish shortly after the commit. > > > >=20 > > > > Patch is available at http://people.freebsd.org/~kib/misc/physbio.5= =2Ediff > > > >=20 > > >=20 > > > Once you bring in said UFS changes, will the use of bus_dmamap_load_c= cb(9) > > > be a requirement for disk controller drivers? > >=20 > > Generally speaking, no. I plan to do the gradual migration of the drive= rs, > > definitely not forcing the unmapped bios down to the drivers which are > > not tested yet. In the patch, driver indicates the support for unmapped > > bios by a DISKFLAG. If flag is set, driver could receive both mapped > > and unmapped bios, and use of the bus_dmamap_load_ccb(), while formally > > is only convenience, is essentially the requirement. > >=20 > > If driver does not set the flag, it receives the same i/o requests as > > it does now. Geom performs transient compat mapping for the unmapped > > requests on its own for such drivers. As result, driver does not need > > a change. > >=20 > > My plan is to convert ahci(4) and then some often used high-profile dri= vers > > like mfi(4) and mps(4). I can also hope for isci(4) help. > >=20 > > Everything else, IMO, could be done on the best efforts basis, when both > > developers time and testing facilities are available. Jeff wanted to do > > all driver conversion in one pass, but IMO this is unrealistic. Still, I > > started write some helpers which should provide the transient one-page > > mappings for PIO modes. >=20 > Okay >=20 > >=20 > > You can look at some previous version of the unmapped patch at > > http://people.freebsd.org/~kib/misc/unmapped.8.patch. It only contain a > > hack for ahci(4), which should be fixed properly after physbio is commi= tted. >=20 > Hrm, the changes to the sparc64 pmap code in the latter patch might > need some more attention as some of the functions used for copying > pages there IIRC have constraints on the aligment of source and > destination as well as on the count. Can you say something about > these properties when pmap_copy_page_offs() is called via > pmap_copy_pages()? There is no constraints on the offsets or length for the pmap_copy_pages() itself. As a consequence, the only (obvious) constraint is that cnt <=3D PAGE_SIZE for sparc64 pmap_copy_page_offs(). I will be glad to obtain the help, both as advise and patch, for any !x86 architecture. For x86 too. Yes, pmap_copy_pages() is the corner-stone of the data moves needed when operating on the unmapped buffers. Our current i/o path already converts user buffers into the page list, but it was enough to use uiomove_fromphys() so far, because buffers were always mapped. Now, unmapped buffers provide the other side of the copy with the page list, so I need operation like pmap_copy_pages(). --YyxSosoRaUW6PdRh Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRDpu2AAoJEJDCuSvBvK1By24P/Rp+VIMl++pMjg6C7/A5ed0I AIZ70QQEPIekITd3AOj1UekTLUsYdGCoHT4ruSpJ3PNXIGyUqzUzVnsjR5sMHVzV 4PlHDEW6TFwm9MLs7mLio5hgQ7V0vj1yJMxhPyMhAD8lPk3sr40wUKIJM5VbT7B2 tli+sFag/J9Gq3kJ2LAROKaoYYD8dBczvP65xZgdQR/B2tWzdaUHQevaowGd8QNS WfuafBivVMKy5PR6zRDzWJF8EuDedlwFNfnbQvCns/7OH65zJuEFql1P2P6Iir8U vHWQRgo/Av8aIgTRIfptxQwCLDtlOzJKxKvgaYAQMDVF75oQhhoKRk5Zm4kfvZvR bHszvl8qIx9jilKdmxdwK86kBSjxpg6Yr5DfeXklYvLL5kycrs60mYWvXwY/OK8A HAXoXi1hwH8hka4G0lfHpCSKrHg7Q47fISwCUTkBa+sWpsnXk9VUipuSJb9tWpzd +5h95ffeil4F56gKAHk1CC+rykN1kNuSbvoZpVRQCe/E467YjqXDJWV29c8IPoMs dUY/p62IBSs8sZu3M62O0582CYzP2gto+zdQYgA9516YeJZYSku7DXbO15NmdlCG wWzOIZ6nsIdUOt50/nmXgWKuqj0V3wh+tjNR8wz7P1uG0Kldrfy1QjFBdmGE+n2P na1lt2H31/EYW8JmRgMy =9bnf -----END PGP SIGNATURE----- --YyxSosoRaUW6PdRh-- From owner-freebsd-arch@FreeBSD.ORG Thu Feb 7 21:37:22 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 41872283 for ; Thu, 7 Feb 2013 21:37:22 +0000 (UTC) (envelope-from christoph.mallon@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) by mx1.freebsd.org (Postfix) with ESMTP id B32A068 for ; Thu, 7 Feb 2013 21:37:21 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.28]) by mrigmx.server.lan (mrigmx002) with ESMTP (Nemesis) id 0MSomp-1UTEvG0sy5-00Ro6f for ; Thu, 07 Feb 2013 22:37:15 +0100 Received: (qmail invoked by alias); 07 Feb 2013 21:37:11 -0000 Received: from p5B132387.dip.t-dialin.net (EHLO rotluchs.lokal) [91.19.35.135] by mail.gmx.net (mp028) with SMTP; 07 Feb 2013 22:37:11 +0100 X-Authenticated: #1673122 X-Provags-ID: V01U2FsdGVkX1+TnIskUuGTKJNO1g+tC3H5In4PzfgckMMzg8civT ToRsW4pnlne79t Message-ID: <51141E33.4080103@gmx.de> Date: Thu, 07 Feb 2013 22:35:47 +0100 From: Christoph Mallon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130129 Thunderbird/17.0.2 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Proposal: Unify printing the function name in panic messages() Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Kirk McKusick X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 21:37:22 -0000 Hi, currently the style for printing panic messages varies greatly. Here are some examples of the most common styles: panic("just a message"); panic("function_name: message"); panic("wrong_function_name: message"); panic("%s: message", __func__); Especially the third -- a wrong function name being shown -- is annoying and quite common. I propose a simple macro, which wraps panic() and ensures that the right name is shown alawys by using __func__. I talked with Kirk about this and he suggested, that the name should be toggleable by a global option. This way, some space can be saved on memory constrained targets, where the size of the kernel is relevant. As an additional benefit, this would shrink the size of the current kernel, because currently many places hardocde the name of the function in the string. I propose the following macro: /* * Panic and automatically prepend the name of the function to the panic * message. If NAMED_PANIC is not set, the name is omitted. */ #ifdef NAMED_PANIC # define PANIC(fmt, ...) panic("%s: " fmt, __func__, ##__VA_ARGS__) #else # define PANIC(fmt) panic(__VA_ARGS__) #endif Using a small awk script, I can automatically convert about 1.500 of the approximately 3.500 panic calls. These are all calls, which either use __func__ (or __FUNCTION__, a gcc extension with the same effect) or hardcode the /correct/ function name. The rest either uses subtly different styles, shows no function name or uses a wrong name. The other calls should then be converted successively. For patches, please see http://tron.homeunix.org/zeug/FreeBSD/panic/. Do you have suggestions to improve this proposal? Chrsopth From owner-freebsd-arch@FreeBSD.ORG Thu Feb 7 21:51:26 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0CE02645 for ; Thu, 7 Feb 2013 21:51:26 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe08.c2i.net [212.247.154.226]) by mx1.freebsd.org (Postfix) with ESMTP id 90C0F12A for ; Thu, 7 Feb 2013 21:51:25 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [176.74.213.204] (account mc467741@c2i.net HELO laptop015.hselasky.homeunix.org) by mailfe08.swip.net (CommuniGate Pro SMTP 5.4.4) with ESMTPA id 376947544; Thu, 07 Feb 2013 22:46:16 +0100 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Subject: Re: Proposal: Unify printing the function name in panic messages() Date: Thu, 7 Feb 2013 22:47:26 +0100 User-Agent: KMail/1.13.7 (FreeBSD/9.1-STABLE; KDE/4.8.4; amd64; ; ) References: <51141E33.4080103@gmx.de> In-Reply-To: <51141E33.4080103@gmx.de> X-Face: ?p&W)c(+80hU; '{.$5K+zq{oC6y| /D'an*6mw>j'f:eBsex\Gi, Cc: Kirk McKusick , Christoph Mallon X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 21:51:26 -0000 On Thursday 07 February 2013 22:35:47 Christoph Mallon wrote: > Hi, > Hi, > /* > * Panic and automatically prepend the name of the function to the panic > * message. If NAMED_PANIC is not set, the name is omitted. > */ > #ifdef NAMED_PANIC > # define PANIC(fmt, ...) panic("%s: " fmt, __func__, ##__VA_ARGS__) > #else > # define PANIC(fmt) panic(__VA_ARGS__) > #endif > You mean: #define PANIC(...) panic(__VA_ARGS__) > > Do you have suggestions to improve this proposal? Why not make two variants to avoid the #ifdef ? PANIC_NAMED(fmt, ...) and PANIC(...) --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Feb 7 22:12:13 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 89945988 for ; Thu, 7 Feb 2013 22:12:13 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A02911EC for ; Thu, 7 Feb 2013 22:12:12 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA03821; Fri, 08 Feb 2013 00:12:09 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U3ZhR-000OtD-4t; Fri, 08 Feb 2013 00:12:09 +0200 Message-ID: <511426B8.2070800@FreeBSD.org> Date: Fri, 08 Feb 2013 00:12:08 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Christoph Mallon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> In-Reply-To: <51141E33.4080103@gmx.de> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 22:12:13 -0000 on 07/02/2013 23:35 Christoph Mallon said the following: > panic("just a message"); This seems perfect. Panic without at least a stack trace or preferably a crash dump is practically useless in most cases. A stack trace already has all the function names. So a function name in a message seems to be redundant. JIMHO. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Feb 7 22:33:03 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B0BD6C20 for ; Thu, 7 Feb 2013 22:33:03 +0000 (UTC) (envelope-from christoph.mallon@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mx1.freebsd.org (Postfix) with ESMTP id 4EB29291 for ; Thu, 7 Feb 2013 22:33:03 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.33]) by mrigmx.server.lan (mrigmx001) with ESMTP (Nemesis) id 0MQ9JV-1U01Fd1x91-005JoG for ; Thu, 07 Feb 2013 23:33:02 +0100 Received: (qmail invoked by alias); 07 Feb 2013 22:32:57 -0000 Received: from p5B132387.dip.t-dialin.net (EHLO rotluchs.lokal) [91.19.35.135] by mail.gmx.net (mp033) with SMTP; 07 Feb 2013 23:32:57 +0100 X-Authenticated: #1673122 X-Provags-ID: V01U2FsdGVkX18AuFBDDFcAlE1LxpBL4rGuMgGc8cA1XyJF0PV5Yy ScTGBy4Gn3sfMB Message-ID: <51142B42.4020909@gmx.de> Date: Thu, 07 Feb 2013 23:31:30 +0100 From: Christoph Mallon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130129 Thunderbird/17.0.2 MIME-Version: 1.0 To: Hans Petter Selasky Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <201302072247.26167.hselasky@c2i.net> In-Reply-To: <201302072247.26167.hselasky@c2i.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Kirk McKusick , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 22:33:03 -0000 On 07.02.2013 22:47, Hans Petter Selasky wrote: > On Thursday 07 February 2013 22:35:47 Christoph Mallon wrote: >> /* >> * Panic and automatically prepend the name of the function to the panic >> * message. If NAMED_PANIC is not set, the name is omitted. >> */ >> #ifdef NAMED_PANIC >> # define PANIC(fmt, ...) panic("%s: " fmt, __func__, ##__VA_ARGS__) >> #else >> # define PANIC(fmt) panic(__VA_ARGS__) >> #endif >> > > You mean: > > #define PANIC(...) panic(__VA_ARGS__) Yes, stupid typo. >> Do you have suggestions to improve this proposal? > > Why not make two variants to avoid the #ifdef ? > > PANIC_NAMED(fmt, ...) > > and > > PANIC(...) NAMED_PANIC is intended to be a global option (think WITNESS or INVARIANTS), not to be set at before including systm.h. All panics should either show the function name or not. This way you can shrink the kernel a bit at the flip of a switch. Christoph From owner-freebsd-arch@FreeBSD.ORG Thu Feb 7 23:24:49 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C9A98913 for ; Thu, 7 Feb 2013 23:24:49 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (unknown [IPv6:2620:64:0:1:223:7dff:fea2:c8f2]) by mx1.freebsd.org (Postfix) with ESMTP id B3D6C6A4 for ; Thu, 7 Feb 2013 23:24:49 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id DF9C52AA35B for ; Thu, 7 Feb 2013 16:24:47 -0700 (MST) Received: by night.db.net (Postfix, from userid 1000) id 277521CCFC; Thu, 7 Feb 2013 18:23:52 -0500 (EST) Date: Thu, 7 Feb 2013 18:23:52 -0500 From: Diane Bruce To: freebsd-arch@freebsd.org Subject: group(5) Group Passwords do not work Message-ID: <20130207232352.GA51387@night.db.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2013 23:24:49 -0000 Hi, I've been looking at pw & friends for a while when this PR was brought to my attention. http://www.freebsd.org/cgi/query-pr.cgi?pr=docs/167741 Right now group passwords in /etc/group are marked with * I'm told some linux distributions are marking this as "NOTUSED" Clearly our man pages should either be changed to make it much more clear that this stuff does not work and will never work in FreeBSD or the code should be changed to make it work. ;) Mark Saad spent some time checking this. If it is stated it is never going to be made to work, by core or whatever, some of the code in libutil + pw can be simplified a bit. It was also suggested on IRC that it is also possible that some pam code does expect group passwords to work or at least passed through. How are we to proceed folks? - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-arch@FreeBSD.ORG Fri Feb 8 08:21:50 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BC038472 for ; Fri, 8 Feb 2013 08:21:50 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 688F3BE3 for ; Fri, 8 Feb 2013 08:21:50 +0000 (UTC) Received: from ds4.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id D3B9F6397; Fri, 8 Feb 2013 08:21:42 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 9FA1F802F; Fri, 8 Feb 2013 09:21:42 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Diane Bruce Subject: Re: group(5) Group Passwords do not work References: <20130207232352.GA51387@night.db.net> Date: Fri, 08 Feb 2013 09:21:42 +0100 In-Reply-To: <20130207232352.GA51387@night.db.net> (Diane Bruce's message of "Thu, 7 Feb 2013 18:23:52 -0500") Message-ID: <86a9rfkyg9.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 08:21:50 -0000 Diane Bruce writes: > It was also suggested on IRC that it is also possible that some pam > code does expect group passwords to work or at least passed through. No, who gave you that idea? The only places where gr_passwd is mentioned in head are: contrib/mtree/getid.c include/grp.h lib/libc/gen/getgrent.3 lib/libc/gen/getgrent.c lib/libutil/gr_util.c libexec/mknetid/parse_group.c share/man/man5/group.5 tools/regression/lib/libc/nss/test-getgr.c tools/regression/lib/libutil/test-grp.c usr.bin/getent/getent.c usr.bin/logins/logins.c usr.bin/newgrp/newgrp.c usr.sbin/makefs/getid.c usr.sbin/nscd/agents/group.c usr.sbin/pw/pw_group.c DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Feb 8 09:47:19 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1C06560F for ; Fri, 8 Feb 2013 09:47:19 +0000 (UTC) (envelope-from Devin.Teske@fisglobal.com) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) by mx1.freebsd.org (Postfix) with ESMTP id BEA648A6 for ; Fri, 8 Feb 2013 09:47:18 +0000 (UTC) Received: from smtp.fisglobal.com ([10.132.206.16]) by ltcfislmsgpa05.fnfis.com (8.14.5/8.14.5) with ESMTP id r189l6Om015750 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 8 Feb 2013 03:47:06 -0600 Received: from LTCFISWMSGMB21.FNFIS.com ([10.132.99.23]) by LTCFISWMSGHT05.FNFIS.com ([10.132.206.16]) with mapi id 14.02.0309.002; Fri, 8 Feb 2013 03:47:05 -0600 From: "Teske, Devin" To: Diane Bruce , "freebsd-arch@freebsd.org" Subject: RE: group(5) Group Passwords do not work Thread-Topic: group(5) Group Passwords do not work Thread-Index: AQHOBYpYe7CgT9S9K0SQjxU66bgzd5hvtJgD Date: Fri, 8 Feb 2013 09:47:04 +0000 Message-ID: <13CA24D6AB415D428143D44749F57D7201EA6244@ltcfiswmsgmb21> References: <20130207232352.GA51387@night.db.net> In-Reply-To: <20130207232352.GA51387@night.db.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.132.253.120] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327, 1.0.431, 0.0.0000 definitions=2013-02-08_05:2013-02-08,2013-02-08,1970-01-01 signatures=0 Cc: "Teske, Devin" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 09:47:19 -0000 On Thu, 7 Feb 2013, Diane Bruce wrote: > Hi, >=20 > I've been looking at pw & friends for a while when this PR > was brought to my attention. >=20 > http://www.freebsd.org/cgi/query-pr.cgi?pr=3Ddocs/167741 >=20 > Right now group passwords in /etc/group are marked with * > I'm told some linux distributions are marking this as "NOTUSED" > Clearly our man pages should either be changed to make it much more clear > that this stuff does not work and will never work in FreeBSD or the > code should be changed to make it work. ;) It secretly does work -- but only for those willing to take the plunge and: WARNING: Not recommended unless you *must* have this functionality... sudo chmod u+s /usr/bin/newgrp NOTE: Assuming /usr/bin/newgrp is already owned by root See newgrp(8) for additional details. > Mark Saad spent some time > checking this. If it is stated it is never going to be made to work, by c= ore > or whatever, some of the code in libutil + pw can be simplified a bit. newgrp(8) ships without the setuid root bit set for security reasons. It's = there to flip for anybody that needs it. Perhaps documentation should be up= dated to mention this. > It was also suggested on IRC that it is also possible that some pam > code does expect group passwords to work or at least passed through. >=20 Nope, not used by PAM. > How are we to proceed folks? I'd rather not see this functionality go away -- in my up-coming release of= bsdconfig(8) I have a module that supports nearly every aspect of pw(8) in= cluding managing group(5) passwords. I see in a later reply to this thread = by des that the list includes things besides newgrp(8) and pw(8) ... add bs= dconfig(8) to that list by way of pw(8) usage. --=20 Devin _____________ The information contained in this message is proprietary and/or confidentia= l. If you are not the intended recipient, please: (i) delete the message an= d all copies; (ii) do not disclose, distribute or use the message in any ma= nner; and (iii) notify the sender immediately. In addition, please be aware= that any message addressed to our domain is subject to archiving and revie= w by persons other than the intended recipient. Thank you. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 8 10:10:13 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C12BAB7B for ; Fri, 8 Feb 2013 10:10:13 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 7DA71999 for ; Fri, 8 Feb 2013 10:10:13 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id r18AA6D1096982 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 8 Feb 2013 11:10:11 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id r18AA6ZB096981 for freebsd-arch@freebsd.org; Fri, 8 Feb 2013 11:10:06 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Fri, 8 Feb 2013 11:10:06 +0100 From: Paul Schenkeveld To: freebsd-arch@freebsd.org Subject: Re: group(5) Group Passwords do not work Message-ID: <20130208101006.GA63171@psconsult.nl> References: <20130207232352.GA51387@night.db.net> <86a9rfkyg9.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86a9rfkyg9.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 10:10:13 -0000 On Fri, Feb 08, 2013 at 09:21:42AM +0100, Dag-Erling Smørgrav wrote: > Diane Bruce writes: > > It was also suggested on IRC that it is also possible that some pam > > code does expect group passwords to work or at least passed through. > > No, who gave you that idea? > > The only places where gr_passwd is mentioned in head are: > > contrib/mtree/getid.c > include/grp.h > lib/libc/gen/getgrent.3 > lib/libc/gen/getgrent.c > lib/libutil/gr_util.c > libexec/mknetid/parse_group.c > share/man/man5/group.5 > tools/regression/lib/libc/nss/test-getgr.c > tools/regression/lib/libutil/test-grp.c > usr.bin/getent/getent.c > usr.bin/logins/logins.c > usr.bin/newgrp/newgrp.c Newgrp still has the capability of letting non-root users change their group to any group that the user can supply a correct group passord for. To enable this capability do "chmod u+s /usr/bin/newgrp". I suppose this setuid bit was turned off by default long time ago as switching groups became more or less redundant when supplementary group ID's were introduced. I've used newgrp quite a lot on old System V UNIX machines back in the 1980's when supplementary group id's were not available. So it's incorrect to sat that the password field in /etc/group does not work and cannot work in FreeBSD. After all /etc/group is just a database containing group records and it's up to programs to decide what to do with that data. AFAIK this field is also part of the Posix standard so it should stay there and the functionality in pw(8) and other programs still matters if you happen to run software on your system that uses this piece of data. > usr.sbin/makefs/getid.c > usr.sbin/nscd/agents/group.c > usr.sbin/pw/pw_group.c HTH, Paul Schenkeveld From owner-freebsd-arch@FreeBSD.ORG Fri Feb 8 13:39:38 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D550BD1C for ; Fri, 8 Feb 2013 13:39:38 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id BD55E71B for ; Fri, 8 Feb 2013 13:39:38 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 77E182AA4F2; Fri, 8 Feb 2013 06:39:29 -0700 (MST) Received: by night.db.net (Postfix, from userid 1000) id 0787B1CCFC; Fri, 8 Feb 2013 08:38:34 -0500 (EST) Date: Fri, 8 Feb 2013 08:38:34 -0500 From: Diane Bruce To: Dag-Erling Sm??rgrav Subject: Re: group(5) Group Passwords do not work Message-ID: <20130208133833.GA62849@night.db.net> References: <20130207232352.GA51387@night.db.net> <86a9rfkyg9.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86a9rfkyg9.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Diane Bruce , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 13:39:38 -0000 On Fri, Feb 08, 2013 at 09:21:42AM +0100, Dag-Erling Sm??rgrav wrote: > Diane Bruce writes: > > It was also suggested on IRC that it is also possible that some pam > > code does expect group passwords to work or at least passed through. > > No, who gave you that idea? Devin on IRC and I see he has replied as well. - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-arch@FreeBSD.ORG Fri Feb 8 13:48:15 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0AF0CF1C for ; Fri, 8 Feb 2013 13:48:15 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id DDAFE776 for ; Fri, 8 Feb 2013 13:48:14 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id DEAC72AA5BB; Fri, 8 Feb 2013 06:48:13 -0700 (MST) Received: by night.db.net (Postfix, from userid 1000) id 688BA1CCFC; Fri, 8 Feb 2013 08:47:18 -0500 (EST) Date: Fri, 8 Feb 2013 08:47:18 -0500 From: Diane Bruce To: "Teske, Devin" Subject: Re: group(5) Group Passwords do not work Message-ID: <20130208134718.GB62849@night.db.net> References: <20130207232352.GA51387@night.db.net> <13CA24D6AB415D428143D44749F57D7201EA6244@ltcfiswmsgmb21> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <13CA24D6AB415D428143D44749F57D7201EA6244@ltcfiswmsgmb21> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Diane Bruce , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 13:48:15 -0000 On Fri, Feb 08, 2013 at 09:47:04AM +0000, Teske, Devin wrote: > On Thu, 7 Feb 2013, Diane Bruce wrote: > ... > > It secretly does work -- but only for those willing to take the plunge and: > > WARNING: Not recommended unless you *must* have this functionality... > > sudo chmod u+s /usr/bin/newgrp > > NOTE: Assuming /usr/bin/newgrp is already owned by root > > See newgrp(8) for additional details. Indeed it will work if it is properly setuid root. The question was whether we should further deprecate it or document it. ;) > > Mark Saad spent some time > > checking this. If it is stated it is never going to be made to work, by core > > or whatever, some of the code in libutil + pw can be simplified a bit. > > newgrp(8) ships without the setuid root bit set for security reasons. It's there to flip for anybody that needs it. Perhaps documentation should be updated to mention this. > Yes, that was the over all question. If we are shipping with it deliberately non setuid are we deprecating it with the aim of further removing it completely, which OpenBSD have already done BTW or are we going to document it. On an OpenBSD machine you will get: $ newgrp ksh: newgrp: not found > > > It was also suggested on IRC that it is also possible that some pam > > code does expect group passwords to work or at least passed through. > > > > Nope, not used by PAM. > > > > How are we to proceed folks? > > I'd rather not see this functionality go away -- in my up-coming release of bsdconfig(8) I have a module that supports nearly every aspect of pw(8) including managing group(5) passwords. I see in a later reply to this thread by des that the list includes things besides newgrp(8) and pw(8) ... add bsdconfig(8) to that list by way of pw(8) usage. > -- > Devin I'm finishing this incomplete reply to move to IRC for now ;) > > _____________ > The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 08:51:51 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B015DA74 for ; Sat, 9 Feb 2013 08:51:51 +0000 (UTC) (envelope-from christoph.mallon@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) by mx1.freebsd.org (Postfix) with ESMTP id 37A83EB8 for ; Sat, 9 Feb 2013 08:51:50 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.35]) by mrigmx.server.lan (mrigmx001) with ESMTP (Nemesis) id 0M38or-1UtioD0o47-00srxa for ; Sat, 09 Feb 2013 09:51:50 +0100 Received: (qmail invoked by alias); 09 Feb 2013 08:51:46 -0000 Received: from p5B132F8B.dip.t-dialin.net (EHLO rotluchs.lokal) [91.19.47.139] by mail.gmx.net (mp035) with SMTP; 09 Feb 2013 09:51:46 +0100 X-Authenticated: #1673122 X-Provags-ID: V01U2FsdGVkX1+XsO9v6R2XFN3z8J8TXtMYdJ94IeLHfxJ9EX48/d Z1zxNQSI/LH3N+ Message-ID: <51160E06.1070404@gmx.de> Date: Sat, 09 Feb 2013 09:51:18 +0100 From: Christoph Mallon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130129 Thunderbird/17.0.2 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> In-Reply-To: <511426B8.2070800@FreeBSD.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 08:51:51 -0000 On 07.02.2013 23:12, Andriy Gapon wrote: > on 07/02/2013 23:35 Christoph Mallon said the following: >> panic("just a message"); > > This seems perfect. > Panic without at least a stack trace or preferably a crash dump is practically > useless in most cases. A stack trace already has all the function names. > So a function name in a message seems to be redundant. This is nice in theory, but infeasible in practice. More than half of the panic() calls include the correct function name in a way or another. I estimate, that another quarter show a wrong name. It is hard to get this number mechanically, because, well, the names are wrong. So most calls include the name (or try to). Having no function name is a minority. I plan to correct and unify this hotchpotch. Also, if stack traces are disabled, you at least can reliably determine, where the panic came from. Finally, you can turn the names on and off with one central switch. Christoph From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 09:08:52 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DF748DA2 for ; Sat, 9 Feb 2013 09:08:52 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 17DC7F1A for ; Sat, 9 Feb 2013 09:08:51 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA19450; Sat, 09 Feb 2013 11:08:49 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U46QS-0007j7-SX; Sat, 09 Feb 2013 11:08:48 +0200 Message-ID: <5116121E.1010601@FreeBSD.org> Date: Sat, 09 Feb 2013 11:08:46 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Christoph Mallon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> In-Reply-To: <51160E06.1070404@gmx.de> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 09:08:52 -0000 on 09/02/2013 10:51 Christoph Mallon said the following: > On 07.02.2013 23:12, Andriy Gapon wrote: >> on 07/02/2013 23:35 Christoph Mallon said the following: >>> panic("just a message"); >> >> This seems perfect. >> Panic without at least a stack trace or preferably a crash dump is practically >> useless in most cases. A stack trace already has all the function names. >> So a function name in a message seems to be redundant. > > This is nice in theory, but infeasible in practice. > More than half of the panic() calls include the correct function name in a way or another. > I estimate, that another quarter show a wrong name. In any case, you just search the code for the message and that's it. I don't suppose that there are many duplicates there. > It is hard to get this number mechanically, because, well, the names are wrong. > So most calls include the name (or try to). > Having no function name is a minority. > I plan to correct and unify this hotchpotch. > Also, if stack traces are disabled, you at least can reliably determine, where the panic came from. Even if you can determine that, without more debugging data that would be useless in most cases. (We paniced because something went wrong, but why did it happen?) > Finally, you can turn the names on and off with one central switch. Well, have you experienced any problems with debugging due to those (absent/misleading) function names? Or do you see many reports about such problems? If this is a solution in search of a problem, then I don't like it, because it requires massive, if mostly mechanical, changes throughout the code. If panic message like ("invalid value of bar") were to be changed to something like ("invalid value of bar (%foo)", bar) I would find that to be a far more useful project. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 09:27:00 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6B83F343; Sat, 9 Feb 2013 09:27:00 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.freebsd.org (Postfix) with ESMTP id C817EFCE; Sat, 9 Feb 2013 09:26:58 +0000 (UTC) Received: from c211-30-173-106.carlnfd1.nsw.optusnet.com.au (c211-30-173-106.carlnfd1.nsw.optusnet.com.au [211.30.173.106]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id r199QlRN027425 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 9 Feb 2013 20:26:49 +1100 Date: Sat, 9 Feb 2013 20:26:47 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Christoph Mallon Subject: Re: Proposal: Unify printing the function name in panic messages() In-Reply-To: <51160E06.1070404@gmx.de> Message-ID: <20130209195417.C1753@besplex.bde.org> References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.0 cv=RbTIkCRv c=1 sm=1 a=38QaZPNW7E4A:10 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=eOSutZEXEbgA:10 a=QiFWfQZKQqc9mlzFMg0A:9 a=CjuIK1q_8ugA:10 a=Q472suVMmv298HXA:21 a=5NugUVhWCmp0UsIz:21 a=TEtd8y5WR3g2ypngnwZWYw==:117 Cc: Kirk McKusick , Andriy Gapon , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 09:27:00 -0000 On Sat, 9 Feb 2013, Christoph Mallon wrote: > On 07.02.2013 23:12, Andriy Gapon wrote: >> on 07/02/2013 23:35 Christoph Mallon said the following: >>> panic("just a message"); >> >> This seems perfect. >> Panic without at least a stack trace or preferably a crash dump is practically >> useless in most cases. A stack trace already has all the function names. >> So a function name in a message seems to be redundant. > > This is nice in theory, but infeasible in practice. This is feasible in practice, and needs less churn. panic() can edit out duplicates. The callers can be changed at leisure. > More than half of the panic() calls include the correct function name in a way or another. If the function name is correct and is normally formatted as a literal followed by a colon and a space at the start of the message, then it is easy to detect and edit out. If the function name is __func__ and is normally formatted with "%s: " at the start of the format, then it is only a little harder to detect and equally easy to edit out. > I estimate, that another quarter show a wrong name. > It is hard to get this number mechanically, because, well, the names are wrong. If the name is wrong, then it is hard to detect the "duplication", either at runtime or by reading the code. > So most calls include the name (or try to). > Having no function name is a minority. > I plan to correct and unify this hotchpotch. > Also, if stack traces are disabled, you at least can reliably determine, where the panic came from. The name should still be printed in the panic message (unless you turn this off), using the same name lookup as stack tracing uses. If stack traces are disabled, then you normally shouldn't disable names in the panic message. Disabling them would normally not disable the deduplication, but deduplication could be controlled by another runtime option. > Finally, you can turn the names on and off with one central switch. A runtime switch works much better: - no code bloat for function names in all cases - no need to recompile to change the option - more control at runtime. Name lookup from the return address is broken for inline functions and for other optimizations like tail calls, but much more is broken for stack traces. Bruce From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 09:28:53 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C040851E for ; Sat, 9 Feb 2013 09:28:53 +0000 (UTC) (envelope-from christoph.mallon@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) by mx1.freebsd.org (Postfix) with ESMTP id 6009961 for ; Sat, 9 Feb 2013 09:28:53 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.34]) by mrigmx.server.lan (mrigmx002) with ESMTP (Nemesis) id 0MCeiK-1UCODG1FGE-009SSI for ; Sat, 09 Feb 2013 10:28:52 +0100 Received: (qmail invoked by alias); 09 Feb 2013 09:28:45 -0000 Received: from p5B132F8B.dip.t-dialin.net (EHLO rotluchs.lokal) [91.19.47.139] by mail.gmx.net (mp034) with SMTP; 09 Feb 2013 10:28:45 +0100 X-Authenticated: #1673122 X-Provags-ID: V01U2FsdGVkX195co0O0NJ51ooiTqkzoA7ODFyrOKZZBPlKoUqFVV KUnnnGuYINQiH3 Message-ID: <511616AC.8080306@gmx.de> Date: Sat, 09 Feb 2013 10:28:12 +0100 From: Christoph Mallon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130129 Thunderbird/17.0.2 MIME-Version: 1.0 To: Andriy Gapon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> In-Reply-To: <5116121E.1010601@FreeBSD.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 09:28:53 -0000 On 09.02.2013 10:08, Andriy Gapon wrote: > In any case, you just search the code for the message and that's it. Often the messages contains parameters (%d, %s, ...) or are split into multiple lines to appease the ancient 80 columns god. These make it harder to grep. Having the /right/ name makes it easier to get to the right place. > If this is a solution in search of a problem, then I don't like it, because it The two problems this change solves are very simple: - There are needlessly about a dozen different ways used to add the function name into the panic message. This change unifies it. - All too often the name is wrong. This change gets it right every time without any manual and error-prone effort of somebody, who adds a use of PANIC(). > requires massive, if mostly mechanical, changes throughout the code. I do not understand, what the problem is. There are bugs and cumbersome code. This simple changes solves it. Christoph From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 09:38:21 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 868FE6B5; Sat, 9 Feb 2013 09:38:21 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 34641BC; Sat, 9 Feb 2013 09:38:20 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so2859736vba.27 for ; Sat, 09 Feb 2013 01:38:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=6VPB4hPPXSL5f19Ss1OFop55MRuiFsNYlyzoGJV+GOg=; b=1IQGGKGORRGI7y3HpOTVZsTssK4QVf46Pr6ASs2Z0vI1C+TdqSYGGWGf+r/elS53IL fiNRyiAnxH6F5TjTrml++HmtgiN2nxFfGUamGm8Dk3DScr3Z2LdM32UHJg1c6gF20Hp2 wMiMdrOWUzA+/53WtJ6+4Rj9Taaqzz3H7QqCMs+KOiVPHlgMv/TYVjGfd6lKozOzIbSF n2I185rMfZxppf147fejdHDHB8/Mmn2jJAQ/w1gzgmAW4DE0Aupmh55yJLTDMK96vUTD 6gTOU69bDGnflLFDjGLqlwzDJrQv9drl2X4k/q93plPcROBGKOpgPZKx5gzJ93UrCG7M z1lQ== MIME-Version: 1.0 X-Received: by 10.52.24.114 with SMTP id t18mr8968622vdf.62.1360402700231; Sat, 09 Feb 2013 01:38:20 -0800 (PST) Received: by 10.58.170.36 with HTTP; Sat, 9 Feb 2013 01:38:20 -0800 (PST) In-Reply-To: <5116121E.1010601@FreeBSD.org> References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> Date: Sat, 9 Feb 2013 01:38:20 -0800 Message-ID: Subject: Re: Proposal: Unify printing the function name in panic messages() From: Mehmet Erol Sanliturk To: Andriy Gapon Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Kirk McKusick , Christoph Mallon , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 09:38:21 -0000 On Sat, Feb 9, 2013 at 1:08 AM, Andriy Gapon wrote: > on 09/02/2013 10:51 Christoph Mallon said the following: > > On 07.02.2013 23:12, Andriy Gapon wrote: > >> on 07/02/2013 23:35 Christoph Mallon said the following: > >>> panic("just a message"); > >> > >> This seems perfect. > >> Panic without at least a stack trace or preferably a crash dump is > practically > >> useless in most cases. A stack trace already has all the function > names. > >> So a function name in a message seems to be redundant. > > > > This is nice in theory, but infeasible in practice. > > More than half of the panic() calls include the correct function name in > a way or another. > > I estimate, that another quarter show a wrong name. > > In any case, you just search the code for the message and that's it. > I don't suppose that there are many duplicates there. > > > It is hard to get this number mechanically, because, well, the names are > wrong. > > So most calls include the name (or try to). > > Having no function name is a minority. > > I plan to correct and unify this hotchpotch. > > Also, if stack traces are disabled, you at least can reliably determine, > where the panic came from. > > Even if you can determine that, without more debugging data that would be > useless in most cases. (We paniced because something went wrong, but why > did it > happen?) > > > Finally, you can turn the names on and off with one central switch. > > Well, have you experienced any problems with debugging due to those > (absent/misleading) function names? Or do you see many reports about such > problems? > > If this is a solution in search of a problem, then I don't like it, > because it > requires massive, if mostly mechanical, changes throughout the code. > > If panic message like ("invalid value of bar") were to be changed to > something > like ("invalid value of bar (%foo)", bar) I would find that to be a far > more > useful project. > > -- > Andriy Gapon > During 1980 years we were using a Burroughs 6700 main frame . Its Fortran compiler produced code was generating a call stack trace on run time errors . That facility was very useful during finding possible causes . That main frame was using ONLY TWO MEGA BYTES of main memory . Now I am using such a facility with G95 Fortran compiler by setting a compiler switch : It is giving a call stack listing with line numbers of calls when a run time error occurs with line number of error point . A similar structure may be used for FreeBSD parts generated during compilation when a compiler switch is set . Making such a setting as default may be more useful because it is unknown when a run time error occurs . By submitting such call stack lists to developers will make their problem solution tasks much more easier than the currently required much more conversations to identify the cause and call steps . Therefore , this is mainly a compilation problem which can be achieved with current structure of sources . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 09:51:50 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A262F9C3 for ; Sat, 9 Feb 2013 09:51:50 +0000 (UTC) (envelope-from christoph.mallon@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by mx1.freebsd.org (Postfix) with ESMTP id 1DF6C115 for ; Sat, 9 Feb 2013 09:51:49 +0000 (UTC) Received: from mailout-de.gmx.net ([10.1.76.2]) by mrigmx.server.lan (mrigmx002) with ESMTP (Nemesis) id 0M5rVP-1UtARp3vT7-00xpRv for ; Sat, 09 Feb 2013 10:51:49 +0100 Received: (qmail invoked by alias); 09 Feb 2013 09:51:44 -0000 Received: from p5B132F8B.dip.t-dialin.net (EHLO rotluchs.lokal) [91.19.47.139] by mail.gmx.net (mp002) with SMTP; 09 Feb 2013 10:51:44 +0100 X-Authenticated: #1673122 X-Provags-ID: V01U2FsdGVkX1/JCwUCz1XJ+0Zc1W02epD7JGBjMs/AHacNwbLtFr Cs6U+b+SWCkpfu Message-ID: <51161C10.9070002@gmx.de> Date: Sat, 09 Feb 2013 10:51:12 +0100 From: Christoph Mallon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130129 Thunderbird/17.0.2 MIME-Version: 1.0 To: Bruce Evans Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <20130209195417.C1753@besplex.bde.org> In-Reply-To: <20130209195417.C1753@besplex.bde.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Kirk McKusick , Andriy Gapon , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 09:51:50 -0000 On 09.02.2013 10:26, Bruce Evans wrote: I knew, this would happen, the moment I typed "ancient god". > Name lookup from the return address is broken for inline functions and > for other optimizations like tail calls, but much more is broken for > stack traces. Thank you for contributing to my case. As bde points out, optimisations interfere with location information gathered at runtime. Having the right name of the function in the panic, ensures that at least the starting point is right. My PANIC() macro guarantees exactly this. Christoph From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 10:19:19 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 057F4576 for ; Sat, 9 Feb 2013 10:19:19 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3109720A for ; Sat, 9 Feb 2013 10:19:17 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA19800; Sat, 09 Feb 2013 12:19:15 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U47Wd-0007nt-Jl; Sat, 09 Feb 2013 12:19:15 +0200 Message-ID: <511622A2.2090601@FreeBSD.org> Date: Sat, 09 Feb 2013 12:19:14 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Christoph Mallon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> <511616AC.8080306@gmx.de> In-Reply-To: <511616AC.8080306@gmx.de> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 10:19:19 -0000 on 09/02/2013 11:28 Christoph Mallon said the following: > On 09.02.2013 10:08, Andriy Gapon wrote: >> In any case, you just search the code for the message and that's it. > > Often the messages contains parameters (%d, %s, ...) or are split into multiple lines to appease the ancient 80 columns god. > These make it harder to grep. > Having the /right/ name makes it easier to get to the right place. Having right tools for the search does that too. And doesn't require any code churn. >> If this is a solution in search of a problem, then I don't like it, because it > > The two problems this change solves are very simple: > - There are needlessly about a dozen different ways used to add the function name into the panic message. > This change unifies it. > - All too often the name is wrong. > This change gets it right every time without any manual and error-prone effort of somebody, who adds a use of PANIC(). There is no need to have a function name in panic message. If it's present and if it's incorrect - these are very minor details. We are not talking about new code that prevents real bugs. >> requires massive, if mostly mechanical, changes throughout the code. > > I do not understand, what the problem is. > There are bugs and cumbersome code. > This simple changes solves it. You conveniently omitted some questions of mine. I'll reproduce them: Well, have you experienced any problems with debugging due to those (absent/misleading) function names? Or do you see many reports about such problems? So I conclude that this is indeed a solution in search of a problem. And that's exactly why i don't like it: - a lot of lines changed for no good reason - code looks uglier / more obfuscated due to macro(s) - no clear benefit, because there is no clear problem that needs solving -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 10:30:42 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 067E9778 for ; Sat, 9 Feb 2013 10:30:42 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4DE65256 for ; Sat, 9 Feb 2013 10:30:40 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA19877; Sat, 09 Feb 2013 12:30:20 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U47hM-0007on-Iy; Sat, 09 Feb 2013 12:30:20 +0200 Message-ID: <5116253B.6030201@FreeBSD.org> Date: Sat, 09 Feb 2013 12:30:19 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Christoph Mallon Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <20130209195417.C1753@besplex.bde.org> <51161C10.9070002@gmx.de> In-Reply-To: <51161C10.9070002@gmx.de> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Kirk McKusick , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 10:30:42 -0000 on 09/02/2013 11:51 Christoph Mallon said the following: > On 09.02.2013 10:26, Bruce Evans wrote: > > I knew, this would happen, the moment I typed "ancient god". > >> Name lookup from the return address is broken for inline functions and >> for other optimizations like tail calls, but much more is broken for >> stack traces. > > Thank you for contributing to my case. > > As bde points out, optimisations interfere with location information gathered at runtime. > Having the right name of the function in the panic, ensures that at least the starting point is right. > My PANIC() macro guarantees exactly this. Optimizations, inlining are completely irrelevant. If the inlined function is inlined in just one place, then there is no difference. If it is inlined in multiple places, then you would need to know where it is inlined and for that you know a stack trace. The function name alone just doesn't carry as much significance as you attribute to it. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 10:46:02 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 22232933; Sat, 9 Feb 2013 10:46:02 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-ie0-x22a.google.com (ie-in-x022a.1e100.net [IPv6:2607:f8b0:4001:c03::22a]) by mx1.freebsd.org (Postfix) with ESMTP id E19F42AC; Sat, 9 Feb 2013 10:46:01 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id c11so6085565ieb.15 for ; Sat, 09 Feb 2013 02:46:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=fJYDIzuDhWt7MVuZMkxJWgey77+gkZjVLGtfMCW6RrA=; b=nXmVyOMKIBphD30wYHkson6S/Iz2ocP/4E23T+UVtB1XDKuZDELiRMvCzGpEAVVtoZ 9OCRCELGIIvBXc55DTXWz1WaZOxyMuk0BpthoasMfLkVZ3dot4jGSetomMgYYvmd9f82 CSGVH5jKqXaNNLBRHNFunQd8eEEazCi7YdHICkjZvZ9MB02eP8OxEsMCglcQO2bJwHIm yaT3lLN1x7TTrdSO5kCa2VHM5oxmB2CveVfd7aqcH6hWqYQsBujSNY4EncaoFpkPqbIE NHFAojoMgSqYzRFeiGMLifu4RH6GcQCidXMpxpR5KVsbfc1G5hS3Fq8mv0lyi2w8r/pL ko2Q== MIME-Version: 1.0 X-Received: by 10.50.197.170 with SMTP id iv10mr4295325igc.62.1360406761620; Sat, 09 Feb 2013 02:46:01 -0800 (PST) Received: by 10.64.16.73 with HTTP; Sat, 9 Feb 2013 02:46:01 -0800 (PST) Received: by 10.64.16.73 with HTTP; Sat, 9 Feb 2013 02:46:01 -0800 (PST) In-Reply-To: <511622A2.2090601@FreeBSD.org> References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> <511616AC.8080306@gmx.de> <511622A2.2090601@FreeBSD.org> Date: Sat, 9 Feb 2013 10:46:01 +0000 Message-ID: Subject: Re: Proposal: Unify printing the function name in panic messages() From: Chris Rees To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Kirk McKusick , Christoph Mallon , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 10:46:02 -0000 On 9 Feb 2013 10:19, "Andriy Gapon" wrote: > > on 09/02/2013 11:28 Christoph Mallon said the following: > > On 09.02.2013 10:08, Andriy Gapon wrote: > >> In any case, you just search the code for the message and that's it. > > > > Often the messages contains parameters (%d, %s, ...) or are split into multiple lines to appease the ancient 80 columns god. > > These make it harder to grep. > > Having the /right/ name makes it easier to get to the right place. > > Having right tools for the search does that too. > And doesn't require any code churn. OK, which tool can one find a panic message split across lines in source code? I would find this very useful. Chris From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 15:27:24 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 796C21D8 for ; Sat, 9 Feb 2013 15:27:24 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C2A12F95 for ; Sat, 9 Feb 2013 15:27:23 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA21298; Sat, 09 Feb 2013 17:27:20 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1U4CKm-00085B-3e; Sat, 09 Feb 2013 17:27:20 +0200 Message-ID: <51166AD5.4090707@FreeBSD.org> Date: Sat, 09 Feb 2013 17:27:17 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130121 Thunderbird/17.0.2 MIME-Version: 1.0 To: Chris Rees Subject: Re: Proposal: Unify printing the function name in panic messages() References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> <511616AC.8080306@gmx.de> <511622A2.2090601@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 15:27:24 -0000 on 09/02/2013 12:46 Chris Rees said the following: > > On 9 Feb 2013 10:19, "Andriy Gapon" > > wrote: >> >> on 09/02/2013 11:28 Christoph Mallon said the following: >> > On 09.02.2013 10:08, Andriy Gapon wrote: >> >> In any case, you just search the code for the message and that's it. >> > >> > Often the messages contains parameters (%d, %s, ...) or are split into > multiple lines to appease the ancient 80 columns god. >> > These make it harder to grep. >> > Having the /right/ name makes it easier to get to the right place. >> >> Having right tools for the search does that too. >> And doesn't require any code churn. > > OK, which tool can one find a panic message split across lines in source code? > I would find this very useful. http://grok.x12.su/source/search?q=%22offset+below+first+LBA%22&project=freebsd Generally, I like opengrok very much. Pity that we don't have an official server. fxr.watson.org is great, but opengrok is superior to glimpse. E.g. try the same kind of search here: http://fxr.watson.org/fxr/search BTW, a local version (slower because no index is used): $ pcregrep -r -M 'offset\W*below\W*first\W*LBA' sys/ sys/geom/part/g_part.c: DPRINTF("partition %d has start offset below first " "LBA: %jd < %jd\n", e1->gpe_index, pcregrep comes from devel/pcre. P.S. my first instinct was to try git grep first, git grep seems to support perl regular expressions in general, but it looks like the port doesn't include the necessary support: fatal: cannot use Perl-compatible regexes when not compiled with USE_LIBPCRE P.P.S. I must say that I use textproc/glimpse (with index weekly updated via cron) and in 99% of cases glimpse 'something' immediately returns useful results. It's quite rare that I have to use other tools for searching the code. I use vim + ctags too :-) -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sat Feb 9 22:51:38 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E8A8E2AC; Sat, 9 Feb 2013 22:51:38 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id E2947C26; Sat, 9 Feb 2013 22:51:19 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.6/8.14.6) with ESMTP id r19MpDlt036260; Sat, 9 Feb 2013 15:51:13 -0700 (MST) (envelope-from ian@FreeBSD.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r19HFXrB037385; Sat, 9 Feb 2013 10:15:33 -0700 (MST) (envelope-from ian@FreeBSD.org) Subject: Re: Proposal: Unify printing the function name in panic messages() From: Ian Lepore To: Andriy Gapon In-Reply-To: <5116121E.1010601@FreeBSD.org> References: <51141E33.4080103@gmx.de> <511426B8.2070800@FreeBSD.org> <51160E06.1070404@gmx.de> <5116121E.1010601@FreeBSD.org> Content-Type: text/plain; charset="us-ascii" Date: Sat, 09 Feb 2013 10:15:33 -0700 Message-ID: <1360430133.4545.68.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Kirk McKusick , Christoph Mallon , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Feb 2013 22:51:39 -0000 On Sat, 2013-02-09 at 11:08 +0200, Andriy Gapon wrote: > on 09/02/2013 10:51 Christoph Mallon said the following: > > On 07.02.2013 23:12, Andriy Gapon wrote: > >> on 07/02/2013 23:35 Christoph Mallon said the following: > >>> panic("just a message"); > >> > >> This seems perfect. > >> Panic without at least a stack trace or preferably a crash dump is practically > >> useless in most cases. A stack trace already has all the function names. > >> So a function name in a message seems to be redundant. > > > > This is nice in theory, but infeasible in practice. > > More than half of the panic() calls include the correct function name in a way or another. > > I estimate, that another quarter show a wrong name. > > In any case, you just search the code for the message and that's it. > I don't suppose that there are many duplicates there. > Exactly. The use of sufficiently descriptive messages is all that's necessary for finding the source of a panic or other error. It has taken me over 30 years to come around to this way of thinking, but during the past 5 years of working in a shop with a large source base and no file/line numbers in the logging, I've seen the light. What I've found to be the far bigger modern problem is not finding the source of a given message in the code, but rather finding out WHO said it, given the growing ubiquity of multi-threaded code. -- Ian > > It is hard to get this number mechanically, because, well, the names are wrong. > > So most calls include the name (or try to). > > Having no function name is a minority. > > I plan to correct and unify this hotchpotch. > > Also, if stack traces are disabled, you at least can reliably determine, where the panic came from. > > Even if you can determine that, without more debugging data that would be > useless in most cases. (We paniced because something went wrong, but why did it > happen?) > > > Finally, you can turn the names on and off with one central switch. > > Well, have you experienced any problems with debugging due to those > (absent/misleading) function names? Or do you see many reports about such problems? > > If this is a solution in search of a problem, then I don't like it, because it > requires massive, if mostly mechanical, changes throughout the code. > > If panic message like ("invalid value of bar") were to be changed to something > like ("invalid value of bar (%foo)", bar) I would find that to be a far more > useful project. >