From owner-freebsd-hackers@freebsd.org Sun Nov 6 19:07:00 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6E3FC33B9F for ; Sun, 6 Nov 2016 19:07:00 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AE88674 for ; Sun, 6 Nov 2016 19:06:59 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190f-a3bff70000007cae-cd-581f7e1d73d4 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id B3.F4.31918.D1E7F185; Sun, 6 Nov 2016 14:01:50 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id uA6J1nEA010501; Sun, 6 Nov 2016 14:01:49 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uA6J1jrY010027 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Nov 2016 14:01:48 -0500 Date: Sun, 6 Nov 2016 13:01:45 -0600 From: Benjamin Kaduk To: Jonathan de Boyne Pollard Cc: FreeBSD Hackers , NetBSD Users Subject: Re: Improved manual page for ul(1) Message-ID: <20161106190144.GL91607@kduck.kaduk.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpgleLIzCtJLcpLzFFi42IR4hTV1pWrk48wWHOQyWL75n+MFufPtjBZ /FxyitmB2WPGp/ksHgsf9jB6XJ24jSmAOYrLJiU1J7MstUjfLoErY8nS36wFj5krZixWaWBs Zu5i5OSQEDCRmNzdxt7FyMUhJNDGJPFl1XsoZwOjxNWzu5ghnCtMElc3tzCBtLAIqEjcu7yZ DcRmE1CTeLy3mRXEFhHwkOj6vxVsLLNAksSjlr8sILawgLbE1K6VYHFeoHXfz5xhBLGFBOwl tj3oYoOIC0qcnPmEBaJXS+LGv5dAuziAbGmJ5f84QMKcAg4Sze/2gLWKCihLNMx4wDyBUWAW ku5ZSLpnIXQvYGRexSibklulm5uYmVOcmqxbnJyYl5dapGuil5tZopeaUrqJERy4kvw7GOc0 eB9iFOBgVOLhFbCRjxBiTSwrrsw9xCjJwaQkyvtsg0yEEF9SfkplRmJxRnxRaU5q8SFGCQ5m JRHeGVVA5bwpiZVVqUX5MClpDhYlcd7/bl/DhQTSE0tSs1NTC1KLYLIyHBxKErzGtUCNgkWp 6akVaZk5JQhpJg5OkOE8QMO7a0CGFxck5hZnpkPkTzHqcrzb/O4BkxBLXn5eqpQ47xuQIgGQ oozSPLg5oIQjkb2/5hWjONBbwry9IFU8wGQFN+kV0BImoCVuITIgS0oSEVJSDYxzd9hdv7Dm 3zYvo5rpZlsOZk90zkxl/HxqaeW50ifeSzodZh6otlgRJfVTwlvizvPsd+0m108wlFlLuHnx iN9tfX6zOexQl1nQhXkv/CdUaoZOij5/kD9st/GRbdufmN1Zx7lyxTaGx/e0og1WdZq8rl+3 iJsjWjEidrrzeqEKs14VCwYh0QYlluKMREMt5qLiRAD45aDUEwMAAA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Nov 2016 19:07:00 -0000 On Sat, Nov 05, 2016 at 11:05:07AM +0000, Jonathan de Boyne Pollard wrote: > So whither does one submit an improved manual page for ul(1) ? Here's > the source: > > * http://jdebp.eu./Proposals/ul.1 Somehow I do not think that a 1993 copyright by the Regents is accurate. I am unwilling to commit without some clarification of the copyright status and confirmation of license terms (noting that for new code, we prefer the two-clause BSD license at this point). -Ben From owner-freebsd-hackers@freebsd.org Mon Nov 7 10:19:06 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D881FC32E20 for ; Mon, 7 Nov 2016 10:19:06 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-1.server.virginmedia.net (know-smtprelay-omc-1.server.virginmedia.net [80.0.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 4EB199C9 for ; Mon, 7 Nov 2016 10:19:05 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.100] ([86.10.211.13]) by know-smtprelay-1-imp with bizsmtp id 4yHu1u0050HtmFq01yHuyi; Mon, 07 Nov 2016 10:17:54 +0000 X-Originating-IP: [86.10.211.13] X-Spam: 0 X-Authority: v=2.1 cv=beuWK77B c=1 sm=1 tr=0 a=SB7hr1IvJSWWr45F2gQiKw==:117 a=SB7hr1IvJSWWr45F2gQiKw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=N659UExz7-8A:10 a=2rVjqWD_AAAA:8 a=k1tVrXxULUTk8nKwLxsA:9 a=pILNOxqGKmIA:10 a=-FEs8UIgK8oA:10 a=NWVoK91CQyQA:10 a=ULaUcM2Ibn9MdPUUwucP:22 Subject: Re: Improved manual page for ul(1) To: FreeBSD Hackers , NetBSD Users References: <20161106190144.GL91607@kduck.kaduk.org> From: Jonathan de Boyne Pollard Cc: Benjamin Kaduk Message-ID: <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> Date: Mon, 7 Nov 2016 10:17:32 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20161106190144.GL91607@kduck.kaduk.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 10:19:06 -0000 Jonathan de Boyne Pollard: > So whither does one submit an improved manual page for ul(1) ? Here's the source: > > * http://jdebp.eu./Proposals/ul.1 Benjamin Kaduk: > Somehow I do not think that a 1993 copyright by the Regents is accurate. I am unwilling to commit without some clarification of the copyright status and confirmation of license terms (noting that for new code, we prefer the two-clause BSD license at this point). It's actually quite clear and straightforward. You are suggesting that the copyright declaration should be changed; that is not permitted, however. The copyright declaration for the UC Regents is entirely accurate for the (small amount of) existing manual text that's still there, and retaining it is mandated by the first of the very licence terms that you are referring to. I took the existing manual and expanded and corrected it, and the world has my work under the exact same licence that I had the original, so that the result is not entangled in BSD Licence Hell; hence the exact same licence remains there as-is. Really, the first reaction should rather be something like "My goodness! Are all of those bugs really there?" or perhaps "Does anything nowadays make any use of that wacky lpr feature?" or even "Given then that it can handle UTF-8 input and output, could ul be fixed to make all of groff's overstrikes work?". (-: I haven't found anything that makes use of the lpr feature. Presumably it dates from a time when ul was once used as a filter for printing. Back then there was an array of programs: collpr, colcrt, iul, ul, ulpr, ... And more interesting is whether less could be fixed to make groff's overstrikes work. Or groff fixed to not need overstrikes for things that can be done in Unicode ... From owner-freebsd-hackers@freebsd.org Tue Nov 8 00:29:29 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12529C358CA for ; Tue, 8 Nov 2016 00:29:29 +0000 (UTC) (envelope-from mark@heily.com) Received: from mail-qk0-x22d.google.com (mail-qk0-x22d.google.com [IPv6:2607:f8b0:400d:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B33ECC65 for ; Tue, 8 Nov 2016 00:29:28 +0000 (UTC) (envelope-from mark@heily.com) Received: by mail-qk0-x22d.google.com with SMTP id x190so196806445qkb.0 for ; Mon, 07 Nov 2016 16:29:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heily-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Tr0SIiPlu0JO8EqGM2+Va7hXqn+mN8XqvfQjFh2ELgY=; b=KEYIDE2AzD6JEolDUz2fy8gVRrw1V9aYdYAPyUZusHEvCB2bMpYyceCxCJEmv83HmO rXLBm1rvDhDpoUM2D7yVhzGmHbvVDLapoX8ivB7VuOMeA52Azr1kRI7KaOiY9q2yWhFp nkL0ztOe2haey3bMazaG68PTf8HRp20bH1h2Ad8x5/296WrFut0F8RFlgfKXNmaMqoMl k3VZ9PFITNVyZzC8GlNBlFLSJu3WLS+YL7/uKcFYsm9qRbNH0W+yY0Iyh/T51qufyPJ2 ZZjhF0xZya8Uf5nuJ7Yprk7aFFLcb5lx6dXkXZsdLjJiZrTvIKSooeUzwmoclIO5jJzk 1c4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Tr0SIiPlu0JO8EqGM2+Va7hXqn+mN8XqvfQjFh2ELgY=; b=UKOiGvKBOuc7s+QNowE+OBHVc0vQfkOgpA/TQPuL9yfjFeakHKyCrXVKET3Bi4evj5 nz4FQwPS8xZnLBXNhM39jzaFWWfO36/oLiU2RYuNRv5hKbOYNrSeauw9bZ4BY0mwsZfk HaBSnYzG5W5xovPwOdIQZb3yisthZOMpb9/9p1tEEke7wjYot1HisBYjSRYmvaKuei/W CGM43KW2bLSzbGaA9Gay5OW385A6Z0cKZMPd5V6lO/miMjclQeV74aXZxyi9ggWRPv1d hbolUjRm+3+GcY0kn3nkMT9WzPAVuNg1cETK1REzxSRhXUKl6O9mu0XcizWKTQXEQ+3Q 7pNQ== X-Gm-Message-State: ABUngvdepYx4IkWTtY+Nvt0mbMsawFZQd5EoV2yxICmaLh4V2Am4OfaXK907xVxaO1H6UrT6MpfzdPlUGowohw== X-Received: by 10.55.9.77 with SMTP id 74mr8676097qkj.225.1478564967958; Mon, 07 Nov 2016 16:29:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.150.39 with HTTP; Mon, 7 Nov 2016 16:29:27 -0800 (PST) X-Originating-IP: [71.70.175.250] In-Reply-To: References: From: Mark Heily Date: Mon, 7 Nov 2016 19:29:27 -0500 Message-ID: Subject: Re: Improved manual page for ul(1) To: Jonathan de Boyne Pollard Cc: FreeBSD Hackers , NetBSD Users Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 00:29:29 -0000 On Sat, Nov 5, 2016 at 7:05 AM, Jonathan de Boyne Pollard < J.deBoynePollard-newsgroups@ntlworld.com> wrote: > So whither does one submit an improved manual page for ul(1) ? Here's the > source: > > * http://jdebp.eu./Proposals/ul.1 > > The current FreeBSD/NetBSD manual page really doesn't describe what ul > does and is even somewhat misleading. So the aforelinked has some > improvements. > > I don't think putting this level of detail into the ul(1) manpage is the best approach. Would it be better to create a new manpage in section 5 to describe the TTY-37 control sequences in all their gory detail? You could call it tty37_control(5) or something similar. For comparison, there is a groff_out(5) manpage that you could use as an example. The benefit of a separate manpage for TTY-37 information is that any other commands that deal with these control sequences can have their manpages updated to refer to the tty37_control(5) manual. - Mark From owner-freebsd-hackers@freebsd.org Tue Nov 8 08:01:05 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FDE4C36FAB for ; Tue, 8 Nov 2016 08:01:05 +0000 (UTC) (envelope-from ap00@mail.ru) Received: from smtp27.mail.ru (smtp27.mail.ru [94.100.181.182]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E530F31F for ; Tue, 8 Nov 2016 08:01:04 +0000 (UTC) (envelope-from ap00@mail.ru) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:To:Message-ID:From:Date; bh=RXk02nWFVaKZz9fbDIVf2vn0db0BWGEd/zy5XM3iYm4=; b=MQDv1WTjxsUU2+JgA8cZ40sFUHz75/fP03t72WWN/zml4tJ9de2aa+kT36j7vRd3fJqGm1yp8cTr0kxOQx387PLKj9OIS5B7b5AYtepHzFRMheNr9EgviFBC6EQEY3tR+/aZg0nESPwGHOZ1H3FaHY3hryFuK1ACTnddgS+f6ts=; Received: from [91.190.121.202] (port=52518 helo=pstation) by smtp27.mail.ru with esmtpa (envelope-from ) id 1c41L1-0006B3-Fl for freebsd-hackers@freebsd.org; Tue, 08 Nov 2016 11:01:00 +0300 Date: Tue, 8 Nov 2016 11:00:56 +0300 From: Anthony Pankov X-Priority: 3 (Normal) Message-ID: <1644757548.20161108110056@mail.ru> To: freebsd-hackers@freebsd.org Subject: nss_ldap seems to not work MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Authentication-Results: smtp27.mail.ru; auth=pass smtp.auth=ap00@mail.ru smtp.mailfrom=ap00@mail.ru X-E1FCDC63: F586714A37AD46779D936A3B9A0429D3B7AB0E7134D2E1A8 X-E1FCDC64: CF9B41FAC5AE2F0DC3D9940DA68EDE6C395F995829CEF92E2F26E4F17A20ACD7DC3326C543D49E8D X-Mailru-Sender: 0489DF6C38DA5EE561E8A477868835DB969DAEDF2C146D3D062233983D541B49730FAF750FE6C77BA5D819213F947FA3 X-Mras: OK X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 08:01:05 -0000 Greetings. nss_ldap seems to not work correctly at least at FreeBSD 10.3. Two configurations 1. FreeBSD 9.2 2. FreeBSD 10.3 sharing nss_ldap settings and using the same LDAP tree (DIT) produce different results. At FreeBSD 10.3 nss_ldap can't enumerate supplementary user groups. Example: FreeBSD 9.2: # id user1 ... groups=basegroup,gr1,gr2,gr3 FreeBSD 10.3: # id user1 ... groups=basegroup The effect is inadequate result of initgroups() calling which lead to various side effects with permissions. P.S. Interesting fact. At FreeBSD 10.3 pw utility produce correct result: #pw usershow user1 ... groups=basegroup,gr1,gr2,gr3 -- Best regards, Anthony mailto:ap00@mail.ru From owner-freebsd-hackers@freebsd.org Tue Nov 8 09:18:19 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BC98C36D3D for ; Tue, 8 Nov 2016 09:18:19 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-1.server.virginmedia.net (know-smtprelay-omc-1.server.virginmedia.net [80.0.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 95CF0F1B for ; Tue, 8 Nov 2016 09:18:17 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.100] ([86.10.211.13]) by know-smtprelay-1-imp with bizsmtp id 5MJG1u0090HtmFq01MJGLk; Tue, 08 Nov 2016 09:18:16 +0000 X-Originating-IP: [86.10.211.13] X-Spam: 0 X-Authority: v=2.1 cv=beuWK77B c=1 sm=1 tr=0 a=SB7hr1IvJSWWr45F2gQiKw==:117 a=SB7hr1IvJSWWr45F2gQiKw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=N659UExz7-8A:10 a=2rVjqWD_AAAA:8 a=pMdQXerZ3W1E7PntnhwA:9 a=pILNOxqGKmIA:10 a=IJuWuSD6ecgA:10 a=-FEs8UIgK8oA:10 a=NWVoK91CQyQA:10 a=ULaUcM2Ibn9MdPUUwucP:22 Subject: Re: Improved manual page for ul(1) To: FreeBSD Hackers , NetBSD Users References: <20161106190144.GL91607@kduck.kaduk.org> <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> From: Jonathan de Boyne Pollard Message-ID: Date: Tue, 8 Nov 2016 09:17:53 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 09:18:19 -0000 Jonathan de Boyne Pollard: > And more interesting is whether less could be fixed to make groff's > overstrikes work. Or groff fixed to not need overstrikes for things > that can be done in Unicode ... > It turns out that this is the man command limiting things. Again. It's hardwiring -Tascii even if it detects a UTF-8 character locale. I'm experimenting with this preliminary patch, applied on top of the patch at http://jdebp.eu./Softwares/nosh/italics-in-manuals.html#FreeBSD . The preliminary results are encouraging, although I haven't yet tried an ISO8859-1 locale on a Latin-1 terminal. A UTF-8 locale produces Unicode bullets on a UTF-8 terminal, and an ASCII locale produces the old overstrikes on an ASCII terminal. OpenBSD's man actually supports a -T option outright. That's also worth thinking about. --- usr.bin/man/man.sh.patch1 2016-05-16 14:26:03.474372847 +0100 +++ usr.bin/man/man.sh 2016-11-08 08:40:50.406495179 +0000 @@ -316,23 +316,23 @@ # device flag (-T) we have to pass to eqn(1) and groff(1). Then, # setup the pipeline of commands based on the user's request. + # I don't pretend to know this; I'm just copying from the + # previous version of man(1). + case "$man_charset" in + KOI8-R) nroff_dev="koi8-r" ;; + ISO8859-1) nroff_dev="latin1" ;; + ISO8859-15) nroff_dev="latin1" ;; + UTF-8) nroff_dev="utf8" ;; + *) nroff_dev="ascii" ;; + esac + + NROFF="$NROFF -T$nroff_dev" + EQN="$EQN -T$nroff_dev" + # If the manpage is from a particular charset, we need to setup nroff # to properly output for the correct device. case "${manpage}" in *.${man_charset}/*) - # I don't pretend to know this; I'm just copying from the - # previous version of man(1). - case "$man_charset" in - KOI8-R) nroff_dev="koi8-r" ;; - ISO8859-1) nroff_dev="latin1" ;; - ISO8859-15) nroff_dev="latin1" ;; - UTF-8) nroff_dev="utf8" ;; - *) nroff_dev="ascii" ;; - esac - - NROFF="$NROFF -T$nroff_dev" - EQN="$EQN -T$nroff_dev" - # Iff the manpage is from the locale and not just the charset, # then we need to define the locale string. case "${manpage}" in @@ -351,9 +351,7 @@ eval "$tool=\${${tool}_$l:-\$$tool}" done ;; - *) NROFF="$NROFF -Tascii" - EQN="$EQN -Tascii" - ;; + *) ;; esac if [ -z "$MANCOLOR" ] && [ -z "$MANITALIC" ]; then From owner-freebsd-hackers@freebsd.org Tue Nov 8 12:30:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47E83C35724 for ; Tue, 8 Nov 2016 12:30:52 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from smtp.rlwinm.de (smtp.rlwinm.de [IPv6:2a01:4f8:201:31ef::e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14427397 for ; Tue, 8 Nov 2016 12:30:52 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from vader9.bultmann.eu (unknown [87.253.189.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.rlwinm.de (Postfix) with ESMTPSA id 3A75D1056B for ; Tue, 8 Nov 2016 13:30:50 +0100 (CET) Subject: Re: nss_ldap seems to not work To: freebsd-hackers@freebsd.org References: <1644757548.20161108110056@mail.ru> From: Jan Bramkamp Message-ID: <2eac83ec-c5d6-6167-2921-66e7c0d34476@rlwinm.de> Date: Tue, 8 Nov 2016 13:30:49 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <1644757548.20161108110056@mail.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 12:30:52 -0000 On 08/11/2016 09:00, Anthony Pankov via freebsd-hackers wrote: > Greetings. > > nss_ldap seems to not work correctly at least at FreeBSD 10.3. The original PADL nss_ldap and pam_ldap modules have been effectively unmaintained by the upstream for years. They inject a lot of code into each process using either NSS or PAM. Do yourself a favor and move on to net/nss-pam-ldapd(-sasl) which is maintained and moved most of the logic and all of network communication to a dedicated daemon process. See https://arthurdejong.org/nss-pam-ldapd/design for more details. > Two configurations > 1. FreeBSD 9.2 > 2. FreeBSD 10.3 > sharing nss_ldap settings and using the same LDAP tree (DIT) produce > different results. > > At FreeBSD 10.3 nss_ldap can't enumerate supplementary user > groups. > > Example: > FreeBSD 9.2: > # id user1 > ... groups=basegroup,gr1,gr2,gr3 > FreeBSD 10.3: > # id user1 > ... groups=basegroup > > The effect is inadequate result of initgroups() calling which lead to > various side effects with permissions. > > P.S. Interesting fact. At FreeBSD 10.3 pw utility produce correct > result: > #pw usershow user1 > ... groups=basegroup,gr1,gr2,gr3 > I suspect that there is a regression in the old nss_ldap module. At this time I would be surprised if anyone wanted to touch the old code with a ten foot pole. -- Jan Bramkamp From owner-freebsd-hackers@freebsd.org Tue Nov 8 14:33:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9180FC37A46 for ; Tue, 8 Nov 2016 14:33:35 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F7F2BAE for ; Tue, 8 Nov 2016 14:33:34 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from [88.198.220.132] (helo=sslproxy03.your-server.de) by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.85_2) (envelope-from ) id 1c473z-0005vL-Pi for freebsd-hackers@freebsd.org; Tue, 08 Nov 2016 15:07:47 +0100 Received: from [82.135.62.35] (helo=mail.embedded-brains.de) by sslproxy03.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.84_2) (envelope-from ) id 1c473z-0000wc-3p for freebsd-hackers@freebsd.org; Tue, 08 Nov 2016 15:07:47 +0100 Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 23B072A180C for ; Tue, 8 Nov 2016 15:07:55 +0100 (CET) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id B-MHgPNIRZBO for ; Tue, 8 Nov 2016 15:07:53 +0100 (CET) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 3D9C02A01AE for ; Tue, 8 Nov 2016 15:07:53 +0100 (CET) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id E5fMCJO3RjtB for ; Tue, 8 Nov 2016 15:07:53 +0100 (CET) Received: from [192.168.96.129] (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTPSA id 0029B2A0003 for ; Tue, 8 Nov 2016 15:07:52 +0100 (CET) To: FreeBSD From: Sebastian Huber Subject: Should page allocator zero the pages for UMA? Message-ID: <5821DC2E.9020302@embedded-brains.de> Date: Tue, 8 Nov 2016 15:07:42 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: smtp-embedded@poldinet.de X-Virus-Scanned: Clear (ClamAV 0.99.2/22499/Tue Nov 8 11:28:14 2016) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 14:33:35 -0000 Hello, we use the FreeBSD network, USB and SD/MMC card stacks for the real-time=20 operating system RTEMS: https://git.rtems.org/rtems-libbsd I update currently from FreeBSD 9.3 to head. We use the UMA from FreeBSD=20 with a custom page allocator: https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-kernel-page.= c The FreeBSD 9.3 based port worked well with uninitialized pages, e.g.=20 random or previous content. However, after the update to head I had to=20 zero initialize the pages. One issue was an incomplete struct inpcb { [...] struct inpcbport *inp_phd; /* (i/h) head of this list */ #define inp_zero_size offsetof(struct inpcb, inp_gencnt) inp_gen_t inp_gencnt; /* (c) generation count */ struct llentry *inp_lle; /* cached L2 information */ struct rwlock inp_lock; rt_gen_t inp_rt_cookie; /* generation for route entry */ union { /* cached L3 information */ struct route inpu_route; struct route_in6 inpu_route6; } inp_rtu; #define inp_route inp_rtu.inpu_route #define inp_route6 inp_rtu.inpu_route6 }; initialization. The initialization consists of two parts: static int udp_inpcb_init(void *mem, int size, int flags) { struct inpcb *inp; inp =3D mem; INP_LOCK_INIT(inp, "inp", "udpinp"); return (0); } /* * Allocate a PCB and associate it with the socket. * On success return with the PCB locked. */ int in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) { struct inpcb *inp; int error; #ifdef INVARIANTS if (pcbinfo =3D=3D &V_tcbinfo) { INP_INFO_RLOCK_ASSERT(pcbinfo); } else { INP_INFO_WLOCK_ASSERT(pcbinfo); } #endif error =3D 0; inp =3D uma_zalloc(pcbinfo->ipi_zone, M_NOWAIT); if (inp =3D=3D NULL) return (ENOBUFS); bzero(inp, inp_zero_size); inp->inp_pcbinfo =3D pcbinfo; inp->inp_socket =3D so; inp->inp_cred =3D crhold(so->so_cred); inp->inp_inc.inc_fibnum =3D so->so_fibnum; [...] This lets at least inp_route uninitialized leading to a crash during=20 destruction, e.g. if (inp->inp_route.ro_rt) { RTFREE(inp->inp_route.ro_rt); inp->inp_route.ro_rt =3D (struct rtentry *)NULL; } uses uninitialized data. Did something in the page allocator change between FreeBSD 9.3 and=20 trunk, so that page are now zero initialized or is this a bug in=20 udp_inpcb_init()? --=20 Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG= . From owner-freebsd-hackers@freebsd.org Tue Nov 8 14:55:42 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50CA6C3508B for ; Tue, 8 Nov 2016 14:55:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D284298B for ; Tue, 8 Nov 2016 14:55:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uA8EtSsV082796 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 8 Nov 2016 16:55:28 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uA8EtSsV082796 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uA8EtS0Z082793; Tue, 8 Nov 2016 16:55:28 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 8 Nov 2016 16:55:28 +0200 From: Konstantin Belousov To: Sebastian Huber Cc: FreeBSD Subject: Re: Should page allocator zero the pages for UMA? Message-ID: <20161108145528.GN54029@kib.kiev.ua> References: <5821DC2E.9020302@embedded-brains.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5821DC2E.9020302@embedded-brains.de> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 14:55:42 -0000 On Tue, Nov 08, 2016 at 03:07:42PM +0100, Sebastian Huber wrote: > Hello, > > we use the FreeBSD network, USB and SD/MMC card stacks for the real-time > operating system RTEMS: > > https://git.rtems.org/rtems-libbsd > > I update currently from FreeBSD 9.3 to head. We use the UMA from FreeBSD > with a custom page allocator: > > https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-kernel-page.c > > The FreeBSD 9.3 based port worked well with uninitialized pages, e.g. > random or previous content. However, after the update to head I had to > zero initialize the pages. One issue was an incomplete > > struct inpcb { > [...] > struct inpcbport *inp_phd; /* (i/h) head of this list */ > #define inp_zero_size offsetof(struct inpcb, inp_gencnt) > inp_gen_t inp_gencnt; /* (c) generation count */ > struct llentry *inp_lle; /* cached L2 information */ > struct rwlock inp_lock; > rt_gen_t inp_rt_cookie; /* generation for route entry */ > union { /* cached L3 information */ > struct route inpu_route; > struct route_in6 inpu_route6; > } inp_rtu; > #define inp_route inp_rtu.inpu_route > #define inp_route6 inp_rtu.inpu_route6 > }; > > initialization. The initialization consists of two parts: > > static int > udp_inpcb_init(void *mem, int size, int flags) > { > struct inpcb *inp; > > inp = mem; > INP_LOCK_INIT(inp, "inp", "udpinp"); > return (0); > } > > /* > * Allocate a PCB and associate it with the socket. > * On success return with the PCB locked. > */ > int > in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) > { > struct inpcb *inp; > int error; > > #ifdef INVARIANTS > if (pcbinfo == &V_tcbinfo) { > INP_INFO_RLOCK_ASSERT(pcbinfo); > } else { > INP_INFO_WLOCK_ASSERT(pcbinfo); > } > #endif > > error = 0; > inp = uma_zalloc(pcbinfo->ipi_zone, M_NOWAIT); > if (inp == NULL) > return (ENOBUFS); > bzero(inp, inp_zero_size); > inp->inp_pcbinfo = pcbinfo; > inp->inp_socket = so; > inp->inp_cred = crhold(so->so_cred); > inp->inp_inc.inc_fibnum = so->so_fibnum; > [...] > > This lets at least inp_route uninitialized leading to a crash during > destruction, e.g. > > if (inp->inp_route.ro_rt) { > RTFREE(inp->inp_route.ro_rt); > inp->inp_route.ro_rt = (struct rtentry *)NULL; > } > > uses uninitialized data. > > Did something in the page allocator change between FreeBSD 9.3 and > trunk, so that page are now zero initialized or is this a bug in > udp_inpcb_init()? Nothing guarantees that returned items are zeroed out automatically. Pages are not returned zeroed by VM page allocator, it might indicate that the allocated page happens to be zeroed by PG_ZERO flag. Unless M_ZERO is passed to uma_zalloc(), item is not zeroed. I am not sure, you did not demonstrated zone initialization, but it seems that your zone is having zone init function. If this is true, than I do not see why do you consider it possible to have items automatically zeroed at all. Uma calls the init function at the bulk allocation time, and constructor for each item returned by uma_zalloc(). No implicit zeroing occurs there. If your zone does not have init/fini functions, nor ctors/dtors, then HEAD and stable/11 actually actively trash items to catch reliance on the uninitialized memory being zero. Look for trash_ctor in uma_core.c. From owner-freebsd-hackers@freebsd.org Tue Nov 8 13:40:25 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28B28C3668F for ; Tue, 8 Nov 2016 13:40:25 +0000 (UTC) (envelope-from bland@bbnest.net) Received: from mail1.asahi-net.or.jp (mail1.asahi-net.or.jp [202.224.39.197]) by mx1.freebsd.org (Postfix) with ESMTP id F3BDB390 for ; Tue, 8 Nov 2016 13:40:24 +0000 (UTC) (envelope-from bland@bbnest.net) Received: from eee.bbnest.net (w133033.ppp.asahi-net.or.jp [121.1.133.33]) by mail1.asahi-net.or.jp (Postfix) with ESMTP id BB456E1E6; Tue, 8 Nov 2016 22:40:21 +0900 (JST) Received: from nest.bbnest.net (nest.bbnest.net [192.168.1.108]) (authenticated bits=0) by eee.bbnest.net (8.15.2/8.15.2) with ESMTPSA id uA8DeKpM005569 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Nov 2016 22:40:22 +0900 (JST) (envelope-from bland@bbnest.net) Subject: Re: nss_ldap seems to not work Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 From: Alexander Nedotsukov X-Priority: 3 (Normal) In-Reply-To: <1644757548.20161108110056@mail.ru> Date: Tue, 8 Nov 2016 22:40:20 +0900 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> References: <1644757548.20161108110056@mail.ru> To: Anthony Pankov X-Mailer: Apple Mail (2.3124) X-Mailman-Approved-At: Tue, 08 Nov 2016 15:39:52 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 13:40:25 -0000 Does it help if you add "nss_schema rfc2307=E2=80=9D to nss_ldap.config? > On 8 =D0=BD=D0=BE=D1=8F=D0=B1. 2016 =D0=B3., at 17:00, Anthony Pankov = via freebsd-hackers wrote: >=20 > Greetings. >=20 > nss_ldap seems to not work correctly at least at FreeBSD 10.3. >=20 > Two configurations > 1. FreeBSD 9.2 > 2. FreeBSD 10.3 > sharing nss_ldap settings and using the same LDAP tree (DIT) = produce > different results. >=20 > At FreeBSD 10.3 nss_ldap can't enumerate supplementary user > groups. >=20 > Example: > FreeBSD 9.2: > # id user1 > ... groups=3Dbasegroup,gr1,gr2,gr3 > FreeBSD 10.3: > # id user1 > ... groups=3Dbasegroup >=20 > The effect is inadequate result of initgroups() calling which lead to > various side effects with permissions. >=20 > P.S. Interesting fact. At FreeBSD 10.3 pw utility produce correct > result: > #pw usershow user1 > ... groups=3Dbasegroup,gr1,gr2,gr3 >=20 > --=20 > Best regards, > Anthony mailto:ap00@mail.ru >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Tue Nov 8 16:02:47 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B03EC37061 for ; Tue, 8 Nov 2016 16:02:47 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from smtp.rlwinm.de (smtp.rlwinm.de [IPv6:2a01:4f8:201:31ef::e]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0853ABEE for ; Tue, 8 Nov 2016 16:02:47 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from vader9.bultmann.eu (unknown [87.253.189.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.rlwinm.de (Postfix) with ESMTPSA id 340B7105AE for ; Tue, 8 Nov 2016 17:02:45 +0100 (CET) Subject: Re: nss_ldap seems to not work To: freebsd-hackers@freebsd.org References: <1644757548.20161108110056@mail.ru> <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> From: Jan Bramkamp Message-ID: <577797f1-9475-918f-6e36-77de59207416@rlwinm.de> Date: Tue, 8 Nov 2016 17:02:44 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 16:02:47 -0000 On 08/11/2016 14:40, Alexander Nedotsukov wrote: > Does it help if you add "nss_schema rfc2307” to nss_ldap.config? Did the default change to rf2307bis? Anthony Pankov: Are your using groups of DNs or groups of uids? -- Jan Bramkamp From owner-freebsd-hackers@freebsd.org Tue Nov 8 19:20:56 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29F89C37ED5 for ; Tue, 8 Nov 2016 19:20:56 +0000 (UTC) (envelope-from ap00@mail.ru) Received: from fallback4.mail.ru (fallback4.mail.ru [94.100.181.169]) by mx1.freebsd.org (Postfix) with ESMTP id DABE3FA6 for ; Tue, 8 Nov 2016 19:20:55 +0000 (UTC) (envelope-from ap00@mail.ru) Received: from smtp29.i.mail.ru (smtp29.i.mail.ru [94.100.177.89]) by fallback4.mail.ru (mPOP.Fallback_MX) with ESMTP id 4E2DB22F60FB for ; Tue, 8 Nov 2016 22:20:46 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Subject:To:Message-ID:From:Date; bh=+4O/PZE0Y/ZpG+ISyEdU0kyCT7RFXHAvFX2X/nQi0jQ=; b=I4EX1l3MbVfaBb9K6jIJKZ8zypzlM8xq4YPh2zO6mnYnQHUK/P2xMCYg4ANs4XFCkdEnw5K/yM4cXMyeTGJeyrRHKuvcCYssRflMpHoKZwu0plMu6D7Ws2B3VrUlsK4uZJgipS7goEZPgSCSpDsiFgVUjUl9t0KU+tp3AalijqM=; Received: from [91.190.121.202] (port=54333 helo=pstation) by smtp29.i.mail.ru with esmtpa (envelope-from ) id 1c4Bwj-0001gr-CD for freebsd-hackers@freebsd.org; Tue, 08 Nov 2016 22:20:37 +0300 Date: Tue, 8 Nov 2016 22:20:34 +0300 From: Anthony Pankov X-Priority: 3 (Normal) Message-ID: <26095845.20161108222034@mail.ru> To: freebsd-hackers@freebsd.org Subject: Re: nss_ldap seems to not work In-Reply-To: <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> References: <1644757548.20161108110056@mail.ru> <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Authentication-Results: smtp29.i.mail.ru; auth=pass smtp.auth=ap00@mail.ru smtp.mailfrom=ap00@mail.ru X-E1FCDC63: 47975F80F6E2B9B30E2709F7B692CA3BA2BA58D3D091CA1F X-E1FCDC64: 136FD621EEE4B2902005B47CA003263070B98512BA0E4D99685376221D61E053B9C4F2611CB3DCF4 X-Mailru-Sender: 0489DF6C38DA5EE561E8A477868835DB5F0E93C26E80CF8FCC19735011D983A7920795A225069ADACEE4813122F9A0DF X-Mras: OK X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 19:20:56 -0000 Hello, Alexander. I'm sorry, but this happen when engaging production server. I fix it by moving to nss-pam-ldap (nslcd) so I can't move back and give this option a chance. > Does it help if you add "nss_schema rfc2307=E2=80=9D to nss_ldap.config? >> On 8 =D0=BD=D0=BE=D1=8F=D0=B1. 2016 =D0=B3., at 17:00, Anthony Pankov vi= a freebsd-hackers wrote: >>=20 >> Greetings. >>=20 >> nss_ldap seems to not work correctly at least at FreeBSD 10.3. >>=20 >> Two configurations >> 1. FreeBSD 9.2 >> 2. FreeBSD 10.3 >> sharing nss_ldap settings and using the same LDAP tree (DIT) pro= duce >> different results. >>=20 >> At FreeBSD 10.3 nss_ldap can't enumerate supplementary user >> groups. >>=20 >> Example: >> FreeBSD 9.2: >> # id user1 >> ... groups=3Dbasegroup,gr1,gr2,gr3 >> FreeBSD 10.3: >> # id user1 >> ... groups=3Dbasegroup >>=20 >> The effect is inadequate result of initgroups() calling which lead to >> various side effects with permissions. >>=20 >> P.S. Interesting fact. At FreeBSD 10.3 pw utility produce correct >> result: >> #pw usershow user1 >> ... groups=3Dbasegroup,gr1,gr2,gr3 >>=20 >> --=20 >> Best regards, >> Anthony mailto:ap00@mail.ru >>=20 >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.or= g" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" --=20 =D0=A1 =D1=83=D0=B2=D0=B0=D0=B6=D0=B5=D0=BD=D0=B8=D0=B5=D0=BC, Anthony mailto:ap00@mail.ru From owner-freebsd-hackers@freebsd.org Tue Nov 8 20:02:38 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0142C36D43 for ; Tue, 8 Nov 2016 20:02:38 +0000 (UTC) (envelope-from ap00@mail.ru) Received: from fallback8.mail.ru (fallback8.mail.ru [94.100.181.110]) by mx1.freebsd.org (Postfix) with ESMTP id 60053E3C for ; Tue, 8 Nov 2016 20:02:37 +0000 (UTC) (envelope-from ap00@mail.ru) Received: from smtp40.i.mail.ru (smtp40.i.mail.ru [94.100.177.100]) by fallback8.mail.ru (mPOP.Fallback_MX) with ESMTP id BB6CCAB79F7F; Tue, 8 Nov 2016 22:29:36 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References:In-Reply-To:Subject:To:Message-ID:From:Date; bh=e4rgqbUy3mMN8HZ1g5LCV6KkYbMm3h9jG7d0eflVRt0=; b=LNx1hMf2S5dMxuG5je1A00qWmW6P6Zn/YL52b9Em+xjkaeMZeJhCzCbMqXFz3LvUYL0k5qelRx1iehwag07T92yQlHCHra3K9c4Gbu+DFj7JJ5nhXmwWg2Xtfqwz1BIPd8J7SuxHNNsGQKpOzjpfm1RlrSvDEoTSH/5RujRA2A8=; Received: from [91.190.121.202] (port=54437 helo=pstation) by smtp40.i.mail.ru with esmtpa (envelope-from ) id 1c4C5F-0003SL-AT; Tue, 08 Nov 2016 22:29:25 +0300 Date: Tue, 8 Nov 2016 22:29:22 +0300 From: Anthony Pankov X-Priority: 3 (Normal) Message-ID: <1022257053.20161108222922@mail.ru> To: Jan Bramkamp , freebsd-hackers@freebsd.org Subject: Re: nss_ldap seems to not work In-Reply-To: <577797f1-9475-918f-6e36-77de59207416@rlwinm.de> References: <1644757548.20161108110056@mail.ru> <4A97463C-6A15-4B79-A52C-9DBBF2A20862@bbnest.net> <577797f1-9475-918f-6e36-77de59207416@rlwinm.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Authentication-Results: smtp40.i.mail.ru; auth=pass smtp.auth=ap00@mail.ru smtp.mailfrom=ap00@mail.ru X-E1FCDC63: 47975F80F6E2B9B3745D855FC68FE5F865B98BC17AE4ABE7 X-E1FCDC64: C90CDBD1C952FE9D60D31AC0B3436FAF70B98512BA0E4D9939D98DD915EAF4EBC9EA2F84C7A79465 X-Mailru-Sender: 0489DF6C38DA5EE561E8A477868835DBBEF8434DEF9628170C2038ABC5A2BD3D599ED0D7F3EBD3D2B7AB0E7134D2E1A8 X-Mras: OK X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2016 20:02:39 -0000 > The original PADL nss_ldap and pam_ldap modules have been effectively > unmaintained by the upstream for years... FreeBSD handbook describe nss_ldap use case. I respect FreeBSD handbook and follow its recommendations. But in my situation nslcd was a cure . > Anthony Pankov: Are your using groups of DNs or groups of uids? I use posixGroup object class with multivalued memberUid. memberUid value is just an uid. > -- Jan Bramkamp > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" -- Best reagrds, Anthony mailto:ap00@mail.ru From owner-freebsd-hackers@freebsd.org Wed Nov 9 05:11:07 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40A0AC377F9 for ; Wed, 9 Nov 2016 05:11:07 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu [18.9.25.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E7E29669 for ; Wed, 9 Nov 2016 05:11:06 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190c-573ff70000007d1a-30-5822aeb5e50d Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id F8.80.32026.5BEA2285; Wed, 9 Nov 2016 00:05:57 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id uA955ugK029567; Wed, 9 Nov 2016 00:05:56 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uA955qHj014005 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Nov 2016 00:05:55 -0500 Date: Tue, 8 Nov 2016 23:05:52 -0600 From: Benjamin Kaduk To: Jonathan de Boyne Pollard Cc: FreeBSD Hackers , NetBSD Users Subject: Re: Improved manual page for ul(1) Message-ID: <20161109050551.GZ91607@kduck.kaduk.org> References: <20161106190144.GL91607@kduck.kaduk.org> <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrKIsWRmVeSWpSXmKPExsUixCmqrLt1nVKEwYYb8hbbN/9jtDh/toXJ 4ueSU8wOzB4zPs1n8Vj4sIfR4+rEbUwBzFFcNimpOZllqUX6dglcGZvXbmIrWChccWO3SwPj Yv4uRk4OCQETiTUda1i6GLk4hATamCRWfz7PBuFsYJT4v/4rK4RzhUli94seVpAWFgEViRl/ /rGD2GwCahKP9zaDxUUEPCS6/m9lBrGZBZIkHrX8ZQGxhQW0JaZ2rQSL8wKt+7b2LjPE0AWM Escb5jBBJAQlTs58wgLRrCVx499LoDgHkC0tsfwfB0iYU8BB4uihpWwgtqiAskTDjAfMExgF ZiHpnoWkexZC9wJG5lWMsim5Vbq5iZk5xanJusXJiXl5qUW6hnq5mSV6qSmlmxjBoSvJs4Px zBuvQ4wCHIxKPLwC9xUjhFgTy4orcw8xSnIwKYny9s5SihDiS8pPqcxILM6ILyrNSS0+xCjB wawkwvtiOVCONyWxsiq1KB8mJc3BoiTO+9/ta7iQQHpiSWp2ampBahFMVoaDQ0mCt2EtUKNg UWp6akVaZk4JQpqJgxNkOA/QcAOQGt7igsTc4sx0iPwpRkUpcV4PkIQASCKjNA+uF5RaJLL3 17xiFAd6RZg3DaSKB5iW4LpfAQ1mAhpcFaMAMrgkESEl1cCom7a06qjv5k5OThu25dz/Zk5+ dmuSjpnL0a4yrpvsCvrqFzxq7j69IXt2coTTT8F7xUV162zuHL+xSW5m1/7lBTqF026LLfow 5XjhLoaIW037p3arWiieWnHwyFmPOPlQ2xVnkwt3LPerm6N68ItWhY/Gih4RPr5D8a5zZ51b 6/VmTrvUXsafSizFGYmGWsxFxYkAp5OJmAgDAAA= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 05:11:07 -0000 On Mon, Nov 07, 2016 at 10:17:32AM +0000, Jonathan de Boyne Pollard wrote: > Jonathan de Boyne Pollard: > > > So whither does one submit an improved manual page for ul(1) ? Here's the source: > > > > * http://jdebp.eu./Proposals/ul.1 > > Benjamin Kaduk: > > Somehow I do not think that a 1993 copyright by the Regents is accurate. I am unwilling to commit without some clarification of the copyright status and confirmation of license terms (noting that for new code, we prefer the two-clause BSD license at this point). > > It's actually quite clear and straightforward. You are suggesting that > the copyright declaration should be changed; that is not permitted, > however. The copyright declaration for the UC Regents is entirely > accurate for the (small amount of) existing manual text that's still > there, and retaining it is mandated by the first of the very licence > terms that you are referring to. I took the existing manual and > expanded and corrected it, and the world has my work under the exact > same licence that I had the original, so that the result is not > entangled in BSD Licence Hell; hence the exact same licence remains > there as-is. Leaving the copyright declaration and license unchanged is akin to disclaiming that you hold any copyright in the work, i.e., that only minor non-copyrightable changes were made. I assumed it went without saying that there remains original text the Regents copyright must stay, rather, that there must be an *addition* to the copyright holders. So, which is it? Do you claim copyright on any of the text present, or disclaim it? My point about our project (and the Regents) having gone to the two-clause form remains. > Really, the first reaction should rather be something like "My goodness! > Are all of those bugs really there?" or perhaps "Does anything nowadays > make any use of that wacky lpr feature?" or even "Given then that it can > handle UTF-8 input and output, could ul be fixed to make all of groff's > overstrikes work?". (-: It would be easier to do so if the contribution was provided as a diff. > I haven't found anything that makes use of the lpr feature. Presumably > it dates from a time when ul was once used as a filter for printing. > Back then there was an array of programs: collpr, colcrt, iul, ul, ulpr, > ... Indeed, many interesting historical things have dropped from common use. -Ben From owner-freebsd-hackers@freebsd.org Wed Nov 9 05:23:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEDC4C37EDC for ; Wed, 9 Nov 2016 05:23:34 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 913CCD9E for ; Wed, 9 Nov 2016 05:23:34 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190f-45bff70000005610-a6-5822b2cdd8ea Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 03.A0.22032.DC2B2285; Wed, 9 Nov 2016 00:23:26 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id uA95NPUM030995; Wed, 9 Nov 2016 00:23:25 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uA95NLtV016822 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Nov 2016 00:23:24 -0500 Date: Tue, 8 Nov 2016 23:23:21 -0600 From: Benjamin Kaduk To: Jonathan de Boyne Pollard Cc: FreeBSD Hackers , NetBSD Users Subject: Re: Improved manual page for ul(1) Message-ID: <20161109052321.GA91607@kduck.kaduk.org> References: <20161106190144.GL91607@kduck.kaduk.org> <87e90ef8-05d2-1480-47ce-cc228fb3fd36@NTLWorld.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrPIsWRmVeSWpSXmKPExsUixCmqrHtuk1KEwe4vyhbbN/9jtDh/toXJ 4ueSU8wOzB4zPs1n8Vj4sIfR4+rEbUwBzFFcNimpOZllqUX6dglcGadfRRZMYq2Y9vE2cwNj G0sXIweHhICJxOPvDl2MXBxCAm1MEl8ufmSDcDYwSnzbehzKucIk0XOoHayDRUBFYud3/y5G Tg42ATWJx3ubWUFsEQEPia7/W5lBbGaBJIlHLX9ZQGxhAW2JqV0rweK8QMsWLFrHCjHzGqPE 4tYd7BAJQYmTM5+wQDRrSdz495IJZBezgLTE8n8cIGFOAQeJWQuvgc0RFVCWaJjxgHkCo8As JN2zkHTPQuhewMi8ilE2JbdKNzcxM6c4NVm3ODkxLy+1SNdELzezRC81pXQTIzhsJfl3MM5p 8D7EKMDBqMTDK3BfMUKINbGsuDL3EKMkB5OSKG/vLKUIIb6k/JTKjMTijPii0pzU4kOMEhzM SiK8gRuBcrwpiZVVqUX5MClpDhYlcd7/bl/DhQTSE0tSs1NTC1KLYLIyHBxKErzfQBoFi1LT UyvSMnNKENJMHJwgw3mAhi8BG15ckJhbnJkOkT/FqMvxbvO7B0xCLHn5ealS4ryzQYoEQIoy SvPg5oDSjUT2/ppXjOJAbwnzWgOTjxAPMFXBTXoFtIQJaElVjALIkpJEhJRUA2NkXLFnLteF ZJ83Vge4s1nrL/7xupDVdyXt2PlXfzZPEXKvK3Fee0Kkd22xzItL3/c9LePJuS3N1rj4yS8r r0lrpV8IrHQSLZ+w81+zJ1fIZZt7D53+L829EbKw+WjF1/+L1yy6fWBdxSnWnxonDdIXvuu4 GltU8Yy5UMX589fS/2a+0aViwmpKLMUZiYZazEXFiQAd5iHPEgMAAA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 05:23:35 -0000 On Tue, Nov 08, 2016 at 09:17:53AM +0000, Jonathan de Boyne Pollard wrote: > Jonathan de Boyne Pollard: > > > And more interesting is whether less could be fixed to make groff's > > overstrikes work. Or groff fixed to not need overstrikes for things > > that can be done in Unicode ... > > > It turns out that this is the man command limiting things. Again. It's > hardwiring -Tascii even if it detects a UTF-8 character locale. Note that the affected codepath is only for the legacy groff fallback; the common case should be using mandoc(1) directly (though I would not be surprised if it still needs patching to pass -Tmumble). -Ben From owner-freebsd-hackers@freebsd.org Wed Nov 9 06:45:48 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 986DCC3708F for ; Wed, 9 Nov 2016 06:45:48 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 56B62CBD for ; Wed, 9 Nov 2016 06:45:48 +0000 (UTC) (envelope-from sebastian.huber@embedded-brains.de) Received: from [88.198.220.132] (helo=sslproxy03.your-server.de) by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.85_2) (envelope-from ) id 1c4Mdl-00009B-9n; Wed, 09 Nov 2016 07:45:45 +0100 Received: from [82.135.62.35] (helo=mail.embedded-brains.de) by sslproxy03.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.84_2) (envelope-from ) id 1c4Mdk-00049w-Jx; Wed, 09 Nov 2016 07:45:44 +0100 Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 913082A0003; Wed, 9 Nov 2016 07:45:54 +0100 (CET) Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id m06-M-Ry5Hax; Wed, 9 Nov 2016 07:45:52 +0100 (CET) Received: from localhost (localhost.localhost [127.0.0.1]) by mail.embedded-brains.de (Postfix) with ESMTP id 307D12A1808; Wed, 9 Nov 2016 07:45:52 +0100 (CET) X-Virus-Scanned: amavisd-new at zimbra.eb.localhost Received: from mail.embedded-brains.de ([127.0.0.1]) by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id kDVT8wttQDvO; Wed, 9 Nov 2016 07:45:52 +0100 (CET) Received: from [192.168.96.129] (unknown [192.168.96.129]) by mail.embedded-brains.de (Postfix) with ESMTPSA id 1154C2A0003; Wed, 9 Nov 2016 07:45:52 +0100 (CET) Subject: Re: Should page allocator zero the pages for UMA? To: Konstantin Belousov References: <5821DC2E.9020302@embedded-brains.de> <20161108145528.GN54029@kib.kiev.ua> Cc: FreeBSD From: Sebastian Huber Message-ID: <5822C615.3060100@embedded-brains.de> Date: Wed, 9 Nov 2016 07:45:41 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <20161108145528.GN54029@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable X-Authenticated-Sender: smtp-embedded@poldinet.de X-Virus-Scanned: Clear (ClamAV 0.99.2/22502/Wed Nov 9 04:11:10 2016) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 06:45:48 -0000 On 08/11/16 15:55, Konstantin Belousov wrote: > On Tue, Nov 08, 2016 at 03:07:42PM +0100, Sebastian Huber wrote: >> Hello, >> >> we use the FreeBSD network, USB and SD/MMC card stacks for the real-ti= me >> operating system RTEMS: >> >> https://git.rtems.org/rtems-libbsd >> >> I update currently from FreeBSD 9.3 to head. We use the UMA from FreeB= SD >> with a custom page allocator: >> >> https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-kernel-pa= ge.c >> >> The FreeBSD 9.3 based port worked well with uninitialized pages, e.g. >> random or previous content. However, after the update to head I had to >> zero initialize the pages. One issue was an incomplete >> >> struct inpcb { >> [...] >> struct inpcbport *inp_phd; /* (i/h) head of this list */ >> #define inp_zero_size offsetof(struct inpcb, inp_gencnt) >> inp_gen_t inp_gencnt; /* (c) generation count */ >> struct llentry *inp_lle; /* cached L2 information */ >> struct rwlock inp_lock; >> rt_gen_t inp_rt_cookie; /* generation for route entry */ >> union { /* cached L3 information */ >> struct route inpu_route; >> struct route_in6 inpu_route6; >> } inp_rtu; >> #define inp_route inp_rtu.inpu_route >> #define inp_route6 inp_rtu.inpu_route6 >> }; >> >> initialization. The initialization consists of two parts: >> >> static int >> udp_inpcb_init(void *mem, int size, int flags) >> { >> struct inpcb *inp; >> >> inp =3D mem; >> INP_LOCK_INIT(inp, "inp", "udpinp"); >> return (0); >> } >> >> /* >> * Allocate a PCB and associate it with the socket. >> * On success return with the PCB locked. >> */ >> int >> in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) >> { >> struct inpcb *inp; >> int error; >> >> #ifdef INVARIANTS >> if (pcbinfo =3D=3D &V_tcbinfo) { >> INP_INFO_RLOCK_ASSERT(pcbinfo); >> } else { >> INP_INFO_WLOCK_ASSERT(pcbinfo); >> } >> #endif >> >> error =3D 0; >> inp =3D uma_zalloc(pcbinfo->ipi_zone, M_NOWAIT); >> if (inp =3D=3D NULL) >> return (ENOBUFS); >> bzero(inp, inp_zero_size); >> inp->inp_pcbinfo =3D pcbinfo; >> inp->inp_socket =3D so; >> inp->inp_cred =3D crhold(so->so_cred); >> inp->inp_inc.inc_fibnum =3D so->so_fibnum; >> [...] >> >> This lets at least inp_route uninitialized leading to a crash during >> destruction, e.g. >> >> if (inp->inp_route.ro_rt) { >> RTFREE(inp->inp_route.ro_rt); >> inp->inp_route.ro_rt =3D (struct rtentry *)NULL; >> } >> >> uses uninitialized data. >> >> Did something in the page allocator change between FreeBSD 9.3 and >> trunk, so that page are now zero initialized or is this a bug in >> udp_inpcb_init()? > Nothing guarantees that returned items are zeroed out automatically. > Pages are not returned zeroed by VM page allocator, it might indicate > that the allocated page happens to be zeroed by PG_ZERO flag. Unless > M_ZERO is passed to uma_zalloc(), item is not zeroed. > > I am not sure, you did not demonstrated zone initialization, but it see= ms > that your zone is having zone init function. If this is true, than I d= o not > see why do you consider it possible to have items automatically zeroed = at > all. Uma calls the init function at the bulk allocation time, and cons= tructor > for each item returned by uma_zalloc(). No implicit zeroing occurs the= re. Now, I am a bit confused. We have /* * Allocate a new slab for a keg. This does not insert the slab onto a=20 list. * * Arguments: * wait Shall we wait? * * Returns: * The slab that was allocated or NULL if there is no memory and the * caller specified M_NOWAIT. */ static uma_slab_t keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int wait) { [...] /* * This reproduces the old vm_zone behavior of zero filling pages th= e * first time they are added to a zone. * * Malloced items are zeroed in uma_zalloc. */ if ((keg->uk_flags & UMA_ZONE_MALLOC) =3D=3D 0) wait |=3D M_ZERO; else wait &=3D ~M_ZERO; [...] /* zone is passed for legacy reasons. */ mem =3D allocf(zone, keg->uk_ppera * PAGE_SIZE, &flags, wait); Firstly, the name "wait" and the comment are confusing, since "wait" is=20 really "flags" which may have M_ZERO set. So, for non UMA_ZONE_MALLOC=20 zones (which should be most zones) the page is supposed to be zeroed? We ignored the flags in our FreeBSD 9.3 based port and this worked quite=20 well. However, on FreeBSD head I changed the page allocator now to do: #ifdef INVARIANTS wait |=3D M_ZERO; #endif if (addr !=3D NULL && (wait & M_ZERO) !=3D 0) { memset(addr, 0, size_in_bytes); } With this change, everything works like before. > > If your zone does not have init/fini functions, nor ctors/dtors, then > HEAD and stable/11 actually actively trash items to catch reliance on > the uninitialized memory being zero. Look for trash_ctor in uma_core.c= . The zone has an init/fini function, see udp_init() /* * For now default to 2-tuple UDP hashing - until the fragment * reassembly code can also update the flowid. * * Once we can calculate the flowid that way and re-establish * a 4-tuple, flip this to 4-tuple. */ in_pcbinfo_init(&V_udbinfo, "udp", &V_udb, UDBHASHSIZE, UDBHASHSIZE, "udp_inpcb", udp_inpcb_init, NULL, 0, IPI_HASHFIELDS_2TUPLE); --=20 Sebastian Huber, embedded brains GmbH Address : Dornierstr. 4, D-82178 Puchheim, Germany Phone : +49 89 189 47 41-16 Fax : +49 89 189 47 41-09 E-Mail : sebastian.huber@embedded-brains.de PGP : Public key available on request. Diese Nachricht ist keine gesch=E4ftliche Mitteilung im Sinne des EHUG. From owner-freebsd-hackers@freebsd.org Wed Nov 9 07:00:42 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9E4AC37681 for ; Wed, 9 Nov 2016 07:00:42 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-1.server.virginmedia.net (know-smtprelay-omc-1.server.virginmedia.net [80.0.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 200C27E1 for ; Wed, 9 Nov 2016 07:00:41 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.100] ([86.10.211.13]) by know-smtprelay-1-imp with bizsmtp id 5j0Z1u00H0HtmFq01j0ZHQ; Wed, 09 Nov 2016 07:00:33 +0000 X-Originating-IP: [86.10.211.13] X-Spam: 0 X-Authority: v=2.1 cv=beuWK77B c=1 sm=1 tr=0 a=SB7hr1IvJSWWr45F2gQiKw==:117 a=SB7hr1IvJSWWr45F2gQiKw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=IkcTkHD0fZMA:10 a=2rVjqWD_AAAA:8 a=VEFJHFMHd6k9yd6FQUcA:9 a=QEXdDO2ut3YA:10 a=-FEs8UIgK8oA:10 a=NWVoK91CQyQA:10 a=ULaUcM2Ibn9MdPUUwucP:22 Subject: Re: Improved manual page for ul(1) To: FreeBSD Hackers , NetBSD Users References: From: Jonathan de Boyne Pollard Message-ID: <504a32f0-fd75-9d8b-d58f-243d77dbf852@NTLWorld.com> Date: Wed, 9 Nov 2016 07:00:10 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 07:00:42 -0000 Jonathan de Boyne Pollard: > So whither does one submit an improved manual page for ul(1) ? Here's > the source: > > * http://jdebp.eu./Proposals/ul.1 I don't want to make this too much of a moving target, but I've updated this. The prior version is now at http://jdebp.eu./Proposals/ul.1-20161107 . * The Options sub-section was a failure of boldness on my part. The description of options was in-line within the description section in the original page, and I kept it in the description section. It's now a top-level section. * I wasn't satisfied with the history section, which has now been expanded. * The description now states what side ul chooses when it comes to the underscore overstriking underscore ambiguity. ( printf '_\b_\n'|ul -i ) As I said, I don't want to make this a constantly moving target. I'm fairly satisfied that I've covered the ground that I set out to cover, now. From owner-freebsd-hackers@freebsd.org Wed Nov 9 07:01:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1019C37727 for ; Wed, 9 Nov 2016 07:01:35 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from know-smtprelay-omc-1.server.virginmedia.net (know-smtprelay-omc-1.server.virginmedia.net [80.0.253.65]) by mx1.freebsd.org (Postfix) with ESMTP id 5754397F for ; Wed, 9 Nov 2016 07:01:35 +0000 (UTC) (envelope-from j.deboynepollard-newsgroups@ntlworld.com) Received: from [192.168.1.100] ([86.10.211.13]) by know-smtprelay-1-imp with bizsmtp id 5j1a1u00A0HtmFq01j1bK0; Wed, 09 Nov 2016 07:01:35 +0000 X-Originating-IP: [86.10.211.13] X-Spam: 0 X-Authority: v=2.1 cv=beuWK77B c=1 sm=1 tr=0 a=SB7hr1IvJSWWr45F2gQiKw==:117 a=SB7hr1IvJSWWr45F2gQiKw==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=IkcTkHD0fZMA:10 a=PM9FOwcr_VXe-W7DZxMA:9 a=QEXdDO2ut3YA:10 Subject: Re: Improved manual page for ul(1) To: FreeBSD Hackers , NetBSD Users References: From: Jonathan de Boyne Pollard Message-ID: <0eed8264-8fb9-7b00-365d-8d9d2316d1ce@NTLWorld.com> Date: Wed, 9 Nov 2016 07:01:11 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2016 07:01:36 -0000 Mark Heily: > The benefit of a separate manpage for TTY-37 information is that any > other commands that deal with these control sequences can have their > manpages updated to refer to the tty37_control(5) manual. It's a good idea, and not without precedent, both in the FreeBSD manual and older manuals. The problem is twofold. The relevant tools don't all agree on what the semantics are. As noted in this manual and in the Stack Exchange answer, less and more don't recognize overstriking that ul and colcrt do recognize. (ul and colcrt are right on the basis that what they do is what a real TTY-37 would do, and less and more are wrong on that basis.) And these semantics are extensions to, and even different from, an actual Teletype Model 37. The differences are given in this manual: the carriage width is different (and different from some other tools, too -- collpr has an 80 column wide carriage, iul a 500-column wide one), half-scrolling is very different (and buggy), and the toolset in general incorporates (also buggy) wide character capability that simply wasn't even an idea for a Teletype Model 37. Also, I don't want this to be too much of a moving target. I've already done one change. I want to leave it stable for a little while. People are (I hope) reading it. From owner-freebsd-hackers@freebsd.org Thu Nov 10 20:59:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 068EEC3A239 for ; Thu, 10 Nov 2016 20:59:35 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C885FE03; Thu, 10 Nov 2016 20:59:34 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: by mail-it0-x231.google.com with SMTP id q124so68593371itd.1; Thu, 10 Nov 2016 12:59:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=4lRNwk/roIeD8gB9Q3ApjOw8qOtZMqnuklaA6yTqQPw=; b=VYj8a7osZ/Bv2ajg5Bitf45w52jZh+ZxiNV+rluGcW0+w22+gPLVGaNaVelQQHlaUK 4KwmaKUbdhQ1J33XKz4CXSqTS+wEXfbgBD0hf4GyolXEZZDkSGRKtqgvkN/9+vh+9vwG 5g1SkSdSNrzLYMeLXLHHgZ6ahcW9CDfkKCAB51y08gJjqKvH/6IuK2+mxZ4umHVCL0IT dVMD83oxfwmNpBH2TPZ8W/ycVny7DCzhxsIb2HI0M8r7cfcQIdG/4mduZ7GQT9wbOoX2 lX0RZwemSf8sBtegBi2aJRpED1+zR9wpDLafZozRsqO2zA0zcAJsC/tHOazq3U8pUFky MJlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4lRNwk/roIeD8gB9Q3ApjOw8qOtZMqnuklaA6yTqQPw=; b=Tc/SjhJfB8Ab+LFJ2Iw6NfwpYV38jwpj8RCfDumscutrEYGmU6LF/+Em/xN7/LARB6 9Ll4wxouF+4K5bdscJV7LwLpFpmyZ09Ip4NIp+ggUma/6GCq5cxROwjrhQOWZo0v9Utp eUdvkkP6BG3zQLmqdpcg6EBcMPTvrNQaMdIMqjBpv8S7ScZ6C/N+2i1+l28UYjrCuAuB URJ4X5bAUdn4sJu7m3FhKzbI5xb43Jaqy0coEF4j9LR4J6vhndtl1FpGr5pYt3gMRzmR Z20lDZqBOPph8XqCRgYKV2YulePJA//hZ5OqQguA8LijVoBy2A643UuM6pGQuolabzQA AQHg== X-Gm-Message-State: ABUngvc4LjnqgrXFVCxgkSbrGv5ngJxsdrUk+AaUrdG7yFosYQRDSBZQ90OO2W7iq/cB5mYjqySllzgc+XAe6w== X-Received: by 10.107.173.195 with SMTP id m64mr8008376ioo.188.1478811573962; Thu, 10 Nov 2016 12:59:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.87.67 with HTTP; Thu, 10 Nov 2016 12:59:33 -0800 (PST) From: Subbsd Date: Thu, 10 Nov 2016 23:59:33 +0300 Message-ID: Subject: EFI_STAGING_SIZE - what exactly is limited and trouble with high value To: freebsd-hackers , Will Andrews Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 20:59:35 -0000 Hello, After https://lists.freebsd.org/pipermail/svn-src-stable-11/2016-September/000503.html commits and with same reason as https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209944 ( i have mfs-based images more than ~512M size) , I tried to increase EFI_STAGING_SIZE up to 1024 ( my server have 16 GB RAM ) and got a system hang with follow message: -- Booting ... Start @ 0xffffffff80301000 ... Efi framebuffer information: addr, size 0xe9000000, 0x500000 dimensions 1280 x 1024 stride 1280 masks 0x00ff0000 0x0000ff00, 0x000000ff, 0xff000000 --- If i reduce EFI_STAGING_SIZE to 768, system booted as usual. Can someone explain -- what exactly is limited EFI_STAGING_SIZE in 1GB, which leads boot to freeze ? This is normal ? And why this value can not be dependent from hw.physmem ? We increase EFI_STAGING_SIZE by 16 Megabytes, from 48 to 64, but i'm sure, and some may have more than 64 MB From owner-freebsd-hackers@freebsd.org Thu Nov 10 21:02:45 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07270C3A519 for ; Thu, 10 Nov 2016 21:02:45 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFF19224; Thu, 10 Nov 2016 21:02:44 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: by mail-it0-x233.google.com with SMTP id e187so285870807itc.0; Thu, 10 Nov 2016 13:02:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=aeAB1y3Jms9nCHIRbpBxC1vneQ+jKtNmZ29abv6y618=; b=F7D2QR71Uer8lH09OHMdaf7C0qE3qPW7labuZdfe8xTYcIb2iCS5c1caqt+EAP/MMe 760Nw+dfJGfZBRIm6s5kyPlPzOOc8gRGLO+vUAuilSpjPYA7KgnGHeJXKUy5wep2BPiR 0M6p5AvOHKR9Xn5Aa6XhYoT8vGI7PktEDZBh7Td44uQMWfPqaCcEl2vHDKlievxbQ4ie nr6gPMQec3qfSoaUn6SOEzq9LEDF/3BN/JngEKC6Zb/DNOr6Um8JMp0xAB58UH30sjAQ Ccgpj6UkvzAPrdHIZ+0KfTnFSEAgjQUh+VGFHL7ejvLMMmCtAMiyHIcqvXIjL52sv8Uv Zczg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=aeAB1y3Jms9nCHIRbpBxC1vneQ+jKtNmZ29abv6y618=; b=YtU+lWBAUBU1SbdJTDv+RgkSv5GO973dWQthf0JeO/hJ+xLmwabtPCvWbu9Tn8BzIQ sBF2djZjkQQ4r4XBVBHZNuwD6l19WPTS4zeVFyStp1xLTFQHry4Xx4mAgQAwjFoU8HXo 3l4I6mwHzN0pbg3RCnbQdO8Qa7/70cd3Cm6LfA20dzKcE9WfgCUvrLW+f7IzoIHM82N3 qDUlBZZyfKh2+6IxqVVbC5LPAbs2vesKn6OMtOJDxE612GmSmJ4n3tTJW+nIASPvTgBJ qcdKn84yKQUed9mOEgDyjghOvOqjKVPYJRMAZsTgXgHRzTrKs1a2f4PtO/scBnNqgnpT Z8Ug== X-Gm-Message-State: ABUngvebyHeGwlnuddXTJBY+WnbNOYbtNGckmNxEvjBq7BLWddto+L+AaYi39EebVIuwqOxVX0LtkH+UK/VODA== X-Received: by 10.36.181.83 with SMTP id j19mr7476118iti.13.1478811761987; Thu, 10 Nov 2016 13:02:41 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.87.67 with HTTP; Thu, 10 Nov 2016 13:02:41 -0800 (PST) In-Reply-To: References: From: Subbsd Date: Fri, 11 Nov 2016 00:02:41 +0300 Message-ID: Subject: Re: EFI_STAGING_SIZE - what exactly is limited and trouble with high value To: freebsd-hackers , Will Andrews Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 21:02:45 -0000 On Thu, Nov 10, 2016 at 11:59 PM, Subbsd wrote: > Hello, > > After https://lists.freebsd.org/pipermail/svn-src-stable-11/2016-September/000503.html > commits and with same reason as > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209944 ( i have > mfs-based images more than ~512M size) , I tried to increase > EFI_STAGING_SIZE up to 1024 ( my server have 16 GB RAM ) and got a > system hang with follow message: > -- > Booting ... > Start @ 0xffffffff80301000 ... > Efi framebuffer information: > addr, size 0xe9000000, 0x500000 > dimensions 1280 x 1024 > stride 1280 > masks 0x00ff0000 0x0000ff00, 0x000000ff, 0xff000000 > --- > > If i reduce EFI_STAGING_SIZE to 768, system booted as usual. > > Can someone explain -- what exactly is limited EFI_STAGING_SIZE in > 1GB, which leads boot to freeze ? This is normal ? And why this value > can not be dependent from hw.physmem ? > > We increase EFI_STAGING_SIZE by 16 Megabytes, from 48 to 64, but i'm > sure, and some may have more than 64 MB Forget to attach screenshot with EFI_STAGING_SIZE after magic 768 value: http://pasteboard.co/pGgkKHOO7.jpg From owner-freebsd-hackers@freebsd.org Thu Nov 10 21:21:35 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3C79C3AB5E for ; Thu, 10 Nov 2016 21:21:35 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C5FBECC for ; Thu, 10 Nov 2016 21:21:34 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mandree.no-ip.org ([77.182.233.153]) by mail.gmx.com (mrgmx101 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M7TQZ-1czg4C0wOV-00xLdk; Thu, 10 Nov 2016 22:21:23 +0100 Received: from [IPv6:::1] (localhost6.localdomain6 [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id E8AE223D87C; Thu, 10 Nov 2016 22:21:18 +0100 (CET) Subject: sbrk(0) replacement for memory resource tracking? (was: [linimon@FreeBSD.org: svn commit: r425823 - in head: benchmarks/stress-ng cad/cider cad/ngspice_rework databases/mariadb100-server databases/mariadb101-server databases/mariadb55-server databases/virtuoso devel/ace deve...]) To: freebsd-hackers@FreeBSD.org References: <20161110012624.GA23701@lonesome.com> From: Matthias Andree Cc: Mark Linimon Message-ID: Date: Thu, 10 Nov 2016 22:21:18 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161110012624.GA23701@lonesome.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K0:dZD0JnCQxdJ9fw/sdT5lu6TzesQYFuJyYGS57nofBNPL5OpjN/8 7jvQoxr1ZsDKG9Mh0+DEzafc46NZwUSSbQgfIF0HYoOwntjx3zJ2X8km0RJGFsA07FGmk5r FiS8goHgq/wcwZj6Y3OWP75ALwgt2f41bM+cQ0CZTWoGb5YM7/2DiYCHbiiO1eyRwkA3vVt d6jGY7uwe7ArutP5G5M+g== X-UI-Out-Filterresults: notjunk:1;V01:K0:P5tM6vyMqGo=:rDoNZZzR2n/UyZYurPhVbp tl9F1HAMZAMzf2C9fm8+fOa6dwGjS7Hz0KJkNpAXA8UCzHbrf4RVyuJoBTZ0Bn9bb+2vWuggN Syb0aSkPcD6YeLpK3AuJuc+9YvtEEdpSobVExKaCDt5Ci2gqdyUbgD2nJb+DDVaGvaxMnsty+ j/GJWL+MEKIgILx5Lz3tx7bbhSwITACKrq5Xq32+4RJgCgx5KpId2jQ7GC8UpEV/8m3eLaLO/ 6TpDtsvrs/10Aw20dBa2PmSfNi3+mCfSWdb7reUG7cCl7oqDaHuiTnYvsdTk5Yz2vJEn8RgJo ++xYfMZb9F07zm2Ns2A/ULV1Tz1+8jU2lmG3KI9et8wNcHEA5YpKnsp2FSiXUC+5leEH1TkYs 0KhAE76BXx3RJPmQaS0lz8arKAQLlcKWKChJhnEV/8sh1gKQs8DLqYnyc9Uy6GorAKEzJ86XJ dPMzqvybDUZQdRAXcVBD38UHkhzfzZ+TtLUjVgGnttqvdkVxbBuizW3JDuoyg/ikQWn/Zsur8 Nhf04j5UIIA08lY3u4iECr0aKWkP5fO3Vshb0faj3aGkzwpE1S5F36l1q1FyG7PWJ6FfA2/7A Edzvqqa3Q7gr0mq4B/3rcQAJiEHLN5t606Gckb41z/B31tcLDjyX1zD9AmCTxj4BX9FMeqnwj GPskUhWRhih+se7tMRHHQ4t9Ck2iI4jjR5NWcqPF1q1J/5Bd/BTrj7hVWMtlYRQYrrRk2avde 2VpCYN5cp9pkAULEQFP01jJ7K2yaFJEAPXFgn8/Y81DplQ2jsBtN1HL5uZe3KBFxpj6d9FOUM YpGlojR X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 21:21:35 -0000 Am 10.11.2016 um 02:26 schrieb Mark Linimon: > FYI. Unfortunately I do not know what the generic fix is yet. But at > least this will prevent the package builders from wasting time right no= w. >=20 > I will try to keep the following page updated as I learn more: >=20 > https://wiki.freebsd.org/PortsBrokenWithSbrk >=20 > (oops, I forgot I have not put in the proper logfile URLs yet. Let me > get started on that.) >=20 > mcl Please help me understand the issue, and if by adding one or two introductory paragraphs to the Wiki. To me it looks like the sbrk() function is going away from our base system underneath a stable 11-* branch. If that is true, I'll have to object to that and request sbrk() be put back, we add a deprecation notice now (if necessary via errata notice) and pull it only from FreeBSD= 12. OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and only to track its resource usage. I'll be happy to help porting to something else that serves the same purpose, aka "how much memory am I using" - but what would that be? >=20 > Modified: head/sysutils/e2fsprogs/Makefile > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/sysutils/e2fsprogs/Makefile Thu Nov 10 01:08:44 2016 (r425822) > +++ head/sysutils/e2fsprogs/Makefile Thu Nov 10 01:21:43 2016 (r425823) > @@ -14,6 +14,8 @@ LICENSE=3D GPLv2 > =20 > PORTSCOUT=3D ignore # cannot handle the version in the directory > =20 > +BROKEN_aarch64=3D Fails to link: missing sbrk > + > USES=3D cpe gmake pkgconfig tar:xz > CPE_VENDOR=3D e2fsprogs_project > USE_CSTD=3D gnu99 >=20 From owner-freebsd-hackers@freebsd.org Thu Nov 10 22:01:04 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2730C3B068 for ; Thu, 10 Nov 2016 22:01:04 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-2.mit.edu (dmz-mailsec-scanner-2.mit.edu [18.9.25.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84198BC2 for ; Thu, 10 Nov 2016 22:01:04 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190d-49bff70000007435-97-5824ecea06de Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 64.21.29749.AECE4285; Thu, 10 Nov 2016 16:55:55 -0500 (EST) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id uAALtrlT017015; Thu, 10 Nov 2016 16:55:53 -0500 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id uAALtn4b018419 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 10 Nov 2016 16:55:52 -0500 Date: Thu, 10 Nov 2016 15:55:50 -0600 From: Benjamin Kaduk To: Matthias Andree Cc: freebsd-hackers@FreeBSD.org, Mark Linimon Subject: Re: sbrk(0) replacement for memory resource tracking? (was: [linimon@FreeBSD.org: svn commit: r425823 - in head: benchmarks/stress-ng cad/cider cad/ngspice_rework databases/mariadb100-server databases/mariadb101-server databases/mariadb55-server databases/virtuoso devel/ace deve...]) Message-ID: <20161110215549.GL91607@kduck.kaduk.org> References: <20161110012624.GA23701@lonesome.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrMIsWRmVeSWpSXmKPExsUixG6novv6jUqEwZ4NahbbN/9jtLi1fimr Rfv7OWwOzB4zPs1n8fjwMc6jafcDxgDmKC6blNSczLLUIn27BK6MdXdeMRdc5q04dLedpYHx AlcXIyeHhICJxNd5y1m7GLk4hATamCSu3L7ACJIQEtjIKHH3CQdE4iqTRPv5LcwgCRYBVYl3 D44zgdhsAioSDd2XweIiAjoST7bMBLOZBVwkGm5fYwdpFhZYyyTx524TO0iCF2hd7+bpTBAb EiWWzVzNBhEXlDg58wkLRLOWxI1/L4FqOIBsaYnl/zhATE4BK4n3p4tBKkQFlCUaZjxgnsAo MAtJ8ywkzbMQmhcwMq9ilE3JrdLNTczMKU5N1i1OTszLSy3SNdLLzSzRS00p3cQICltOSd4d jP/ueh1iFOBgVOLhlehUiRBiTSwrrsw9xCjJwaQkyqs4HSjEl5SfUpmRWJwRX1Sak1p8iFGC g1lJhNfzFVCONyWxsiq1KB8mJc3BoiTO+9/ta7iQQHpiSWp2ampBahFMVoaDQ0mC9/lroEbB otT01Iq0zJwShDQTByfIcB6g4QdAaniLCxJzizPTIfKnGBWlxHnZQRICIImM0jy4XlBakcje X/OKURzoFWFed2CSEeIBpiS47ldAg5mABs+IAxtckoiQkmpgnGm0anbNgumdKgUtvB6njxzL UvvV+a+B++DTmcEZlwvv5IeEvtnSFHkk5dw5ocl+IpbvygNLpiwy03/peHLCEZOOtFX+ip9/ OXyfvGz5O7ePrxv73jEk/l/xVSqLIW+XYNCCtRGB7F5rtatvlU0yWW3LkumSW1ejICX8YtJq JrvJDkzr7voeVWIpzkg01GIuKk4EADQesgUGAwAA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 22:01:04 -0000 On Thu, Nov 10, 2016 at 10:21:18PM +0100, Matthias Andree wrote: > Am 10.11.2016 um 02:26 schrieb Mark Linimon: > > FYI. Unfortunately I do not know what the generic fix is yet. But at > > least this will prevent the package builders from wasting time right now. > > > > I will try to keep the following page updated as I learn more: > > > > https://wiki.freebsd.org/PortsBrokenWithSbrk > > > > (oops, I forgot I have not put in the proper logfile URLs yet. Let me > > get started on that.) > > > > mcl > > Please help me understand the issue, and if by adding one or two > introductory paragraphs to the Wiki. Looks like r300303 is the relevant one for aarch64 (RISC-V got a similar treatment later?). > To me it looks like the sbrk() function is going away from our base > system underneath a stable 11-* branch. If that is true, I'll have to > object to that and request sbrk() be put back, we add a deprecation > notice now (if necessary via errata notice) and pull it only from FreeBSD12. At the time I somehow convinced myself that it had never been in a stable release and was thus okay, but maybe I'm misremembering. Hmm, or maybe it is okay for a tier-2 architecture [in the mind of the committer, not necessarly me, to be clear]. > OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and > only to track its resource usage. I'll be happy to help porting to Same for zephyr. > something else that serves the same purpose, aka "how much memory am I > using" - but what would that be? I think there isn't really a drop-in replacement. N.B. that the number from sbrk(0) has been meaningless for quite some time, since jemalloc uses mmap to get more space. -Ben From owner-freebsd-hackers@freebsd.org Thu Nov 10 22:27:03 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26B64C3B91E for ; Thu, 10 Nov 2016 22:27:03 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9549CE73 for ; Thu, 10 Nov 2016 22:27:01 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mandree.no-ip.org ([77.182.233.153]) by mail.gmx.com (mrgmx103 [212.227.17.168]) with ESMTPSA (Nemesis) id 0M6ioC-1cypiX3g16-00wXgf; Thu, 10 Nov 2016 23:26:42 +0100 Received: from [IPv6:::1] (localhost6.localdomain6 [IPv6:::1]) by apollo.emma.line.org (Postfix) with ESMTP id D42BC23D87C; Thu, 10 Nov 2016 23:26:37 +0100 (CET) Subject: Re: sbrk(0) replacement for memory resource tracking? To: Benjamin Kaduk References: <20161110012624.GA23701@lonesome.com> <20161110215549.GL91607@kduck.kaduk.org> Cc: "freebsd-hackers@FreeBSD.org" , Mark Linimon From: Matthias Andree Message-ID: Date: Thu, 10 Nov 2016 23:26:37 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161110215549.GL91607@kduck.kaduk.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K0:wZOcRZzytzSMrDqUaNAFgPfZYaSz7LrmAGN+KqmQQX4aM136Z6r v6pBPxybkDnijjjunQp6CD1+JfqdACKboV+x5aQDxxS9avXJOfdyvGRVu+kWUBsO7irCEic ZJK4KhkHy8bEAadqYFM8e9yFRy/haMsCTYYM55bkxKkDqWKV6sQjHUQ9ZmV4RTVucC6u11A TDtCseQ6xMGIZAF0BkPHw== X-UI-Out-Filterresults: notjunk:1;V01:K0:u9HThrfUORk=:MavIeAZMCotgofVMXg5GB6 +lfL61R2s+I1Qla6M75YkIBQC3U2IsLfP2nqrT97ZQOtNdfxk2hx+beZ84Bh6vI7JiodEwlBr 3okox19mJJ1GWxzwZJckV7vtNKT34pQNVFCB1J9xXjEf86Un9NjN7cRvSthJuprNckNdvt4z5 fPy9tLQF4DMKro95Dzz8J5QvFOyVXu7b/fVoG2w0rVD3urFEG74UPYioCfQLRC0DnOMEsGQEw zC1G8YcfF2JEQatSCzVge/xDabEtFilG8HJUZhZqnDfwqrgrUzojE6bp6jjtCJb5GxqoU49kM QfUZ8P4x7czEIydJ5k1gcB+e8f8/a06MApgV9h4+LdxVXCt0NfTqzxbE8mfZQTf1QHstCN5l6 WwGTM5QKZXnB1C+G5HNDqj4/csYXlIa3CCPFJ3PV5S5MtlBnJFSGyIQZW1nJXd9TW3sSMOeLI q163x+rTdNpK2l6ZjViqDBSuZTzdw+fDjBHF2ubpNGL1kcrOS6KsU086dNu3gj1q48H/n9WsA PdAtAitR+dzbgEkuPZ3C8JOHrk2oZ3KbG3H+61VPRxNiqaf4TxHmEWuktBtVBjDrqo6B+ojkX WB6qEk/isFAAQL2lQRf36lS9FEZAP4UZweqy0V+FOL/IByOrWPD+uQp1IZMn0f/oQvZR0jw2e AO/oFnjoHJl70qcLnKWS3TpcfmObT0z71P233WHdz4nL/CROVWERiv4X4eBG6YR9BYLW3AsAx ANnXvQho2kJloFJ+qCK37LvFsU67wCLL+v7DgYPdwR1hIf5Fi3NFU0wtMfEjI6/i+KXasj/Tr 6rvvOeR X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 22:27:03 -0000 Am 10.11.2016 um 22:55 schrieb Benjamin Kaduk: > On Thu, Nov 10, 2016 at 10:21:18PM +0100, Matthias Andree wrote: >> Am 10.11.2016 um 02:26 schrieb Mark Linimon: >>> FYI. Unfortunately I do not know what the generic fix is yet. But a= t >>> least this will prevent the package builders from wasting time right = now. >>> >>> I will try to keep the following page updated as I learn more: >>> >>> https://wiki.freebsd.org/PortsBrokenWithSbrk >>> >>> (oops, I forgot I have not put in the proper logfile URLs yet. Let m= e >>> get started on that.) >>> >>> mcl >> Please help me understand the issue, and if by adding one or two >> introductory paragraphs to the Wiki. > > Looks like r300303 is the relevant one for aarch64 (RISC-V got a simila= r treatment > later?). > >> To me it looks like the sbrk() function is going away from our base >> system underneath a stable 11-* branch. If that is true, I'll have to= >> object to that and request sbrk() be put back, we add a deprecation >> notice now (if necessary via errata notice) and pull it only from Free= BSD12. > At the time I somehow convinced myself that it had never been in a stab= le > release and was thus okay, but maybe I'm misremembering. Hmm, or maybe= it > is okay for a tier-2 architecture [in the mind of the committer, not ne= cessarly > me, to be clear]. > >> OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and= >> only to track its resource usage. I'll be happy to help porting to > Same for zephyr. > >> something else that serves the same purpose, aka "how much memory am I= >> using" - but what would that be? > I think there isn't really a drop-in replacement. N.B. that the number= > from sbrk(0) has been meaningless for quite some time, since jemalloc > uses mmap to get more space. OK. So the quick and dirty way to re-enable e2fsprogs on those architectures whilst scrapping any memory statistics would be to #define sbrk(a) (a) which would just invalidate stats, providing the application handles bogus data. Other than that, it would seem that mallctl("epoch", ...) to synch up stats, and mallctl("stats.active", ...) or perhaps or "stats.mapped" gets me close to what comparing sbrk(0) over process lifetime would have achieved, wouldn't it? This is assuming sbrk() had page granularity anyhow and stats.active provides exactly that (gross memory allocated). Possibly this also wants mallctlnametomib and mallctlbymib for optimization if called often. Right? From owner-freebsd-hackers@freebsd.org Thu Nov 10 23:08:34 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ABCCC3AC54 for ; Thu, 10 Nov 2016 23:08:34 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E8B6BB7 for ; Thu, 10 Nov 2016 23:08:33 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 16A4E5A9F13; Thu, 10 Nov 2016 23:08:26 +0000 (UTC) Date: Thu, 10 Nov 2016 23:08:26 +0000 From: Brooks Davis To: Benjamin Kaduk Cc: Matthias Andree , freebsd-hackers@FreeBSD.org, Mark Linimon Subject: Re: sbrk(0) replacement for memory resource tracking? (was: [linimon@FreeBSD.org: svn commit: r425823 - in head: benchmarks/stress-ng cad/cider cad/ngspice_rework databases/mariadb100-server databases/mariadb101-server databases/mariadb55-server databases/virtuoso devel/ace deve...]) Message-ID: <20161110230825.GA94990@spindle.one-eyed-alien.net> References: <20161110012624.GA23701@lonesome.com> <20161110215549.GL91607@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VS++wcV0S1rZb1Fb" Content-Disposition: inline In-Reply-To: <20161110215549.GL91607@kduck.kaduk.org> User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Nov 2016 23:08:34 -0000 --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 10, 2016 at 03:55:50PM -0600, Benjamin Kaduk wrote: > On Thu, Nov 10, 2016 at 10:21:18PM +0100, Matthias Andree wrote: > > Am 10.11.2016 um 02:26 schrieb Mark Linimon: > > > FYI. Unfortunately I do not know what the generic fix is yet. But at > > > least this will prevent the package builders from wasting time right = now. > > >=20 > > > I will try to keep the following page updated as I learn more: > > >=20 > > > https://wiki.freebsd.org/PortsBrokenWithSbrk > > >=20 > > > (oops, I forgot I have not put in the proper logfile URLs yet. Let me > > > get started on that.) > > >=20 > > > mcl > >=20 > > Please help me understand the issue, and if by adding one or two > > introductory paragraphs to the Wiki. >=20 >=20 > Looks like r300303 is the relevant one for aarch64 (RISC-V got a similar = treatment > later?). >=20 > > To me it looks like the sbrk() function is going away from our base > > system underneath a stable 11-* branch. If that is true, I'll have to > > object to that and request sbrk() be put back, we add a deprecation > > notice now (if necessary via errata notice) and pull it only from FreeB= SD12. >=20 > At the time I somehow convinced myself that it had never been in a stable > release and was thus okay, but maybe I'm misremembering. Hmm, or maybe it > is okay for a tier-2 architecture [in the mind of the committer, not nece= ssarly > me, to be clear]. It's never been in a release on either architecture. > > OTOH, e2fsprogs uses only sbrk(0) to track its overall memory use, and > > only to track its resource usage. I'll be happy to help porting to >=20 > Same for zephyr. >=20 > > something else that serves the same purpose, aka "how much memory am I > > using" - but what would that be? >=20 > I think there isn't really a drop-in replacement. N.B. that the number > from sbrk(0) has been meaningless for quite some time, since jemalloc > uses mmap to get more space. As you point out, sbrk(0) does not show memory usage in a useful way for normal programs. If you enable ASR/ASLR it gets even more useless. An implementation that returns NULL should satisfy most code that abuses it for memory use and provide no less value. -- Brooks --VS++wcV0S1rZb1Fb Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJYJP3pAAoJEKzQXbSebgfA9REH/2okeLCzM2ySFrUIdyqKTKxa fsYE9JgIpUe8Ay4FHo3Fau3GCjajOYmYWToLZ/Wp+HhRdE+WFbx9hYWaL8/AnI7E 99GDRaCEDUJWsUNYEW5GUJeyNpBsT5M1TMdKPqoOI9VXMrY0pH3nZT2pVVqDVN13 ghOW0RcvWjUeV9UVSSUEs0frXoKxrTz+vAvvLqD11+ifSPoYCUcdtwfUJYWu2Dt6 eKWoBvcvlR1cWI3RoQzHnvnjCmW17XAd52FuJUsOzJPBlFPmEFiBW6zjxXFmXJoJ VGdpQO/gwq+g67GjJL/b7Sx2hBq3xENFJJbIUpz2Ic/Zo31Lj75C7U6ZgB9PEGQ= =O7U5 -----END PGP SIGNATURE----- --VS++wcV0S1rZb1Fb-- From owner-freebsd-hackers@freebsd.org Fri Nov 11 12:26:59 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55F06C3B1F9; Fri, 11 Nov 2016 12:26:59 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 7945E18D8; Fri, 11 Nov 2016 12:26:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA24110; Fri, 11 Nov 2016 14:26:56 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c5Av1-0007xj-Vj; Fri, 11 Nov 2016 14:26:56 +0200 To: FreeBSD Current , FreeBSD Hackers From: Andriy Gapon Subject: firewire panic Message-ID: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> Date: Fri, 11 Nov 2016 14:25:59 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 12:26:59 -0000 Does anyone still use firewire or hack on code? I've recently tried to connect an external firewire HDD enclosure and got this: Unread portion of the kernel message buffer: lock order reversal: 1st 0xfffff8002b0f2f48 sbp (sbp) @ /usr/src/sys/kern/kern_mutex.c:158 2nd 0xfffff8003f86f460 CAM device lock (CAM device lock) @ /usr/src/sys/cam/scsi/scsi_xpt.c:2323 stack backtrace: #0 0xffffffff8068d220 at witness_debugger+0x70 #1 0xffffffff8068cd81 at witness_checkorder+0x7a1 #2 0xffffffff8061bab8 at __mtx_lock_flags+0x98 #3 0xffffffff802b663d at scsi_scan_lun+0x11d #4 0xffffffff802b51f7 at scsi_action+0x67 #5 0xffffffff802a756a at xpt_action+0x1a #6 0xffffffff8047459e at sbp_cam_scan_target+0xce #7 0xffffffff8064f856 at softclock_call_cc+0x2d6 #8 0xffffffff8064fbf7 at softclock+0x47 #9 0xffffffff80602190 at intr_event_execute_handlers+0xe0 #10 0xffffffff806029ec at ithread_execute_handlers+0x2c #11 0xffffffff8060285b at ithread_loop+0x5b #12 0xffffffff805ff72f at fork_exit+0xdf #13 0xffffffff8082483e at fork_trampoline+0xe lock order reversal: panic: mutex sbp not owned at /usr/src/sys/dev/firewire/sbp.c:967 cpuid = 2 curthread: 0xfffff8000ada5000 stack: 0xfffffe0504ded000 - 0xfffffe0504df1000 stack pointer: 0xfffffe0504df0a00 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff80420bbb = db_trace_self_wrapper+0x2b/frame 0xfffffe0504df0930 kdb_backtrace() at 0xffffffff80670359 = kdb_backtrace+0x39/frame 0xfffffe0504df09e0 vpanic() at 0xffffffff8063986c = vpanic+0x14c/frame 0xfffffe0504df0a20 panic() at 0xffffffff806395b3 = panic+0x43/frame 0xfffffe0504df0a80 __mtx_assert() at 0xffffffff8061c40d = __mtx_assert+0xed/frame 0xfffffe0504df0ac0 sbp_cam_scan_lun() at 0xffffffff80474667 = sbp_cam_scan_lun+0x37/frame 0xfffffe0504df0af0 xpt_done_process() at 0xffffffff802aacfa = xpt_done_process+0x2da/frame 0xfffffe0504df0b30 xpt_done_td() at 0xffffffff802ac2e5 = xpt_done_td+0xd5/frame 0xfffffe0504df0b80 fork_exit() at 0xffffffff805ff72f = fork_exit+0xdf/frame 0xfffffe0504df0bf0 fork_trampoline() at 0xffffffff8082483e = fork_trampoline+0xe/frame 0xfffffe0504df0bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- -- Andriy Gapon From owner-freebsd-hackers@freebsd.org Sat Nov 12 09:10:26 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71474C28228; Sat, 12 Nov 2016 09:10:26 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F16861537; Sat, 12 Nov 2016 09:10:25 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id u144so2302665wmu.0; Sat, 12 Nov 2016 01:10:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=u0eIQGomDTJydbJospEPAr8toYd46akfulqWnnvS630=; b=Sc5lc0426nYxKwhEAxI1FEdSOw4e6PG3k3gpQuSxrkRpjbi34/LBvuvNFHuNqljrGe Iw3j48zxumJNFnJnHV9mFrlGvNbAIpAYaczh/OC/Aenh3B86NjELdg303iAE/eSzx1fj vv+wE2X9cczgCcBpocNQNekUPD5wZq7t8RDeT7FBL7nAToAE3N16XW6XEkib5viPlCXR vy55UmCOqlBKqbHIOUVJ7dZ3TIRzSQxlFBTasQprG+rCxiTYrBxXd3Lhl6KCWv6qyCZI BWNVH3B9g4qorLyBdiYZhfl78NoThkYgHK9C4haAcR2KVF7v5D4Qh/sdJgF8pyERynsp CbNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=u0eIQGomDTJydbJospEPAr8toYd46akfulqWnnvS630=; b=dKARPLQrWJ/j+A6DPfnSMkMXtuGSoV/ttDSTg0OdlO1F6xGg4H2hhAo+4BdtzHclkW SObIja5kY/uF8x2etAW/3TvEjCS1WP48aN56vefCMeT/zQIcy1zjvwiVTYO9O+kWCvRr bIi39wwbXpHdUre5sJ7NRlEo/TjDEgmr136aBNRRkMy1UJpj9o1UAgkJfI5jjJyxH0tH mWKZ6+vmSh7lqXyxndXo4imfTg4tkhORfwX7n8qvnSudyEmYYxZ36p1cF4kYupmwBMjP xafTO6KCRXWvtIiMK8PvQkJI+cu2E+VlpkphYZf62WzBWErahRKNMEKiRXqMBVYOof3j bYJQ== X-Gm-Message-State: ABUngvfsE+BMQXFXPTw/HLOXE3wW/nKhRp86aPahWmw9Urg78xGtsG+Yc0eUdTtKn9mxcA== X-Received: by 10.28.156.10 with SMTP id f10mr1980737wme.63.1478941824312; Sat, 12 Nov 2016 01:10:24 -0800 (PST) Received: from ernst.home (p578E14E0.dip0.t-ipconnect.de. [87.142.20.224]) by smtp.gmail.com with ESMTPSA id f10sm15946645wjl.28.2016.11.12.01.10.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 12 Nov 2016 01:10:23 -0800 (PST) Date: Sat, 12 Nov 2016 10:10:20 +0100 From: Gary Jennejohn To: Andriy Gapon Cc: FreeBSD Current , FreeBSD Hackers Subject: Re: firewire panic Message-ID: <20161112101020.0ddf33ba@ernst.home> In-Reply-To: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> References: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.29; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 09:10:26 -0000 On Fri, 11 Nov 2016 14:25:59 +0200 Andriy Gapon wrote: > Does anyone still use firewire or hack on code? I've recently > tried to connect an external firewire HDD enclosure and got > this: > Works for me on FreeBSD 12.0-CURRENT #14 r308123M amd64. Nov 12 10:05:25 ernst kernel: firewire0: Nov 12 10:05:25 ernst kernel: bus manager 1 Nov 12 10:05:25 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0: ordered:1 type:0 EUI:0001a3000005ee4c node:0 speed:2 maxrec:8 Nov 12 10:05:25 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0 'Genesys ' '' '' Nov 12 10:05:26 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 Nov 12 10:05:26 ernst kernel: da0: Fixed Direct Access SCSI device Nov 12 10:05:26 ernst kernel: da0: 50.000MB/s transfers Nov 12 10:05:26 ernst kernel: da0: 305245MB (625142448 512 byte sectors) Nov 12 10:05:26 ernst kernel: da0: quirks=0x2 [snip kernel backtrace] -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Sat Nov 12 09:34:21 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E45CC28C71; Sat, 12 Nov 2016 09:34:21 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 396761577; Sat, 12 Nov 2016 09:34:21 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm0-x242.google.com with SMTP id u144so2389026wmu.0; Sat, 12 Nov 2016 01:34:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=mU0TE7kzzdsWjZWY6TmBtJbXOcLIGzH6qRthYOpE9iY=; b=Tk4VGe732uAeZ4g9vj/EXLnbcUnn09rMRqoXUv4Odp7wm5ztksv6B5Is6t2emtDEqJ ORJME66KwYy/MvkPPXdfrzR4yq/6VHY6fH7yy61FxYbOQhQfuIOkXhWCv/n13r4340Jg 8LZqRbfvHG6NbNF0Ihbx9yBH7AdEn47Zf2ldu3nIHuo+KxXHF+AcmtNNmNHys3RuQU38 Zv8ppWI7yGrMDmvWjEcO/sbN52FWozizOHDseBUtjqXHRE0NGIY5ecruUcU5KEBQVfuG CShGwEo0kMOoMbL7rj/Tw5vuLaxy+ELW8LTBHkGt6P6BP2UEsLea95p9lt+sEKRe8iJL 70Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=mU0TE7kzzdsWjZWY6TmBtJbXOcLIGzH6qRthYOpE9iY=; b=KMFjwFa6c+BoL4G7vPW3nS0W8QLFotYlg/LIzSA9Pox35q19AS+IlFj+Q/heFfhXl3 bCJNFeuAXyCiZvvN1opr8b7p/I9dAPoZgHbISdPg53Op+w0HhbMBSCFIZaHKHFEERrlL VEPmvUd3aKRlL0cJxybCJdAqZK68ggxBjEDEmj3CcVGlemK7HhDk2eJL+PLFG+fDqfZQ DNEpWPJU+bT4fppZefDjhbjb9UzTF/T9jflHnvhhPSBmBD5Ci510RuxNKhr4vW38a6ga ZUfk1ee51kmho3NTW9b5gBD2lna9IgHntONhI1TQOEs1IUovts5NvnFsczKFIqphvwOp vUJw== X-Gm-Message-State: ABUngvfMCEh64FdmapxPbGurCbou/qmSpYLXaRYZXESJZ830IMy1wFOuRkDLOof1FVg9Lw== X-Received: by 10.28.55.203 with SMTP id e194mr2150890wma.97.1478943259379; Sat, 12 Nov 2016 01:34:19 -0800 (PST) Received: from ernst.home (p578E14E0.dip0.t-ipconnect.de. [87.142.20.224]) by smtp.gmail.com with ESMTPSA id n3sm16002641wjq.34.2016.11.12.01.34.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 12 Nov 2016 01:34:18 -0800 (PST) Date: Sat, 12 Nov 2016 10:34:17 +0100 From: Gary Jennejohn To: Andriy Gapon Cc: FreeBSD Current , FreeBSD Hackers Subject: Re: firewire panic Message-ID: <20161112103417.7ae9b439@ernst.home> In-Reply-To: <20161112101020.0ddf33ba@ernst.home> References: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> <20161112101020.0ddf33ba@ernst.home> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.29; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 09:34:21 -0000 On Sat, 12 Nov 2016 10:10:20 +0100 Gary Jennejohn wrote: > On Fri, 11 Nov 2016 14:25:59 +0200 > Andriy Gapon wrote: > > > Does anyone still use firewire or hack on code? I've recently > > tried to connect an external firewire HDD enclosure and got > > this: > > > > Works for me on FreeBSD 12.0-CURRENT #14 r308123M amd64. > > Nov 12 10:05:25 ernst kernel: firewire0: > Nov 12 10:05:25 ernst kernel: bus manager 1 > Nov 12 10:05:25 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0: ordered:1 type:0 EUI:0001a3000005ee4c node:0 speed:2 maxrec:8 > Nov 12 10:05:25 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0 'Genesys ' '' '' > Nov 12 10:05:26 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 > Nov 12 10:05:26 ernst kernel: da0: Fixed Direct Access SCSI device > Nov 12 10:05:26 ernst kernel: da0: 50.000MB/s transfers > Nov 12 10:05:26 ernst kernel: da0: 305245MB (625142448 512 byte sectors) > Nov 12 10:05:26 ernst kernel: da0: quirks=0x2 > > [snip kernel backtrace] > FYI also works on FreeBSD 12.0-CURRENT r308568 amd64. Nov 12 10:24:31 ernst kernel: fwohci0: fwohci_intr_core: BUS reset Nov 12 10:24:31 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=3, non CYCLEMASTER mode Nov 12 10:24:31 ernst kernel: firewire0: 2 nodes, maxhop <= 1 cable IRM irm(0) (me) Nov 12 10:24:31 ernst kernel: firewire0: root node is not cycle master capable Nov 12 10:24:31 ernst kernel: firewire0: bus manager 0 Nov 12 10:24:31 ernst kernel: fwohci0: too many cycles lost, no cycle master present? Nov 12 10:24:31 ernst kernel: firewire0: split transaction timeout: tl=0x1 flag=0x04 Nov 12 10:24:31 ernst kernel: send: dst=0x01 tl=0x01 rt=0 tcode=0x4 pri=0x0 src=0x000 Nov 12 10:24:34 ernst kernel: fwohci0: fwohci_intr_core: BUS reset Nov 12 10:24:34 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000001, SelfID Count=4, CYCLEMASTER mode Nov 12 10:24:34 ernst kernel: firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) (me) Nov 12 10:24:34 ernst kernel: firewire0: bus manager 1 Nov 12 10:24:34 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0: ordered:1 type:0 EUI:0001a3000005ee4c node:0 speed:2 maxrec:8 Nov 12 10:24:34 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0 'Genesys ' '' '' Nov 12 10:24:35 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 Nov 12 10:24:35 ernst kernel: da0: Fixed Direct Access SCSI device Nov 12 10:24:35 ernst kernel: da0: 50.000MB/s transfers Nov 12 10:24:35 ernst kernel: da0: 305245MB (625142448 512 byte sectors) Nov 12 10:24:35 ernst kernel: da0: quirks=0x2 Nov 12 10:25:00 ernst kernel: fwohci0: fwohci_intr_core: BUS reset Nov 12 10:25:00 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=5, CYCLEMASTER mode Nov 12 10:25:00 ernst kernel: firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) Nov 12 10:25:00 ernst kernel: firewire0: bus manager 0 Nov 12 10:25:00 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 Nov 12 10:25:00 ernst kernel: da0: detached Nov 12 10:25:00 ernst kernel: (da0:sbp0:0:0:0): Periph destroyed -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Sat Nov 12 10:03:59 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06581C3B991 for ; Sat, 12 Nov 2016 10:03:59 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ADDEF15D9 for ; Sat, 12 Nov 2016 10:03:58 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bs.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1c5V9z-000Kh3-Lx for freebsd-hackers@FreeBSD.org; Sat, 12 Nov 2016 12:03:43 +0200 From: Daniel Braniss Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: ipmitool crashing, getpass() Message-Id: <65C0C11B-123E-4536-B7C1-2EBC0DDA37FE@cs.huji.ac.il> Date: Sat, 12 Nov 2016 12:03:43 +0200 To: Freebsd hackers list Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 10:03:59 -0000 Hi, I have no idea when this started to happen, but with a relative new = 11.3-stable, ipmitool core dumps. ipmi_main() calls getpass(=E2=80=A6) and coredumps trying to strdup the = password. My guess it=E2=80=99s not calling libc=E2=80=99s getpass(), because adding this patch t works: --- lib/ipmi_main.c~ 2016-05-06 17:48:54.000000000 +0300 +++ lib/ipmi_main.c 2016-11-12 11:55:39.396032000 +0200 @@ -97,6 +97,20 @@ =20 static struct ipmi_intf * ipmi_main_intf =3D NULL; =20 +#include +#include + + +static char * +getpass(const char *prompt) +{ + static char buf[ _PASSWORD_LEN + 1]; + =20 + if (readpassphrase(prompt, buf, sizeof(buf), RPP_ECHO_OFF)=3D=3D = NULL) + buf[0] =3D '\0'; + return(buf); +} + /* ipmi_password_file_read - Open file and read password from it * * @filename: file name to read from the code was cut-n-pasted from libc. cheers, danny From owner-freebsd-hackers@freebsd.org Sat Nov 12 10:15:19 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32B71C3BDFB for ; Sat, 12 Nov 2016 10:15:19 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E36221E4C for ; Sat, 12 Nov 2016 10:15:18 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from imac.bs.cs.huji.ac.il ([132.65.179.42]) by kabab.cs.huji.ac.il with esmtp id 1c5VL6-0005Rt-RK for freebsd-hackers@FreeBSD.org; Sat, 12 Nov 2016 12:15:12 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: ipmitool crashing, getpass() From: Daniel Braniss In-Reply-To: <65C0C11B-123E-4536-B7C1-2EBC0DDA37FE@cs.huji.ac.il> Date: Sat, 12 Nov 2016 12:15:12 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <65C0C11B-123E-4536-B7C1-2EBC0DDA37FE@cs.huji.ac.il> To: Freebsd hackers list X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 10:15:19 -0000 > On 12 Nov 2016, at 12:03 PM, Daniel Braniss = wrote: >=20 > Hi, > I have no idea when this started to happen, but with a relative new = 11.3-stable, ipmitool core dumps. s/11.3/10.3/ > ipmi_main() calls getpass(=E2=80=A6) and coredumps trying to strdup = the password. My guess it=E2=80=99s not > calling libc=E2=80=99s getpass(), because adding this patch t works: >=20 > --- lib/ipmi_main.c~ 2016-05-06 17:48:54.000000000 +0300 > +++ lib/ipmi_main.c 2016-11-12 11:55:39.396032000 +0200 > @@ -97,6 +97,20 @@ >=20 > static struct ipmi_intf * ipmi_main_intf =3D NULL; >=20 > +#include > +#include > + > + > +static char * > +getpass(const char *prompt) > +{ > + static char buf[ _PASSWORD_LEN + 1]; > + =20 > + if (readpassphrase(prompt, buf, sizeof(buf), RPP_ECHO_OFF)=3D=3D = NULL) > + buf[0] =3D '\0'; > + return(buf); > +} > + > /* ipmi_password_file_read - Open file and read password from it > * > * @filename: file name to read from >=20 > the code was cut-n-pasted from libc. >=20 > cheers, > danny >=20 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to = "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Sat Nov 12 15:58:52 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48FE4C3D8EA for ; Sat, 12 Nov 2016 15:58:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2D981C96 for ; Sat, 12 Nov 2016 15:58:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uACFwV1j059146 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 12 Nov 2016 17:58:32 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uACFwV1j059146 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uACFwU66059145; Sat, 12 Nov 2016 17:58:30 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Nov 2016 17:58:30 +0200 From: Konstantin Belousov To: Sebastian Huber Cc: FreeBSD Subject: Re: Should page allocator zero the pages for UMA? Message-ID: <20161112155830.GC54029@kib.kiev.ua> References: <5821DC2E.9020302@embedded-brains.de> <20161108145528.GN54029@kib.kiev.ua> <5822C615.3060100@embedded-brains.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5822C615.3060100@embedded-brains.de> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 15:58:52 -0000 On Wed, Nov 09, 2016 at 07:45:41AM +0100, Sebastian Huber wrote: > On 08/11/16 15:55, Konstantin Belousov wrote: > > On Tue, Nov 08, 2016 at 03:07:42PM +0100, Sebastian Huber wrote: > >> Hello, > >> > >> we use the FreeBSD network, USB and SD/MMC card stacks for the real-time > >> operating system RTEMS: > >> > >> https://git.rtems.org/rtems-libbsd > >> > >> I update currently from FreeBSD 9.3 to head. We use the UMA from FreeBSD > >> with a custom page allocator: > >> > >> https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-kernel-page.c > >> > >> The FreeBSD 9.3 based port worked well with uninitialized pages, e.g. > >> random or previous content. However, after the update to head I had to > >> zero initialize the pages. One issue was an incomplete > >> > >> struct inpcb { > >> [...] > >> struct inpcbport *inp_phd; /* (i/h) head of this list */ > >> #define inp_zero_size offsetof(struct inpcb, inp_gencnt) > >> inp_gen_t inp_gencnt; /* (c) generation count */ > >> struct llentry *inp_lle; /* cached L2 information */ > >> struct rwlock inp_lock; > >> rt_gen_t inp_rt_cookie; /* generation for route entry */ > >> union { /* cached L3 information */ > >> struct route inpu_route; > >> struct route_in6 inpu_route6; > >> } inp_rtu; > >> #define inp_route inp_rtu.inpu_route > >> #define inp_route6 inp_rtu.inpu_route6 > >> }; > >> > >> initialization. The initialization consists of two parts: > >> > >> static int > >> udp_inpcb_init(void *mem, int size, int flags) > >> { > >> struct inpcb *inp; > >> > >> inp = mem; > >> INP_LOCK_INIT(inp, "inp", "udpinp"); > >> return (0); > >> } > >> > >> /* > >> * Allocate a PCB and associate it with the socket. > >> * On success return with the PCB locked. > >> */ > >> int > >> in_pcballoc(struct socket *so, struct inpcbinfo *pcbinfo) > >> { > >> struct inpcb *inp; > >> int error; > >> > >> #ifdef INVARIANTS > >> if (pcbinfo == &V_tcbinfo) { > >> INP_INFO_RLOCK_ASSERT(pcbinfo); > >> } else { > >> INP_INFO_WLOCK_ASSERT(pcbinfo); > >> } > >> #endif > >> > >> error = 0; > >> inp = uma_zalloc(pcbinfo->ipi_zone, M_NOWAIT); > >> if (inp == NULL) > >> return (ENOBUFS); > >> bzero(inp, inp_zero_size); > >> inp->inp_pcbinfo = pcbinfo; > >> inp->inp_socket = so; > >> inp->inp_cred = crhold(so->so_cred); > >> inp->inp_inc.inc_fibnum = so->so_fibnum; > >> [...] > >> > >> This lets at least inp_route uninitialized leading to a crash during > >> destruction, e.g. > >> > >> if (inp->inp_route.ro_rt) { > >> RTFREE(inp->inp_route.ro_rt); > >> inp->inp_route.ro_rt = (struct rtentry *)NULL; > >> } > >> > >> uses uninitialized data. > >> > >> Did something in the page allocator change between FreeBSD 9.3 and > >> trunk, so that page are now zero initialized or is this a bug in > >> udp_inpcb_init()? > > Nothing guarantees that returned items are zeroed out automatically. > > Pages are not returned zeroed by VM page allocator, it might indicate > > that the allocated page happens to be zeroed by PG_ZERO flag. Unless > > M_ZERO is passed to uma_zalloc(), item is not zeroed. > > > > I am not sure, you did not demonstrated zone initialization, but it seems > > that your zone is having zone init function. If this is true, than I do not > > see why do you consider it possible to have items automatically zeroed at > > all. Uma calls the init function at the bulk allocation time, and constructor > > for each item returned by uma_zalloc(). No implicit zeroing occurs there. > > Now, I am a bit confused. We have > > /* > * Allocate a new slab for a keg. This does not insert the slab onto a > list. > * > * Arguments: > * wait Shall we wait? > * > * Returns: > * The slab that was allocated or NULL if there is no memory and the > * caller specified M_NOWAIT. > */ > static uma_slab_t > keg_alloc_slab(uma_keg_t keg, uma_zone_t zone, int wait) > { > [...] > /* > * This reproduces the old vm_zone behavior of zero filling pages the > * first time they are added to a zone. > * > * Malloced items are zeroed in uma_zalloc. > */ > > if ((keg->uk_flags & UMA_ZONE_MALLOC) == 0) > wait |= M_ZERO; > else > wait &= ~M_ZERO; > [...] > /* zone is passed for legacy reasons. */ > mem = allocf(zone, keg->uk_ppera * PAGE_SIZE, &flags, wait); > > Firstly, the name "wait" and the comment are confusing, since "wait" is > really "flags" which may have M_ZERO set. So, for non UMA_ZONE_MALLOC > zones (which should be most zones) the page is supposed to be zeroed? > > We ignored the flags in our FreeBSD 9.3 based port and this worked quite > well. However, on FreeBSD head I changed the page allocator now to do: > > #ifdef INVARIANTS > wait |= M_ZERO; > #endif > > if (addr != NULL && (wait & M_ZERO) != 0) { > memset(addr, 0, size_in_bytes); > } > > With this change, everything works like before. Is this in your page allocator ? If yes, then this is how things are suppossed to work. If uma requested M_ZERO page, the page must be zeroed. This is very different from 'always zeroing', which you asked about before. > > > > > If your zone does not have init/fini functions, nor ctors/dtors, then > > HEAD and stable/11 actually actively trash items to catch reliance on > > the uninitialized memory being zero. Look for trash_ctor in uma_core.c. > > The zone has an init/fini function, see udp_init() > > /* > * For now default to 2-tuple UDP hashing - until the fragment > * reassembly code can also update the flowid. > * > * Once we can calculate the flowid that way and re-establish > * a 4-tuple, flip this to 4-tuple. > */ > in_pcbinfo_init(&V_udbinfo, "udp", &V_udb, UDBHASHSIZE, UDBHASHSIZE, > "udp_inpcb", udp_inpcb_init, NULL, 0, > IPI_HASHFIELDS_2TUPLE); > If you use init/ctr which do not set the allocated object to the pre-defined values, and do not specify M_ZERO flag, the content of the bytes which were not touched by init/ctr is not defined. It is not guaranteed to be zeroed. From owner-freebsd-hackers@freebsd.org Sat Nov 12 18:52:33 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDAE9C3EF33; Sat, 12 Nov 2016 18:52:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E53C48A7; Sat, 12 Nov 2016 18:52:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA28479; Sat, 12 Nov 2016 20:52:29 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c5dPh-000ALS-B5; Sat, 12 Nov 2016 20:52:29 +0200 Subject: Re: firewire panic To: gljennjohn@gmail.com References: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> <20161112101020.0ddf33ba@ernst.home> <20161112103417.7ae9b439@ernst.home> Cc: FreeBSD Current , FreeBSD Hackers From: Andriy Gapon Message-ID: <45171db9-20db-bb7b-0f08-603898948677@FreeBSD.org> Date: Sat, 12 Nov 2016 20:51:33 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161112103417.7ae9b439@ernst.home> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Nov 2016 18:52:33 -0000 On 12/11/2016 11:34, Gary Jennejohn wrote: > FYI also works on FreeBSD 12.0-CURRENT r308568 amd64. > > Nov 12 10:24:31 ernst kernel: fwohci0: fwohci_intr_core: BUS reset > Nov 12 10:24:31 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=3, non CYCLEMASTER mode > Nov 12 10:24:31 ernst kernel: firewire0: 2 nodes, maxhop <= 1 cable IRM irm(0) (me) > Nov 12 10:24:31 ernst kernel: firewire0: root node is not cycle master capable > Nov 12 10:24:31 ernst kernel: firewire0: bus manager 0 > Nov 12 10:24:31 ernst kernel: fwohci0: too many cycles lost, no cycle master present? > Nov 12 10:24:31 ernst kernel: firewire0: split transaction timeout: tl=0x1 flag=0x04 > Nov 12 10:24:31 ernst kernel: send: dst=0x01 tl=0x01 rt=0 tcode=0x4 pri=0x0 src=0x000 > Nov 12 10:24:34 ernst kernel: fwohci0: fwohci_intr_core: BUS reset > Nov 12 10:24:34 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000001, SelfID Count=4, CYCLEMASTER mode > Nov 12 10:24:34 ernst kernel: firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) (me) > Nov 12 10:24:34 ernst kernel: firewire0: bus manager 1 > Nov 12 10:24:34 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0: ordered:1 type:0 EUI:0001a3000005ee4c node:0 speed:2 maxrec:8 > Nov 12 10:24:34 ernst kernel: sbp0: sbp_show_sdev_info: sbp0:0:0 'Genesys ' '' '' > Nov 12 10:24:35 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 > Nov 12 10:24:35 ernst kernel: da0: Fixed Direct Access SCSI device > Nov 12 10:24:35 ernst kernel: da0: 50.000MB/s transfers > Nov 12 10:24:35 ernst kernel: da0: 305245MB (625142448 512 byte sectors) > Nov 12 10:24:35 ernst kernel: da0: quirks=0x2 > Nov 12 10:25:00 ernst kernel: fwohci0: fwohci_intr_core: BUS reset > Nov 12 10:25:00 ernst kernel: fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=5, CYCLEMASTER mode > Nov 12 10:25:00 ernst kernel: firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) > Nov 12 10:25:00 ernst kernel: firewire0: bus manager 0 > Nov 12 10:25:00 ernst kernel: da0 at sbp0 bus 0 scbus6 target 0 lun 0 > Nov 12 10:25:00 ernst kernel: da0: detached > Nov 12 10:25:00 ernst kernel: (da0:sbp0:0:0:0): Periph destroyed Is this with INVARIANTS ? -- Andriy Gapon