From owner-freebsd-arch@FreeBSD.ORG Sun Feb 23 11:59:52 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E2C62D5A; Sun, 23 Feb 2014 11:59:52 +0000 (UTC) Received: from vms173007pub.verizon.net (vms173007pub.verizon.net [206.46.173.7]) by mx1.freebsd.org (Postfix) with ESMTP id B8DA61577; Sun, 23 Feb 2014 11:59:52 +0000 (UTC) Received: from localhost.localdomain ([unknown] [71.178.10.220]) by vms173007.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N1G00I966NFCV60@vms173007.mailsrvcs.net>; Sun, 23 Feb 2014 05:59:44 -0600 (CST) Received: from localhost.localdomain (aerie [127.0.0.1]) by localhost.localdomain (8.14.3/8.14.3/Debian-9.4) with ESMTP id s1NBxd6E004390 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 23 Feb 2014 06:59:39 -0500 Received: (from tom@localhost) by localhost.localdomain (8.14.3/8.14.3/Submit) id s1NBxd9j004388; Sun, 23 Feb 2014 06:59:39 -0500 Date: Sun, 23 Feb 2014 06:59:39 -0500 From: Thomas Dickey To: Ed Schouten Subject: Re: terminfo Message-id: <20140223115939.GB4084@aerie.jexium-island.net> References: <5304A0CC.5000505@FreeBSD.org> MIME-version: 1.0 Content-type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary=DBIVS5p969aUjpLe Content-disposition: inline In-reply-to: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: arch@freebsd.org, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dickey@his.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 11:59:53 -0000 --DBIVS5p969aUjpLe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Feb 21, 2014 at 01:05:52PM +0100, Ed Schouten wrote: > Hi Bryan, >=20 > On 19 February 2014 13:17, Bryan Drewery wrote: > > Why do we not use terminfo? Our termcap is quite aged and missing a lot > > of modern terminals/clients. >=20 > It is true that our termcap is quite aged, but the fact is, once you > add entries for a certain terminal, there's little need to update it > after that. ncurses itself is not really a moving target. What kind of > modern terminals/clients are missing? hmm - I continue to add/improve the database (and always have todo's) http://invisible-island.net/ncurses/terminfo.src-history.html > > Using terminfo would allow us to use the already well maintained databa= se from ncurses. Is it just a matter of someone doing the work or are there= other reasons? >=20 > It's just a matter of someone doing the work. It would be nice if we > ever made this change. It's more than that - there's some reluctance of the users to switch to terminfo. So what FreeBSD has is a facade wrapped around terminfo, with the legacy termcap database updated occasionally. =20 > I won't deny that termcap was really useful at one point in time, but > let's be honest: the variety of terminals out there has massively > dropped over time. Terminal emulation has become a solved problem. As > of FreeBSD 9, syscons supports all the sequences described in > xterm-256color, though it isn't able to print more than 8 colours, I'm inclined to disagree with that. Here's a working entry for the console, along with notes: http://invisible-island.net/ncurses/terminfo.src.html#tic-teken (though I see that I overlooked adding the note which I made for wsvt25, that none of the xterm-specific items is supported). Counting differences using infocmp: versus cons25 (37 lines) versus xterm-new (120 lines) One of the differences against cons25 is partial, as noted in my comments on acsc - there's no corresponding adjustment to make against xterm. > which is why we use TERM=3Dxterm. Tools like screen, tmux, etc., they > use a different TERM type, but this is mainly used to detect whether > the process is running inside of screen or tmux. It does not strongly > affect the kinds of sequences that are being emitted. They work > perfectly fine if you just set TERM to xterm or xterm-256color. >=20 > I suspect the following logic would be sufficient for at least 99.5% > of our users: >=20 > if $TERM contains 256 > use xterm-256color > else > use xterm I didn't see any 256-color support in testing this configuration. It does _something_, but would tend to disappoint users. ftp://invisible-island.net/temp/teken-88colors.png ftp://invisible-island.net/temp/teken-256colors.png (screenshots made using VMware Fusion with FreeBSD 10 - ymmv) =20 > It's a shame I am so short on time nowadays, but I think it would make > so much sense to just come up with some kind of document that > standardizes the intersection of the features supported by most common > terminal emulators and get it rubber stamped by the maintainers of > various terminal emulators. If it turns out some kind of terminal > emulator does something in a non-standard way, we can just slap this > document in the author's face. That would not only benefit FreeBSD, > but also most of the other flavours of UNIX. >=20 > $TERM should die. certainly not while there are such large differences among the implementati= ons. :-) --=20 Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net --DBIVS5p969aUjpLe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAlMJ4qsACgkQcCNT4PfkjtuhgQCdFvrZJbPSj1B0oqTdtWb6RyXb kYEAniEYpH45cNdgLPQI3bB8RuvaNpXp =adgk -----END PGP SIGNATURE----- --DBIVS5p969aUjpLe-- From owner-freebsd-arch@FreeBSD.ORG Sun Feb 23 13:49:27 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2E7A156B; Sun, 23 Feb 2014 13:49:27 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C7AF01DD2; Sun, 23 Feb 2014 13:49:26 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id im17so4779048vcb.5 for ; Sun, 23 Feb 2014 05:49:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=AnFC+WBnkqGztrTZrWVmZjgYfGCczsXDcJNVDgKY5PU=; b=GUugPbl5M7Njftbac0JekmHdtqNYcLtfgSJvXEOU0yVp6oMa4DmK2ZSzqQU6DnH3VE jeusS0U1waSut4BIYjH1CG6jsD/7cU0XHqIQH4v/EyyZZcufMzBYYb0FrPORbu3AY6up /bmn8YmrPYLFQY+6kz0MV+S6CSt5F68E6eMjtFxKHAMzIiFQAh5NRqZvGFBi6aQ8B4fc MmChpyVk+m1cD9P8dKiYLpCCCkQbRZ7qrpRO64TZjVM+LKBbWMmNpUtqnHsB2H2woqxW wzlvnezD9T0Sv4al8HUNhkr98PXBZmgESI4mYI9geVEWKYMHf7WrsbsmbZG733PYnlFr dtfg== MIME-Version: 1.0 X-Received: by 10.220.67.18 with SMTP id p18mr9942119vci.14.1393163365859; Sun, 23 Feb 2014 05:49:25 -0800 (PST) Sender: edschouten@gmail.com Received: by 10.220.105.140 with HTTP; Sun, 23 Feb 2014 05:49:25 -0800 (PST) In-Reply-To: <20140223115939.GB4084@aerie.jexium-island.net> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> Date: Sun, 23 Feb 2014 14:49:25 +0100 X-Google-Sender-Auth: a5uZkc1bDxu0dtwaZqpjTJJTGNI Message-ID: Subject: Re: terminfo From: Ed Schouten To: dickey@his.com Content-Type: text/plain; charset=UTF-8 Cc: arch@freebsd.org, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 13:49:27 -0000 Thomas, On 23 February 2014 12:59, Thomas Dickey wrote: >> I won't deny that termcap was really useful at one point in time, but >> let's be honest: the variety of terminals out there has massively >> dropped over time. Terminal emulation has become a solved problem. As >> of FreeBSD 9, syscons supports all the sequences described in >> xterm-256color, though it isn't able to print more than 8 colours, > > I'm inclined to disagree with that. > > Here's a working entry for the console, along with notes: > > http://invisible-island.net/ncurses/terminfo.src.html#tic-teken > > (though I see that I overlooked adding the note which I made for wsvt25, > that none of the xterm-specific items is supported). > > Counting differences using infocmp: > versus cons25 (37 lines) > versus xterm-new (120 lines) > > One of the differences against cons25 is partial, as noted in my comments > on acsc - there's no corresponding adjustment to make against xterm. This terminfo entry is incorrect. It will make ACS work on a standard setup where the user has the BIOS CP437 font loaded, but will break horribly if a user uses a different character set, like ISO-8859-1. There is a reason why syscons displays ASCII characters for ACS box drawing, namely that there is no guarantee that the actual font loaded into the graphics card has any ACS characters to begin with. >> which is why we use TERM=xterm. Tools like screen, tmux, etc., they >> use a different TERM type, but this is mainly used to detect whether >> the process is running inside of screen or tmux. It does not strongly >> affect the kinds of sequences that are being emitted. They work >> perfectly fine if you just set TERM to xterm or xterm-256color. >> >> I suspect the following logic would be sufficient for at least 99.5% >> of our users: >> >> if $TERM contains 256 >> use xterm-256color >> else >> use xterm > > I didn't see any 256-color support in testing this configuration. > It does _something_, but would tend to disappoint users. > > ftp://invisible-island.net/temp/teken-88colors.png > ftp://invisible-island.net/temp/teken-256colors.png > > (screenshots made using VMware Fusion with FreeBSD 10 - ymmv) As I mentioned before, teken supports 256 colours internally, but this is sampled down to 8 different colours later on. This is due to the fact that syscons only reserves 1 byte per cell to keep track of display attributes, which is not sufficient to keep track of 256 foreground colours, 256 background colours and bold, blink, etc. >> It's a shame I am so short on time nowadays, but I think it would make >> so much sense to just come up with some kind of document that >> standardizes the intersection of the features supported by most common >> terminal emulators and get it rubber stamped by the maintainers of >> various terminal emulators. If it turns out some kind of terminal >> emulator does something in a non-standard way, we can just slap this >> document in the author's face. That would not only benefit FreeBSD, >> but also most of the other flavours of UNIX. >> >> $TERM should die. > > certainly not while there are such large differences among the implementations. > > :-) And I think that these differences only exist, because -- don't take this as a personal insult, please read on -- the maintainer of terminfo/ncurses/vttest has done little to nothing to introduce any convergence of implementations over time. I think the comment you've added above the termcap entry and this email demonstrate this. Because realistically speaking: - Who cares about VT52 support? Those things are from the Viking Age. - Who cares about double-width characters? I don't know a single program that uses this. In the worst case, you can just use fullwidth CJK for certain characters. - Who cares about send/receive mode? - Who cares about 88 colour support? Just use 256 colours. - Who cares about ACS? Unicode already has those characters. There are so many legacy features out there, that nobody actually *knows* how terminals should behave anymore. Also combined with the fact that the de facto reference implementation of terminal emulation (i.e., xterm) cannot easily be decoupled from X11, people just make stuff up. People are nowadays only interested in having a 16 or 256 colour, UTF-8 enabled terminal. It would be so incredibly easy for you as the maintainer of terminfo to say: "Here's an ideal terminfo entry. It's compact, easy to implement, etc. This is how I want terminals to behave. And this menu in vttest can be used to easily test conformance." Then you just file bugs against the authors of commonly used terminal emulators and 3-5 years later, the ecosystem has converged. Done. As I mentioned before, my spare time is limited nowadays, but if you would be interested in doing something like that, I would be more than interested to get this sorted out on FreeBSD's side. -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Sun Feb 23 17:27:14 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CD6E11DD for ; Sun, 23 Feb 2014 17:27:14 +0000 (UTC) Received: from mail-ee0-f42.google.com (mail-ee0-f42.google.com [74.125.83.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5993012C7 for ; Sun, 23 Feb 2014 17:27:13 +0000 (UTC) Received: by mail-ee0-f42.google.com with SMTP id e53so1078773eek.15 for ; Sun, 23 Feb 2014 09:27:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=8p1D2J83V6rNjfyRGYcrcn9HJTcd/q1t+8VapTl1NTM=; b=YYz6BT0M73nemIqly9jYAzSA2qiBAQ+WFDfi8wjMMJMbRSL+zg6Px7V/wubfyxzOVF YAXDIV5bZ2voAYd84c5nIAWJnPJcPtymCrOeT2JmFSStX3Ru7gdS2D3Jj7f3UVP8la8p VNTYy9b8KarjB5CtIvQPXIe5eMKumrfR/SdcoOYJ0Pb6thtruCK4cfsgcCnZlSHs5g10 YN1V+wm2vvuf1d2TOtRg9JVPjfEP2Fugqp0LZhNSz5ZaIgdctXf23GdF0r1RMvnVwtuw 7pKpdonyNGEh9goqLtZkNrS/P/3rccbccalGWqfogXd+xmhrSUB5vKI0/k2sGtOC5/Ld aYqw== X-Gm-Message-State: ALoCoQmPdyp8VH87IL5ZvEm2Iy2LKg8mD59KfC8lrt881YZxXgzAfWnlOhksrHSSxjNcwv5WNxiN X-Received: by 10.14.103.134 with SMTP id f6mr19948394eeg.41.1393174893492; Sun, 23 Feb 2014 09:01:33 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id m8sm43094836eef.14.2014.02.23.09.01.32 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 23 Feb 2014 09:01:32 -0800 (PST) Message-ID: <530A296E.9090805@freebsd.org> Date: Sun, 23 Feb 2014 21:01:34 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Ed Schouten , dickey@his.com Subject: Re: terminfo References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> In-Reply-To: X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 17:27:14 -0000 On 23.02.2014 17:49, Ed Schouten wrote: > This terminfo entry is incorrect. It will make ACS work on a standard > setup where the user has the BIOS CP437 font loaded, but will break > horribly if a user uses a different character set, like ISO-8859-1. > > There is a reason why syscons displays ASCII characters for ACS box > drawing, namely that there is no guarantee that the actual font loaded > into the graphics card has any ACS characters to begin with. Old syscons was able to displays pseudographics, and several cons25-like termcap entries exists to correspond ACS for each console font. It is your emulation which always displays ASCII now, which is step backward. -- http://ache.vniz.net/ From owner-freebsd-arch@FreeBSD.ORG Sun Feb 23 18:09:11 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA3C3C56; Sun, 23 Feb 2014 18:09:11 +0000 (UTC) Received: from mail-ve0-x22e.google.com (mail-ve0-x22e.google.com [IPv6:2607:f8b0:400c:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5EA90165D; Sun, 23 Feb 2014 18:09:11 +0000 (UTC) Received: by mail-ve0-f174.google.com with SMTP id oy12so329205veb.19 for ; Sun, 23 Feb 2014 10:09:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=riZQ8wIYaJWJDxDdUqM4Y/Yi6/vAiTMubtgnJCwpyLU=; b=dBqOpXwx6Defi4VPwMX6gW9CIquQR+R7HLOih4aFNf04XDpdhUy9eJV8ET8zE8b1ST dBVuv6w0wlqXfuuAATwh/0uWk8dOsUDULeKovJi55Y5QKKuZfbyMBeXxRjLwyQVpDtLo NVHHIVskfAbi/h0ZTy433FH59qnZ+BfKMuUumO2S0zA0443NtkkIe8kKJhpvTJs27gaG lNnCpIVF4pmFrw9VyYDw6GRia8xGbTMAh+wJSTN1B9mSADUsL9rUfJDbOHcGKLbKFoRy UNataYmerohXYa/xYDh8Qcz6lro3AQS9yltO3PQ7hsuZgv/j9mIRlTRTRye/dcF1JMlU 7WCA== MIME-Version: 1.0 X-Received: by 10.220.116.136 with SMTP id m8mr10284234vcq.77.1393178950575; Sun, 23 Feb 2014 10:09:10 -0800 (PST) Sender: edschouten@gmail.com Received: by 10.220.105.140 with HTTP; Sun, 23 Feb 2014 10:09:10 -0800 (PST) In-Reply-To: <530A296E.9090805@freebsd.org> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> <530A296E.9090805@freebsd.org> Date: Sun, 23 Feb 2014 19:09:10 +0100 X-Google-Sender-Auth: _5uh2xDvDXga_kovX_iRaG3fUqA Message-ID: Subject: Re: terminfo From: Ed Schouten To: Andrey Chernov Content-Type: text/plain; charset=UTF-8 Cc: arch@freebsd.org, dickey@his.com, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 18:09:11 -0000 On 23 February 2014 18:01, Andrey Chernov wrote: > It is your emulation which always displays ASCII now, which is step backward. Indeed. But you know what the awesome thing is? Because ACS is now part of the terminal emulator itself, you can actually make this work with a single TERM type, regardless of the character set used. Be sure to extend sys/teken/teken_scs.h to include additional maps for each character set. Alternatively, enable vt(9), which translates it to the right Unicode characters. -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Sun Feb 23 19:14:06 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 13BD1677 for ; Sun, 23 Feb 2014 19:14:06 +0000 (UTC) Received: from mail-ee0-f49.google.com (mail-ee0-f49.google.com [74.125.83.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9522F1BA0 for ; Sun, 23 Feb 2014 19:14:05 +0000 (UTC) Received: by mail-ee0-f49.google.com with SMTP id d17so2672467eek.36 for ; Sun, 23 Feb 2014 11:13:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=1RLQsMepEnvJ2SofQKS0nzVOZRO/ML3wEjaNcsQ8Wsc=; b=jJ7o/n/8eQP5wE18Bhx9EOzR5xsHiHXXJyITy+kD4eS6ZT3KYBRNM8hBo8oOR4odtf w7BmqtuGt/R1TcNqsDzau82UeAvWY0s5dSxfV3LOkhlLMjFBhXOMRq7q4nDSKEfIC/rz 6N3r+QALDfbQiKgm/hOJs4APBYZ1eexjoZd7cBwrPHSfJrnr8AJLi8+m15U3o4R3r5oY X94ty9TKKtgv86YEd7+FHaEN39pmEJ0fjXXQZrplX3OvE6y2h4h+tNKFW771Zpzo9NUQ sJC/S0Xtf0ajw+lwadYGGv6ATG/BNiisiV5t8fOdwHCrp2kig666zH+39LWiiB463AGM Uf0A== X-Gm-Message-State: ALoCoQmmgWwslGs46C3kmiscTKb4RHjd7vrdjexLiS2CZcHhsdnWJoF03XVzggSFgac1Zbuohhc+ X-Received: by 10.15.75.66 with SMTP id k42mr20494614eey.2.1393181499837; Sun, 23 Feb 2014 10:51:39 -0800 (PST) Received: from [192.168.1.2] ([89.169.173.68]) by mx.google.com with ESMTPSA id x2sm54167075eeo.8.2014.02.23.10.51.38 for (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 23 Feb 2014 10:51:39 -0800 (PST) Message-ID: <530A433D.1040907@freebsd.org> Date: Sun, 23 Feb 2014 22:51:41 +0400 From: Andrey Chernov User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Ed Schouten Subject: Re: terminfo References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> <530A296E.9090805@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.7a1pre Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, dickey@his.com, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Feb 2014 19:14:06 -0000 On 23.02.2014 22:09, Ed Schouten wrote: > On 23 February 2014 18:01, Andrey Chernov wrote: >> It is your emulation which always displays ASCII now, which is step backward. > > Indeed. But you know what the awesome thing is? Because ACS is now > part of the terminal emulator itself, you can actually make this work > with a single TERM type, regardless of the character set used. > > Be sure to extend sys/teken/teken_scs.h to include additional maps for > each character set. In the /sys/teken/teken_scs.h you use just one bit to distinguish between ASCII fallback and Unicode boxdrawing, it is not enough. There must be a field for the current character set somewhere in the teken internals and vidcontrol(1) must be able to set it. The whole processing chain is architectural decision I prefer to not interfere with. If you'll make it ready, I could give you KOI8-R map for it. -- http://ache.vniz.net/ From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 01:35:35 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 982C0A4F; Tue, 25 Feb 2014 01:35:35 +0000 (UTC) Received: from vms173005pub.verizon.net (vms173005pub.verizon.net [206.46.173.5]) by mx1.freebsd.org (Postfix) with ESMTP id 6C79E1E26; Tue, 25 Feb 2014 01:35:34 +0000 (UTC) Received: from localhost.localdomain ([unknown] [71.178.10.220]) by vms173005.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N1J00B4R32HE250@vms173005.mailsrvcs.net>; Mon, 24 Feb 2014 19:35:10 -0600 (CST) Received: from localhost.localdomain (aerie [127.0.0.1]) by localhost.localdomain (8.14.3/8.14.3/Debian-9.4) with ESMTP id s1P1Z50E007333 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 24 Feb 2014 20:35:05 -0500 Received: (from tom@localhost) by localhost.localdomain (8.14.3/8.14.3/Submit) id s1P1Z5e6007331; Mon, 24 Feb 2014 20:35:05 -0500 Date: Mon, 24 Feb 2014 20:35:04 -0500 From: Thomas Dickey To: Peter Wemm Subject: Re: terminfo Message-id: <20140225013504.GA7294@aerie.jexium-island.net> References: <5304A0CC.5000505@FreeBSD.org> MIME-version: 1.0 Content-type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary=pf9I7BMVVzbSWLtt Content-disposition: inline In-reply-to: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: FreeBSD Arch , Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dickey@his.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:35:35 -0000 --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Feb 21, 2014 at 11:14:22AM -0800, Peter Wemm wrote: > HOWEVER, if I were to do it today, I would be inclined to use netbsd's > libtinfo and cdb modules and switch to terminfo as the source of > truth. It is my understanding that netbsd's libtinfo "compiles" into > a .cdb format file and gives us the same flexibility that we have with > termcap/termcap.db without the non-extensible sysv binary format > lock-in. Then have ncurses use it instead, like netbsd does. We get ncurses has been extensible since release 5.0 in 1999 (long enough to notic= e). It's supported hashed-database since 2006 (which is the way I build it on m= y *BSD machines). --=20 Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net --pf9I7BMVVzbSWLtt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAlML80gACgkQcCNT4PfkjtuDUgCfW/e5sHkWSfI5bVGGa0Nr5Ijd XAoAnRYbhoqDiT+pfhzgsBnnN0oWmg2Q =flSa -----END PGP SIGNATURE----- --pf9I7BMVVzbSWLtt-- From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 01:50:21 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4C00FE7; Tue, 25 Feb 2014 01:50:21 +0000 (UTC) Received: from mhc-out-201.his.com (mhc-out-201.his.com [216.194.251.12]) by mx1.freebsd.org (Postfix) with ESMTP id 8E0AF1017; Tue, 25 Feb 2014 01:50:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mhc-out-201.his.com (Postfix) with ESMTP id 448EF60E97; Mon, 24 Feb 2014 20:40:32 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mhc-out-201.his.com X-Spam-Flag: NO X-Spam-Score: -1.911 X-Spam-Level: X-Spam-Status: No, score=-1.911 tagged_above=-99 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mhc-out-201.his.com ([127.0.0.1]) by localhost (mhc-out-201.his.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uqZn-F13JUcv; Mon, 24 Feb 2014 20:40:28 -0500 (EST) Received: from mail-sterling.his.com (mail-sterling.his.com [216.194.248.141]) by mhc-out-201.his.com (Postfix) with ESMTP id 3878560FDA; Mon, 24 Feb 2014 20:40:28 -0500 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail-sterling.his.com (Postfix) with ESMTP id A96223FB001D; Mon, 24 Feb 2014 20:40:27 -0500 (EST) X-Virus-Scanned: amavisd-new at mail-sterling.his.com Received: from mail-sterling.his.com ([127.0.0.1]) by localhost (mail-sterling.his.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PLO5fPZ2phbb; Mon, 24 Feb 2014 20:40:27 -0500 (EST) Received: from mail-sterling.his.com (mail-sterling.his.com [216.194.248.141]) by mail-sterling.his.com (Postfix) with ESMTP id 301E73FB0013; Mon, 24 Feb 2014 20:40:27 -0500 (EST) Date: Mon, 24 Feb 2014 20:40:27 -0500 (EST) From: Thomas Dickey To: dickey@his.com Message-ID: <154665408.16871741.1393292427092.JavaMail.root@his.com> In-Reply-To: <20140225013504.GA7294@aerie.jexium-island.net> Subject: Re: terminfo MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [71.178.10.220] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Mac)/7.2.6_GA_2926) Cc: FreeBSD Arch , Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 01:50:21 -0000 for example - http://aerie.jexium-island.net/ncurses/terminfo.src.html#toc-_N_C_U_R_S_E_S__U_S_E_R-_D_E_F_I_N_A_B_L_E__C_A_P_A_B_I_L_I_T_I_E_S ----- Original Message ----- | From: "Thomas Dickey" ... | ncurses has been extensible since release 5.0 in 1999 (long enough to notice). -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 02:06:17 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0294D327; Tue, 25 Feb 2014 02:06:17 +0000 (UTC) Received: from mhc-out-201.his.com (mhc-out-201.his.com [216.194.251.12]) by mx1.freebsd.org (Postfix) with ESMTP id BE6C91190; Tue, 25 Feb 2014 02:06:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mhc-out-201.his.com (Postfix) with ESMTP id 6B22C60FCD; Mon, 24 Feb 2014 21:06:15 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mhc-out-201.his.com X-Spam-Flag: NO X-Spam-Score: -1.911 X-Spam-Level: X-Spam-Status: No, score=-1.911 tagged_above=-99 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham Received: from mhc-out-201.his.com ([127.0.0.1]) by localhost (mhc-out-201.his.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9q3838pK4N14; Mon, 24 Feb 2014 21:06:11 -0500 (EST) Received: from mail-sterling.his.com (mail-sterling.his.com [216.194.248.141]) by mhc-out-201.his.com (Postfix) with ESMTP id 5A7C660FCB; Mon, 24 Feb 2014 21:06:11 -0500 (EST) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail-sterling.his.com (Postfix) with ESMTP id CB6B13F90018; Mon, 24 Feb 2014 21:06:10 -0500 (EST) X-Virus-Scanned: amavisd-new at mail-sterling.his.com Received: from mail-sterling.his.com ([127.0.0.1]) by localhost (mail-sterling.his.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vz4PA4qwKfxH; Mon, 24 Feb 2014 21:06:10 -0500 (EST) Received: from mail-sterling.his.com (mail-sterling.his.com [216.194.248.141]) by mail-sterling.his.com (Postfix) with ESMTP id 234B33F9000E; Mon, 24 Feb 2014 21:06:10 -0500 (EST) Date: Mon, 24 Feb 2014 21:06:10 -0500 (EST) From: Thomas Dickey To: dickey@his.com Message-ID: <380328500.16877350.1393293970026.JavaMail.root@his.com> In-Reply-To: <154665408.16871741.1393292427092.JavaMail.root@his.com> Subject: Re: terminfo MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [71.178.10.220] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Mac)/7.2.6_GA_2926) Cc: FreeBSD Arch , Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 02:06:17 -0000 sorry - my internal mirrors look just like the real pages: http://invisible-island.net/ncurses/terminfo.src.html#toc-_N_C_U_R_S_E_S__U_S_E_R-_D_E_F_I_N_A_B_L_E__C_A_P_A_B_I_L_I_T_I_E_S ----- Original Message ----- | From: "Thomas Dickey" | To: dickey@his.com | Cc: "FreeBSD Arch" , "Bryan Drewery" , "Peter Wemm" | Sent: Monday, February 24, 2014 8:40:27 PM | Subject: Re: terminfo | | for example - | | http://aerie.jexium-island.net/ncurses/terminfo.src.html#toc-_N_C_U_R_S_E_S__U_S_E_R-_D_E_F_I_N_A_B_L_E__C_A_P_A_B_I_L_I_T_I_E_S | | ----- Original Message ----- | | From: "Thomas Dickey" | ... | | ncurses has been extensible since release 5.0 in 1999 (long enough | | to notice). | | -- | Thomas E. Dickey | http://invisible-island.net | ftp://invisible-island.net | -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 02:39:26 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D4CFBCF; Tue, 25 Feb 2014 02:39:26 +0000 (UTC) Received: from vms173019pub.verizon.net (vms173019pub.verizon.net [206.46.173.19]) by mx1.freebsd.org (Postfix) with ESMTP id 5CA5E13AD; Tue, 25 Feb 2014 02:39:25 +0000 (UTC) Received: from localhost.localdomain ([unknown] [71.178.10.220]) by vms173019.mailsrvcs.net (Sun Java(tm) System Messaging Server 7u2-7.02 32bit (built Apr 16 2009)) with ESMTPA id <0N1J00LQB38BI230@vms173019.mailsrvcs.net>; Mon, 24 Feb 2014 19:38:40 -0600 (CST) Received: from localhost.localdomain (aerie [127.0.0.1]) by localhost.localdomain (8.14.3/8.14.3/Debian-9.4) with ESMTP id s1P1cZNc007383 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 24 Feb 2014 20:38:35 -0500 Received: (from tom@localhost) by localhost.localdomain (8.14.3/8.14.3/Submit) id s1P1cZFH007382; Mon, 24 Feb 2014 20:38:35 -0500 Date: Mon, 24 Feb 2014 20:38:35 -0500 From: Thomas Dickey To: Andrey Chernov Subject: Re: terminfo Message-id: <20140225013835.GB7294@aerie.jexium-island.net> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> <530A296E.9090805@freebsd.org> MIME-version: 1.0 Content-type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary=+g7M9IMkV8truYOl Content-disposition: inline In-reply-to: <530A296E.9090805@freebsd.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Ed Schouten , Bryan Drewery , dickey@his.com, arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: dickey@his.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 02:39:26 -0000 --+g7M9IMkV8truYOl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Feb 23, 2014 at 09:01:34PM +0400, Andrey Chernov wrote: > On 23.02.2014 17:49, Ed Schouten wrote: > > This terminfo entry is incorrect. It will make ACS work on a standard > > setup where the user has the BIOS CP437 font loaded, but will break > > horribly if a user uses a different character set, like ISO-8859-1. > >=20 > > There is a reason why syscons displays ASCII characters for ACS box > > drawing, namely that there is no guarantee that the actual font loaded > > into the graphics card has any ACS characters to begin with. >=20 > Old syscons was able to displays pseudographics, and several cons25-like > termcap entries exists to correspond ACS for each console font. It is > your emulation which always displays ASCII now, which is step backward. my changes on Saturday were for a default install of FreeBSD 9 and 10 (the "Old syscons" pseudographics is, except for the 4 noted, quite alive). I took a look for documentation per se regarding teken, but found none. It would be nice if someone pointed where the documentation for the feature= is. --=20 Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net --+g7M9IMkV8truYOl Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAlML9BsACgkQcCNT4PfkjtuWOQCfYgFZN6PfM619PFz/4a+R+Krv aLAAn1gV/8ZOKTbeC/1uSjZOLOGORlyJ =V1VG -----END PGP SIGNATURE----- --+g7M9IMkV8truYOl-- From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 12:55:11 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D0A0B21; Tue, 25 Feb 2014 12:55:11 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3AF2C1656; Tue, 25 Feb 2014 12:55:10 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 0FD245523; Tue, 25 Feb 2014 12:55:10 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id E0746D89; Tue, 25 Feb 2014 13:55:10 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Subject: Re: terminfo References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> Date: Tue, 25 Feb 2014 13:55:10 +0100 In-Reply-To: (Warner Losh's message of "Fri, 21 Feb 2014 12:14:35 -0700") Message-ID: <86ios31he9.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@FreeBSD.org, Ed Schouten , Ian Lepore , Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 12:55:11 -0000 Warner Losh writes: > I've had good luck with TERM=3Dvt100 in those situations, but I use tip > and apart from eating ^P it does a good job of being transparent > enough... Am I the only one still using plain, unadorned cu(1)? (yes, I know it's the same binary as tip) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 13:09:57 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0185D92; Tue, 25 Feb 2014 13:09:57 +0000 (UTC) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 36029173F; Tue, 25 Feb 2014 13:09:56 +0000 (UTC) Received: from [194.32.164.24] (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id s1PD4s6E054159; Tue, 25 Feb 2014 13:04:54 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: terminfo From: Bob Bishop In-Reply-To: <86ios31he9.fsf@nine.des.no> Date: Tue, 25 Feb 2014 13:04:49 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: <48887C45-6C57-43B2-9D3D-98B715976BA4@gid.co.uk> References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> <86ios31he9.fsf@nine.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1827) Cc: arch@freebsd.org, Bryan Drewery , Warner Losh , Ian Lepore , Ed Schouten X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 13:09:57 -0000 Hi, On 25 Feb 2014, at 12:55, Dag-Erling Sm=F8rgrav wrote: > Am I the only one still using plain, unadorned cu(1)? You are not alone :-) (Lots of Soekris boxen around here.) -- Bob Bishop rb@gid.co.uk From owner-freebsd-arch@FreeBSD.ORG Tue Feb 25 18:51:21 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 594B5CA6; Tue, 25 Feb 2014 18:51:21 +0000 (UTC) Received: from mail.vnode.se (mail.vnode.se [212.247.52.13]) by mx1.freebsd.org (Postfix) with ESMTP id 10AD11B9E; Tue, 25 Feb 2014 18:51:20 +0000 (UTC) Received: from mail.vnode.se (localhost [127.0.0.1]) by mail.vnode.se (Postfix) with ESMTP id C6DB8E3F07A; Tue, 25 Feb 2014 19:44:33 +0100 (CET) X-Virus-Scanned: amavisd-new at vnode.se Received: from mail.vnode.se ([127.0.0.1]) by mail.vnode.se (mail.vnode.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RoFxhM-LuQqN; Tue, 25 Feb 2014 19:44:31 +0100 (CET) Received: from [10.10.10.116] (unknown [83.223.1.131]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.vnode.se (Postfix) with ESMTPSA id CEB05E3F079; Tue, 25 Feb 2014 19:44:29 +0100 (CET) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: terminfo From: Joel Dahl In-Reply-To: <48887C45-6C57-43B2-9D3D-98B715976BA4@gid.co.uk> Date: Tue, 25 Feb 2014 19:44:28 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> <86ios31he9.fsf@nine.des.no> <48887C45-6C57-43B2-9D3D-98B715976BA4@gid.co.uk> To: Bob Bishop X-Mailer: Apple Mail (2.1827) Cc: Ed Schouten , Ian Lepore , Bryan Drewery , arch@freebsd.org, Warner Losh , =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 18:51:21 -0000 25 feb 2014 kl. 14:04 skrev Bob Bishop : > Hi, >=20 > On 25 Feb 2014, at 12:55, Dag-Erling Sm=F8rgrav wrote: >=20 >> Am I the only one still using plain, unadorned cu(1)? >=20 > You are not alone :-) (Lots of Soekris boxen around here.) I use it almost every day. =97 Joel= From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 09:00:21 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62FDD545 for ; Wed, 26 Feb 2014 09:00:21 +0000 (UTC) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C74221C91 for ; Wed, 26 Feb 2014 09:00:20 +0000 (UTC) Received: from server.rulingia.com (c220-239-232-212.belrs5.nsw.optusnet.com.au [220.239.232.212]) by vps.rulingia.com (8.14.7/8.14.7) with ESMTP id s1Q8xlP1051460 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 26 Feb 2014 19:59:48 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.7/8.14.7) with ESMTP id s1Q8xfmx007489 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 26 Feb 2014 19:59:41 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.7/8.14.7/Submit) id s1Q8xdba007487; Wed, 26 Feb 2014 19:59:39 +1100 (EST) (envelope-from peter) Date: Wed, 26 Feb 2014 19:59:39 +1100 From: Peter Jeremy To: Ed Schouten Subject: Re: terminfo Message-ID: <20140226085939.GC2705@server.rulingia.com> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="raC6veAxrt5nqIoY" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.22 (2013-10-16) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 09:00:21 -0000 --raC6veAxrt5nqIoY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2014-Feb-23 14:49:25 +0100, Ed Schouten wrote: >- Who cares about double-width characters? I don't know a single >program that uses this. In the worst case, you can just use fullwidth >CJK for certain characters. I wrote an app at $previous-job that used double-height/double-width characters in xterm. I might have been able to use unicode but then everyone who wanted to use in would need to have configured their xterm for UTF-8 - and I don't believe had done that. >- Who cares about 88 colour support? Just use 256 colours. Assuming your app supports that. >- Who cares about ACS? Unicode already has those characters. Again, that assumes your app supports unicode. >People are nowadays only interested in having a 16 or 256 colour, >UTF-8 enabled terminal. And running legacy apps that predate unicode. --=20 Peter Jeremy --raC6veAxrt5nqIoY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iKYEARECAGYFAlMNrPtfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDBCRjc3QTcyNTg5NEVCRTY0RjREN0VFRUZF OEE0N0JGRjAwRkI4ODcACgkQ/opHv/APuIeMcQCfXDVwRgn8qUwJ1HFa5W31AljI ejkAoK/flBPdpgyjiy0pL976dmWgXNYj =GOqp -----END PGP SIGNATURE----- --raC6veAxrt5nqIoY-- From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 12:41:41 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 962179FE; Wed, 26 Feb 2014 12:41:41 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 4ED1115CE; Wed, 26 Feb 2014 12:41:41 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1WIdnv-000LfX-Cl; Wed, 26 Feb 2014 16:41:39 +0400 Date: Wed, 26 Feb 2014 16:41:39 +0400 From: Slawa Olhovchenkov To: Dag-Erling Sm??rgrav Subject: Re: terminfo Message-ID: <20140226124139.GB79819@zxy.spb.ru> References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> <86ios31he9.fsf@nine.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86ios31he9.fsf@nine.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: arch@FreeBSD.org, Bryan Drewery , Warner Losh , Ian Lepore , Ed Schouten X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 12:41:41 -0000 On Tue, Feb 25, 2014 at 01:55:10PM +0100, Dag-Erling Sm??rgrav wrote: > Warner Losh writes: > > I've had good luck with TERM=vt100 in those situations, but I use tip > > and apart from eating ^P it does a good job of being transparent > > enough... > > Am I the only one still using plain, unadorned cu(1)? I too From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 18:26:08 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 700984E1; Wed, 26 Feb 2014 18:26:08 +0000 (UTC) Received: from mail3.westryn.net (mail3.westryn.net [72.28.112.3]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4A7CE196D; Wed, 26 Feb 2014 18:26:08 +0000 (UTC) Received: from sneffels.westryn.net (204.16.225.169.westryn.net [204.16.225.169]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail3.westryn.net (Postfix) with ESMTPSA id 57AB373A6E; Wed, 26 Feb 2014 12:16:22 -0600 (CST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: terminfo From: Kim Shrier In-Reply-To: <20140226124139.GB79819@zxy.spb.ru> Date: Wed, 26 Feb 2014 11:16:20 -0700 Content-Transfer-Encoding: 7bit Message-Id: References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> <86ios31he9.fsf@nine.des.no> <20140226124139.GB79819@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.1827) Cc: Ed Schouten , Ian Lepore , Bryan Drewery , arch@FreeBSD.org, Warner Losh , Dag-Erling Sm??rgrav X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 18:26:08 -0000 On Feb 26, 2014, at 5:41 AM, Slawa Olhovchenkov wrote: > On Tue, Feb 25, 2014 at 01:55:10PM +0100, Dag-Erling Sm??rgrav wrote: > >> Warner Losh writes: >>> I've had good luck with TERM=vt100 in those situations, but I use tip >>> and apart from eating ^P it does a good job of being transparent >>> enough... >> >> Am I the only one still using plain, unadorned cu(1)? > > I too and I From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 21:48:17 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 107F2A57 for ; Wed, 26 Feb 2014 21:48:17 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E4197106D for ; Wed, 26 Feb 2014 21:48:16 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1QLmGs6041462 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 26 Feb 2014 13:48:16 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1QLmGDG041461 for arch@FreeBSD.org; Wed, 26 Feb 2014 13:48:16 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Feb 2014 13:48:16 -0800 From: John-Mark Gurney To: arch@FreeBSD.org Subject: small kernel kernel option... Message-ID: <20140226214816.GB92037@funkthat.com> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Feb 2014 13:48:16 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 21:48:17 -0000 I'm about to commit a change to sha256 to speed it up, but the cost of that speed up is an increase in code/data size from just under 1k to almost 9k (as measured on amd64)... this increase is from unrolling a loop.. Maybe we should have a global kernel option, SMALL_KERNEL, or something similar that can be used to shrink code size for those that are trying to build small embedded devices... Or do we already have this option, but I just don't know about it? I know 8k isn't much, but, a billion here and a billion there and pretty soon you're talking about real money.. :) Comments? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:02:41 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1AAF6E for ; Wed, 26 Feb 2014 22:02:41 +0000 (UTC) Received: from mail-pb0-x233.google.com (mail-pb0-x233.google.com [IPv6:2607:f8b0:400e:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 768E411D0 for ; Wed, 26 Feb 2014 22:02:41 +0000 (UTC) Received: by mail-pb0-f51.google.com with SMTP id un15so1600475pbc.10 for ; Wed, 26 Feb 2014 14:02:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=5zxdkEejge/I4I8AnnwmS+QDpwqw3R7lrw3RjWrkzZ8=; b=0dk6CPjXQyl43Ttmyxkak4VNB3zZt634q99Ul/zyJr021eLBg/sNjkB4+t77YCfoFc 3sdlPNcLkgOoCD5yGlTpnWv8wNc/KZ/P0PDyW/SIsJLGN6F4jgzM8hlW8ycqxIMl3YJt z+DtWEGy3FBQm0t8Y29MBmEB8asMrhc/zCnhs568uRODAAF7i8i/Ivg4uHdO0jkaqIvX 0HF2xtOEiwEF52ZKxn9Uv2XQ3MqkvaK12wgEg+LbN4NsUit/fflKcTh2Z5MU3nFvDtzw UZThA/+Fb+Q01fdbDldcleVD3KbW7NbgAaA9FFHUlsukJzi+GATO9Haw2bfq6oMZNv0c +A2Q== X-Received: by 10.66.41.106 with SMTP id e10mr11305165pal.109.1393452161078; Wed, 26 Feb 2014 14:02:41 -0800 (PST) Received: from [10.64.26.169] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id pe3sm6694883pbc.23.2014.02.26.14.02.23 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Feb 2014 14:02:32 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: small kernel kernel option... From: Warner Losh In-Reply-To: <20140226214816.GB92037@funkthat.com> Date: Wed, 26 Feb 2014 14:02:08 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <5BA6FA29-0BBE-45C1-B734-29013C5A2D29@bsdimp.com> References: <20140226214816.GB92037@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.1874) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:02:41 -0000 On Feb 26, 2014, at 1:48 PM, John-Mark Gurney wrote: > I'm about to commit a change to sha256 to speed it up, but the cost > of that speed up is an increase in code/data size from just under 1k > to almost 9k (as measured on amd64)... this increase is from = unrolling > a loop.. >=20 > Maybe we should have a global kernel option, SMALL_KERNEL, or = something > similar that can be used to shrink code size for those that are trying > to build small embedded devices=85 I=92d prefer something that=92s more like OPTIMIZE_SIZE or = OPTIMIZE_SPEED. > Or do we already have this option, but I just don't know about it? We don=92t have it. We have a billion tiny knobs. > I know 8k isn't much, but, a billion here and a billion there and = pretty > soon you're talking about real money.. :) yea=85 There=92s lots that can be done here, since most folks don=92t = bother with this=85 Warner > Comments? >=20 > --=20 > John-Mark Gurney Voice: +1 415 225 5579 >=20 > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:05:06 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7D2922F for ; Wed, 26 Feb 2014 22:05:06 +0000 (UTC) Received: from mail-lb0-x236.google.com (mail-lb0-x236.google.com [IPv6:2a00:1450:4010:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6068A11F3 for ; Wed, 26 Feb 2014 22:05:06 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id n15so1109912lbi.27 for ; Wed, 26 Feb 2014 14:05:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=AMyX24j+FI6jcJuk/6ifFIyQ+3YNDiFdUe2DiS37/38=; b=VquwgrnszNlvWMeJHY/w7WpM38xiUbxJFzi0GSVPf8QbioURr5Xbnb0wpl3gJCsl4k jgY7nfNbj50ErzgVkL58IkCHevISQ239DwZJNZuTMXteaHWtWB2B9bUawsp90zM1edSt gjVtEQHzXQjLETcTvw4sBc2BZRtPvXVqhm3CGIh8M6KpagiQaU6uiTGmKfi1wvTGWfas /tClQ+t9+bo8Q9ZEkiORlj/MlKCGBClz1LYuQtC6w6z5drZWMzortI0mK49TVDYtpHF8 Rrf8gx5zLH/zubxtS133ZS5YfLoUxhKn6ExcJ42C95BHl47JDfdpdvuTaNOi5suPoSjq +RuA== MIME-Version: 1.0 X-Received: by 10.152.1.168 with SMTP id 8mr40612lan.74.1393452304588; Wed, 26 Feb 2014 14:05:04 -0800 (PST) Sender: rizzo.unipi@gmail.com Received: by 10.115.4.162 with HTTP; Wed, 26 Feb 2014 14:05:04 -0800 (PST) In-Reply-To: <20140226214816.GB92037@funkthat.com> References: <20140226214816.GB92037@funkthat.com> Date: Wed, 26 Feb 2014 14:05:04 -0800 X-Google-Sender-Auth: qumyQCqvmy08SpaLFi4CITQsnFU Message-ID: Subject: Re: small kernel kernel option... From: Luigi Rizzo To: "arch@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:05:07 -0000 On Wednesday, February 26, 2014, John-Mark Gurney wrote: > I'm about to commit a change to sha256 to speed it up, but the cost > of that speed up is an increase in code/data size from just under 1k > to almost 9k (as measured on amd64)... this increase is from unrolling > a loop.. > > Maybe we should have a global kernel option, SMALL_KERNEL, or something > similar that can be used to shrink code size for those that are trying > to build small embedded devices... > > Or do we already have this option, but I just don't know about it? > > I know 8k isn't much, but, a billion here and a billion there and pretty > soon you're talking about real money.. :) > Comments? > > It is very nice that you care about this. I am just not really sure that for such small numbers the extra data size matters much. The downside, as you can imagine, is now having to make sure that lint kernels do not trip on some "unused" warning. Cheers Luigi -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org > " > -- -----------------------------------------+------------------------------- Prof. Luigi RIZZO, rizzo@iet.unipi.it . Dip. di Ing. dell'Informazione http://www.iet.unipi.it/~luigi/ . Universita` di Pisa TEL +39-050-2211611 . via Diotisalvi 2 Mobile +39-338-6809875 . 56122 PISA (Italy) -----------------------------------------+------------------------------- From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:21:34 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 737EF876 for ; Wed, 26 Feb 2014 22:21:34 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 0629012F4 for ; Wed, 26 Feb 2014 22:21:33 +0000 (UTC) Received: from [192.168.0.100] ([87.139.233.65]) by mail.gmx.com (mrgmx002) with ESMTPSA (Nemesis) id 0MGjL7-1WWFFN2FX7-00DU5S for ; Wed, 26 Feb 2014 23:21:25 +0100 Message-ID: <530E68E5.6080702@gmx.de> Date: Wed, 26 Feb 2014 23:21:25 +0100 From: olli hauer User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: Re: terminfo References: <5304A0CC.5000505@FreeBSD.org> <1392997589.1145.91.camel@revolution.hippie.lan> <86ios31he9.fsf@nine.des.no> In-Reply-To: <86ios31he9.fsf@nine.des.no> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:8n4QSn5RXBc1EdZ6fYC2dOsoyGlrbDV04eTSVIEIA0B09rI+r6L cmZYX1tJB5cUSofVEgRhu0hHUlF/1IRDNKp6N3moVOx/GAjjZl82KvbTQ/Ils81UDHNzwkO XjF1cXMwSp4aCUryxoQw1XU9HL+M/H5AMRmunRyiSNstG7lGteDMcQVp6TdqJa6RKZtyJ+X Qgu6J2QEQhi51fUM3e1gA== Cc: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:21:34 -0000 On 2014-02-25 13:55, Dag-Erling Smørgrav wrote: > Warner Losh writes: >> I've had good luck with TERM=vt100 in those situations, but I use tip >> and apart from eating ^P it does a good job of being transparent >> enough... > > Am I the only one still using plain, unadorned cu(1)? > Take me to the list of cu users. cu with an USB-to-RS232 adapter is really handy to setup or fix network equipment -- olli From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:24:47 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 062D394A for ; Wed, 26 Feb 2014 22:24:47 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D185F137D for ; Wed, 26 Feb 2014 22:24:46 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1QMOjId041991 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 26 Feb 2014 14:24:45 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1QMOjlx041990; Wed, 26 Feb 2014 14:24:45 -0800 (PST) (envelope-from jmg) Date: Wed, 26 Feb 2014 14:24:45 -0800 From: John-Mark Gurney To: Warner Losh , Luigi Rizzo Subject: Re: small kernel kernel option... Message-ID: <20140226222445.GD92037@funkthat.com> Mail-Followup-To: Warner Losh , Luigi Rizzo , arch@freebsd.org References: <20140226214816.GB92037@funkthat.com> <20140226214816.GB92037@funkthat.com> <5BA6FA29-0BBE-45C1-B734-29013C5A2D29@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5BA6FA29-0BBE-45C1-B734-29013C5A2D29@bsdimp.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 26 Feb 2014 14:24:46 -0800 (PST) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:24:47 -0000 Warner Losh wrote this message on Wed, Feb 26, 2014 at 14:02 -0800: > > On Feb 26, 2014, at 1:48 PM, John-Mark Gurney wrote: > > > I'm about to commit a change to sha256 to speed it up, but the cost > > of that speed up is an increase in code/data size from just under 1k > > to almost 9k (as measured on amd64)... this increase is from unrolling > > a loop.. > > > > Maybe we should have a global kernel option, SMALL_KERNEL, or something > > similar that can be used to shrink code size for those that are trying > > to build small embedded devices? > > I?d prefer something that?s more like OPTIMIZE_SIZE or OPTIMIZE_SPEED. Shall I add OPTIMIZE_SIZE and make OPTIMIZE_SPEED the default (by being unspecified)? I don't know of a good way w/ config to make sure only one or the other is specified (besides having some shared header #error if they are.. and though you could say if neither is specified, balance that optimization, but I don't see a use for it... > > Or do we already have this option, but I just don't know about it? > > We don?t have it. We have a billion tiny knobs. Then we could through a bunch of those knobs under an #ifdef OPTIMIZED_SPEED block.. > > I know 8k isn't much, but, a billion here and a billion there and pretty > > soon you're talking about real money.. :) > > yea? There?s lots that can be done here, since most folks don?t bother with this? Every little bit helps... :) Luigi Rizzo wrote this message on Wed, Feb 26, 2014 at 14:05 -0800: > It is very nice that you care about this. > I am just not really sure that for such small numbers the extra data size > matters much. The downside, as you can imagine, is now having to make sure > that lint kernels do not trip on some "unused" warning. I do know that there are some people trying to fix a bootable image in really small flash, like 4meg or something like that, so 8k is .2% of 4meg, so it is something... I do think we could save more space by making some standard things optional, or detecting when code isn't used and isn't part of the public KPI, but we have to start somwhere... Catching build errors shouldn't be too bad as I'm sure we'll have a few arm kernels add this option almost immediately... It does mean we either need to add a LINT-SPEED kernel like we have -NOINET, -NOIP, etc. or simply let tinderbox catch these... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:43:27 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43DFDDEC for ; Wed, 26 Feb 2014 22:43:27 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F1A041563 for ; Wed, 26 Feb 2014 22:43:26 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id f11so3123873qae.3 for ; Wed, 26 Feb 2014 14:43:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=Jc18ZQSr9P88CA6nG4m90YULLrcXDIzSsdyRuC9ztuk=; b=dxkzPMXVSgF4LOXbRTq4F4C5STqKVr6ukskVwOWrCoCMPn22BFsWI8TSzjFmisjjFu BvbEEKu0TGldCkudiAX3eEfTalyy2j4DREFzkhU9RtXNNfRjX/RbwE4sZoVhk4z6CnEx CqUmY0It47b+mxE4v7Fcrc8YpldmV+C1UlJuU3BuhE5Zun9Mk9yEG5C6UwdfN7O8blzF VlVUrZUKeviL1qoZQlk5+aJC5Tad5YKMVsGqihrZTjfykY2u9WsEA8QDEQ8ne9fC0iN/ 24U1+bB/MdSfS/JCtpg0ykKbK2CfdXj96Qd691JYrGyfGZef9mF64AbzApIP3n9fbDwq 0uNg== MIME-Version: 1.0 X-Received: by 10.229.56.200 with SMTP id z8mr14059382qcg.1.1393454605326; Wed, 26 Feb 2014 14:43:25 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Wed, 26 Feb 2014 14:43:25 -0800 (PST) In-Reply-To: <20140226222445.GD92037@funkthat.com> References: <20140226214816.GB92037@funkthat.com> <5BA6FA29-0BBE-45C1-B734-29013C5A2D29@bsdimp.com> <20140226222445.GD92037@funkthat.com> Date: Wed, 26 Feb 2014 14:43:25 -0800 X-Google-Sender-Auth: xl7Cx4zGcxjjmw_h9Rv00TUVzoU Message-ID: Subject: Re: small kernel kernel option... From: Adrian Chadd To: Warner Losh , Luigi Rizzo , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:43:27 -0000 I like the idea of optimise_speed versus optimise_size. Thanks for taking the time to at least consider this. :-) -a On 26 February 2014 14:24, John-Mark Gurney wrote: > Warner Losh wrote this message on Wed, Feb 26, 2014 at 14:02 -0800: >> >> On Feb 26, 2014, at 1:48 PM, John-Mark Gurney wrote: >> >> > I'm about to commit a change to sha256 to speed it up, but the cost >> > of that speed up is an increase in code/data size from just under 1k >> > to almost 9k (as measured on amd64)... this increase is from unrolling >> > a loop.. >> > >> > Maybe we should have a global kernel option, SMALL_KERNEL, or something >> > similar that can be used to shrink code size for those that are trying >> > to build small embedded devices? >> >> I?d prefer something that?s more like OPTIMIZE_SIZE or OPTIMIZE_SPEED. > > Shall I add OPTIMIZE_SIZE and make OPTIMIZE_SPEED the default (by being > unspecified)? I don't know of a good way w/ config to make sure only > one or the other is specified (besides having some shared header #error > if they are.. and though you could say if neither is specified, balance > that optimization, but I don't see a use for it... > >> > Or do we already have this option, but I just don't know about it? >> >> We don?t have it. We have a billion tiny knobs. > > Then we could through a bunch of those knobs under an #ifdef > OPTIMIZED_SPEED block.. > >> > I know 8k isn't much, but, a billion here and a billion there and pretty >> > soon you're talking about real money.. :) >> >> yea? There?s lots that can be done here, since most folks don?t bother with this? > > Every little bit helps... :) > > Luigi Rizzo wrote this message on Wed, Feb 26, 2014 at 14:05 -0800: >> It is very nice that you care about this. >> I am just not really sure that for such small numbers the extra data size >> matters much. The downside, as you can imagine, is now having to make sure >> that lint kernels do not trip on some "unused" warning. > > I do know that there are some people trying to fix a bootable image > in really small flash, like 4meg or something like that, so 8k is > .2% of 4meg, so it is something... I do think we could save more > space by making some standard things optional, or detecting when code > isn't used and isn't part of the public KPI, but we have to start > somwhere... > > Catching build errors shouldn't be too bad as I'm sure we'll have a few > arm kernels add this option almost immediately... It does mean we > either need to add a LINT-SPEED kernel like we have -NOINET, -NOIP, > etc. or simply let tinderbox catch these... > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Feb 26 22:58:32 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8900E2DE for ; Wed, 26 Feb 2014 22:58:32 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A3A21663 for ; Wed, 26 Feb 2014 22:58:32 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id bj1so1611977pad.31 for ; Wed, 26 Feb 2014 14:58:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=LVayOy4tMeMDKrqwE0SXOkeoOiKnsPLrKo3T1tA3+qs=; b=wOyu0deN2RYJZFM5Q1odCrJohcvbdODkxYct59kEtZzzQWetGiCL3AS36uWPh8ZDiz K+C4FuchZlXzNZ7E48w34QGCsIPA6+mJmwy1mzBFvDgKz7BFDAoMa0U1Yb8eYlgoSfLd s/asC45RLTvNfHXoZB5fJa0XYMicuER/S7mONMMuhJBh+yjwk9BcgiKQjl7HeRmPJ5em 3/JTPQ67mN09WClHysHzZDM+HjJv8HvdEY1FjWmFYvhAtQuA1iPO4+JzXfi7v8R9XLBB giScvGLTWV1Laonl+VKlcjpgum3Zrw0M4JdDCHJNO48xsVyxNIA76Xxifuk8Boe4eAUn htXQ== X-Received: by 10.67.1.202 with SMTP id bi10mr11790851pad.68.1393455512069; Wed, 26 Feb 2014 14:58:32 -0800 (PST) Received: from [10.64.26.169] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id gj9sm6998585pbc.7.2014.02.26.14.58.23 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Feb 2014 14:58:30 -0800 (PST) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: small kernel kernel option... From: Warner Losh In-Reply-To: <20140226222445.GD92037@funkthat.com> Date: Wed, 26 Feb 2014 14:58:20 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140226214816.GB92037@funkthat.com> <20140226214816.GB92037@funkthat.com> <5BA6FA29-0BBE-45C1-B734-29013C5A2D29@bsdimp.com> <20140226222445.GD92037@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Feb 2014 22:58:32 -0000 On Feb 26, 2014, at 2:24 PM, John-Mark Gurney wrote: > Warner Losh wrote this message on Wed, Feb 26, 2014 at 14:02 -0800: >>=20 >> On Feb 26, 2014, at 1:48 PM, John-Mark Gurney = wrote: >>=20 >>> I'm about to commit a change to sha256 to speed it up, but the cost >>> of that speed up is an increase in code/data size from just under 1k >>> to almost 9k (as measured on amd64)... this increase is from = unrolling >>> a loop.. >>>=20 >>> Maybe we should have a global kernel option, SMALL_KERNEL, or = something >>> similar that can be used to shrink code size for those that are = trying >>> to build small embedded devices? >>=20 >> I?d prefer something that?s more like OPTIMIZE_SIZE or = OPTIMIZE_SPEED. >=20 > Shall I add OPTIMIZE_SIZE and make OPTIMIZE_SPEED the default (by = being > unspecified)? I don't know of a good way w/ config to make sure only > one or the other is specified (besides having some shared header = #error > if they are.. and though you could say if neither is specified, = balance > that optimization, but I don't see a use for it=85 I=92d toss it into a header. No reason that config needs to error out=85 Warner From owner-freebsd-arch@FreeBSD.ORG Thu Feb 27 23:05:31 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 642A87F7 for ; Thu, 27 Feb 2014 23:05:31 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 257A7113A for ; Thu, 27 Feb 2014 23:05:31 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id D03A3B808F; Fri, 28 Feb 2014 00:05:28 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id B6C6228497; Fri, 28 Feb 2014 00:05:28 +0100 (CET) Date: Fri, 28 Feb 2014 00:05:28 +0100 From: Jilles Tjoelker To: Peter Jeremy Subject: Re: terminfo Message-ID: <20140227230528.GB8295@stack.nl> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> <20140226085939.GC2705@server.rulingia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140226085939.GC2705@server.rulingia.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Ed Schouten , arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Feb 2014 23:05:31 -0000 On Wed, Feb 26, 2014 at 07:59:39PM +1100, Peter Jeremy wrote: > On 2014-Feb-23 14:49:25 +0100, Ed Schouten wrote: > >- Who cares about double-width characters? I don't know a single > >program that uses this. In the worst case, you can just use fullwidth > >CJK for certain characters. > I wrote an app at $previous-job that used double-height/double-width > characters in xterm. I might have been able to use unicode but then > everyone who wanted to use in would need to have configured their xterm > for UTF-8 - and I don't believe had done that. > >- Who cares about 88 colour support? Just use 256 colours. > Assuming your app supports that. > >- Who cares about ACS? Unicode already has those characters. > Again, that assumes your app supports unicode. > >People are nowadays only interested in having a 16 or 256 colour, > >UTF-8 enabled terminal. > And running legacy apps that predate unicode. The proposal here is not to change terminal emulators to remove all legacy features. Rather, it is proposed to create one termcap or terminfo entry that will work with all modern terminal emulators. Applications using termcap or terminfo (either directly or indirectly through ncurses) will then stop using various legacy features. In the case of features like legacy ACS and 88-colour, the application does not lose any user-visible functionality. In the long run, support for the legacy features will probably deteriorate, especially in terminal emulators with maintainers less passionate about the details of terminal emulation than xterm's maintainer. I think it is worth it to simplify terminals. Instead of having people choose how they want their terminal to work improperly, provide one way and concentrate on making that work as well as possible (no visual artifacts, 256 colours, Unicode, key combinations like Ctrl+Home, etc.) I also like the idea of some applications sending control sequences directly, without using curses or even termcap/terminfo. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Fri Feb 28 05:15:06 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 102374A0 for ; Fri, 28 Feb 2014 05:15:06 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D965D1FF1 for ; Fri, 28 Feb 2014 05:15:05 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s1S5EtMo062710 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 27 Feb 2014 21:14:58 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <53101B49.2010400@freebsd.org> Date: Fri, 28 Feb 2014 13:14:49 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: Re: small kernel kernel option... References: <20140226214816.GB92037@funkthat.com> In-Reply-To: <20140226214816.GB92037@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 05:15:06 -0000 On 2/27/14, 5:48 AM, John-Mark Gurney wrote: > I'm about to commit a change to sha256 to speed it up, but the cost > of that speed up is an increase in code/data size from just under 1k > to almost 9k (as measured on amd64)... this increase is from unrolling > a loop.. > > Maybe we should have a global kernel option, SMALL_KERNEL, or something > similar that can be used to shrink code size for those that are trying > to build small embedded devices... > > Or do we already have this option, but I just don't know about it? > > I know 8k isn't much, but, a billion here and a billion there and pretty > soon you're talking about real money.. :) > > Comments? > it's the same as the old "space vs time" optimisations in UFS. maybe "OPTIMIZE_FOR_SPACE" vs "OPTIMIZE_FOR_SPEED" From owner-freebsd-arch@FreeBSD.ORG Fri Feb 28 11:43:07 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9999DEF1 for ; Fri, 28 Feb 2014 11:43:07 +0000 (UTC) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 06EC41239 for ; Fri, 28 Feb 2014 11:43:06 +0000 (UTC) Received: from server.rulingia.com (c220-239-232-212.belrs5.nsw.optusnet.com.au [220.239.232.212]) by vps.rulingia.com (8.14.7/8.14.7) with ESMTP id s1SBgUkY002985 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Fri, 28 Feb 2014 22:42:30 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.7/8.14.7) with ESMTP id s1SBgOC9039152 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 28 Feb 2014 22:42:24 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.7/8.14.7/Submit) id s1SBgOJ3039151 for arch@FreeBSD.org; Fri, 28 Feb 2014 22:42:24 +1100 (EST) (envelope-from peter) Date: Fri, 28 Feb 2014 22:42:24 +1100 From: Peter Jeremy To: arch@FreeBSD.org Subject: Re: small kernel kernel option... Message-ID: <20140228114224.GE2705@server.rulingia.com> References: <20140226214816.GB92037@funkthat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="q9KOos5vDmpwPx9o" Content-Disposition: inline In-Reply-To: <20140226214816.GB92037@funkthat.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 11:43:07 -0000 --q9KOos5vDmpwPx9o Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2014-Feb-26 13:48:16 -0800, John-Mark Gurney wrote: >I'm about to commit a change to sha256 to speed it up, but the cost >of that speed up is an increase in code/data size from just under 1k >to almost 9k (as measured on amd64)... this increase is from unrolling >a loop.. Out of interest, how much of a speedup and what CPU/compiler combinations did you test your change on? I ask because several years ago, I tried about 7 different SHA-256 implementations (basically, all the C ones I could easily find in FreeBSD and ports I had installed, as well as one I tweaked myself) across a range of CPUs and compilers. I found that not only was there a very wide variation in speed between implementations but that the best on one CPU often ran quite poorly on another and unrolling loops didn't necessarily help. --=20 Peter Jeremy --q9KOos5vDmpwPx9o Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iKYEARECAGYFAlMQdiBfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDBCRjc3QTcyNTg5NEVCRTY0RjREN0VFRUZF OEE0N0JGRjAwRkI4ODcACgkQ/opHv/APuIfJAgCcCDPc8M7zXY+CXQP2RLcPgA2/ I+sAoIKZYCrb1xoKofOPwniloOR7yofx =WjFN -----END PGP SIGNATURE----- --q9KOos5vDmpwPx9o-- From owner-freebsd-arch@FreeBSD.ORG Fri Feb 28 18:18:06 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D8C42868 for ; Fri, 28 Feb 2014 18:18:06 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9CC28171F for ; Fri, 28 Feb 2014 18:18:06 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s1SII5Xd077871 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Feb 2014 10:18:06 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s1SII5jL077870; Fri, 28 Feb 2014 10:18:05 -0800 (PST) (envelope-from jmg) Date: Fri, 28 Feb 2014 10:18:05 -0800 From: John-Mark Gurney To: Peter Jeremy Subject: Re: small kernel kernel option... Message-ID: <20140228181804.GQ47921@funkthat.com> Mail-Followup-To: Peter Jeremy , arch@freebsd.org References: <20140226214816.GB92037@funkthat.com> <20140228114224.GE2705@server.rulingia.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140228114224.GE2705@server.rulingia.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 28 Feb 2014 10:18:06 -0800 (PST) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 18:18:06 -0000 Peter Jeremy wrote this message on Fri, Feb 28, 2014 at 22:42 +1100: > On 2014-Feb-26 13:48:16 -0800, John-Mark Gurney wrote: > >I'm about to commit a change to sha256 to speed it up, but the cost > >of that speed up is an increase in code/data size from just under 1k > >to almost 9k (as measured on amd64)... this increase is from unrolling > >a loop.. > > Out of interest, how much of a speedup and what CPU/compiler > combinations did you test your change on? I ask because several years > ago, I tried about 7 different SHA-256 implementations (basically, all > the C ones I could easily find in FreeBSD and ports I had installed, > as well as one I tweaked myself) across a range of CPUs and compilers. > I found that not only was there a very wide variation in speed between > implementations but that the best on one CPU often ran quite poorly on > another and unrolling loops didn't necessarily help. I did not do an exhaustive search.. I only benchmarked the two easy ones, the one from libmd and the kernel one... I ran my tests on an A10-5700@3.4GHz, Core i7@2GHz (though under MacOSX) and an Opteron-4228 HE@2.8Ghz... All tests were on amd64... There were a few people who also ran the tests for me but I don't remeber what processors they ran on.. In all the cases we saw an improvement, and mostly saw a ~20% improvement by using cperciva's libmd version than the kernel version... These were proven w/ ministat... Part of the reason I didn't to an exhaustive search is that many implementations (OpenSSL, NSS) are very difficult to extract w/o major work.. If you'd like to run my test suite, it can be run by d/l'ing: https://www.funkthat.com/~jmg/sha256.test.unr.tgz just run the script gennumbers and wait for a while... it'll compile and validate and perform the tests.. This also includes the tests to test various numbers of loop unrolling which exposed some weird timing behavior... Enough that the only option is either unrolled or completely rolled, no partial loops.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Feb 28 19:58:57 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D131AAE for ; Fri, 28 Feb 2014 19:58:57 +0000 (UTC) Received: from mail-la0-x243.google.com (mail-la0-x243.google.com [IPv6:2a00:1450:4010:c03::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 18AEB11A9 for ; Fri, 28 Feb 2014 19:58:56 +0000 (UTC) Received: by mail-la0-f67.google.com with SMTP id gl10so1019610lab.10 for ; Fri, 28 Feb 2014 11:58:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=KxWCXsr5GVMiwid8IXYZPgZ2y4vHm3C1NPeCBouFimY=; b=gFGgBlAFOUUVppVJ6jD2IAC5pgx2ljDdX0jpKVG82RSCtCt73UwP/IMmphG0Jhcws0 9WZ1DG4rGVicX+vO06FttTqIEkX9I1+NMGqrxnzo3XQtiEKcStJWQTtF2QN26bXAbQ4O cLhWCmSD0lL7FDtmCHDP3cEtVwXRjqSrFJnFwFNJ4p8ROQaxzhToVqw6ZZN4ytJbtwBX Urj2qAIUTybCBKx7B116GXL/kTATRTuD9Kgjge5ByEbJU5/HQzWHy/eOZ7pSmyeHkqXP I7qiUAucHNcuHAt7oL1aVaGWjSX20XwvFUuKxloAzQx1kgFSti/TCaUpWxE1dy+HHx7O m5hw== MIME-Version: 1.0 X-Received: by 10.153.8.194 with SMTP id dm2mr3084822lad.54.1393617535191; Fri, 28 Feb 2014 11:58:55 -0800 (PST) Received: by 10.112.55.69 with HTTP; Fri, 28 Feb 2014 11:58:55 -0800 (PST) Received: by 10.112.55.69 with HTTP; Fri, 28 Feb 2014 11:58:55 -0800 (PST) Date: Fri, 28 Feb 2014 20:58:55 +0100 Message-ID: Subject: From: Grazia Valentino To: freebsd-arch@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Feb 2014 19:58:57 -0000 Ho ricevuto un premio una ricarica del valore100 From owner-freebsd-arch@FreeBSD.ORG Sun Mar 2 16:55:17 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DF98A37 for ; Sun, 2 Mar 2014 16:55:17 +0000 (UTC) Received: from mail-pd0-x236.google.com (mail-pd0-x236.google.com [IPv6:2607:f8b0:400e:c02::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E2E4E12ED for ; Sun, 2 Mar 2014 16:55:16 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id g10so2779857pdj.27 for ; Sun, 02 Mar 2014 08:55:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; bh=968Hb13M3Ejw8ied2GV49G6bFKVCS5bA/JcgnSOK5wU=; b=rg899mTnyf0CYYCVDaW3kf4ubMDADpxF8aAjeSJZTLfC50OF4ucg3qNu4tng0v/bAX Uv5WEKV5qXmvSccUiRB5ZOVG7WvGW8VLfcr18N5DazJuvDsxHnnkYDNi3m3QubV7C6PM 5EojXJWMNcve68OZmg/yYhCCNJFZ4p6FZwHLt+rpLlh5AZycLTyfYpHAIBptb8kqfMg4 1QIxsNPXsFa2h3wUghA3Se+othmhTCVxauyQSanGI0ohkHUOAvafpkVpbV7+QYO2EEN7 +t3NE9LS31eZGUHvx7jlu3RQvniX3EFtSWh/ZFeKmXVTwTV8jiwzWDZ9cw/gQ57hofuj 0gAQ== X-Received: by 10.67.23.135 with SMTP id ia7mr14648483pad.5.1393779316587; Sun, 02 Mar 2014 08:55:16 -0800 (PST) Received: from zhabar.gateway.2wire.net (76-253-2-5.lightspeed.sntcca.sbcglobal.net. [76.253.2.5]) by mx.google.com with ESMTPSA id qf7sm65226582pac.14.2014.03.02.08.55.15 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Sun, 02 Mar 2014 08:55:16 -0800 (PST) Sender: Justin Hibbits Date: Sun, 2 Mar 2014 08:55:11 -0800 From: jhibbits@freebsd.org To: Subject: newcons fb driver Message-ID: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; powerpc64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Mar 2014 16:55:17 -0000 I've been looking into the slowness of the newcons framebuffer backend, and after discussing with Nathan Whitehorn, we've determined that the slowness is caused by single byte writes to the framebuffer, which are very much suboptimal. His conclusion is that, on nVidia hardware, the card first performs a read, a very slow operation on the hardware, then a single-byte write, and I'm thinking the same. So, to accomodate this limitation, I have a question and proposal: (q) From looking at the code, it appears to support, to some degree, a background image (the character blitting uses a mask, it doesn't just write background in that mask). Is this truly the case? (p) If it is the case that it supports a background image, I'm thinking it would make sense to buffer the framebuffer, such that we write to the internal representation, and blit whole words or lines at a time to the hardware. With an 8-bit buffer, at 1920x1080 (common resolution) we would be using approximately 2MB for the buffer. On memory constrained devices this can be quite significant, so it becomes a tradeoff. Thoughts? - Justin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 00:11:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 99DE0A6E; Mon, 3 Mar 2014 00:11:12 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 34533196B; Mon, 3 Mar 2014 00:11:12 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id j5so2874283qaq.28 for ; Sun, 02 Mar 2014 16:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ia3dBPsKT8KMmv1rstIuXhm/u8wxoZOF0kSKvHkcYro=; b=cLmlqomivb3Gt2+rZM6VU+yLmvmHrGbsQG+1VpbdrLWJ25nU+MtoA9wr2eqVIX4BG6 Rtyod72g+tUn3pCOoBOg46LAGjXmFPnThk5lRZUZmh0Ul9tahJMPqnYOZwayGr91illy S4I6tfLu1AyQt5NtCvWIptAxAKcf8IDv+7ZJT7DGtjxRYiBeocDVdKqb+hFJhXZDoFlJ je8gQU8Hh6KnSoRsJNWwP7caJGsjzv3bjcCoTQ3oBYr7NzY7ViWy2E3qt6H4seaAhdlN nmbxuyp/cEtHQM9Ha7HiH2iBvWVZfqU5o4hiTUwA5icB6Ho+5VYwhjQL0E0z+lSiFXs0 XDSg== MIME-Version: 1.0 X-Received: by 10.224.136.67 with SMTP id q3mr15634266qat.8.1393805471350; Sun, 02 Mar 2014 16:11:11 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Sun, 2 Mar 2014 16:11:11 -0800 (PST) In-Reply-To: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> Date: Sun, 2 Mar 2014 16:11:11 -0800 X-Google-Sender-Auth: glDqLmthvkkQcDmYsh-n9kFCd14 Message-ID: Subject: Re: newcons fb driver From: Adrian Chadd To: Justin Hibbits , Poul-Henning Kamp Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 00:11:12 -0000 .. i'm pretty sure there was a reason for why it's done in byte sizes. Maybe speak to phk? -a On 2 March 2014 08:55, wrote: > I've been looking into the slowness of the newcons framebuffer backend, > and after discussing with Nathan Whitehorn, we've determined that the > slowness is caused by single byte writes to the framebuffer, which are > very much suboptimal. His conclusion is that, on nVidia hardware, the > card first performs a read, a very slow operation on the hardware, then > a single-byte write, and I'm thinking the same. So, to accomodate this > limitation, I have a question and proposal: > > (q) From looking at the code, it appears to support, to some degree, a > background image (the character blitting uses a mask, it doesn't just > write background in that mask). Is this truly the case? > > (p) If it is the case that it supports a background image, I'm thinking > it would make sense to buffer the framebuffer, such that we write to > the internal representation, and blit whole words or lines at a time > to the hardware. With an 8-bit buffer, at 1920x1080 (common resolution) > we would be using approximately 2MB for the buffer. On memory > constrained devices this can be quite significant, so it becomes a > tradeoff. > > Thoughts? > > - Justin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 06:52:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0BCF178; Mon, 3 Mar 2014 06:52:23 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id B24E3FF6; Mon, 3 Mar 2014 06:52:23 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 5C0543EB36; Mon, 3 Mar 2014 06:52:16 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.7/8.14.7) with ESMTP id s236qFFx042131; Mon, 3 Mar 2014 06:52:16 GMT (envelope-from phk@phk.freebsd.dk) To: Adrian Chadd Subject: Re: newcons fb driver In-reply-to: From: "Poul-Henning Kamp" References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 03 Mar 2014 06:52:15 +0000 Message-ID: <42130.1393829535@critter.freebsd.dk> Cc: Justin Hibbits , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 06:52:24 -0000 In message , Adrian Chadd writes: >.. i'm pretty sure there was a reason for why it's done in byte sizes. >Maybe speak to phk? Buggy video hardware, which does not do larger writes correctly, the most recent one being an Intel Laptop, but I can't remember which. At the very least, byte writes needs to be a boot-time option. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 06:58:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 892CB404; Mon, 3 Mar 2014 06:58:20 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 34ECC89; Mon, 3 Mar 2014 06:58:20 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so3395716qcx.27 for ; Sun, 02 Mar 2014 22:58:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hrtviet2s2nVnr7sbT1dOuVcRSuDEa5p0wttZ6ejT4A=; b=bWDABcBEWLbcWBSP4S4mJVUiFMQA99MpEhNqfMwIjI9isKu5PKvhRrZu1yskWcFSWt 5KEZtx0ecXKyX7ZzAt+OBLbeKowLSKheVMGutebjQ17f1jJRICtZjSxGDFmOljXQgbmH NvmvcE43Y/mI/09MFRLVX6lGZtgVB2G5dYnA0OcPKe2ekcbx+4JMye7XmfEzMAQqqoVt vpWZ5zP/oBNF2As5I5TS7puDCvd9nLAssPmofZKN6NsL6AwEeqeyYMkKeDxfQI8//lBs WQt18dsl05Y+9OEzYvINalHPno4GNEFnyeO131plDj2ASBDhuqbSq9z+zRzUrW2/4Osf IVkQ== MIME-Version: 1.0 X-Received: by 10.140.22.167 with SMTP id 36mr20657352qgn.0.1393829899307; Sun, 02 Mar 2014 22:58:19 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Sun, 2 Mar 2014 22:58:19 -0800 (PST) In-Reply-To: <42130.1393829535@critter.freebsd.dk> References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> <42130.1393829535@critter.freebsd.dk> Date: Sun, 2 Mar 2014 22:58:19 -0800 X-Google-Sender-Auth: yeFZXQzFYPJN_jThrfAip1bBW_o Message-ID: Subject: Re: newcons fb driver From: Adrian Chadd To: Poul-Henning Kamp Content-Type: text/plain; charset=ISO-8859-1 Cc: Justin Hibbits , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 06:58:20 -0000 On 2 March 2014 22:52, Poul-Henning Kamp wrote: > In message > , Adrian Chadd writes: > >>.. i'm pretty sure there was a reason for why it's done in byte sizes. >>Maybe speak to phk? > > Buggy video hardware, which does not do larger writes correctly, > the most recent one being an Intel Laptop, but I can't remember which. > > At the very least, byte writes needs to be a boot-time option. Is there a way to detect this? Ie, writing to video memory and then reading it back a byte at a time to see if the patterns match? -a From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 07:29:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30558E02; Mon, 3 Mar 2014 07:29:26 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id E3827315; Mon, 3 Mar 2014 07:29:24 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id DAD843EB96; Mon, 3 Mar 2014 07:29:23 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.7/8.14.7) with ESMTP id s237TNAK051907; Mon, 3 Mar 2014 07:29:23 GMT (envelope-from phk@phk.freebsd.dk) To: Adrian Chadd Subject: Re: newcons fb driver In-reply-to: From: "Poul-Henning Kamp" References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> <42130.1393829535@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 03 Mar 2014 07:29:23 +0000 Message-ID: <51906.1393831763@critter.freebsd.dk> Cc: Justin Hibbits , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 07:29:26 -0000 In message , Adrian Chadd writes: >> Buggy video hardware, which does not do larger writes correctly, >> the most recent one being an Intel Laptop, but I can't remember which. >> >> At the very least, byte writes needs to be a boot-time option. > >Is there a way to detect this? Ie, writing to video memory and then >reading it back a byte at a time to see if the patterns match? Never tried that. Given that the display was bogus, it would probably work. Maybe do a "verify" on the first 10 screen updates ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 10:48:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87CAF61C for ; Mon, 3 Mar 2014 10:48:44 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5D063763 for ; Mon, 3 Mar 2014 10:48:43 +0000 (UTC) Received: from Julian-MBP3.local ([12.157.112.67]) (authenticated bits=0) by vps1.elischer.org (8.14.7/8.14.7) with ESMTP id s23Amg8u073647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 3 Mar 2014 02:48:43 -0800 (PST) (envelope-from julian@freebsd.org) Message-ID: <53145E05.1070207@freebsd.org> Date: Mon, 03 Mar 2014 02:48:37 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: newcons fb driver References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> In-Reply-To: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 10:48:44 -0000 On 3/2/14, 8:55 AM, jhibbits@freebsd.org wrote: > I've been looking into the slowness of the newcons framebuffer backend, > and after discussing with Nathan Whitehorn, we've determined that the > slowness is caused by single byte writes to the framebuffer, which are > very much suboptimal. His conclusion is that, on nVidia hardware, the > card first performs a read, a very slow operation on the hardware, then > a single-byte write, and I'm thinking the same. So, to accomodate this > limitation, I have a question and proposal: > > (q) From looking at the code, it appears to support, to some degree, a > background image (the character blitting uses a mask, it doesn't just > write background in that mask). Is this truly the case? > > (p) If it is the case that it supports a background image, I'm thinking > it would make sense to buffer the framebuffer, such that we write to > the internal representation, and blit whole words or lines at a time > to the hardware. With an 8-bit buffer, at 1920x1080 (common resolution) > we would be using approximately 2MB for the buffer. On memory > constrained devices this can be quite significant, so it becomes a > tradeoff. oh how history repeats itself.. hit this in the newcons driver (never completed) in 1993. the oldest hardware had this problem.. solution: keep a virtual framebuffer and update the hardware (only lines that changed) on 1/50 of a second ticks... (or some other timer value faster than that) requires keeping the data in smaller subbuffers to track whether the entire creen needs ot be updated. > > Thoughts? > > - Justin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 14:21:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B776CC3 for ; Mon, 3 Mar 2014 14:21:48 +0000 (UTC) Received: from mail-ie0-f171.google.com (mail-ie0-f171.google.com [209.85.223.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 36416E06 for ; Mon, 3 Mar 2014 14:21:48 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id ar20so940686iec.30 for ; Mon, 03 Mar 2014 06:21:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=gYuIoHtIKpe0fWzH8bWHjaJSPTqzKEDwyDIP1ZRFvHI=; b=QhT9CyM5LXLLOCVLvFOlGY4hDYR5raVGA/Z+r7NjtZxW3jncb41tKo8HAoH8xLMXtr ZGfo/554vpO3PcZiwOOXprklnJR8htdcbm6R5hr0F17xDdIuZUaMvujJPfQQHWE1DnW9 HKmfZsYJWj6vx4UO65mjRC7x27s08IuQepcgF7DKkFl2VlygCKu7R4WGGlMRy+pFEtMj rjsZADjhIawdbYtvU9p/1VATzT45HSqX5EdxBYfjI0gnCPi34Zts2ZMgZzp5rur9HOm/ +hZhzAJgGYp6V/3JUbOHiqN+RURXAHYC0UgsTRprjaW5jkqK0vqfUnWeyaO0y2YA99hK YfHg== X-Gm-Message-State: ALoCoQnVE3rPrUdRPz98xG5GLykCSnvwtbbbeK4P3IQlNHSlheHHo4et09MzBPDCRXIkRO/dox3n X-Received: by 10.50.253.70 with SMTP id zy6mr22233426igc.28.1393856507325; Mon, 03 Mar 2014 06:21:47 -0800 (PST) Received: from netflix-mac.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id y9sm39651440igl.7.2014.03.03.06.21.46 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Mar 2014 06:21:46 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: newcons fb driver From: Warner Losh In-Reply-To: <53145E05.1070207@freebsd.org> Date: Mon, 3 Mar 2014 07:21:45 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> <53145E05.1070207@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 14:21:48 -0000 On Mar 3, 2014, at 3:48 AM, Julian Elischer wrote: > On 3/2/14, 8:55 AM, jhibbits@freebsd.org wrote: >> I've been looking into the slowness of the newcons framebuffer = backend, >> and after discussing with Nathan Whitehorn, we've determined that the >> slowness is caused by single byte writes to the framebuffer, which = are >> very much suboptimal. His conclusion is that, on nVidia hardware, = the >> card first performs a read, a very slow operation on the hardware, = then >> a single-byte write, and I'm thinking the same. So, to accomodate = this >> limitation, I have a question and proposal: >>=20 >> (q) =46rom looking at the code, it appears to support, to some = degree, a >> background image (the character blitting uses a mask, it doesn't just >> write background in that mask). Is this truly the case? >>=20 >> (p) If it is the case that it supports a background image, I'm = thinking >> it would make sense to buffer the framebuffer, such that we write to >> the internal representation, and blit whole words or lines at a time >> to the hardware. With an 8-bit buffer, at 1920x1080 (common = resolution) >> we would be using approximately 2MB for the buffer. On memory >> constrained devices this can be quite significant, so it becomes a >> tradeoff. > oh how history repeats itself.. hit this in the newcons driver (never = completed) in 1993. >=20 > the oldest hardware had this problem.. > solution: > keep a virtual framebuffer and update the hardware (only lines that = changed) on 1/50 of a second ticks... > (or some other timer value faster than that) >=20 > requires keeping the data in smaller subbuffers to track whether the = entire creen needs ot be updated. I know back in this kind of day the reason we had to do byte writes was = because 8-bit video cards would drop every other character if you did word writes of the same data=85 = But that=92s back in the earliest of the early days when 386sx-25=92s ran the face of the earth and 8-bit video = cards had not yet gone extinct=85 But I think that either a verify option or a simpler =91do single byte = writes=92 if the larger writes cause hangish problems. Since this is a big =91go uber-slow=92 option, we shouldn=92t = do it unless we really sure we need this. Warner >>=20 >> Thoughts? >>=20 >> - Justin >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" >>=20 >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 16:49:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2F0D980 for ; Mon, 3 Mar 2014 16:49:40 +0000 (UTC) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 74715DAD for ; Mon, 3 Mar 2014 16:49:40 +0000 (UTC) Received: from torb.pix.net (torb.pix.net [IPv6:2001:470:e254:10:12dd:b1ff:febf:eca9]) (authenticated bits=0) by hydra.pix.net (8.14.5/8.14.5) with ESMTP id s23GncVI099602; Mon, 3 Mar 2014 11:49:38 -0500 (EST) (envelope-from lidl@pix.net) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98 at mail.pix.net Message-ID: <5314B2A2.3060100@pix.net> Date: Mon, 03 Mar 2014 11:49:38 -0500 From: Kurt Lidl User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: newcons fb driver References: <42130.1393829535@critter.freebsd.dk> In-Reply-To: <42130.1393829535@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 16:49:40 -0000 > In message > , Adrian Chadd writes: > >>.. i'm pretty sure there was a reason for why it's done in byte sizes. >>Maybe speak to phk? > > Buggy video hardware, which does not do larger writes correctly, > the most recent one being an Intel Laptop, but I can't remember which. > > At the very least, byte writes needs to be a boot-time option. Intel Atom boards, as I recall: http://freshbsd.org/commit/freebsd/r237203 -Kurt From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 17:44:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E1DD53BE for ; Mon, 3 Mar 2014 17:44:20 +0000 (UTC) Received: from mail-ea0-x22c.google.com (mail-ea0-x22c.google.com [IPv6:2a00:1450:4013:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7BF14371 for ; Mon, 3 Mar 2014 17:44:20 +0000 (UTC) Received: by mail-ea0-f172.google.com with SMTP id l9so4741803eaj.3 for ; Mon, 03 Mar 2014 09:44:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=nC/Y4r43zOqPqXTtKDPQUgv4wINuVT1wuPAb/jt8aig=; b=x0/Tz2C3E+g/5wFXkVWG69ew9g5bCanUV+bVuPLLH4vR/IPv+ujwRJ5iL0wdcGt5P2 j9VSD0L3OQYrA2fJ4eifBdO+uS/cj8fGakCuBjqRSEx6qeLoiESqxKLq2iLkKX2K3Y9s ewtYrOCElYWZB34OQjKKgBYHH8xzM/3DlZKWaIuBTN5FUS5PHuuBhtLdfrzRdc1s+6ch FI9NwPdFKFxQUKf/QSDCePDnqhVPRoPhlPfXKSQcVbQySnHGAUjy0a7tHn/gEJ5nd4gl okLaABrMWYdu3OBG5e3UE9e81msLTST5pJz098wSft0U6ZOk+IGxrqez2Kl4LrvfUxjD jX7w== MIME-Version: 1.0 X-Received: by 10.205.106.7 with SMTP id ds7mr28166bkc.158.1393868658553; Mon, 03 Mar 2014 09:44:18 -0800 (PST) Sender: chmeeedalf@gmail.com Received: by 10.205.21.68 with HTTP; Mon, 3 Mar 2014 09:44:18 -0800 (PST) In-Reply-To: <5314B2A2.3060100@pix.net> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> Date: Mon, 3 Mar 2014 09:44:18 -0800 X-Google-Sender-Auth: D5Vd_A0iErLy9TtplyqA17off64 Message-ID: Subject: Re: newcons fb driver From: Justin Hibbits To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:44:20 -0000 On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: >> In message > mail.gmail.com> >> >> , Adrian Chadd writes: >> >>> .. i'm pretty sure there was a reason for why it's done in byte sizes. >>> Maybe speak to phk? >> >> >> Buggy video hardware, which does not do larger writes correctly, >> the most recent one being an Intel Laptop, but I can't remember which. >> >> At the very least, byte writes needs to be a boot-time option. > > > Intel Atom boards, as I recall: > > http://freshbsd.org/commit/freebsd/r237203 > > -Kurt All great knowledge, but really only answers half of what I'm looking for (always good to know potential pitfalls). Assuming a tunable/sysctl is added, what's the best way to optimize from my original post? Use a backing buffer (potentially with a tunable to not)? Or assume we don't support background images, and write the background color in the masked pixels? - Justin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 17:46:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A837D496 for ; Mon, 3 Mar 2014 17:46:48 +0000 (UTC) Received: from mail-pb0-f41.google.com (mail-pb0-f41.google.com [209.85.160.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7AD5F391 for ; Mon, 3 Mar 2014 17:46:48 +0000 (UTC) Received: by mail-pb0-f41.google.com with SMTP id jt11so4069338pbb.0 for ; Mon, 03 Mar 2014 09:46:42 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Hy8V9qo4KWoMoeSukhVFznsVGBuWqjJY+3fZUV55zvQ=; b=MsbM0ylhI9xBD6Ed7HdetMUy6TiZRNiqCb1R6rCR212vZl/zLc3izeuGYDpUiQdOym pG1o0kbvl00luHrKhs37xVuYRVZQZESl7OxFMy98zKtVbY9NatJzwvg4StlVdOdUSk64 DPByHuguhNaT+IQAhMO9g24yBJ9nAhdqO6u8RjnrbR1MOPHPa1K8FVVpaj0KyRVB8OEL X9hUYUQbq4LgeaB+Vx9uW5Dt/R6G0DDIUQfBci9fUNzPSenAbu/0ZTiNSEmdLrD6TKJF Brv8WYoX4vihbMVtjS3PPyWc2UuUHiDT1vazVZPYlcwWd+Q0rgUUxSwuv5+V/bHKzZYY mV0w== X-Gm-Message-State: ALoCoQlQnV/THyXHEiabTxvq2DBZV5I3+7QaXHZfFaO5zJ4aZ3pt8mFdxZkHt+E79te1mNXYNM9R X-Received: by 10.68.232.134 with SMTP id to6mr4961976pbc.107.1393868802363; Mon, 03 Mar 2014 09:46:42 -0800 (PST) Received: from [10.64.27.32] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id yx3sm11893891pbb.6.2014.03.03.09.46.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 03 Mar 2014 09:46:41 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: newcons fb driver From: Warner Losh In-Reply-To: Date: Mon, 3 Mar 2014 10:46:38 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> To: Justin Hibbits X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:46:48 -0000 On Mar 3, 2014, at 10:44 AM, Justin Hibbits = wrote: > On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: >>> In message >> mail.gmail.com> >>>=20 >>> , Adrian Chadd writes: >>>=20 >>>> .. i'm pretty sure there was a reason for why it's done in byte = sizes. >>>> Maybe speak to phk? >>>=20 >>>=20 >>> Buggy video hardware, which does not do larger writes correctly, >>> the most recent one being an Intel Laptop, but I can't remember = which. >>>=20 >>> At the very least, byte writes needs to be a boot-time option. >>=20 >>=20 >> Intel Atom boards, as I recall: >>=20 >> http://freshbsd.org/commit/freebsd/r237203 >>=20 >> -Kurt >=20 > All great knowledge, but really only answers half of what I'm looking > for (always good to know potential pitfalls). Assuming a > tunable/sysctl is added, what's the best way to optimize from my > original post? Use a backing buffer (potentially with a tunable to > not)? Or assume we don't support background images, and write the > background color in the masked pixels? Most of the issues are with the CHARACTER buffer, not the actual frame = (pixel) buffer=85 Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 17:56:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D328B9B2 for ; Mon, 3 Mar 2014 17:56:34 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id BD46D666 for ; Mon, 3 Mar 2014 17:56:34 +0000 (UTC) Received: from AlfredMacbookAir.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id 49DDF1A3C19 for ; Mon, 3 Mar 2014 09:56:26 -0800 (PST) Message-ID: <5314C249.9000808@mu.org> Date: Mon, 03 Mar 2014 09:56:25 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: newcons fb driver References: <20140302085511.6354f9ac@zhabar.gateway.2wire.net> <42130.1393829535@critter.freebsd.dk> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 17:56:34 -0000 On 3/2/14, 10:58 PM, Adrian Chadd wrote: > On 2 March 2014 22:52, Poul-Henning Kamp wrote: >> In message >> , Adrian Chadd writes: >> >>> .. i'm pretty sure there was a reason for why it's done in byte sizes. >>> Maybe speak to phk? >> Buggy video hardware, which does not do larger writes correctly, >> the most recent one being an Intel Laptop, but I can't remember which. >> >> At the very least, byte writes needs to be a boot-time option. > Is there a way to detect this? Ie, writing to video memory and then > reading it back a byte at a time to see if the patterns match? I seem to recall this being a technique used back in my 386/msdos days. :) -Alfred From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 19:50:14 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17111587; Mon, 3 Mar 2014 19:50:14 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id CE02C382; Mon, 3 Mar 2014 19:50:13 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 26DDE3AE30; Mon, 3 Mar 2014 19:50:12 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.7/8.14.7) with ESMTP id s23JoBV4060476; Mon, 3 Mar 2014 19:50:11 GMT (envelope-from phk@phk.freebsd.dk) To: Justin Hibbits Subject: Re: newcons fb driver In-reply-to: From: "Poul-Henning Kamp" References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 03 Mar 2014 19:50:11 +0000 Message-ID: <60475.1393876211@critter.freebsd.dk> Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 19:50:14 -0000 In message , Justin Hibbits writes: >On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: >All great knowledge, but really only answers half of what I'm looking >for (always good to know potential pitfalls). Assuming a >tunable/sysctl is added, what's the best way to optimize from my >original post? Use a backing buffer (potentially with a tunable to >not)? Or assume we don't support background images, and write the >background color in the masked pixels? You should probably drop sos@ aka Søren Schmidt an email, he did syscons and has 10+ years experience in this stuff. One advantage of a backing buffer is that you can implement things like the 1000 line scroll buffer syscons did etc, I personally use that a lot to see dmesg output etc. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 3 22:33:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E49734B5; Mon, 3 Mar 2014 22:33:29 +0000 (UTC) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 7411FA8D; Mon, 3 Mar 2014 22:33:29 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id BA4C6D643BA; Tue, 4 Mar 2014 09:33:20 +1100 (EST) Date: Tue, 4 Mar 2014 09:33:11 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Poul-Henning Kamp Subject: Re: newcons fb driver In-Reply-To: <60475.1393876211@critter.freebsd.dk> Message-ID: <20140304071445.Y3158@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=HZAtEE08 c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=nlC_4_pT8q9DhB4Ho9EA:9 a=cz2ZRIgtxKwA:10 a=wJWlkF7cXJYA:10 a=pGLkceISAAAA:8 a=shYRFTWRAAAA:8 a=dfneXY-hSJqyxupU7d8A:9 a=45ClL6m2LaAA:10 a=Wpccw3uv0jEA:10 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Justin Hibbits , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Mar 2014 22:33:30 -0000 On Mon, 3 Mar 2014, Poul-Henning Kamp wrote: > In message > , Justin Hibbits writes: >> On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: > >> All great knowledge, but really only answers half of what I'm looking >> for (always good to know potential pitfalls). Assuming a >> tunable/sysctl is added, what's the best way to optimize from my >> original post? Use a backing buffer (potentially with a tunable to >> not)? Or assume we don't support background images, and write the >> background color in the masked pixels? > > You should probably drop sos@ aka S=F8ren Schmidt an email, he did > syscons and has 10+ years experience in this stuff. > > One advantage of a backing buffer is that you can implement things > like the 1000 line scroll buffer syscons did etc, I personally use > that a lot to see dmesg output etc. Is newcons so much worse than syscons that it doesn't even have a backing buffer? Backing buffers are a fundamental part of virtual consoles. Only one virtual console at a time uses the frame buffer, and the output to the others is virtual. Output to the one using the frame buffer is not much different to output for others when they are switched to active. The output should be delayed as long as possible to give a refresh rate faster than anyone's eyes can notice. 100-200 Hz is adequate, but I try to make it thousands of times faster than that so that 100% of 1 CPU isn't needed for screen output. syscons in an old version of FreeBSD now gives me i/o speeds of 26MB/S to an active console and 29MB/S to an inactive console. In text mode of course. That is quite slow even for 10+ year old hardware that it was re-tested on (old tests gave approximately the frame buffer speed and I mostly stopped running them when frame buffers got fast enough for text). Except with 20+ year old hardware, the frame buffer speed makes little difference in this test. Graphics mode is harder to make acceptably fast. Why would newcons need to start supporting bytewise i/o now? Hardware was rarely broken enough to need it even in FreeBSD-1, and syscons has always been sloppy about it. i386 (but not amd64) has a bogus function bcopyb() for doing such i/o. This was used mainly by the 4+ (3+ too many) console drivers in 386BSD, FreeBSD-1 and/or FreeBSD-2.0. syscons never used it. In FreeBSD-4, it was used by pcvt. pcvt used it only for the character set initialization where speed is unimportant. Now bcopyb() still exists, but is never used. Another reason that bcopyb() is bogus is that i/o should be done using bus space. dev/fb still has many hacks to avoid using bus-space. Only ia64 is clean there. Other arches use massive ifdef tangles to do things wrong. Old versions of x86 used mostly bcopy() on frame buffers, but now use a home made 16-bit copying function. arm and mips still use bcopy() but misspell it memcpy(). Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Mar 4 01:17:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCA3FD93; Tue, 4 Mar 2014 01:17:09 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 790E6B36; Tue, 4 Mar 2014 01:17:09 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id m5so4256646qaj.39 for ; Mon, 03 Mar 2014 17:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/BSvHUXI/1mzcKfA02fxoe/7E4W8rPdRl519zAHMYvI=; b=VILdh2p3V1Tnetbgmee2b/7R2ceKK5LeZBFUsAYfaaTFB8Vb0r4bokQgMsc1XenZ0L SfAR+gqP+hk19O6awjdsAG3R9oLN4kUEMaShq4xYMEtStnn5sMEk9GmzQbqmkO0L2PoU DUFF4q2R0yXZp+j4zCvrTyY7V+8+LkFdR8RLrS+z0v/gUJcOtl3MLlJD0qiwFbf/bXbl PqaFLFMoTZXRJKW7uL34nxJIOhJ7CLkL25SHALGvrJRX3OQ6II2nnPdoKqVsTBqLt5Rt EloINpoVTN+SwBCtyBOrTw1gEq0NqFURX9n6ed3JgfTs1PKTgjUJGrDKzBVJMBl6W8+P AWOw== MIME-Version: 1.0 X-Received: by 10.140.42.138 with SMTP id c10mr26749023qga.24.1393895828549; Mon, 03 Mar 2014 17:17:08 -0800 (PST) Received: by 10.224.16.10 with HTTP; Mon, 3 Mar 2014 17:17:08 -0800 (PST) Date: Mon, 3 Mar 2014 17:17:08 -0800 Message-ID: Subject: UDP transmit and no flowid From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 01:17:09 -0000 Hi, So something I was told about whilst investigating RSS at Netflix was that the UDP path doesn't actually get set by anything unless you're using flowtable. So, if one doesn't define flowtable, the transmit path congests quite heavily through one TX queue. I was about to do up a simple hack to do a toeplitz hash in the UDP send path before it gets bumped up to ip_output() - that way it can set the flowid correctly. The other option is to do a topelitz hash in ip_output() if m_flowid isn't set. The downside there is that we'd have to check whether it's actually a TCP, UDP, or IP flow before we assign it a flowid. In any case, this would likely decongest the transmit path quite a bit for UDP send. It's still not completely RSS-y (we would still need to line up transmit and receive paths to minimise lock contention) but it seems like a necessary first step in that direction. What do people think? I'll try this out in the next week or two once I've sorted out my employment situation and I can reserve some time on the netperf cluster. (before people ask "why udp?" - I'm kinda fed up with memcached performing much worse on freebsd than linux. Since fixing the UDP side of things requires a subset of the same work needed for TCP, I figure we should tackle UDP first.) Thanks, -adrian From owner-freebsd-arch@FreeBSD.ORG Tue Mar 4 01:25:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F31921B4; Tue, 4 Mar 2014 01:25:29 +0000 (UTC) Received: from mail-ve0-x231.google.com (mail-ve0-x231.google.com [IPv6:2607:f8b0:400c:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9BFFFC0A; Tue, 4 Mar 2014 01:25:29 +0000 (UTC) Received: by mail-ve0-f177.google.com with SMTP id sa20so4597635veb.36 for ; Mon, 03 Mar 2014 17:25:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=w1yJ+v2OasfnlKYxCV23Os7Uko9DjCD+Y3Rg828bKMQ=; b=ITTZeWi9zO7FuUv7ciqzLQNKfPfmMByWCMfsS/dBJiT29bl4SAoKkCZTnqq1p1hpcW sY82ocjiCV7Ts4bB6cBvwbmRoAWX/dxuTyPVzC6s+Ei7HjaOcFihfaLE9AA8fLDd4GRb zsgHNiRCMvJ8O5kQ1GWpHFTPJ1I921karVXCZcI1Iik8cyYxGZP4bhz/f4bsahWLQcci bNomixxtGa8p4b+JlgZszfQI5t1obPPgI41bjPMRqCv+ap5SQfUe900TT0/XtmT4LE22 4USXIZOY6uni5QV9tSwTj7HtNy2Ku6msDJE+yS583sPBj1OaIElEC9KEsyoTwVAlAzAL 4SOg== MIME-Version: 1.0 X-Received: by 10.220.136.6 with SMTP id p6mr18945741vct.9.1393896328763; Mon, 03 Mar 2014 17:25:28 -0800 (PST) Received: by 10.221.11.135 with HTTP; Mon, 3 Mar 2014 17:25:28 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Mar 2014 17:25:28 -0800 Message-ID: Subject: Re: UDP transmit and no flowid From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 01:25:30 -0000 Funny, Jeff Pieper and I were just talking about how it seems UDP performance, errr is less than optimal, so hey trying something that might help sounds like a good idea to me. Will be curious to hear the results! Cheers, Jack On Mon, Mar 3, 2014 at 5:17 PM, Adrian Chadd wrote: > Hi, > > So something I was told about whilst investigating RSS at Netflix was > that the UDP path doesn't actually get set by anything unless you're > using flowtable. So, if one doesn't define flowtable, the transmit > path congests quite heavily through one TX queue. > > I was about to do up a simple hack to do a toeplitz hash in the UDP > send path before it gets bumped up to ip_output() - that way it can > set the flowid correctly. > > The other option is to do a topelitz hash in ip_output() if m_flowid > isn't set. The downside there is that we'd have to check whether it's > actually a TCP, UDP, or IP flow before we assign it a flowid. > > In any case, this would likely decongest the transmit path quite a bit > for UDP send. It's still not completely RSS-y (we would still need to > line up transmit and receive paths to minimise lock contention) but it > seems like a necessary first step in that direction. > > What do people think? > > I'll try this out in the next week or two once I've sorted out my > employment situation and I can reserve some time on the netperf > cluster. > > (before people ask "why udp?" - I'm kinda fed up with memcached > performing much worse on freebsd than linux. Since fixing the UDP side > of things requires a subset of the same work needed for TCP, I figure > we should tackle UDP first.) > > Thanks, > > > -adrian > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Mar 4 11:16:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A53C7C9A; Tue, 4 Mar 2014 11:16:32 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 6608A355; Tue, 4 Mar 2014 11:16:32 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1WKnKp-000Nlf-0x; Tue, 04 Mar 2014 15:16:31 +0400 Date: Tue, 4 Mar 2014 15:16:31 +0400 From: Slawa Olhovchenkov To: Adrian Chadd Subject: Re: UDP transmit and no flowid Message-ID: <20140304111630.GA84822@zxy.spb.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 11:16:32 -0000 On Mon, Mar 03, 2014 at 05:17:08PM -0800, Adrian Chadd wrote: > So something I was told about whilst investigating RSS at Netflix was > that the UDP path doesn't actually get set by anything unless you're > using flowtable. So, if one doesn't define flowtable, the transmit > path congests quite heavily through one TX queue. Now you touch UDP? Can you add "back pressure" for UDP sockets? (currenly UDP socket don't signaled about output buffer full and just drop packets, regardless) From owner-freebsd-arch@FreeBSD.ORG Tue Mar 4 12:14:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 321B0475; Tue, 4 Mar 2014 12:14:15 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 00934B6F; Tue, 4 Mar 2014 12:14:13 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s24CE83e004033 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 4 Mar 2014 13:14:09 +0100 (CET) (envelope-from egrosbein@rdtc.ru) X-Envelope-From: egrosbein@rdtc.ru X-Envelope-To: freebsd-arch@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s24CE2jF039046; Tue, 4 Mar 2014 19:14:02 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <5315C38A.1010508@rdtc.ru> Date: Tue, 04 Mar 2014 19:14:02 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP transmit and no flowid References: In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.2 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Report: * 2.5 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 12:14:15 -0000 On 04.03.2014 08:17, Adrian Chadd wrote: > I'll try this out in the next week or two once I've sorted out my > employment situation and I can reserve some time on the netperf > cluster. There is a patch written by melifaro@freebsd.org to introduce IP flow id generation and modified by me to add sysctl net.inet.ip.skip_flowid_gen to disable id generation in run time and return pre-patched behavior: http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff I've tested it in production for mpd5-based high loaded BRAS, it works just fine. It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() and to fill m->m_pkthdr.flowid. Eugene Grosbein. From owner-freebsd-arch@FreeBSD.ORG Tue Mar 4 21:11:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6987EBC2; Tue, 4 Mar 2014 21:11:34 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 19C5AB77; Tue, 4 Mar 2014 21:11:34 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so128566qcy.28 for ; Tue, 04 Mar 2014 13:11:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qKQ4j+oxLmZOqFB3ih1qQC9E/xuWFYtMliMjPcAe5Xk=; b=CL670+jLHEWklOlz1EtT/mzMgZRor9IdQjT+8P3GHza7TtFHzT5LMsCNnx87deofrd zzMP0kLo6QHVKd30KR1+mXtm6xN8DiMB8kBvLV5A1T8OVBq07cCcrbNjyEw8CC0fkCus z+IrGGQbPIQ9W2F/V+CzLFeWp6IVK6l/2NRW/Gj+HvoAcSrf0eJhEQ5QAUmjhyyCdz+4 WMQbKwrAbo0rKuOeSwKeC5yiuvYvCQ1ksaUKKK1xO7bNqVoZLIXZhAjVuSI5rU4ZiKN7 l4FgzLT1PBGefAF3Pr133dIqakjiTalAV9Oa3ZFfQAYkFUWhiLgJuJDib1i12zE6LNpy 19Kw== MIME-Version: 1.0 X-Received: by 10.229.171.8 with SMTP id f8mr2433919qcz.13.1393967492543; Tue, 04 Mar 2014 13:11:32 -0800 (PST) Received: by 10.224.16.10 with HTTP; Tue, 4 Mar 2014 13:11:32 -0800 (PST) In-Reply-To: <5315C38A.1010508@rdtc.ru> References: <5315C38A.1010508@rdtc.ru> Date: Tue, 4 Mar 2014 13:11:32 -0800 Message-ID: Subject: Re: UDP transmit and no flowid From: Adrian Chadd To: Eugene Grosbein Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2014 21:11:34 -0000 Hi, On 4 March 2014 04:14, Eugene Grosbein wrote: > On 04.03.2014 08:17, Adrian Chadd wrote: > >> I'll try this out in the next week or two once I've sorted out my >> employment situation and I can reserve some time on the netperf >> cluster. > > There is a patch written by melifaro@freebsd.org to introduce > IP flow id generation and modified by me to add sysctl > net.inet.ip.skip_flowid_gen to disable id generation in run time > and return pre-patched behavior: > > http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff > > I've tested it in production for mpd5-based high loaded BRAS, it works just fine. > It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() > and to fill m->m_pkthdr.flowid. Hm, is this actually going to work for all paths through ip_output? Only a couple of paths go via netisr_queue(); the rest go via ifp->if_output. Is that going to loop back through the netisr outbound path? For some workloads we'll want to fill this in with the topelitz hash that matches the RX flow from the NIC, just to keep locking/processing of that flow on the same core. And to answer Slawa's email - yes, mainly because it's a subset of what's needed for doing this for TCP. In the TCP case, the hashing is already done for us on inbound connections; but there's still the whole missing bits from Robert's work to tie in the pcbgroup handling and the whole multi-queue / multi-listener thing that Linux and now DragonflyBSD does. But there's a handful of silly things that need to be first investigated and checked - like, ensuring that it works fine with fragmented IP frames and re-encapsulated things - just to ensure that they don't break the RSS model. Why'd you have to do an m_pullup() here in this path, which ideally should just be a lookup only path? Are you actually seeing the IP/TCP headers spread across multiple mbufs? They're not being snuck into mbuf headroom at all? Thanks, -a From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 01:00:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E0F5B60 for ; Wed, 5 Mar 2014 01:00:41 +0000 (UTC) Received: from mail-vc0-x229.google.com (mail-vc0-x229.google.com [IPv6:2607:f8b0:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3ACC1699 for ; Wed, 5 Mar 2014 01:00:41 +0000 (UTC) Received: by mail-vc0-f169.google.com with SMTP id hq11so337637vcb.0 for ; Tue, 04 Mar 2014 17:00:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=+MGyzf2PIaCrGnN4QgNeET1jB+byo+4HvjqNwhgcOlc=; b=t8nr65aP6dT4o9CgJrSmGMmmJUZEhSHDB7Np+yr9LQDIexEZkW1LsJ3aLuH/R4sJ8G S05Qx++/35q3xhDiw3AqXpmMHpu9Y0f799DgNsCMeiWx7S5i14ygVU//v/KsZOv5cIWw oAf7vch7Sy7GBpCbg6cE1p+rPfV+pEPVDUtYVksdkRmtd9EGMH2gwsIy2FHeN0CakZQm ILh8EUxkTb6sWBiDsEct5kjfGhYuKJw8nnj/RkVbKBwJVlNVnU4rA4tFE0nkeNlCxdjr je6oat15mpL1cHe8XLMKxMb6Ns2xv7MQgeSKSM8mhAqGUU4WnOjCPU82qtQNYmTzxImU ErHA== MIME-Version: 1.0 X-Received: by 10.52.190.1 with SMTP id gm1mr1768734vdc.21.1393981240223; Tue, 04 Mar 2014 17:00:40 -0800 (PST) Sender: chmeeedalf@gmail.com Received: by 10.58.133.193 with HTTP; Tue, 4 Mar 2014 17:00:40 -0800 (PST) In-Reply-To: <20140304071445.Y3158@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> Date: Tue, 4 Mar 2014 17:00:40 -0800 X-Google-Sender-Auth: GmH0iGmlfb8niWinFvch1tg6dxU Message-ID: Subject: Re: newcons fb driver From: Justin Hibbits To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 01:00:41 -0000 On Mon, Mar 3, 2014 at 2:33 PM, Bruce Evans wrote: > On Mon, 3 Mar 2014, Poul-Henning Kamp wrote: > >> In message >> >> , Justin Hibbits writes: >>> >>> On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: >> >> >>> All great knowledge, but really only answers half of what I'm looking >>> for (always good to know potential pitfalls). Assuming a >>> tunable/sysctl is added, what's the best way to optimize from my >>> original post? Use a backing buffer (potentially with a tunable to >>> not)? Or assume we don't support background images, and write the >>> background color in the masked pixels? >> >> >> You should probably drop sos@ aka S=C3=B8ren Schmidt an email, he did >> syscons and has 10+ years experience in this stuff. >> >> One advantage of a backing buffer is that you can implement things >> like the 1000 line scroll buffer syscons did etc, I personally use >> that a lot to see dmesg output etc. > > > Is newcons so much worse than syscons that it doesn't even have a > backing buffer? Backing buffers are a fundamental part of virtual > consoles. Only one virtual console at a time uses the frame buffer, > and the output to the others is virtual. Output to the one using the > frame buffer is not much different to output for others when they are > switched to active. The output should be delayed as long as possible > to give a refresh rate faster than anyone's eyes can notice. 100-200 > Hz is adequate, but I try to make it thousands of times faster than > that so that 100% of 1 CPU isn't needed for screen output. syscons > in an old version of FreeBSD now gives me i/o speeds of 26MB/S to an > active console and 29MB/S to an inactive console. In text mode of > course. That is quite slow even for 10+ year old hardware that it was > re-tested on (old tests gave approximately the frame buffer speed and > I mostly stopped running them when frame buffers got fast enough for > text). Except with 20+ year old hardware, the frame buffer speed makes > little difference in this test. Graphics mode is harder to make > acceptably fast. > > Why would newcons need to start supporting bytewise i/o now? Hardware > was rarely broken enough to need it even in FreeBSD-1, and syscons has > always been sloppy about it. i386 (but not amd64) has a bogus function > bcopyb() for doing such i/o. This was used mainly by the 4+ (3+ too > many) console drivers in 386BSD, FreeBSD-1 and/or FreeBSD-2.0. syscons > never used it. In FreeBSD-4, it was used by pcvt. pcvt used it only > for the character set initialization where speed is unimportant. Now > bcopyb() still exists, but is never used. > > Another reason that bcopyb() is bogus is that i/o should be done using > bus space. dev/fb still has many hacks to avoid using bus-space. Only > ia64 is clean there. Other arches use massive ifdef tangles to do > things wrong. Old versions of x86 used mostly bcopy() on frame buffers, > but now use a home made 16-bit copying function. arm and mips still > use bcopy() but misspell it memcpy(). > > Bruce Newcons does have a backing text buffer. I'm not even sure how we got on topic of a text buffer, when my question was regarding the frame buffer (sys/dev/vt/hw/fb/vt_fb.c), specifically vt_fb_bitbltchr(), and consists of only the following: * Does newcons support a background image, or is the mask there simply for drawing the text? * If it does support a background image, would it be good to double-buffer this, writing to the frame buffer only after the text is blitted? * If it doesn't support an image, would it be acceptable to prime the word to lay down with the background, and modify the pixels in the word accordingly? Without some kind of optimization, newcons on powerpc is unacceptably slow. - Justin From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 02:17:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E65DCA48; Wed, 5 Mar 2014 02:17:07 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8F574D89; Wed, 5 Mar 2014 02:17:07 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so461123qcr.8 for ; Tue, 04 Mar 2014 18:17:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=APvbzFweJRLwRCKxhyGsy92i+YpwWzV+lU2y12E6tgI=; b=M9m0UHvnVjPGyQ8qnZyvIi0N57SFORNzm4ajLdX0kjRhrPsvmJ41o71OOgjCnDzwjF Bc1mi+AEZ7DONyzepLg0WSDAhhAxlAslHVZA1jNLj1amcwsWD5952xfVl9U6KbtMuOJ8 nvSMCAWRSyOjvXqfIw7DtKDPQya4G6Lhlq7akyOcmzAD6W4EpeTJO5GVvKYr6UlSNPCR xYYTvGNoulgESTrAN+hIe4DWMzQrqBSI7bvJGVeBvb8qlDIvKvT6N4gNylEOANNPlLuA qLBCjHgXbGCSkgp3fQN4I5cfpjEbbJcpaPNk1uVjC4hleGBf6GNy/C4la/TZmqr0Dkml r9lA== MIME-Version: 1.0 X-Received: by 10.224.11.136 with SMTP id t8mr3832964qat.26.1393985826718; Tue, 04 Mar 2014 18:17:06 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.16.10 with HTTP; Tue, 4 Mar 2014 18:17:06 -0800 (PST) In-Reply-To: References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> Date: Tue, 4 Mar 2014 18:17:06 -0800 X-Google-Sender-Auth: WeNrl6MRpnT-zZLHXdnZFeHGN3w Message-ID: Subject: Re: newcons fb driver From: Adrian Chadd To: Justin Hibbits Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 02:17:08 -0000 I think the whole thread here is "this crap may not be needed for framebuffer modes." So, maybe for newcons-in-text-mode, you need the 8 bit writes but newcons-in-FB-mode you can get away with 32 bit writes. What would be nice would be to add some boot time checks to see if you can write 32 bit test patterns and then read them back in 8, 16, 32 bit chunks. See if they match. If they do, great. If they don't, then you need to downgrade. But the other thing is whether it's slow because you're doing scrolling/painting using copy operations on a very large/deep buffer, or whether you're doing anything hardware accelerated. Is newcons taking advantage of doing things like hardware-assisted scrolling, blitting/copying, copy-and-mask operations, updating only part of the screen if the backing text buffer changes, etc? -a On 4 March 2014 17:00, Justin Hibbits wrote: > On Mon, Mar 3, 2014 at 2:33 PM, Bruce Evans wrote: >> On Mon, 3 Mar 2014, Poul-Henning Kamp wrote: >> >>> In message >>> >>> , Justin Hibbits writes: >>>> >>>> On Mon, Mar 3, 2014 at 8:49 AM, Kurt Lidl wrote: >>> >>> >>>> All great knowledge, but really only answers half of what I'm looking >>>> for (always good to know potential pitfalls). Assuming a >>>> tunable/sysctl is added, what's the best way to optimize from my >>>> original post? Use a backing buffer (potentially with a tunable to >>>> not)? Or assume we don't support background images, and write the >>>> background color in the masked pixels? >>> >>> >>> You should probably drop sos@ aka S=F8ren Schmidt an email, he did >>> syscons and has 10+ years experience in this stuff. >>> >>> One advantage of a backing buffer is that you can implement things >>> like the 1000 line scroll buffer syscons did etc, I personally use >>> that a lot to see dmesg output etc. >> >> >> Is newcons so much worse than syscons that it doesn't even have a >> backing buffer? Backing buffers are a fundamental part of virtual >> consoles. Only one virtual console at a time uses the frame buffer, >> and the output to the others is virtual. Output to the one using the >> frame buffer is not much different to output for others when they are >> switched to active. The output should be delayed as long as possible >> to give a refresh rate faster than anyone's eyes can notice. 100-200 >> Hz is adequate, but I try to make it thousands of times faster than >> that so that 100% of 1 CPU isn't needed for screen output. syscons >> in an old version of FreeBSD now gives me i/o speeds of 26MB/S to an >> active console and 29MB/S to an inactive console. In text mode of >> course. That is quite slow even for 10+ year old hardware that it was >> re-tested on (old tests gave approximately the frame buffer speed and >> I mostly stopped running them when frame buffers got fast enough for >> text). Except with 20+ year old hardware, the frame buffer speed makes >> little difference in this test. Graphics mode is harder to make >> acceptably fast. >> >> Why would newcons need to start supporting bytewise i/o now? Hardware >> was rarely broken enough to need it even in FreeBSD-1, and syscons has >> always been sloppy about it. i386 (but not amd64) has a bogus function >> bcopyb() for doing such i/o. This was used mainly by the 4+ (3+ too >> many) console drivers in 386BSD, FreeBSD-1 and/or FreeBSD-2.0. syscons >> never used it. In FreeBSD-4, it was used by pcvt. pcvt used it only >> for the character set initialization where speed is unimportant. Now >> bcopyb() still exists, but is never used. >> >> Another reason that bcopyb() is bogus is that i/o should be done using >> bus space. dev/fb still has many hacks to avoid using bus-space. Only >> ia64 is clean there. Other arches use massive ifdef tangles to do >> things wrong. Old versions of x86 used mostly bcopy() on frame buffers, >> but now use a home made 16-bit copying function. arm and mips still >> use bcopy() but misspell it memcpy(). >> >> Bruce > > Newcons does have a backing text buffer. I'm not even sure how we got > on topic of a text buffer, when my question was regarding the frame > buffer (sys/dev/vt/hw/fb/vt_fb.c), specifically vt_fb_bitbltchr(), and > consists of only the following: > > * Does newcons support a background image, or is the mask there simply > for drawing the text? > * If it does support a background image, would it be good to > double-buffer this, writing to the frame buffer only after the text is > blitted? > * If it doesn't support an image, would it be acceptable to prime the > word to lay down with the background, and modify the pixels in the > word accordingly? > > Without some kind of optimization, newcons on powerpc is unacceptably slo= w. > > - Justin > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 02:32:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2158EDDF; Wed, 5 Mar 2014 02:32:01 +0000 (UTC) Received: from mail-qc0-x233.google.com (mail-qc0-x233.google.com [IPv6:2607:f8b0:400d:c01::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C905FEE0; Wed, 5 Mar 2014 02:32:00 +0000 (UTC) Received: by mail-qc0-f179.google.com with SMTP id m20so473805qcx.24 for ; Tue, 04 Mar 2014 18:32:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=0/yvPaEF6D+HGQRqnWFmJ1KAoRkSl1SaX+cqSH2c3r8=; b=CV50IwlIBttDQZ0Hcc3xnA3iWlk8JbxdO5BRfn9PPXWzMvqBlL12OsCwO8Cpl3bdUz OotEleIr6vx9V7YWBvi2XQgxxiPJAk8ouMyrYIhFDXwfqhg32AGHZB5FZwWelV9SrIqw 5bfMh2KVawMB/8wVWX8R7XHJJ4w6/YSR6Bv1YnSjGoBByBTf4zAEC1WnCpHm74jm/kCB sWQ7+4MTbc7lr9F1Nf6w2XWFxRHJzMQ3rafFmzzPvZP3kh8c5Gz4eOZVRgsIJHm9y6Ei eLHM9/HIsxfC/hJGmYTGPjEUyCTmnSIrDxZKD6Ex6bkemNESlYlK3fpd7XcM2B7U4UdZ 802g== MIME-Version: 1.0 X-Received: by 10.140.101.162 with SMTP id u31mr3342170qge.107.1393986719861; Tue, 04 Mar 2014 18:31:59 -0800 (PST) Received: by 10.224.16.10 with HTTP; Tue, 4 Mar 2014 18:31:59 -0800 (PST) Date: Tue, 4 Mar 2014 18:31:59 -0800 Message-ID: Subject: importing sam leffler's libstatfoo into -HEAd From: Adrian Chadd To: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 02:32:01 -0000 hi, i'd like to import the libstatfoo code from sam into -HEAD. It's used by some of the wifi tools to print out periodic and global statistics. It abstracts away the gathering, printing and formatting bits. I'm hoping that we can adapt some of the other base tools to use it and then teach it to print out statistics in other (machine parsable) formats, so we don't have to keep hacking at the tools to do this. The initial import: http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 04:15:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04F02CD4; Wed, 5 Mar 2014 04:15:15 +0000 (UTC) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80403B67; Wed, 5 Mar 2014 04:15:13 +0000 (UTC) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.7/8.14.7) with ESMTP id s254F3ZZ008167 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Mar 2014 05:15:03 +0100 (CET) (envelope-from egrosbein@rdtc.ru) X-Envelope-From: egrosbein@rdtc.ru X-Envelope-To: melifaro@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.7/8.14.7) with ESMTP id s254EwAj044959; Wed, 5 Mar 2014 11:14:58 +0700 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <5316A4C2.6040100@rdtc.ru> Date: Wed, 05 Mar 2014 11:14:58 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130415 Thunderbird/17.0.5 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP transmit and no flowid References: <5315C38A.1010508@rdtc.ru> In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.7 required=5.0 tests=BAYES_00, DATE_IN_FUTURE_96_Q, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-Spam-Report: * 3.0 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: date * 0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net Cc: FreeBSD Net , "Alexander V. Chernikov" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 04:15:15 -0000 On 05.03.2014 04:11, Adrian Chadd wrote: > Hi, > > On 4 March 2014 04:14, Eugene Grosbein wrote: >> On 04.03.2014 08:17, Adrian Chadd wrote: >> >>> I'll try this out in the next week or two once I've sorted out my >>> employment situation and I can reserve some time on the netperf >>> cluster. >> >> There is a patch written by melifaro@freebsd.org to introduce >> IP flow id generation and modified by me to add sysctl >> net.inet.ip.skip_flowid_gen to disable id generation in run time >> and return pre-patched behavior: >> >> http://www.grosbein.net/freebsd/patches/netisr_ip_flowid.diff >> >> I've tested it in production for mpd5-based high loaded BRAS, it works just fine. >> It uses IP src/dst addresses and TCP/UDP/SCTP ports to feed jenkins_hashword() >> and to fill m->m_pkthdr.flowid. > > Hm, is this actually going to work for all paths through ip_output? > Only a couple of paths go via netisr_queue(); the rest go via > ifp->if_output. Is that going to loop back through the netisr outbound > path? > > For some workloads we'll want to fill this in with the topelitz hash > that matches the RX flow from the NIC, just to keep locking/processing > of that flow on the same core. > > And to answer Slawa's email - yes, mainly because it's a subset of > what's needed for doing this for TCP. In the TCP case, the hashing is > already done for us on inbound connections; but there's still the > whole missing bits from Robert's work to tie in the pcbgroup handling > and the whole multi-queue / multi-listener thing that Linux and now > DragonflyBSD does. > > But there's a handful of silly things that need to be first > investigated and checked - like, ensuring that it works fine with > fragmented IP frames and re-encapsulated things - just to ensure that > they don't break the RSS model. > > Why'd you have to do an m_pullup() here in this path, which ideally > should just be a lookup only path? Are you actually seeing the IP/TCP > headers spread across multiple mbufs? They're not being snuck into > mbuf headroom at all? I cannot answer these questions, CC'ing author of the patch, Alexander Chernikov. Eugene Grosbein From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 04:19:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59B09185; Wed, 5 Mar 2014 04:19:22 +0000 (UTC) Received: from felyko.com (felyko.com [174.136.100.2]) by mx1.freebsd.org (Postfix) with ESMTP id 44474B94; Wed, 5 Mar 2014 04:19:21 +0000 (UTC) Received: from [10.0.1.3] (c-24-6-115-18.hsd1.ca.comcast.net [24.6.115.18]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 8604B39828; Tue, 4 Mar 2014 20:19:20 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Rui Paulo In-Reply-To: Date: Tue, 4 Mar 2014 20:19:23 -0800 Content-Transfer-Encoding: 7bit Message-Id: <9BA45ADB-3005-4689-A9D3-932CB9F2F84E@FreeBSD.org> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 04:19:22 -0000 On 4 Mar 2014, at 18:31, Adrian Chadd wrote: > hi, > > i'd like to import the libstatfoo code from sam into -HEAD. > > It's used by some of the wifi tools to print out periodic and global > statistics. It abstracts away the gathering, printing and formatting > bits. > > I'm hoping that we can adapt some of the other base tools to use it > and then teach it to print out statistics in other (machine parsable) > formats, so we don't have to keep hacking at the tools to do this. > > The initial import: > > http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff While there, could you please rename it? libstatfoo is a pretty bad name... -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 04:19:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 865D2186; Wed, 5 Mar 2014 04:19:23 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2607:f2f8:a528::3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id 6E98FB95; Wed, 5 Mar 2014 04:19:23 +0000 (UTC) Received: from [10.0.1.3] (c-24-6-115-18.hsd1.ca.comcast.net [24.6.115.18]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 26F453988D; Tue, 4 Mar 2014 20:19:22 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Rui Paulo In-Reply-To: Date: Tue, 4 Mar 2014 20:19:23 -0800 Content-Transfer-Encoding: 7bit Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 04:19:23 -0000 On 4 Mar 2014, at 18:31, Adrian Chadd wrote: > hi, > > i'd like to import the libstatfoo code from sam into -HEAD. > > It's used by some of the wifi tools to print out periodic and global > statistics. It abstracts away the gathering, printing and formatting > bits. > > I'm hoping that we can adapt some of the other base tools to use it > and then teach it to print out statistics in other (machine parsable) > formats, so we don't have to keep hacking at the tools to do this. > > The initial import: > > http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff While there, could you please rename it? libstatfoo is a pretty bad name... -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 05:17:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54BDD37F; Wed, 5 Mar 2014 05:17:01 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E14E2128; Wed, 5 Mar 2014 05:17:00 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id e89so1646868qgf.5 for ; Tue, 04 Mar 2014 21:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=5VXT6BkZzuFLdOBKO84HWkFP79mpo+ehEJk8HQmZ+fk=; b=nV2YbM9d14N4HBkp8dr+9h8sPw1YdE2mOri0UN5+uhzfITZr8T6e+8pbbb/GP1g0Hl XsjmfBSCm5TbSNY13eoB5SG/GB5k3Yt67vduIxgbXwkshxph18ZdEPHEogH6GRnKtz9p YlQMmnjJQDz55bDp6V/s+hMdoBIFggPraR139vR37eepsujs1X+JGqClrDmqWlLRCZJf H7c0FASDjyPLE+dfwl98+oCZTWf9ZmHF373YYqU4I/BsESfSOQEfHgROBm4EQc3eX+bq xu4knwQipFS6UeCIcoL4aocLzBDcc/0vUGIlFlrNYAMMYOipPllJ7eyvAyjNltRTUe3l 42HQ== MIME-Version: 1.0 X-Received: by 10.140.101.162 with SMTP id u31mr35115qge.107.1393996619962; Tue, 04 Mar 2014 21:16:59 -0800 (PST) Received: by 10.224.16.10 with HTTP; Tue, 4 Mar 2014 21:16:59 -0800 (PST) In-Reply-To: <9BA45ADB-3005-4689-A9D3-932CB9F2F84E@FreeBSD.org> References: <9BA45ADB-3005-4689-A9D3-932CB9F2F84E@FreeBSD.org> Date: Tue, 4 Mar 2014 21:16:59 -0800 Message-ID: Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Adrian Chadd To: Rui Paulo Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 05:17:01 -0000 On 4 March 2014 20:19, Rui Paulo wrote: > On 4 Mar 2014, at 18:31, Adrian Chadd wrote: > >> hi, >> >> i'd like to import the libstatfoo code from sam into -HEAD. >> >> It's used by some of the wifi tools to print out periodic and global >> statistics. It abstracts away the gathering, printing and formatting >> bits. >> >> I'm hoping that we can adapt some of the other base tools to use it >> and then teach it to print out statistics in other (machine parsable) >> formats, so we don't have to keep hacking at the tools to do this. >> >> The initial import: >> >> http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff > > While there, could you please rename it? libstatfoo is a pretty bad name... I'm open to suggestions, but I don't want to get bogged down by sheds. -a From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 14:07:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 048A59F1; Wed, 5 Mar 2014 14:07:49 +0000 (UTC) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id A1577CBB; Wed, 5 Mar 2014 14:07:48 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 03F2F4212EF; Thu, 6 Mar 2014 01:07:39 +1100 (EST) Date: Thu, 6 Mar 2014 01:07:38 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Justin Hibbits Subject: Re: newcons fb driver In-Reply-To: Message-ID: <20140305230458.L1053@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=ddC5gxne c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=wcSzQ5qOX8Chkg5hSN0A:9 a=CjuIK1q_8ugA:10 Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 14:07:49 -0000 On Tue, 4 Mar 2014, Justin Hibbits wrote: > On Mon, Mar 3, 2014 at 2:33 PM, Bruce Evans wrote: >> Is newcons so much worse than syscons that it doesn't even have a >> backing buffer? Backing buffers are a fundamental part of virtual >> consoles. ... >> >> Why would newcons need to start supporting bytewise i/o now? Hardware >> was rarely broken enough to need it even in FreeBSD-1, and syscons has >> always been sloppy about it. ... > > Newcons does have a backing text buffer. I'm not even sure how we got > on topic of a text buffer, when my question was regarding the frame > buffer (sys/dev/vt/hw/fb/vt_fb.c), specifically vt_fb_bitbltchr(), and > consists of only the following: Since it takes very large hardware pessimizations to give slowness with a correctly implemented backing text buffer. Newcons doesn't seem to have one of those, even for vga (sys/dev/vt/hw/vga/vga.c) where it has a character output routine. vga supports text mode using slow code, but I think it is only 2 times slower than syscons in most cases. Bitmapped mode with 16x8 characters and 256 colors takes 128 bytes/character (most wasted for coloring individual pixels in characters), so it is inherently 64 times slower than text mode. vt_fb_bitbltchr() supports this. It doesn't seem to be especially pessimal, except for its per-char interface. It seems to use bytewise accesses a bit too much for simplicity, although it sometimes does 32-bit. This should only give a further slowness factor of 2, 4 or 8 unless the frame buffer hardware is stupid. vt_fb_setpixel() is an example of a really slow interface. I thought at first that vt_fb_bitbltchr() was much faster, but now see that it is not much more than a wrapper around a manually inlined vt_fb_setpixel(). The problems are easier to see in it. Setting pixels one at a time is too slow. For 16x8 characters, that is at least 128 memory accesses per character, and the interface prevents merging these accesses. vt_fb_bitbltchr() could do some merging, but doesn't seem to do any. Anyway, the support seems to be limited to modes with 2**8, 2**16 or 2**32 colors, so that there are 1, 2 or 4 bytes per pixel. For modes with < 8 bits per pixel, vt_fb_setpixel() is even worse. Maybe I misremember how colors are packed, but this is very hardware dependent and I think vt only supports 1 simple type of packing. A correctly implemented backing text buffer consists of characters (and possibly attributes) stored in fast memory in a form for copying to a frame buffer, with the frame buffer in text mode. A not so correctly implemented backing text buffer is the same, except with the frame buffer in pixel mode. This is much more needed if the frame buffer doesn't support text mode. > * Does newcons support a background image, or is the mask there simply > for drawing the text? > * If it does support a background image, would it be good to > double-buffer this, writing to the frame buffer only after the text is > blitted? > * If it doesn't support an image, would it be acceptable to prime the > word to lay down with the background, and modify the pixels in the > word accordingly? I don't know what the mask does. How would you lay down the background much faster than with blitting? > Without some kind of optimization, newcons on powerpc is unacceptably slow. How slow is that exactly? If the frame buffer speed is 50MB/S, then 16x8 characters with 256 colors can be written at 390K/S. This is acceptable. If the frame buffer is much slower than that, then it is too slow, at least without hardware scrolling. Sample frame buffer speeds: my first PC (1982): 1MB/sec (slower through the CPU) ISA ET4000: 2.4MB/sec read, 5.9MB/sec write VLB ET4000/W32i: 6.8MB/sec read, 25.5MB/sec write PCI S3/868: 3.5MB/sec read, 23.1MB/sec write PCI S3/Virge: 4.1MB/sec read, 40.0MB/sec write PCI S3/Savage: 3.3MB/sec read, 25.8MB/sec write PCI Xpert: 5.3MB/sec read, 21.8MB/sec write PCI R9200SE: 5.8MB/sec read, 60.2MB/sec write (but 120MB/sec through FPU) I stopped measuring these speeds often about 20 years ago when they became fast enough for simple i/o in text mode. The last 2 are newer. Read speeds are still much lower for some reason (just the usual PCI slowness?). This would amplify any slowness from the hardware doing read-modify-write accesses for to convert to aligned 32/64/128/...-bit accesses. I think bit blitting can't do any better than the hardware by doing the read cycles itself. Everything needs to be buffered in fast memory just to avoid reading slow memory. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 15:02:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D16E5B4C; Wed, 5 Mar 2014 15:02:55 +0000 (UTC) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 7F90333B; Wed, 5 Mar 2014 15:02:55 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 3B453D613E6; Thu, 6 Mar 2014 02:02:47 +1100 (EST) Date: Thu, 6 Mar 2014 02:02:46 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Bruce Evans Subject: Re: newcons fb driver In-Reply-To: <20140305230458.L1053@besplex.bde.org> Message-ID: <20140306013120.H1530@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> <20140305230458.L1053@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=fbeUlSgF c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=S3Ji9MKQCi1dntuHwz4A:9 a=CjuIK1q_8ugA:10 Cc: Justin Hibbits , Poul-Henning Kamp , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 15:02:55 -0000 On Thu, 6 Mar 2014, Bruce Evans wrote: > On Tue, 4 Mar 2014, Justin Hibbits wrote: >> Without some kind of optimization, newcons on powerpc is unacceptably slow. > > How slow is that exactly? If the frame buffer speed is 50MB/S, then > 16x8 characters with 256 colors can be written at 390K/S. This is > acceptable. If the frame buffer is much slower than that, then it is > too slow, at least without hardware scrolling. I forgot to mention that u$ doesn't solve this problem in WinXP. The speed is not too bad in normal mode (only about 30 times slower than syscons - the scrolling speed is 30000 lines/second instead of 1000000). But in safe mode, a simple graphics driver is used and it is horribly slow. Safe mode with command prompt gives an MSDOS prompt in a maximized terminal window, so the problem is very noticeable (the scrolling speed is 4 lines per second = 12 seconds per screenful). Apparently normal mode uses acceleration to be not very slow, but safe mode is too simple to do that. The FreeBSD console driver needs to be even simpler so that it works as a low level console. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 16:34:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B98B1B62; Wed, 5 Mar 2014 16:34:22 +0000 (UTC) Received: from mail-ie0-x229.google.com (mail-ie0-x229.google.com [IPv6:2607:f8b0:4001:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 73D0CE99; Wed, 5 Mar 2014 16:34:22 +0000 (UTC) Received: by mail-ie0-f169.google.com with SMTP id to1so1336114ieb.0 for ; Wed, 05 Mar 2014 08:34:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=igr/KjTMBLkRPqMiMj9GlXDb4DFb98aOygsQTQJm4gs=; b=WDsEF7+kT7ZHYbu3ASlxvXNwef7bkSIc90lVTiIe1uargmii/HO0OlZQy9tlYTqGcd yYA9debdPa3Wjeg8SLw+KqaANBnIF+zZNIpUXHa4ji2dXOQ0NQXZw/S3wWgvF0I0poND ooZoqtLo2/d7w/BEzJfdP4qK8Q0BEiPrCt3TnnB6FX/MNmiYaisae9T9hBm6et7PVnAu 5TwgQRNAEOXI7tOUFfPOu045NaRSi53+1X/XOB+rMNpSrKmYe0d2pvHqFIEfT47jJzxr co7yfOMl+Krh6IwXAnZOkh+b32nzEwb4u3sWfiYSd9CIT8vRgOvPwpuqHoltv1JqFbU8 9ARg== MIME-Version: 1.0 X-Received: by 10.42.16.199 with SMTP id q7mr5578372ica.16.1394037261988; Wed, 05 Mar 2014 08:34:21 -0800 (PST) Received: by 10.50.164.227 with HTTP; Wed, 5 Mar 2014 08:34:21 -0800 (PST) In-Reply-To: References: <9BA45ADB-3005-4689-A9D3-932CB9F2F84E@FreeBSD.org> Date: Wed, 5 Mar 2014 10:34:21 -0600 Message-ID: Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Scot Hetzel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 16:34:22 -0000 On Tue, Mar 4, 2014 at 11:16 PM, Adrian Chadd wrote: > On 4 March 2014 20:19, Rui Paulo wrote: >> On 4 Mar 2014, at 18:31, Adrian Chadd wrote: >> >>> hi, >>> >>> i'd like to import the libstatfoo code from sam into -HEAD. >>> >>> It's used by some of the wifi tools to print out periodic and global >>> statistics. It abstracts away the gathering, printing and formatting >>> bits. >>> >>> I'm hoping that we can adapt some of the other base tools to use it >>> and then teach it to print out statistics in other (machine parsable) >>> formats, so we don't have to keep hacking at the tools to do this. >>> >>> The initial import: >>> >>> http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff >> >> While there, could you please rename it? libstatfoo is a pretty bad name... > > I'm open to suggestions, but I don't want to get bogged down by sheds. > lib/libbsdstatfoo -> lib/libbsdstat[,istics] lib/libbsdstatfoo/statfoo.c -> lib/libbsdstat[,istics]/bsdstat[,istics].c lib/libbsdstatfoo/statfoo.h -> lib/libbsdstat[,istics]/bsdstat[,istics].h Then change all occurrences of statfoo/STATFOO in those files to bsdstat[,istics]/BSDSTAT[,ISTICS]. -- DISCLAIMER: No electrons were maimed while sending this message. Only slightly bruised. From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 16:42:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B74F9DF4; Wed, 5 Mar 2014 16:42:37 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4FBD2F6A; Wed, 5 Mar 2014 16:42:37 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so1218673qaj.7 for ; Wed, 05 Mar 2014 08:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=CaziTuBFKH4HB27ic5HQTwCah5C40u5HgkxzbF2OQ0M=; b=BwhQDZ73vzeca+4PJ10ISYbI3D1NvJ/8L9l6ZLty2vuKyxcR10v99Uh3FRLDmXHveN DJCGkBVGx+MsRpILDrUDXpzCbR70Bws9tCHfIdk/8Vgb0ekyFbsDU7eATkjjrU0lmix+ VcL7bSS60HcRGmjNa6Y6cbkie620naPowEuZejNuvTJNhVb54dvixI6BgZZucfqYKrs9 o3C+ZFIE7kDGFa6sSXbsvsVh/22tcaO9KDuqrNuNzYGD25q7hPOB7Bq+JWDW6Ht/5Nzb xP1ZOS52iDz12piUcFzygRkVhcu7WLfG8vvhBvLeC6g7Tj44a8aKGrJhD5OJcrwhUdoz UrBw== MIME-Version: 1.0 X-Received: by 10.140.42.138 with SMTP id c10mr7726451qga.24.1394037756533; Wed, 05 Mar 2014 08:42:36 -0800 (PST) Received: by 10.224.8.137 with HTTP; Wed, 5 Mar 2014 08:42:36 -0800 (PST) In-Reply-To: References: <9BA45ADB-3005-4689-A9D3-932CB9F2F84E@FreeBSD.org> Date: Wed, 5 Mar 2014 08:42:36 -0800 Message-ID: Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Adrian Chadd To: Scot Hetzel Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 16:42:37 -0000 Yup, I've renamed it libbsdstat. http://people.freebsd.org/~adrian/patches/20140304-libbsdstat-2.diff I'm doing a universe run right now. -a On 5 March 2014 08:34, Scot Hetzel wrote: > On Tue, Mar 4, 2014 at 11:16 PM, Adrian Chadd wrote: >> On 4 March 2014 20:19, Rui Paulo wrote: >>> On 4 Mar 2014, at 18:31, Adrian Chadd wrote: >>> >>>> hi, >>>> >>>> i'd like to import the libstatfoo code from sam into -HEAD. >>>> >>>> It's used by some of the wifi tools to print out periodic and global >>>> statistics. It abstracts away the gathering, printing and formatting >>>> bits. >>>> >>>> I'm hoping that we can adapt some of the other base tools to use it >>>> and then teach it to print out statistics in other (machine parsable) >>>> formats, so we don't have to keep hacking at the tools to do this. >>>> >>>> The initial import: >>>> >>>> http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff >>> >>> While there, could you please rename it? libstatfoo is a pretty bad name... >> >> I'm open to suggestions, but I don't want to get bogged down by sheds. >> > > lib/libbsdstatfoo -> lib/libbsdstat[,istics] > lib/libbsdstatfoo/statfoo.c -> lib/libbsdstat[,istics]/bsdstat[,istics].c > lib/libbsdstatfoo/statfoo.h -> lib/libbsdstat[,istics]/bsdstat[,istics].h > > Then change all occurrences of statfoo/STATFOO in those files to > bsdstat[,istics]/BSDSTAT[,ISTICS]. > > > -- > DISCLAIMER: > > No electrons were maimed while sending this message. Only slightly bruised. From owner-freebsd-arch@FreeBSD.ORG Wed Mar 5 23:43:10 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1B79F57 for ; Wed, 5 Mar 2014 23:43:10 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5D11CE32 for ; Wed, 5 Mar 2014 23:43:10 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so4961000qga.4 for ; Wed, 05 Mar 2014 15:43:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=jPUy+xS9wlTnZdxvCFGYxLmLLjJm16E8eviO96aKpXI=; b=TJakij1b0P9IlaCWhGpvEUG9TgOCf7YtPIUJuYpC0JDxr9iXXBdVDVeqUKDJEfAtlE nMQudAGo/UZHYAGsPgXhZnohZ6jH/AExnCp1DsNVl8rILWhVnivMNu8VMCZDcmwCvDPw r7s5PEC9qWbMQisQnfNHU16pRB13nLI7QjznQupEqWAlV6Hb91+JGQ8CRu7A+w3ethbj xTK94y54/Rp9WPAo+tbgwKPOziuddKyXCFENiVGmITtJu56ON/DlcBaVjSClfBTCQz/E 57kdxHs2K9cVrt2YKIi2qOTh+3p1p2C9bo6b3L2p6Bx5CmCr79buSeBFcg+M3hCbZ1Yr u0Bw== MIME-Version: 1.0 X-Received: by 10.140.81.244 with SMTP id f107mr5550714qgd.104.1394062989537; Wed, 05 Mar 2014 15:43:09 -0800 (PST) Sender: chmeeedalf@gmail.com Received: by 10.140.28.131 with HTTP; Wed, 5 Mar 2014 15:43:09 -0800 (PST) In-Reply-To: <20140306013120.H1530@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> <20140305230458.L1053@besplex.bde.org> <20140306013120.H1530@besplex.bde.org> Date: Wed, 5 Mar 2014 15:43:09 -0800 X-Google-Sender-Auth: lpBGcF5KnyVo22Dmn5BoDDDvb8M Message-ID: Subject: Re: newcons fb driver From: Justin Hibbits To: Bruce Evans Content-Type: text/plain; charset=UTF-8 Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Mar 2014 23:43:10 -0000 On Wed, Mar 5, 2014 at 7:02 AM, Bruce Evans wrote: > On Thu, 6 Mar 2014, Bruce Evans wrote: > >> On Tue, 4 Mar 2014, Justin Hibbits wrote: >>> >>> Without some kind of optimization, newcons on powerpc is unacceptably >>> slow. >> >> >> How slow is that exactly? If the frame buffer speed is 50MB/S, then >> 16x8 characters with 256 colors can be written at 390K/S. This is >> acceptable. If the frame buffer is much slower than that, then it is >> too slow, at least without hardware scrolling. > > > I forgot to mention that u$ doesn't solve this problem in WinXP. The > speed is not too bad in normal mode (only about 30 times slower than > syscons - the scrolling speed is 30000 lines/second instead of 1000000). > But in safe mode, a simple graphics driver is used and it is horribly > slow. Safe mode with command prompt gives an MSDOS prompt in a > maximized terminal window, so the problem is very noticeable (the > scrolling speed is 4 lines per second = 12 seconds per screenful). > Apparently normal mode uses acceleration to be not very slow, but safe > mode is too simple to do that. The FreeBSD console driver needs to > be even simpler so that it works as a low level console. > > Bruce The ofwfb syscons driver is much faster than newcons, so a frame buffer can be made relatively fast (within reason). It's newcons that's slow, and my assumption currently is the byte-oriented writes instead of word size. - Justin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 6 00:31:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C21BBEB for ; Thu, 6 Mar 2014 00:31:13 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A213C277 for ; Thu, 6 Mar 2014 00:31:13 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WLMDQ-00011R-P6 for freebsd-arch@freebsd.org; Wed, 05 Mar 2014 16:31:12 -0800 Date: Wed, 5 Mar 2014 16:31:12 -0800 (PST) From: "Paul \"LeoNerd\" Evans" To: freebsd-arch@freebsd.org Message-ID: <1394065872771-5892010.post@n5.nabble.com> In-Reply-To: References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> Subject: Re: terminfo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 00:31:13 -0000 Ed Schouten-3 wrote > - Who cares about VT52 support? Those things are from the Viking Age. > - Who cares about double-width characters? I don't know a single > program that uses this. In the worst case, you can just use fullwidth > CJK for certain characters. >=20 > - Who cares about send/receive mode? >=20 > - Who cares about 88 colour support? Just use 256 colours. >=20 > - Who cares about ACS? Unicode already has those characters. I'm with you on all those points. Ed Schouten-3 wrote > There are so many legacy features out there, that nobody actually > *knows* how terminals should behave anymore. Also combined with the > fact that the de facto reference implementation of terminal emulation > (i.e., xterm) cannot easily be decoupled from X11, people just make > stuff up. >=20 > People are nowadays only interested in having a 16 or 256 colour, > UTF-8 enabled terminal. >=20 > It would be so incredibly easy for you as the maintainer of terminfo > to say: "Here's an ideal terminfo entry. It's compact, easy to > implement, etc. This is how I want terminals to behave. And this menu > in vttest can be used to easily test conformance." Then you just file > bugs against the authors of commonly used terminal emulators and 3-5 > years later, the ecosystem has converged. Done. >=20 > As I mentioned before, my spare time is limited nowadays, but if you > would be interested in doing something like that, I would be more than > interested to get this sorted out on FreeBSD's side. I believe that may be where I come in. Two of my larger current projects are a terminal emulator, and a terminal emulation library; two parts of this particular issue. libvterm is a totally-abstracted library in pure C99 that aims to entirely emulate a VTxxx/xterm and similar terminals. https://launchpad.net/libvterm/ This isn't entirely finished yet but already it understands the vast majority of sequences actually found in the wild. The list of sequences it currently understands (as compared VT1xx, 2xx, etc..) can be found at =20 http://bazaar.launchpad.net/~leonerd/libvterm/trunk/view/head:/doc/seqs.txt (a document not entirely unlike xterm's ctlseqs ;) ) Of particular interest to FreeBSD may be the fact that libvterm abstracts a= s much as possible out to the embedding program; all byte IO, keyboard/mouse interaction, and drawing operations. It even optionally abstracts out mallo= c itself, and makes special care not to call the allocator function on the "critical" path of normal drawing; this is only used for initialisation and resize. This feature I intended specially for use in things like UNIX kernels, as sometimes they have critical memory requirements. ((This library also drives, among others, pangoterm, a GTK-based terminal I wrote around it as proof-of-concept. This terminal has been my one terminal I've actually used for both personal and work use, for the past two years: https://launchpad.net/pangoterm But to be clear - all the clever "understanding terminal stuff" happens in the abstracted libvterm library; the actual GTK program is a tiny wrapper that basically just provides "put these letters here in these colours/attributes" functions.)) I've also been working on the "terminal UI applications" end of the chain, by writing a set of libraries that programs can use for driving a terminal, which more completely understand these modern features of terminals, such a= s provided by libvterm. http://www.leonerd.org.uk/code/libtickit/ This aims to provide support at a number of increasingly-abstracted levels, though right now only the lowermost (the raw "terminal" level) is available in the C library. The remaining (the "window" and "widget" levels) are currently implemented in the Perl module of the same name (Tickit on CPAN), where I'm developing it, slowly migrating code as I go into the C library. Inbetween the two I have been trying my best to both follow and implement "current" terminal technology, as exemplified by xterm and compatible, and also to extend it where it makes sense - for example, my definition of CSI = u to allow terminals to encode arbitrary modified Unicode keypresses, and thu= s to provide a solution to cases like Ctrl-Shift-A and Ctrl-I as distinct fro= m Tab. http://www.leonerd.org.uk/hacks/fixterms/ *deep breath* Sorry if the-above comes across as somewhat of a r=E1=BA=BFsum=C3=A9-style = advert or shameless self-promotion. I was pointed at this thread because of a terminal-based discussion in Freenode's #vim, where I'm usually the person to field any kind of terminal-related question, often with reference to one or more of my projects above. I got to this point in the message thread and felt I should reply and speak up, if only to say "Hey, so yes I am trying t= o be this person who cares about modern terminal technology and wishes to see it continue to expand and grow into the 21st century". So if I can be of any assistance with these issues, I'd be happy to take an= y questions or comments. --=20 Paul "LeoNerd" Evans leonerd@leonerd.org.uk http://www.leonerd.org.uk/ -- View this message in context: http://freebsd.1045724.n5.nabble.com/terminfo= -tp5887280p5892010.html Sent from the freebsd-arch mailing list archive at Nabble.com. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 6 01:24:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C611FCD6 for ; Thu, 6 Mar 2014 01:24:11 +0000 (UTC) Received: from sam.nabble.com (sam.nabble.com [216.139.236.26]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A61BA8BE for ; Thu, 6 Mar 2014 01:24:11 +0000 (UTC) Received: from [192.168.236.26] (helo=sam.nabble.com) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1WLN2g-0007bn-Hn for freebsd-arch@freebsd.org; Wed, 05 Mar 2014 17:24:10 -0800 Date: Wed, 5 Mar 2014 17:24:10 -0800 (PST) From: "Paul \"LeoNerd\" Evans" To: freebsd-arch@freebsd.org Message-ID: <1394069050543-5892023.post@n5.nabble.com> In-Reply-To: <20140227230528.GB8295@stack.nl> References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> <20140226085939.GC2705@server.rulingia.com> <20140227230528.GB8295@stack.nl> Subject: Re: terminfo MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 01:24:11 -0000 Jilles Tjoelker wrote > I think it is worth it to simplify terminals. Instead of having people > choose how they want their terminal to work improperly, provide one way > and concentrate on making that work as well as possible (no visual > artifacts, 256 colours, Unicode, key combinations like Ctrl+Home, etc.) I agree here too. I'm sure overall, application writers waste more time trying to accommodate every possible variation in terminals, than terminal authors would just getting to a place where they all agree. I for one have wasted far too long getting my libraries to properly understand that bce is default-off in screen, and how to work around it. Lets not forget; the original reason we're in this state has to do with DEC vs Wyse vs IBM vs everyone else, trying to build glass teletypes. We're past that age now. Jilles Tjoelker wrote > I also like the idea of some applications sending control sequences > directly, without using curses or even termcap/terminfo. libtickit does exactly this. If $TERM implies a recent enough terminal (i.e. TERM=xterm or variations) then it switches to an active probing mode where it just asks the terminal whether it supports 256 colours, and so on. It then uses an entirely built-in set of sequences, not looked up from termcap/terminfo. In fact, more or less the only reason libtickit links against unibilium or tinfo of ncurses is so that it can look up the afore-mentioned bce flag, in order to work out if it should print spaces or use ECH. I've always felt this a far better way to write terminal applications. Which of these is the best way to know if a terminal supports a given feature: a) use a possibly-wrong local environment variable as a key into a disk database on the local machine (where the terminal may not be running; e.g. consider ssh) to find a fixed set of capabilities that were current about the terminal when those files were written, or b) ask it I vote "b" :) For anyone who still thinks "a" I will leave you to consider how your termcap file knows whether I compiled my terminal using ./configure --with-256colours --with-italics --without-underline -- Paul "LeoNerd" Evans leonerd@leonerd.org.uk http://www.leonerd.org.uk/ -- View this message in context: http://freebsd.1045724.n5.nabble.com/terminfo-tp5887280p5892023.html Sent from the freebsd-arch mailing list archive at Nabble.com. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 6 04:21:19 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92CD2514; Thu, 6 Mar 2014 04:21:19 +0000 (UTC) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 2289990E; Thu, 6 Mar 2014 04:21:18 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id F28551A28FA; Thu, 6 Mar 2014 15:21:10 +1100 (EST) Date: Thu, 6 Mar 2014 15:21:08 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Justin Hibbits Subject: Re: newcons fb driver In-Reply-To: Message-ID: <20140306133834.P2064@besplex.bde.org> References: <42130.1393829535@critter.freebsd.dk> <5314B2A2.3060100@pix.net> <60475.1393876211@critter.freebsd.dk> <20140304071445.Y3158@besplex.bde.org> <20140305230458.L1053@besplex.bde.org> <20140306013120.H1530@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=PqKqMW83 c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=KxXyRG7pkqQyeGXE110A:9 a=Q858Yj1g5SAHSpJR:21 a=NmfiDTQT_qFu96JG:21 a=CjuIK1q_8ugA:10 Cc: Poul-Henning Kamp , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 04:21:19 -0000 On Wed, 5 Mar 2014, Justin Hibbits wrote: > On Wed, Mar 5, 2014 at 7:02 AM, Bruce Evans wrote: >> On Thu, 6 Mar 2014, Bruce Evans wrote: >> >>> On Tue, 4 Mar 2014, Justin Hibbits wrote: >>>> >>>> Without some kind of optimization, newcons on powerpc is unacceptably >>>> slow. >>> >>> >>> How slow is that exactly? If the frame buffer speed is 50MB/S, then >>> 16x8 characters with 256 colors can be written at 390K/S. This is >>> acceptable. If the frame buffer is much slower than that, then it is >>> too slow, at least without hardware scrolling. >> >> I forgot to mention that u$ doesn't solve this problem in WinXP. The >> speed is not too bad in normal mode (only about 30 times slower than >> syscons - the scrolling speed is 30000 lines/second instead of 1000000). >> But in safe mode, a simple graphics driver is used and it is horribly >> slow. Safe mode with command prompt gives an MSDOS prompt in a >> maximized terminal window, so the problem is very noticeable (the >> scrolling speed is 4 lines per second = 12 seconds per screenful). >> Apparently normal mode uses acceleration to be not very slow, but safe >> mode is too simple to do that. The FreeBSD console driver needs to >> be even simpler so that it works as a low level console. This hardware can do 120 MB/s with 64-bit writes. The window size in safe mode was 80x48 (I think with 16x8 characters). The number of pixels to move for dumb scrolling of a screenful is this 80x48*x16*8*48 ~= 23.5 million. I don't know if the mode is 256 colors, but assume that. So there are 23.5 million bytes and they should be moved in about 0.2 seconds instead of 12. Probably many of the following pessimizations are applied: - halve the speed by not using a backing buffer but reading the data to be scrolled from the frame buffer. 0.4 seconds at least. Probably much more. - use 8-bit accesses instead of 64-bit ones. Probably 8 times slower (see below). 3.2 seconds With a bit more work, 12 seconds can probably be attained. The 6 times slower read speed measured below on different hardware would do it easily. On a laptop that is likely to have a faster frame buffer and a more customized BIOS (I suspect safe mode uses the BIOS), the scrolling speed in safe mode is 3-4 times faster. > The ofwfb syscons driver is much faster than newcons, so a frame > buffer can be made relatively fast (within reason). It's newcons > that's slow, and my assumption currently is the byte-oriented writes > instead of word size. I suspect syscons is doing fewer updates, but the potential slowdowns are indeed very large: On my test hardware (AGP GeForce2 GTS; not quite the same as used above; slower), I get an almost perfectly linear slowdown as the access sizes decreases: for writing 100MB of data (50M characters in text mode): 64 bits: 1.37 seconds 32 bits: 2.35 seconds (42.5MB/sec) 16 bits: 4.70 seconds 8 bits: 9.40 seconds For reading: 32 bits: 14.5 seconds (6.9MB/sec) 8 bits: 58.1 seconds Reading is 6 times slower, so using 8-bit accesses instead of 64-bit ones doesn't just pessimize by a factor of 8 if it causes the hardware to do a read for every write. With the above timing, it changes from 1.37 seconds to 9.40 + 58.1 = 67.5 seconds (49 times slower). Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Mar 6 23:27:01 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B463D1BF; Thu, 6 Mar 2014 23:27:01 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 51ABEAC6; Thu, 6 Mar 2014 23:26:58 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WLhgm-000PoA-5v; Thu, 06 Mar 2014 23:26:56 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s26NQrE3051291; Thu, 6 Mar 2014 16:26:53 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/owmJLhMAGtPjedeQTb5Im Subject: Teach mdmfs about tmpfs and use tmpfs in rc scripts From: Ian Lepore To: freebsd-arch , freebsd-rc@FreeBSD.org Content-Type: multipart/mixed; boundary="=-Cvk3LbT8clLtJ22ceTxC" Date: Thu, 06 Mar 2014 16:26:53 -0700 Message-ID: <1394148413.1149.348.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2014 23:27:01 -0000 --=-Cvk3LbT8clLtJ22ceTxC Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit A recent discussion on arm@ about using tmpfs instead of md(4) spurred me to do something I've been wanting to do for a while: enhance mdmfs so that it can configure tmpfs as well as md+ffs, and update the rc scripts to make use the the feature. Attached are diffs to do this (man page updates not in there yet). mdmfs recognizes two new values for the md-device argument: tmpfs and auto. If you request 'tmpfs' you get it, or it fails if tmpfs is not present in the kernel. If you request 'auto' it will use tmpfs if it's present in the kernel and the other options don't include multilabel MAC (which tmpfs doesn't handle, as near as I can tell). Other options which have no meaning with tmpfs (such as setting the number of inodes) are silently ignored. The rc scripts are updated to add a new mfs_type knob, which defaults to "auto". This in effect automatically updates folks to use tmpfs instead of md as long as it's present and they aren't using the one option that forbids it (multilabel MAC). Thoughts? -- Ian --=-Cvk3LbT8clLtJ22ceTxC Content-Disposition: inline; filename="mdmfs_tmpfs.diff" Content-Type: text/x-patch; name="mdmfs_tmpfs.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: sys/fs/tmpfs/tmpfs_vfsops.c =================================================================== --- sys/fs/tmpfs/tmpfs_vfsops.c (revision 262759) +++ sys/fs/tmpfs/tmpfs_vfsops.c (working copy) @@ -60,6 +60,8 @@ __FBSDID("$FreeBSD$"); #include +FEATURE(tmpfs, "Efficient memory file system"); + /* * Default permission for root node */ Index: sbin/mdmfs/mdmfs.c =================================================================== --- sbin/mdmfs/mdmfs.c (revision 262759) +++ sbin/mdmfs/mdmfs.c (working copy) @@ -51,6 +51,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include typedef enum { false, true } bool; @@ -78,7 +79,8 @@ static void debugprintf(const char *, ...) __prin static void do_mdconfig_attach(const char *, const enum md_types); static void do_mdconfig_attach_au(const char *, const enum md_types); static void do_mdconfig_detach(void); -static void do_mount(const char *, const char *); +static void do_mount_md(const char *, const char *); +static void do_mount_tmpfs(const char *, const char *); static void do_mtptsetup(const char *, struct mtpt_info *); static void do_newfs(const char *); static void extract_ugid(const char *, struct mtpt_info *); @@ -90,11 +92,11 @@ main(int argc, char **argv) { struct mtpt_info mi; /* Mountpoint info. */ char *mdconfig_arg, *newfs_arg, /* Args to helper programs. */ - *mount_arg; + *mount_arg, *size_arg; enum md_types mdtype; /* The type of our memory disk. */ - bool have_mdtype; + bool have_mdtype, mlmac; bool detach, softdep, autounit, newfs; - char *mtpoint, *unitstr; + const char *mtpoint, *unitstr; char *p; int ch; void *set; @@ -105,6 +107,7 @@ main(int argc, char **argv) detach = true; softdep = true; autounit = false; + mlmac = false; newfs = true; have_mdtype = false; mdtype = MD_SWAP; @@ -119,6 +122,7 @@ main(int argc, char **argv) mdconfig_arg = strdup(""); newfs_arg = strdup(""); mount_arg = strdup(""); + size_arg = NULL; /* If we were started as mount_mfs or mfs, imply -C. */ if (strcmp(getprogname(), "mount_mfs") == 0 || @@ -175,6 +179,7 @@ main(int argc, char **argv) loudsubs = true; break; case 'l': + mlmac = true; argappend(&newfs_arg, "-l"); break; case 'M': @@ -213,7 +218,7 @@ main(int argc, char **argv) softdep = false; break; case 's': - argappend(&mdconfig_arg, "-s %s", optarg); + size_arg = optarg; break; case 't': argappend(&newfs_arg, "-t"); @@ -239,42 +244,64 @@ main(int argc, char **argv) if (argc < 2) usage(); - /* Derive 'unit' (global). */ + /* + * Based on the command line 'md-device' either mount a tmpfs filesystem + * or configure the md device then format and mount a filesystem on it. + * If the device is "auto" use tmpfs if it is available and there is no + * request for multilabel MAC on the filesystem. + */ unitstr = argv[0]; - if (strncmp(unitstr, "/dev/", 5) == 0) - unitstr += 5; - if (strncmp(unitstr, mdname, mdnamelen) == 0) - unitstr += mdnamelen; - if (!isdigit(*unitstr)) { - autounit = true; - unit = -1; - mdsuffix = unitstr; + mtpoint = argv[1]; + + if (strcmp(unitstr, "auto") == 0) { + if (feature_present("tmpfs") && !mlmac) + unitstr = "tmpfs"; + else + unitstr = "md"; + } + + if (strcmp(unitstr, "tmpfs") == 0) { + if (size_arg != NULL) + argappend(&mount_arg, "-o size=%s", size_arg); + do_mount_tmpfs(mount_arg, mtpoint); } else { - ul = strtoul(unitstr, &p, 10); - if (ul == ULONG_MAX) - errx(1, "bad device unit: %s", unitstr); - unit = ul; - mdsuffix = p; /* can be empty */ + if (size_arg != NULL) + argappend(&mdconfig_arg, "-s %s", size_arg); + if (strncmp(unitstr, "/dev/", 5) == 0) + unitstr += 5; + if (strncmp(unitstr, mdname, mdnamelen) == 0) + unitstr += mdnamelen; + if (!isdigit(*unitstr)) { + autounit = true; + unit = -1; + mdsuffix = unitstr; + } else { + ul = strtoul(unitstr, &p, 10); + if (ul == ULONG_MAX) + errx(1, "bad device unit: %s", unitstr); + unit = ul; + mdsuffix = p; /* can be empty */ + } + + if (!have_mdtype) + mdtype = MD_SWAP; + if (softdep) + argappend(&newfs_arg, "-U"); + if (mdtype != MD_VNODE && !newfs) + errx(1, "-P requires a vnode-backed disk"); + + /* Do the work. */ + if (detach && !autounit) + do_mdconfig_detach(); + if (autounit) + do_mdconfig_attach_au(mdconfig_arg, mdtype); + else + do_mdconfig_attach(mdconfig_arg, mdtype); + if (newfs) + do_newfs(newfs_arg); + do_mount_md(mount_arg, mtpoint); } - mtpoint = argv[1]; - if (!have_mdtype) - mdtype = MD_SWAP; - if (softdep) - argappend(&newfs_arg, "-U"); - if (mdtype != MD_VNODE && !newfs) - errx(1, "-P requires a vnode-backed disk"); - - /* Do the work. */ - if (detach && !autounit) - do_mdconfig_detach(); - if (autounit) - do_mdconfig_attach_au(mdconfig_arg, mdtype); - else - do_mdconfig_attach(mdconfig_arg, mdtype); - if (newfs) - do_newfs(newfs_arg); - do_mount(mount_arg, mtpoint); do_mtptsetup(mtpoint, &mi); return (0); @@ -431,7 +458,7 @@ do_mdconfig_detach(void) * Mount the configured memory disk. */ static void -do_mount(const char *args, const char *mtpoint) +do_mount_md(const char *args, const char *mtpoint) { int rv; @@ -442,6 +469,19 @@ static void } /* + * Mount the configured tmpfs. + */ +static void +do_mount_tmpfs(const char *args, const char *mtpoint) +{ + int rv; + + rv = run(NULL, "%s -t tmpfs %s tmp %s", _PATH_MOUNT, args, mtpoint); + if (rv) + errx(1, "mount exited with error code %d", rv); +} + +/* * Various configuration of the mountpoint. Mostly, enact 'mip'. */ static void Index: etc/defaults/rc.conf =================================================================== --- etc/defaults/rc.conf (revision 262759) +++ etc/defaults/rc.conf (working copy) @@ -51,6 +51,7 @@ tmpmfs_flags="-S" # Extra mdmfs options for the mf varmfs="AUTO" # Set to YES to always create an mfs /var, NO to never varsize="32m" # Size of mfs /var if created varmfs_flags="-S" # Extra mount options for the mfs /var +mfs_type="auto" # 'md', 'tmpfs', or "auto" to choose tmpfs when available populate_var="AUTO" # Set to YES to always (re)populate /var, NO to never cleanvar_enable="YES" # Clean the /var directory local_startup="/usr/local/etc/rc.d" # startup script dirs. Index: etc/rc.subr =================================================================== --- etc/rc.subr (revision 262759) +++ etc/rc.subr (working copy) @@ -1474,7 +1474,7 @@ mount_md() if [ -n "$3" ]; then flags="$3" fi - /sbin/mdmfs $flags -s $1 md $2 + /sbin/mdmfs $flags -s $1 ${mfs_type} $2 } # Code common to scripts that need to load a kernel module Index: etc/rc.initdiskless =================================================================== --- etc/rc.initdiskless (revision 262759) +++ etc/rc.initdiskless (working copy) @@ -198,7 +198,7 @@ handle_remount() { # $1 = mount point # Create a generic memory disk # mount_md() { - /sbin/mdmfs -S -i 4096 -s $1 -M md $2 + /sbin/mdmfs -S -i 4096 -s $1 -M auto $2 } # Create the memory filesystem if it has not already been created --=-Cvk3LbT8clLtJ22ceTxC-- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 04:52:38 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 795D415F; Fri, 7 Mar 2014 04:52:38 +0000 (UTC) Received: from mail-oa0-x22c.google.com (mail-oa0-x22c.google.com [IPv6:2607:f8b0:4003:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2CC36A0C; Fri, 7 Mar 2014 04:52:38 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id n16so3634621oag.17 for ; Thu, 06 Mar 2014 20:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=pQh8V+wqyyb/EQR12a5YHPsDqjtKDfUuVcIabMQxhUk=; b=rmiLYKY3Lcq/zniDG0bhdxlKvDvmdJFp+Lm++6H8PsbbGXIB0r/el7cotmd8RwqiTV LaF4eVjV/ZniUSDuAAtAQ/8RF3dEOvC+1W5BCHHvx0233g6f5bE9c5eAbRMZXvXUhG7W eWRLFYB1QxJsxja6q9fAuf+VNS4oPRGroGg0qSd0a+canxH8JmtgNuqSlIGJsAI+/U5c Tg12XzGGpMCY2taJwkbIQnSbrM0zsLewc7FSpW7Np9ientS9VN3gpypZzYYaW/KtmPXw fZMy9Y2Mygj1d+V1CzZUrcvcg9gJwQrY9JTx5fgzQKuJHU/YueCmhj2JIA1RttR5s6QE kXgQ== X-Received: by 10.182.230.135 with SMTP id sy7mr13454675obc.24.1394167957612; Thu, 06 Mar 2014 20:52:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.76.80.194 with HTTP; Thu, 6 Mar 2014 20:52:07 -0800 (PST) In-Reply-To: <1394148413.1149.348.camel@revolution.hippie.lan> References: <1394148413.1149.348.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Fri, 7 Mar 2014 12:52:07 +0800 Message-ID: Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-rc@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 04:52:38 -0000 On Fri, Mar 7, 2014 at 7:26 AM, Ian Lepore wrote: > > Thoughts? > Is it ok to default mdmfs to tmpfs behavior? Not sure if anyone would like to have explicit allocation e.g. failing early on insufficient memory, rather than failing on write. If so then at least 'md' should be in the options in addition to 'auto' and 'tmpfs' when both md and tmpfs are available from kernel. -Jia-Shiun. From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 10:52:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3786259; Fri, 7 Mar 2014 10:52:00 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id A2EFFE09; Fri, 7 Mar 2014 10:51:59 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id BA8DA5E11; Fri, 7 Mar 2014 10:51:52 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 40ECEEE3; Fri, 7 Mar 2014 11:51:28 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Adrian Chadd Subject: Re: importing sam leffler's libstatfoo into -HEAd References: Date: Fri, 07 Mar 2014 11:51:27 +0100 In-Reply-To: (Adrian Chadd's message of "Tue, 4 Mar 2014 18:31:59 -0800") Message-ID: <86ha7a7040.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 10:52:01 -0000 Adrian Chadd writes: > http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff Why did you rename it? The whole point of PRIVATELIB is to avoid having to rename libraries. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 13:37:01 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2CC4E99; Fri, 7 Mar 2014 13:37:01 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9534DF9B; Fri, 7 Mar 2014 13:37:01 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WLuxQ-00026e-3e; Fri, 07 Mar 2014 13:37:00 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s27DauOA052131; Fri, 7 Mar 2014 06:36:56 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19mnmnJIAT0VsBzl0Q+d/i3 Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts From: Ian Lepore To: Jia-Shiun Li In-Reply-To: References: <1394148413.1149.348.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Fri, 07 Mar 2014 06:36:56 -0700 Message-ID: <1394199416.1149.367.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-rc@FreeBSD.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 13:37:01 -0000 On Fri, 2014-03-07 at 12:52 +0800, Jia-Shiun Li wrote: > On Fri, Mar 7, 2014 at 7:26 AM, Ian Lepore wrote: > > > > Thoughts? > > > > Is it ok to default mdmfs to tmpfs behavior? Not sure if anyone would > like to have explicit allocation e.g. failing early on insufficient memory, > rather than failing on write. If so then at least 'md' should be in the > options in addition to 'auto' and 'tmpfs' when both md and tmpfs are > available from kernel. > I'm not sure what you mean. If the device on the command line is md the program behaves as it always has. If you ask for 'auto' you get the "best" memory filesystem available for some definition of "best". If you don't trust someone else's definition of best (like you need failure at allocation time) then you choose the one that behaves the way you like. -- Ian From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 13:52:19 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B94EC88A; Fri, 7 Mar 2014 13:52:19 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8FC501D8; Fri, 7 Mar 2014 13:52:19 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WLvCE-000A1z-7N; Fri, 07 Mar 2014 13:52:18 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s27DqFrK052142; Fri, 7 Mar 2014 06:52:15 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18B/tGzgsguymLLGEZx/nOf Subject: option NEW_PCIB From: Ian Lepore To: freebsd-arch , freebsd-arm Content-Type: text/plain; charset="us-ascii" Date: Fri, 07 Mar 2014 06:52:15 -0700 Message-ID: <1394200335.1149.370.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 13:52:19 -0000 Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except arm and mips. Is that on purpose? What are the implications of adding it? Or maybe more importantly, what are the implications of it not being there? -- Ian From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 14:38:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 043B96ED; Fri, 7 Mar 2014 14:38:04 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CFA4F7AC; Fri, 7 Mar 2014 14:38:03 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A20E5B96B; Fri, 7 Mar 2014 09:38:01 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: importing sam leffler's libstatfoo into -HEAd Date: Fri, 7 Mar 2014 08:37:23 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <86ha7a7040.fsf@nine.des.no> In-Reply-To: <86ha7a7040.fsf@nine.des.no> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201403070837.23590.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 07 Mar 2014 09:38:01 -0500 (EST) Cc: Adrian Chadd , Dag-Erling =?utf-8?q?Sm=C3=B8rgrav?= , freebsd-current X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 14:38:04 -0000 On Friday, March 07, 2014 5:51:27 am Dag-Erling Sm=C3=B8rgrav wrote: > Adrian Chadd writes: > > http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff >=20 > Why did you rename it? The whole point of PRIVATELIB is to avoid having > to rename libraries. Because 'statfoo' is a pretty silly name? (This is detailed in another subthread, did you not read that?) =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 14:38:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CAD6ABA0 for ; Fri, 7 Mar 2014 14:38:42 +0000 (UTC) Received: from mail-ie0-f170.google.com (mail-ie0-f170.google.com [209.85.223.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 937077CF for ; Fri, 7 Mar 2014 14:38:42 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id rd18so4362671iec.1 for ; Fri, 07 Mar 2014 06:38:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=W27fH1I87Ha8zkZSOY93njfGatefchgF18HihHKw/H8=; b=A2q1kP/FLoybY+yI5aqoeBoERtOKWoXZJE7LkU7d6ee5RLV6zmmcrNnC87NoJigGWe 1h/XzUr+YrwtKrtm1VJMt/OUSM9ouk+YaZqLYmCpUhWGQLtb3KAF1nqtnwnrk+gTK62U DxWfNA0/LfXKzVkkSQ7LJ56ljYiAZdpFGnIwicFCJn8PxRpcDPjg/JEPQ7dlbp6Qvi5t 4lDf9Q7GhsecSfLV7CsYLQAkJrRw8SxHmG3vzqPXjIARCg12NCRMdyCMUWF1qEInZ3C5 KmwAju41g/Ba71j4KcgXdb0KV3vgHPLWRZNNurWvnSr32F+bwz+3MZpefhvKHHw5Mmeo IOCQ== X-Gm-Message-State: ALoCoQk/8iYRpv0JfK0NDm2R/hu+an54necv+5ptD+2Sat/Fka7hJGW6OkEqgP77G/U5Nrq8drU2 X-Received: by 10.50.137.100 with SMTP id qh4mr3212252igb.34.1394203115760; Fri, 07 Mar 2014 06:38:35 -0800 (PST) Received: from netflix-mac.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id f1sm4531995igy.2.2014.03.07.06.38.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 07 Mar 2014 06:38:35 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: option NEW_PCIB From: Warner Losh In-Reply-To: <1394200335.1149.370.camel@revolution.hippie.lan> Date: Fri, 7 Mar 2014 07:38:33 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <58AB4C66-4267-414D-80D4-B97FF86A94A5@bsdimp.com> References: <1394200335.1149.370.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1874) Cc: freebsd-arm , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 14:38:42 -0000 On Mar 7, 2014, at 6:52 AM, Ian Lepore wrote: > Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except = arm > and mips. Is that on purpose? What are the implications of adding = it? > Or maybe more importantly, what are the implications of it not being > there? This is John Baldwin=92s option for his reworked PCI bridge code. He did = that as a fallback in case he really messed up something. It introduces = renumbering of busses that don=92t already have numbers assigned. It should be = enabled on ARM, but the required resource isn=92t defined on arm, and some of the = other required glue doesn=92t seem to be implemented for arm yet, which is why = things are the way they are at the moment. I think John intends for the option = to go away, and everything it covers will be =91standard=92. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 15:19:53 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 36F79B84; Fri, 7 Mar 2014 15:19:53 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AE836B4B; Fri, 7 Mar 2014 15:19:52 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id s27FJj4Q057002; Fri, 7 Mar 2014 17:19:45 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s27FJj4Q057002 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id s27FJjWe057001; Fri, 7 Mar 2014 17:19:45 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Mar 2014 17:19:45 +0200 From: Konstantin Belousov To: Ian Lepore Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts Message-ID: <20140307151945.GK24664@kib.kiev.ua> References: <1394148413.1149.348.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SpYKH75o2aPywVKk" Content-Disposition: inline In-Reply-To: <1394148413.1149.348.camel@revolution.hippie.lan> User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-rc@FreeBSD.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 15:19:53 -0000 --SpYKH75o2aPywVKk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 06, 2014 at 04:26:53PM -0700, Ian Lepore wrote: > A recent discussion on arm@ about using tmpfs instead of md(4) spurred > me to do something I've been wanting to do for a while: enhance mdmfs > so that it can configure tmpfs as well as md+ffs, and update the rc > scripts to make use the the feature. >=20 > Attached are diffs to do this (man page updates not in there yet). >=20 > mdmfs recognizes two new values for the md-device argument: tmpfs and > auto. If you request 'tmpfs' you get it, or it fails if tmpfs is not > present in the kernel. If you request 'auto' it will use tmpfs if it's > present in the kernel and the other options don't include multilabel MAC > (which tmpfs doesn't handle, as near as I can tell). Other options > which have no meaning with tmpfs (such as setting the number of inodes) > are silently ignored. >=20 > The rc scripts are updated to add a new mfs_type knob, which defaults to > "auto". This in effect automatically updates folks to use tmpfs instead > of md as long as it's present and they aren't using the one option that > forbids it (multilabel MAC). >=20 > Thoughts? >=20 > -- Ian >=20 > Index: sys/fs/tmpfs/tmpfs_vfsops.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/fs/tmpfs/tmpfs_vfsops.c (revision 262759) > +++ sys/fs/tmpfs/tmpfs_vfsops.c (working copy) > @@ -60,6 +60,8 @@ __FBSDID("$FreeBSD$"); > =20 > #include > =20 > +FEATURE(tmpfs, "Efficient memory file system"); > + The existing way to check for the presence of the filesystem module in the kernel is sysctl vfs.conflist or getvfsbyname(3) wrapper around it. I suspect that the interface is unknown because the information about fs module presence in kernel is pretty much useless: nmount(2) syscall auto-loads required fs module if not already present. That said, the feature for tmpfs is redundand and should not be added. I do not see adding the ability to issue 'mount -t tmpfs ...' command to mdmfs(8) as right approach. The helper is already complicated and its single-purpose functionality of newfs-ing md(4) IMO should be left as is. If you want to to have /var on tmpfs, why not change rc.d/var ? It would be much simpler, since the change would be one-liner for mount_md line. --SpYKH75o2aPywVKk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTGeOQAAoJEJDCuSvBvK1BTAIP/139L6LJqJHByfKHXdILpHkt rFoP+tidt0Q9gXbko5wQLCSgujLPTe/y0KF1nz46EyJanqf3iwmz+CHKXfdOxBcy Gefrypx3WAHCLkgAOIIyFecTFaTh7TKMy1KTtpstj4a3pIKKwgJUd0RC7xZQB8it 6tM7DDIlDxP4Sribio0sSgjzxDY0pTQyKRq5gDz/eV2f3/vxBSm91O4ew+0tZTXy 0Cy1mqPlwZ5zC3Oh3XufSa5SPz5c22LCN0Z29gizhnDkreeGWZO3THPMWU8kZ31u jb+VegWA5k4uAkcmAbJNvk4fRTqAal8pBSLhIbowFz4Bppgm8kIOTEIR75R+Rccz MW6V9yt316+ngzuUwHNlB1lgPclXCv/beskbLZQ3e3uSaCU8pNc1pxNWmJtTL7p7 dfugxnRpAdizoYfMQsABf1fjQH8oQ08PIJsGFqdP2gP9AHUgtzI7p3Yg2XQiaaaJ unhyrYMEezrTt2bc9gmOAqDiOX6GepLjEn6lR0wq9kNXYswfZBHjvdafvivTaWaq oLLqjKiXNHagFqfPQtIb5WPBRQjPtCWCw1nTBiY7O8FR+T5HRiNqlgvwZ9z4OEqI TeemIfL39l+7n7IT8cHlRPyOeEeBfLMQqBLkv7WyvkNcvDdR7iCtbyO2CDXkhV60 qMhfOZBqh7rEeyCz4bBh =y87W -----END PGP SIGNATURE----- --SpYKH75o2aPywVKk-- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 7 17:44:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AA39CFD; Fri, 7 Mar 2014 17:44:13 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C389EAB9; Fri, 7 Mar 2014 17:44:12 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id i50so8014018qgf.0 for ; Fri, 07 Mar 2014 09:44:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=lpNTwTQY35dI8OvUHHhg6MEsgBsc3NkQT/3s6VSjv7s=; b=zRer8urcN3FxaPvR6boahbw4cQWx1ghC+33hgwnl+XAoJazD1+Es6OypeM1O5TUVhl tI3RwYHgCJ87LiL761leDO8b9vGbXaQWip7ANZ0PUITisGyQurf0D2MSa9sMWh1fPbSR +RHwPdQDSxI6198r42WrlA1z3R/njhRn5yUa3KjjYcreQTuRbN3bpp/McKeS5fAlF6Zn rJ3GmJWbEYzMpsL+eI/K988XoFhxsBW27gN8AnsFCOplweI6aQ0+IaVHyZ9J9zgFAcl5 uB69D+qQ6Q0PMWHzu5/7jzlsQKyjBx76pCw3XqaB7oue1YLhUBXvpfoZULpWSgcc5t42 MSOA== MIME-Version: 1.0 X-Received: by 10.229.66.202 with SMTP id o10mr16903567qci.7.1394214252024; Fri, 07 Mar 2014 09:44:12 -0800 (PST) Received: by 10.224.8.137 with HTTP; Fri, 7 Mar 2014 09:44:11 -0800 (PST) In-Reply-To: <201403070837.23590.jhb@freebsd.org> References: <86ha7a7040.fsf@nine.des.no> <201403070837.23590.jhb@freebsd.org> Date: Fri, 7 Mar 2014 09:44:11 -0800 Message-ID: Subject: Re: importing sam leffler's libstatfoo into -HEAd From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Mar 2014 17:44:13 -0000 what he said. -a On 7 March 2014 05:37, John Baldwin wrote: > On Friday, March 07, 2014 5:51:27 am Dag-Erling Sm=F8rgrav wrote: >> Adrian Chadd writes: >> > http://people.freebsd.org/~adrian/patches/20140304-libbsdstatfoo.diff >> >> Why did you rename it? The whole point of PRIVATELIB is to avoid having >> to rename libraries. > > Because 'statfoo' is a pretty silly name? (This is detailed in another > subthread, did you not read that?) > > -- > John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Mar 8 13:31:59 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70F43980; Sat, 8 Mar 2014 13:31:59 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3038D25E; Sat, 8 Mar 2014 13:31:58 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id CFBB05D98; Sat, 8 Mar 2014 13:31:51 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id B7BE836D03; Sat, 8 Mar 2014 14:31:22 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: John Baldwin Subject: Re: importing sam leffler's libstatfoo into -HEAd References: <86ha7a7040.fsf@nine.des.no> <201403070837.23590.jhb@freebsd.org> Date: Sat, 08 Mar 2014 14:31:20 +0100 In-Reply-To: <201403070837.23590.jhb@freebsd.org> (John Baldwin's message of "Fri, 7 Mar 2014 08:37:23 -0500") Message-ID: <86lhwkn7fb.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Adrian Chadd , freebsd-current , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 13:31:59 -0000 John Baldwin writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Why did you rename it? The whole point of PRIVATELIB is to avoid > > having to rename libraries. > Because 'statfoo' is a pretty silly name? (This is detailed in > another subthread, did you not read that?) The original patch renamed it to "bsdstatfoo", did you not read that? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 8 14:24:38 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46E858B8; Sat, 8 Mar 2014 14:24:38 +0000 (UTC) Received: from mail-oa0-x231.google.com (mail-oa0-x231.google.com [IPv6:2607:f8b0:4003:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EBC4882C; Sat, 8 Mar 2014 14:24:37 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id g12so5339687oah.36 for ; Sat, 08 Mar 2014 06:24:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=s5CFWtJqPqzYLDANsAuTSQV9pW5Ds/mR2DACmLBZ35U=; b=SqCRyUbx7DqZesLdZRi+ZAlLn39M3+LeYIGPONmZU/ZPP3KOv1XgZoNveBXPYMDowE d9rxrPNQU48FZlMZDkRXjjHZUdY8c7h6zKiuTY2xsz4cS2KHa5Yu7CLdFC5eoznBezPA AwNRQcxLlufKoTem4S8y86cTcSKlwy0nkk3QfUxkKLCsf7Y80IcpE8SMRlSxVzBka3k4 AFou+ahoBywFepxxp+Sxsnwg9jfARnqecW6gdaiQtkpj3fkg+tfzgGOCH/5tgn75Zc8k Vu6lkLtOVeinGalrT+Pvt+ygJZQmGIrQ3JD2IWR6omoB1OIVJHoWXoTadLdeOuh+FoEV iLxA== X-Received: by 10.60.115.129 with SMTP id jo1mr16256116oeb.0.1394288677357; Sat, 08 Mar 2014 06:24:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.76.80.194 with HTTP; Sat, 8 Mar 2014 06:24:07 -0800 (PST) In-Reply-To: <1394199416.1149.367.camel@revolution.hippie.lan> References: <1394148413.1149.348.camel@revolution.hippie.lan> <1394199416.1149.367.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Sat, 8 Mar 2014 22:24:07 +0800 Message-ID: Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-rc@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 14:24:38 -0000 On Fri, Mar 7, 2014 at 9:36 PM, Ian Lepore wrote: > I'm not sure what you mean. If the device on the command line is md the > program behaves as it always has. If you ask for 'auto' you get the > "best" memory filesystem available for some definition of "best". If > you don't trust someone else's definition of best (like you need failure > at allocation time) then you choose the one that behaves the way you > like. ehh.. sorry read too fast and did not realize you 'added' new options in addition to existing 'md'. My bad. I did not use mdmfs often. My understanding is that it is the help to simplify mdconfig-newfs-mount process to replace a one-step mount_mfs. Then I agree with Konstantin, it does not look too appealing. If the goal is to merge all memory-backed fs to single versatile command, then the proposal does the job. Otherwise one mount_tmpfs comamnd does all user needs for tmpfs, and I am not sure the auto decision is good if user did not know what he need or care. It seems better to leave the decision to user. And for rc usage, I think we can just change it to tmpfs. If in the future tmpfs grows ability to populate content with e.g. a cpio archive at boot time or passed from command line, md usage can be further replaced. -Jia-Shiun. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 8 14:54:41 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3870D123; Sat, 8 Mar 2014 14:54:41 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 08B2FA2C; Sat, 8 Mar 2014 14:54:40 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WMIdz-0005yr-A6; Sat, 08 Mar 2014 14:54:33 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s28EsTjB053504; Sat, 8 Mar 2014 07:54:29 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18WkOkKEtxc0xY+V/NE2Rk0 Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts From: Ian Lepore To: Jia-Shiun Li In-Reply-To: References: <1394148413.1149.348.camel@revolution.hippie.lan> <1394199416.1149.367.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sat, 08 Mar 2014 07:54:28 -0700 Message-ID: <1394290468.1149.397.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-rc@FreeBSD.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 14:54:41 -0000 On Sat, 2014-03-08 at 22:24 +0800, Jia-Shiun Li wrote: > On Fri, Mar 7, 2014 at 9:36 PM, Ian Lepore wrote: > > I'm not sure what you mean. If the device on the command line is md the > > program behaves as it always has. If you ask for 'auto' you get the > > "best" memory filesystem available for some definition of "best". If > > you don't trust someone else's definition of best (like you need failure > > at allocation time) then you choose the one that behaves the way you > > like. > > ehh.. sorry read too fast and did not realize you 'added' new options > in addition to existing 'md'. My bad. > > I did not use mdmfs often. My understanding is that it is the help to simplify > mdconfig-newfs-mount process to replace a one-step mount_mfs. > Then I agree with Konstantin, it does not look too appealing. > If the goal is to merge all memory-backed fs to single versatile command, > then the proposal does the job. Otherwise one mount_tmpfs comamnd does > all user needs for tmpfs, and I am not sure the auto decision is good if > user did not know what he need or care. It seems better to leave the decision > to user. > > And for rc usage, I think we can just change it to tmpfs. If in the future tmpfs > grows ability to populate content with e.g. a cpio archive at boot time or > passed from command line, md usage can be further replaced. > > -Jia-Shiun. Okay, if people think that all this work should be done in the rc scripts rather than in a program, then the rc scripts need to be changed to do what I did in the program: honor existing options that make sense for tmpfs (any -o options for the mount, translate -s to size=, and don't use tmpfs if the config requests multilabel MAC). And the changes need to happen in rc.subr and in rc.initdiskless which doesn't use rc.subr. Oh, and of course, don't do any of it if tmpfs isn't available. If this isn't done, peoples' existing configurations may break (in the case of the MAC option, break in a way with potential security implications). I've gotta say, I don't understand the basic resistance to having a single unified tool for configuring a memory filesystem. -- Ian From owner-freebsd-arch@FreeBSD.ORG Sat Mar 8 17:12:25 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 371BF63D; Sat, 8 Mar 2014 17:12:25 +0000 (UTC) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DB23C6B8; Sat, 8 Mar 2014 17:12:24 +0000 (UTC) Received: by mail-ob0-f178.google.com with SMTP id wp18so5412625obc.23 for ; Sat, 08 Mar 2014 09:12:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=lTewJyv+dA1klkTiv2wkMblGiNrwBwdS2SGlSBgvW8g=; b=CX8yLLZiYx5CJO+OlwXZ/ywdKe9MafyDLbeLPMxOh+wxqZoNyHfcuwAtRuTOhdzR9p mtuoFn152EPQd8ydoyHBX0JITFr24cRnsUNaT6L3+3VWfcvpCMD1eZqYG6BfHdufbLw4 TqnGu6h8HlTcSrK+VS6a0Of8E+VsyYrGNF0v+tyV6p1wiWvYpAtUAdqHiOzrpe/696Bg X/J0p1UxWKtAX5qRvmxVyW4wrfhwd+U/lOa19Mwa/Sbeeq40lG5BEtL1wsHMQUOZduRi OFbZ1+HYRF1GwrUjaJuBWeMZgXwwK4OsonrjMt95ZLMWt+MBhgwO9LApRPDVkyOqb2GA JfTg== X-Received: by 10.60.51.230 with SMTP id n6mr16650075oeo.35.1394298744183; Sat, 08 Mar 2014 09:12:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.76.80.194 with HTTP; Sat, 8 Mar 2014 09:11:54 -0800 (PST) In-Reply-To: <1394290468.1149.397.camel@revolution.hippie.lan> References: <1394148413.1149.348.camel@revolution.hippie.lan> <1394199416.1149.367.camel@revolution.hippie.lan> <1394290468.1149.397.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Sun, 9 Mar 2014 01:11:54 +0800 Message-ID: Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-rc@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 17:12:25 -0000 On Sat, Mar 8, 2014 at 10:54 PM, Ian Lepore wrote: > Okay, if people think that all this work should be done in the rc > scripts rather than in a program, then the rc scripts need to be changed > to do what I did in the program: honor existing options that make sense > for tmpfs (any -o options for the mount, translate -s to size=, and > don't use tmpfs if the config requests multilabel MAC). And the changes > need to happen in rc.subr and in rc.initdiskless which doesn't use > rc.subr. Oh, and of course, don't do any of it if tmpfs isn't > available. > > If this isn't done, peoples' existing configurations may break (in the > case of the MAC option, break in a way with potential security > implications). > > I've gotta say, I don't understand the basic resistance to having a > single unified tool for configuring a memory filesystem. > Probably because we were focusing on the usual rc case? For the usual rc case it is easier to change scripts to use tmpfs altogether. But it will break compatibility of rc.conf flags too. Upgraded users will need to modify them manually. And I am not sure if there are more advanced uses, like setting multilabel for /var & /tmp. In these cases your patch makes more sense. Think that's just us not catching your point yet. ;) -Jia-Shiun. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 8 17:43:00 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5BEE9D3B; Sat, 8 Mar 2014 17:43:00 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2B36594A; Sat, 8 Mar 2014 17:42:59 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WMLH0-000KCF-Kq; Sat, 08 Mar 2014 17:42:58 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s28Hgu4n053669; Sat, 8 Mar 2014 10:42:56 -0700 (MST) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+jpFxbKLw3qfqfEcPT4hCI Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts From: Ian Lepore To: Jia-Shiun Li In-Reply-To: References: <1394148413.1149.348.camel@revolution.hippie.lan> <1394199416.1149.367.camel@revolution.hippie.lan> <1394290468.1149.397.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sat, 08 Mar 2014 10:42:56 -0700 Message-ID: <1394300576.1149.413.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-rc@FreeBSD.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2014 17:43:00 -0000 On Sun, 2014-03-09 at 01:11 +0800, Jia-Shiun Li wrote: > On Sat, Mar 8, 2014 at 10:54 PM, Ian Lepore wrote: > > Okay, if people think that all this work should be done in the rc > > scripts rather than in a program, then the rc scripts need to be changed > > to do what I did in the program: honor existing options that make sense > > for tmpfs (any -o options for the mount, translate -s to size=, and > > don't use tmpfs if the config requests multilabel MAC). And the changes > > need to happen in rc.subr and in rc.initdiskless which doesn't use > > rc.subr. Oh, and of course, don't do any of it if tmpfs isn't > > available. > > > > If this isn't done, peoples' existing configurations may break (in the > > case of the MAC option, break in a way with potential security > > implications). > > > > I've gotta say, I don't understand the basic resistance to having a > > single unified tool for configuring a memory filesystem. > > > > Probably because we were focusing on the usual rc case? > For the usual rc case it is easier to change scripts to use > tmpfs altogether. But it will break compatibility of rc.conf > flags too. Upgraded users will need to modify them manually. > And I am not sure if there are more advanced uses, like > setting multilabel for /var & /tmp. In these cases your > patch makes more sense. > > Think that's just us not catching your point yet. ;) > Part of the reason I jumped on this when you brought it up on the arm list is because we're heavy users of the readonly rootfs support rc scripts at $work (and not many people are), so I wanted to make sure the change wasn't done on the assumption that you can just hard-code tmpfs in place of md in the rc scripts "and stuff will just keep working". That's only true for some limited subset of "stuff" and while it's certainly possible to manually change your rc.conf to match when upgrading, that's actually hard to do when you're trying to manage software ranging from freebsd 6 through 11 all with a common config and build and deployment system. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Mar 10 10:16:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31F0C191; Mon, 10 Mar 2014 10:16:42 +0000 (UTC) Received: from mail-ob0-x22e.google.com (mail-ob0-x22e.google.com [IPv6:2607:f8b0:4003:c01::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D5F99114; Mon, 10 Mar 2014 10:16:41 +0000 (UTC) Received: by mail-ob0-f174.google.com with SMTP id wo20so6690546obc.5 for ; Mon, 10 Mar 2014 03:16:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=SwVL4hyzFtyCDbIF7z/S0rI7GukauMjo6j/udBKHVis=; b=zWg25xq88IcvIviZtjTjjTiruGpjmyWpE2hGdJdfpG7e3lx+4JG/RYPX9AJloMwLNk E4iDNTdFJdjF9Gb1PHo1Yf217X4uYYgjjdie6vIWeA+xo2GWggIx4d3zQO8AUymNWaRE yQlUxwkFFDkKI/Ndj+vnCGsPZCkx2q1YH/ZN3EehZj2fPiSEcAO09UXYG/bsd3+CghV/ Atn3l0CezcSPzXWHzLEQZMBMmz8pGo/DEKVGC1IVtEBjQoGjEf6c+n5lTyq5YaK+DwoS GWX+eKU3497KSha+g65vK8xeLkmKzUXU9tO+Z186V7TtjV8e0WgcxVy9cNY44X0eEMZZ 6nuQ== X-Received: by 10.182.2.42 with SMTP id 10mr22308obr.73.1394446601158; Mon, 10 Mar 2014 03:16:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.80.194 with HTTP; Mon, 10 Mar 2014 03:16:11 -0700 (PDT) In-Reply-To: <1394300576.1149.413.camel@revolution.hippie.lan> References: <1394148413.1149.348.camel@revolution.hippie.lan> <1394199416.1149.367.camel@revolution.hippie.lan> <1394290468.1149.397.camel@revolution.hippie.lan> <1394300576.1149.413.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Mon, 10 Mar 2014 18:16:11 +0800 Message-ID: Subject: Re: Teach mdmfs about tmpfs and use tmpfs in rc scripts To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-rc@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 10:16:42 -0000 On Sun, Mar 9, 2014 at 1:42 AM, Ian Lepore wrote: > > Part of the reason I jumped on this when you brought it up on the arm > list is because we're heavy users of the readonly rootfs support rc > scripts at $work (and not many people are), so I wanted to make sure the > change wasn't done on the assumption that you can just hard-code tmpfs > in place of md in the rc scripts "and stuff will just keep working". > That's only true for some limited subset of "stuff" and while it's > certainly possible to manually change your rc.conf to match when > upgrading, that's actually hard to do when you're trying to manage > software ranging from freebsd 6 through 11 all with a common config and > build and deployment system. > Does FreeBSD have policy or guideline on how to maintain compatibility? After all, consider the tmpfs case it will be literally neither md nor mfs. That said, guess that is too trivial to bikeshed. Functionally I agree it is good. It maintains compatibility while making use of better tmpfs. Do you plan to commit it? -Jia-Shiun. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 10 15:43:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4659F60; Mon, 10 Mar 2014 15:43:52 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B7199AD; Mon, 10 Mar 2014 15:43:52 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D3814B926; Mon, 10 Mar 2014 11:43:48 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: option NEW_PCIB Date: Mon, 10 Mar 2014 09:45:20 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <1394200335.1149.370.camel@revolution.hippie.lan> <58AB4C66-4267-414D-80D4-B97FF86A94A5@bsdimp.com> In-Reply-To: <58AB4C66-4267-414D-80D4-B97FF86A94A5@bsdimp.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201403100945.20298.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 10 Mar 2014 11:43:48 -0400 (EDT) Cc: freebsd-arm , Ian Lepore X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 15:43:52 -0000 On Friday, March 07, 2014 9:38:33 am Warner Losh wrote: >=20 > On Mar 7, 2014, at 6:52 AM, Ian Lepore wrote: >=20 > > Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except arm > > and mips. Is that on purpose? What are the implications of adding it? > > Or maybe more importantly, what are the implications of it not being > > there? >=20 > This is John Baldwin=92s option for his reworked PCI bridge code. He did = that as > a fallback in case he really messed up something. It introduces renumberi= ng > of busses that don=92t already have numbers assigned. It should be enable= d on > ARM, but the required resource isn=92t defined on arm, and some of the ot= her > required glue doesn=92t seem to be implemented for arm yet, which is why = things > are the way they are at the moment. I think John intends for the option t= o go > away, and everything it covers will be =91standard=92. Yes. I just added a page on the wiki about NEW_PCIB explaining the changes each platform needs for it in a bit more detail on Friday: https://wiki.freebsd.org/NEW_PCIB I have posted patches in the past to arm@ to handle step 2 in the NEW_PCIB base requirements for arm@ but haven't been able to get folks to test them. I just recently made a new pass through sys/arm in a p4 tree to refresh thi= s. I haven't even compiled these yet, but you can find the patch here: http://people.freebsd.org/~jhb/patches/arm_activate2.patch I don't know how best to think about fixing i80321_pci to work with NEW_PCI= B. It has some hack that I don't fully understand. I think it uses an alternate mapping of the same resource range to use a different base address for the mapping. Longer term I think the bus_map_resource() think I suggest at the bottom is how to handle that, but even then there would still need to be a way to know which base address a given resource wanted to use. It may be that we need to implement that differently (bus-specific rman flag?) =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 10 16:20:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2BC2F79 for ; Mon, 10 Mar 2014 16:20:08 +0000 (UTC) Received: from mail-pd0-f169.google.com (mail-pd0-f169.google.com [209.85.192.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 74AE8CF6 for ; Mon, 10 Mar 2014 16:20:08 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id fp1so7259589pdb.14 for ; Mon, 10 Mar 2014 09:20:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=y3de8dFHuByGitLfgxttDdBSNrutbxKFQEbGTTC6OBY=; b=AqsUU0j2f9eC42+ZpbB6gVSCwbetURpQ2EhV63qIOev+nwDIrxgja39jjMtymqKla2 QfV5Inx95hEZPUkSfCCqa65i+W8fuSMD7vGZvxOMQMcPjkzvhaJOTPT1TUNZRbxYZZBJ Lg4hqQ0kFwtgXqPp6eYRHrGEval75jiSs9+ykL8yCp4fL2Xi70IHgklcw5fMryr6GFWS SNhZ6Hu6Y61GK6/GEkgAuIS6qtgH65bKkJ5a2nrywGrI+ZJOs6HsccLGAZlcZRUT3Fnq eG2q3t8VB3decbTY3VzjfmROlwmRyBTbZ6j/AqFB1Bnum4IQgYW7w5Us+sjYhwPc9s6f qA8A== X-Gm-Message-State: ALoCoQl8NKm7f/BOpwhqtuVMPE2kN6zoHYoJyPyOzBspDA+GxqlHAxR9Qook1VU6bLyzKdJ2OOlf X-Received: by 10.68.229.106 with SMTP id sp10mr41460628pbc.23.1394468404764; Mon, 10 Mar 2014 09:20:04 -0700 (PDT) Received: from [10.64.26.252] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id bc4sm65919122pbb.2.2014.03.10.09.20.03 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 10 Mar 2014 09:20:04 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: option NEW_PCIB From: Warner Losh In-Reply-To: <201403100945.20298.jhb@freebsd.org> Date: Mon, 10 Mar 2014 10:20:02 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <51AC226F-7516-48EA-BC9D-B67F6508BCBF@bsdimp.com> References: <1394200335.1149.370.camel@revolution.hippie.lan> <58AB4C66-4267-414D-80D4-B97FF86A94A5@bsdimp.com> <201403100945.20298.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1874) Cc: freebsd-arm , Ian Lepore , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 16:20:08 -0000 On Mar 10, 2014, at 7:45 AM, John Baldwin wrote: > On Friday, March 07, 2014 9:38:33 am Warner Losh wrote: >>=20 >> On Mar 7, 2014, at 6:52 AM, Ian Lepore wrote: >>=20 >>> Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except = arm >>> and mips. Is that on purpose? What are the implications of adding = it? >>> Or maybe more importantly, what are the implications of it not being >>> there? >>=20 >> This is John Baldwin=92s option for his reworked PCI bridge code. He = did that as >> a fallback in case he really messed up something. It introduces = renumbering >> of busses that don=92t already have numbers assigned. It should be = enabled on >> ARM, but the required resource isn=92t defined on arm, and some of = the other >> required glue doesn=92t seem to be implemented for arm yet, which is = why things >> are the way they are at the moment. I think John intends for the = option to go >> away, and everything it covers will be =91standard=92. >=20 > Yes. I just added a page on the wiki about NEW_PCIB explaining the = changes > each platform needs for it in a bit more detail on Friday: >=20 > https://wiki.freebsd.org/NEW_PCIB >=20 > I have posted patches in the past to arm@ to handle step 2 in the = NEW_PCIB > base requirements for arm@ but haven't been able to get folks to test = them. > I just recently made a new pass through sys/arm in a p4 tree to = refresh this. > I haven't even compiled these yet, but you can find the patch here: >=20 > http://people.freebsd.org/~jhb/patches/arm_activate2.patch I=92ll take a look=85 Thanks for putting these together... > I don't know how best to think about fixing i80321_pci to work with = NEW_PCIB. > It has some hack that I don't fully understand. I think it uses an > alternate mapping of the same resource range to use a different base = address > for the mapping. Longer term I think the bus_map_resource() think I = suggest > at the bottom is how to handle that, but even then there would still = need to > be a way to know which base address a given resource wanted to use. = It may > be that we need to implement that differently (bus-specific rman = flag?) While I=92m not normally a big fan of retiring code, I think in this = case it may be easier to just retire the i80321 support. It is really old, released in 2002. = There=92s not too many boards that had this on it, and I think all the ones that ever ran = FreeBSD are in cognet@=91s lab. On the whole, it is yet another machine to update that slows down = any modernization processes we have... Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 10 17:52:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3676E490; Mon, 10 Mar 2014 17:52:57 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CECD29A7; Mon, 10 Mar 2014 17:52:56 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s2AHqnIY071921 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 10 Mar 2014 10:52:50 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s2AHqn1u071920; Mon, 10 Mar 2014 10:52:49 -0700 (PDT) (envelope-from jmg) Date: Mon, 10 Mar 2014 10:52:49 -0700 From: John-Mark Gurney To: John Baldwin Subject: Re: option NEW_PCIB Message-ID: <20140310175249.GH32089@funkthat.com> Mail-Followup-To: John Baldwin , freebsd-arch@freebsd.org, freebsd-arm , Ian Lepore References: <1394200335.1149.370.camel@revolution.hippie.lan> <58AB4C66-4267-414D-80D4-B97FF86A94A5@bsdimp.com> <201403100945.20298.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201403100945.20298.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 10 Mar 2014 10:52:50 -0700 (PDT) Cc: freebsd-arm , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 17:52:57 -0000 John Baldwin wrote this message on Mon, Mar 10, 2014 at 09:45 -0400: > On Friday, March 07, 2014 9:38:33 am Warner Losh wrote: > > > > On Mar 7, 2014, at 6:52 AM, Ian Lepore wrote: > > > > > Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except arm > > > and mips. Is that on purpose? What are the implications of adding it? > > > Or maybe more importantly, what are the implications of it not being > > > there? > > > > This is John Baldwin?s option for his reworked PCI bridge code. He did that as > > a fallback in case he really messed up something. It introduces renumbering > > of busses that don?t already have numbers assigned. It should be enabled on > > ARM, but the required resource isn?t defined on arm, and some of the other > > required glue doesn?t seem to be implemented for arm yet, which is why things > > are the way they are at the moment. I think John intends for the option to go > > away, and everything it covers will be ?standard?. > > Yes. I just added a page on the wiki about NEW_PCIB explaining the changes > each platform needs for it in a bit more detail on Friday: > > https://wiki.freebsd.org/NEW_PCIB > > I have posted patches in the past to arm@ to handle step 2 in the NEW_PCIB > base requirements for arm@ but haven't been able to get folks to test them. > I just recently made a new pass through sys/arm in a p4 tree to refresh this. > I haven't even compiled these yet, but you can find the patch here: > > http://people.freebsd.org/~jhb/patches/arm_activate2.patch Do you need a fully work system, or is booting fine? If you need a fully working system, then I need some help figure out my panic.. I sent email to alc and kib a few days ago about it, but I haven't heard anything back.. it's a problem w/ vm_page... If booting is fine, I can test on my AVILA board I have now.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Mon Mar 10 19:01:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA1D1E39; Mon, 10 Mar 2014 19:01:55 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8E2089B; Mon, 10 Mar 2014 19:01:55 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2E41AB9A0; Mon, 10 Mar 2014 15:01:53 -0400 (EDT) From: John Baldwin To: "John-Mark Gurney" Subject: Re: option NEW_PCIB Date: Mon, 10 Mar 2014 14:20:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <1394200335.1149.370.camel@revolution.hippie.lan> <201403100945.20298.jhb@freebsd.org> <20140310175249.GH32089@funkthat.com> In-Reply-To: <20140310175249.GH32089@funkthat.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403101420.16117.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 10 Mar 2014 15:01:53 -0400 (EDT) Cc: freebsd-arm , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Mar 2014 19:01:55 -0000 On Monday, March 10, 2014 1:52:49 pm John-Mark Gurney wrote: > John Baldwin wrote this message on Mon, Mar 10, 2014 at 09:45 -0400: > > On Friday, March 07, 2014 9:38:33 am Warner Losh wrote: > > > > > > On Mar 7, 2014, at 6:52 AM, Ian Lepore wrote: > > > > > > > Every architecture has "option NEW_PCIB" in its conf/DEFAULTS except arm > > > > and mips. Is that on purpose? What are the implications of adding it? > > > > Or maybe more importantly, what are the implications of it not being > > > > there? > > > > > > This is John Baldwin?s option for his reworked PCI bridge code. He did that as > > > a fallback in case he really messed up something. It introduces renumbering > > > of busses that don?t already have numbers assigned. It should be enabled on > > > ARM, but the required resource isn?t defined on arm, and some of the other > > > required glue doesn?t seem to be implemented for arm yet, which is why things > > > are the way they are at the moment. I think John intends for the option to go > > > away, and everything it covers will be ?standard?. > > > > Yes. I just added a page on the wiki about NEW_PCIB explaining the changes > > each platform needs for it in a bit more detail on Friday: > > > > https://wiki.freebsd.org/NEW_PCIB > > > > I have posted patches in the past to arm@ to handle step 2 in the NEW_PCIB > > base requirements for arm@ but haven't been able to get folks to test them. > > I just recently made a new pass through sys/arm in a p4 tree to refresh this. > > I haven't even compiled these yet, but you can find the patch here: > > > > http://people.freebsd.org/~jhb/patches/arm_activate2.patch > > Do you need a fully work system, or is booting fine? If you need a > fully working system, then I need some help figure out my panic.. I sent > email to alc and kib a few days ago about it, but I haven't heard anything > back.. it's a problem w/ vm_page... > > If booting is fine, I can test on my AVILA board I have now.. Well, the system needs to boot far enough to actually use memory and I/O port resources for PCI devices. That probably means getting to at least single user and being able to pass a network packet, etc. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 13 01:55:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF4A932F for ; Thu, 13 Mar 2014 01:55:12 +0000 (UTC) Received: from eastrmfepi106.cox.net (eastrmfepi106.cox.net [68.230.241.202]) by mx1.freebsd.org (Postfix) with ESMTP id 8650D122 for ; Thu, 13 Mar 2014 01:55:12 +0000 (UTC) Received: from eastrmimpo210 ([68.230.241.225]) by eastrmfepo103.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140313002401.TKAK28095.eastrmfepo103.cox.net@eastrmimpo210> for ; Wed, 12 Mar 2014 20:24:01 -0400 Received: from [192.168.3.22] ([72.219.207.160]) by eastrmimpo210 with cox id coQ11n0013UAdxU01oQ1YB; Wed, 12 Mar 2014 20:24:01 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020207.5320FAA1.015F,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=aZC/a2Ut c=1 sm=1 a=o76Gp7hjYHlWfsOj36Wk/Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=IkcTkHD0fZMA:10 a=kviXuzpPAAAA:8 a=danhDmx_AAAA:8 a=XJCMxI6UAAAA:8 a=6GvH2X3pQOj0P9pxOeEA:9 a=QEXdDO2ut3YA:10 a=V_1K7DYXpSZz2i0W:21 a=cMBHtJfcAmDyxZiW:21 a=o76Gp7hjYHlWfsOj36Wk/Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <5320FAB4.6040909@cox.net> Date: Wed, 12 Mar 2014 19:24:20 -0500 From: "John D. Hendrickson and Sara Darnell" User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: "Paul \"LeoNerd\" Evans" Subject: Re: terminfo References: <5304A0CC.5000505@FreeBSD.org> <20140223115939.GB4084@aerie.jexium-island.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: johnandsara2@cox.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Mar 2014 01:55:12 -0000 how do PEOPLE login remotely over serial line after you INTENTIONALLY break send / receive ?? is it your job to break things that already work for others for your own personal satisfaction ? no. and since when is VT52 a default in BSD ? it isn't since when does everyone need your dependance on number of colors. they don't and may be bothered by ANY color .... Ed Schouten-3 wrote > > - Who cares about VT52 support? Those things are from the Viking Age. > > - Who cares about double-width characters? I don't know a single > > program that uses this. In the worst case, you can just use fullwidth > > CJK for certain charactersPaul "LeoNerd" Evans wrote: > Ed Schouten-3 wrote >> - Who cares about VT52 support? Those things are from the Viking Age. >> - Who cares about double-width characters? I don't know a single >> program that uses this. In the worst case, you can just use fullwidth >> CJK for certain characters. >> >> - Who cares about send/receive mode? >> >> - Who cares about 88 colour support? Just use 256 colours. >> >> - Who cares about ACS? Unicode already has those characters. > > I'm with you on all those points. > > > Ed Schouten-3 wrote >> There are so many legacy features out there, that nobody actually >> *knows* how terminals should behave anymore. Also combined with the >> fact that the de facto reference implementation of terminal emulation >> (i.e., xterm) cannot easily be decoupled from X11, people just make >> stuff up. >> >> People are nowadays only interested in having a 16 or 256 colour, >> UTF-8 enabled terminal. >> >> It would be so incredibly easy for you as the maintainer of terminfo >> to say: "Here's an ideal terminfo entry. It's compact, easy to >> implement, etc. This is how I want terminals to behave. And this menu >> in vttest can be used to easily test conformance." Then you just file >> bugs against the authors of commonly used terminal emulators and 3-5 >> years later, the ecosystem has converged. Done. >> >> As I mentioned before, my spare time is limited nowadays, but if you >> would be interested in doing something like that, I would be more than >> interested to get this sorted out on FreeBSD's side. > > I believe that may be where I come in. > > Two of my larger current projects are a terminal emulator, and a terminal > emulation library; two parts of this particular issue. > > libvterm is a totally-abstracted library in pure C99 that aims to entirely > emulate a VTxxx/xterm and similar terminals. > https://launchpad.net/libvterm/ > > This isn't entirely finished yet but already it understands the vast > majority of sequences actually found in the wild. The list of sequences it > currently understands (as compared VT1xx, 2xx, etc..) can be found at > > http://bazaar.launchpad.net/~leonerd/libvterm/trunk/view/head:/doc/seqs.txt > (a document not entirely unlike xterm's ctlseqs ;) ) > > Of particular interest to FreeBSD may be the fact that libvterm abstracts as > much as possible out to the embedding program; all byte IO, keyboard/mouse > interaction, and drawing operations. It even optionally abstracts out malloc > itself, and makes special care not to call the allocator function on the > "critical" path of normal drawing; this is only used for initialisation and > resize. This feature I intended specially for use in things like UNIX > kernels, as sometimes they have critical memory requirements. > > ((This library also drives, among others, pangoterm, a GTK-based terminal I > wrote around it as proof-of-concept. This terminal has been my one terminal > I've actually used for both personal and work use, for the past two years: > https://launchpad.net/pangoterm > But to be clear - all the clever "understanding terminal stuff" happens in > the abstracted libvterm library; the actual GTK program is a tiny wrapper > that basically just provides "put these letters here in these > colours/attributes" functions.)) > > I've also been working on the "terminal UI applications" end of the chain, > by writing a set of libraries that programs can use for driving a terminal, > which more completely understand these modern features of terminals, such as > provided by libvterm. > http://www.leonerd.org.uk/code/libtickit/ > > This aims to provide support at a number of increasingly-abstracted levels, > though right now only the lowermost (the raw "terminal" level) is available > in the C library. The remaining (the "window" and "widget" levels) are > currently implemented in the Perl module of the same name (Tickit on CPAN), > where I'm developing it, slowly migrating code as I go into the C library. > > Inbetween the two I have been trying my best to both follow and implement > "current" terminal technology, as exemplified by xterm and compatible, and > also to extend it where it makes sense - for example, my definition of CSI u > to allow terminals to encode arbitrary modified Unicode keypresses, and thus > to provide a solution to cases like Ctrl-Shift-A and Ctrl-I as distinct from > Tab. > http://www.leonerd.org.uk/hacks/fixterms/ > > > *deep breath* > > Sorry if the-above comes across as somewhat of a rếsumé-style advert or > shameless self-promotion. I was pointed at this thread because of a > terminal-based discussion in Freenode's #vim, where I'm usually the person > to field any kind of terminal-related question, often with reference to one > or more of my projects above. I got to this point in the message thread and > felt I should reply and speak up, if only to say "Hey, so yes I am trying to > be this person who cares about modern terminal technology and wishes to see > it continue to expand and grow into the 21st century". > > So if I can be of any assistance with these issues, I'd be happy to take any > questions or comments. > From owner-freebsd-arch@FreeBSD.ORG Fri Mar 14 14:20:28 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7142B1EA for ; Fri, 14 Mar 2014 14:20:28 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A12D1000 for ; Fri, 14 Mar 2014 14:20:28 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2EEKSgn017600 for ; Fri, 14 Mar 2014 14:20:28 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2EEKSQ7017599 for freebsd-arch@FreeBSD.org; Fri, 14 Mar 2014 14:20:28 GMT (envelope-from bdrewery) Received: (qmail 11477 invoked from network); 14 Mar 2014 09:20:22 -0500 Received: from unknown (HELO roundcube.xk42.net) (10.10.5.5) by sweb.xzibition.com with SMTP; 14 Mar 2014 09:20:22 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 14 Mar 2014 09:20:22 -0500 From: Bryan Drewery To: freebsd-arch@FreeBSD.org Subject: Renaming cnt to =?UTF-8?Q?vm=5Fcnt?= Organization: FreeBSD Message-ID: <1627768f8d03e34c68dbe72038033a2d@shatow.net> X-Sender: bdrewery@FreeBSD.org User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 14:20:28 -0000 Hi, http://people.freebsd.org/~bdrewery/patches/cnt_vm_cnt.diff I would like to rename the vmmeter.h global cnt to vm_cnt. It is quite a generic variable to be polluting the global namespace. The variable is only defined in _KERNEL, so there is no userland/KBI risk. We've made this same change at Isilon as well. The only downside I see is that it will cause conflicts when merging from project branches and pending patches. It also will essentially need to be redone for MFC with a direct commit. I don't think any conflicts would be painful though. Is anyone strongly against such a change? I generated with: sed -i '' -e 's,[[:<:]]cnt[[:>:]]\.v_,vm_cnt.v_,g' $(git grep -l "[[:<:]]cnt[[:>:]]\.") I have not yet ran make universe to check if it is sufficient/proper, but will before commit of course. -- Regards, Bryan Drewery From owner-freebsd-arch@FreeBSD.ORG Fri Mar 14 14:24:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C82C82EF for ; Fri, 14 Mar 2014 14:24:23 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AFF4213A for ; Fri, 14 Mar 2014 14:24:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s2EEONxU017680 for ; Fri, 14 Mar 2014 14:24:23 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s2EEONR4017679 for freebsd-arch@freebsd.org; Fri, 14 Mar 2014 14:24:23 GMT (envelope-from bdrewery) Received: (qmail 54103 invoked from network); 14 Mar 2014 09:24:21 -0500 Received: from unknown (HELO roundcube.xk42.net) (10.10.5.5) by sweb.xzibition.com with SMTP; 14 Mar 2014 09:24:21 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 14 Mar 2014 09:24:21 -0500 From: Bryan Drewery To: freebsd-arch@freebsd.org Subject: Re: Renaming cnt to =?UTF-8?Q?vm=5Fcnt?= Organization: FreeBSD In-Reply-To: <1627768f8d03e34c68dbe72038033a2d@shatow.net> References: <1627768f8d03e34c68dbe72038033a2d@shatow.net> Message-ID: X-Sender: bdrewery@FreeBSD.org User-Agent: Roundcube Webmail/0.9.5 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 14:24:23 -0000 On 2014-03-14 09:20, Bryan Drewery wrote: > Hi, > > http://people.freebsd.org/~bdrewery/patches/cnt_vm_cnt.diff > > I would like to rename the vmmeter.h global cnt to vm_cnt. > It is quite a generic variable to be polluting the global > namespace. The variable is only defined in _KERNEL, so > there is no userland/KBI risk. I take that back. I will do an exp-run at least to ensure no ports with modules need fixing. > We've made this same > change at Isilon as well. > > The only downside I see is that it will cause conflicts > when merging from project branches and pending patches. > It also will essentially need to be redone for MFC > with a direct commit. > > I don't think any conflicts would be painful though. > > Is anyone strongly against such a change? > > I generated with: sed -i '' -e 's,[[:<:]]cnt[[:>:]]\.v_,vm_cnt.v_,g' > $(git grep -l "[[:<:]]cnt[[:>:]]\.") > I have not yet ran make universe to check if it is > sufficient/proper, but will before commit of course. -- Regards, Bryan Drewery From owner-freebsd-arch@FreeBSD.ORG Fri Mar 14 17:14:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D428CEB for ; Fri, 14 Mar 2014 17:14:15 +0000 (UTC) Received: from mail-ve0-x231.google.com (mail-ve0-x231.google.com [IPv6:2607:f8b0:400c:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C80B6669 for ; Fri, 14 Mar 2014 17:14:14 +0000 (UTC) Received: by mail-ve0-f177.google.com with SMTP id sa20so2966502veb.36 for ; Fri, 14 Mar 2014 10:14:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Q7cNBDXOXMcnanIQl7owX2i38cU5FJlRSLpGqrjQ3BI=; b=CGLsjHz2WP3uebNPG6UHZNUX2nxwIkWFY9DaGdijMk/ALoW2JdFnzccdL2rkDwlWRo VCtsOzCWzGBo0qM1RqPN05ttq8kEcKZkiekIyzv+rfGomQu5rzrC17nyVZleYTTiTmDj GuvjV1UndY2+JvQl20PLyUzzfgQzSczNIMASE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=Q7cNBDXOXMcnanIQl7owX2i38cU5FJlRSLpGqrjQ3BI=; b=adYdTn4IA4/uj1ChWmUlPqmVCRYZYxvst5zWq49k6e/JK3cNrhxu01E/M3M17clQB5 8UfTQa18qjX6gROMvhNfVQq6hXX+mTOR4l5A8Iy0iQ1ovHyfD8meexpL0MQCDinEUXdM svHr3P8UYaJR0BUxXSaI/jIIdbTnmChw4S4lM7P04ndssh1/2wcLq6sP+UIHtz4WvV/N ZgCd+NsydT38EFhlyMpDN+lI/1u68H4gvr1ac0zCYVzHv0EYM2Ktcia1nxyMYV+pb07U F2VTConiUGuAajNrqmbWYAjbCEPCvzf56pHCDI/C/h+Mh+1KoTlnvv+1+xl64gN2Xod+ lsJQ== X-Gm-Message-State: ALoCoQkO9GCJW8hbjqbohq8eq7uUnLiG1oODjlia0O/kSVwi1zafwE++/QjzxdCgNzVKhsNZFibX MIME-Version: 1.0 X-Received: by 10.220.99.72 with SMTP id t8mr7299464vcn.10.1394817253854; Fri, 14 Mar 2014 10:14:13 -0700 (PDT) Received: by 10.220.30.69 with HTTP; Fri, 14 Mar 2014 10:14:13 -0700 (PDT) In-Reply-To: References: <1627768f8d03e34c68dbe72038033a2d@shatow.net> Date: Fri, 14 Mar 2014 10:14:13 -0700 Message-ID: Subject: Re: Renaming cnt to vm_cnt From: Peter Wemm To: Bryan Drewery Content-Type: text/plain; charset=ISO-8859-1 Cc: FreeBSD Arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Mar 2014 17:14:15 -0000 On Fri, Mar 14, 2014 at 7:24 AM, Bryan Drewery wrote: > On 2014-03-14 09:20, Bryan Drewery wrote: >> >> Hi, >> >> http://people.freebsd.org/~bdrewery/patches/cnt_vm_cnt.diff >> >> I would like to rename the vmmeter.h global cnt to vm_cnt. >> It is quite a generic variable to be polluting the global >> namespace. The variable is only defined in _KERNEL, so >> there is no userland/KBI risk. > > > I take that back. I will do an exp-run at least to ensure no ports with > modules need fixing. It's more than just modules - things like lsof, net-snmp etc are likely candidates to check too. Anything that uses libkvm or /dev/kmem. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV Yes, I know, gmail sucks now. If you see this then I forgot. Habits are hard to break. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 15 14:28:25 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5ED49531 for ; Sat, 15 Mar 2014 14:28:25 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 31A19E22 for ; Sat, 15 Mar 2014 14:28:24 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WOpZS-000IDm-9g for freebsd-arch@FreeBSD.org; Sat, 15 Mar 2014 14:28:18 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s2FESFIg065291 for ; Sat, 15 Mar 2014 08:28:15 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18ZirkdTp3ZQcjrcVMPMFaN Subject: Per-arch CFLAGS From: Ian Lepore To: freebsd-arch Content-Type: multipart/mixed; boundary="=-rfFM8btMGEB7fOLA+pEa" Date: Sat, 15 Mar 2014 08:28:15 -0600 Message-ID: <1394893695.1149.553.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 14:28:25 -0000 --=-rfFM8btMGEB7fOLA+pEa Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit I've run into a situation where I need to pass architecture specific CFLAGS to the world stage of a cross-build but not have those flags in effect for the build-tools and cross-tools that run on the build host. I tried several things based on $TARGET_ARCH and manipulating the environment in Makefile.inc1 but everything I tried either didn't work or had unhappy side effects such as overriding automatically-supplied internal flags with the user-supplied flags, usually resulting in build failure because some crucial CPUTYPE stuff would be missing. After consulting with Warner a bit I came up with the attached rather simple change that does the job perfectly. It adds support for a CFLAGS.arch variable that can be set in make.conf or the command line or environment. Any objections to this? -- Ian --=-rfFM8btMGEB7fOLA+pEa Content-Disposition: inline; filename="cflags.arch.diff" Content-Type: text/x-patch; name="cflags.arch.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: share/mk/bsd.cpu.mk =================================================================== --- share/mk/bsd.cpu.mk (revision 263112) +++ share/mk/bsd.cpu.mk (working copy) @@ -260,3 +260,7 @@ CFLAGS += -G0 .if !defined(NO_CPU_CFLAGS) CFLAGS += ${_CPUCFLAGS} .endif + +# Add in any architecture-specific CFLAGS. +# These come from make.conf or the command line or the environment. +CFLAGS += ${CFLAGS.${MACHINE_ARCH}} Index: share/examples/etc/make.conf =================================================================== --- share/examples/etc/make.conf (revision 263112) +++ share/examples/etc/make.conf (working copy) @@ -60,6 +60,12 @@ # nonstandard optimization settings # before submitting bug reports without patches to the developers. # +# CFLAGS.arch provides a mechanism for applying CFLAGS only when building +# the given architecture. This is useful primarily on a system used for +# cross-building, when you have a set of flags to apply to the TARGET_ARCH +# being cross-built but don't want those settings applied to building the +# cross-tools or other components that run on the build host machine. +# # CXXFLAGS controls the compiler settings used when compiling C++ code. # Note that CXXFLAGS is initially set to the value of CFLAGS. If you wish # to add to CXXFLAGS value, "+=" must be used rather than "=". Using "=" @@ -71,6 +77,7 @@ # # CFLAGS+= -msse3 # CXXFLAGS+= -msse3 +# CFLAGS.armv6+= -mfloat-abi=softfp # # MAKE_SHELL controls the shell used internally by make(1) to process the # command scripts in makefiles. Three shells are supported, sh, ksh, and --=-rfFM8btMGEB7fOLA+pEa-- From owner-freebsd-arch@FreeBSD.ORG Sat Mar 15 16:59:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA6A7820 for ; Sat, 15 Mar 2014 16:59:32 +0000 (UTC) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 85B14D4A for ; Sat, 15 Mar 2014 16:59:32 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id lx4so3880224iec.10 for ; Sat, 15 Mar 2014 09:59:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=kZ76/iE/I3hjksKskQDX7bDR7usKPDj3wvBQUy1wyL8=; b=MRExCIENZ5A8OQoRoPL7GvMM+M176v4WpnM5Lhx+QnJKlCXUVk/Ud91H9W0IIP6s8F Tw/AaBHYvt0ZTwvpMQIjHWhV5BTJ4uNDSBGR1VZGJdbU9udE/xZ3R5BLtSjYhoGsKDsk ouHzIgCNxgK7gnsHk9wDuDJ2vEhH5zjCXdtvsL4RPA3k8pIhxwbeMw7DKfFF6+WKXmsI CTPeBLFUh7jc2DQYXWCAv6SOmpJTAJvfI7CRflGHO1FDRlr1CcNvxBKwPn66v0B9iuP7 krDT8/4+Y/E2ecqTl672M/lHYdTXBKHPxvZkHN8w2tKgyRBNFkTtmQyy3aC1D4f7ihnO 0kZg== X-Gm-Message-State: ALoCoQlDpXbd6wD16pBadzR1ZaUubitgBSS900BJjQtX3T5MY7yREfngazOfQVOdfpm9WJLmgsfe X-Received: by 10.42.62.196 with SMTP id z4mr2343761ich.49.1394902765753; Sat, 15 Mar 2014 09:59:25 -0700 (PDT) Received: from netflix-mac.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id kb5sm7186252igb.1.2014.03.15.09.59.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 15 Mar 2014 09:59:25 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Per-arch CFLAGS From: Warner Losh In-Reply-To: <1394893695.1149.553.camel@revolution.hippie.lan> Date: Sat, 15 Mar 2014 10:59:23 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <0EBF508F-6504-414A-A2A0-39C5C2EEA6D0@bsdimp.com> References: <1394893695.1149.553.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 16:59:32 -0000 On Mar 15, 2014, at 8:28 AM, Ian Lepore wrote: > I've run into a situation where I need to pass architecture specific > CFLAGS to the world stage of a cross-build but not have those flags in > effect for the build-tools and cross-tools that run on the build host.=20= >=20 > I tried several things based on $TARGET_ARCH and manipulating the > environment in Makefile.inc1 but everything I tried either didn't work > or had unhappy side effects such as overriding automatically-supplied > internal flags with the user-supplied flags, usually resulting in = build > failure because some crucial CPUTYPE stuff would be missing. >=20 > After consulting with Warner a bit I came up with the attached rather > simple change that does the job perfectly. It adds support for a > CFLAGS.arch variable that can be set in make.conf or the command line = or > environment. Any objections to this? This looks good to me. I like it better than earlier patches, and = includes the TARGET_ARCH->MACHINE_ARCH change I was going to suggest after thinking about our last conversation. Warner > -- Ian >=20 > Index: share/mk/bsd.cpu.mk > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- share/mk/bsd.cpu.mk (revision 263112) > +++ share/mk/bsd.cpu.mk (working copy) > @@ -260,3 +260,7 @@ CFLAGS +=3D -G0 > .if !defined(NO_CPU_CFLAGS) > CFLAGS +=3D ${_CPUCFLAGS} > .endif > + > +# Add in any architecture-specific CFLAGS. =20 > +# These come from make.conf or the command line or the environment. > +CFLAGS +=3D ${CFLAGS.${MACHINE_ARCH}} > Index: share/examples/etc/make.conf > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- share/examples/etc/make.conf (revision 263112) > +++ share/examples/etc/make.conf (working copy) > @@ -60,6 +60,12 @@ > # nonstandard optimization settings > # before submitting bug reports without patches to the developers. > # > +# CFLAGS.arch provides a mechanism for applying CFLAGS only when = building=20 > +# the given architecture. This is useful primarily on a system used = for=20 > +# cross-building, when you have a set of flags to apply to the = TARGET_ARCH=20 > +# being cross-built but don't want those settings applied to building = the=20 > +# cross-tools or other components that run on the build host machine. = =20 > +# > # CXXFLAGS controls the compiler settings used when compiling C++ = code. > # Note that CXXFLAGS is initially set to the value of CFLAGS. If you = wish > # to add to CXXFLAGS value, "+=3D" must be used rather than "=3D". = Using "=3D" > @@ -71,6 +77,7 @@ > # > # CFLAGS+=3D -msse3 > # CXXFLAGS+=3D -msse3 > +# CFLAGS.armv6+=3D -mfloat-abi=3Dsoftfp > # > # MAKE_SHELL controls the shell used internally by make(1) to process = the > # command scripts in makefiles. Three shells are supported, sh, ksh, = and > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Mar 15 23:53:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23D1CFEE for ; Sat, 15 Mar 2014 23:53:12 +0000 (UTC) Received: from eastrmfepo103.cox.net (eastrmfepo103.cox.net [68.230.241.215]) by mx1.freebsd.org (Postfix) with ESMTP id C153A313 for ; Sat, 15 Mar 2014 23:53:11 +0000 (UTC) Received: from eastrmimpo209 ([68.230.241.224]) by eastrmfepo103.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140315235305.VFLI28095.eastrmfepo103.cox.net@eastrmimpo209> for ; Sat, 15 Mar 2014 19:53:05 -0400 Received: from [192.168.3.22] ([72.219.207.160]) by eastrmimpo209 with cox id dzt41n00E3UAdxU01zt4PU; Sat, 15 Mar 2014 19:53:05 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020208.5324E7E1.002D,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=H/cFNZki c=1 sm=1 a=o76Gp7hjYHlWfsOj36Wk/Q==:17 a=f5xKl4ys9bwA:10 a=LGN1LpB6GecA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=8nJEP1OIZ-IA:10 a=kviXuzpPAAAA:8 a=FP58Ms26AAAA:8 a=Fu2VrRXrEpl2uIX9_sAA:9 a=wPNLvfGTeEIA:10 a=YkDor_TC5ScA:10 a=o76Gp7hjYHlWfsOj36Wk/Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <5324E7E5.1070207@cox.net> Date: Sat, 15 Mar 2014 18:53:09 -0500 From: "John D. Hendrickson and Sara Darnell" User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: freebsd-arch Subject: Re: user quest. is BSD 4.2 ok to install ? References: <1394893695.1149.553.camel@revolution.hippie.lan> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: johnandsara2@cox.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2014 23:53:12 -0000 user question. could somebody reply to my email? is BSD 4.2 ok to install ? i've never installed BSD i did read about portage. i wrote ** i have an older pc and want to stay away from a huge number of new problems / breakage of old apps / new hw i can't get. something stable, quick(r) and simple ie, i want a stable awk and an older web browser that sucks is just fine i also don't want to kick my own butt. what's a good release to start would you vote ? ----------------- * http://sourceforge.net/projects/dep-trace/ * which can quickly and fully solve total (end-to-end) depends for bin or source (does for debian, can for BSD) From owner-freebsd-arch@FreeBSD.ORG Sun Mar 16 00:52:09 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2D5D817 for ; Sun, 16 Mar 2014 00:52:09 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id 63A9F93C for ; Sun, 16 Mar 2014 00:52:09 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id s2G0nqar092760; Sat, 15 Mar 2014 19:49:52 -0500 (CDT) (envelope-from mike@karels.net) Message-Id: <201403160049.s2G0nqar092760@mail.karels.net> To: johnandsara2@cox.net From: Mike Karels Subject: Re: user quest. is BSD 4.2 ok to install ? In-reply-to: Your message of Sat, 15 Mar 2014 18:53:09 -0500. <5324E7E5.1070207@cox.net> Date: Sat, 15 Mar 2014 19:49:52 -0500 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: mike@karels.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 00:52:09 -0000 > user question. could somebody reply to my email? > is BSD 4.2 ok to install ? 4.2BSD will run fine on a VAX 11/780, 750 or 730, or maybe a 785. It does not run on a PC. Or did you mean something else? > i've never installed BSD i did read about portage. i wrote ** > i have an older pc and want to stay away from a huge number of new > problems / breakage of old apps / new hw i can't get. something > stable, quick(r) and simple > ie, i want a stable awk and an older web browser that sucks is just fine 4.2BSD predates the web by quite a few years, and doesn't have a graphical user interface. It has awk, though. It is certainly simple by today's standards. > i also don't want to kick my own butt. what's a good release to start > would you vote ? For a PC: FreeBSD 9.2, given your concerns. Mike From owner-freebsd-arch@FreeBSD.ORG Sun Mar 16 06:52:46 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76EBDBCD; Sun, 16 Mar 2014 06:52:46 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe002.messaging.microsoft.com [216.32.181.182]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2E7E17D5; Sun, 16 Mar 2014 06:52:45 +0000 (UTC) Received: from mail207-ch1-R.bigfish.com (10.43.68.245) by CH1EHSOBE007.bigfish.com (10.43.70.57) with Microsoft SMTP Server id 14.1.225.22; Sun, 16 Mar 2014 06:52:38 +0000 Received: from mail207-ch1 (localhost [127.0.0.1]) by mail207-ch1-R.bigfish.com (Postfix) with ESMTP id B83301201DD; Sun, 16 Mar 2014 06:52:38 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kzz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24d7h2516h2545h255eh25cch25f6h2605h1155h) Received-SPF: softfail (mail207-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail207-ch1 (localhost.localdomain [127.0.0.1]) by mail207-ch1 (MessageSwitch) id 139495275749407_25411; Sun, 16 Mar 2014 06:52:37 +0000 (UTC) Received: from CH1EHSMHS014.bigfish.com (snatpool3.int.messaging.microsoft.com [10.43.68.227]) by mail207-ch1.bigfish.com (Postfix) with ESMTP id 0751C440055; Sun, 16 Mar 2014 06:52:37 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS014.bigfish.com (10.43.70.14) with Microsoft SMTP Server (TLS) id 14.16.227.3; Sun, 16 Mar 2014 06:52:36 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Sat, 15 Mar 2014 23:52:35 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s2G6qYV32531; Sat, 15 Mar 2014 23:52:35 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id AB42D58097; Sat, 15 Mar 2014 23:52:34 -0700 (PDT) To: Ian Lepore Subject: Re: Per-arch CFLAGS In-Reply-To: <1394893695.1149.553.camel@revolution.hippie.lan> References: <1394893695.1149.553.camel@revolution.hippie.lan> Comments: In-reply-to: Ian Lepore message dated "Sat, 15 Mar 2014 08:28:15 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Sat, 15 Mar 2014 23:52:34 -0700 Message-ID: <20140316065234.AB42D58097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Mar 2014 06:52:46 -0000 On Sat, 15 Mar 2014 08:28:15 -0600, Ian Lepore writes: >After consulting with Warner a bit I came up with the attached rather >simple change that does the job perfectly. It adds support for a >CFLAGS.arch variable that can be set in make.conf or the command line or >environment. Any objections to this? No, looks fine. FWIW we do much the same in the Junos build, but ${MACHINE} rather than ${MACHINE_ARCH} both have their uses in fact. Esp since we use the pseudo machine "host" to represent the build host (for host tools etc). From owner-freebsd-arch@FreeBSD.ORG Mon Mar 17 23:23:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18E36F45; Mon, 17 Mar 2014 23:23:21 +0000 (UTC) Received: from mail-ie0-x230.google.com (mail-ie0-x230.google.com [IPv6:2607:f8b0:4001:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B463B8E0; Mon, 17 Mar 2014 23:23:20 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id rd18so6281243iec.35 for ; Mon, 17 Mar 2014 16:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=7zOEl8FKXdd3ckNwS10UOh1ra6e6OLytoYvYrSr2cxI=; b=xrI/3dP0jQR8Dz3x1Q5xU2cKKpWhhn+WNjpYpjCa+/DoQSYTGh3g1S8ru65JOzLsmJ P02WMarAlaYdUCteOlOabEsVFQQY+GHPpS2xdN4Mc/aNxzRQ4FEGE/3TXySNurorPoDX uhQOMxt+ohWkEMnbNApVwOMX+qWVcxMbmjfp6KR6KrsuqJW3v5MxOoj0wxW1EfJgw3G/ 65djdsaCogSEUakDEwgrBQU2DWugmmE5N71eqzbpcs+i0pP1Fed0p8xU33OFPnKiV5ES 4rhYM4AKbt/zohd/EfW8tVIzTd5QYGrlVcME+g9HlMRtAYIGAJoPJ2ISi16APsIOZPbx Qdzg== MIME-Version: 1.0 X-Received: by 10.42.27.136 with SMTP id j8mr3307491icc.69.1395098600169; Mon, 17 Mar 2014 16:23:20 -0700 (PDT) Sender: oshogbo.vx@gmail.com Received: by 10.50.95.34 with HTTP; Mon, 17 Mar 2014 16:23:19 -0700 (PDT) Date: Tue, 18 Mar 2014 00:23:19 +0100 X-Google-Sender-Auth: rlxw3VDwpqrpJhTfI433aU1h014 Message-ID: Subject: Hello fdclose From: Mariusz Zaborski To: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Content-Type: multipart/mixed; boundary=20cf3042705a7d7ed004f4d5b4f2 Cc: jilles@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Mar 2014 23:23:21 -0000 --20cf3042705a7d7ed004f4d5b4f2 Content-Type: text/plain; charset=ISO-8859-1 Hi, After our previous discuss [1] I prepare fdclosedir(3) function which was committed by Pawel (cc'ed) in commit r254499. A while ago I also prepare the fdclose function. Unfortunately, this new function is a little bit more tricky then previous one. Can I ask you for a review of this patch? Thanks, Mariusz [1] http://lists.freebsd.org/pipermail/freebsd-arch/2013-August/014688.html --20cf3042705a7d7ed004f4d5b4f2 Content-Type: text/x-patch; charset=US-ASCII; name="fdclose.patch" Content-Disposition: attachment; filename="fdclose.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hswdobnv0 LS0tIC8vZGVwb3QvdXNlci9vc2hvZ2JvL2NhcHNpY3VtL2luY2x1ZGUvc3RkaW8uaAkyMDEzLTA2 LTI4IDA4OjUxOjI4LjAwMDAwMDAwMCAwMDAwCisrKyAvaG9tZS9vc2hvZ2JvL3A0L2NhcHNpY3Vt L2luY2x1ZGUvc3RkaW8uaAkyMDEzLTA2LTI4IDA4OjUxOjI4LjAwMDAwMDAwMCAwMDAwCkBAIC0z OTYsNiArMzk2LDcgQEAKIGludAkgYXNwcmludGYoY2hhciAqKiwgY29uc3QgY2hhciAqLCAuLi4p IF9fcHJpbnRmbGlrZSgyLCAzKTsKIGNoYXIJKmN0ZXJtaWRfcihjaGFyICopOwogdm9pZAkgZmNs b3NlYWxsKHZvaWQpOworaW50CSBmZGNsb3NlKEZJTEUgKik7CiBjaGFyCSpmZ2V0bG4oRklMRSAq LCBzaXplX3QgKik7CiBjb25zdCBjaGFyICpmbXRjaGVjayhjb25zdCBjaGFyICosIGNvbnN0IGNo YXIgKikgX19mb3JtYXRfYXJnKDIpOwogaW50CSBmcHVyZ2UoRklMRSAqKTsKLS0tIC8vZGVwb3Qv dXNlci9vc2hvZ2JvL2NhcHNpY3VtL2xpYi9saWJjL3N0ZGlvL1N5bWJvbC5tYXAJMjAxMy0wNi0y OCAwODo1MToyOC4wMDAwMDAwMDAgMDAwMAorKysgL2hvbWUvb3Nob2diby9wNC9jYXBzaWN1bS9s aWIvbGliYy9zdGRpby9TeW1ib2wubWFwCTIwMTMtMDYtMjggMDg6NTE6MjguMDAwMDAwMDAwIDAw MDAKQEAgLTE1Niw2ICsxNTYsNyBAQAogCXB1dHdjX2w7CiAJcHV0d2NoYXJfbDsKIAlmbWVtb3Bl bjsKKwlmZGNsb3NlOwogCW9wZW5fbWVtc3RyZWFtOwogCW9wZW5fd21lbXN0cmVhbTsKIH07Ci0t LSAvL2RlcG90L3VzZXIvb3Nob2diby9jYXBzaWN1bS9saWIvbGliYy9zdGRpby9mY2xvc2UuMwky MDEzLTA2LTI4IDA4OjUxOjI4LjAwMDAwMDAwMCAwMDAwCisrKyAvaG9tZS9vc2hvZ2JvL3A0L2Nh cHNpY3VtL2xpYi9saWJjL3N0ZGlvL2ZjbG9zZS4zCTIwMTMtMDYtMjggMDg6NTE6MjguMDAwMDAw MDAwIDAwMDAKQEAgLTEsNSArMSw2IEBACi0uXCIgQ29weXJpZ2h0IChjKSAxOTkwLCAxOTkxLCAx OTkzCi0uXCIJVGhlIFJlZ2VudHMgb2YgdGhlIFVuaXZlcnNpdHkgb2YgQ2FsaWZvcm5pYS4gIEFs bCByaWdodHMgcmVzZXJ2ZWQuCisuXCIgQ29weXJpZ2h0IChjKSAxOTkwLCAxOTkxLCAxOTkzIFRo ZSBSZWdlbnRzIG9mIHRoZSBVbml2ZXJzaXR5IG9mIENhbGlmb3JuaWEuCisuXCIgQ29weXJpZ2h0 IChjKSAyMDE0IE1hcml1c3ogWmFib3Jza2kgPG9zaG9nYm9ARnJlZUJTRC5vcmc+CisuXCIgQWxs IHJpZ2h0cyByZXNlcnZlZC4KIC5cIgogLlwiIFRoaXMgY29kZSBpcyBkZXJpdmVkIGZyb20gc29m dHdhcmUgY29udHJpYnV0ZWQgdG8gQmVya2VsZXkgYnkKIC5cIiBDaHJpcyBUb3JlayBhbmQgdGhl IEFtZXJpY2FuIE5hdGlvbmFsIFN0YW5kYXJkcyBDb21taXR0ZWUgWDMsCkBAIC0zMiwxMSArMzMs MTIgQEAKIC5cIiAgICAgQCgjKWZjbG9zZS4zCTguMSAoQmVya2VsZXkpIDYvNC85MwogLlwiICRG cmVlQlNEOiBoZWFkL2xpYi9saWJjL3N0ZGlvL2ZjbG9zZS4zIDE2NTkwMyAyMDA3LTAxLTA5IDAw OjI4OjE2WiBpbXAgJAogLlwiCi0uRGQgQXByaWwgMjIsIDIwMDYKKy5EZCBNYXJjaCAxNywgMjAx NAogLkR0IEZDTE9TRSAzCiAuT3MKIC5TaCBOQU1FCiAuTm0gZmNsb3NlICwKKy5ObSBmZGNsb3Nl ICwKIC5ObSBmY2xvc2VhbGwKIC5OZCBjbG9zZSBhIHN0cmVhbQogLlNoIExJQlJBUlkKQEAgLTQ1 LDYgKzQ3LDggQEAKIC5JbiBzdGRpby5oCiAuRnQgaW50CiAuRm4gZmNsb3NlICJGSUxFICpzdHJl YW0iCisuRnQgaW50CisuRm4gZmRjbG9zZSAiRklMRSAqc3RyZWFtIgogLkZ0IHZvaWQKIC5GbiBm Y2xvc2VhbGwgdm9pZAogLlNoIERFU0NSSVBUSU9OCkBAIC01OSwyMiArNjMsNjQgQEAKIC5YciBm Zmx1c2ggMyAuCiAuUHAKIFRoZQorLkZuIGZkY2xvc2UKK2Z1bmN0aW9uIGlzIGVxdWl2YWxlbnQg dG8gdGhlCisuRm4gZmNsb3NlCitmdW5jdGlvbiBleGNlcHQgdGhhdCB0aGlzIGZ1bmN0aW9uIHJl dHVybnMgZmlsZSBkZXNjcmlwdG9yIGluc3RlYWQgb2YKK2Nsb3NpbmcgaXQuCisuUHAKK1RoZQog LkZuIGZjbG9zZWFsbAogZnVuY3Rpb24gY2FsbHMKIC5GbiBmY2xvc2UKIG9uIGFsbCBvcGVuIHN0 cmVhbXMuCiAuU2ggUkVUVVJOIFZBTFVFUwotVXBvbiBzdWNjZXNzZnVsIGNvbXBsZXRpb24gMCBp cyByZXR1cm5lZC4KK1RoZQorLkZuIGZjbG9zZWFsbAorZnVuY3Rpb24gcmV0dXJuIG5vIHZhbHVl LgorLlBwCitVcG9uIHN1Y2Nlc3NmdWwgY29tcGxldGlvbgorLkZuIGZjbG9zZQorcmV0dXJuIDAu CitPdGhlcndpc2UsCisuRHYgRU9GCitpcyByZXR1cm5lZCBhbmQgdGhlIGdsb2JhbCB2YXJpYWJs ZQorLlZhIGVycm5vCitpcyBzZXQgdG8gaW5kaWNhdGUgdGhlIGVycm9yLgorLlBwCitUaGUKKy5G biBmZGNsb3NlCitmdW5jdGlvbiByZXR1cm4gdGhlIGZpbGUgZGVzY3JpcHRvciBpZiBzdWNjZXNz ZnVsbC4KIE90aGVyd2lzZSwKIC5EdiBFT0YKIGlzIHJldHVybmVkIGFuZCB0aGUgZ2xvYmFsIHZh cmlhYmxlCiAuVmEgZXJybm8KIGlzIHNldCB0byBpbmRpY2F0ZSB0aGUgZXJyb3IuCisuUHAKIElu IGVpdGhlciBjYXNlIG5vIGZ1cnRoZXIgYWNjZXNzIHRvIHRoZSBzdHJlYW0gaXMgcG9zc2libGUu CiAuU2ggRVJST1JTCisuQmwgLXRhZyAtd2lkdGggRXIKKy5JdCBCcSBFciBFT1BOT1RTVVBQCiBU aGUKKy5GYSBfY2xvc2UKK21ldGhvZCBpbgorLkZhIHN0cmVhbQorYXJndW1lbnQgdG8KKy5GbiBm ZGNsb3NlICwKK3dhcyBub3QgZGVmYXVsdC4KKy5JdCBCcSBFciBFQkFERgorVGhlCisuRmEgc3Ry ZWFtCithcmd1bWVudCB0bworLkZuIGZkY2xvc2UgLAorZG9lcyBub3QgY29udGFpbnMgdmFsaWQg ZmlsZSBkZXNjcmlwdG9yLgorLkVsCisuUHAKK1RoZQogLkZuIGZjbG9zZQotZnVuY3Rpb24KK2Fu ZAorLkZuIGZkY2xvc2UKK2Z1bmN0aW9ucwogbWF5IGFsc28gZmFpbCBhbmQgc2V0CiAuVmEgZXJy bm8KIGZvciBhbnkgb2YgdGhlIGVycm9ycyBzcGVjaWZpZWQgZm9yIHRoZSByb3V0aW5lcwpAQCAt ODQsNyArMTMwLDkgQEAKIC5TaCBOT1RFUwogVGhlCiAuRm4gZmNsb3NlCi1mdW5jdGlvbgorYW5k CisuRm4gZmRjbG9zZQorZnVuY3Rpb25zCiBkb2VzIG5vdCBoYW5kbGUgTlVMTCBhcmd1bWVudHM7 IHRoZXkgd2lsbCByZXN1bHQgaW4gYSBzZWdtZW50YXRpb24KIHZpb2xhdGlvbi4KIFRoaXMgaXMg aW50ZW50aW9uYWwgLSBpdCBtYWtlcyBpdCBlYXNpZXIgdG8gbWFrZSBzdXJlIHByb2dyYW1zIHdy aXR0ZW4KQEAgLTEwNCw4ICsxNTIsMTMgQEAKIGZ1bmN0aW9uCiBjb25mb3JtcyB0bwogLlN0IC1p c29DIC4KLS5QcAorLlNoIEhpc3RvcnkKIFRoZQogLkZuIGZjbG9zZWFsbAogZnVuY3Rpb24gZmly c3QgYXBwZWFyZWQgaW4KIC5GeCA3LjAgLgorLlBwCitUaGUKKy5GbiBmZGNsb3NlCitmdW5jdGlv biBmaXJzdCBhcHBlYXJlZCBpbgorLkZ4IDExLjAgLgotLS0gLy9kZXBvdC91c2VyL29zaG9nYm8v Y2Fwc2ljdW0vbGliL2xpYmMvc3RkaW8vZmNsb3NlLmMJMjAxMy0wNi0yOCAwODo1MToyOC4wMDAw MDAwMDAgMDAwMAorKysgL2hvbWUvb3Nob2diby9wNC9jYXBzaWN1bS9saWIvbGliYy9zdGRpby9m Y2xvc2UuYwkyMDEzLTA2LTI4IDA4OjUxOjI4LjAwMDAwMDAwMCAwMDAwCkBAIC0xLDYgKzEsNyBA QAogLyotCi0gKiBDb3B5cmlnaHQgKGMpIDE5OTAsIDE5OTMKLSAqCVRoZSBSZWdlbnRzIG9mIHRo ZSBVbml2ZXJzaXR5IG9mIENhbGlmb3JuaWEuICBBbGwgcmlnaHRzIHJlc2VydmVkLgorICogQ29w eXJpZ2h0IChjKSAxOTkwLCAxOTkzIFRoZSBSZWdlbnRzIG9mIHRoZSBVbml2ZXJzaXR5IG9mIENh bGlmb3JuaWEuCisgKiBDb3B5cmlnaHQgKGMpIDIwMTQgTWFyaXVzeiBaYWJvcnNraSA8b3Nob2di b0BGcmVlQlNELm9yZz4KKyAqIEFsbCByaWdodHMgcmVzZXJ2ZWQuCiAgKgogICogVGhpcyBjb2Rl IGlzIGRlcml2ZWQgZnJvbSBzb2Z0d2FyZSBjb250cmlidXRlZCB0byBCZXJrZWxleSBieQogICog Q2hyaXMgVG9yZWsuCkBAIC0zOCw2ICszOSw3IEBACiAKICNpbmNsdWRlICJuYW1lc3BhY2UuaCIK ICNpbmNsdWRlIDxlcnJuby5oPgorI2luY2x1ZGUgPHN0ZGJvb2wuaD4KICNpbmNsdWRlIDxzdGRp by5oPgogI2luY2x1ZGUgPHN0ZGxpYi5oPgogI2luY2x1ZGUgInVuLW5hbWVzcGFjZS5oIgpAQCAt NDUsMTkgKzQ3LDE3IEBACiAjaW5jbHVkZSAibGliY19wcml2YXRlLmgiCiAjaW5jbHVkZSAibG9j YWwuaCIKIAotaW50Ci1mY2xvc2UoRklMRSAqZnApCitzdGF0aWMgaW50CitjbGVhbmZpbGUoRklM RSAqZnAsIGJvb2wgYykKIHsKIAlpbnQgcjsKIAotCWlmIChmcC0+X2ZsYWdzID09IDApIHsJLyog bm90IG9wZW4hICovCi0JCWVycm5vID0gRUJBREY7Ci0JCXJldHVybiAoRU9GKTsKKwlyID0gZnAt Pl9mbGFncyAmIF9fU1dSID8gX19zZmx1c2goZnApIDogMDsKKwlpZiAoYykgeworCQlpZiAoZnAt Pl9jbG9zZSAhPSBOVUxMICYmICgqZnAtPl9jbG9zZSkoZnAtPl9jb29raWUpIDwgMCkKKwkJCXIg PSBFT0Y7CiAJfQotCUZMT0NLRklMRShmcCk7Ci0JciA9IGZwLT5fZmxhZ3MgJiBfX1NXUiA/IF9f c2ZsdXNoKGZwKSA6IDA7Ci0JaWYgKGZwLT5fY2xvc2UgIT0gTlVMTCAmJiAoKmZwLT5fY2xvc2Up KGZwLT5fY29va2llKSA8IDApCi0JCXIgPSBFT0Y7CisKIAlpZiAoZnAtPl9mbGFncyAmIF9fU01C RikKIAkJZnJlZSgoY2hhciAqKWZwLT5fYmYuX2Jhc2UpOwogCWlmIChIQVNVQihmcCkpCkBAIC04 MCw2ICs4MCw1NSBAQAogCVNURElPX1RIUkVBRF9MT0NLKCk7CiAJZnAtPl9mbGFncyA9IDA7CQkv KiBSZWxlYXNlIHRoaXMgRklMRSBmb3IgcmV1c2UuICovCiAJU1RESU9fVEhSRUFEX1VOTE9DSygp OworCisJcmV0dXJuIChyKTsKK30KKworaW50CitmZGNsb3NlKEZJTEUgKmZwKQoreworCWludCBm ZCwgciwgZXJyOworCisJaWYgKGZwLT5fZmxhZ3MgPT0gMCkgewkvKiBub3Qgb3BlbiEgKi8KKwkJ ZXJybm8gPSBFQkFERjsKKwkJcmV0dXJuIChFT0YpOworCX0KKworCXIgPSAwOworCUZMT0NLRklM RShmcCk7CisJZmQgPSBmcC0+X2ZpbGU7CisJaWYgKGZwLT5fY2xvc2UgIT0gX19zY2xvc2UpIHsK KwkJciA9IEVPRjsKKwkJZXJybm8gPSBFT1BOT1RTVVBQOworCX0gZWxzZSBpZiAoZmQgPCAwKSB7 CisJCXIgPSBFT0Y7CisJCWVycm5vID0gRUJBREY7CisJfQorCWlmIChyID09IEVPRikgeworCQll cnIgPSBlcnJubzsKKwkJKHZvaWQpY2xlYW5maWxlKGZwLCB0cnVlKTsKKwkJZXJybm8gPSBlcnI7 CisJfSBlbHNlIHsKKwkJciA9IGNsZWFuZmlsZShmcCwgZmFsc2UpOworCX0KIAlGVU5MT0NLRklM RShmcCk7CisKKwlyZXR1cm4gKHIgPT0gMCA/IGZkIDogcik7Cit9CisKK2ludAorZmNsb3NlKEZJ TEUgKmZwKQoreworCWludCByOworCisJaWYgKGZwLT5fZmxhZ3MgPT0gMCkgewkvKiBub3Qgb3Bl biEgKi8KKwkJZXJybm8gPSBFQkFERjsKKwkJcmV0dXJuIChFT0YpOworCX0KKworCUZMT0NLRklM RShmcCk7CisJciA9IGNsZWFuZmlsZShmcCwgdHJ1ZSk7CisJRlVOTE9DS0ZJTEUoZnApOworCiAJ cmV0dXJuIChyKTsKIH0K --20cf3042705a7d7ed004f4d5b4f2-- From owner-freebsd-arch@FreeBSD.ORG Tue Mar 18 18:18:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 77D40752; Tue, 18 Mar 2014 18:18:24 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4A63093E; Tue, 18 Mar 2014 18:18:24 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3C9CFB917; Tue, 18 Mar 2014 14:18:23 -0400 (EDT) From: John Baldwin To: Mariusz Zaborski Subject: Re: Hello fdclose Date: Tue, 18 Mar 2014 14:04:51 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201403181404.52197.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 18 Mar 2014 14:18:23 -0400 (EDT) Cc: jilles@freebsd.org, freebsd-current@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 18:18:24 -0000 On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: > Hi, > > After our previous discuss [1] I prepare fdclosedir(3) function which > was committed by Pawel (cc'ed) in commit r254499. > > A while ago I also prepare the fdclose function. Unfortunately, this > new function is a little bit more tricky then previous one. Can I ask > you for a review of this patch? I think the code is fine. I have a few suggestions on the manpage wording: The +.Fn fdclose +function is equivalent to the +.Fn fclose +function except that this function returns file descriptor instead of +closing it. +.Pp +The I would move fdclose() to its own paragraph and reword this sentence as: "The fdclose() function is equivalent to fclose() except that it does not close the underlying file descriptor." .Sh RETURN VALUES -Upon successful completion 0 is returned. +The +.Fn fcloseall +function return no value. +.Pp +Upon successful completion +.Fn fclose +return 0. +Otherwise, +.Dv EOF +is returned and the global variable +.Va errno +is set to indicate the error. +.Pp +The +.Fn fdclose +function return the file descriptor if successfull. Otherwise, .Dv EOF One of English's arcane rules is that most verbs append an 's' when used with singular subjects, so "function returns" shoud be used instead of "function return", etc. I do think for this section it would be good to combine the descriptions of fclose() and fdclose() when possible, so perhaps something like: "The fcloseall() function returns no value. Upon successful completion, fclose() returns 0 and fdclose() returns the file descriptor of the underlying file. Otherwise, EOF is returned and the global variable errno is set to indicate the error. In either case no further access to the stream is possible." This allows "in either case" to still read correctly and makes it clear it applies to both fclose() and fdclose(). .Sh ERRORS +.Bl -tag -width Er +.It Bq Er EOPNOTSUPP The +.Fa _close +method in +.Fa stream +argument to +.Fn fdclose , +was not default. +.It Bq Er EBADF +The +.Fa stream +argument to +.Fn fdclose , +does not contains valid file descriptor. +.El +.Pp +The .Fn fclose -function +and +.Fn fdclose +functions may also fail and set .Va errno For the errors section, the first error list needs some sort of introductory text. Also, this shouldn't claim that fdclose() can return an errno value for close(2). "ERRORS The fdclose() function may will fail if: [EOPNOTSUPP] The stream to close uses a non-default close method. [EBADF] The stream is not backed by a valid file descriptor. The fclose() and fdclose() functions may also fail and set errno for any of the errors specified for fflush(3). The fclose() functino may also fail and set errno for any of the errors specified for close(2)." @@ -84,7 +130,9 @@ .Sh NOTES The .Fn fclose -function +and +.Fn fdclose +functions does not handle NULL arguments; they will result in a segmentation violation. This is intentional - it makes it easier to make sure programs written "do not handle". -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Mar 18 21:21:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A976747B; Tue, 18 Mar 2014 21:21:48 +0000 (UTC) Received: from mail106.syd.optusnet.com.au (mail106.syd.optusnet.com.au [211.29.132.42]) by mx1.freebsd.org (Postfix) with ESMTP id 51155F91; Tue, 18 Mar 2014 21:21:47 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail106.syd.optusnet.com.au (Postfix) with ESMTPS id 91B813C37B1; Wed, 19 Mar 2014 08:21:40 +1100 (EST) Date: Wed, 19 Mar 2014 08:21:39 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin Subject: Re: Hello fdclose In-Reply-To: <201403181404.52197.jhb@freebsd.org> Message-ID: <20140319071034.H996@besplex.bde.org> References: <201403181404.52197.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=fbeUlSgF c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=205o-tYZI0HVFv6DISIA:9 a=CjuIK1q_8ugA:10 Cc: jilles@freebsd.org, freebsd-arch@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 21:21:48 -0000 On Tue, 18 Mar 2014, John Baldwin wrote: > On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: > ... > I think the code is fine. I have a few suggestions on the manpage wording: > > .Sh RETURN VALUES > -Upon successful completion 0 is returned. > +The > +.Fn fcloseall > +function return no value. > +.Pp > +Upon successful completion > +.Fn fclose > +return 0. > +Otherwise, > +.Dv EOF > +is returned and the global variable > +.Va errno > +is set to indicate the error. The .Rv macro should be used whenever possible. Unfortunately, it doesn't support the EOF return, but only -1, so stdio man pages can rarely use it, and this one is no exception. Using it gives standard wording that is quite different from the above: standard wording: The close() function returns the value 0 if successful; otherwise the value -1 is returned and the global variable errno is set to indicate the error. above wording (previous): Upon successful completion 0 is returned. Otherwise, EOF is returned and the global variable errno is set to indicate the error. above wording (new): Upon successful completion fclose() return [sic] 0. Otherwise, EOF is returned and the global variable errno is set to indicate the error. These are excessively formal in different ways: - I don't like "the foo() function". Why not just "foo()"? The standard wording uses this, and so does the new wording, but the previous wording omits the function name (that only works for man pages that only have a single function, as they should). - I don't like "the value N". Why not just "N"? The standard wording uses this, but the previous and new wordings don't. - "returns N" is better than "N is returned". Some man pages use worse wordings like "N will be returned". - "the global variable errno" is excessively detailed/verbose, without the details even being correct. Why not just "errno", with this identifier documented elsewhere? errno isn't a global variable in most implementations. It is can be, and usually is, a macro that expands to a modifiable lvalue of type int. In FreeBSD, the macro expands to a function that returns a pointer to int. - "Upon sucessful completion" is correct but verbose. The standard wording doesn't even use it. - the standard wording uses a conjunction instead of a new sentence before "otherwise" (this is better). It is missing a comma after "otherwise" (this is worse). > +.Pp > +The > +.Fn fdclose > +function return the file descriptor if successfull. > Otherwise, > .Dv EOF "successfull is consistently misspelled. > One of English's arcane rules is that most verbs append an 's' when used with > singular subjects, so "function returns" shoud be used instead of "function > return", etc. I do think for this section it would be good to combine the > descriptions of fclose() and fdclose() when possible, so perhaps something > like: > > "The fcloseall() function returns no value. > > Upon successful completion, fclose() returns 0 and fdclose() returns the > file descriptor of the underlying file. Otherwise, EOF is returned and > the global variable errno is set to indicate the error. In either case > no further access to the stream is possible." OK. You kept "return[s] N" and and deverbosified "the foo() function". "Upon successful completion" is needed more with several functions. "the global variable errno" remains consistently bad. There should be a comma after "In either case". > This allows "in either case" to still read correctly and makes it clear it > applies to both fclose() and fdclose(). Better "In every case". > > .Sh ERRORS > +.Bl -tag -width Er > +.It Bq Er EOPNOTSUPP > The > +.Fa _close > +method in > +.Fa stream > +argument to > +.Fn fdclose , > +was not default. > +.It Bq Er EBADF The ERRORS section should be sorted. > For the errors section, the first error list needs some sort of introductory > text. Also, this shouldn't claim that fdclose() can return an errno value for > close(2). > > "ERRORS > > The fdclose() function may will fail if: I don't like the tense given by "will" in man pages. POSIX says "shall fail" in similar contexts, and "will fail" is a mistranslation of this ("shall" is a technical term that doesn't suggest future tense). deshallify.sh does the not-incorrect translation s/shall fail/fails/ (I think this is too simple to always work). It doesn't translate anything to "will". I can't parse "may will" :-). deshallify.txt doesn't translate "may" or "should" to anything (these are also technical terms in some contexts, so they might need translation. IIRC, "may" is optional behaviour, mostly for the implementation, while "shall" is required behaviour, only for the implementation, but "should" is recommended practice, mostly for applications). Man pages are very unlikely to be as consistent as POSIX with these terms. > [EOPNOTSUPP] The stream to close uses a non-default close method. > > [EBADF] The stream is not backed by a valid file descriptor. > > The fclose() and fdclose() functions may also fail and set errno for any of > the errors specified for fflush(3). Most stdio man pages just point to underlying functions for errors. This avoids duplication. The above EBADF seems to be redundant, since fflush(3) already has it, with a different and longer description. > > The fclose() functino may also fail and set errno for any of the errors > specified for close(2)." fclose() is now a small function :-). > @@ -84,7 +130,9 @@ > .Sh NOTES > The > .Fn fclose > -function > +and > +.Fn fdclose > +functions > does not handle NULL arguments; they will result in a segmentation > violation. > This is intentional - it makes it easier to make sure programs written > > "do not handle". "they" is now ambiguous. In the old version: Em-dash seems to be handled poorly by mdoc. It seems to be necessary to hard code it. It shouldn't be hard coded as a hyphen. The double "it" is ambiguous. Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Mar 18 21:35:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 874D1C85; Tue, 18 Mar 2014 21:35:20 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4B595167; Tue, 18 Mar 2014 21:35:20 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id A602AB80AB; Tue, 18 Mar 2014 22:35:16 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 8730C28497; Tue, 18 Mar 2014 22:35:16 +0100 (CET) Date: Tue, 18 Mar 2014 22:35:16 +0100 From: Jilles Tjoelker To: Mariusz Zaborski Subject: Re: Hello fdclose Message-ID: <20140318213516.GA71491@stack.nl> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 21:35:20 -0000 On Tue, Mar 18, 2014 at 12:23:19AM +0100, Mariusz Zaborski wrote: > After our previous discuss [1] I prepare fdclosedir(3) function which > was committed by Pawel (cc'ed) in commit r254499. > A while ago I also prepare the fdclose function. Unfortunately, this > new function is a little bit more tricky then previous one. Can I ask > you for a review of this patch? Does this patch allow perl to stop writing to FILE._file? As pointed out in http://lists.freebsd.org/pipermail/freebsd-current/2013-January/039024.html perlio.c in the perl source contains a function PerlIOStdio_invalidate_fileno() that should modify a FILE such that fclose() does not close the file descriptor but still frees all memory (Perl has already called fflush()). Although using fdclose() could solve this without touching the internals of FILE, this will make perlio.c uglier with even more #ifdefs. > [snip] > --- //depot/user/oshogbo/capsicum/lib/libc/stdio/Symbol.map 2013-06-28 08:51:28.000000000 0000 > +++ /home/oshogbo/p4/capsicum/lib/libc/stdio/Symbol.map 2013-06-28 08:51:28.000000000 0000 > @@ -156,6 +156,7 @@ > putwc_l; > putwchar_l; > fmemopen; > + fdclose; > open_memstream; > open_wmemstream; > }; This should be in the FBSD_1.4 namespace (which does not exist yet). > [snip] > --- //depot/user/oshogbo/capsicum/lib/libc/stdio/fclose.c 2013-06-28 08:51:28.000000000 0000 > +++ /home/oshogbo/p4/capsicum/lib/libc/stdio/fclose.c 2013-06-28 08:51:28.000000000 0000 > [snip] > +int > +fdclose(FILE *fp) > +{ > + int fd, r, err; > + > + if (fp->_flags == 0) { /* not open! */ > + errno = EBADF; > + return (EOF); > + } > + > + r = 0; > + FLOCKFILE(fp); > + fd = fp->_file; > + if (fp->_close != __sclose) { > + r = EOF; > + errno = EOPNOTSUPP; > + } else if (fd < 0) { > + r = EOF; > + errno = EBADF; > + } > + if (r == EOF) { > + err = errno; > + (void)cleanfile(fp, true); > + errno = err; > + } else { > + r = cleanfile(fp, false); > + } > FUNLOCKFILE(fp); > + > + return (r == 0 ? fd : r); If a file descriptor would be returned but cleanfile() returns an error (e.g. write error on flush), the file descriptor is not returned and not closed. I think that in cases where fdclose() would be used, it is essential that the file descriptor is never closed. This means that the API needs to be different so it can report a write error but still return a file descriptor. One way to do this is to return the file descriptor by reference. Another is to expect the application to call fileno() and not return the descriptor from the new function. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Wed Mar 19 04:39:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8859C9FF; Wed, 19 Mar 2014 04:39:00 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 396E5C7D; Wed, 19 Mar 2014 04:38:59 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s2J4cvTh045626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 18 Mar 2014 22:38:58 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s2J4cvWK045623; Tue, 18 Mar 2014 22:38:57 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Tue, 18 Mar 2014 22:38:57 -0600 (MDT) From: Warren Block To: John Baldwin Subject: Re: Hello fdclose In-Reply-To: <201403181404.52197.jhb@freebsd.org> Message-ID: References: <201403181404.52197.jhb@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Tue, 18 Mar 2014 22:38:58 -0600 (MDT) Cc: jilles@freebsd.org, freebsd-arch@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 04:39:00 -0000 On Tue, 18 Mar 2014, John Baldwin wrote: > On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: >> Hi, >> >> After our previous discuss [1] I prepare fdclosedir(3) function which >> was committed by Pawel (cc'ed) in commit r254499. >> >> A while ago I also prepare the fdclose function. Unfortunately, this >> new function is a little bit more tricky then previous one. Can I ask >> you for a review of this patch? > > I think the code is fine. I have a few suggestions on the manpage wording: > > The > +.Fn fdclose > +function is equivalent to the > +.Fn fclose > +function except that this function returns file descriptor instead of > +closing it. > +.Pp > +The > > I would move fdclose() to its own paragraph and reword this sentence as: > > "The fdclose() function is equivalent to fclose() except that it does > not close the underlying file descriptor." .Fn fdclose is equivalent to .Fn fclose , but the file descriptor is returned rather than closed. Likewise in other sections, the markup is supposed to do the job of pointing out that something is a function. textproc/igor can identify some problems with wording. It also checks for rudimentary mdoc(7) requirements. If desired, I'm willing to edit this man page. (I've learned far too recently that most people do not want to be consulted on wording, they just want it fixed. That's now the approach I take: make all the corrections and return it, rather than a back-and-forth with the danger of edit fatigue.) From owner-freebsd-arch@FreeBSD.ORG Wed Mar 19 19:49:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7734CE0A; Wed, 19 Mar 2014 19:49:09 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4DE563F7; Wed, 19 Mar 2014 19:49:09 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 57EC5B96E; Wed, 19 Mar 2014 15:49:08 -0400 (EDT) From: John Baldwin To: Warren Block Subject: Re: Hello fdclose Date: Wed, 19 Mar 2014 15:23:33 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201403181404.52197.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403191523.33275.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 19 Mar 2014 15:49:08 -0400 (EDT) Cc: jilles@freebsd.org, freebsd-arch@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 19:49:09 -0000 On Wednesday, March 19, 2014 12:38:57 am Warren Block wrote: > On Tue, 18 Mar 2014, John Baldwin wrote: > > > On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: > >> Hi, > >> > >> After our previous discuss [1] I prepare fdclosedir(3) function which > >> was committed by Pawel (cc'ed) in commit r254499. > >> > >> A while ago I also prepare the fdclose function. Unfortunately, this > >> new function is a little bit more tricky then previous one. Can I ask > >> you for a review of this patch? > > > > I think the code is fine. I have a few suggestions on the manpage wording: > > > > The > > +.Fn fdclose > > +function is equivalent to the > > +.Fn fclose > > +function except that this function returns file descriptor instead of > > +closing it. > > +.Pp > > +The > > > > I would move fdclose() to its own paragraph and reword this sentence as: > > > > "The fdclose() function is equivalent to fclose() except that it does > > not close the underlying file descriptor." > > .Fn fdclose > is equivalent to > .Fn fclose , > but the file descriptor is returned rather than closed. > > Likewise in other sections, the markup is supposed to do the job of > pointing out that something is a function. Yes, but this has the 'no capital letter at the start of a sentence' problem. Also, I do think reusing the 'underlying file descriptor' language is important in the context of the earlier description of fclose(). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Mar 19 20:28:17 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3C77E30; Wed, 19 Mar 2014 20:28:17 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 76C5B9C7; Wed, 19 Mar 2014 20:28:17 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s2JKSFDo053021 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 19 Mar 2014 14:28:15 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s2JKSF6w053018; Wed, 19 Mar 2014 14:28:15 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Wed, 19 Mar 2014 14:28:15 -0600 (MDT) From: Warren Block To: John Baldwin Subject: Re: Hello fdclose In-Reply-To: <201403191523.33275.jhb@freebsd.org> Message-ID: References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Wed, 19 Mar 2014 14:28:15 -0600 (MDT) Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Mar 2014 20:28:18 -0000 On Wed, 19 Mar 2014, John Baldwin wrote: > On Wednesday, March 19, 2014 12:38:57 am Warren Block wrote: >> On Tue, 18 Mar 2014, John Baldwin wrote: >> >>> On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: >>>> Hi, >>>> >>>> After our previous discuss [1] I prepare fdclosedir(3) function which >>>> was committed by Pawel (cc'ed) in commit r254499. >>>> >>>> A while ago I also prepare the fdclose function. Unfortunately, this >>>> new function is a little bit more tricky then previous one. Can I ask >>>> you for a review of this patch? >>> >>> I think the code is fine. I have a few suggestions on the manpage wording: >>> >>> The >>> +.Fn fdclose >>> +function is equivalent to the >>> +.Fn fclose >>> +function except that this function returns file descriptor instead of >>> +closing it. >>> +.Pp >>> +The >>> >>> I would move fdclose() to its own paragraph and reword this sentence as: >>> >>> "The fdclose() function is equivalent to fclose() except that it does >>> not close the underlying file descriptor." >> >> .Fn fdclose >> is equivalent to >> .Fn fclose , >> but the file descriptor is returned rather than closed. >> >> Likewise in other sections, the markup is supposed to do the job of >> pointing out that something is a function. > > Yes, but this has the 'no capital letter at the start of a sentence' problem. I've heard that mentioned before, but have never seen any actual rule regarding it. And we do have actual rules about avoiding redundant phrases: http://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/book.html#writing-style-guidelines While normal words should be capitalized as the first word in a sentence, special words that are case-sensitive override that (IMO). > Also, I do think reusing the 'underlying file descriptor' language is important > in the context of the earlier description of fclose(). Sorry, a problem with my micro-optimization: .Fn fdclose is equivalent to .Fn fclose , but does not close the underlying file descriptor. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 20 14:22:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9265C4C; Thu, 20 Mar 2014 14:22:05 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8FB7FBA6; Thu, 20 Mar 2014 14:22:05 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8B14EB918; Thu, 20 Mar 2014 10:22:03 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Hello fdclose Date: Thu, 20 Mar 2014 09:32:02 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <20140318213516.GA71491@stack.nl> In-Reply-To: <20140318213516.GA71491@stack.nl> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403200932.02294.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 20 Mar 2014 10:22:03 -0400 (EDT) Cc: freebsd-current@freebsd.org, Jilles Tjoelker , Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 14:22:05 -0000 On Tuesday, March 18, 2014 5:35:16 pm Jilles Tjoelker wrote: > On Tue, Mar 18, 2014 at 12:23:19AM +0100, Mariusz Zaborski wrote: > > After our previous discuss [1] I prepare fdclosedir(3) function which > > was committed by Pawel (cc'ed) in commit r254499. > > > A while ago I also prepare the fdclose function. Unfortunately, this > > new function is a little bit more tricky then previous one. Can I ask > > you for a review of this patch? > > Does this patch allow perl to stop writing to FILE._file? As pointed out > in > http://lists.freebsd.org/pipermail/freebsd-current/2013-January/039024.html > perlio.c in the perl source contains a function > PerlIOStdio_invalidate_fileno() that should modify a FILE such that > fclose() does not close the file descriptor but still frees all memory > (Perl has already called fflush()). Although using fdclose() could solve > this without touching the internals of FILE, this will make perlio.c > uglier with even more #ifdefs. I hope it does. I want to have some sort of API for Perl to use so it stops (ab)using _file before I move forward with full int _file. > I think that in cases where fdclose() would be used, it is essential > that the file descriptor is never closed. This means that the API needs > to be different so it can report a write error but still return a file > descriptor. One way to do this is to return the file descriptor by > reference. Another is to expect the application to call fileno() and not > return the descriptor from the new function. I would prefer the latter of requiring the application to use fileno(). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 20 14:22:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D843F3; Thu, 20 Mar 2014 14:22:22 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 64B65BBF; Thu, 20 Mar 2014 14:22:22 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5FA1BB9B2; Thu, 20 Mar 2014 10:22:21 -0400 (EDT) From: John Baldwin To: Warren Block Subject: Re: Hello fdclose Date: Thu, 20 Mar 2014 10:21:51 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201403191523.33275.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201403201021.51602.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 20 Mar 2014 10:22:21 -0400 (EDT) Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 14:22:22 -0000 On Wednesday, March 19, 2014 4:28:15 pm Warren Block wrote: > On Wed, 19 Mar 2014, John Baldwin wrote: > > > On Wednesday, March 19, 2014 12:38:57 am Warren Block wrote: > >> On Tue, 18 Mar 2014, John Baldwin wrote: > >> > >>> On Monday, March 17, 2014 7:23:19 pm Mariusz Zaborski wrote: > >>>> Hi, > >>>> > >>>> After our previous discuss [1] I prepare fdclosedir(3) function which > >>>> was committed by Pawel (cc'ed) in commit r254499. > >>>> > >>>> A while ago I also prepare the fdclose function. Unfortunately, this > >>>> new function is a little bit more tricky then previous one. Can I ask > >>>> you for a review of this patch? > >>> > >>> I think the code is fine. I have a few suggestions on the manpage wording: > >>> > >>> The > >>> +.Fn fdclose > >>> +function is equivalent to the > >>> +.Fn fclose > >>> +function except that this function returns file descriptor instead of > >>> +closing it. > >>> +.Pp > >>> +The > >>> > >>> I would move fdclose() to its own paragraph and reword this sentence as: > >>> > >>> "The fdclose() function is equivalent to fclose() except that it does > >>> not close the underlying file descriptor." > >> > >> .Fn fdclose > >> is equivalent to > >> .Fn fclose , > >> but the file descriptor is returned rather than closed. > >> > >> Likewise in other sections, the markup is supposed to do the job of > >> pointing out that something is a function. > > > > Yes, but this has the 'no capital letter at the start of a sentence' problem. > > I've heard that mentioned before, but have never seen any actual rule > regarding it. All of my rules for that come from elementary school. :) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 20 20:13:10 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10DF8C35; Thu, 20 Mar 2014 20:13:10 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C31A5C29; Thu, 20 Mar 2014 20:13:09 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id E73C2644A; Thu, 20 Mar 2014 20:13:08 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 71B2CF23; Thu, 20 Mar 2014 21:13:01 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warren Block Subject: Re: Hello fdclose References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> Date: Thu, 20 Mar 2014 21:13:01 +0100 In-Reply-To: (Warren Block's message of "Wed, 19 Mar 2014 14:28:15 -0600 (MDT)") Message-ID: <86zjkkr5ma.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: jilles@freebsd.org, freebsd-arch@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Mar 2014 20:13:10 -0000 Warren Block writes: > John Baldwin writes: > > Warren Block writes: > > > .Fn fdclose > > > is equivalent to > > > .Fn fclose , > > > but the file descriptor is returned rather than closed. > > Yes, but this has the 'no capital letter at the start of a sentence' > > problem. > I've heard that mentioned before, but have never seen any actual rule > regarding it. And we do have actual rules about avoiding redundant > phrases: > http://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/book.html#wri= ting-style-guidelines We always use The .Nm foo utility or The .Fn foo function instead of just .Nm or .Fn at the start of a sentence, but never (or rarely) within a sentence. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 13:48:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4EB7EF8; Fri, 21 Mar 2014 13:48:50 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 62B8EBDD; Fri, 21 Mar 2014 13:48:50 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s2LDmf8S013767 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 21 Mar 2014 07:48:41 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s2LDmequ013764; Fri, 21 Mar 2014 07:48:41 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Fri, 21 Mar 2014 07:48:40 -0600 (MDT) From: Warren Block To: =?ISO-8859-15?Q?Dag-Erling_Sm=F8rgrav?= Subject: Re: Hello fdclose In-Reply-To: <86zjkkr5ma.fsf@nine.des.no> Message-ID: References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Fri, 21 Mar 2014 07:48:41 -0600 (MDT) Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 13:48:50 -0000 On Thu, 20 Mar 2014, Dag-Erling Smørgrav wrote: > Warren Block writes: >> John Baldwin writes: >>> Warren Block writes: >>>> .Fn fdclose >>>> is equivalent to >>>> .Fn fclose , >>>> but the file descriptor is returned rather than closed. >>> Yes, but this has the 'no capital letter at the start of a sentence' >>> problem. >> I've heard that mentioned before, but have never seen any actual rule >> regarding it. And we do have actual rules about avoiding redundant >> phrases: >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/book.html#writing-style-guidelines > > We always use > > The > .Nm foo > utility > > or > > The > .Fn foo > function > > instead of just .Nm or .Fn at the start of a sentence, but never (or > rarely) within a sentence. By "we", do you mean the FreeBSD project or your local organization? Is there a specific reason the redundant wording, or is it just customary? I suspect that usage is similar to that for an acronym, defining it on the first usage but only then. Both mdoc and DocBook markup should accomplish the same thing. From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 14:19:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1641EF2; Fri, 21 Mar 2014 14:19:04 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id AB361F27; Fri, 21 Mar 2014 14:19:04 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id A64EB6DAE; Fri, 21 Mar 2014 14:19:03 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id C67577FB; Fri, 21 Mar 2014 15:18:52 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warren Block Subject: Re: Hello fdclose References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> Date: Fri, 21 Mar 2014 15:18:52 +0100 In-Reply-To: (Warren Block's message of "Fri, 21 Mar 2014 07:48:40 -0600 (MDT)") Message-ID: <86fvmbzlbn.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:19:05 -0000 Warren Block writes: > Dag-Erling Sm=C3=B8rgrav writes: > > We always use [The .Nm foo utility or The .Fn foo function] instead > > of just .Nm or .Fn at the start of a sentence, but never (or rarely) > > within a sentence. > By "we", do you mean the FreeBSD project or your local organization? > Is there a specific reason the redundant wording, or is it just > customary? I mean the FreeBSD project, and the reason is as John stated: all sentences must start with a capital letter. I've gotten so used to this over the past 15 years that I even do it in email and other non-FreeBSD written material. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 14:22:38 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDB8D411 for ; Fri, 21 Mar 2014 14:22:38 +0000 (UTC) Received: from mail-pb0-f50.google.com (mail-pb0-f50.google.com [209.85.160.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B8D7FE3 for ; Fri, 21 Mar 2014 14:22:38 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id md12so2480033pbc.9 for ; Fri, 21 Mar 2014 07:22:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Dj8/J9O9N5MQJ30L3vY2Aq5UDbm13po9CGB364JotJc=; b=VAvjGQzB6CbdY68JcYc0L6A4pr49weNSlk/auNVGPZBxpY/NKNqKZAVME6r/6M2f6P FW3TSpsjrRxGgoUKatMt6aq9tIn8siD6EBrDdP4U9/sbO6jtVzUom4ez4ShCU4e9Evd3 3KbVgXA8Ob9OM/TARLldEk0BOGsKq6dtF2ggtkII2It+9VqhrG6AFmwOxOtSbCMIAXrU 2rf73ZuOv2yjlmt/zzVpA5Ev7logp5dR+TLLmH/eJaTxRdBygw3rwUvlOLGqwqJ+wyUN SnPcO2hmAtejydqDaqxrjG8QHaYlta20v2HgzcbqO/BygiNDcVeHV8n9rybED5NQfibq f3Ww== X-Gm-Message-State: ALoCoQldIQrdwYKHF0+49kuTJNKa+4pYgKDaVsVmfhpNBJJfFAtRudpAuuVBWB7SICPu0QyEVExH X-Received: by 10.66.144.227 with SMTP id sp3mr54829333pab.100.1395411751007; Fri, 21 Mar 2014 07:22:31 -0700 (PDT) Received: from lgmac-scingram.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id h6sm10182001pbl.75.2014.03.21.07.22.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Mar 2014 07:22:30 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Hello fdclose From: Warner Losh In-Reply-To: <86fvmbzlbn.fsf@nine.des.no> Date: Fri, 21 Mar 2014 08:22:27 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1874) Cc: Warren Block , jilles@freebsd.org, freebsd-arch@freebsd.org, FreeBSD Current , Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 14:22:38 -0000 On Mar 21, 2014, at 8:18 AM, Dag-Erling Sm=F8rgrav wrote: > Warren Block writes: >> Dag-Erling Sm=F8rgrav writes: >>> We always use [The .Nm foo utility or The .Fn foo function] instead >>> of just .Nm or .Fn at the start of a sentence, but never (or rarely) >>> within a sentence. >> By "we", do you mean the FreeBSD project or your local organization? >> Is there a specific reason the redundant wording, or is it just >> customary? >=20 > I mean the FreeBSD project, and the reason is as John stated: all > sentences must start with a capital letter. I've gotten so used to = this > over the past 15 years that I even do it in email and other = non-FreeBSD > written material. Yes, it is style that was settled upon 15 years ago or so in the FreeBSD project. Even more than finding myself using this in email, etc, I find = that when I see people using other conventions it just looks wrong to my = eyes. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 16:40:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5030E7ED; Fri, 21 Mar 2014 16:40:21 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D6DD6172; Fri, 21 Mar 2014 16:40:20 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s2LGeG24014951 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 21 Mar 2014 10:40:17 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s2LGeG07014948; Fri, 21 Mar 2014 10:40:16 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Fri, 21 Mar 2014 10:40:16 -0600 (MDT) From: Warren Block To: =?ISO-8859-15?Q?Dag-Erling_Sm=F8rgrav?= Subject: Re: Hello fdclose In-Reply-To: <86fvmbzlbn.fsf@nine.des.no> Message-ID: References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Fri, 21 Mar 2014 10:40:17 -0600 (MDT) Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 16:40:21 -0000 On Fri, 21 Mar 2014, Dag-Erling Smørgrav wrote: > Warren Block writes: >> Dag-Erling Smørgrav writes: >>> We always use [The .Nm foo utility or The .Fn foo function] instead >>> of just .Nm or .Fn at the start of a sentence, but never (or rarely) >>> within a sentence. >> By "we", do you mean the FreeBSD project or your local organization? >> Is there a specific reason the redundant wording, or is it just >> customary? > > I mean the FreeBSD project, and the reason is as John stated: all > sentences must start with a capital letter. I've gotten so used to this > over the past 15 years that I even do it in email and other non-FreeBSD > written material. > > DES "Because it's been that way for 15 years" is not always a justification (consider BIND in base, for example :). I'll let this drop for now, but still find it interesting that it is not mentioned in the FDP Primer, and those style guide rules date back at least 15 years. It may be that the original doc project members felt that capitalizing the first word of a sentence was so obvious that it did not need to be stated. Or possibly, that it was a rule that could be usefully broken at times. From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 16:54:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C31E73D9; Fri, 21 Mar 2014 16:54:22 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7DDD7370; Fri, 21 Mar 2014 16:54:22 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 626E48D67; Fri, 21 Mar 2014 16:54:21 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 6B157960; Fri, 21 Mar 2014 17:54:13 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warren Block Subject: Re: Hello fdclose References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> Date: Fri, 21 Mar 2014 17:54:13 +0100 In-Reply-To: (Warren Block's message of "Fri, 21 Mar 2014 10:40:16 -0600 (MDT)") Message-ID: <86mwgj7aru.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: jilles@freebsd.org, freebsd-current@freebsd.org, Mariusz Zaborski , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 16:54:22 -0000 Warren Block writes: > Dag-Erling Sm=C3=B8rgrav writes: > > I mean the FreeBSD project, and the reason is as John stated: all > > sentences must start with a capital letter. I've gotten so used to > > this over the past 15 years that I even do it in email and other > > non-FreeBSD written material. > "Because it's been that way for 15 years" is not always a > justification (consider BIND in base, for example :). "I don't like your answer, so I will ignore it" is not a justification either. It's kindergarden behavior and beneath the dignity of a FreeBSD committer. We have a rule that sentences must always start with a capital letter. The fact that this rule was instituted 15 years ago does not automatically invalidate it, and neither does the fact that Joe Random Committer disagrees with it. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 17:20:27 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 679C1292; Fri, 21 Mar 2014 17:20:27 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 35FDF7F7; Fri, 21 Mar 2014 17:20:26 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WR37E-000GXT-KH; Fri, 21 Mar 2014 17:20:20 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s2LHKHV4072021; Fri, 21 Mar 2014 11:20:17 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/0mIVqU4NTCgACqjRlEzw1 Subject: Re: Hello fdclose From: Ian Lepore To: Dag-Erling =?ISO-8859-1?Q?Sm=F8rgrav?= In-Reply-To: <86mwgj7aru.fsf@nine.des.no> References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> Content-Type: text/plain; charset="ISO-8859-1" Date: Fri, 21 Mar 2014 11:20:17 -0600 Message-ID: <1395422417.81853.14.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by damnhippie.dyndns.org id s2LHKHV4072021 Cc: Warren Block , jilles@FreeBSD.org, freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 17:20:27 -0000 On Fri, 2014-03-21 at 17:54 +0100, Dag-Erling Sm=F8rgrav wrote: > Warren Block writes: > > Dag-Erling Sm=F8rgrav writes: > > > I mean the FreeBSD project, and the reason is as John stated: all > > > sentences must start with a capital letter. I've gotten so used to > > > this over the past 15 years that I even do it in email and other > > > non-FreeBSD written material. > > "Because it's been that way for 15 years" is not always a > > justification (consider BIND in base, for example :). >=20 > "I don't like your answer, so I will ignore it" is not a justification > either. It's kindergarden behavior and beneath the dignity of a FreeBS= D > committer. >=20 > We have a rule that sentences must always start with a capital letter. > The fact that this rule was instituted 15 years ago does not > automatically invalidate it, and neither does the fact that Joe Random > Committer disagrees with it. >=20 > DES Just as the age of a moronic "rule" such as this doesn't in any way justify the idea that it could never be changed. Especially when the "rule" appears to be an undocumented prejudice. People love to throw around assertions about "rules" of the English language. It doesn't have many rules (subject has to agree in number with the verb, that's about it for unbreakable rules), but it has as many opinions on proper style as there are readers and writers. -- Ian From owner-freebsd-arch@FreeBSD.ORG Fri Mar 21 19:23:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DFBE46A for ; Fri, 21 Mar 2014 19:23:05 +0000 (UTC) Received: from mail-pb0-f50.google.com (mail-pb0-f50.google.com [209.85.160.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id DE10A802 for ; Fri, 21 Mar 2014 19:23:04 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id md12so2817514pbc.9 for ; Fri, 21 Mar 2014 12:23:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Ni6vJ+PsNCtSifKSdpJhnt6fxZ9hnr42KmjMjz22hXA=; b=CK0S2MpbWNG0szVkFdjemNUiHRSc8K+UaTLz9seiSeptV/NI14YL7IaD6bMemeX0VO qIWq9Ri4y1TkBoy0Pui+xSViV46ihbnBLGXiEqPCx/YxyKQLkBnJCaprAv3KCSiwcE1j vbVa8rlBifWNEhuEyPOgFiYN19axQNmy0ApNmS4usSYSpj0CJft8mdIMDDgn5qiDyYpY wwyJ8k7Hh7A6AyubcSU81xyO2yYm2AsTh5yHxOkjTj/Jhb8ic8aW41lbdZBH5SCnWrRF R5kBRBwc0phTAFfGfOQeHocKQ4Y/XqrX/HgdFZAmMpjEh1X+KY+wXxlGx10qJr7s4ojb U3GA== X-Gm-Message-State: ALoCoQk8VIHK0N5snvhnMKCo679kQ8jGmr0KoWoz5+gpNeqbDNFanBArE7nW+KH2IMcqrAzY3cZa X-Received: by 10.66.20.10 with SMTP id j10mr57357297pae.11.1395428287863; Fri, 21 Mar 2014 11:58:07 -0700 (PDT) Received: from lgmac-scingram.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id om6sm11284596pbc.43.2014.03.21.11.58.06 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 21 Mar 2014 11:58:07 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Hello fdclose From: Warner Losh In-Reply-To: <1395422417.81853.14.camel@revolution.hippie.lan> Date: Fri, 21 Mar 2014 12:58:05 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <98DE2314-1233-4C12-8346-5D1240B9BAAF@bsdimp.com> References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> <1395422417.81853.14.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1874) Cc: Warren Block , Mariusz Zaborski , freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org, jilles@FreeBSD.org, =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Mar 2014 19:23:05 -0000 On Mar 21, 2014, at 11:20 AM, Ian Lepore wrote: > On Fri, 2014-03-21 at 17:54 +0100, Dag-Erling Sm=F8rgrav wrote: >> Warren Block writes: >>> Dag-Erling Sm=F8rgrav writes: >>>> I mean the FreeBSD project, and the reason is as John stated: all >>>> sentences must start with a capital letter. I've gotten so used to >>>> this over the past 15 years that I even do it in email and other >>>> non-FreeBSD written material. >>> "Because it's been that way for 15 years" is not always a >>> justification (consider BIND in base, for example :). >>=20 >> "I don't like your answer, so I will ignore it" is not a = justification >> either. It's kindergarden behavior and beneath the dignity of a = FreeBSD >> committer. >>=20 >> We have a rule that sentences must always start with a capital = letter. >> The fact that this rule was instituted 15 years ago does not >> automatically invalidate it, and neither does the fact that Joe = Random >> Committer disagrees with it. >>=20 >> DES >=20 > Just as the age of a moronic "rule" such as this doesn't in any way > justify the idea that it could never be changed. Especially when the > "rule" appears to be an undocumented prejudice. It has been the way things have been for a long time, and everybody that has been writing man pages in the project for any length of time knows it. Ruslan was especially good about fixing this issue back in the day. This rule was hashed out and there=92s really no compelling = reason to change it. Unlike BIND, we have had no complaints about it in the almost two decades it has been around=85 It does serve a useful purpose, though, which is why it has endured. If you were to have a man page that said =91Putc(3) returns =85=92 then = the automated tools (and web links) that find Putc.3 wouldn=92t be able to = since it doesn=92t exist. > People love to throw around assertions about "rules" of the English > language. It doesn't have many rules (subject has to agree in number > with the verb, that's about it for unbreakable rules), but it has as > many opinions on proper style as there are readers and writers. This is the proper style for FreeBSD man page, by convention. It is so basic that it doesn=92t surprise me it isn=92t in the docs, but it also = surprises me that it isn=92t. There=92s numerous instances of this rule being mentioned in commit = messages: http://lists.freebsd.org/pipermail/cvs-doc/2008-February/017562.html http://osdir.com/ml/os.freebsd.devel.cvs.doc/2004-07/msg00284.html = http://blog.gmane.org/gmane.os.freebsd.devel.documentation/month=3D2005020= 1/page=3D6 or the discussions in the archives like: http://osdir.com/ml/freebsd.devel.documentation/2002-05/msg00589.html = http://markmail.org/thread/jx2jr5myngp5jg5d#query:+page:1+mid:lasvnzqoojxf= eoet+state:results or a few other places. These discussions span the last 15 years (the oldest one is 1999, but = the convention is easily a few years older than that). Warner From owner-freebsd-arch@FreeBSD.ORG Sat Mar 22 00:33:45 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14AD5873; Sat, 22 Mar 2014 00:33:45 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id C2C8D8F7; Sat, 22 Mar 2014 00:33:44 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id C3A8C6E18; Sat, 22 Mar 2014 00:33:42 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 620E2CB6; Sat, 22 Mar 2014 01:33:34 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Ian Lepore Subject: Re: Hello fdclose References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> <1395422417.81853.14.camel@revolution.hippie.lan> Date: Sat, 22 Mar 2014 01:33:34 +0100 In-Reply-To: <1395422417.81853.14.camel@revolution.hippie.lan> (Ian Lepore's message of "Fri, 21 Mar 2014 11:20:17 -0600") Message-ID: <86ior76pi9.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Warren Block , jilles@FreeBSD.org, freebsd-arch@FreeBSD.org, freebsd-current@FreeBSD.org, Mariusz Zaborski X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 00:33:45 -0000 Ian Lepore writes: > Just as the age of a moronic "rule" such as this doesn't in any way > justify the idea that it could never be changed. I never claimed it did, but I can see how it would be convenient to pretend that I did so you can attack that instead of discussing the actual issue. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 22 00:51:48 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0F99FA2; Sat, 22 Mar 2014 00:51:48 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 991D6A6B; Sat, 22 Mar 2014 00:51:48 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 3753C6E6C; Sat, 22 Mar 2014 00:51:47 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 09231CD9; Sat, 22 Mar 2014 01:51:39 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Subject: Re: Hello fdclose References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> <1395422417.81853.14.camel@revolution.hippie.lan> <98DE2314-1233-4C12-8346-5D1240B9BAAF@bsdimp.com> Date: Sat, 22 Mar 2014 01:51:38 +0100 In-Reply-To: <98DE2314-1233-4C12-8346-5D1240B9BAAF@bsdimp.com> (Warner Losh's message of "Fri, 21 Mar 2014 12:58:05 -0600") Message-ID: <86a9cj6oo5.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Warren Block , Ian Lepore , Mariusz Zaborski , freebsd-current@FreeBSD.org, freebsd-arch@FreeBSD.org, jilles@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 00:51:49 -0000 Warner Losh writes: > It does serve a useful purpose, though, which is why it has endured. > If you were to have a man page that said =E2=80=98Putc(3) returns =E2=80= =A6=E2=80=99 then the > automated tools (and web links) that find Putc.3 wouldn=E2=80=99t be able= to since > it doesn=E2=80=99t exist. Moreover - if FreeBSD were written in Pascal, it might not matter, but in C, _exit(2) and _Exit(3) are two different functions. (I'm sure there are other examples without a leading underscore) (eww, starting a sentence with a non-alphabetic character would be even worse...) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 22 07:04:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 930754CE; Sat, 22 Mar 2014 07:04:31 +0000 (UTC) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4B601B6F; Sat, 22 Mar 2014 07:04:31 +0000 (UTC) Received: by mail-pd0-f175.google.com with SMTP id x10so3218345pdj.6 for ; Sat, 22 Mar 2014 00:04:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Irx6AB7AtYfkcr61ySQvzFDnoXCBjZE0tlX1tccc/gs=; b=jLWNObI8npum3Pk5J6HWXsm4R+syb2d/1ofaDOLmSU36RCuJhJmQ6zyJePkufm+ULD La7CDw/Ik32w9dJiIHM8dZOerTB8XNfjhtU/YUtRZC8jrrBxJNgmNCjk+G3bMgrVvXFf oF3X3B++tDh+sN4cucrJyBgyQeR7OSrZi2TksrvJrNmYPB2K/dG1dR/7pP1GubO3l8gq LRjult3GgvQZMxWncjGjVvRXuvgKGE79S8CWgzMQ69x0oRDteNtoC84qpFkknjlEIdZU 6jdkdrVInc/vS8ykx+wyTJdGlApbDvqOZcwgTdjk93wHYF3LFxiZfR8P6RrYgBLu12fA 5GGQ== MIME-Version: 1.0 X-Received: by 10.68.193.130 with SMTP id ho2mr36211818pbc.141.1395471870983; Sat, 22 Mar 2014 00:04:30 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.0.164 with HTTP; Sat, 22 Mar 2014 00:04:30 -0700 (PDT) In-Reply-To: <86a9cj6oo5.fsf@nine.des.no> References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> <1395422417.81853.14.camel@revolution.hippie.lan> <98DE2314-1233-4C12-8346-5D1240B9BAAF@bsdimp.com> <86a9cj6oo5.fsf@nine.des.no> Date: Sat, 22 Mar 2014 00:04:30 -0700 X-Google-Sender-Auth: t2S2K2uhPJDiY0EsFCyXmjmtzwQ Message-ID: Subject: Re: Hello fdclose From: Kevin Oberman To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Warren Block , Ian Lepore , Mariusz Zaborski , FreeBSD Current , freebsd-arch@freebsd.org, jilles@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Mar 2014 07:04:31 -0000 On Fri, Mar 21, 2014 at 5:51 PM, Dag-Erling Sm=C3=B8rgrav wrot= e: > Warner Losh writes: > > It does serve a useful purpose, though, which is why it has endured. > > If you were to have a man page that said =E2=80=98Putc(3) returns =E2= =80=A6=E2=80=99 then the > > automated tools (and web links) that find Putc.3 wouldn=E2=80=99t be ab= le to > since > > it doesn=E2=80=99t exist. > > Moreover - if FreeBSD were written in Pascal, it might not matter, but > in C, _exit(2) and _Exit(3) are two different functions. > > (I'm sure there are other examples without a leading underscore) > > (eww, starting a sentence with a non-alphabetic character would be even > worse...) > 3Com was once a very important networking company. 3M still is. /me duck and runs. --=20 R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-arch@FreeBSD.ORG Tue Mar 25 22:25:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F336AA9D; Tue, 25 Mar 2014 22:25:28 +0000 (UTC) Received: from mail-ie0-x235.google.com (mail-ie0-x235.google.com [IPv6:2607:f8b0:4001:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B9AF8D1B; Tue, 25 Mar 2014 22:25:28 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id tp5so1070276ieb.12 for ; Tue, 25 Mar 2014 15:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=b00+oq0P4Sk4aVuY3TcbZo2wK4tawRsyLTjizIOzfes=; b=psLhsAd1fvR/twOrRX5KPvhyyZsuhJNvh690NHcKLr7u+mMnV2uPIj4Gyv1o5xDB4u g+Ar1MHcwPjeGrsV49srcyaHXuZINikDcKzOUlYcw59cCY4YXMP2BDvHY0sFNhKBNjJU D+AdaITfBLVD/3/anz5G1e7AaD9aHPEkmeyXeJkUNBBnz45lpNj4ybX95it1QgdwA5Is Cq/IhsVQhc/tyEWcnmms80axBnCTR3MXK5TUveJf0Y1iI2tgtd+/sWBrzvnjfOmQHgZV wXivf7At5DVzgFk0bF0WZEs9HIZuWiwPX8wKaSMvp05GXBfc4wmde67T+rRAUUdzy6Lu 2WAw== MIME-Version: 1.0 X-Received: by 10.50.62.178 with SMTP id z18mr10543677igr.49.1395786328236; Tue, 25 Mar 2014 15:25:28 -0700 (PDT) Sender: oshogbo.vx@gmail.com Received: by 10.50.95.34 with HTTP; Tue, 25 Mar 2014 15:25:28 -0700 (PDT) In-Reply-To: <20140318213516.GA71491@stack.nl> References: <20140318213516.GA71491@stack.nl> Date: Tue, 25 Mar 2014 23:25:28 +0100 X-Google-Sender-Auth: y2FtSCV_tynAA_mUJ-fSV3uN4Ts Message-ID: Subject: Re: Hello fdclose From: Mariusz Zaborski To: Jilles Tjoelker Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:25:29 -0000 > Does this patch allow perl to stop writing to FILE._file? As pointed out > in > http://lists.freebsd.org/pipermail/freebsd-current/2013-January/039024.html > perlio.c in the perl source contains a function > PerlIOStdio_invalidate_fileno() that should modify a FILE such that > fclose() does not close the file descriptor but still frees all memory > (Perl has already called fflush()). Although using fdclose() could solve > this without touching the internals of FILE, this will make perlio.c > uglier with even more #ifdefs. Yes it should help. I start some work on it but i have some troubles with perl (I'm not perl hacker :(), so I will try to prepare some patch for it in the nearest feature. There are some other places also where we could use it. >> [snip] >> --- //depot/user/oshogbo/capsicum/lib/libc/stdio/Symbol.map 2013-06-28 08:51:28.000000000 0000 >> +++ /home/oshogbo/p4/capsicum/lib/libc/stdio/Symbol.map 2013-06-28 08:51:28.000000000 0000 >> @@ -156,6 +156,7 @@ >> putwc_l; >> putwchar_l; >> fmemopen; >> + fdclose; >> open_memstream; >> open_wmemstream; >> }; > > This should be in the FBSD_1.4 namespace (which does not exist yet). Oh - thx, so this will be good now? @@ -160,6 +160,10 @@ open_wmemstream; }; +FBSD_1.4 { + fdclose; +}; + >> [snip] >> --- //depot/user/oshogbo/capsicum/lib/libc/stdio/fclose.c 2013-06-28 08:51:28.000000000 0000 >> +++ /home/oshogbo/p4/capsicum/lib/libc/stdio/fclose.c 2013-06-28 08:51:28.000000000 0000 >> [snip] >> +int >> +fdclose(FILE *fp) >> +{ >> + int fd, r, err; >> + >> + if (fp->_flags == 0) { /* not open! */ >> + errno = EBADF; >> + return (EOF); >> + } >> + >> + r = 0; >> + FLOCKFILE(fp); >> + fd = fp->_file; >> + if (fp->_close != __sclose) { >> + r = EOF; >> + errno = EOPNOTSUPP; >> + } else if (fd < 0) { >> + r = EOF; >> + errno = EBADF; >> + } >> + if (r == EOF) { >> + err = errno; >> + (void)cleanfile(fp, true); >> + errno = err; >> + } else { >> + r = cleanfile(fp, false); >> + } >> FUNLOCKFILE(fp); >> + >> + return (r == 0 ? fd : r); > > If a file descriptor would be returned but cleanfile() returns an error > (e.g. write error on flush), the file descriptor is not returned and not > closed. > > I think that in cases where fdclose() would be used, it is essential > that the file descriptor is never closed. This means that the API needs > to be different so it can report a write error but still return a file > descriptor. One way to do this is to return the file descriptor by > reference. Another is to expect the application to call fileno() and not > return the descriptor from the new function. You have very good point. The first question is where function will return error: 1* When there is different _close function from std (it will behave like fclose with some errno) 2* When __sflush fails (and it will free structure) 3* When fd in structure is not correct (it will behave like fclose with some errno) I think those assumptions about when close fd are reasonable. When I wrote this function I discouse this with Pawel, and we decided that if _close function is different from std we should work same as fclose function plus return errno about EOPNOTSUPP. So in my opinion only second point is unwanted by us. So if __sflush fails we could not return any err (I don't thing this is wanted solution) or we could return error returned by __sflush and not free structure. In my opinion last option will be the best one. What you think Jilles? In this moment I don't like idea of changing API of this function. Cheers, Mariusz From owner-freebsd-arch@FreeBSD.ORG Tue Mar 25 22:29:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9EC9D40; Tue, 25 Mar 2014 22:29:49 +0000 (UTC) Received: from mail-ie0-x22c.google.com (mail-ie0-x22c.google.com [IPv6:2607:f8b0:4001:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 92B3FD66; Tue, 25 Mar 2014 22:29:49 +0000 (UTC) Received: by mail-ie0-f172.google.com with SMTP id as1so1084775iec.3 for ; Tue, 25 Mar 2014 15:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=rbMCoYMAkXVkvD4h6AfOoA5Kf2bQGmesakmjUbhzQhs=; b=lUjm8AiYw3Lk2OspwwOWpuDAHKSQ0f6Ww8HLMPOKcRoJfUDmgsUcWgKYQ0jOoVOLbf VMvQLtSjpSQkH3xh95zcoVCYxLP2LSAmQpL5GXEahe2gyvq31Tu2Jg5p36lkOTS+r823 GvFkgyqhkKqxak2bAEW98HVd/3sT1vGbD90XHIResERAaCm7otwtMxfd5ZMWrG2Wocnu eWoGFl3dprI8kbbi+hgqwHLSERCRTqNu8MmVbrrhWn4u05YXvxMpdLGLgxbM01ghf9we 5sKVpTdTsCeXnkHiQwXsl8mQMqL1kf/xUKwFsEDzvf8TsMYsQyvFjgSP7klFEeoDRmUF 5Ggw== MIME-Version: 1.0 X-Received: by 10.42.228.65 with SMTP id jd1mr4014935icb.62.1395786589086; Tue, 25 Mar 2014 15:29:49 -0700 (PDT) Sender: oshogbo.vx@gmail.com Received: by 10.50.95.34 with HTTP; Tue, 25 Mar 2014 15:29:49 -0700 (PDT) In-Reply-To: References: <201403181404.52197.jhb@freebsd.org> <201403191523.33275.jhb@freebsd.org> <86zjkkr5ma.fsf@nine.des.no> <86fvmbzlbn.fsf@nine.des.no> <86mwgj7aru.fsf@nine.des.no> <1395422417.81853.14.camel@revolution.hippie.lan> <98DE2314-1233-4C12-8346-5D1240B9BAAF@bsdimp.com> <86a9cj6oo5.fsf@nine.des.no> Date: Tue, 25 Mar 2014 23:29:49 +0100 X-Google-Sender-Auth: dtVUTIhiTd56UWKXqvv2AzvgNvk Message-ID: Subject: Re: Hello fdclose From: Mariusz Zaborski To: Kevin Oberman Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Warren Block , Ian Lepore , FreeBSD Current , freebsd-arch@freebsd.org, jilles@freebsd.org, =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:29:50 -0000 On 22 March 2014 08:04, Kevin Oberman wrote: > On Fri, Mar 21, 2014 at 5:51 PM, Dag-Erling Sm=F8rgrav wrote= : >> >> Warner Losh writes: >> > It does serve a useful purpose, though, which is why it has endured. >> > If you were to have a man page that said 'Putc(3) returns ...' then th= e >> > automated tools (and web links) that find Putc.3 wouldn't be able to >> > since >> > it doesn't exist. >> >> Moreover - if FreeBSD were written in Pascal, it might not matter, but >> in C, _exit(2) and _Exit(3) are two different functions. >> >> (I'm sure there are other examples without a leading underscore) >> >> (eww, starting a sentence with a non-alphabetic character would be even >> worse...) > > > 3Com was once a very important networking company. 3M still is. > > /me duck and runs. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com Hello, Thx guys for such big interested of the fdclose function :) When we decide how API will look and how the fdclose function will behave then I will correct man page. Thx, Mariusz From owner-freebsd-arch@FreeBSD.ORG Tue Mar 25 22:34:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F2C0109; Tue, 25 Mar 2014 22:34:09 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B1344E0A; Tue, 25 Mar 2014 22:34:08 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.8/8.14.8) with ESMTP id s2PMY3v0043464; Wed, 26 Mar 2014 00:34:03 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s2PMY3v0043464 Received: (from kostik@localhost) by tom.home (8.14.8/8.14.8/Submit) id s2PMY3E2043463; Wed, 26 Mar 2014 00:34:03 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 26 Mar 2014 00:34:03 +0200 From: Konstantin Belousov To: Mariusz Zaborski Subject: Re: Hello fdclose Message-ID: <20140325223403.GJ21331@kib.kiev.ua> References: <20140318213516.GA71491@stack.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="D85EWoRcDXv1uhH1" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-current@freebsd.org, Jilles Tjoelker , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:34:09 -0000 --D85EWoRcDXv1uhH1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 25, 2014 at 11:25:28PM +0100, Mariusz Zaborski wrote: > >> [snip] > >> --- //depot/user/oshogbo/capsicum/lib/libc/stdio/Symbol.map 2013-06-= 28 08:51:28.000000000 0000 > >> +++ /home/oshogbo/p4/capsicum/lib/libc/stdio/Symbol.map 2013-06-= 28 08:51:28.000000000 0000 > >> @@ -156,6 +156,7 @@ > >> putwc_l; > >> putwchar_l; > >> fmemopen; > >> + fdclose; > >> open_memstream; > >> open_wmemstream; > >> }; > > > > This should be in the FBSD_1.4 namespace (which does not exist yet). >=20 > Oh - thx, so this will be good now? >=20 > @@ -160,6 +160,10 @@ > open_wmemstream; > }; >=20 > +FBSD_1.4 { > + fdclose; > +}; > + You also need to update the lib/libc/Versions.def file. Look e.g. at the r226217 for the proper way to introduce new version. --D85EWoRcDXv1uhH1 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTMgRbAAoJEJDCuSvBvK1BIwkP/0U0VREO2spHRx4sihbXYDMb oHREQROUJgnbajBLDPKj+x+dvd7pTsp372fYwZeJqnZy/uKXGc6P4v2csKF4ujNS vxbEhVfWQZWKJbKf/+dQz+ubULvaGNjzEeXWRel2ZFblBFH80bwvndNn2WNKFDfX +TeZF+++rh6scscNSGa1R5TG6+fXbS8m83re8F8rfH98Q/jbezVAf0fKPqiB0KcB 3HsHgulPoGPq/Q6LZ7Pe3lSkPeuofjd8sdQbAPMzBd9aHedXHXIp0opMh9/l0pU3 Vnq2EThg8MHsn80tAZiQWio1JQSK1dw48V8haqJ+qxoiDUcoSsGJwrww/70Vy77M 4d8MXuVHDCgHli1l1rLgM6fV9tUviHlv9bUhropdNP0AXYg09YdKG1C/m2uZ/8X2 aCRXIikNBhzZb1MjDDrbIpX2XZSf+ZgeTyhBiRzby/5bhcqyjZpMd4lVYkR5IzVU pCt3hTVsZdQ53VFcvgEAovM+roQrnoYiW9Jf43rBYOB10j6Hx0nXDG7FM3H4oEMC JIJNuzobehzXYcPHRakT9xO/MC+GSaYgRMPqozQOSxlg1AnX08uybz4iFoghGvxP RIlG5SjU+8O788dIMJhoCbw9co1FvBOZqGSINY+YNzL7MDTgrVAi478Bs0aphsdy 9Cqn+0NcFspHLbKIQGNp =ojLX -----END PGP SIGNATURE----- --D85EWoRcDXv1uhH1-- From owner-freebsd-arch@FreeBSD.ORG Tue Mar 25 22:46:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA82D59C; Tue, 25 Mar 2014 22:46:07 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7DA84EF4; Tue, 25 Mar 2014 22:46:07 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id rd18so1091539iec.29 for ; Tue, 25 Mar 2014 15:46:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GTfqS51OVQ7zb4D0HiYFRk/2I/iGsqansg2pN0JNbCY=; b=p0On+1kQRbkP7FC8L1YYk+YON4Ocmv2TVuVB6lXSjrcwmyYBV/zs0NzcHiTdefwtoN DMz5dvTmUkKtPIeDzrUO905KDoe44DES9BYY5q1JxHSb50lVQk9cUa43QsB8Y7azRKsG G0aMfcExN530YNHbEk+eKy2qAPYQhMvmwKdFzPzcfRcVTxFn/cf1peiAf1vpGjWYZ4JT B9JOVXRSv7urwW9BjfXt7jcUidu+wxE5LKO6F9Scf7BrduDxtBT9gXbfY/vu7ApSvuWJ K1RimdjA66pc5g6qCaYZ6UBYFvzIDVNLvA5SfcfVd42mdjNBuxH3fgKA6RwQdSbzhzd1 Mz9A== MIME-Version: 1.0 X-Received: by 10.42.58.130 with SMTP id i2mr4184297ich.66.1395787566997; Tue, 25 Mar 2014 15:46:06 -0700 (PDT) Sender: oshogbo.vx@gmail.com Received: by 10.50.95.34 with HTTP; Tue, 25 Mar 2014 15:46:06 -0700 (PDT) In-Reply-To: <20140325223403.GJ21331@kib.kiev.ua> References: <20140318213516.GA71491@stack.nl> <20140325223403.GJ21331@kib.kiev.ua> Date: Tue, 25 Mar 2014 23:46:06 +0100 X-Google-Sender-Auth: B9YvPP6NjTIrKeJOm2JRH5D7g-w Message-ID: Subject: Re: Hello fdclose From: Mariusz Zaborski To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Current , Jilles Tjoelker , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 22:46:07 -0000 > > > +FBSD_1.4 { > > + fdclose; > > +}; > > + > > You also need to update the lib/libc/Versions.def file. Look e.g. at > the r226217 for the proper way to introduce new version. > Updated - thank you. Cheers, Mariusz From owner-freebsd-arch@FreeBSD.ORG Wed Mar 26 09:49:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 612BF6B5 for ; Wed, 26 Mar 2014 09:49:50 +0000 (UTC) Received: from smtp-newslist-212.md02.com (smtp-newslist-212.md02.com [209.172.40.212]) by mx1.freebsd.org (Postfix) with ESMTP id 3125BF52 for ; Wed, 26 Mar 2014 09:49:50 +0000 (UTC) Received: by smtp-newslist-212.md02.com id h6ahbs19sl8i for ; Wed, 26 Mar 2014 05:39:08 -0400 (envelope-from ) Subject: CCTV From: "Phil" To: freebsd-arch@freebsd.org Tag-id: 208377.3279879.2136996.59.m.9bbf62d9 IP-Address: 209.172.40.212 Interface-id: 443 Parent-id: 93133 Client-id: 208377 MIME-Version: 1.0 Message-Id: <20140326093908.8E1C28BD69@pmta4-1-08> Date: Wed, 26 Mar 2014 05:39:08 -0400 (EDT) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 09:49:50 -0000 Hi, We can install a CCTV at your house or business for the following price. 1 Camera system =C2=A3399 2 Camera system =C2=A3599 3 Camera system =C2=A3699 4 Camera system =C2=A3899 Would you like me to arrange this for you? You get the following with all our systems. - See the cameras on your mobile. - High quality cameras with wide lens. - High capacity storage. - Infrared night vision. - Includes all parts and labour. - Includes cameras and recorder. - 1 years hardware warranty. We can beat any like for like price. Our engineers can install a system anywhere in the UK* Regards, CCTV Warrington Telephone: 01204 216810 Website: www.cctvwarrington.co.uk Address: 3 Churchbank, Bolton, BL1 1HX * Additional costs for any location which is more than 50 miles from our= offices. Unsubscribe from this mailing list: http://link.mailanimal01.com/u/443/0= e038a683332c8b28eeff1b9767edb34a3a380962ac20e9f From owner-freebsd-arch@FreeBSD.ORG Wed Mar 26 16:42:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1690ECE; Wed, 26 Mar 2014 16:42:32 +0000 (UTC) Received: from land.berklix.org (land.berklix.org [144.76.10.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 803EBB1D; Wed, 26 Mar 2014 16:42:32 +0000 (UTC) Received: from mart.js.berklix.net (p5DCBEFA9.dip0.t-ipconnect.de [93.203.239.169]) (authenticated bits=128) by land.berklix.org (8.14.5/8.14.5) with ESMTP id s2QGf4Cd018544; Wed, 26 Mar 2014 16:41:05 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id s2QGgIjx068661; Wed, 26 Mar 2014 17:42:18 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.7/8.14.7) with ESMTP id s2QGg6uG069790; Wed, 26 Mar 2014 17:42:18 +0100 (CET) (envelope-from jhs@berklix.com) Message-Id: <201403261642.s2QGg6uG069790@fire.js.berklix.net> To: webmaster@freebsd.org Subject: Re: CCTV From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultants, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Wed, 26 Mar 2014 05:39:08 -0400." <20140326093908.8E1C28BD69@pmta4-1-08> Date: Wed, 26 Mar 2014 17:42:06 +0100 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 16:42:33 -0000 > From: "Phil" > Date: Wed, 26 Mar 2014 05:39:08 -0400 (EDT) "Phil" wrote: > Hi, > > We can install a CCTV at your house or business for the following price. Hi webmaster@freebsd.org, cc: freebsd-arch@freebsd.org (Not cc'd to spammer) Please for a while do not purge previous post: http://lists.freebsd.org/pipermail/freebsd-arch/2014-March/015193.html URL has been forwarded to UK nominet (registars' TLD .co.uk overseer) + to police in spammer's town, as evidence of illegal spam + false supressed identity, whois reports: "Registrant's address: The registrant is a non-trading individual who has opted to have their address omitted from the WHOIS service." The spammer's domain registrar needs more evidence before he will terminate his customer cctvwarrington.co.uk I suggest a number of you also forward the original spam with full headers to Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com Interleave replies below like a play script. Indent old text with "> ". Boycott Putin's Russia, invaders of Ukraine. From owner-freebsd-arch@FreeBSD.ORG Thu Mar 27 19:07:07 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3912DCDC for ; Thu, 27 Mar 2014 19:07:07 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0C1F38A1 for ; Thu, 27 Mar 2014 19:07:06 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WTFdp-0004B2-M4 for freebsd-arch@FreeBSD.org; Thu, 27 Mar 2014 19:07:05 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s2RJ73hW079051 for ; Thu, 27 Mar 2014 13:07:03 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX19GPfeqtAP0qHm5HalR4Avr Subject: Supporting variable-frequency event timers From: Ian Lepore To: freebsd-arch Content-Type: multipart/mixed; boundary="=-6fkTI/i94V3xVMnPriRG" Date: Thu, 27 Mar 2014 13:07:03 -0600 Message-ID: <1395947223.81853.121.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 19:07:07 -0000 --=-6fkTI/i94V3xVMnPriRG Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Increasingly, ARM systems are using techniques such as dynamic voltage and frequency scaling (DVFS) to save power. Sometimes the frequency changes affect the clocks being used as event timers. I'd like to commit the attached patch which allows an event timer driver to call et_change_frequency() to reconfigure the timer (if it's the active one) on all CPUs when the frequency changes. -- Ian --=-6fkTI/i94V3xVMnPriRG Content-Disposition: inline; filename="eventtimer_varfreq.diff" Content-Type: text/x-patch; name="eventtimer_varfreq.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: sys/sys/timeet.h =================================================================== --- sys/sys/timeet.h (revision 263112) +++ sys/sys/timeet.h (working copy) @@ -89,6 +89,7 @@ extern struct mtx et_eventtimers_mtx; /* Driver API */ int et_register(struct eventtimer *et); int et_deregister(struct eventtimer *et); +void et_change_frequency(struct eventtimer *et, uint64_t newfreq); /* Consumer API */ struct eventtimer *et_find(const char *name, int check, int want); int et_init(struct eventtimer *et, et_event_cb_t *event, Index: sys/sys/systm.h =================================================================== --- sys/sys/systm.h (revision 263112) +++ sys/sys/systm.h (working copy) @@ -168,6 +168,7 @@ struct ucred; struct uio; struct _jmp_buf; struct trapframe; +struct eventtimer; int setjmp(struct _jmp_buf *) __returns_twice; void longjmp(struct _jmp_buf *, int) __dead2; @@ -286,6 +287,7 @@ void cpu_stopprofclock(void); sbintime_t cpu_idleclock(void); void cpu_activeclock(void); void cpu_new_callout(int cpu, sbintime_t bt, sbintime_t bt_opt); +void cpu_et_frequency(struct eventtimer *et, uint64_t newfreq); extern int cpu_can_deep_sleep; extern int cpu_disable_deep_sleep; Index: sys/kern/kern_et.c =================================================================== --- sys/kern/kern_et.c (revision 263112) +++ sys/kern/kern_et.c (working copy) @@ -113,6 +113,18 @@ et_deregister(struct eventtimer *et) } /* + * Change the frequency of the given timer. If it is the active timer, + * reconfigure it on all CPUs (reschedules all current events based on the new + * timer frequency). + */ +void +et_change_frequency(struct eventtimer *et, uint64_t newfreq) +{ + + cpu_et_frequency(et, newfreq); +} + +/* * Find free event timer hardware with specified parameters. */ struct eventtimer * Index: sys/kern/kern_clocksource.c =================================================================== --- sys/kern/kern_clocksource.c (revision 263112) +++ sys/kern/kern_clocksource.c (working copy) @@ -799,6 +799,25 @@ cpu_activeclock(void) spinlock_exit(); } +/* + * Change the frequency of the given timer. This changes et->et_frequency and + * if et is the active timer it reconfigures the timer on all CPUs. This is + * intended to be a private interface for the use of et_change_frequency() only. + */ +void +cpu_et_frequency(struct eventtimer *et, uint64_t newfreq) +{ + + ET_LOCK(); + if (et == timer) { + configtimer(0); + et->et_frequency = newfreq; + configtimer(1); + } else + et->et_frequency = newfreq; + ET_UNLOCK(); +} + #ifdef KDTRACE_HOOKS void clocksource_cyc_set(const struct bintime *bt) --=-6fkTI/i94V3xVMnPriRG-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 05:13:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0743D1D33 for ; Tue, 1 Apr 2014 05:13:40 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe001.messaging.microsoft.com [216.32.181.181]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B47EABB1 for ; Tue, 1 Apr 2014 05:13:39 +0000 (UTC) Received: from mail184-ch1-R.bigfish.com (10.43.68.252) by CH1EHSOBE007.bigfish.com (10.43.70.57) with Microsoft SMTP Server id 14.1.225.22; Tue, 1 Apr 2014 05:13:32 +0000 Received: from mail184-ch1 (localhost [127.0.0.1]) by mail184-ch1-R.bigfish.com (Postfix) with ESMTP id 492A12601FD for ; Tue, 1 Apr 2014 05:13:32 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h2148h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kzz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1dc1h1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24d7h2516h2545h255eh25cch25f6h2605h268bh1155h) Received-SPF: softfail (mail184-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail184-ch1 (localhost.localdomain [127.0.0.1]) by mail184-ch1 (MessageSwitch) id 1396329210401438_927; Tue, 1 Apr 2014 05:13:30 +0000 (UTC) Received: from CH1EHSMHS001.bigfish.com (snatpool2.int.messaging.microsoft.com [10.43.68.236]) by mail184-ch1.bigfish.com (Postfix) with ESMTP id 5CD0EC004F for ; Tue, 1 Apr 2014 05:13:30 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS001.bigfish.com (10.43.70.1) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 1 Apr 2014 05:13:29 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Mon, 31 Mar 2014 22:13:28 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s315DSV06440; Mon, 31 Mar 2014 22:13:28 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id F20F958097; Mon, 31 Mar 2014 22:13:27 -0700 (PDT) To: Subject: make WITH[OUT]_* more useful? X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Mon, 31 Mar 2014 22:13:27 -0700 From: Simon Gerraty Message-ID: <20140401051327.F20F958097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 05:13:40 -0000 I really like the idea of WITH[OUT]_* knobs translating to MK_* knobs, but I find the current implementation much less useful than I think it could be. Not the least of its problems is being implemented in bsd.own.mk which ties policy and mechanism together. It is not always (rarely) safe to include the in-tree bsd.own.mk which means that in many cases you cannot rely on MK_* at all, but have to re-implement the WITH_* vs WITHOUT_* logic repeatedly. It is also generally bad to include bsd.own.mk from sys.mk, which means any knobs you need early must re-implement the WITH_* vs WITHOUT_* logic repeatedly. contrib/bmake/mk/options.mk is an example of a more generic implementation with (I think) some advantages. The key semantic changes are (DOMINANT_* is from a newer version than in contrib): # NO_* takes precedence # If both WITH_* and WITHOUT_* are defined, WITHOUT_ wins unless # DOMINANT_* is set to "yes" # Otherwise WITH_* and WITHOUT_* override the default. and MK_* can be pre-set without causing an error. The key advantage is that the mechanism is separate from the policy. You can thus have knobs that get set much earlier (eg during sys.mk) and other knobs that get set later. Ie. both sys.mk and bsd.own.mk can include options.mk to process options that they care about, allowing MK_* to be used more consistently - you could use different prefix to avoid overlap, but that's probably not necessary. You can in fact have per-makefile option lists if you want (see contrib/bmake/Makefile) Thoughts? From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 06:43:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EF0713A9 for ; Tue, 1 Apr 2014 06:43:34 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8459B6BB for ; Tue, 1 Apr 2014 06:43:34 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id k14so6977442wgh.10 for ; Mon, 31 Mar 2014 23:43:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=/DWIJhCHmV/+CpC6vdH7c7UtwpFlDMrRye0MzGbFU2Y=; b=ioYGFq2i8fcgmfg23NyMc7CUsGteGt3bKRy+JafyXOBmwlxZgL63DZscQ8GMykpm0V a2qsyBerujdkQiOLvTGxY7yHASZ9aUhcxRsGSl/hKOv07gBYjyDYGrbc7vFanxfi0PPR xHaAfFMf373f3gyABMNdjQmGsliozIqT2LKoY3lF/Etl7iCUENp7HwRprNuyDSX9wDKR 3E97dh1ccJme+qa7vvQiQttlgs88V7qyUJg4gKsOLf8SLevugR2EVaYK5NuWgN4VOM+R 894EnN05vT+SqUbUWwf9WFzMALwtnk3SzHdjgczpiFH9UKrNQ2kNi5hmdu/f9iOoX9WC cWEA== X-Received: by 10.180.109.107 with SMTP id hr11mr6728723wib.4.1396334612795; Mon, 31 Mar 2014 23:43:32 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id gz1sm31216304wib.14.2014.03.31.23.43.31 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 31 Mar 2014 23:43:31 -0700 (PDT) Sender: Baptiste Daroussin Date: Tue, 1 Apr 2014 08:43:16 +0200 From: Baptiste Daroussin To: Simon Gerraty Subject: Re: make WITH[OUT]_* more useful? Message-ID: <20140401064316.GQ99393@ivaldir.etoilebsd.net> References: <20140401051327.F20F958097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9+VnUxDxRuy97YQ+" Content-Disposition: inline In-Reply-To: <20140401051327.F20F958097@chaos.jnpr.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 06:43:35 -0000 --9+VnUxDxRuy97YQ+ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 31, 2014 at 10:13:27PM -0700, Simon Gerraty wrote: > I really like the idea of WITH[OUT]_* knobs translating to MK_* knobs, > but I find the current implementation much less useful than I think it > could be. Not the least of its problems is being implemented in > bsd.own.mk which ties policy and mechanism together. >=20 > It is not always (rarely) safe to include the in-tree bsd.own.mk which > means that in many cases you cannot rely on MK_* at all, but have to > re-implement the WITH_* vs WITHOUT_* logic repeatedly. >=20 > It is also generally bad to include bsd.own.mk from sys.mk, which means > any knobs you need early must re-implement the WITH_* vs WITHOUT_* logic > repeatedly. >=20 > contrib/bmake/mk/options.mk is an example of a more generic > implementation with (I think) some advantages. >=20 > The key semantic changes are (DOMINANT_* is from a newer version > than in contrib): >=20 > # NO_* takes precedence > # If both WITH_* and WITHOUT_* are defined, WITHOUT_ wins unless > # DOMINANT_* is set to "yes" > # Otherwise WITH_* and WITHOUT_* override the default. > and > MK_* can be pre-set without causing an error. >=20 > The key advantage is that the mechanism is separate from the policy. > You can thus have knobs that get set much earlier (eg during sys.mk) > and other knobs that get set later. Ie. both sys.mk and bsd.own.mk can > include options.mk to process options that they care about, allowing > MK_* to be used more consistently - you could use different prefix to > avoid overlap, but that's probably not necessary. >=20 > You can in fact have per-makefile option lists if you want (see > contrib/bmake/Makefile)=20 >=20 > Thoughts? I would be interested in having your opinion on what we did for ports. Basically we have in the end a variable: PORT_OPTIONS that contains the the options that are considered like "MK_*" =3D yes and all the one considerer = as are not it. one can activate variables via make.conf: OPTIONS_SET=3D OPT1 OPT2 OPTIONS_UNSET=3D OPT3 We added a couple of sugar so that options are not on yes/no but can be a selection in a list etc. Can be looked at here: http://svnweb.freebsd.org/ports/head/Mk/bsd.options.= mk Having it creating in the end the MK_* variables would be really realy easy. regards, Bapt --9+VnUxDxRuy97YQ+ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlM6YAQACgkQ8kTtMUmk6EwqCwCcC19iLPOR3T7h+CxzzimZZ179 FDEAn3LXCkxynr91lPnVyRGhS9qtWIc3 =YKB2 -----END PGP SIGNATURE----- --9+VnUxDxRuy97YQ+-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 14:22:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 706435EA for ; Tue, 1 Apr 2014 14:22:22 +0000 (UTC) Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com [209.85.192.176]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41FA1A2A for ; Tue, 1 Apr 2014 14:22:21 +0000 (UTC) Received: by mail-pd0-f176.google.com with SMTP id r10so9566157pdi.7 for ; Tue, 01 Apr 2014 07:22:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=JQW4rEOlfUPXsYL+wsv/KbEs+3UOghKgZc/Qawnbqyg=; b=XfyZQu8oq7T+wopJhu8YFsUCkRDEtEWDJtGDVgbBquT/6D1IiC/n0lEH3wzJYqwH3O Ey3Cg7Z1ORMMnhsIB1sJg1HNjaTEqQp+oXbSl5XTEyiV/ZGV3sHU1KXZY6+73MaHfa1t YIIxNyBic3l7TJfHu8UO7GKlTRYDcwB7ErJRMqiRLR95ignwAJc/FLWtudAe8WlpSVkH A8G4x7QntDFR4Zeuf3kZ6DjD4+vOrHvZBdyb/XSrzf2jkHhPUpP60/LFAzBy322g9VLz DLKMoQDIAcsg1jgzT9zhDy5/Ek2rbZ8+t+mb4x9pUeFXyR34G22ZHjTAwJbqz53Gbg44 UWMw== X-Gm-Message-State: ALoCoQlk+eOpTtEuNzG2k/Ux6HHuNGUe8dtesA2H27hgn4ARUPtvtMDDJ5YNtyBDUJDaZX3ecTIp X-Received: by 10.66.163.2 with SMTP id ye2mr15576578pab.110.1396362141462; Tue, 01 Apr 2014 07:22:21 -0700 (PDT) Received: from [10.64.26.254] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id my6sm50345980pbc.36.2014.04.01.07.22.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 07:22:20 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140401064316.GQ99393@ivaldir.etoilebsd.net> Date: Tue, 1 Apr 2014 08:22:18 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org, Simon Gerraty X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 14:22:22 -0000 On Apr 1, 2014, at 12:43 AM, Baptiste Daroussin = wrote: > On Mon, Mar 31, 2014 at 10:13:27PM -0700, Simon Gerraty wrote: >> I really like the idea of WITH[OUT]_* knobs translating to MK_* = knobs, >> but I find the current implementation much less useful than I think = it >> could be. Not the least of its problems is being implemented in >> bsd.own.mk which ties policy and mechanism together. >>=20 >> It is not always (rarely) safe to include the in-tree bsd.own.mk = which >> means that in many cases you cannot rely on MK_* at all, but have to >> re-implement the WITH_* vs WITHOUT_* logic repeatedly. Where, exactly, do we do this? In my recent gripping I=92ve found almost no instances of this. The ones I did find were trivial to fix, and I = have either committed or will commit fixes for them. So perhaps you could come up with an actual example of the problem you are talking about. >> It is also generally bad to include bsd.own.mk from sys.mk, which = means >> any knobs you need early must re-implement the WITH_* vs WITHOUT_* = logic >> repeatedly. Again, I find no evidence of this in the tree, can you cite specific = examples? I=92ve been going through the tree fixing many abuses that did this = when, in fact, they didn=92t need to. >> contrib/bmake/mk/options.mk is an example of a more generic >> implementation with (I think) some advantages. >>=20 >> The key semantic changes are (DOMINANT_* is from a newer version >> than in contrib): >>=20 >> # NO_* takes precedence >> # If both WITH_* and WITHOUT_* are defined, WITHOUT_ wins unless >> # DOMINANT_* is set to "yes" >> # Otherwise WITH_* and WITHOUT_* override the default. >> and >> MK_* can be pre-set without causing an error. I mostly hate this. Specifically, I don=92t like the DOMINANT bits. They = are unnecessary. WITH/WITHOUT is a user-only knob. The build system should never use it, = but always set MK_* directly. I recently fixed it so the build system can start = doing that. This solves the WITH and WITHOUT problem internally. I have a round of changes that I=92m pushing in this morning once I = check to make sure that universe completed on them successfully. >> The key advantage is that the mechanism is separate from the policy. >> You can thus have knobs that get set much earlier (eg during sys.mk) >> and other knobs that get set later. Ie. both sys.mk and bsd.own.mk = can >> include options.mk to process options that they care about, allowing >> MK_* to be used more consistently - you could use different prefix to >> avoid overlap, but that's probably not necessary. I=92m still not sure I see the big win. >> You can in fact have per-makefile option lists if you want (see >> contrib/bmake/Makefile)=20 Since the base system is supposed to be one, integrated whole, I=92m not = sure what this will buy you... >> Thoughts? >=20 > I would be interested in having your opinion on what we did for ports. >=20 > Basically we have in the end a variable: PORT_OPTIONS that contains = the the > options that are considered like "MK_*" =3D yes and all the one = considerer as > are not it. There is overlap in concept between the PORT_OPTIONS, but the MK/NO = knobs control more fundamental and sometimes global options to the build. = Also, there are thousands of instances of PORT_OPTIONS in a large port build, one = for each port. We have one set for the src tree. > one can activate variables via make.conf: > OPTIONS_SET=3D OPT1 OPT2 > OPTIONS_UNSET=3D OPT3 >=20 > We added a couple of sugar so that options are not on yes/no but can = be a > selection in a list etc. >=20 > Can be looked at here: = http://svnweb.freebsd.org/ports/head/Mk/bsd.options.mk >=20 > Having it creating in the end the MK_* variables would be really realy = easy. This is a potentially interesting concept, but I don=92t think it scales = well to the src tree. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 14:55:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2092715 for ; Tue, 1 Apr 2014 14:55:20 +0000 (UTC) Received: from smtp-newslist-212.md02.com (smtp-newslist-212.md02.com [209.172.40.212]) by mx1.freebsd.org (Postfix) with ESMTP id 77BA1E54 for ; Tue, 1 Apr 2014 14:55:20 +0000 (UTC) Received: by smtp-newslist-212.md02.com id h7b9lg19sl8o for ; Tue, 1 Apr 2014 10:55:14 -0400 (envelope-from ) Subject: CCTV From: "Phil Law" <3@boltoncctv.com> To: freebsd-arch@freebsd.org Tag-id: 209494.3306923.2147983.59.m.3b9f04fa IP-Address: 209.172.40.212 Interface-id: 443 Parent-id: 93133 Client-id: 209494 MIME-Version: 1.0 Message-Id: <20140401145514.453968BE18@pmta4-1-06> Date: Tue, 1 Apr 2014 10:55:14 -0400 (EDT) Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 14:55:20 -0000 Hi, We can install a CCTV at your house or business for the following price. 1 Camera system =C2=A3370 2 Camera system =C2=A3420 3 Camera system =C2=A3595 4 Camera system =C2=A3645 Would you like me to arrange this for you? You get the following with all our systems. - See the cameras on your mobile. - High quality cameras with wide lens. - High capacity storage. - Infrared night vision. - Includes all parts and labour. - Includes cameras and recorder. - 1 years hardware warranty. We can beat any like for like price. Our engineers can install a system anywhere in the UK* Regards, Bolton CCTV Telephone: 01204 216810 Website: http://link.mailanimal01.com/c/443/0208232bdba84e2c76c0a1d09ccf= 1643b2aa54783420edd279d4a3b3d0148ef5 Address: 3 Churchbank, Bolton, BL1 1HX * Additional costs for any location which is more than 50 miles from our= offices. Unsubscribe from this mailing list: http://link.mailanimal01.com/u/443/0= 208232bdba84e2c76c0a1d09ccf1643a3a380962ac20e9f From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 16:59:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96B6DAAE; Tue, 1 Apr 2014 16:59:48 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe005.messaging.microsoft.com [216.32.181.185]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BA41EC8; Tue, 1 Apr 2014 16:59:47 +0000 (UTC) Received: from mail174-ch1-R.bigfish.com (10.43.68.246) by CH1EHSOBE004.bigfish.com (10.43.70.54) with Microsoft SMTP Server id 14.1.225.22; Tue, 1 Apr 2014 16:29:29 +0000 Received: from mail174-ch1 (localhost [127.0.0.1]) by mail174-ch1-R.bigfish.com (Postfix) with ESMTP id 01E511004D7; Tue, 1 Apr 2014 16:29:29 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kzz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail174-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail174-ch1 (localhost.localdomain [127.0.0.1]) by mail174-ch1 (MessageSwitch) id 1396369766733884_5064; Tue, 1 Apr 2014 16:29:26 +0000 (UTC) Received: from CH1EHSMHS031.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.244]) by mail174-ch1.bigfish.com (Postfix) with ESMTP id AED00C00B2; Tue, 1 Apr 2014 16:29:26 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS031.bigfish.com (10.43.70.31) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 1 Apr 2014 16:29:22 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Tue, 1 Apr 2014 09:29:21 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s31GTCV98295; Tue, 1 Apr 2014 09:29:16 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 7A9D058097; Tue, 1 Apr 2014 09:29:12 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Tue, 01 Apr 2014 08:22:18 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 1 Apr 2014 09:29:12 -0700 Message-ID: <20140401162912.7A9D058097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 16:59:48 -0000 On Tue, 1 Apr 2014 08:22:18 -0600, Warner Losh writes: >>> re-implement the WITH_* vs WITHOUT_* logic repeatedly. > >Where, exactly, do we do this? In my recent gripping I=92ve found almost I hit this just about every possible way while trying to support WITH[OUT]_BMAKE in such a way that people could build head on 9 etc. Incluing in-tree bsd.own.mk so MK_BMAKE would get set - broke building on 9 IIRC. Being able to simply do MK_BMAKE?= {yes,no} would have been best solution. Also I normally want to built WITH_CTF, but of course you cannot simply put that in make.conf you have to .if !defined(WITHOUT_CTF) && !defined(NO_CTF) WITH_CTF=yes .endif else you get errors during buildworld. >>> It is also generally bad to include bsd.own.mk from sys.mk, which means >>> any knobs you need early must re-implement the WITH_* vs WITHOUT_* logic >>> repeatedly. > >Again, I find no evidence of this in the tree, can you cite specific exampl= It isn't done anymore (was certainly done back in 2.x, don't recall when it was removed), which is good, but means that sys.mk cannot use any MK_* that the user can influence via WITH[OUT]_*. That's not so much a problem for existing options, but makes it impractical to leverage the mechanism for things you might want to set during sys.mk - like whether to use meta mode or auto objdir creation. >I mostly hate this. Specifically, I don=92t like the DOMINANT bits. They ar= >e unnecessary. I mostly agree - I find it quite reasonable to simply allow WITHOUT to win. DOMINANT_* is just an "out" in case there's some future case I haven't thought of. The default behavior remains that WITHOUT wins. >WITH/WITHOUT is a user-only knob. Agreed, but the implementation renders it user unfriendly. If everywhere I want to set a default (eg make.conf) I have to do a dance like .if !defined(WITHOUT_CTF) && !defined(NO_CTF) WITH_CTF=yes .endif it isn't exactly helping me as much as it could. If the build stops using WITH/WITHOUT internally that probably helps. >The build system should never use it, but always >set MK_* directly. Agreed - that would be most useful and is one of the main changes in my version. >I recently fixed it so the build system can start doing = >that. This solves >the WITH and WITHOUT problem internally. That's good - being able to set MK_* directly without causing error does address the issue for the build itself. Alone though it doesn't necessarily make life any better for users who (per my example above) want to set defaults like WITH_CTF= in make.conf but occasionally override them. Unless they too are supposed to set MK_* directly but then what is the point of WITH[OUT]_* ? >I'm still not sure I see the big win. I have a number of knobs to be set during sys.mk AUTO_OBJ automatically create objdirs META_MODE use meta mode setting MK_* is fine, but it is nice if you allow the user to interact using WITH[OUT]_*, and for that it is best if sys.mk can safely include something like options.mk Of course the user can learn to MK_AUTO_OBJ=no but that simply devalues WITH[OUT]_* It is a neat mechanism, that with some minor tweaks could be much more useful. Baptiste writes: >> I would be interested in having your opinion on what we did for ports. It is a bit more elaborate (422 lines vs 59 in options.mk) that's probably a necessity for ports, but not so sure about inclusion by sys.mk From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 18:58:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEEDB12F for ; Tue, 1 Apr 2014 18:58:01 +0000 (UTC) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F3FED3E for ; Tue, 1 Apr 2014 18:58:01 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id kq14so10270904pab.37 for ; Tue, 01 Apr 2014 11:58:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=ep8exWb1lFv+AunpPzCNvkzew7r+YL1GoqfBDATXms8=; b=SP3YrJDQJ3lV2VBeagQRIv2GYLTXppT3R+2xymh5LrZA+MNxCr01iJGmSfBEfDemiS KQFo7jf6zymBp5eZYtQOfi41BRuYq97zyucOTWdz5wFjWhQOUmh9ofuLoEtTOs6Qf+zD VhPtdqfySSQzJLTHG0h4JpocD7O7tufc4D53gFWJ0hfJwc+sBGzTabepX3e6l7oazWgI bI/TJdIlp7LUWvKntuIbmFrw2QPG+2nkGfxAy2t60Me2iCLS/XkuqyXeirGuy8bN6GWr FWGHJzd1tnDpzygKhUEUENZVxX/5IH2ZydHJ51VLPkjGp67jnmcFx0eth5HE/Yh+xXVd CUkQ== X-Gm-Message-State: ALoCoQl6bxh8JIr0L5pwcudC/+JLeRaavW0/q4tm+a3lvL7QdSXyMWfQRdd/McLJyLw/KRaCsrJ2 X-Received: by 10.66.146.105 with SMTP id tb9mr3459836pab.157.1396378680783; Tue, 01 Apr 2014 11:58:00 -0700 (PDT) Received: from [10.64.24.154] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id aj7sm59696490pad.29.2014.04.01.11.57.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 11:58:00 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140401162912.7A9D058097@chaos.jnpr.net> Date: Tue, 1 Apr 2014 12:57:57 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 18:58:01 -0000 On Apr 1, 2014, at 10:29 AM, Simon J. Gerraty wrote: >=20 > On Tue, 1 Apr 2014 08:22:18 -0600, Warner Losh writes: >>>> re-implement the WITH_* vs WITHOUT_* logic repeatedly. >>=20 >> Where, exactly, do we do this? In my recent gripping I=3D92ve found = almost >=20 > I hit this just about every possible way while trying to support > WITH[OUT]_BMAKE=20 > in such a way that people could build head on 9 etc. > Incluing in-tree bsd.own.mk so MK_BMAKE would get set - broke building > on 9 IIRC. >=20 > Being able to simply do MK_BMAKE?=3D {yes,no} would have been best = solution. >=20 > Also I normally want to built WITH_CTF, but of course you cannot = simply > put that in make.conf you have to=20 >=20 > .if !defined(WITHOUT_CTF) && !defined(NO_CTF) > WITH_CTF=3Dyes > .endif >=20 > else you get errors during build world. That=92s a bug I plan on fixing. But an interesting one because much of = the extra garbage there is due to the duality in the build system where we bogusly = (IMHO) set WITHOUT_CTF on the make command line, plus have an extra special NO_CTF option which does the same sort of thing. Bugs in bmake prevent the simple undefining of certain symbols on the = command line, which makes at least part of the above construct necessary. >>>> It is also generally bad to include bsd.own.mk from sys.mk, which = means >>>> any knobs you need early must re-implement the WITH_* vs WITHOUT_* = logic >>>> repeatedly. >>=20 >> Again, I find no evidence of this in the tree, can you cite specific = exampl=3D >=20 > It isn't done anymore (was certainly done back in 2.x, don't recall = when > it was removed), which is good, but means that sys.mk cannot use > any MK_* that the user can influence via WITH[OUT]_*. =20 > That's not so much a problem for existing options, but makes it > impractical to leverage the mechanism for things you might want to set > during sys.mk - like whether to use meta mode or auto objdir creation. I=92ll have to think about this point. It is a good point, but I=92m = unsure how the proposed changes would fix that. Perhaps you could explain that a = bit. >> I mostly hate this. Specifically, I don=3D92t like the DOMINANT bits. = They ar=3D >> e unnecessary. >=20 > I mostly agree - I find it quite reasonable to simply allow WITHOUT to = win. > DOMINANT_* is just an "out" in case there's some future case I haven't > thought of. The default behavior remains that WITHOUT wins. >=20 >> WITH/WITHOUT is a user-only knob.=20 >=20 > Agreed, but the implementation renders it user unfriendly. > If everywhere I want to set a default (eg make.conf) I have to do a > dance like >=20 > .if !defined(WITHOUT_CTF) && !defined(NO_CTF) > WITH_CTF=3Dyes > .endif >=20 > it isn't exactly helping me as much as it could. > If the build stops using WITH/WITHOUT internally that probably helps. Yea, that=92s just a bug=85 We have severe options like this, but CTF is = one of the more annoying. >> The build system should never use it, but always >> set MK_* directly.=20 >=20 > Agreed - that would be most useful and is one of the main changes in = my > version. Already in the tree. >> I recently fixed it so the build system can start doing =3D >> that. This solves >> the WITH and WITHOUT problem internally. >=20 > That's good - being able to set MK_* directly without causing error > does address the issue for the build itself. >=20 > Alone though it doesn't necessarily make life any better for users > who (per my example above) want to set defaults like > WITH_CTF=3D in make.conf but occasionally override them. Unless they = too > are supposed to set MK_* directly but then what is the point of > WITH[OUT]_* ? Setting defaults in make.conf, and then overriding them is tough because bmake doesn't have a tracking system to know where in the food-chain different variables are set. However, this is also a good point, but I must be being dense to see how your proposal would fix this. >> I'm still not sure I see the big win. >=20 > I have a number of knobs to be set during sys.mk > AUTO_OBJ automatically create objdirs > META_MODE use meta mode objdirs set in sys.mk? I thought that was done bad.obj.mk since it isn=92t= part of the global system. META_MODE might belong there. > setting MK_* is fine, but it is nice if you allow the user to interact > using WITH[OUT]_*, and for that it is best if sys.mk can safely = include > something like options.mk Not sure how creating a new file solves the bsd.own.mk problem, apart perhaps from some name space pollution differences. > Of course the user can learn to MK_AUTO_OBJ=3Dno but that simply = devalues > WITH[OUT]_*=20 >=20 > It is a neat mechanism, that with some minor tweaks could be much more > useful. It is neat and simple now, but the WITH/WITHOUT stuff was neat and = simple when we started and it has devolved as people have hacked on it without proper care. > Baptiste writes: >>> I would be interested in having your opinion on what we did for = ports. >=20 > It is a bit more elaborate (422 lines vs 59 in options.mk) > that's probably a necessity for ports, but not so sure about inclusion > by sys.mk Yea, back to the point I don=92t understand: how is your new file going = to be materially different than the current mechanism. I=92m OK with change, = but I need to understand how the change is going to make things better. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 19:25:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0CC2F89E for ; Tue, 1 Apr 2014 19:25:05 +0000 (UTC) Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 93886FBC for ; Tue, 1 Apr 2014 19:25:04 +0000 (UTC) Received: by mail-we0-f170.google.com with SMTP id w61so6848495wes.29 for ; Tue, 01 Apr 2014 12:25:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Zr/o8zRxGNWbsymSoHSbjUJIiomPxxiVl2EcxeG4VQg=; b=tS5A0dArxw6HWQTkvMFRsJgtGi4QG1iSl5797HlQF72OgWiGg64mdzuI2cZ0WsH/vb YgALAZN3qZqtwBOA66keFwJqQ/kX/gtrZpjMv00dNfpvRRyQLKYbdzRtUxFaIHifJ4pL otHXRQXccVjYQaCL9w4nUz37t7Ej8F7lNHjBWnaabRlnojr1qHbZRyF5WS52QRz/W1KI TSBuwhos6ugS/vJ6BjIY3gyee21gKVl4aElN7JeyDRgc1g3Qdch+Cx/81T3y/O83ghaA wnPu1q7RQ5rSA3RJuiy1p/6V/0PH+sC2W2EuMFWLcUb5Tc8V1rbKS8mm3RNOefKGdmgd s1mA== X-Received: by 10.180.103.227 with SMTP id fz3mr22374258wib.29.1396380302910; Tue, 01 Apr 2014 12:25:02 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id pm5sm5761167wjc.11.2014.04.01.12.24.59 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Apr 2014 12:25:01 -0700 (PDT) Sender: Baptiste Daroussin Date: Tue, 1 Apr 2014 21:24:43 +0200 From: Baptiste Daroussin To: "Simon J. Gerraty" Subject: Re: make WITH[OUT]_* more useful? Message-ID: <20140401192443.GV99393@ivaldir.etoilebsd.net> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4QzzFpAwbwI4hdN6" Content-Disposition: inline In-Reply-To: <20140401162912.7A9D058097@chaos.jnpr.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 19:25:05 -0000 --4QzzFpAwbwI4hdN6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 01, 2014 at 09:29:12AM -0700, Simon J. Gerraty wrote: >=20 > On Tue, 1 Apr 2014 08:22:18 -0600, Warner Losh writes: > >>> re-implement the WITH_* vs WITHOUT_* logic repeatedly. > > > >Where, exactly, do we do this? In my recent gripping I=3D92ve found almo= st >=20 > I hit this just about every possible way while trying to support > WITH[OUT]_BMAKE=20 > in such a way that people could build head on 9 etc. > Incluing in-tree bsd.own.mk so MK_BMAKE would get set - broke building > on 9 IIRC. >=20 > Being able to simply do MK_BMAKE?=3D {yes,no} would have been best soluti= on. >=20 > Also I normally want to built WITH_CTF, but of course you cannot simply > put that in make.conf you have to=20 >=20 > .if !defined(WITHOUT_CTF) && !defined(NO_CTF) > WITH_CTF=3Dyes > .endif >=20 > else you get errors during buildworld. >=20 > >>> It is also generally bad to include bsd.own.mk from sys.mk, which mea= ns > >>> any knobs you need early must re-implement the WITH_* vs WITHOUT_* lo= gic > >>> repeatedly. > > > >Again, I find no evidence of this in the tree, can you cite specific exa= mpl=3D >=20 > It isn't done anymore (was certainly done back in 2.x, don't recall when > it was removed), which is good, but means that sys.mk cannot use > any MK_* that the user can influence via WITH[OUT]_*. =20 > That's not so much a problem for existing options, but makes it > impractical to leverage the mechanism for things you might want to set > during sys.mk - like whether to use meta mode or auto objdir creation. >=20 > >I mostly hate this. Specifically, I don=3D92t like the DOMINANT bits. Th= ey ar=3D > >e unnecessary. >=20 > I mostly agree - I find it quite reasonable to simply allow WITHOUT to wi= n. > DOMINANT_* is just an "out" in case there's some future case I haven't > thought of. The default behavior remains that WITHOUT wins. >=20 > >WITH/WITHOUT is a user-only knob.=20 >=20 > Agreed, but the implementation renders it user unfriendly. > If everywhere I want to set a default (eg make.conf) I have to do a > dance like >=20 > .if !defined(WITHOUT_CTF) && !defined(NO_CTF) > WITH_CTF=3Dyes > .endif >=20 > it isn't exactly helping me as much as it could. > If the build stops using WITH/WITHOUT internally that probably helps. >=20 > >The build system should never use it, but always > >set MK_* directly.=20 >=20 > Agreed - that would be most useful and is one of the main changes in my > version. >=20 > >I recently fixed it so the build system can start doing =3D > >that. This solves > >the WITH and WITHOUT problem internally. >=20 > That's good - being able to set MK_* directly without causing error > does address the issue for the build itself. >=20 > Alone though it doesn't necessarily make life any better for users > who (per my example above) want to set defaults like > WITH_CTF=3D in make.conf but occasionally override them. Unless they too > are supposed to set MK_* directly but then what is the point of > WITH[OUT]_* ? >=20 > >I'm still not sure I see the big win. >=20 > I have a number of knobs to be set during sys.mk > AUTO_OBJ automatically create objdirs > META_MODE use meta mode >=20 > setting MK_* is fine, but it is nice if you allow the user to interact > using WITH[OUT]_*, and for that it is best if sys.mk can safely include > something like options.mk >=20 > Of course the user can learn to MK_AUTO_OBJ=3Dno but that simply devalues > WITH[OUT]_*=20 >=20 > It is a neat mechanism, that with some minor tweaks could be much more > useful. >=20 > Baptiste writes: > >> I would be interested in having your opinion on what we did for ports. >=20 > It is a bit more elaborate (422 lines vs 59 in options.mk) > that's probably a necessity for ports, but not so sure about inclusion > by sys.mk >=20 >=20 >=20 >=20 >=20 it is more elaborate because it sets ports only things which is not needed = at all for src. It also handles compat with precious versions of options. Extracting the pure framework will lead to around 30 lines no more. and pro= bably way less. I can make a PoC if you want regards, Bapt --4QzzFpAwbwI4hdN6 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlM7EnsACgkQ8kTtMUmk6ExJFgCgrSL1uGMr4GiESjMyTJEs7fcD dgQAnAtv81RIu7mAU0+BPiTAdBsCbk6b =ccaq -----END PGP SIGNATURE----- --4QzzFpAwbwI4hdN6-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 19:40:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48544EFF for ; Tue, 1 Apr 2014 19:40:58 +0000 (UTC) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 00884171 for ; Tue, 1 Apr 2014 19:40:58 +0000 (UTC) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 35F4125D3810; Tue, 1 Apr 2014 19:40:55 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 60C5FC22B96; Tue, 1 Apr 2014 19:40:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id hR2bWhF4U6vF; Tue, 1 Apr 2014 19:40:53 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:590a:5d80:8a04:284f] (unknown [IPv6:fde9:577b:c1a9:4410:590a:5d80:8a04:284f]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id A7FD6C22C67; Tue, 1 Apr 2014 19:40:52 +0000 (UTC) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: "Bjoern A. Zeeb" In-Reply-To: <20140401051327.F20F958097@chaos.jnpr.net> Date: Tue, 1 Apr 2014 19:40:18 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140401051327.F20F958097@chaos.jnpr.net> To: Simon Gerraty X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 19:40:58 -0000 On 01 Apr 2014, at 05:13 , Simon Gerraty wrote: > # NO_* takes precedence > # If both WITH_* and WITHOUT_* are defined, WITHOUT_ wins unless > # DOMINANT_* is set to "yes" > # Otherwise WITH_* and WITHOUT_* override the default. > and > MK_* can be pre-set without causing an error. ... > Thoughts? I am a it worried that we are increasing the number of prefixes again = rather than reducing it. I was hoping a while ago that NO_ would die. =97=20 Bjoern A. Zeeb ????????? ??? ??????? ??????: '??? ??? ???? ?????? ??????? ?? ?? ??????? ??????? ??? ????? ????? ???? ?????? ?? ????? ????', ????????? ?????????, "??? ????? ?? ?????", ?.??? From owner-freebsd-arch@FreeBSD.ORG Tue Apr 1 19:53:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E145267D; Tue, 1 Apr 2014 19:53:09 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe006.messaging.microsoft.com [216.32.181.186]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95E0B2D6; Tue, 1 Apr 2014 19:53:09 +0000 (UTC) Received: from mail14-ch1-R.bigfish.com (10.43.68.231) by CH1EHSOBE012.bigfish.com (10.43.70.62) with Microsoft SMTP Server id 14.1.225.22; Tue, 1 Apr 2014 19:53:02 +0000 Received: from mail14-ch1 (localhost [127.0.0.1]) by mail14-ch1-R.bigfish.com (Postfix) with ESMTP id 7F50C40758; Tue, 1 Apr 2014 19:53:02 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Idb82hzz1f42h208ch1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h1082kzz1de097hz31h2a8h839hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail14-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail14-ch1 (localhost.localdomain [127.0.0.1]) by mail14-ch1 (MessageSwitch) id 1396381980729658_29557; Tue, 1 Apr 2014 19:53:00 +0000 (UTC) Received: from CH1EHSMHS027.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.246]) by mail14-ch1.bigfish.com (Postfix) with ESMTP id A4DBB380108; Tue, 1 Apr 2014 19:53:00 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS027.bigfish.com (10.43.70.27) with Microsoft SMTP Server (TLS) id 14.16.227.3; Tue, 1 Apr 2014 19:52:51 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Tue, 1 Apr 2014 12:52:50 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s31JqnV48957; Tue, 1 Apr 2014 12:52:49 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 8752258097; Tue, 1 Apr 2014 12:52:49 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> Comments: In-reply-to: Warner Losh message dated "Tue, 01 Apr 2014 12:57:57 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 1 Apr 2014 12:52:49 -0700 Message-ID: <20140401195249.8752258097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Apr 2014 19:53:09 -0000 On Tue, 1 Apr 2014 12:57:57 -0600, Warner Losh writes: >> .if !defined(WITHOUT_CTF) && !defined(NO_CTF) >> WITH_CTF=3Dyes >> .endif >>=20 >> else you get errors during build world. > >That's a bug I plan on fixing. Agreed, and cool. >> That's not so much a problem for existing options, but makes it >> impractical to leverage the mechanism for things you might want to set >> during sys.mk - like whether to use meta mode or auto objdir creation. > >I=92ll have to think about this point. It is a good point, but I=92m = >unsure how >the proposed changes would fix that. Perhaps you could explain that a = >bit. Sure. For the sake of argument assume I can put something like this in local.sys.mk: OPTIONS_DEFAULT_NO+= \ META_MODE OPTIONS_DEFAULT_YES+= \ AUTO_OBJ .include "options.mk" .if ${MK_AUTO_OBJ} == "yes" .include "auto.obj.mk" .endif .if ${MK_META_MODE} == "yes" .include "meta.sys.mk" .endif and both MK_META_MODE={yes,no} MK_AUTO_OBJ={yes,no} which are needed during sys.mk can be influenced by user's -DWITH[OUT]_* >Setting defaults in make.conf, and then overriding them is tough because >bmake doesn't have a tracking system to know where in the food-chain >different variables are set. However, this is also a good point, but I >must be being dense to see how your proposal would fix this. It "fixes" it by not throwing an error when both WITH_FOO and WITHOUT_FOO are defined. Instead, you decree that one wins (eg WITHOUT_FOO overrides WITH_FOO). You could also address the conflict by giving preference to command line. ${.MAKEFLAGS:MWITH*} will give you any WITH_* and WITHOUT_* from command line. There might be a rare case where an error is desired on conflict, but if you can sensibly resolve it that is better. >> I have a number of knobs to be set during sys.mk >> AUTO_OBJ automatically create objdirs >> META_MODE use meta mode > >objdirs set in sys.mk? yes. >I thought that was done bad.obj.mk since it isn't part of the global system. Traditionally done in bsd.obj.mk - but that requires a separate invocation of make. The junos build for example has auto-created objdirs for 10+ years. The projects/bmake build does the same. It is a good way to ensure consistent/reliable results and avoid probelms when users forget to 'make obj'. When you have to support 2000+ developers you want to be able to make it hard for them to shoot themselves in the foot. If making objdirs automatically, you really want to do it early, since the result influences how make then behaves. >META_MODE might belong there. > >> setting MK_* is fine, but it is nice if you allow the user to interact >> using WITH[OUT]_*, and for that it is best if sys.mk can safely = >include >> something like options.mk > >Not sure how creating a new file solves the bsd.own.mk problem, apart >perhaps from some name space pollution differences. An options.mk which does nothing except process the list of options provided to it, is always safe to include even by sys.mk. Whereas bsd.own.mk does other stuff which makes it unsuitable to include eg. during the early stages of buildworld. bsd.own.mk can of course set its own list of options and then include the same options.mk a simple matter of refactoring to allow re-use. >Yea, back to the point I don=92t understand: how is your new file going = >to be >materially different than the current mechanism. I=92m OK with change, = >but I >need to understand how the change is going to make things better. Separating the mechanism of option processing from the definition of options makes re-use easier - see above, sorry I don't know how to make it clearer. The other aspects you have at least partially addressed already, by allowing MK_* to be set independently. Though: .if defined(MK_${var}) .if ${.MAKE.LEVEL} == 0 .error MK_${var} can't be set by a user. .endif would seem to negate that. Why can a makefile at level 0 not set MK_*? The outstanding question is dealing with conflict when both WITH_FOO and WITHOUT_FOO are defined. Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Wed Apr 2 03:03:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC36D88E for ; Wed, 2 Apr 2014 03:03:15 +0000 (UTC) Received: from mail-pd0-f169.google.com (mail-pd0-f169.google.com [209.85.192.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8DFFCD4F for ; Wed, 2 Apr 2014 03:03:15 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id fp1so10512103pdb.0 for ; Tue, 01 Apr 2014 20:03:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=G8vAEKmTl1S9G4Wl6hqniB3gG4KGbdMq+oTCDUWB9Mo=; b=K9V60D9bgswNUc7bZ7TXWKvFycQaoHqOOxtEtBDZoTLFVlSoRpOzY1FYaxQ006q/81 16g0X3AKQz33ciwrRHCrRjZl+0zExLJ+hipjxM+jzhcnZzcrxVyHTU6ix/F+XTHZGn6k 67hpWzCJLZjc3JLMxYnV2H9jYOv/wMTEZhej/l7IgwoTnK+4zHQyA9MPFt3UEbgbP33V mT7T2Cth5pNhWqewDR0mx147eYfj7MZ/+gGmIDI04drnpqJxv8gV5Ph9I5wFnUgY11pX xbHVS/0LcOnPIQRYK0F2B0gvfodQJKsKESzWnFQ3Nq2yKqVLYJjDZU3sNUXkV5eTsFxj sWiQ== X-Gm-Message-State: ALoCoQnUdzD55wfPecXElHZdH/2sZcFvJOJGc/TPESShdEth4lLpxabsvbCJZhkq1td5CyAXYzmo X-Received: by 10.68.226.197 with SMTP id ru5mr34773688pbc.77.1396407788966; Tue, 01 Apr 2014 20:03:08 -0700 (PDT) Received: from [10.64.24.154] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id x1sm924662pbk.47.2014.04.01.20.03.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 01 Apr 2014 20:03:08 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: Date: Tue, 1 Apr 2014 21:03:05 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <00367FFD-4F88-4257-B25F-D4B3F0644FE1@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org, Simon Gerraty X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 03:03:15 -0000 On Apr 1, 2014, at 1:40 PM, Bjoern A. Zeeb = wrote: >=20 > On 01 Apr 2014, at 05:13 , Simon Gerraty wrote: >=20 >> # NO_* takes precedence >> # If both WITH_* and WITHOUT_* are defined, WITHOUT_ wins unless >> # DOMINANT_* is set to "yes" >> # Otherwise WITH_* and WITHOUT_* override the default. >> and >> MK_* can be pre-set without causing an error. > ... >> Thoughts? >=20 > I am a it worried that we are increasing the number of prefixes again = rather than reducing it. I was hoping a while ago that NO_ would die. That was my plan=85 Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 2 17:59:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD782963 for ; Wed, 2 Apr 2014 17:59:15 +0000 (UTC) Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B9C31B7 for ; Wed, 2 Apr 2014 17:59:15 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id uo5so537331pbc.32 for ; Wed, 02 Apr 2014 10:59:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=bHe4ySkWdhn6IfeLYh2+Qo4Hva1XRp11+6zWgUdXR2M=; b=YK8l2I1s9hXLenZGvFCkmeszypFOdrJpD3SoiE+9G2+1ylQFKO/mMOJ4r3WkvkgPOq FdpXjQH+Vz3z9AFV+wfMAYY5VnV93wy6uHFIVxYRnQ2vAQ4dhm0l5ZpIU2/bMuVzfhM5 bycgwpFpBX2L8Bimx2J6LXgGQvzoHEPA2e4pWT8WdgQhPKcr8wY9wB4F3y4yQk6LKMq2 Xei0tDUn14OzOh3iRDCfF4AM/H59C5qgXaGPria5tKYD81h3Y9YDYxFRWzxEfzJeH+WC o5WOjyd4WIlByKw+b0WkSiKQ44oTTOuQx1BVugs3wmshVP9lpbk/6ArMjTFxiVEC+gN3 jRyw== X-Gm-Message-State: ALoCoQmxlQ2QRAn1ZrO9IsAGCYfrRLgzyhxva0J8mq0WL+MLbLtvRyPTJHMTGdThRC2H+v2Cw4Y1 X-Received: by 10.68.201.97 with SMTP id jz1mr1743710pbc.26.1396461554425; Wed, 02 Apr 2014 10:59:14 -0700 (PDT) Received: from [10.64.24.154] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id yi3sm5828593pbb.51.2014.04.02.10.59.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 02 Apr 2014 10:59:13 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140401195249.8752258097@chaos.jnpr.net> Date: Wed, 2 Apr 2014 11:59:10 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 17:59:15 -0000 Hi Simon, thanks for the good dialog, and the patient explanation of things. Sorry = I=92ve taken a bit to reply... On Apr 1, 2014, at 1:52 PM, Simon J. Gerraty wrote: > On Tue, 1 Apr 2014 12:57:57 -0600, Warner Losh writes: >>> .if !defined(WITHOUT_CTF) && !defined(NO_CTF) >>> WITH_CTF=3D3Dyes >>> .endif >>> =3D20 >>> else you get errors during build world. >>=20 >> That's a bug I plan on fixing.=20 >=20 > Agreed, and cool. I almost have this bug fixed. There=92s one instance where it isn=92t = that helps illustrate the point you are making. >>> That's not so much a problem for existing options, but makes it >>> impractical to leverage the mechanism for things you might want to = set >>> during sys.mk - like whether to use meta mode or auto objdir = creation. >>=20 >> I=3D92ll have to think about this point. It is a good point, but = I=3D92m =3D >> unsure how >> the proposed changes would fix that. Perhaps you could explain that a = =3D >> bit. >=20 > Sure. For the sake of argument assume I can put something like this = in > local.sys.mk: >=20 > OPTIONS_DEFAULT_NO+=3D \ > META_MODE >=20 > OPTIONS_DEFAULT_YES+=3D \ > AUTO_OBJ >=20 > .include "options.mk" >=20 > .if ${MK_AUTO_OBJ} =3D=3D "yes" > .include "auto.obj.mk" > .endif > .if ${MK_META_MODE} =3D=3D "yes" > .include "meta.sys.mk" > .endif >=20 > and both >=20 > MK_META_MODE=3D{yes,no} > MK_AUTO_OBJ=3D{yes,no} >=20 > which are needed during sys.mk can be influenced by user's = -DWITH[OUT_* OK. A bit of a contrived example, but I suppose if I understood the meta = mode a bit better I might think differently :) I=92m a bit hesitant, though, to have this affect sys.mk because that = affects all users of make, not just /usr/src. >> Setting defaults in make.conf, and then overriding them is tough = because >> bmake doesn't have a tracking system to know where in the food-chain >> different variables are set. However, this is also a good point, but = I >> must be being dense to see how your proposal would fix this. >=20 > It "fixes" it by not throwing an error when both WITH_FOO and = WITHOUT_FOO > are defined. Instead, you decree that one wins (eg WITHOUT_FOO > overrides WITH_FOO). In some cases, you can declare a winner. In other cases that might be = harder to do. With almost all options defaulting to YES, having WITHOUT win = makes good sense. When there=92s more of a mix, I=92m less sure. > You could also address the conflict by giving preference to command = line. > ${.MAKEFLAGS:MWITH*} will give you any WITH_* and WITHOUT_* from = command > line. =20 >=20 > There might be a rare case where an error is desired on conflict, but > if you can sensibly resolve it that is better. >=20 >>> I have a number of knobs to be set during sys.mk >>> AUTO_OBJ automatically create objdirs >>> META_MODE use meta mode >>=20 >> objdirs set in sys.mk?=20 >=20 > yes. >=20 >> I thought that was done bad.obj.mk since it isn't part of the global = system. >=20 > Traditionally done in bsd.obj.mk - but that requires a separate > invocation of make. Right, but can=92t that be done automatically w/o that extra invocation? > The junos build for example has auto-created objdirs for 10+ years. > The projects/bmake build does the same. > It is a good way to ensure consistent/reliable results and avoid > probelms when users forget to 'make obj'. >=20 > When you have to support 2000+ developers you want to be able to > make it hard for them to shoot themselves in the foot. >=20 > If making objdirs automatically, you really want to do it early, since > the result influences how make then behaves. True, all good stuff. Just got caught up quibbling over its need to be in sys.mk vs enhancing bsd.obj.mk to do this... >> META_MODE might belong there. >>=20 >>> setting MK_* is fine, but it is nice if you allow the user to = interact >>> using WITH[OUT]_*, and for that it is best if sys.mk can safely =3D >> include >>> something like options.mk >>=20 >> Not sure how creating a new file solves the bsd.own.mk problem, apart >> perhaps from some name space pollution differences. >=20 > An options.mk which does nothing except process the list of > options provided to it, is always safe to include even by sys.mk. > Whereas bsd.own.mk does other stuff which makes it unsuitable to = include=20 > eg. during the early stages of buildworld. > bsd.own.mk can of course set its own list of options and then include > the same options.mk a simple matter of refactoring to allow re-use. Back to the NO_CTF stuff I talked about above: I totally get this. If = you look at bsd.own.mk it is in 3 sections (plus a stray line at the end that really = belongs in the first section. # section 1 - stuff about ownership # section 2 - stuff to do options # section 3 - setting defaults based on options It is this last section that=92s a problem to CTF specifically. = Normally, we have a paradigm in the tree where we do .include .if ${MK_FOO} !=3D =93no=94 CFLAGS+=3D-DSUPPORT_FOO .endif There=92s one place in the tree that wants to turn off CTF. This is = mostly fixed by just setting MK_CTF=3Dno in that makefile after we include bsd.own.mk. I = say mostly fixed because we wind up with a NULL command where we really want a @: command, though the former I believe is harmless but verbose. = Except one could unset WITH_CTF (which doesn=92t completely work, it still = shows up as defined) and set WITHOUT_CTF before bsd.own.mk and it would work, modulo this bug. This can really only be fixed by making bsd.own.mk look more like # section 1 -same .include # section 3 - same with bsd.options.mk looking a lot like section 2 of bsd.own.mk. Also, = we=92d have to include bsd.options.mk at the head of the Makefile rather than = bsd.own.mk where we do now. This sounds a lot like what you=92re trying to describe, though = placement of bsd.options.mk may be different than I described. The only reason we need it where I suggested it is compatibility with the past. Though we = may be able to get away with it in sys.mk, I=92m hesitant to place it in = there because that=92s global to everything, including ports, etc. Plus, I think it is = too early, due to meta MK_ variables, that I talk about below. >> Yea, back to the point I don=3D92t understand: how is your new file = going =3D >> to be >> materially different than the current mechanism. I=3D92m OK with = change, =3D >> but I >> need to understand how the change is going to make things better. >=20 > Separating the mechanism of option processing from the definition of > options makes re-use easier - see above, sorry I don't know how to = make > it clearer. >=20 > The other aspects you have at least partially addressed already, by > allowing MK_* to be set independently. > Though: >=20 > .if defined(MK_${var}) > .if ${.MAKE.LEVEL} =3D=3D 0 > .error MK_${var} can't be set by a user. > .endif >=20 > would seem to negate that. Why can a makefile at level 0 not set = MK_*? Well, the notion now is that if you want to test MK_* variables, you = need to include bsd.own.mk first. The notion I was going for with the above is = that you=92d have to include bsd.own.mk first, then set the MK_* variables in your = Makefile, so the tests won=92t be hit. But there=92s a problem even if we take the approach above. Section 2 in = bsd.own.mk is actually two separate sections. One that sets the MK_* variables = based on WITH_ or WITHOUT_ and then a second section that cascades the MK_ = variables to other MK_ variables (like MK_CRYPT=3D=3Dno turning of OpenSSL, = Kerberos, etc). If you wanted to set one of those variables in your Makefile, you=92d have = a chicken and egg problem. If you set it before bsd.options.mk, then you=92d get = the cascade effects but hit the warning. If you set it after, you dodge the warning, but = don=92t get the cascade. This problem suggests, perhaps, that the test be deleted. > The outstanding question is dealing with conflict when both WITH_FOO = and > WITHOUT_FOO are defined. True. That=92s a tougher problem than you might think on first blush, as = we=92ve been talking about. For now, I=92d suggest WITHOUT wins, and we see how = far that gets us and what problems come up. Anything else seems like it = would get uber complicated quite quickly as we play whack-a-mole with = different edge use cases. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 2 20:28:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 489E3120 for ; Wed, 2 Apr 2014 20:28:58 +0000 (UTC) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe001.messaging.microsoft.com [216.32.180.11]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED68B91A for ; Wed, 2 Apr 2014 20:28:57 +0000 (UTC) Received: from mail22-va3-R.bigfish.com (10.7.14.227) by VA3EHSOBE006.bigfish.com (10.7.40.26) with Microsoft SMTP Server id 14.1.225.22; Wed, 2 Apr 2014 20:28:50 +0000 Received: from mail22-va3 (localhost [127.0.0.1]) by mail22-va3-R.bigfish.com (Postfix) with ESMTP id 2DD2C140123; Wed, 2 Apr 2014 20:28:50 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zzzz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzzz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail22-va3: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail22-va3 (localhost.localdomain [127.0.0.1]) by mail22-va3 (MessageSwitch) id 1396470528272190_12099; Wed, 2 Apr 2014 20:28:48 +0000 (UTC) Received: from VA3EHSMHS038.bigfish.com (unknown [10.7.14.232]) by mail22-va3.bigfish.com (Postfix) with ESMTP id 3A4F74C0070; Wed, 2 Apr 2014 20:28:48 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by VA3EHSMHS038.bigfish.com (10.7.99.48) with Microsoft SMTP Server (TLS) id 14.16.227.3; Wed, 2 Apr 2014 20:28:45 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 2 Apr 2014 13:28:44 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s32KShV17565; Wed, 2 Apr 2014 13:28:44 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 37C8F58097; Wed, 2 Apr 2014 13:28:43 -0700 (PDT) To: "Bjoern A. Zeeb" Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: References: <20140401051327.F20F958097@chaos.jnpr.net> Comments: In-reply-to: "Bjoern A. Zeeb" message dated "Tue, 01 Apr 2014 19:40:18 -0000." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 2 Apr 2014 13:28:43 -0700 Message-ID: <20140402202843.37C8F58097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 20:28:58 -0000 On Tue, 1 Apr 2014 19:40:18 +0000, "Bjoern A. Zeeb" writes: >I am a it worried that we are increasing the number of prefixes again rathe= >r than reducing it. I was hoping a while ago that NO_ would die. NO_ is/was potentially useful. It allows a makefile to say, "I cannot do that" Of couse by allowing makefiles to set MK_* directly, they can just as easily do MK_*=no Even then, until you eradicate NO_* it is necessary to handle it? From owner-freebsd-arch@FreeBSD.ORG Wed Apr 2 21:23:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82BCC68E; Wed, 2 Apr 2014 21:23:40 +0000 (UTC) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe005.messaging.microsoft.com [216.32.180.31]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FA7BDF2; Wed, 2 Apr 2014 21:23:40 +0000 (UTC) Received: from mail6-va3-R.bigfish.com (10.7.14.236) by VA3EHSOBE001.bigfish.com (10.7.40.21) with Microsoft SMTP Server id 14.1.225.22; Wed, 2 Apr 2014 21:23:33 +0000 Received: from mail6-va3 (localhost [127.0.0.1]) by mail6-va3-R.bigfish.com (Postfix) with ESMTP id 5533B46057E; Wed, 2 Apr 2014 21:23:33 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 4 X-BigFish: VPS4(zz1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz1de097h74efjz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail6-va3: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail6-va3 (localhost.localdomain [127.0.0.1]) by mail6-va3 (MessageSwitch) id 1396473810418663_19572; Wed, 2 Apr 2014 21:23:30 +0000 (UTC) Received: from VA3EHSMHS041.bigfish.com (unknown [10.7.14.249]) by mail6-va3.bigfish.com (Postfix) with ESMTP id 5796820074; Wed, 2 Apr 2014 21:23:30 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by VA3EHSMHS041.bigfish.com (10.7.99.51) with Microsoft SMTP Server (TLS) id 14.16.227.3; Wed, 2 Apr 2014 21:23:30 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 2 Apr 2014 14:23:29 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s32LNSV53393; Wed, 2 Apr 2014 14:23:28 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 7CE8958097; Wed, 2 Apr 2014 14:23:28 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Wed, 02 Apr 2014 11:59:10 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 2 Apr 2014 14:23:28 -0700 Message-ID: <20140402212328.7CE8958097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2014 21:23:40 -0000 >> which are needed during sys.mk can be influenced by user's -DWITH[OUT_* > >OK. A bit of a contrived example, but I suppose if I understood the meta mo= >de Actually not contrived at all - we have that internally. Ie. we have a local.sys.mk to set all our cool stuff but we currently just test for .ifdef WITH[OUT]*, hence this thread. >a bit better I might think differently :) > >I=92m a bit hesitant, though, to have this affect sys.mk because that affec= >ts all users >of make, not just /usr/src. That's why I'd put such things in local.sys.mk or some other makefile that affects /usr/src but isn't install into /usr/share/mk/ >In some cases, you can declare a winner. In other cases that might be harder >to do. With almost all options defaulting to YES, having WITHOUT win makes >good sense. When there=92s more of a mix, I=92m less sure. Agreed, that's why I took the easy way out by allowing the winner to be configured if the need should arise ;-) >> Traditionally done in bsd.obj.mk - but that requires a separate >> invocation of make. > >Right, but can=92t that be done automatically w/o that extra invocation? Yes provided you do it early enough (ie during sys.mk) eg. before you've evaluated things that affect .PATH >Back to the NO_CTF stuff I talked about above: I totally get this. If you l= >There=92s one place in the tree that wants to turn off CTF. This is mostly = >fixed by >just setting MK_CTF=3Dno in that makefile after we include bsd.own.mk. I say Wouldn't it be simpler to set MK_CTF=no *before* including bsd.own.mk ? >mostly fixed because we wind up with a NULL command where we really want >a @: command, though the former I believe is harmless but verbose. Except >one could unset WITH_CTF (which doesn=92t completely work, it still shows >up as defined) and set WITHOUT_CTF before bsd.own.mk and it would work, >modulo this bug. > >This can really only be fixed by making bsd.own.mk look more like > ># section 1 -same >.include ># section 3 - same > >with bsd.options.mk looking a lot like section 2 of bsd.own.mk. Also, we= >=92d have Yes, that's essentially what I was proposing. By extracting the mechanism to its own file, it can be re-used. >This sounds a lot like what you=92re trying to describe, though placement of >bsd.options.mk may be different than I described. The only reason we >need it where I suggested it is compatibility with the past. Though we may Yes I assumed it would be included as above - to avoid changing behavior unnecessarily. Note: tweaking the semantics and making it re-usable are somewhat orthogonal. >be able to get away with it in sys.mk, I=92m hesitant to place it in there = >because >that=92s global to everything, including ports, etc. Plus, I think it is to= Calling it bsd.options.mk is a conflict with ports. Though including it as "bsd.options.mk" both in bsd.own.mk and in the relevant ports makefile, should probably mitigate that. >o early, due to meta MK_ variables, that I talk about below. >> .if defined(MK_${var}) >> .if ${.MAKE.LEVEL} =3D=3D 0 >> .error MK_${var} can't be set by a user. >> .endif >> = > >> would seem to negate that. Why can a makefile at level 0 not set MK_*? > >Well, the notion now is that if you want to test MK_* variables, you need to >include bsd.own.mk first. The notion I was going for with the above is that= Setting MK_* before bsd.own.mk vs after has semantic differences but that shouldn't preclude doing either. Eg. the knob names below describe the semantics # these remove choice from user MK_CANNOT= no MK_MUST= yes .include # these respect user choices MK_LIKE?= yes MK_DISLIKE?= no >But there=92s a problem even if we take the approach above. Section 2 in bs= >d.own.mk >is actually two separate sections. One that sets the MK_* variables based on >WITH_ or WITHOUT_ and then a second section that cascades the MK_ variables >to other MK_ variables (like MK_CRYPT=3D=3Dno turning of OpenSSL, Kerberos,= > etc). If >you wanted to set one of those variables in your Makefile, you=92d have a c= >hicken >and egg problem. If you set it before bsd.options.mk, then you=92d get the = >cascade effects >but hit the warning. If you set it after, you dodge the warning, but don=92= >t get the cascade. Per above setting MK_* before including bsd.own.mk is just supressing user input, probably not something to do a lot, but handy at times - eg. allows doing away with NO_* If that has cascading effects, we assume they are intended. Currently, what warning do you hit btw? I only see .errors if MK_* is pre-defined or WITH[OUT]* both defined. >This problem suggests, perhaps, that the test be deleted. > >> The outstanding question is dealing with conflict when both WITH_FOO and >> WITHOUT_FOO are defined. > >True. That=92s a tougher problem than you might think on first blush, as we= >=92ve >been talking about. For now, I=92d suggest WITHOUT wins, and we see how far Agreed. I cannot currently think of a case where that wouldn't be right, but as I mentioned wrt my options.mk it is easy to allow configurability should the need arise. Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Sat Apr 5 10:00:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A83A2E24; Sat, 5 Apr 2014 10:00:44 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BA1685C; Sat, 5 Apr 2014 10:00:44 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id k15so4126586qaq.15 for ; Sat, 05 Apr 2014 03:00:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=KxtoKD506BfcM7fJeS6nEyuU/KK38iGXKPv3io/jw84=; b=OM9IP+IA4SI2uQ9mHwzlV09WyT0xcz+nI//660HBYTPAtN3KtBEKMzm+i5dcj56LHG gWV8NJnnPpww7tcPh8bUaUtqQxCPNCQtEeg8MIIOdSqughUKs05Ls4mMbDHF18xLp3pN BrcWMYZtIJxyLwKTgEmhm/fHlqzy8boqi9tfbMnbkdCydm9hQ8fEHPlZAeJt6CXniZ4g TCt1Qo8t4B3lE12JHxaum8oXGp+2OM2u489ZO4wameCUW0SpGDEy/HL10pMQhdfm4unJ VdYBmhdVs+nukz5XXMsidBquZ9eu+pFuCePKXnHVqwY7pGyE7zJxf3x3ZwDy4QGeCNtO l/IQ== MIME-Version: 1.0 X-Received: by 10.224.36.129 with SMTP id t1mr928918qad.88.1396692043517; Sat, 05 Apr 2014 03:00:43 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.140.96.203 with HTTP; Sat, 5 Apr 2014 03:00:43 -0700 (PDT) Date: Sat, 5 Apr 2014 12:00:43 +0200 X-Google-Sender-Auth: psMQTcnKFhi6nm-19Q5X-JQ_jd0 Message-ID: Subject: [Patch] kqueue(2) <-> procdesc(4): EVFILT_PROCDESC From: Ed Schouten To: Robert Watson Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 10:00:44 -0000 Hi Robert, The other day I was playing around with procdesc(4). Quite a nice piece of work. I did notice, though, that there is no way you can use kqueue(2) to monitor process descriptors. That's quite a shame, because it would be awesome if we could just use EVFILT_PROC on process descriptors directly. This is why I thought it would be nice to introduce a variant called EVFILT_PROCDESC. I initially tried altering kqueue(2) in such a way that EVFILT_PROCDESC would use a mixture of filt_fileattach() to attach to the descriptor and filt_proc() to watch for events, but this approach didn't really work out all that well, for the reason that a kevent either has a file descriptor or a process associated; not both. In the end I decided to not make things more complex than needed and just implement it like a regular file descriptor probe. This means that we can get NOTE_EXIT to work, but NOTE_FORK, NOTE_EXEC and NOTE_TRACK would require some more work. What are your thoughts on the following patch? http://80386.nl/pub/kqueue-evfilt-procdesc.txt Some notes on this patch: - I decided to just reuse the obsolete EVFILT_NETDEV. EVFILT_PROCDESC will be used on completely different file descriptor types, so I can't think of a way this would cause ABI issues. - pd->pd_proc is protected by proctree_lock. It's a bit hard to pick this up inside of the kqfilter, so simply make procdesc_exit() copy out pd->pd_proc->p_xstat. We also want this copy, because we don't want to run into a race condition where wait4() already reaps the process before the kqfilter is called. This is works for EVFILT_PROC anyway. The nice thing about this patch is that even though pdwait4(2) is still unimplemented, it does at least allow people to now extract the exit code without accessing any global namespaces. Thanks, -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Sat Apr 5 15:36:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BEEF7B54; Sat, 5 Apr 2014 15:36:34 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 464C8DCC; Sat, 5 Apr 2014 15:36:34 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.8/8.14.8) with ESMTP id s35FaLMd055575; Sat, 5 Apr 2014 18:36:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s35FaLMd055575 Received: (from kostik@localhost) by tom.home (8.14.8/8.14.8/Submit) id s35FaLdI055574; Sat, 5 Apr 2014 18:36:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 5 Apr 2014 18:36:21 +0300 From: Konstantin Belousov To: Ed Schouten Subject: Re: [Patch] kqueue(2) <-> procdesc(4): EVFILT_PROCDESC Message-ID: <20140405153621.GH21331@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="11IAkegDWp8TRrA/" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: Robert Watson , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Apr 2014 15:36:34 -0000 --11IAkegDWp8TRrA/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 05, 2014 at 12:00:43PM +0200, Ed Schouten wrote: > Hi Robert, >=20 > The other day I was playing around with procdesc(4). Quite a nice > piece of work. I did notice, though, that there is no way you can use > kqueue(2) to monitor process descriptors. That's quite a shame, > because it would be awesome if we could just use EVFILT_PROC on > process descriptors directly. This is why I thought it would be nice > to introduce a variant called EVFILT_PROCDESC. >=20 > I initially tried altering kqueue(2) in such a way that > EVFILT_PROCDESC would use a mixture of filt_fileattach() to attach to > the descriptor and filt_proc() to watch for events, but this approach > didn't really work out all that well, for the reason that a kevent > either has a file descriptor or a process associated; not both. >=20 > In the end I decided to not make things more complex than needed and > just implement it like a regular file descriptor probe. This means > that we can get NOTE_EXIT to work, but NOTE_FORK, NOTE_EXEC and > NOTE_TRACK would require some more work. >=20 > What are your thoughts on the following patch? >=20 > http://80386.nl/pub/kqueue-evfilt-procdesc.txt >=20 > Some notes on this patch: >=20 > - I decided to just reuse the obsolete EVFILT_NETDEV. EVFILT_PROCDESC > will be used on completely different file descriptor types, so I can't > think of a way this would cause ABI issues. >=20 > - pd->pd_proc is protected by proctree_lock. It's a bit hard to pick > this up inside of the kqfilter, so simply make procdesc_exit() copy > out pd->pd_proc->p_xstat. We also want this copy, because we don't > want to run into a race condition where wait4() already reaps the > process before the kqfilter is called. This is works for EVFILT_PROC > anyway. >=20 > The nice thing about this patch is that even though pdwait4(2) is > still unimplemented, it does at least allow people to now extract the > exit code without accessing any global namespaces. procdesc_kqops_event() should mimic the filt_proc() much more closely than you do. In particular: - I do not see why do you check for NOTE_EXIT in kqops_event(). If enforcing NOTE_EXIT anywhere, procdesc_kqfilter() is more natural place to put the temporal restriction. I do think that signalling and execing are equally useful as exiting notifications, but this is for future. - The whole structure of filt_proc(), which copies masked sfflags to fflags is easier to extend later. It seems that just repeating most of the code from filt_proc() would do what needed.=20 I think you must do knlist_clear() before knlist_destroy(). --11IAkegDWp8TRrA/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTQCL1AAoJEJDCuSvBvK1BLVYQAKfTXhLBPY7bAzGacGlUJLu8 uG/GrCsEryr49oFw6C73g6y0JYGW7kcWGYyknQ3Iz0/52c5c8DzIj2q5gSvrG0Ih RpgdrDYRMiVRcoQzoQxmT2pJ9e2LzNnk6Hnz6zWwOgBlkz9EYVs44s0LLOlbCzEx 0+uEYMr6ACOspE421/y4c4H8jB1EZ6VMvxjONWqVzZpHR5xzxzFtZ73/JAKrCwt9 DANQqHImXykhTqsomv4edeJ0e/TJRykHP2HqojYsRqSlE2boneJGrsoDX18BqKwi H6KqNMpgYOx+4cngsxhKJznIoOVSfmp0gvILZgLn6ySatn1cTKfle9lum0PHEHqZ o5bl8GTVmEqols9q14CzxXbGNg/FW2WBobb7iQUNK1TIyBE+wwMZl3ArN8jSg8LS rjTs2lp65O+ZU0+/yex2MVaX2DwCyX+iN1Li0E5P8Z/0D26txc3TNGyxUm31e+w6 wV4szGVy8a0l5oz9CplVfkjGYfGEKIZ5qz0PK+PwODxsP0r2wqmbZnd4XaUAPNpP 4r03lqqO0It8QdYo0u3or6luGKHB6EV99xy3M9CLvI9dPyaVXkNAq7KF1xp0gvhB bOaVYhWFb0zoe+/HN/GCJCFTvzfKeDdgbyggV/IswdmVnS8CdLtmc4jo/sarcVD0 hyfmf3LOPdURY7lxjxFT =fPku -----END PGP SIGNATURE----- --11IAkegDWp8TRrA/-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 6 15:16:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8973C963; Sun, 6 Apr 2014 15:16:51 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A58B80F; Sun, 6 Apr 2014 15:16:51 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id e16so5447047qcx.34 for ; Sun, 06 Apr 2014 08:16:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=w1V1H0iBj1f/nzCtdOiOjrxXAlcR6CeM7O7KJKEyQ8A=; b=NaGxBWmsMZudexdV0RUvGgR3tf6G077DAmw+VYhmcJqdBxSTDKGenwFMfZfqsNOshs KVZppmO6MpYE1du8U74UDbwi3t4JQisPjsUslchQT3BAAVV1QxcKlaXM+WRKydwUoGm/ 3Ny/HJwPJTP1e4/X2x9edahKi+JxbA+pShW/FEU5uNW1fe/gqJtslcANTwSyBryjpRI1 t7wnFqDzX9rKnIQDS413b55db7RH6c1c/LocxheZOKwBFJ/H6uO3Zf2hfU/YZo+SHCK7 M7k5D9bYzwEiZHVgUFHwQuK1xtWZwJDp2lwHoaXAC8Yd8HcyH6PhdWeEIiG51EhUDR3H vIZQ== MIME-Version: 1.0 X-Received: by 10.229.198.2 with SMTP id em2mr1720202qcb.21.1396797409571; Sun, 06 Apr 2014 08:16:49 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.140.96.203 with HTTP; Sun, 6 Apr 2014 08:16:49 -0700 (PDT) In-Reply-To: <20140405153621.GH21331@kib.kiev.ua> References: <20140405153621.GH21331@kib.kiev.ua> Date: Sun, 6 Apr 2014 17:16:49 +0200 X-Google-Sender-Auth: ENszJSg_2Mt60BLpmnN5lBllZ4U Message-ID: Subject: Re: [Patch] kqueue(2) <-> procdesc(4): EVFILT_PROCDESC From: Ed Schouten To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 Cc: Robert Watson , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 15:16:51 -0000 Hi there, First of all, thanks for the quick review. Well appreciated! On 5 April 2014 17:36, Konstantin Belousov wrote: > procdesc_kqops_event() should mimic the filt_proc() much more closely > than you do. In particular: > - I do not see why do you check for NOTE_EXIT in kqops_event(). > If enforcing NOTE_EXIT anywhere, procdesc_kqfilter() is more natural place > to put the temporal restriction. I do think that signalling and execing > are equally useful as exiting notifications, but this is for future. > - The whole structure of filt_proc(), which copies masked sfflags to > fflags is easier to extend later. It seems that just repeating > most of the code from filt_proc() would do what needed. My plan was initially to mimick filt_proc() almost entirely and just use KNOTE_LOCKED(..., NOTE_EXIT) just like EVFILT_PROC does, but unfortunately this doesn't seem possible, because if we went down this path, there would be no way for us to activate the knote in procdesc_kqfilter() immediately, as KNOTE_ACTIVATE() is not available outside of kern_event.c. This is why I just changed the code, so that procdesc_kqops_event() is almost literally a copy of filt_proc(), which the difference that it tests against PDF_EXITED instead of using the hint. While there, I made some style fixes to filt_proc(). Thoughts? http://80386.nl/pub/kqueue-evfilt-procdesc.txt (refresh) > I think you must do knlist_clear() before knlist_destroy(). If I understand kqueue internals properly, I am pretty certain that knlist_clear() is not needed in this case, for two reasons: - knlist_clear man page entry: "This function must not be used when f_isfd is set in struct filterops, as the td argument of fdrop() will be NULL." Other file descriptor types also don't seem to call this function. - struct procdesc is only deallocated if the file descriptor is closed. This means that the knlist is guaranteed to be empty at this point in time. knlist_destroy() also has an assertion for this, which is good. -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Sun Apr 6 19:06:47 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A06F825; Sun, 6 Apr 2014 19:06:47 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C6EF5FD3; Sun, 6 Apr 2014 19:06:46 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id i17so5572542qcy.14 for ; Sun, 06 Apr 2014 12:06:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=q231UmC4L2z7zXy4V77Hu6WRIOrE1S/EUQOqgyHMuoE=; b=lPbab/t09wlLMKfPRIj4CKBAubMu2CWprnNUds0XiLmJPFjEo/pKsbE72+cViirtDO 7OSOAkaObRA3bzrdJ9mm4LWw47h22rv/ZLp4u6F5MlJhFDD7m/b/e7fdj39PvoERLJCN MFIK8tGZ3jL1+qAG+QcUGaHvxrGuzs9hj9U9IkDlPWl6A4+g35gdi4szLEOmtowHFu9m gawITw4gnwFFq9cvJ+w3YC9jWGKHEX8NheqgUwbR2DmwiMSVKks7e92VxHPm0wudSyzH 5eXOdwtIAMYQ5uCOD70yeePdc7JlvgAWbJWLIzUc2vQHQzVKsAjBazd/Og2WL5RnOp0r ZuxA== MIME-Version: 1.0 X-Received: by 10.140.21.8 with SMTP id 8mr9664345qgk.55.1396811205960; Sun, 06 Apr 2014 12:06:45 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Sun, 6 Apr 2014 12:06:45 -0700 (PDT) In-Reply-To: References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> Date: Sun, 6 Apr 2014 12:06:45 -0700 X-Google-Sender-Auth: WzW-xgNagbWwMXvajc_qUhCD3H0 Message-ID: Subject: Re: Compiler toolchain roadmap From: Adrian Chadd To: Jordan Hubbard Content-Type: text/plain; charset=ISO-8859-1 Cc: Baptiste Daroussin , David Chisnall , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 19:06:47 -0000 Hi, (shifting to -arch.) Sure. Qualcomm is a great example here. They're ARM, but they're not really just "arm". They also have their embedded CPU architecture that they stick inside basebands and mobile base stations that can run with an MMU and do all that kind of jazz. They are increasingly playing in the LLVM space, but I doubt they'll release LLVM patches for their latest ARM or whatnot fork before it's mainstreamed. They may want to bring up an operating system (eg android) on a new CPU family without having to wait for the upstream compiler to have support. So they'll jump through hoops to make sure they can use their internal compiler fork and then merge stuff back into upstream LLVM when things have shaken out. I can't speak for Apple but from what I've heard they have also done their own compiler and CPU architecture hacking. They employ LLVM developers who will add new CPU features to existing CPUs as well as hack on new CPUs. There are plenty of CPU features in the existing Intel (and likely AMD) 64 bit x86 stuff that we just plain aren't using yet in LLVM. Then there's the research folk like Robert's group(s). They're creating a new CPU architecture and they're also building the toolchain for OS bringup. It's a great example of what a company making new platforms end up doing - except here it's research, not production. The same issues are faced. I'm definitely not arguing "make everything work for old MIPS and ARM platforms". I'm totally on board with the "C99 and then C11 should just be required at this point." But part of the point of supporting external compiler stuff isn't just to make MIPS, ARM and PPC stuff work. It's to keep us honest. We as a project treat non-x86 as a second class citizen and it shows in our build infrastructure (ie, everything is native.) It's similar to the 32 vs 64 bit platform stuff - it again made the codebase better, even if it was "ew, DEC alpha." As an example here - I get the feeling that people care not about 32-bit x86 and would like it to die. The lack of interest and use of freebsd/i386 by developers sometimes shows - suddenly KVA isn't infinite anymore and a lot more stuff assumes large amounts of physical RAM. But besides the recent push by Intel for all of their x86 32-bit embedded stuff (which is all Linux, by the way), those decisions also impact the 32 bit ARM platforms. Those aren't going away. -a From owner-freebsd-arch@FreeBSD.ORG Sun Apr 6 19:49:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61F8E2EE for ; Sun, 6 Apr 2014 19:49:15 +0000 (UTC) Received: from mail-ie0-f174.google.com (mail-ie0-f174.google.com [209.85.223.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 215A6345 for ; Sun, 6 Apr 2014 19:49:14 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id rp18so5207413iec.5 for ; Sun, 06 Apr 2014 12:49:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=0x0kHuG8qZ2jU2rto7nWGRSijJq1TZIkgqd6rYwHbr8=; b=N07Ooa5DgtDwvpPIHFXhJbhKSaW2kW7sPFkCFBEcksazhHdDvzXjYxtuooCnrF04a+ 11j/W8L0LTgjyv6o0tgc18fIZZ7ustqJHQa7z9jsK3B9EWodspyM0QWV+2xvGoquOtTK XI/I+fquCDop0hmzlyAc0kGYN00PrqGHA3ba12dgDz8bnDgIuA6PV3img9RYzMfOIOOJ T/UmnaEgV/EfyQLd/mrlWwpUIuVe3EjvxU4dImGUzJZ2hjorZ4ldgZsbiEpgHlFWyJcs kMMb+t5nitSfVYQZoSgMZzEZTB7PLG90zWNKmdt+HwyyMVG5vORLxjeAXYfEiUC54eyP yNsg== X-Gm-Message-State: ALoCoQkGwGwta6PnPfpxdx7Ss9G397Tynl12ssFTsGbHwZf4f1smVoP1/OH/jgb6mZ4JaSGsB8WI X-Received: by 10.50.25.7 with SMTP id y7mr14866011igf.17.1396813272337; Sun, 06 Apr 2014 12:41:12 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id b6sm23952115igm.2.2014.04.06.12.41.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 06 Apr 2014 12:41:11 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Warner Losh In-Reply-To: Date: Sun, 6 Apr 2014 13:41:10 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <61229525-FD28-4311-B0C8-E3D32DAA27A8@bsdimp.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: David Chisnall , Baptiste Daroussin , Jordan Hubbard , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 19:49:15 -0000 On Apr 6, 2014, at 1:06 PM, Adrian Chadd wrote: > They may want to bring up an operating system (eg android) on a new > CPU family without having to wait for the upstream compiler to have > support. So they'll jump through hoops to make sure they can use their > internal compiler fork and then merge stuff back into upstream LLVM > when things have shaken out. We have basic out-of-tree toolchain support today (though it is a bit = clang centric due to esoteric differences in things like -B). I=92ve = been working in the background to improve this support, as well as bring = FreeBSD gcc changes forward to newer gcc=92s (at least the ones that = make sense to me: all the !x86 mods, the format changes, the platform = support). This will be critical if we=92re going to support, say, cross = building sparc64 from a ports-based compiler from scratch (which we have = extremely limited/poor support for today due to bootstrapping issues). I know there=92s also a big desire on some fronts to go all in on gcc = 4.9 or some other gplv3 compiler where the ARM or MIPS or = is so much better than the ancient 4.2.1 we = have in the tree. So it isn=92t even just the experimental bleeding edge = R&D lab folks that need to do this. Plus we have a boatload of issues with just doing a buildworld and/or = buildkernel with, say, gcc49=85 And our build infrastructure needs to = grow better version support for gcc and clang (right now you get one = version, which makes for some rockiness if you are building current in = some scenarios on 9.2 where clang versions differ). That=92s much to be done, and not a lot of doing. I=92m likely = forgetting other areas we need to focus on as well... > I'm definitely not arguing "make everything work for old MIPS and ARM > platforms". I'm totally on board with the "C99 and then C11 should > just be required at this point." But part of the point of supporting > external compiler stuff isn't just to make MIPS, ARM and PPC stuff > work. It's to keep us honest. We as a project treat non-x86 as a > second class citizen and it shows in our build infrastructure (ie, > everything is native.) It's similar to the 32 vs 64 bit platform stuff > - it again made the codebase better, even if it was "ew, DEC alpha.=94 You can do arm and mips native builds, it is just that the machines that = you have available are significantly under resourced compared to x86 = servers. I mean, you aren=92t going to find any ARM or MIPS machines = that can do a build world in 12 minutes=85 > As an example here - I get the feeling that people care not about > 32-bit x86 and would like it to die. The lack of interest and use of > freebsd/i386 by developers sometimes shows - suddenly KVA isn't > infinite anymore and a lot more stuff assumes large amounts of > physical RAM. But besides the recent push by Intel for all of their > x86 32-bit embedded stuff (which is all Linux, by the way), those > decisions also impact the 32 bit ARM platforms. Those aren't going > away. There=92s two issues here. One is people assuming KVA is cheap and other = things like this. The second is bodies to help (a) educate and (b) = implement for the smaller platforms. Over the years we=92ve grown up a = fair amount of embedded platform support. Now the time is ripe to push = to the next level of support: where the server guys talk to the embedded = guys and vice versa on a more regular basis. I see this happening a bit = already, and would encourage more of it. Warner From owner-freebsd-arch@FreeBSD.ORG Sun Apr 6 21:56:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D2D95EF for ; Sun, 6 Apr 2014 21:56:35 +0000 (UTC) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2F44E0 for ; Sun, 6 Apr 2014 21:56:34 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id lx4so5443760iec.38 for ; Sun, 06 Apr 2014 14:56:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=FAV8J86MypwqsrPpxkALYu1tr/Sszj/0Vd2/luHOkpY=; b=k7VnbAVflbpNL/KzT2pzRP4oyMhomPm+c4UyiYWSQtfXjBpyDvSgCFn5KVXZa7ywMs IsVyp7u/CjNAVZpIJjRnlrKynuk1Zb3k/t0z27CSeilzPBi30o3cdmoRenIATQjYaYHh pja6P42z67bc6rvRJQA/bar2Dp2aiouAGHLfAgSDG7HpJC1qHg/pDGf/3Dyz/m+XRb1v 7X2PwScm+GKAYpU6oNz2IyJyhEhcO3EQ1oNdj4U9b/7lTzIy56V3uirxQCw2f+tEJKkh dNspRPFyHwZd6UAA7e9pDG75swXOAhEoxtZqwMk+bOhLt2Iqxins0AgRCsI4hkDcwu9f XAIA== X-Gm-Message-State: ALoCoQnwshfNftXi+BUyzl1HvMnqXjoXtY17V6u0aoEHzy4TE8qs95IbhRc00SEk2/ofNwe2sa9M X-Received: by 10.50.143.34 with SMTP id sb2mr16550262igb.11.1396821387760; Sun, 06 Apr 2014 14:56:27 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id u1sm24684443igm.8.2014.04.06.14.56.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 06 Apr 2014 14:56:27 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Warner Losh In-Reply-To: Date: Sun, 6 Apr 2014 15:56:26 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> To: Jordan Hubbard X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2014 21:56:35 -0000 On Apr 6, 2014, at 4:16 AM, Jordan Hubbard wrote: > Now it=92s 2014 and apparently we can=92t have nice things in the tree = because of MIPS? Maybe I=92m over-simplifying the argument, but even = simplistically I would easily understand First off, nobody every said we can=92t have nice things in the tree = because of MIPS. Where was that said? It can=92t possibly be true = because gcc supports blocks in the tree, so there=92s no impediment. = LLVM-based things? Show me the money and bring one to the table and we = can talk, but even then there=92s clang support for mips, so again = that=92s not a big deal. As for numbers, perhaps you are right about mips, perhaps not. There=92s = a thriving community, the code isn=92t holding the tree back, and things = do get fixed there. Maybe not as well as our ARM community, but it still = us. I hear a lot of FUD and chest pounding about how it is holding us = back, but I=92ve yet to see any real evidence of that proffered. Mips = and powerpc are in the tree because Juniper needs/wants them, and has = been contributing fixes to the tree over the years. External toolchain is coming along nicely given the timelines for 11, = which is where we committed to having it done and removing gcc/binutils = from the tree. clang is good, but it isn=92t quite ready for the = binutils removal yet. If the time comes and it isn=92t done, then we can = talk about how mips is holding things up (assuming the clang mips stuff = doesn=92t go in). But until then, show me the concrete examples where = there=92s an actual problem. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Apr 7 04:33:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58AAFFA4; Mon, 7 Apr 2014 04:33:02 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D28C6BF; Mon, 7 Apr 2014 04:33:01 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so5604647qga.4 for ; Sun, 06 Apr 2014 21:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Lms3+avPd4KOEtTPBwlXvv3706m9FAp0wsJKI4N+Quk=; b=nQPZv6L8sMXB1Jd+de7kqSSc+jFlk3UEkBtuQ2SGmVMx0ULvmamyR26GhLsNLHGK7P ZDP1yazgoN70/LwgMeJvsGryqlfY4AmkQsSxBtY9vGY0aSHn9h6gk7OLEtGybBwJymMy gsKjrNxm1GnV4ZohLRqsGFZdT9I8lvxTkhM6nEECZ7pZTKRnNSRQqgYTFbV3gxpi2/zj j3utj6gF092ayvf+gmrU+2lsQbu+T6F6tKb9lXTdk539n3Drqs18uIAsEsKq1MqPMuB4 E3YGGQzQILgefFs1ZLusuvwranspO6zCdM3s8ccqHSomP+qg014C7IOCMh7JU8H7jOlU bxhg== MIME-Version: 1.0 X-Received: by 10.140.93.22 with SMTP id c22mr11572948qge.53.1396845181185; Sun, 06 Apr 2014 21:33:01 -0700 (PDT) Received: by 10.224.50.206 with HTTP; Sun, 6 Apr 2014 21:33:01 -0700 (PDT) Date: Sun, 6 Apr 2014 21:33:01 -0700 Message-ID: Subject: patch: report per-CPU per-state wakeup counts From: Adrian Chadd To: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 04:33:02 -0000 Hi, We don't report how many actual sleeps (ie, wakeups) that we're doing via ACPI. I'd like to add support for this. http://people.freebsd.org/~adrian/misc/20140406-acpica-wakeup-stat-1.diff comments? -a From owner-freebsd-arch@FreeBSD.ORG Mon Apr 7 09:06:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD6601C4 for ; Mon, 7 Apr 2014 09:06:29 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B4C4FEF1 for ; Mon, 7 Apr 2014 09:06:29 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id BE5B072F7C; Mon, 7 Apr 2014 02:06:27 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 75693-01; Mon, 7 Apr 2014 02:06:27 -0700 (PDT) Received: from [10.8.0.22] (unknown [10.8.0.22]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 8F80C72F72; Mon, 7 Apr 2014 02:06:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396861587; bh=/5RvYXW8QyMnV1/NfQP961BNT7uLiKSWl0qcUVssR1k=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=tWP23QylWXhfbrQCWwFhPW7R1UdbOkdePp/Es1TROd3V8ie0rCj5w8lxNOLpMpQec o7/72pMDE8O6Iwd5dC2tqWvLGWSacqa5NOBVemJU8vBGKBPenZO3qpn2PlFrdNlEQS MeWoGSLTj5ztNcrL+/w0VIWG44UMNYF4PQLT7fh4= Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Jordan Hubbard In-Reply-To: <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> Date: Mon, 7 Apr 2014 14:06:12 +0500 Content-Transfer-Encoding: quoted-printable Message-Id: <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> To: Warner Losh X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 09:06:29 -0000 On Apr 7, 2014, at 2:56 AM, Warner Losh wrote: > First off, nobody every said we can=92t have nice things in the tree = because of MIPS. Where was that said? It can=92t possibly be true = because gcc supports blocks in the tree, so there=92s no impediment. = LLVM-based things? Show me the money and bring one to the table and we = can talk, but even then there=92s clang support for mips, so again = that=92s not a big deal. Perhaps I got lost somewhere along the way during the conversation = thread with David Chisnall, but it sounded like we were both in favor of = libdispatch and willing to at least grudgingly accept the proposition = that we=92d never break the deadlock between having an absolute = locked-in use case for it first vs having the fundamental technology = available and in a position to change the way we do async programming = (and perhaps get more dynamism into FreeBSD as a whole) when he said: "I would certainly be in favour of importing it. The package seems to = be on every FreeBSD machine that I use, so I've become accustomed to = having it there and just work.=94 Then he followed up with: "The slight problem, however, is that we would still like to be able to = build the base system with a more or less standard C compiler. Blocks = are in clang and are slowly making their way into commercial compilers, = but the only two versions of gcc that support them are the ones shipped = by Apple and FreeBSD.=94 and then: "For embedded uses, we'd also like to build FreeBSD with = vendor's-ugly-hacked-up-gcc-of-the-week. This is less of an issue now = for ARM, but MIPS vendors still hack up gcc in such a way that there's = no way that they can get their changes upstreamed and then ship the = result with their chips.=94 So what I took away from that was that my long and somewhat quixotic = attempts to get libdispatch into FreeBSD (notionally scheduled for 8.1, = then pushed out indefinitely) would probably remain quixotic due to a = desire to keep base buildable by a fairly broad and non-freebsd = controlled compiler toolchains in the case of MIPS. Did I = misunderstand something? I=92d still love to get libdispatch into base = such that other services can be layered on top of it. There=92s a = fundamental reason why we stuck it into Libsystem in OS X, and the = notification system and other IPC technologies I'm hoping for as part of = the =93services hub=94 work we=92ll be doing will all depend on = libdispatch and blocks. This is also, just in case anyone is wondering, far from academic. = Notification services are currently the bane of our existence over in = FreeNAS, with services like Samba depending on a very buggy libinotify = port for FreeBSD that we=92ve made some fixes to but ultimately just had = to disable entirely (so Samba in FreeNAS will not support kernel = notifications for awhile), it=92s that bad. Just judging by the blatant = nature of the bugs we=92ve found and fixed so far, it=92s also fairly = clear that nobody has been using that port very much, or very = intensively. Unfortunately, the mobile computing space (to say nothing of the = services space) needs to be a lot more aware of constant environmental = changes (network links coming and going, time zones changing on the fly, = service pairing relationships being created/broken, etc) and this takes = a bit more architecture. I=92m more than willing to help drive some of = that architecture, too, but I sure don=92t want to do it on top of = pthreads and raw socket I/O. :) - Jordan From owner-freebsd-arch@FreeBSD.ORG Mon Apr 7 15:03:27 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42261DA9 for ; Mon, 7 Apr 2014 15:03:27 +0000 (UTC) Received: from mail-ob0-f180.google.com (mail-ob0-f180.google.com [209.85.214.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1CA6AC1 for ; Mon, 7 Apr 2014 15:03:26 +0000 (UTC) Received: by mail-ob0-f180.google.com with SMTP id wn1so6609021obc.39 for ; Mon, 07 Apr 2014 08:03:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=tQoiiosKny/KYrqJ4RU1ayKbIS+PNloOv8JCsZ+cLLU=; b=QHzn6vyNnzXE/n4tHBAZf8LTsQ4wwosmpH1lPSHjr+g00sRTWI/UmzxRAOBaRom0r6 JmYWOcNodf7/I2yyKmlky3pA4rI7IpW2U4ADnCVSCOZnqdPhsPEEpvtYcW6a4X/t90WO xvchz/AnYQsjb8qVwyNXTOhQlgaxANH7BywpRWPoF/q2aKNgJIPZpqoRFikxpoQaB/q/ R1cBMY6U8mLwndO9czU2mpo3y2lEnpFn5BYdPigM1fuvqsxEIa0KFJSqVXVUsk5t58qu ZPO4cGq2gg6+KwH64hX2VeK5yCp1L5btOHqwXzWTUuq67phQTf+ajj+HHsr+69PgxCPj xCTw== X-Gm-Message-State: ALoCoQkKMhS9+0qKBA5hYPpcY/xJYmD8q0cLu95l7s/G1g80lYMPt3r+E6pjbFyZ5neWdfc7ppBO X-Received: by 10.182.55.103 with SMTP id r7mr1047923obp.78.1396883000201; Mon, 07 Apr 2014 08:03:20 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id h3sm13128780obh.8.2014.04.07.08.03.19 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Apr 2014 08:03:19 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Warner Losh In-Reply-To: <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> Date: Mon, 7 Apr 2014 09:03:18 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> To: Jordan Hubbard X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 15:03:27 -0000 On Apr 7, 2014, at 3:06 AM, Jordan Hubbard wrote: > "For embedded uses, we'd also like to build FreeBSD with = vendor's-ugly-hacked-up-gcc-of-the-week. This is less of an issue now = for ARM, but MIPS vendors still hack up gcc in such a way that there's = no way that they can get their changes upstreamed and then ship the = result with their chips.=94 This is a desire, not a hard requirement. As such, there may be missing = bits if you chose to go this route. At least that=92s been the notion in = the past when using llvm features has come up. The notion for this path = has always been =91It is possible, but only a subset of the = functionality may be available.=94 So if one were to add blocks, it would need a knob WITHOUT_BLOCKS that = would disable all functionality tied to them. For many applications, = this is a reasonable subset (based on my guesses at how intrusive this = would be to the system). And it might even be automatically selected = based on compiler support, but that=92s another can of worms. > So what I took away from that was that my long and somewhat quixotic = attempts to get libdispatch into FreeBSD (notionally scheduled for 8.1, = then pushed out indefinitely) would probably remain quixotic due to a = desire to keep base buildable by a fairly broad and non-freebsd = controlled compiler toolchains in the case of MIPS. Did I = misunderstand something? I=92d still love to get libdispatch into base = such that other services can be layered on top of it. There=92s a = fundamental reason why we stuck it into Libsystem in OS X, and the = notification system and other IPC technologies I'm hoping for as part of = the =93services hub=94 work we=92ll be doing will all depend on = libdispatch and blocks. I think you misunderstood. While there=92s a desire to be able to build = from vendor supplied gcc for latest and greatest silicon, that desire = has consequences. We already have a number of FreeBSD extensions in the = compiler, and those would have to be disabled for this vendor supplied = gcc compiler. > This is also, just in case anyone is wondering, far from academic. = Notification services are currently the bane of our existence over in = FreeNAS, with services like Samba depending on a very buggy libinotify = port for FreeBSD that we=92ve made some fixes to but ultimately just had = to disable entirely (so Samba in FreeNAS will not support kernel = notifications for awhile), it=92s that bad. Just judging by the blatant = nature of the bugs we=92ve found and fixed so far, it=92s also fairly = clear that nobody has been using that port very much, or very = intensively. >=20 > Unfortunately, the mobile computing space (to say nothing of the = services space) needs to be a lot more aware of constant environmental = changes (network links coming and going, time zones changing on the fly, = service pairing relationships being created/broken, etc) and this takes = a bit more architecture. I=92m more than willing to help drive some of = that architecture, too, but I sure don=92t want to do it on top of = pthreads and raw socket I/O. :) Understandable. So to take this a step further=85 There=92s many levels of integration = here=85 First, there=92s the kernel, which is most often the bit of = code people want/need the special compiler for. Next, there=92s having = the feature available in user land. Finally, there=92d be a wide-scale = integration of this feature. I see very few programs in base benefiting = from libdispatch, honestly, but that doesn=92t mean the set is empty. Do = you have a longer write up on what you=92d like to do here? I see a continuum of answer here: If you want to modify the kernel = extensive to use blocks, then that=92s going to be a much bigger problem = than having a few daemons and a library in the tree that require them = which is a bigger problem than having a few daemons using it, but = optionally, and a library which is a bigger problem than a library in = the tree which is a bigger problem than the status quo. Somewhere along = this continuum will be FreeBSD=92s sweet spot. I=92m guessing it might = be on the second notch (no wide-spread kernel changes to fundamental = bits, but with daemons that require it to work at all) or third notch = (daemons still work, but work better, faster, harder with blocks). So I do agree with you in many ways: this crazy fringe desire shouldn=92t = drive out all innovation in the tree. Where to drop the needle on the = continuum is an area where there=92s a rough, fuzzy consensus, but a = more concrete set of code desiring to be in the tree is needed. And 8.1 for libdispatch? Is it really so portable it would work with our = old, crappy gcc pre blocks update? Warner= From owner-freebsd-arch@FreeBSD.ORG Mon Apr 7 18:13:38 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B5DFA16; Mon, 7 Apr 2014 18:13:38 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF4F5FDF; Mon, 7 Apr 2014 18:13:37 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id z60so6683619qgd.28 for ; Mon, 07 Apr 2014 11:13:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Mhx0hnO2BuLb0m/xzWKsIL95DNg3sm6xUA2BZKiplHo=; b=F+4K/kosqbBqutx0OJojLF7ccrTCekY/TD8YMlr9P/Ys/+AbVwDv9+IIIRcJuQ3edf 2XiApPIrqXDJlt4ZISdlXR2Tgb+vbDtfuQ5VfZs8Zsc8F+vwhb4+DXxURnvHSBqnm/bJ C5qA5QCU8FY+SSU+UWP6BZkjRJquhqZ4kUvTIxqeRChPBfO0kXONRHdG4asl5hkdVuk9 5nZ7e5P89rQnzdruCD11LWw/e/13Uif0biu5S4A8G6Ht5zLhJO4X30pZgIFAynr5OC+T fjFxcxYIF79u/OEApFR+tGxg1dL3ZcXfVhxCks2Q/wInkgabC5a2AA31TwONB7G7HDlM 3f6g== MIME-Version: 1.0 X-Received: by 10.224.16.69 with SMTP id n5mr35570888qaa.7.1396894417171; Mon, 07 Apr 2014 11:13:37 -0700 (PDT) Sender: edschouten@gmail.com Received: by 10.140.96.203 with HTTP; Mon, 7 Apr 2014 11:13:37 -0700 (PDT) In-Reply-To: References: <20140405153621.GH21331@kib.kiev.ua> Date: Mon, 7 Apr 2014 20:13:37 +0200 X-Google-Sender-Auth: BNjOoqilOLMjQuz26W9CrcmBksc Message-ID: Subject: Re: [Patch] kqueue(2) <-> procdesc(4): EVFILT_PROCDESC From: Ed Schouten To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 Cc: Robert Watson , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Apr 2014 18:13:38 -0000 Hi, On 6 April 2014 17:16, Ed Schouten wrote: > My plan was initially to mimick filt_proc() almost entirely and just > use KNOTE_LOCKED(..., NOTE_EXIT) just like EVFILT_PROC does, but > unfortunately this doesn't seem possible, because if we went down this > path, there would be no way for us to activate the knote in > procdesc_kqfilter() immediately, as KNOTE_ACTIVATE() is not available > outside of kern_event.c. > > This is why I just changed the code, so that procdesc_kqops_event() is > almost literally a copy of filt_proc(), which the difference that it > tests against PDF_EXITED instead of using the hint. While there, I > made some style fixes to filt_proc(). Thoughts? After looking at the kqueue code in a bit more detail, I think I have now found a nice middle way. kevent(2) itself will only call into the kqfilter with hint == 0. I have decided to let procdesc_kqops_event() be a literal copy of filt_proc(), with the difference that if hint == 0, it tests against PDF_EXITED. As it seems to work quite well, I've decided to push this in (r264231). Thanks for the review! -- Ed Schouten From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 07:03:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78DBC733 for ; Tue, 8 Apr 2014 07:03:37 +0000 (UTC) Received: from mail.iXsystems.com (newknight.ixsystems.com [206.40.55.70]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CA991BEB for ; Tue, 8 Apr 2014 07:03:36 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 8995B73FDF; Tue, 8 Apr 2014 00:03:35 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 69944-07; Tue, 8 Apr 2014 00:03:35 -0700 (PDT) Received: from [10.8.0.18] (unknown [10.8.0.18]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 5515273FD9; Tue, 8 Apr 2014 00:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1396940615; bh=vhi29iPZoQ6K5RaYBl0csmQAHA3vTaP/f86XNqZPCIM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=EnV5v+vlHHsoHknMmdq6ROLTfYg4me3CDRb/42WX+hRFPxpEd/CZz5T+pBqSNKIiK dd+6dc8+TcuElkWDsL8Nx3jhAbxzA7qzCuRm/MbQ69dIQOlkL5gPZq2lOCZgK1z5ma O0jjUnCkkgebDv6zxlcjAV4qUHBOaikqVgpcOiOk= Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Jordan Hubbard In-Reply-To: Date: Tue, 8 Apr 2014 12:03:31 +0500 Content-Transfer-Encoding: quoted-printable Message-Id: <482357EF-5596-45FD-8D2C-3742BB4872E8@ixsystems.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> To: Warner Losh X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 07:03:37 -0000 On Apr 7, 2014, at 8:03 PM, Warner Losh wrote: > This is a desire, not a hard requirement. As such, there may be = missing bits if you chose to go this route. At least that=92s been the = notion in the past when using llvm features has come up. The notion for = this path has always been =91It is possible, but only a subset of the = functionality may be available.=94 >=20 > So if one were to add blocks, it would need a knob WITHOUT_BLOCKS that = would disable all functionality tied to them. For many applications, = this is a reasonable subset (based on my guesses at how intrusive this = would be to the system). And it might even be automatically selected = based on compiler support, but that=92s another can of worms. I=92m glad to hear that building with foreign compilers is more of a = desire than a hard constraint. I=92m still not sure what the scenario = would look like where libdispatch has been built WITHOUT_BLOCKS and is = now essentially API incompatible with the libdispatch compiled = WITH_BLOCKS (since you wouldn=92t get functions like dispatch_async() = but would get the dispatch_async_f() function which takes a function = pointer/context variable instead), but I guess that=92s a problem for = the hypothetical wacky-compiler user to figure out? You seemed to imply that when you said FreeBSD was already using a = number of custom compiler extensions, I just wasn=92t sure that this = extended as far as =93you can build FreeBSD, for the most part, with = your weird-ass compiler but you can also expect it to be a different = FreeBSD than what=92s in base for x86_64 (for example) because various = things will be switched off, API incompatible, or simply not compiled at = all." If that=92s the case, then woohoo, when can I import libdispatch into = base? :) [I=92m being semi-facetious here, I realize there are a few = more steps to go]. > So to take this a step further=85 There=92s many levels of integration = here=85 First, there=92s the kernel, which is most often the bit of = code people want/need the special compiler for. Next, there=92s having = the feature available in user land. Finally, there=92d be a wide-scale = integration of this feature. I see very few programs in base benefiting = from libdispatch, honestly, but that doesn=92t mean the set is empty. Do = you have a longer write up on what you=92d like to do here? Sure, I can write something up as part of my =93any objections to doing = this?=94 step, but just to summarize briefly for this conversation: 1. Libdispatch does not need to touch the kernel at all, nor does the = kernel require blocks support. The pthread_workqueue() stuff that = Stacey Son did all those years ago would be a nice optimization (this = lets libdispatch balance its thread pool loads across multiple = applications, not just within a single application) but it=92s not = strictly necessary, and it can also be added later without any consumer = of libdispatch (henceforth referred to by its other name of GCD, because = that=92s easier to type!) needing to know or care. 2. Once we have GCD in base, I can then start looking at adding = libnotify and its associated notifyd daemon, which is a generic system = notification mechanism. 3. Once libnotify is in base, I=92ll then add some basic notifications = to Libc, like timezone changes, hostname changes, and so on, = invalidating some of its internal caches in the face of said changes = (there are some fairly expensive stat() calls you can eliminate once you = have the far lighter-weight shared memory flag checking that notifyd = provides, which also helps save power by not walking around the = filesystem so much). The notifyd service can also translate filesystem = notifications into system notifications, which means services like samba = and netatalk can detect (cheaply) when something has changed out from = under them. Beyond that, we probably need to talk about another daemon, which on OS = X is called configd, which deals with larger scope configuration changes = like network devices coming and going, default routes changing, and so = on. This does require some kernel up-call mechanisms, but it will = =93publish=94 its results in terms of notify(3) notifications so = applications can subscribe to those events the same way. If we can hack = usbd into doing the same for USB events, that=92s great, otherwise I = would imagine configd dealing with that sort of hardware notification = service as well. > I see a continuum of answer here: If you want to modify the kernel = extensive to use blocks, then that=92s going to be a much bigger problem = than having a few daemons and a library in the tree that require them = which is a bigger problem than having a few daemons using it, but = optionally, and a library which is a bigger problem than a library in = the tree which is a bigger problem than the status quo. I.. think=85 I follow that sentence enough to hope I=92ve answered it. = No blocks in the kernel. Nothing more than a few daemons and a library = and, at some point, a way of figuring out when the kernel has made = certain changes to the system configuration, if and where said changes = can=92t be intercepted and dealt with at the libc layer or otherwise = noticed by a user-land daemon which can post the notifications for any = subscribers of them. As to the subscribers, they won=92t have to use blocks unless they = choose to consume one or more of notify(3)=92s block APIs. The notify = mechanism was written with both old-style and new-style clients in mind. = If your daemon wants a signal when the notification it=92s subscribing = to changes, fine. If it wants some data on a file descriptor which it=92s= poll(3)/kqueue(3)ing, fine. See = https://developer.apple.com/library/mac/documentation/Darwin/Reference/Man= Pages/man3/notify.3.html and, of course, ignore the mach-specific = delivery methods as N/A. > And 8.1 for libdispatch? Is it really so portable it would work with = our old, crappy gcc pre blocks update? Sorry, misunderstanding. I was referring to the original slashdot / = phoronix article which stated that GCD would be coming to FreeBSD in = 8.1. Apparently, according to FreeBSD=92s own wiki = (https://wiki.freebsd.org/GCD) it works, too. - Jordan From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 15:42:25 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB13E9A8 for ; Tue, 8 Apr 2014 15:42:25 +0000 (UTC) Received: from mail-pb0-f43.google.com (mail-pb0-f43.google.com [209.85.160.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7196410CF for ; Tue, 8 Apr 2014 15:42:25 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id um1so1209057pbc.2 for ; Tue, 08 Apr 2014 08:42:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=XDoKCl2QJS1groSbN8A2CA7qVzl3hh7baCFz8d6wuEs=; b=KXUIJ0sU+Zu9BCqv/72+CApbBKZdH7JJQWLs2OmQq50Fzqipo4hW//1waM37c4aIjv r3DlxgNrJu8q6cgEHaaZTTgRhFJo8t/QsgGiZK2VghM8nkGKzGKzhsG4e2btpp/I11Y1 2LISrSYlwuuX5aydDVglDkdwAvnxN1uSVQ/cI5N6ePbmiTwYGyE48fzXjZexHGxm0/HD jB147YEekkwAS7o0qB9m6mndx/S4zuCjuUoLsXccBAS7Ll43ApyrosPlAIMTOtf9uXSU R+47MFVFSkRp/V9I5g74U9fjww0T1MqMRQmrq7Q4HDPiIg1dm9ooKj0rxao2jZEzLYaw FMiA== X-Gm-Message-State: ALoCoQmjUIoBDeZ4N8wPpV0QcvZH6QvfIl4hJKiqk0AhH5bceL+/CXMjhDmxJJ0LHH43eM80jH2t X-Received: by 10.68.198.97 with SMTP id jb1mr5511052pbc.104.1396971739522; Tue, 08 Apr 2014 08:42:19 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id pq3sm5363311pbb.57.2014.04.08.08.42.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 08:42:18 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Warner Losh In-Reply-To: <482357EF-5596-45FD-8D2C-3742BB4872E8@ixsystems.com> Date: Tue, 8 Apr 2014 09:42:16 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <94AE42FF-9F01-4A92-AAB9-86D9705AE27C@bsdimp.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> <482357EF-5596-45FD-8D2C-3742BB4872E8@ixsystems.com> To: Jordan Hubbard X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 15:42:25 -0000 On Apr 8, 2014, at 1:03 AM, Jordan Hubbard wrote: >=20 > On Apr 7, 2014, at 8:03 PM, Warner Losh wrote: >=20 >> This is a desire, not a hard requirement. As such, there may be = missing bits if you chose to go this route. At least that=92s been the = notion in the past when using llvm features has come up. The notion for = this path has always been =91It is possible, but only a subset of the = functionality may be available.=94 >>=20 >> So if one were to add blocks, it would need a knob WITHOUT_BLOCKS = that would disable all functionality tied to them. For many = applications, this is a reasonable subset (based on my guesses at how = intrusive this would be to the system). And it might even be = automatically selected based on compiler support, but that=92s another = can of worms. >=20 > I=92m glad to hear that building with foreign compilers is more of a = desire than a hard constraint. I=92m still not sure what the scenario = would look like where libdispatch has been built WITHOUT_BLOCKS and is = now essentially API incompatible with the libdispatch compiled = WITH_BLOCKS (since you wouldn=92t get functions like dispatch_async() = but would get the dispatch_async_f() function which takes a function = pointer/context variable instead), but I guess that=92s a problem for = the hypothetical wacky-compiler user to figure out? Yes.=20 > You seemed to imply that when you said FreeBSD was already using a = number of custom compiler extensions, I just wasn=92t sure that this = extended as far as =93you can build FreeBSD, for the most part, with = your weird-ass compiler but you can also expect it to be a different = FreeBSD than what=92s in base for x86_64 (for example) because various = things will be switched off, API incompatible, or simply not compiled at = all.=94 The extensions right now are fairly modest. The biggest one being the = extended format checker for the kernel printf %b. You can expect it to = be a bit different than the amd64 FreeBSD right now if it is a different = architecture due to differing levels of pmap maturity and optimization, = for example. While most of the experience is the same, the further you = get away from the solidly Tier 1 platforms the more roll your own = FreeBSD becomes and the more handholding it requires. This doesn=92t = make that experience bad, mind you, just that more care is needed for = tier 2 and tier 3 platforms (whatever those terms really mean :). > If that=92s the case, then woohoo, when can I import libdispatch into = base? :) [I=92m being semi-facetious here, I realize there are a few = more steps to go]. If it is either completely disabled when the compiler doesn=92t support = blocks, or degrades gracefully automatically to avoiding blocks, I have = no problems with it. >> So to take this a step further=85 There=92s many levels of = integration here=85 First, there=92s the kernel, which is most often = the bit of code people want/need the special compiler for. Next, there=92s= having the feature available in user land. Finally, there=92d be a = wide-scale integration of this feature. I see very few programs in base = benefiting from libdispatch, honestly, but that doesn=92t mean the set = is empty. Do you have a longer write up on what you=92d like to do here? >=20 > Sure, I can write something up as part of my =93any objections to = doing this?=94 step, but just to summarize briefly for this = conversation: >=20 > 1. Libdispatch does not need to touch the kernel at all, nor does the = kernel require blocks support. The pthread_workqueue() stuff that = Stacey Son did all those years ago would be a nice optimization (this = lets libdispatch balance its thread pool loads across multiple = applications, not just within a single application) but it=92s not = strictly necessary, and it can also be added later without any consumer = of libdispatch (henceforth referred to by its other name of GCD, because = that=92s easier to type!) needing to know or care. >=20 > 2. Once we have GCD in base, I can then start looking at adding = libnotify and its associated notifyd daemon, which is a generic system = notification mechanism. >=20 > 3. Once libnotify is in base, I=92ll then add some basic notifications = to Libc, like timezone changes, hostname changes, and so on, = invalidating some of its internal caches in the face of said changes = (there are some fairly expensive stat() calls you can eliminate once you = have the far lighter-weight shared memory flag checking that notifyd = provides, which also helps save power by not walking around the = filesystem so much). The notifyd service can also translate filesystem = notifications into system notifications, which means services like samba = and netatalk can detect (cheaply) when something has changed out from = under them. I see no reason in principle that this couldn=92t be done. There might = be bumps along the way, but there always are=85 I=92d love to see this = stuff in FreeBSD, but I=92m also not in a position to devote time to it. > Beyond that, we probably need to talk about another daemon, which on = OS X is called configd, which deals with larger scope configuration = changes like network devices coming and going, default routes changing, = and so on. This does require some kernel up-call mechanisms, but it = will =93publish=94 its results in terms of notify(3) notifications so = applications can subscribe to those events the same way. If we can hack = usbd into doing the same for USB events, that=92s great, otherwise I = would imagine configd dealing with that sort of hardware notification = service as well. usbd? Where have you been. We=92ve not had a usbd in the base system = since before you started at Apple :). What you call configd is basically = handled by devd at the moment. It would be nice if devd grew = notification support, since right now it just has a tiny character = device interface with a workable, but imperfect, protocol. >> I see a continuum of answer here: If you want to modify the kernel = extensive to use blocks, then that=92s going to be a much bigger problem = than having a few daemons and a library in the tree that require them = which is a bigger problem than having a few daemons using it, but = optionally, and a library which is a bigger problem than a library in = the tree which is a bigger problem than the status quo. >=20 > I.. think=85 I follow that sentence enough to hope I=92ve answered it. = No blocks in the kernel. Nothing more than a few daemons and a library = and, at some point, a way of figuring out when the kernel has made = certain changes to the system configuration, if and where said changes = can=92t be intercepted and dealt with at the libc layer or otherwise = noticed by a user-land daemon which can post the notifications for any = subscribers of them. Yea, it was a tortured way of stating the continuum... > As to the subscribers, they won=92t have to use blocks unless they = choose to consume one or more of notify(3)=92s block APIs. The notify = mechanism was written with both old-style and new-style clients in mind. = If your daemon wants a signal when the notification it=92s subscribing = to changes, fine. If it wants some data on a file descriptor which it=92s= poll(3)/kqueue(3)ing, fine. See = https://developer.apple.com/library/mac/documentation/Darwin/Reference/Man= Pages/man3/notify.3.html and, of course, ignore the mach-specific = delivery methods as N/A. That=92s really cool stuff. I=92d love to see it. >> And 8.1 for libdispatch? Is it really so portable it would work with = our old, crappy gcc pre blocks update? >=20 > Sorry, misunderstanding. I was referring to the original slashdot / = phoronix article which stated that GCD would be coming to FreeBSD in = 8.1. Apparently, according to FreeBSD=92s own wiki = (https://wiki.freebsd.org/GCD) it works, too. Yea, I=92m not sure what happened with that. But the FreeBSD wiki is = editable by any developer at any time, or any friend of a developer that = creates an account, so with controls like that, there=92s bound to be = some inaccurate information there :) Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 20:41:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 086D948A for ; Tue, 8 Apr 2014 20:41:44 +0000 (UTC) Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D1F0C11F4 for ; Tue, 8 Apr 2014 20:41:43 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id kx10so1538757pab.19 for ; Tue, 08 Apr 2014 13:41:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:content-type :content-transfer-encoding:subject:message-id:date:to:mime-version; bh=1Na0CRKStcgbo+4gyqwksP+ec2bbVxfB0XiNUF8ooBE=; b=KQx/y6HAUggdriV6BZDUXUlFFCDEJbk4aDPloq264Hs3S3YjZ7H3+luF0qdEBmHsjm 8klktNZgFpfIpjji7nwXhGnQ4H9jQSy5+7v6elQlkD3pXGapVmITE0vNaf1R/LIgZ0Wc exdFMZ4yNpIE0GV0hVPPGBiLuX9j1xJdDJwo+9uotZhfpl0SxSdhs/yp/uqXI2cn3Ma5 p2RwScDtLgBeH922r7+CHZWPC6nqifHd5HJwrKze1nZXi9Y4v6g+pf21MQoklJ30m1aj zcQMXQjgmH5Emx/Y+wcpdvPSX/IokOlq8vEdpYh3sctgcgpCyCjfXMmvx2iSIWx8+YaZ mOlQ== X-Gm-Message-State: ALoCoQl82g+pIGBUIJf6ZYbXQeI0PPsgDPR9aksOOaGnVR2CoqFTVh3d4ou+59hJucuS3cECDEfY X-Received: by 10.66.253.170 with SMTP id ab10mr7115657pad.53.1396989277904; Tue, 08 Apr 2014 13:34:37 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id db3sm6561283pbb.10.2014.04.08.13.34.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 13:34:37 -0700 (PDT) Sender: Warner Losh From: Warner Losh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Time for turning off gdb by default? Or worse... Message-Id: Date: Tue, 8 Apr 2014 14:34:35 -0600 To: freebsd-arch Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 20:41:44 -0000 Greetings, The gdb in the tree seems to be of very limited usefulness these days. = It doesn=92t seem to work on clang-enabled architectures w/o building = -gdwarf-2, it doesn=92t seem to work with threaded applications, and on = some architectures it doesn=92t seem to work at all (mips comes to mind, = but it may have been the two binaries I tried). It seems like we=92d be doing our users a favor by applying: diff -r 8bfca9de870e share/mk/bsd.own.mk --- a/share/mk/bsd.own.mk +++ b/share/mk/bsd.own.mk @@ -266,7 +266,6 @@ WITH_HESIOD=3D FREEBSD_UPDATE \ GAMES \ GCOV \ - GDB \ GNU \ GNU_GREP_COMPAT \ GPIB \ @@ -355,6 +354,7 @@ WITH_HESIOD=3D CLANG_EXTRAS \ CTF \ DEBUG_FILES \ + GDB \ HESIOD \ INSTALL_AS_USER \ LLDB \ to the tree, which will turn gdb off by default. It may make more sense = to just remove it entirely, but I=92m not sure I want to go there just = yet in case there are things that I=92m missing. I believe that the port = will be adequate for all architectures we support, but haven=92t tested = this directly yet. I do know that on amd64, the port just worked, where = the in-tree gdb was an epic fail. Comments? Warner= From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 21:24:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 816B6EE2 for ; Tue, 8 Apr 2014 21:24:57 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask.apl.washington.edu", Issuer "troutmask.apl.washington.edu" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4469B1739 for ; Tue, 8 Apr 2014 21:24:57 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.8/8.14.8) with ESMTP id s38LOa4e075453 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 8 Apr 2014 14:24:36 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.8/8.14.8/Submit) id s38LOZuq075452; Tue, 8 Apr 2014 14:24:35 -0700 (PDT) (envelope-from sgk) Date: Tue, 8 Apr 2014 14:24:35 -0700 From: Steve Kargl To: Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... Message-ID: <20140408212435.GA75404@troutmask.apl.washington.edu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 21:24:57 -0000 On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: (courtesy line wrap to something well below 80 characters) > The gdb in the tree seems to be of very limited usefulness > these days. It doesn?t seem to work on clang-enabled > architectures w/o building -gdwarf-2, it doesn?t seem to work > with threaded applications, and on some architectures it > doesn?t seem to work at all (mips comes to mind, but it may > have been the two binaries I tried). > (patch removed) > to the tree, which will turn gdb off by default. It may make > more sense to just remove it entirely, but I?m not sure I want > to go there just yet in case there are things that I?m missing. > I believe that the port will be adequate for all architectures > we support, but haven?t tested this directly yet. I do know > that on amd64, the port just worked, where the in-tree gdb > was an epic fail. I suppose the obvious questions are: 1) Is lldb ready for prime time? 2) What effect does this have on kgdb? Note, /sys/conf/NOTES contains #makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Should this be updates to DEBUG=-gdwarf-2? PS: You'll need to sweep src/ for references to gdb(1). -- Steve From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 21:26:36 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3147A1E7 for ; Tue, 8 Apr 2014 21:26:36 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4C371759 for ; Tue, 8 Apr 2014 21:26:35 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.8/8.14.8) with ESMTP id s38LQTvX077615; Wed, 9 Apr 2014 00:26:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s38LQTvX077615 Received: (from kostik@localhost) by tom.home (8.14.8/8.14.8/Submit) id s38LQTme077614; Wed, 9 Apr 2014 00:26:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Apr 2014 00:26:29 +0300 From: Konstantin Belousov To: Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... Message-ID: <20140408212629.GD21331@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2W4Na7kn/Mq3HLmY" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 21:26:36 -0000 --2W4Na7kn/Mq3HLmY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: > Greetings, >=20 > The gdb in the tree seems to be of very limited usefulness these days. It= doesn?t seem to work on clang-enabled architectures w/o building -gdwarf-2= , it doesn?t seem to work with threaded applications, and on some architect= ures it doesn?t seem to work at all (mips comes to mind, but it may have be= en the two binaries I tried). >=20 > It seems like we?d be doing our users a favor by applying: >=20 > diff -r 8bfca9de870e share/mk/bsd.own.mk > --- a/share/mk/bsd.own.mk > +++ b/share/mk/bsd.own.mk > @@ -266,7 +266,6 @@ WITH_HESIOD=3D > FREEBSD_UPDATE \ > GAMES \ > GCOV \ > - GDB \ > GNU \ > GNU_GREP_COMPAT \ > GPIB \ > @@ -355,6 +354,7 @@ WITH_HESIOD=3D > CLANG_EXTRAS \ > CTF \ > DEBUG_FILES \ > + GDB \ > HESIOD \ > INSTALL_AS_USER \ > LLDB \ >=20 > to the tree, which will turn gdb off by default. It may make more sense t= o just remove it entirely, but I?m not sure I want to go there just yet in = case there are things that I?m missing. I believe that the port will be ade= quate for all architectures we support, but haven?t tested this directly ye= t. I do know that on amd64, the port just worked, where the in-tree gdb was= an epic fail. >=20 > Comments? Do we need kgdb ? Do we need a debugger that is aware of our signal frames which are not annotated with the dwarf ? I personally need and use this. I also use standalone build of the stock gdb. Both in-tree gdb/kgdb and stock gdb are incommutable (for me). --2W4Na7kn/Mq3HLmY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTRGmEAAoJEJDCuSvBvK1BXGkP/3e5CRas+yyrWYfnUBHlA00J /a0V4VFexVOj/IATSn0i4NqGO+iF7KHy3SFaq290YB11KYUg22poNHw0Klx/jK7C 4Sp7FF2FNN4Js0nuGSK+gHRLchU36X+6rKNviC6G2WVTovZ4H+RuEkHDxD/GkYtp 5j6viibiTutntAGExCLe0LyJQpvdRmnp9eJ+TNM2nH6EehY9fcET01RqpYY1SZMn T99uwHrcxyHsrDGLe0o6+KS7OaJqs0BhsVUtbFbCAHB0mcZMV4RaDiyysfGcinp+ hHjGC+uEJLz2hKzyQHw6RUsS1ncw8apASFA1GPeFblvDV9TzbxDiGPItYtEOslv3 urhd2d2QE4kH+o2DbKBwmIg2q647ADzKGAUE8uWuJcgShwHiV6hO4jWZY+dH72sJ a+UF784Vb6HUWBG0KvsAKC/hFcXaD+oggLdAnYlpQ4Uz7bwzTnDZ2h7FNoK4hXBv H0VIn/aMHI44AAaidrA1VSS5b8phgppLw5En+GoT2iUhi12P67hz0ADsG2NhmCvA YH0YkrpvqKk889IpVKHF+NnP/Zu2w7uACmYW/32+Y96JpINInZ5dBhfszEKiUiG2 GSOjpPGySMk0jzfTx2jHaLfRhyawEFV2tCjzNd2QIS0HbA26r94izM2V9IIFkrKe EgMUw9ZVOVvKeBOovQXu =grSe -----END PGP SIGNATURE----- --2W4Na7kn/Mq3HLmY-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 21:43:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0464E722 for ; Tue, 8 Apr 2014 21:43:18 +0000 (UTC) Received: from mail-pd0-f178.google.com (mail-pd0-f178.google.com [209.85.192.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA86618FA for ; Tue, 8 Apr 2014 21:43:17 +0000 (UTC) Received: by mail-pd0-f178.google.com with SMTP id x10so1541590pdj.23 for ; Tue, 08 Apr 2014 14:43:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=UrvBY7QDP21jHZ4q9QOb7XbHiLDdK4iw1PnTXN36AYo=; b=drxxNbJdjjela2QOMSWcxhJcZKanY7e9eO9e/NnAR14w9BKiKfAQVJEpXgdzbmCgfr BVUGLVhvElcabsjG73KguKSgcr8hYduWCs1K/BUvIs6nrdGPwjFRKnnPVoKUX0mOXmCW C5IiGZzQvoLz57jFeoRX01LxclMoy4sGSvsak+uml2om2So277jYpWVB3zCmNEsDhMC/ oqj8HmM3YDRy8QbUVOSiuHm1I90JPN+TDySUudqtvBttsX/Bx7+60OpfuWdJNRV81vjI jiOod3cpq9XY9T98HvJ58IndU4tCdmbYc4TqeDUUrdkUZZwbF1Y4V/UK1/7bfm15JC+h Fj0g== X-Gm-Message-State: ALoCoQlUaTYq2SO4LQWxOpZXfdx4rGHaDNV58bBezC+0jDtDF3QwAT9a/HNPFLpWqhNFRsPjhdTX X-Received: by 10.68.240.5 with SMTP id vw5mr7225049pbc.113.1396993391018; Tue, 08 Apr 2014 14:43:11 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id sm5sm15554245pab.19.2014.04.08.14.43.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 14:43:10 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Warner Losh In-Reply-To: <20140408212435.GA75404@troutmask.apl.washington.edu> Date: Tue, 8 Apr 2014 15:43:08 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> References: <20140408212435.GA75404@troutmask.apl.washington.edu> To: Steve Kargl X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 21:43:18 -0000 On Apr 8, 2014, at 3:24 PM, Steve Kargl = wrote: > On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: >=20 > (courtesy line wrap to something well below 80 characters) >=20 >> The gdb in the tree seems to be of very limited usefulness >> these days. It doesn?t seem to work on clang-enabled >> architectures w/o building -gdwarf-2, it doesn?t seem to work >> with threaded applications, and on some architectures it >> doesn?t seem to work at all (mips comes to mind, but it may >> have been the two binaries I tried). >>=20 >=20 > (patch removed) >=20 >> to the tree, which will turn gdb off by default. It may make >> more sense to just remove it entirely, but I?m not sure I want >> to go there just yet in case there are things that I?m missing. >> I believe that the port will be adequate for all architectures >> we support, but haven?t tested this directly yet. I do know >> that on amd64, the port just worked, where the in-tree gdb >> was an epic fail. >=20 > I suppose the obvious questions are: >=20 > 1) Is lldb ready for prime time? Doesn=92t matter. > 2) What effect does this have on kgdb? Note, /sys/conf/NOTES contains Unfortunately, kgdb isn=92t available as a port, so that does matter. It = is one thing arguing against this change. > #makeoptions DEBUG=3D-g #Build kernel with gdb(1) debug = symbols >=20 > Should this be updates to DEBUG=3D-gdwarf-2? Nope. It should stay exactly as it is. We convert -g to -gdwarf-2 for = those compilers that need it. > PS: You'll need to sweep src/ for references to gdb(1). No. It is just not built by default, not being kicked out of the tree. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 21:47:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C98898FC for ; Tue, 8 Apr 2014 21:47:02 +0000 (UTC) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89D171947 for ; Tue, 8 Apr 2014 21:47:02 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id x3so1850462qcv.12 for ; Tue, 08 Apr 2014 14:47:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=53a6FhEdW+WCvEJXC9IBfnzTV9GtTa4lpZ8Wj0fV9RM=; b=B93GNRFz8qHMe7HhMdF987nYKN2POCj7TV53wz09sAXTbfJLbxf2z7pVa7QOjPjXvZ AzPEkVTu8Qbig1wbeVLLZlZY30hI9wZRsFULZgqRmudXmncyN86PSs/zY/utQJMw0xTd qwj0nZvkt5ykivqOAIu3mfzq7rltWkJYzE9VvvSMzyeWSUlRw3wSVObUevkRKo5kJ46A /8gEEs6Q3xe0NZ43JUxrwGT3XfmXHD24YNdNb9l7aCfQZjL7QPJFU04BWa/aNfquskXe nCc/yASsPEdZgm3c/+7fGxxlM5PXLtx/XToTOAWqqPU8/kCTahGLtisAX9VkuPz8J3k3 rlpQ== MIME-Version: 1.0 X-Received: by 10.229.230.68 with SMTP id jl4mr8064522qcb.2.1396993621748; Tue, 08 Apr 2014 14:47:01 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Tue, 8 Apr 2014 14:47:01 -0700 (PDT) In-Reply-To: <20140408212435.GA75404@troutmask.apl.washington.edu> References: <20140408212435.GA75404@troutmask.apl.washington.edu> Date: Tue, 8 Apr 2014 17:47:01 -0400 X-Google-Sender-Auth: vPE3emF1Z3wYeP0zn3zAb_HHTcs Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Ed Maste To: Steve Kargl Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 21:47:02 -0000 On 8 April 2014 17:24, Steve Kargl wrote: > I suppose the obvious questions are: > > 1) Is lldb ready for prime time? It's fairly usable for amd64 today, especially in comparison to our ancient GDB. It works reasonably well as a cross-debugger for 64-bit MIPS userland core files too. There is some non-functional i386 support, and no support beyond that for other architectures on FreeBSD yet. I'd appreciate hearing test results from anyone willing to give it a try. From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 22:05:47 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC88EF67 for ; Tue, 8 Apr 2014 22:05:47 +0000 (UTC) Received: from mail-ve0-x236.google.com (mail-ve0-x236.google.com [IPv6:2607:f8b0:400c:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92B051AD6 for ; Tue, 8 Apr 2014 22:05:47 +0000 (UTC) Received: by mail-ve0-f182.google.com with SMTP id jw12so1339564veb.41 for ; Tue, 08 Apr 2014 15:05:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=5RLJkx1LFitKo/vtk7+rhejs80C+F2FSQTjuUdsyRyI=; b=vXd+j9d938sLmJ44UkGY4lp0ecovfXbqciYjiJE+mGL+jCVfOtRG01r36N4NT8NoRN yggBfWD5xoembcmwF+vzxXavIVGBTUEQG8bw2mgxCfJyPdCfDvgObzEZHR3q9bEjGr4t GJ6gDchtA953nON/3hsn07hUjlHCrZmAG7tFs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=5RLJkx1LFitKo/vtk7+rhejs80C+F2FSQTjuUdsyRyI=; b=fdqIJIjqN8sMqcdJylbQp2X4uFGaL9Px3k8A23fNjTjLWKLd5dqW25clRozBqZKQBD +NfhYPEs71fHUaa8FtRMThczb4I9h4GH25RqpPvVgVeSByl24lHjukRUT/dDPp3O4HFX uMxCW6BxPyDWiV/92iUG+z+pTO1wfOfUeMVJ3oNClDVczk87ukyc/3J1SfwkWNzOeCJo AH6sfF6OjEjEMrjv9q57tcPqobzFn86C3R5bmXZaOlz5xR2LLtIpVjPBs1fV0xYbFFfC tipIUEfo+W5l8oX00kwtLb2+y2OsbLiLVG89jC09EqcmvLBGgspxF+KoZSXV5l8H6egh 3/tA== X-Gm-Message-State: ALoCoQk5qFyGmYRGxfgtwXFeEd82mw0KEKHmNsGACydIuxJR6H7jxlRnpLPkIA8GxiDz7HvUmOlw MIME-Version: 1.0 X-Received: by 10.52.253.75 with SMTP id zy11mr4516954vdc.10.1396994746565; Tue, 08 Apr 2014 15:05:46 -0700 (PDT) Received: by 10.220.30.69 with HTTP; Tue, 8 Apr 2014 15:05:46 -0700 (PDT) In-Reply-To: <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> Date: Tue, 8 Apr 2014 15:05:46 -0700 Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Peter Wemm To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Cc: Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 22:05:47 -0000 On Tue, Apr 8, 2014 at 2:43 PM, Warner Losh wrote: > > On Apr 8, 2014, at 3:24 PM, Steve Kargl wrote: [..] >> 2) What effect does this have on kgdb? Note, /sys/conf/NOTES contains > > Unfortunately, kgdb isn't available as a port, so that does matter. It is one > thing arguing against this change. This is a big deal, unfortunately. kgdb would be a big loss. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 22:12:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFEBB2FF for ; Tue, 8 Apr 2014 22:12:48 +0000 (UTC) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 811321BBE for ; Tue, 8 Apr 2014 22:12:48 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id rd3so1621291pab.25 for ; Tue, 08 Apr 2014 15:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=OTw090Fm/o2Mmv6fu3NR8bkOlsEoTXVuFXTgQikG4Z4=; b=ZuEJx2uHAjiboMn6SeALM6bQ0PyHZ3COUfNgqX+/7RBfonIe+KBOaVG5UGVj5naqtx ILlPuRnKha3FY0OtBY6rZmgc7AtvuQkcCzUtUSsf+XcZ/qXzRu1B+pe/qRF/Rx2LCiuH z+7kKr88bX8R8mSPEIxbF7dviDEGfND3oTJHTjPE9pWx8t2papESHZF2RgGkMDjY8pQ+ wLqQRdCXcXJQnMWSURGIqJTxUrPKGQq8qrDyrzoXL6truIprzrAXBd3B3rRYuGSmUplb lYzEfmBeNqjrWIzsYvTU5C6QI8fRDjxQZHzx3NrFlM+APBAY+fbZQJCQRSBD1tPkAvVB U35w== X-Received: by 10.68.197.8 with SMTP id iq8mr7626419pbc.124.1396995168153; Tue, 08 Apr 2014 15:12:48 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id nc1sm6858746pbc.32.2014.04.08.15.12.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 15:12:45 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <5344745C.90509@FreeBSD.org> Date: Tue, 08 Apr 2014 15:12:44 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 22:12:48 -0000 On 04/08/14 15:05, Peter Wemm wrote: > On Tue, Apr 8, 2014 at 2:43 PM, Warner Losh wrote: >> >> On Apr 8, 2014, at 3:24 PM, Steve Kargl wrote: > [..] > >>> 2) What effect does this have on kgdb? Note, /sys/conf/NOTES contains >> >> Unfortunately, kgdb isn't available as a port, so that does matter. It is one >> thing arguing against this change. > > > This is a big deal, unfortunately. kgdb would be a big loss. > +1. kgdb shouldn't just disappear without an alternate in place. Navdeep From owner-freebsd-arch@FreeBSD.ORG Tue Apr 8 22:17:11 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E946740C for ; Tue, 8 Apr 2014 22:17:11 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA22F1BF3 for ; Tue, 8 Apr 2014 22:17:11 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WXeKM-0008aV-63; Tue, 08 Apr 2014 22:17:10 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s38MH7qS093170; Tue, 8 Apr 2014 16:17:07 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+JfnYozdJmDKkMdVZndhTy Subject: Re: Time for turning off gdb by default? Or worse... From: Ian Lepore To: Warner Losh In-Reply-To: <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> Content-Type: text/plain; charset="iso-8859-7" Date: Tue, 08 Apr 2014 16:17:07 -0600 Message-ID: <1396995427.81853.449.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by damnhippie.dyndns.org id s38MH7qS093170 Cc: Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 22:17:12 -0000 On Tue, 2014-04-08 at 15:43 -0600, Warner Losh wrote: > On Apr 8, 2014, at 3:24 PM, Steve Kargl wrote: >=20 > > On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: > >=20 > > (courtesy line wrap to something well below 80 characters) > >=20 > >> The gdb in the tree seems to be of very limited usefulness > >> these days. It doesn?t seem to work on clang-enabled > >> architectures w/o building -gdwarf-2, it doesn?t seem to work > >> with threaded applications, and on some architectures it > >> doesn?t seem to work at all (mips comes to mind, but it may > >> have been the two binaries I tried). > >>=20 > >=20 > > (patch removed) > >=20 > >> to the tree, which will turn gdb off by default. It may make > >> more sense to just remove it entirely, but I?m not sure I want > >> to go there just yet in case there are things that I?m missing. > >> I believe that the port will be adequate for all architectures > >> we support, but haven?t tested this directly yet. I do know > >> that on amd64, the port just worked, where the in-tree gdb > >> was an epic fail. > >=20 > > I suppose the obvious questions are: > >=20 > > 1) Is lldb ready for prime time? >=20 > Doesn=A2t matter. >=20 > > 2) What effect does this have on kgdb? Note, /sys/conf/NOTES contain= s >=20 > Unfortunately, kgdb isn=A2t available as a port, so that does matter. I= t is one thing arguing against this change. >=20 > > #makeoptions DEBUG=3D-g #Build kernel with gdb(1) debug symbols > >=20 > > Should this be updates to DEBUG=3D-gdwarf-2? >=20 > Nope. It should stay exactly as it is. We convert -g to -gdwarf-2 for t= hose compilers that need it. Only when building the kernel. For userland we've got nothing. gdb aside, even addr2line doesn't work on userland binaries anymore. It used to be hard to do debugging for arm. Now it's impossible. -- Ian From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 02:50:16 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34638534 for ; Wed, 9 Apr 2014 02:50:16 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 069911A60 for ; Wed, 9 Apr 2014 02:50:15 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s392nx7K084204 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 8 Apr 2014 19:50:04 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5344B551.30200@freebsd.org> Date: Wed, 09 Apr 2014 10:49:53 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Konstantin Belousov , Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... References: <20140408212629.GD21331@kib.kiev.ua> In-Reply-To: <20140408212629.GD21331@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 02:50:16 -0000 On 4/9/14, 5:26 AM, Konstantin Belousov wrote: > On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: > I personally need and use this. I also use standalone build of the > stock gdb. Both in-tree gdb/kgdb and stock gdb are incommutable (for > me). if lldb works then fine for userland, but I REALLY need kgdb to work. also does lldb work with the xxgdb and ddd frontends? does it have its own frontend that makes it usable? From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 02:57:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00EBD797; Wed, 9 Apr 2014 02:57:03 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A296F1B23; Wed, 9 Apr 2014 02:57:03 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id dc16so554324qab.25 for ; Tue, 08 Apr 2014 19:57:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=B4q/SBNPt0ug7KiJbeQoPY4XhxqaqvykN+gWgBXvpuc=; b=XfwhGX1erUKPAoopj1GIJmne4h1WuDsemStvsj2aUBrramPFt+SJMnqYB/8Bca7Ws6 DNbfmIo3/SIKiq8sdNsI8+IgNLKbGZtMMswtcyFhTRfn00kmouwpqO1+P8mABYnC/MV4 /EBalqhQRxTcYVwvfcprfXqqL4OiSlpNbjGGD7bPjCMXNb/9HzDcgYTCSnpKiQxvzkjs rBI9HOYb4WRMcN7KDXB7ZVpjBCMjt1mY5lVsDaX5V9xZbdS7oMLkRsO5ChEpEU9o06dY DUJjpJVsQA6a4VMd0ynvMOKPcCvBiZsXFvIBCbiZzoxqr68j4LzDeQh0McNkALWu3jzG 9mvQ== MIME-Version: 1.0 X-Received: by 10.140.87.5 with SMTP id q5mr8923662qgd.31.1397012222830; Tue, 08 Apr 2014 19:57:02 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Tue, 8 Apr 2014 19:57:02 -0700 (PDT) In-Reply-To: <5344B551.30200@freebsd.org> References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> Date: Tue, 8 Apr 2014 19:57:02 -0700 X-Google-Sender-Auth: 4Dq_-KMMxq0nMZ5NrZ3EqCENrYg Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 02:57:04 -0000 On 8 April 2014 19:49, Julian Elischer wrote: > On 4/9/14, 5:26 AM, Konstantin Belousov wrote: >> >> On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: >> I personally need and use this. I also use standalone build of the stock >> gdb. Both in-tree gdb/kgdb and stock gdb are incommutable (for me). > > > if lldb works then fine for userland, but I REALLY need kgdb to work. > also does lldb work with the xxgdb and ddd frontends? > does it have its own frontend that makes it usable? > Or can we just hack llvm to generate dwarf-2 by default, and only flip it to dwarf-4 when it's done? because this whole situation is just plain dumb. -a From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 03:18:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0FA0C2C for ; Wed, 9 Apr 2014 03:18:02 +0000 (UTC) Received: from mail-pb0-f45.google.com (mail-pb0-f45.google.com [209.85.160.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 816781D8A for ; Wed, 9 Apr 2014 03:18:02 +0000 (UTC) Received: by mail-pb0-f45.google.com with SMTP id uo5so1900304pbc.18 for ; Tue, 08 Apr 2014 20:17:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=BcwC6/OK8k5QcmNtGi4tGsDbIsBBupgTip94vuPpJQs=; b=PpTuO3bmQ2HVcsdSzzBgM+fiUUiAXxby08bdoARuUR5Kf5xuHZ7BLS5dmL9JQZAnVM jV7uTGzgfX4DNQ8KEsDvJiF7kKSpoMptk76ybdx9Ye+vj1jSbMMwJefrGRmgWuTYznZf LdgVHdHrhB394mrHjN8Vk9faCNdKLi3ZWFbSASqEESyfO/Zl2IsojFpRG4k7PLbzt6n4 FC+/hFZUnPZZbikhG5zC4ExWSeuOTSNh/KUn9XO7T+dJqArzw5a/OOriXvwcVq2KB7CI D6hwGrwf326E1usOfoJ1G6Fqg8U5xvTcO//Y9jAF2BwUQLJo/rBODnc4NKAbpbICHIa7 SMAw== X-Gm-Message-State: ALoCoQkds0h6eMEfahBM1+6Wvratf7NvGCaVuMV/VsX+Gv0O70uvs1sM2tSOc5VYBs4L+Jeq2wHc X-Received: by 10.68.250.3 with SMTP id yy3mr8991187pbc.56.1397013476423; Tue, 08 Apr 2014 20:17:56 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id qq5sm7896935pbb.24.2014.04.08.20.17.55 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 20:17:55 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=iso-8859-7 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Warner Losh In-Reply-To: <1396995427.81853.449.camel@revolution.hippie.lan> Date: Tue, 8 Apr 2014 21:17:53 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> <1396995427.81853.449.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1874) Cc: Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 03:18:02 -0000 On Apr 8, 2014, at 4:17 PM, Ian Lepore wrote: > On Tue, 2014-04-08 at 15:43 -0600, Warner Losh wrote: >> On Apr 8, 2014, at 3:24 PM, Steve Kargl = wrote: >>=20 >>> On Tue, Apr 08, 2014 at 02:34:35PM -0600, Warner Losh wrote: >>>=20 >>> (courtesy line wrap to something well below 80 characters) >>>=20 >>>> The gdb in the tree seems to be of very limited usefulness >>>> these days. It doesn?t seem to work on clang-enabled >>>> architectures w/o building -gdwarf-2, it doesn?t seem to work >>>> with threaded applications, and on some architectures it >>>> doesn?t seem to work at all (mips comes to mind, but it may >>>> have been the two binaries I tried). >>>>=20 >>>=20 >>> (patch removed) >>>=20 >>>> to the tree, which will turn gdb off by default. It may make >>>> more sense to just remove it entirely, but I?m not sure I want >>>> to go there just yet in case there are things that I?m missing. >>>> I believe that the port will be adequate for all architectures >>>> we support, but haven?t tested this directly yet. I do know >>>> that on amd64, the port just worked, where the in-tree gdb >>>> was an epic fail. >>>=20 >>> I suppose the obvious questions are: >>>=20 >>> 1) Is lldb ready for prime time? >>=20 >> Doesn=A2t matter. >>=20 >>> 2) What effect does this have on kgdb? Note, /sys/conf/NOTES = contains >>=20 >> Unfortunately, kgdb isn=A2t available as a port, so that does matter. = It is one thing arguing against this change. >>=20 >>> #makeoptions DEBUG=3D-g #Build kernel with = gdb(1) debug symbols >>>=20 >>> Should this be updates to DEBUG=3D-gdwarf-2? >>=20 >> Nope. It should stay exactly as it is. We convert -g to -gdwarf-2 for = those compilers that need it. >=20 > Only when building the kernel. For userland we've got nothing. gdb > aside, even addr2line doesn't work on userland binaries anymore. It > used to be hard to do debugging for arm. Now it's impossible. I thought I=A2ve had it work when I used DEBUG_FLAGS=3D-gdwarf-2 rather = than DEBUG_FLAGS=3D-g. Warner= From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 03:33:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7A57D308 for ; Wed, 9 Apr 2014 03:33:12 +0000 (UTC) Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F2B61067 for ; Wed, 9 Apr 2014 03:33:12 +0000 (UTC) Received: by mail-pd0-f171.google.com with SMTP id r10so1877086pdi.30 for ; Tue, 08 Apr 2014 20:33:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:content-type :content-transfer-encoding:subject:message-id:date:to:mime-version; bh=EX4XMNEVCXUXqFBS2E8KvbuxmG7DWV3y4qtlCDNceig=; b=UJJ1e5OvdrGPz4ROarDuAIJqQOzYNd1NGeY9qaXWzfJHxcXUK+7MLo/wGSq9GbJH/P BSBLl0Y0uiU/FfQY/zUydzT3zU6npnzkop2BI6VJOzxB528eDw10DdVhh2OBOkK7inoc H0q8yFyXgrGksVcpQ0elRNbUzo4tP8Rda5HBAnqyWy440lIO96ndknSHE0BM6X2Vzj8G Ye/1KOQzreEFYlZlq6YOjfV0lG+1g/k2hmrgICnwOoeR2T5T4LiT7l8mUrZI7vBu+aL0 8ZM/OZHfB2nzwtfJWBy8pLXPrYF51j2NkH1WL89c60bX0l1to4WZXB+aL1qZ0Lt4vHqa e4Ww== X-Gm-Message-State: ALoCoQkH9PRnumu5pOdN7Mxkl3PT1nYIOuuxAzA5+OUdz7t3oySgvZty47ll+q443+OCkaha3GsT X-Received: by 10.68.239.137 with SMTP id vs9mr8959667pbc.84.1397013918056; Tue, 08 Apr 2014 20:25:18 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ci4sm7912931pbb.50.2014.04.08.20.25.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Apr 2014 20:25:17 -0700 (PDT) Sender: Warner Losh From: Warner Losh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Separating out building bootstrap and system compilers Message-Id: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> Date: Tue, 8 Apr 2014 21:25:15 -0600 To: freebsd-arch Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 03:33:12 -0000 I=92d love to be able to say make buildworld WITHOUT_GCC=3Dt WITHOUT_CLANG=3Dt and get a working system out of it, without compilers. Too bad I can=92t = right now. Luckily, I worked up these patches. Here=92s my proposed commit message. = Please comment on the patch (which can be found at = http://people.freebsd.org/~imp/patch-queue/bootstrap) Separate out enabling building clang and/or gcc for the system and building clang and/or gcc as the bootstrap compiler. Normally, the default compiler is used. WITH_CLANG_BOOTSTRAP and/or WITH_GCC_BOOTSTRAP will enable building these compilers as part bootstrap phase. WITH/WITHOUT_CLANG_IS_CC controls which compiler is used by default for the bootstrap phase, as well as which compiler is installed as cc. buildworld now successfully completes building the cross compiler with WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt and produces a built system with neither of these included. MK_CROSS_COMPILER will now force MK_CLANG_BOOTSTRAP=3Dno and MK_GCC_BOOTSTRAP=3Dno. BOOTSTRAP_COMPILER was considered, but rejected, since pc98 needs both clang and gcc to bootstrap still. It should be revisisted in the future if this requirement goes away. Values should be gcc, clang or none. Chances are good that MK_BINUTILS is a good candidate for similar treatment. We likely need to fold Xxx causing things to magically not happen into this scheme as well, but that may be a larger, more = disruptive change. Comments? Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 03:38:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A159525 for ; Wed, 9 Apr 2014 03:38:23 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D039510CF for ; Wed, 9 Apr 2014 03:38:22 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s393cK8Y041456 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Apr 2014 20:38:20 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s393cKiU041455; Tue, 8 Apr 2014 20:38:20 -0700 (PDT) (envelope-from jmg) Date: Tue, 8 Apr 2014 20:38:20 -0700 From: John-Mark Gurney To: Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... Message-ID: <20140409033820.GL34745@funkthat.com> Mail-Followup-To: Warner Losh , freebsd-arch References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 08 Apr 2014 20:38:20 -0700 (PDT) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 03:38:23 -0000 Warner Losh wrote this message on Tue, Apr 08, 2014 at 14:34 -0600: > The gdb in the tree seems to be of very limited usefulness these days. It doesn?t seem to work on clang-enabled architectures w/o building -gdwarf-2, it doesn?t seem to work with threaded applications, and on some architectures it doesn?t seem to work at all (mips comes to mind, but it may have been the two binaries I tried). > > It seems like we?d be doing our users a favor by applying: > > diff -r 8bfca9de870e share/mk/bsd.own.mk > --- a/share/mk/bsd.own.mk > +++ b/share/mk/bsd.own.mk > @@ -266,7 +266,6 @@ WITH_HESIOD= > FREEBSD_UPDATE \ > GAMES \ > GCOV \ > - GDB \ > GNU \ > GNU_GREP_COMPAT \ > GPIB \ > @@ -355,6 +354,7 @@ WITH_HESIOD= > CLANG_EXTRAS \ > CTF \ > DEBUG_FILES \ > + GDB \ > HESIOD \ > INSTALL_AS_USER \ > LLDB \ > > to the tree, which will turn gdb off by default. It may make more sense to just remove it entirely, but I?m not sure I want to go there just yet in case there are things that I?m missing. I believe that the port will be adequate for all architectures we support, but haven?t tested this directly yet. I do know that on amd64, the port just worked, where the in-tree gdb was an epic fail. Does the gdb in ports support cross debugging? Or at least easily? I don't see anything in the Makefile that implies that it does... Also, I noticed this: ONLY_FOR_ARCHS= i386 amd64 powerpc powerpc64 # untested elsewhere, might work That's missing a lot of archs... Though it looks like 66 might have better support... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 08:46:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01BD76EE; Wed, 9 Apr 2014 08:46:07 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C8BE1733; Wed, 9 Apr 2014 08:46:05 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id hr17so903212lab.32 for ; Wed, 09 Apr 2014 01:46:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cXPzOIqTA1Xv1GXqe5A4oMjDk82K+lLhJ+7qTSAhd8o=; b=k0RqRYQHTq3l7k5N7YYnq0GAM7n2oNV7sYLf+LnGBz3ags9IFWwphy6FpG+9M9u3Tq DW7/tfzX1Badwnw9z6zuwu9S+rIPufw9Mioyga5vahSyIJhBLbymFgB5YO5aitdqNOgZ iKnqn1FKCW9JtUpXiy0CbQhi9lFrMwnU3vSFHQRtNEgFathhKR8gMt9oAAOLFfPd9Z9Q az3qLJ9KG5VAhFqUpRx8BldJZDUrQXOvrcQZlghvkt1uLpfAHi5ZSHuzWq4XfhHlu7ty CYVbyMQa3oEcQAihCwcYXP3bo/ikVdYlk7FnyenA5m3bPCMke4DJ0um9LGs0s5SJlCnX gWNw== MIME-Version: 1.0 X-Received: by 10.112.142.105 with SMTP id rv9mr794348lbb.42.1397033163932; Wed, 09 Apr 2014 01:46:03 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.169.68 with HTTP; Wed, 9 Apr 2014 01:46:03 -0700 (PDT) In-Reply-To: References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> Date: Wed, 9 Apr 2014 01:46:03 -0700 X-Google-Sender-Auth: N_Fz4l2u17yoY7RZ4Jj2AbjTUzI Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Craig Rodrigues To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 08:46:07 -0000 On Tue, Apr 8, 2014 at 7:57 PM, Adrian Chadd wrote: > > Or can we just hack llvm to generate dwarf-2 by default, and only flip > it to dwarf-4 when it's done? > > because this whole situation is just plain dumb. That's the behavior of llvm for FreeBSD 10.x and earlier: http://lists.freebsd.org/pipermail/svn-src-all/2014-March/082850.html but not for CURRENT. -- Craig From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 08:47:14 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F9CA794; Wed, 9 Apr 2014 08:47:14 +0000 (UTC) Received: from mail-qa0-x235.google.com (mail-qa0-x235.google.com [IPv6:2607:f8b0:400d:c00::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF4191741; Wed, 9 Apr 2014 08:47:13 +0000 (UTC) Received: by mail-qa0-f53.google.com with SMTP id w8so2091695qac.26 for ; Wed, 09 Apr 2014 01:47:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+ILMB8BRcJ8kPPlPe3L6uPprG179KN6WVt5p/mawrIY=; b=ZH920P+e6cKfcd7YeSNtad8woEp2luJYWeHQ7gxIlc6dHeXzI0+rvRROK/neGFASyx h5hIfVRKnpcbe3w7b8hb1FMfaL3BV16haIV1R45h0FrinbHvKfskURpxHmQh6SaxJBhJ UOR/VbmkCRrCRHDtJDctgocBPDmgBAZWxya/k7hNM3rYc4ec6ruO+H7LHPF8DpA9qQH9 LRUfUp2SCtfSOJ2mMmGDHNseA5JBEK1b5g3ZvehKHjQOYUJN0VjL4GPTSlWGQlN3wo0z vWUDebbMF5ZasYawI8QsRs0abvEzBcd6ezYhFFA5lBpxn+nwXbtenqBbge6T66d0s0MU MkYQ== MIME-Version: 1.0 X-Received: by 10.140.38.149 with SMTP id t21mr10252900qgt.24.1397033233100; Wed, 09 Apr 2014 01:47:13 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Wed, 9 Apr 2014 01:47:13 -0700 (PDT) In-Reply-To: References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> Date: Wed, 9 Apr 2014 01:47:13 -0700 X-Google-Sender-Auth: NIRQPvbRulD5m2nE9X-q3fn_dKo Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: Craig Rodrigues Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 08:47:14 -0000 Can we please flip this on for -head? -a On 9 April 2014 01:46, Craig Rodrigues wrote: > On Tue, Apr 8, 2014 at 7:57 PM, Adrian Chadd wrote: >> >> Or can we just hack llvm to generate dwarf-2 by default, and only flip >> it to dwarf-4 when it's done? >> >> because this whole situation is just plain dumb. > > That's the behavior of llvm for FreeBSD 10.x and earlier: > http://lists.freebsd.org/pipermail/svn-src-all/2014-March/082850.html > > but not for CURRENT. > -- > Craig From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 13:03:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2142FF6C; Wed, 9 Apr 2014 13:03:51 +0000 (UTC) Received: from mail-qg0-x233.google.com (mail-qg0-x233.google.com [IPv6:2607:f8b0:400d:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C70041542; Wed, 9 Apr 2014 13:03:50 +0000 (UTC) Received: by mail-qg0-f51.google.com with SMTP id q108so2283724qgd.10 for ; Wed, 09 Apr 2014 06:03:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=lgwgw/qIZcJzCZlv8cUbWkMTF7pTZj1OMmafW+h5tnI=; b=Q2mcYFxoLmcIfd2aksSW6c3ysp6WV19xipKOdq2y20Xb0OcW0NMVLEoRajzGUljiYV bSlHdbzlQVGEyiAvR2guCGYMIcozGmtMuxsnKG5dcuZsCw71bCmXHPiaVyEvfNJyxQcn QI9i3WL0yOb+U4w6vIIAX8NKU0UsNyA2aM+LC4AzB7QPxB78/ML7XyAh2f1DPOEoy2gS zLKooZWXt2QvE9CiQKIyGE4SnxG8TQegu1S9fhle7hdJDpxsDB3jN+pioIuPveQlqhlY WPkS2pKs+YxB6g3xR3DcHmELDwLQ5mAaYbpJ6OUvl0YMQOTt/yK4WOnVvPcAHzZya+yJ aukw== MIME-Version: 1.0 X-Received: by 10.229.112.5 with SMTP id u5mr12347019qcp.3.1397048629963; Wed, 09 Apr 2014 06:03:49 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Wed, 9 Apr 2014 06:03:49 -0700 (PDT) In-Reply-To: <5344B551.30200@freebsd.org> References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> Date: Wed, 9 Apr 2014 09:03:49 -0400 X-Google-Sender-Auth: D3d3AjaQGROziyLH6S6mWT2b32I Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Ed Maste To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 13:03:51 -0000 On 8 April 2014 22:49, Julian Elischer wrote: > > also does lldb work with the xxgdb and ddd frontends? > does it have its own frontend that makes it usable? LLDB recently gained a curses-based UI. It seems pretty usable, although I generally prefer to use a debugger's command-line interface. I've put a screenshot of it here: http://people.freebsd.org/~emaste/lldb/lldb-gui.png -Ed From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 16:11:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC4DF7DE for ; Wed, 9 Apr 2014 16:11:03 +0000 (UTC) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B235B1BF2 for ; Wed, 9 Apr 2014 16:11:02 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id s39GAxPo016200; Wed, 9 Apr 2014 11:10:59 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id s39GAx3q016199; Wed, 9 Apr 2014 11:10:59 -0500 (CDT) (envelope-from brooks) Date: Wed, 9 Apr 2014 11:10:59 -0500 From: Brooks Davis To: Warner Losh Subject: Re: Separating out building bootstrap and system compilers Message-ID: <20140409161059.GA14501@lor.one-eyed-alien.net> References: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 16:11:03 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 08, 2014 at 09:25:15PM -0600, Warner Losh wrote: > I?d love to be able to say >=20 > make buildworld WITHOUT_GCC=3Dt WITHOUT_CLANG=3Dt >=20 > and get a working system out of it, without compilers. Too bad I can?t ri= ght now. >=20 > Luckily, I worked up these patches. Here?s my proposed commit message. Pl= ease comment on the patch > (which can be found at http://people.freebsd.org/~imp/patch-queue/bootstr= ap) >=20 > Separate out enabling building clang and/or gcc for the system and > building clang and/or gcc as the bootstrap compiler. Normally, the > default compiler is used. WITH_CLANG_BOOTSTRAP and/or > WITH_GCC_BOOTSTRAP will enable building these compilers as part > bootstrap phase. WITH/WITHOUT_CLANG_IS_CC controls which compiler is > used by default for the bootstrap phase, as well as which compiler is > installed as cc. buildworld now successfully completes building the > cross compiler with WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt and produces a > built system with neither of these included. >=20 > MK_CROSS_COMPILER will now force MK_CLANG_BOOTSTRAP=3Dno and > MK_GCC_BOOTSTRAP=3Dno. >=20 > BOOTSTRAP_COMPILER was considered, but rejected, since pc98 needs both > clang and gcc to bootstrap still. It should be revisisted in the > future if this requirement goes away. Values should be gcc, clang or > none. >=20 > Chances are good that MK_BINUTILS is a good candidate for similar > treatment. We likely need to fold Xxx causing things to magically not > happen into this scheme as well, but that may be a larger, more disruptive > change. >=20 > Comments? Looks good to me. Thanks for taking the next step in disentangling these variables. -- Brooks --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iKYEARECAGYFAlNFcRNfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDY1NUQ1MTlDMjZBNzgyRTcyNTI5OUJGMDVE OEU4QkU5RjIzODFBRDQACgkQXY6L6fI4GtRs/wCgooYMLB1WsRu9mE9IcG4D5p7H qrUAoI4xtag7j2wl6pR38/oYOzNG11Ki =+SWV -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 17:21:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A3121FA for ; Wed, 9 Apr 2014 17:21:06 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 733671526 for ; Wed, 9 Apr 2014 17:21:06 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 93CEBB95D; Wed, 9 Apr 2014 13:21:04 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Time for turning off gdb by default? Or worse... Date: Wed, 9 Apr 2014 11:45:58 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201404091145.58792.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 09 Apr 2014 13:21:04 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 17:21:06 -0000 On Tuesday, April 08, 2014 4:34:35 pm Warner Losh wrote: > Greetings, >=20 > The gdb in the tree seems to be of very limited usefulness these days. It > doesn=92t seem to work on clang-enabled architectures w/o building -gdwar= f-2, > it doesn=92t seem to work with threaded applications, and on some > architectures it doesn=92t seem to work at all (mips comes to mind, but i= t may > have been the two binaries I tried). >=20 > It seems like we=92d be doing our users a favor by applying: >=20 > diff -r 8bfca9de870e share/mk/bsd.own.mk > --- a/share/mk/bsd.own.mk > +++ b/share/mk/bsd.own.mk > @@ -266,7 +266,6 @@ WITH_HESIOD=3D > FREEBSD_UPDATE \ > GAMES \ > GCOV \ > - GDB \ > GNU \ > GNU_GREP_COMPAT \ > GPIB \ > @@ -355,6 +354,7 @@ WITH_HESIOD=3D > CLANG_EXTRAS \ > CTF \ > DEBUG_FILES \ > + GDB \ > HESIOD \ > INSTALL_AS_USER \ > LLDB \ >=20 > to the tree, which will turn gdb off by default. It may make more sense to > just remove it entirely, but I=92m not sure I want to go there just yet in > case there are things that I=92m missing. I believe that the port will be > adequate for all architectures we support, but haven=92t tested this dire= ctly > yet. I do know that on amd64, the port just worked, where the in-tree gdb > was an epic fail. kgdb is a must. I think it would be less work to forward port kgdb support into gdb7 from ports than to keep our ancient gdb alive. Some things I can think of for gdb7: 1) The threads patch could be greatly simplified if we fixed the ptrace backend to properly handle inferiors with tids. This would remove a lot of the threads patch where the thread inferior tries to invoke ptrace directly and convert registers, etc. The way it does this now is a total hack and requires much larger patches. This would also make it a lot easier to get thread debugging working on more architectures as the thread-db bits would become mostly MI (if not entirely) 2) Porting the kgdb frontend to work with gdb7. It would be nicer to have a more modern base for kgdb and the ability to use python scripting with kgdb, custom printers for in-kernel structures, etc. I think if we have a useful devel/kgdb that builds against devel/gdb we can probably think about retiring gdb, but it's premature right now. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 19:48:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B23AABD2 for ; Wed, 9 Apr 2014 19:48:30 +0000 (UTC) Received: from nm38-vm9.bullet.mail.bf1.yahoo.com (nm38-vm9.bullet.mail.bf1.yahoo.com [72.30.239.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 332CD16C4 for ; Wed, 9 Apr 2014 19:48:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1397072803; bh=DmoHsASxZy7AhSDZgIfa860PIayJx0Lxo/5XtP9GP4I=; h=Received:Received:Received:X-Yahoo-Newman-Id:X-Yahoo-Newman-Property:X-YMail-OSG:X-Yahoo-SMTP:X-Rocket-Received:Message-ID:Date:From:Organization:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=Ug1kYCzvELDG0pwxHYzOqa6aVj/2nt/obi82+u9oF8zuEZseWiM0gKVktBJ4F//kh9RoWk1yhgDlhZ1mns1GuahwtZaXfP87rTIwzBSAq5/L1OujOQIK1YCBIK4iY+ivfAa8CBF45AidpXrRLoFG5jlbFYE1BqEM0P0k0DmiB25Mo/nChc42lpmxKieCzFAU2az3HoOYw5+uZ11b6kCSgTmeJyjwF7LhyAwMlBcERWjmqnoUUZOiAgutm/anGqjKFHHfYCfHVmwSdiC5o3CHc+l+k+hmoakakK3ZwML3C0iq3H6So3AbLJ1yZ1LG34+WIJfvo/424lg5rVJkg/f18Q== DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s2048; d=yahoo.com; b=nRRpha5EZVv+Pb/RWa8PIYeXM9R4WgJb8sWPL18w+3lqwGuDZH85bESkHQ6UB5CUUxsrID28JQtT/WvkOLq6l5Dx1m6qN2cYC+VjdBF3KfEXhKErgNplgc2LAX3rvDOd8wGqgTiLnU4LO+ie68lwIWqzKKwnKyQQnY+4Ih+YjC0fWwwGa3TQED5AmHLWedBjlfhgixbFmhKxvZWhcCsCocWxN+tj3baKeAmpzEKt3BYeJXpRglZfdUGQWDx+Zm9NN3QaahtrLSZpVdWVjDzAqHCcwu6d+hOpzBD9WCDQmJlSRLciyDEmr8aBhoePBRDT9pQZzA5q7jIqlgLlrG+/0A==; Received: from [98.139.215.141] by nm38.bullet.mail.bf1.yahoo.com with NNFMP; 09 Apr 2014 19:46:43 -0000 Received: from [98.139.211.196] by tm12.bullet.mail.bf1.yahoo.com with NNFMP; 09 Apr 2014 19:46:43 -0000 Received: from [127.0.0.1] by smtp205.mail.bf1.yahoo.com with NNFMP; 09 Apr 2014 19:46:43 -0000 X-Yahoo-Newman-Id: 68033.58350.bm@smtp205.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: SbvSNZwVM1mh2R1dJVkM7Oy99Yu2mFBcVFaaCHgdjUtrMzY P42rGFlOLeUjdwI4qEb5FTigZiohbfId0WBNnIOl9BkvGYC3_yBV9_Z3wQHC yqpgWOkJ68.je.V_7Vu7Wj2Q8P4SL8HuTnkTMtMnE9MYiDTO685jnkzdxpYT PbDQGs4VKwT6hlowYysVFnuXmbi8VQxCIx4noVtuplJpy1J6atPEuCjhqzsK gTU0ywjZbM_nSIwUHngF4t7YGW_D7HYW6pDsPMeVrjbdy4AA0E.mLvqxyXzA zddjj3Hzq_KmhMLISeflu8ZDdnA33XaEyq2lPA09xGv5dCOziJDonXlgGa2N IS.h5hXbsLZUx0a7jzBI5PiFq2AcGKW23uRFmff2jNxJ48QbBu1bSwEQZh3i Nl2mAgsKmTWHzo32f4B84rueBTlSvkNfsSqJxbgYjBcHvFGxmQTHPMDL4VkJ 97.aYIirMRR6bgpOU.ippGzkn9NSfenQsQhATIZ9vEVenEM_N9rqOdoPCifs QlqsosBnrPRhKvBH.s9ClQPx4dFU- X-Yahoo-SMTP: xcjD0guswBAZaPPIbxpWwLcp9Unf X-Rocket-Received: from [192.168.0.102] (pfg@190.157.126.109 with plain [98.139.211.125]) by smtp205.mail.bf1.yahoo.com with SMTP; 09 Apr 2014 12:46:43 -0700 PDT Message-ID: <5345A3A5.5020400@FreeBSD.org> Date: Wed, 09 Apr 2014 14:46:45 -0500 From: Pedro Giffuni Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Jordan Hubbard Subject: Re: Compiler toolchain roadmap References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 19:48:30 -0000 FWIW; I have been looking at what in the ports tree could make some use of libdispatch. So far the only candidates seem to be Apache's httpd MPM and maybe cups and Webkit. I, for one, would welcome more work to finish Apple's GCD support and make more use of libdispatch in base (sort(1) would be a good candidate). Don't expect explicit permission to get started though, just do such work in a branch and see how far it goes. Pedro. From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 21:13:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2FA3458E for ; Wed, 9 Apr 2014 21:13:43 +0000 (UTC) Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F26181145 for ; Wed, 9 Apr 2014 21:13:42 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id y10so2950842pdj.27 for ; Wed, 09 Apr 2014 14:13:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=g1TBZZ58RJtzGWfbkEfRcViZI1SNvfbEdDR0AsDgUN4=; b=Z1E6zD21TzgW+JIwDzD77mhlPe06klWO2a24SR/lzCQN1IJjy+/xyoSFfLK9fw8BFs fqO4a2WWYezBohOR1nWYG8F1SGQYz1YTuUlhcQ2pzZzJlnjmZtPFLel1ZFpR979cB+AG zeNEo1D8PcQdi9g8TY6ftmJ1N48NYbnxKyqdpe+R1rwu9A8HA5T8wQ4X1byf8bv9uwUE wq+bbpY0QJV1zsKFb+fK0vyoqA44VfABJUdDyToEPVEaeEhn4mU9tA1Ls4JblnFcayl9 TZNdp/W/yproOJDfzXvDDyZOmfd1T7lwMUuKK2gEeCNPSyllqbF+QJifI2yd5guGpJRj /ExQ== X-Gm-Message-State: ALoCoQlMoaii4k6UwFsi1LsA0slNjKgA0AWuHUNxxa5zsAg3wbWwOKl0qQNSixBqUqat3pn77/CA X-Received: by 10.66.142.201 with SMTP id ry9mr14969367pab.14.1397078015959; Wed, 09 Apr 2014 14:13:35 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id dn1sm4541569pbb.54.2014.04.09.14.13.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Apr 2014 14:13:35 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140402212328.7CE8958097@chaos.jnpr.net> Date: Wed, 9 Apr 2014 15:13:32 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 21:13:43 -0000 Hi Simon, Sorry to take a week to reply. Other things came up and it slipped my = mind until now. It sounds like we=92re in agreement on how to proceed, or very nearly = so. I have some more comments below, but I think the next order of business would be patches. = Do you have some ready to roll? If so, then we should iterate on them. If not, I can = find some time over the coming days/weeks. I=92d like to have this done before BSDcan, but = if not we can chat then, assuming you are going. On Apr 2, 2014, at 3:23 PM, Simon J. Gerraty wrote: >>> which are needed during sys.mk can be influenced by user's = -DWITH[OUT_* >>=20 >> OK. A bit of a contrived example, but I suppose if I understood the = meta mo=3D >> de >=20 > Actually not contrived at all - we have that internally. > Ie. we have a local.sys.mk to set all our cool stuff > but we currently just test for .ifdef WITH[OUT]*, hence this thread. Makes sense... >> a bit better I might think differently :) >>=20 >> I=3D92m a bit hesitant, though, to have this affect sys.mk because = that affec=3D >> ts all users >> of make, not just /usr/src. >=20 > That's why I'd put such things in local.sys.mk or some other makefile > that affects /usr/src but isn't install into /usr/share/mk/ That=92s a bit of a departure over how we=92ve done things, but one that = may make a good amount of sense. Where would the src build pick this up from if = it isn=92t installed? >> In some cases, you can declare a winner. In other cases that might be = harder >> to do. With almost all options defaulting to YES, having WITHOUT win = makes >> good sense. When there=3D92s more of a mix, I=3D92m less sure. >=20 > Agreed, that's why I took the easy way out by allowing the winner to = be > configured if the need should arise ;-) >=20 >>> Traditionally done in bsd.obj.mk - but that requires a separate >>> invocation of make. >>=20 >> Right, but can=3D92t that be done automatically w/o that extra = invocation? >=20 > Yes provided you do it early enough (ie during sys.mk) > eg. before you've evaluated things that affect .PATH Ah, I was aware of that restriction. >> Back to the NO_CTF stuff I talked about above: I totally get this. If = you l=3D >=20 >> There=3D92s one place in the tree that wants to turn off CTF. This is = mostly =3D >> fixed by >> just setting MK_CTF=3D3Dno in that makefile after we include = bsd.own.mk. I say >=20 > Wouldn't it be simpler to set MK_CTF=3Dno *before* including = bsd.own.mk ? Well, kinda=85 Then the issue becomes, in what I think you are = proposing, what happens to the meta variables, or MK_foo that sets a lot of MK_bar. Assuming we = move all of them to their own file, we have two sections. One that sets MK_xxx = variables based on WITH/WITHOUT, and a second one that sets them based on other MK_xxx = variables. If I set MK_CTF=3Dno in my makefile, and it caused other MK_ flags to be = set, then I=92d have to include something to take another run at setting those = meta-variables. >> mostly fixed because we wind up with a NULL command where we really = want >> a @: command, though the former I believe is harmless but verbose. = Except >> one could unset WITH_CTF (which doesn=3D92t completely work, it still = shows >> up as defined) and set WITHOUT_CTF before bsd.own.mk and it would = work, >> modulo this bug. >>=20 >> This can really only be fixed by making bsd.own.mk look more like >>=20 >> # section 1 -same >> .include >> # section 3 - same >>=20 >> with bsd.options.mk looking a lot like section 2 of bsd.own.mk. = Also, we=3D >> =3D92d have >=20 > Yes, that's essentially what I was proposing. > By extracting the mechanism to its own file, it can be re-used. Do you have patches? I think I like it... >> This sounds a lot like what you=3D92re trying to describe, though = placement of >> bsd.options.mk may be different than I described. The only reason we >> need it where I suggested it is compatibility with the past. Though = we may >=20 > Yes I assumed it would be included as above - to avoid changing = behavior > unnecessarily. Note: tweaking the semantics and making it re-usable = are > somewhat orthogonal. Yes. I can=92t fix MK_CTF without a fix due to side effects. >> be able to get away with it in sys.mk, I=3D92m hesitant to place it = in there =3D >> because >> that=3D92s global to everything, including ports, etc. Plus, I think = it is to=3D >=20 > Calling it bsd.options.mk is a conflict with ports. > Though including it as "bsd.options.mk" both in bsd.own.mk and in the > relevant ports makefile, should probably mitigate that. I thought ports used a different mechanism and defined special magic so = the src tree mechanism was disabled. >> o early, due to meta MK_ variables, that I talk about below. >=20 >>> .if defined(MK_${var}) >>> .if ${.MAKE.LEVEL} =3D3D=3D3D 0 >>> .error MK_${var} can't be set by a user. >>> .endif >>> =3D >>=20 >>> would seem to negate that. Why can a makefile at level 0 not set = MK_*? >>=20 >> Well, the notion now is that if you want to test MK_* variables, you = need to >> include bsd.own.mk first. The notion I was going for with the above = is that=3D >=20 > Setting MK_* before bsd.own.mk vs after has semantic differences > but that shouldn't preclude doing either. >=20 > Eg. the knob names below describe the semantics >=20 > # these remove choice from user > MK_CANNOT=3D no > MK_MUST=3D yes >=20 > .include >=20 > # these respect user choices > MK_LIKE?=3D yes > MK_DISLIKE?=3D no >=20 >> But there=3D92s a problem even if we take the approach above. Section = 2 in bs=3D >> d.own.mk >> is actually two separate sections. One that sets the MK_* variables = based on >> WITH_ or WITHOUT_ and then a second section that cascades the MK_ = variables >> to other MK_ variables (like MK_CRYPT=3D3D=3D3Dno turning of OpenSSL, = Kerberos,=3D >> etc). If >> you wanted to set one of those variables in your Makefile, you=3D92d = have a c=3D >> hicken >> and egg problem. If you set it before bsd.options.mk, then you=3D92d = get the =3D >> cascade effects >> but hit the warning. If you set it after, you dodge the warning, but = don=3D92=3D >> t get the cascade. >=20 > Per above setting MK_* before including bsd.own.mk is just supressing > user input, probably not something to do a lot, but handy at times - > eg. allows doing away with NO_* >=20 > If that has cascading effects, we assume they are intended. >=20 > Currently, what warning do you hit btw?=20 > I only see .errors if MK_* is pre-defined or WITH[OUT]* both defined. Those errors, although many have been corrected... >> This problem suggests, perhaps, that the test be deleted. >>=20 >>> The outstanding question is dealing with conflict when both WITH_FOO = and >>> WITHOUT_FOO are defined. >>=20 >> True. That=3D92s a tougher problem than you might think on first = blush, as we=3D >> =3D92ve >> been talking about. For now, I=3D92d suggest WITHOUT wins, and we see = how far >=20 > Agreed. I cannot currently think of a case where that wouldn't be > right, but as I mentioned wrt my options.mk it is easy to allow > configurability should the need arise. I=92m agnostic on a mechanism to declare winners. If we can do the rest = easily w/o it I don=92t care. If it is just a couple of lines to add, I won=92t = complain. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 9 21:16:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE5027C5; Wed, 9 Apr 2014 21:16:51 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B169E1172; Wed, 9 Apr 2014 21:16:51 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s39LGosd046560; Wed, 9 Apr 2014 21:16:50 GMT (envelope-from jonathan.robert.anderson@gmail.com) Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Compiler toolchain roadmap From: Jonathan Anderson In-Reply-To: <5345A3A5.5020400@FreeBSD.org> Date: Wed, 9 Apr 2014 22:16:49 +0100 Message-Id: <420BB3D7-7F17-4E98-8442-910704C63312@gmail.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <5345A3A5.5020400@FreeBSD.org> To: Pedro Giffuni X-Mailer: Apple Mail (2.1874) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Jordan Hubbard , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2014 21:16:52 -0000 On 9 Apr 2014, at 20:46, Pedro Giffuni wrote: > FWIW; >=20 > I have been looking at what in the ports tree could make some use of = libdispatch. So far the only candidates seem to be Apache's httpd MPM = and maybe cups and Webkit. If we had libdispatch in the base system, it would be fantastic to = implement a '-j auto' switch for make. As-is, the -j flag is a bit of a = hack: there is no static "correct" number of jobs to run in parallel, = there is only "would I benefit from running some more work alongside the = current mix". Jon -- Jonathan Anderson jonathan@FreeBSD.org http://freebsd.org/~jonathan/= From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 03:46:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D980E46; Thu, 10 Apr 2014 03:46:57 +0000 (UTC) Received: from ch1outboundpool.messaging.microsoft.com (ch1ehsobe001.messaging.microsoft.com [216.32.181.181]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3FF8019A1; Thu, 10 Apr 2014 03:46:56 +0000 (UTC) Received: from mail44-ch1-R.bigfish.com (10.43.68.228) by CH1EHSOBE009.bigfish.com (10.43.70.59) with Microsoft SMTP Server id 14.1.225.22; Thu, 10 Apr 2014 03:46:28 +0000 Received: from mail44-ch1 (localhost [127.0.0.1]) by mail44-ch1-R.bigfish.com (Postfix) with ESMTP id 277661204ED; Thu, 10 Apr 2014 03:46:28 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.11; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF02-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail44-ch1: transitioning domain of juniper.net does not designate 66.129.239.11 as permitted sender) client-ip=66.129.239.11; envelope-from=sjg@juniper.net; helo=P-EMF02-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail44-ch1 (localhost.localdomain [127.0.0.1]) by mail44-ch1 (MessageSwitch) id 139710158669426_24333; Thu, 10 Apr 2014 03:46:26 +0000 (UTC) Received: from CH1EHSMHS039.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.245]) by mail44-ch1.bigfish.com (Postfix) with ESMTP id 0B2561801B4; Thu, 10 Apr 2014 03:46:26 +0000 (UTC) Received: from P-EMF02-SAC.jnpr.net (66.129.239.11) by CH1EHSMHS039.bigfish.com (10.43.69.248) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 10 Apr 2014 03:46:25 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 9 Apr 2014 20:46:46 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s3A3kkV16230; Wed, 9 Apr 2014 20:46:46 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id C058658097; Wed, 9 Apr 2014 20:46:45 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Wed, 09 Apr 2014 15:13:32 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 9 Apr 2014 20:46:45 -0700 Message-ID: <20140410034645.C058658097@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 03:46:57 -0000 >It sounds like we=92re in agreement on how to proceed, or very nearly = >so. I have some more >comments below, but I think the next order of business would be patches. = >Do you have some ready to roll? No, not yet - I figured getting some level of consensus fist was worthwhile. My cunning plan was simply to remove the mechanism from bsd.own.mk, and include the new file at approximately the original location. Thus not necessarily any functional change. The devil's in the details of course. I'm obviously biased, but I think contrib/bmake/mk/options.mk is close to what is desired - just the mechanism with no policy. >If so, then we should iterate on them. If not, I can = >find some time over >the coming days/weeks. I=92d like to have this done before BSDcan, but = >if not we can >chat then, assuming you are going. Sounds good, yes I'll be there. >> That's why I'd put such things in local.sys.mk or some other makefile >> that affects /usr/src but isn't install into /usr/share/mk/ > >That=92s a bit of a departure over how we=92ve done things, but one that = >may make a good amount of sense. I've been distributing my *.mk files for a long time, and I prefer people leave them as is, so I make it easy for them to customize without hacking. Adding .-include "local.*.mk" allows for a lot of customization. When I wrote dirdeps.mk et al it was with the understanding that Juniper would make them publicly available, so used the same model. IIRC there was some discussion at dev summit or BSDCan last year about having src/ look for src.conf, make.conf etc inside the tree rather than /etc. I know in our internal builds we want to ensure that our src/ is self contained. >Where would the src build pick this up from if = >it isn=92t installed? src/share/mk should work fine. >> Wouldn't it be simpler to set MK_CTF=3Dno *before* including = >bsd.own.mk ? > >Well, kinda=85 Then the issue becomes, in what I think you are = >proposing, what happens >to the meta variables, or MK_foo that sets a lot of MK_bar. Assuming we = >move all of >them to their own file, we have two sections. One that sets MK_xxx = >variables based on >WITH/WITHOUT, and a second one that sets them based on other MK_xxx = >variables. >If I set MK_CTF=3Dno in my makefile, and it caused other MK_ flags to be = >set, then I=92d have >to include something to take another run at setting those = >meta-variables. Not necessarily. The stuff in bsd.own.mk like: .if ${MK_TOOLCHAIN} == "no" MK_BINUTILS:= no MK_CLANG:= no MK_GCC:= no MK_GDB:= no .endif doesn't need to change, and for more elaborate stuff, simply being able to include *options.mk more than once may help quite a bit. >> Yes, that's essentially what I was proposing. >> By extracting the mechanism to its own file, it can be re-used. > >Do you have patches? I think I like it... No, but could do so pretty quickly. I need to resync projects/bmake and start writing my material for BSDCan ;-) I could look doing this as part of that. The hardest bit is naming the new makefile ;-) bsd.options.mk would be an obvious choice - though that conflicts with ports usage. >> Calling it bsd.options.mk is a conflict with ports. >> Though including it as "bsd.options.mk" both in bsd.own.mk and in the >> relevant ports makefile, should probably mitigate that. > >I thought ports used a different mechanism and defined special magic so = >the >src tree mechanism was disabled. The ports makefile is much more elaborate but I believe achieves similar results. The name conflict could probably be worked around if bsd.options.mk is considered the best name choice. --sjg From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 15:06:14 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2D0380A for ; Thu, 10 Apr 2014 15:06:14 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 97BA61D04 for ; Thu, 10 Apr 2014 15:06:14 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id A98476398 for ; Thu, 10 Apr 2014 15:06:08 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 6E437C58; Thu, 10 Apr 2014 17:06:09 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: arch@freebsd.org Subject: ar and ranlib -D Date: Thu, 10 Apr 2014 17:06:09 +0200 Message-ID: <86eh15usv2.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:06:14 -0000 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which defaults to -D. This ensures that all timestamps inside static libraries in the base system are hardcoded to 0 (aka the epoch), which is a huge step towards fully reproducible builds. Any objections? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=ar-D.diff Index: share/mk/bsd.lib.mk =================================================================== --- share/mk/bsd.lib.mk (revision 264317) +++ share/mk/bsd.lib.mk (working copy) @@ -172,7 +172,7 @@ .else @${AR} ${ARFLAGS} ${.TARGET} `NM='${NM}' lorder ${OBJS} ${STATICOBJS} | tsort -q` ${ARADD} .endif - ${RANLIB} ${.TARGET} + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} .endif .if !defined(INTERNALLIB) @@ -189,7 +189,7 @@ .else @${AR} ${ARFLAGS} ${.TARGET} `NM='${NM}' lorder ${POBJS} | tsort -q` ${ARADD} .endif - ${RANLIB} ${.TARGET} + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} .endif .if defined(SHLIB_NAME) || \ @@ -246,7 +246,7 @@ @${ECHO} building special pic ${LIB} library @rm -f ${.TARGET} @${AR} ${ARFLAGS} ${.TARGET} ${SOBJS} ${ARADD} - ${RANLIB} ${.TARGET} + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} .endif .if defined(WANT_LINT) && !defined(NO_LINT) && defined(LIB) && !empty(LIB) Index: share/mk/sys.mk =================================================================== --- share/mk/sys.mk (revision 264317) +++ share/mk/sys.mk (working copy) @@ -37,11 +37,12 @@ AR ?= ar .if defined(%POSIX) -ARFLAGS ?= -rv +ARFLAGS ?= -rDv .else -ARFLAGS ?= cru +ARFLAGS ?= crD .endif RANLIB ?= ranlib +RANLIBFLAGS ?= -D AS ?= as AFLAGS ?= --=-=-=-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 15:22:20 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02510DC5 for ; Thu, 10 Apr 2014 15:22:20 +0000 (UTC) Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B84011FF3 for ; Thu, 10 Apr 2014 15:22:19 +0000 (UTC) Received: by mail-qc0-f173.google.com with SMTP id r5so4431527qcx.4 for ; Thu, 10 Apr 2014 08:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=/GGrOE5CDh816DFPbcT2o8hpVaF+P3PCFyHQtW/M9pk=; b=SQve7aRUPiWLGw21B2eo2gcLjdwyp6T/GXb6YWETPcB3Y+Bx3TQ6lCLEMbWyW9t72X Yx09OPdhK8/OF/iIlmBU3Bu4t5F5Z+n5ZAvVX0vp/kt7FstRP+V2wfMcX/xHiB3y/DDi V7QARPFETXHUi1ZQlgPhr9lYjtePTbUpDsOeONVilEWjm8XJbkKCGRTlBgTVGSYeU6L6 rGmuqByCu+sEx/U+mMddmWF+zGZtJOF1N45m6pvt/pDzNc71aWhQBOGchd3nq7TM37N/ FX/x9y30DeLpTTNBpC6rMCXCdDMZ5DKrrTh27zRMzaawkAwNvQjp0ReKQYNKeCNhVUPL H+JA== MIME-Version: 1.0 X-Received: by 10.224.115.3 with SMTP id g3mr11559621qaq.19.1397143338905; Thu, 10 Apr 2014 08:22:18 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Thu, 10 Apr 2014 08:22:18 -0700 (PDT) In-Reply-To: <86eh15usv2.fsf@nine.des.no> References: <86eh15usv2.fsf@nine.des.no> Date: Thu, 10 Apr 2014 11:22:18 -0400 X-Google-Sender-Auth: KnxGxi_D-pfi4SWsHlEi33kxU9o Message-ID: Subject: Re: ar and ranlib -D From: Ed Maste To: =?ISO-8859-1?Q?Dag=2DErling_Sm=F8rgrav?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:22:20 -0000 On 10 April 2014 11:06, Dag-Erling Sm=F8rgrav wrote: > The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which > defaults to -D. This ensures that all timestamps inside static > libraries in the base system are hardcoded to 0 (aka the epoch), which > is a huge step towards fully reproducible builds. Any objections? Looks good to me, I'm not sure why this didn't happen long ago. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 15:46:04 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46711615 for ; Thu, 10 Apr 2014 15:46:04 +0000 (UTC) Received: from mail-pb0-f42.google.com (mail-pb0-f42.google.com [209.85.160.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19DA91225 for ; Thu, 10 Apr 2014 15:46:03 +0000 (UTC) Received: by mail-pb0-f42.google.com with SMTP id rr13so4162685pbb.1 for ; Thu, 10 Apr 2014 08:45:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=X+FtFpL3jYqJcTRY7nURVpJHnVMoJeelwIMF//SVdI0=; b=Ohcj2w4w0r3k3AMCnygPf0pJBksn3XkINz8COnt8jeD9aMehC5nEr1E1hHiCrmQQTb 6S/kgMRMumQ06qH0bcIyibMEF7DRFI4q4EgqUrcjy1Ar6p3BrX/Vk9ThErMhMRQm8Z9Z VNIM400OFeAsniyIoWyXxTcnJfb9ZN1040uRb2ObcK09XYHtqoglsyKyVzg2HbDW0CXt THG0BlZyAqMVHd4Avy94yW7QsOXQsgBD5zNU9WPchXWDu1JQ/VqatFsWhLSeA28KZI79 1/6bkpRVRQtecnhTfoN9AjDQCTufBxBOzfd+hjWqjjB58pFWEZlOyvbC4y02A7r/YXun ZNsg== X-Gm-Message-State: ALoCoQnQndTWlg+eNRa5h4Unc3vIBJ5D0oLrBVFfrYqHqgYxsVwtbzwKlht0/lOc7dqOCQPxi0/k X-Received: by 10.66.192.73 with SMTP id he9mr20626961pac.88.1397144757204; Thu, 10 Apr 2014 08:45:57 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id dn1sm9852600pbb.54.2014.04.10.08.45.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 08:45:56 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Warner Losh In-Reply-To: <86eh15usv2.fsf@nine.des.no> Date: Thu, 10 Apr 2014 09:45:53 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <79CBA7AC-998E-46EE-8F94-F92C7C00FF75@bsdimp.com> References: <86eh15usv2.fsf@nine.des.no> To: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:46:04 -0000 My only concern is with the %POSIX section. That change isn=92t needed = for reproducible builds. Warner On Apr 10, 2014, at 9:06 AM, Dag-Erling Sm=F8rgrav wrote: > The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which > defaults to -D. This ensures that all timestamps inside static > libraries in the base system are hardcoded to 0 (aka the epoch), which > is a huge step towards fully reproducible builds. Any objections? >=20 > DES > --=20 > Dag-Erling Sm=F8rgrav - des@des.no >=20 > Index: share/mk/bsd.lib.mk > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- share/mk/bsd.lib.mk (revision 264317) > +++ share/mk/bsd.lib.mk (working copy) > @@ -172,7 +172,7 @@ > .else > @${AR} ${ARFLAGS} ${.TARGET} `NM=3D'${NM}' lorder ${OBJS} = ${STATICOBJS} | tsort -q` ${ARADD} > .endif > - ${RANLIB} ${.TARGET} > + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} > .endif >=20 > .if !defined(INTERNALLIB) > @@ -189,7 +189,7 @@ > .else > @${AR} ${ARFLAGS} ${.TARGET} `NM=3D'${NM}' lorder ${POBJS} | = tsort -q` ${ARADD} > .endif > - ${RANLIB} ${.TARGET} > + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} > .endif >=20 > .if defined(SHLIB_NAME) || \ > @@ -246,7 +246,7 @@ > @${ECHO} building special pic ${LIB} library > @rm -f ${.TARGET} > @${AR} ${ARFLAGS} ${.TARGET} ${SOBJS} ${ARADD} > - ${RANLIB} ${.TARGET} > + ${RANLIB} ${RANLIBFLAGS} ${.TARGET} > .endif >=20 > .if defined(WANT_LINT) && !defined(NO_LINT) && defined(LIB) && = !empty(LIB) > Index: share/mk/sys.mk > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- share/mk/sys.mk (revision 264317) > +++ share/mk/sys.mk (working copy) > @@ -37,11 +37,12 @@ >=20 > AR ?=3D ar > .if defined(%POSIX) > -ARFLAGS ?=3D -rv > +ARFLAGS ?=3D -rDv > .else > -ARFLAGS ?=3D cru > +ARFLAGS ?=3D crD > .endif > RANLIB ?=3D ranlib > +RANLIBFLAGS ?=3D -D >=20 > AS ?=3D as > AFLAGS ?=3D > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 15:57:46 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4EFD954 for ; Thu, 10 Apr 2014 15:57:46 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 95CBC131B for ; Thu, 10 Apr 2014 15:57:46 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 868E2649E; Thu, 10 Apr 2014 15:57:45 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 4B1F2CA1; Thu, 10 Apr 2014 17:57:46 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Subject: Re: ar and ranlib -D References: <86eh15usv2.fsf@nine.des.no> <79CBA7AC-998E-46EE-8F94-F92C7C00FF75@bsdimp.com> Date: Thu, 10 Apr 2014 17:57:46 +0200 In-Reply-To: <79CBA7AC-998E-46EE-8F94-F92C7C00FF75@bsdimp.com> (Warner Losh's message of "Thu, 10 Apr 2014 09:45:53 -0600") Message-ID: <86a9btuqh1.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 15:57:46 -0000 Warner Losh writes: > My only concern is with the %POSIX section. That change isn=E2=80=99t nee= ded > for reproducible builds. Is it harmful? (what is that, anyway? It's not documented in make(1)) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 16:17:24 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD940F2D for ; Thu, 10 Apr 2014 16:17:24 +0000 (UTC) Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B010B156E for ; Thu, 10 Apr 2014 16:17:24 +0000 (UTC) Received: by mail-pa0-f41.google.com with SMTP id fa1so4197143pad.28 for ; Thu, 10 Apr 2014 09:17:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=lZYMb7QNsisJ0q4iu/j8P4nn1DsWfYsDuFH0OUTyBGQ=; b=A6ek7Jx4PSMNnd1mzXDtzjan0vJBHoZMBI5b7vwjib+MPlZjWCezeQUmaqL7pqBzUH DnUtBxVZ1SpQipHeC0tVHyjczlf5ruZCB/BMC7Cet4m5lI0fwHUdiyEIMoceeXN1vW43 K0RtBaEtjtS2mTfPmn8pM4VjMq5A5wcOfjTsvJeVzfgY9Qq8FqvF+8ifm9DUq1jPiZa+ Fbb49lD0mq7DVucOab5x9sKuVYkbbHDt+7W/6hMdjvXl6r0LYzccUIxKxsLBa5WTeEAX xY95NggGIMnmsnBc1r71m+kj+VSCzjyzHKNmCWybWacK/du1LiA+Gd+0D75Ue/MPtSk0 I5zg== X-Gm-Message-State: ALoCoQnhR0ipRMMY1mFrw596Q+Jgs8ZJyePS1BrT9PbOmC1hd31O7LeIk3prTp6w6xHUA9LKM2Xw X-Received: by 10.66.240.130 with SMTP id wa2mr21016522pac.73.1397146638558; Thu, 10 Apr 2014 09:17:18 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id qc8sm9988097pbc.68.2014.04.10.09.17.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 09:17:17 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Warner Losh In-Reply-To: Date: Thu, 10 Apr 2014 10:17:15 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <925E4F91-1DCD-4002-9E23-5AD8FD582EF8@bsdimp.com> References: <86eh15usv2.fsf@nine.des.no> To: Ed Maste X-Mailer: Apple Mail (2.1874) Cc: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 16:17:24 -0000 On Apr 10, 2014, at 9:22 AM, Ed Maste wrote: > On 10 April 2014 11:06, Dag-Erling Sm=F8rgrav wrote: >> The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS = which >> defaults to -D. This ensures that all timestamps inside static >> libraries in the base system are hardcoded to 0 (aka the epoch), = which >> is a huge step towards fully reproducible builds. Any objections? >=20 > Looks good to me, I'm not sure why this didn't happen long ago. Once upon a time, ranlib didn=92t like this too well and complained that the index was older than the file. Then it was made a special case. = These days (and these days includes time since ~1995 or 2000), people always rebuild the entire .a anyway, so the value of having a timestamp in there is low, at best, so always doing this has become so boring that i=92m surprised this isn=92t the default behavior. Given that we = always rebuild, though, this change is totally safe. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 16:43:22 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90C31916 for ; Thu, 10 Apr 2014 16:43:22 +0000 (UTC) Received: from mail-pd0-f169.google.com (mail-pd0-f169.google.com [209.85.192.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63CD61870 for ; Thu, 10 Apr 2014 16:43:22 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id fp1so4116671pdb.28 for ; Thu, 10 Apr 2014 09:43:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=W6MVnWkC+vfRwwZwHKbTqufLRTMidcG8Vwl7s17fnrQ=; b=k2bevPjktNVd+2tw/67KjBMia6C7z0Kv+6/y77D89scRn1apiRITzt9J1CbQkKrng+ lCaSzUgX71Yw9+bIZEQL8+TbXIaUvCIFmJ5JQsJ9SPvXlWrCCXdkVVxaoiXxlWtMabOw xJqCghyHpd/Z67zdCbifV+7K9quCCkvLZVg2fXCqoWQaH7ie63fNWbXxDrM1IkAKbqsi n8yixGDm5cT1I+1Ba/cwhvy/I2OZ2E3JOOadrK56nRh83B7+/YpJWFh4tdiwOBgHF4FT KCWxu80dnFaR29z/J/rsmeqxS3LBGSTos/mrJltr0vfpHLaaxtu8tqUXlfkyh1sonuT1 rckg== X-Gm-Message-State: ALoCoQn0EqUmXo1C6ufLknWoXmiYXd6Rb7Y8dGUUyq9VpeKgB/ojvuru4x2UBzwvzursIbQiKp8H X-Received: by 10.66.136.71 with SMTP id py7mr21283460pab.2.1397148196537; Thu, 10 Apr 2014 09:43:16 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id wp3sm10109001pbc.67.2014.04.10.09.43.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 09:43:16 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Warner Losh In-Reply-To: <86a9btuqh1.fsf@nine.des.no> Date: Thu, 10 Apr 2014 10:43:13 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <257F4CC6-78AC-4729-B0F7-50FDA296D46D@bsdimp.com> References: <86eh15usv2.fsf@nine.des.no> <79CBA7AC-998E-46EE-8F94-F92C7C00FF75@bsdimp.com> <86a9btuqh1.fsf@nine.des.no> To: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 16:43:22 -0000 On Apr 10, 2014, at 9:57 AM, Dag-Erling Sm=F8rgrav wrote: > Warner Losh writes: >> My only concern is with the %POSIX section. That change isn=92t = needed >> for reproducible builds. >=20 > Is it harmful? >=20 > (what is that, anyway? It's not documented in make(1)) Short answer: It isn=92t defined by POSIX 1003.2, so yes. POSIX mode, which is barely documented in the .POSIX target, causes make(1) to try hard to comply with POSIX requirements. Make, itself, = winds up setting %POSIX to =931003.2=94 and not remaking the Makefiles. The = global sys.mk system responds to this variable by only using commands defined by POSIX 1003.2, so it uses c89, instead of cc. It adds the dash to the = ar command, and a bunch of other silly differences that are none-the-less mandated by POSIX. Since -D isn=92t defined by POSIX ar, your change breaks that and so is harmful... Warner From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 20:44:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D745CF8 for ; Thu, 10 Apr 2014 20:44:43 +0000 (UTC) Received: from mail-pb0-f43.google.com (mail-pb0-f43.google.com [209.85.160.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E473D12B4 for ; Thu, 10 Apr 2014 20:44:42 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id um1so4430954pbc.16 for ; Thu, 10 Apr 2014 13:44:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=tyaIXj8qS/qU9qBjMcjhzgUJ00J4qVvyMGIRxQu5v0s=; b=jdUhdkEb2UqDCEpvRPMaRQgUl8OmQVnnNL5AIOh+5oThWLUp3bTIgLE7AyIZ7ZBvkb Ori/sEsrqPYHRtphdZ9i5/t0Ynn5sj5LX9MceImxLGZcN7A5RB2nSGXM+/0LStrnEI/8 Ve0H/+S7RGPax+zbtigZ3m/If7FQm8x3G8RovWBj+2JX7htFyJZLiUCyByV2FISOP2xt VB8dGLxFHBqTVnxjmLt0Gutl3qI8mI/In+nBUlUacIhoWUMuhw5OhkTcMomnJLMHqNBV 3dyUDCYZPD8Hfg9hgmwSIo24LYTIB5KiAA3spX+tJbrIvsorYgPUvQ11MThY+iU+8BOK J9Ug== X-Gm-Message-State: ALoCoQmVO7fDRQNWeGYYWoZUyH8ZREF8tsFJute4IwGxAPtapaUM2cq3igHbkpcYKifaJB4Z1s1m X-Received: by 10.66.221.99 with SMTP id qd3mr22499987pac.46.1397162240844; Thu, 10 Apr 2014 13:37:20 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id v1sm11059894pbl.1.2014.04.10.13.37.19 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 13:37:20 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Warner Losh In-Reply-To: <201404091145.58792.jhb@freebsd.org> Date: Thu, 10 Apr 2014 14:37:17 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> References: <201404091145.58792.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 20:44:43 -0000 OK. Here=92s the summary of the thread: (1) gdb in tree is ancient (2) kgdb is quite useful, and only in tree (3) ports gdb rocks, but=85 (4) ports gdb exists only for a few architectures (5) Fixing ptrace will allow us to use a more-stock gdb Action items: (1) Create a wiki page with timeline to deactivation and removal. (2) Create milestones along the path for (a) kgdb + devel/gdb* (b) architectural coverage (c) ptrace fixes (3) profit. I=92ve done these steps and documented them at = https://wiki.freebsd.org/GdbRetirement to allow work to progress (or = not) without repeating this discussion. Thanks for everybody=92s = feedback. Feel free to comment on the wiki page or edit it for missing = items (or testing you=92ve done). At this point, I=92m withdrawing the gdb disabled by default patches. Warner On Apr 9, 2014, at 9:45 AM, John Baldwin wrote: > On Tuesday, April 08, 2014 4:34:35 pm Warner Losh wrote: >> Greetings, >>=20 >> The gdb in the tree seems to be of very limited usefulness these = days. It >> doesn=92t seem to work on clang-enabled architectures w/o building = -gdwarf-2, >> it doesn=92t seem to work with threaded applications, and on some >> architectures it doesn=92t seem to work at all (mips comes to mind, = but it may >> have been the two binaries I tried). >>=20 >> It seems like we=92d be doing our users a favor by applying: >>=20 >> diff -r 8bfca9de870e share/mk/bsd.own.mk >> --- a/share/mk/bsd.own.mk >> +++ b/share/mk/bsd.own.mk >> @@ -266,7 +266,6 @@ WITH_HESIOD=3D >> FREEBSD_UPDATE \ >> GAMES \ >> GCOV \ >> - GDB \ >> GNU \ >> GNU_GREP_COMPAT \ >> GPIB \ >> @@ -355,6 +354,7 @@ WITH_HESIOD=3D >> CLANG_EXTRAS \ >> CTF \ >> DEBUG_FILES \ >> + GDB \ >> HESIOD \ >> INSTALL_AS_USER \ >> LLDB \ >>=20 >> to the tree, which will turn gdb off by default. It may make more = sense to >> just remove it entirely, but I=92m not sure I want to go there just = yet in >> case there are things that I=92m missing. I believe that the port = will be >> adequate for all architectures we support, but haven=92t tested this = directly >> yet. I do know that on amd64, the port just worked, where the in-tree = gdb >> was an epic fail. >=20 > kgdb is a must. I think it would be less work to forward port kgdb = support > into gdb7 from ports than to keep our ancient gdb alive. Some things = I can > think of for gdb7: >=20 > 1) The threads patch could be greatly simplified if we fixed the = ptrace > backend to properly handle inferiors with tids. This would remove = a > lot of the threads patch where the thread inferior tries to invoke > ptrace directly and convert registers, etc. The way it does this = now > is a total hack and requires much larger patches. This would also > make it a lot easier to get thread debugging working on more > architectures as the thread-db bits would become mostly MI (if not > entirely) >=20 > 2) Porting the kgdb frontend to work with gdb7. It would be nicer to > have a more modern base for kgdb and the ability to use python > scripting with kgdb, custom printers for in-kernel structures, etc. >=20 > I think if we have a useful devel/kgdb that builds against devel/gdb = we > can probably think about retiring gdb, but it's premature = right > now. >=20 > --=20 > John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 21:03:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4AD84D6 for ; Thu, 10 Apr 2014 21:03:03 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8D1C814A2 for ; Thu, 10 Apr 2014 21:03:03 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 42BEAB945; Thu, 10 Apr 2014 17:03:02 -0400 (EDT) From: John Baldwin To: Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... Date: Thu, 10 Apr 2014 17:02:52 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> In-Reply-To: <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201404101702.52622.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Apr 2014 17:03:02 -0400 (EDT) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 21:03:03 -0000 On Thursday, April 10, 2014 4:37:17 pm Warner Losh wrote: > OK. Here=92s the summary of the thread: >=20 > (1) gdb in tree is ancient > (2) kgdb is quite useful, and only in tree > (3) ports gdb rocks, but=85 > (4) ports gdb exists only for a few architectures > (5) Fixing ptrace will allow us to use a more-stock gdb >=20 > Action items: >=20 > (1) Create a wiki page with timeline to deactivation and removal. > (2) Create milestones along the path for > (a) kgdb + devel/gdb* > (b) architectural coverage > (c) ptrace fixes I would actually invert these. I think (c) is the simplest to do (in regards to the thread changes I mentioned) and I think it makes (b) a lot easier to do. > (3) profit. Otherwise, sounds good to me. > I=92ve done these steps and documented them at https://wiki.freebsd.org/G= dbRetirement to allow work to progress (or not) without repeating this disc= ussion. Thanks=20 for everybody=92s feedback. Feel free to comment on the wiki page or edit i= t for missing items (or testing you=92ve done). >=20 > At this point, I=92m withdrawing the gdb disabled by default patches. So one thing we kicked around on IRC is that I think it would be nice to have some sort of place to collaborate on maintaining useful GPLv3 toolchain bits. I don't think they belong in the main tree. However, it might be nice to someday have another SVN repo that can be overlaid into an existing src checkout (maybe using SVN external references?) to allow GPLv3 gdb, etc. to be built as part of a world build. I don't know that we need an SVN repo on a FreeBSD.org machine right now, but something would be nice. =2D-=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 21:40:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5FFCFDB; Thu, 10 Apr 2014 21:40:58 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5621A1804; Thu, 10 Apr 2014 21:40:58 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id m5so2469838qaj.6 for ; Thu, 10 Apr 2014 14:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=LrUI/AOYhUNj0tYWbBeIVysYVKho+dtCpfgqsqxoxCc=; b=qWZah2DDIK2WjFwrshEfVAI5Pi6D8OJN03vsxcsCbOhMn6rlEBoPnEVQ2ssQh7V6bx DW+4U7LQVEvegQdbGLvI+oUmhuUL+LjHhY/Kp2lbFNTaoKvrYPm/wfQ6FKuQNNzLGfjp fASpQJ/WnjiDpkRJ5nR2OWcVKG5Oetnmgly6UekXep6hWJI53YTFZLDgsuOdLL30yY6f O3HUDwqW1KuR2xKoZmHzK9I5YF6TvRTBm969gV85NhpFUGKERNK9ovoSged+bLNsozxp GimRmIyaZ98lLAkaYFGceBILG+F06G6LbFR6V7w+1IbW4Ki4HhZubnHt9FJBvbpV8iOH bhFw== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr24105596qaf.26.1397166057374; Thu, 10 Apr 2014 14:40:57 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 14:40:57 -0700 (PDT) In-Reply-To: <201404101702.52622.jhb@freebsd.org> References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> Date: Thu, 10 Apr 2014 14:40:57 -0700 X-Google-Sender-Auth: I8YZTGXCB7RlkTk3pZih1oDNJR4 Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 21:40:58 -0000 [snip] My (-1) action item: * make llvm in -HEAD generate dwarf-2 code by default so the base system gdb can be again used against binaries that have debug symbols, at least until a viable replacement is ready. -a From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 21:53:53 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2A49655 for ; Thu, 10 Apr 2014 21:53:52 +0000 (UTC) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95BF21901 for ; Thu, 10 Apr 2014 21:53:52 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id lx4so4594315iec.38 for ; Thu, 10 Apr 2014 14:53:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=ALr1nfxHFuW/YtcxFQ5q2kiI33UC9hi1Pb5bLJTgeuU=; b=m93Rnxj66zfM93mlp6EM/vx6j/OI16qOKaNTdEGYJng5ilRpezLXptwpKCCtljucas N1w7Yg8GOFmVm7wAqHgiGNq7n4lMQOR42YWTxjycJO2TPIftsP2SZygAkTHMZFtk4R/5 rzF3bPD/LWAVJcDcWJam2TQTyQ38TFVRYP8cLC/5g7rqA7I0E/3fVeyB9tVPvibglqNP Xiej2u8i/CxN5gMAbcfJCaEAdNd6wDTcSnj8Zsv4PuYXU2HZlQzdVb03aIuOTTxMk5MV UgtwIo9ouLOkSrAME1uD4N4wtKqqjdq+EtfCrlOSqIO/zYoO+BAFp75Ad55jwBDl3gba 6U0g== X-Gm-Message-State: ALoCoQld8eRdSxShrbWr5jzeUtW9RSyMXNtpbHrZ90ga+V/l2DiIrdT8RhhLLxNsjqfHIb5dq3RC X-Received: by 10.42.199.144 with SMTP id es16mr3247757icb.87.1397166826564; Thu, 10 Apr 2014 14:53:46 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id s9sm513074igw.16.2014.04.10.14.53.45 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 14:53:45 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Warner Losh In-Reply-To: <534710F4.1040908@cox.net> Date: Thu, 10 Apr 2014 15:53:44 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <5C84C980-BBD6-48F0-914A-220B5DB77A5A@bsdimp.com> References: <86eh15usv2.fsf@nine.des.no> <534710F4.1040908@cox.net> To: johnandsara2@cox.net X-Mailer: Apple Mail (2.1874) Cc: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= , Ed Maste , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 21:53:53 -0000 On Apr 10, 2014, at 3:45 PM, John D. Hendrickson and Sara Darnell = wrote: > Warner Losh wrote: >> On Apr 10, 2014, at 9:22 AM, Ed Maste wrote: >>> On 10 April 2014 11:06, Dag-Erling Sm=F8rgrav wrote: >>>> The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS = which >>>> defaults to -D. This ensures that all timestamps inside static >>>> libraries in the base system are hardcoded to 0 (aka the epoch), = which >>>> is a huge step towards fully reproducible builds. Any objections? >>> Looks good to me, I'm not sure why this didn't happen long ago. >> Once upon a time, ranlib didn=92t like this too well and complained = that >> the index was older than the file. Then it was made a special case. = These >> days (and these days includes time since ~1995 or 2000), people >> always rebuild the entire .a anyway, so the value of having a = timestamp >> in there is low, at best, so always doing this has become so boring >> that i=92m surprised this isn=92t the default behavior. Given that we = always >> rebuild, though, this change is totally safe. >> Warner >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" >=20 > i can't confirm a ar needs inner timestamps as an interface dependancy = issue or that software depends on this format >=20 > however it does make sense IF AND ONLY IF the timestamp are used for = compilation order dependancy, and compilation is about mapping holes by = dependancy, it's ok for releasetime AFTER compiling (NOT during = compiling) >=20 > but you all aren't saying you've done any research >=20 > but i don't know if any kernels use it with lib version as a link to = and extended interface or as a flag. i don't know if any existing = software crashes if the value is 0. >=20 > does install(1) strip this info at the final stage ? never mind >=20 > no. these days people always build such and such a way. bull. >=20 > ar is used by assemblers not just gcc(1) >=20 > some people insert and remove asm libraries by hand. all what the job = requires to get done. >=20 > any argument "these days we will only allow". no. that's messing up = what had already worked right ? >=20 > ok well good luck In the tree, we do none of these things, so the change is safe. The = kernel doesn=92t care, the user land .a files don=92t care since we = always rebuild them and never insert. Outside the tree, people use bsd.lib.mk, which does things the way I say = so the change is safe. Outside of that, these changes make no difference, especially for the = use cases you describe, so that is safe. What am I missing? Warner= From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 21:56:50 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBD0F7F7; Thu, 10 Apr 2014 21:56:50 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BC92C191B; Thu, 10 Apr 2014 21:56:50 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WYMxl-000DUG-2m; Thu, 10 Apr 2014 21:56:49 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s3ALulFK095518; Thu, 10 Apr 2014 15:56:47 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18pF1/F6F57E2R3+TGyTy8s Subject: Re: Time for turning off gdb by default? Or worse... From: Ian Lepore To: Adrian Chadd In-Reply-To: References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> Content-Type: text/plain; charset="us-ascii" Date: Thu, 10 Apr 2014 15:56:46 -0600 Message-ID: <1397167006.1124.66.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 21:56:51 -0000 On Thu, 2014-04-10 at 14:40 -0700, Adrian Chadd wrote: > [snip] > > My (-1) action item: > > * make llvm in -HEAD generate dwarf-2 code by default so the base > system gdb can be again used against binaries that have debug symbols, > at least until a viable replacement is ready. > +1 -- Ian From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 22:33:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90E665CB; Thu, 10 Apr 2014 22:33:39 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B58F1C7C; Thu, 10 Apr 2014 22:33:39 +0000 (UTC) Received: from coleburn.home.andric.com (coleburn.home.andric.com [192.168.0.15]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 08CA45C46; Fri, 11 Apr 2014 00:33:25 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_3111FAE6-3C64-41D7-B2A3-1A8A96B86F59"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Dimitry Andric In-Reply-To: Date: Fri, 11 Apr 2014 00:33:21 +0200 Message-Id: <59CD3E44-42EE-435A-8953-AA548EA04FE3@FreeBSD.org> References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:33:39 -0000 --Apple-Mail=_3111FAE6-3C64-41D7-B2A3-1A8A96B86F59 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 10 Apr 2014, at 23:40, Adrian Chadd wrote: > [snip] > > My (-1) action item: > > * make llvm in -HEAD generate dwarf-2 code by default so the base > system gdb can be again used against binaries that have debug symbols, > at least until a viable replacement is ready. Just do: pkg install gdb -Dimitry --Apple-Mail=_3111FAE6-3C64-41D7-B2A3-1A8A96B86F59 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlNHHDQACgkQsF6jCi4glqMmgACfTpgr8QxgGwgfPzQC90thDObk AW4AnAmUm2sXibNIHaiGSZe9Xp5cHBqN =tb4R -----END PGP SIGNATURE----- --Apple-Mail=_3111FAE6-3C64-41D7-B2A3-1A8A96B86F59-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 22:42:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7EC31876; Thu, 10 Apr 2014 22:42:23 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14E401D1E; Thu, 10 Apr 2014 22:42:23 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id cm18so3627371qab.4 for ; Thu, 10 Apr 2014 15:42:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=6k4puJcMRi4wzg7JijLgaQEbZQmdJ55ZnvuZYYHwJDo=; b=NQTg4lzWlaQq9DVR7gkIJc/q3lPCT2g9QAjkqxcpsKfDpcVkEVhUN1vgbyBhF1xdKF jcXyWULlCM+iMwxzkRxigGlo63vR40azZBFW+tuZG14g/uxZwz9GGbcPZAs8AGamgbld jEWL/dcQSF9dXCijDblPxDLz6x4OwDWVqZMNAQ8IiADsPmwc/17mYEzAycoFWQKBLU8B sr5y1bXlT0vNSmxkweIjWbKmCHcoFUWbEyGOioHnWpYgaUmVwlaWefSG6LNNzZctuWx7 lafiXIjDC0pvMM0j1Qt2GX89/QTvIWE0QXapfghso1dkvYHsF4DBGvrfgVdhE5WuX1CK E/Uw== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr24403283qaf.26.1397169742264; Thu, 10 Apr 2014 15:42:22 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 15:42:22 -0700 (PDT) In-Reply-To: <59CD3E44-42EE-435A-8953-AA548EA04FE3@FreeBSD.org> References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> <59CD3E44-42EE-435A-8953-AA548EA04FE3@FreeBSD.org> Date: Thu, 10 Apr 2014 15:42:22 -0700 X-Google-Sender-Auth: szjPYx27PzLDx4Zn_M9jnPbnMnw Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: Dimitry Andric Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:42:23 -0000 On 10 April 2014 15:33, Dimitry Andric wrote: > On 10 Apr 2014, at 23:40, Adrian Chadd wrote: >> [snip] >> >> My (-1) action item: >> >> * make llvm in -HEAD generate dwarf-2 code by default so the base >> system gdb can be again used against binaries that have debug symbols, >> at least until a viable replacement is ready. > > Just do: > > pkg install gdb I think you're missing the whole point. Also, that doesn't work on MIPS. Or ARM. Or PPC. Or IA-64. -a (All the world isn't amd64.) From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 22:52:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78A63AF7 for ; Thu, 10 Apr 2014 22:52:09 +0000 (UTC) Received: from mail-ie0-f179.google.com (mail-ie0-f179.google.com [209.85.223.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E47B1E03 for ; Thu, 10 Apr 2014 22:52:09 +0000 (UTC) Received: by mail-ie0-f179.google.com with SMTP id lx4so4648374iec.38 for ; Thu, 10 Apr 2014 15:52:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=xP47P4k4sSG987fAWosgiBSc6HIxxz5widEFUrwKlS0=; b=QWvI3SxRn6QZtpmQHQci7unTing6RTyjxpYlD5kp7T8TfdcMUc/4VWOTsNOXXycvfl Gs9bp0KT6bsh6aT/3osk7WtXV0XkBkQYZn6TV1vG0fPjxtB1bQ/zLrO2uZIO5wXu7dUt IEHT5mBhSb+kBAdTdftEmNkg7PMEFheT+1/dcjSxGtEzNrpH1ZFzAJNP4z0rHDaGTEA3 YekX9xJRZbvxMJ6+Y+yeg3QwW+qjZbMZSM1+X6d1ACPIe63zTiCnUIrpyOe6DdBFwkD5 +11gPZq67fbuuuwr9QT0WcUzAeaJ3BNsDZt6l3XEEgv4zN76KlIXvsl224hDpNaPwiLU aN0g== X-Gm-Message-State: ALoCoQlI22HB5CBE8A6v5y7NqQ8VuWNAju+TYI7ntLJ9T+obVSzBg4nvJJRFvoTn6b29O95sP2+X X-Received: by 10.42.99.8 with SMTP id u8mr3723400icn.31.1397169953563; Thu, 10 Apr 2014 15:45:53 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id vc5sm870323igb.3.2014.04.10.15.45.52 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 15:45:52 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Warner Losh In-Reply-To: Date: Thu, 10 Apr 2014 16:45:51 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> <59CD3E44-42EE-435A-8953-AA548EA04FE3@FreeBSD.org> To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: Dimitry Andric , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 22:52:09 -0000 On Apr 10, 2014, at 4:42 PM, Adrian Chadd wrote: > On 10 April 2014 15:33, Dimitry Andric wrote: >> On 10 Apr 2014, at 23:40, Adrian Chadd wrote: >>> [snip] >>>=20 >>> My (-1) action item: >>>=20 >>> * make llvm in -HEAD generate dwarf-2 code by default so the base >>> system gdb can be again used against binaries that have debug = symbols, >>> at least until a viable replacement is ready. >>=20 >> Just do: >>=20 >> pkg install gdb >=20 > I think you're missing the whole point. >=20 > Also, that doesn't work on MIPS. Or ARM. Or PPC. Or IA-64. >=20 If it did, we could just turn off gdb and life would be good=85 Please see https://wiki.freebsd.org/GdbRetirement and edit it based on = testing that you can do. That wiki page is there to drive this issue to = Dimitry=92s answer in the fullness of time. If we had kgdb support in ports and arm worked, we might be able to get = away with it, but we=92re not even close on either of those at the = moment... Warner P.S. Minor quibble: it does work on powerpc, but not sparc64. Doesn=92t = change your point though.= From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 23:16:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6A4B1BE; Thu, 10 Apr 2014 23:16:55 +0000 (UTC) Received: from mail-qa0-x233.google.com (mail-qa0-x233.google.com [IPv6:2607:f8b0:400d:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54CC5113F; Thu, 10 Apr 2014 23:16:55 +0000 (UTC) Received: by mail-qa0-f51.google.com with SMTP id j7so4664708qaq.10 for ; Thu, 10 Apr 2014 16:16:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=3Z+tTHxWf+REskjofdey8hXTyO6CeqPDOvR4mYAX01o=; b=o8vd3fCXWwMlwJ0vubakH+uyoVoxx3K3zdPsePYjXDBVeZUGc64GNftxBx7+9zOpMn Izv83FwHqlrXabGnRrx833GfFJRb9KhZtrzaPRPsYBd1QH3SqSeh42tHF6TUmdMJrEfL DD/sbQQCssRPbZY+4maTGWCAp3ox+jdpRr+BvVrJON+oLQ5k7GkPtoIFFxDRfYj2Ngcx OktpocbGimtoqj4V3xpe+OUwvRQU7mZoxIGIx+ddWf6dcFDrFMN3EbybyVK0F+oqamoC 9bNJmm5suo9cfBWJCwx0777FR29azyYKzAbfeBOAy/LbeI14wM9uqa5eN5bQLvWxvXNs 0/vQ== MIME-Version: 1.0 X-Received: by 10.224.14.14 with SMTP id e14mr14681905qaa.80.1397171814465; Thu, 10 Apr 2014 16:16:54 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Thu, 10 Apr 2014 16:16:54 -0700 (PDT) In-Reply-To: <1396995427.81853.449.camel@revolution.hippie.lan> References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> <1396995427.81853.449.camel@revolution.hippie.lan> Date: Thu, 10 Apr 2014 19:16:54 -0400 X-Google-Sender-Auth: cz5oEq3lk-04i9njL3UNO4_SQVc Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Ed Maste To: Ian Lepore Content-Type: text/plain; charset=ISO-8859-1 Cc: Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 23:16:55 -0000 On 8 April 2014 18:17, Ian Lepore wrote: > > Only when building the kernel. For userland we've got nothing. gdb > aside, even addr2line doesn't work on userland binaries anymore. It > used to be hard to do debugging for arm. Now it's impossible. The elftoolchain-based binutils replacements (nm, addr2line, etc.) work well, although there are a few remaining features that need to be implemented in some of them. They also inherently support (at least some) cross-arch use cases. I'm hopeful that they'll see these features added, and be imported, before too long. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 23:21:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1832A271; Thu, 10 Apr 2014 23:21:24 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A92AD11B9; Thu, 10 Apr 2014 23:21:23 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id w7so5098623qcr.8 for ; Thu, 10 Apr 2014 16:21:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ohlTfnvaL44EmmjhjAnZKYcKof9niXKSatK8D4NYEiM=; b=thg0KgicBfLlOZ4tGfBEdnmIEsq0w4HeiNK2AnugIC5aLgfqaLzaSkTJBN+ICoGYjK VB356O5bRAgsc6vDhRcOTbpP7rHBjKy/3cZ8Ij31D/P4EdDNno6GE+263BkAJrLwhCda 65h80MCub4mMr1FCjGHq4+naAQblRgjCoRi/WDmOzdjXl/4kRQbFgs6WVMyBHBK/jZTE xRrXaceBASlHiRIGmvCaMyS+Y/C7MTGOmQhZpL+q8x6gjXznE3el8sGLSOHwcPFHtF4B WjIDIZlv0mVXpZZOAxZf/GWgAQFGh6KxmwbTSprrVONhj85PihTuA9fsePQV49IsyeHq fLVg== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr24563478qaf.26.1397172082830; Thu, 10 Apr 2014 16:21:22 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 16:21:22 -0700 (PDT) In-Reply-To: References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> <1396995427.81853.449.camel@revolution.hippie.lan> Date: Thu, 10 Apr 2014 16:21:22 -0700 X-Google-Sender-Auth: FLwCnJTqXwOm1-narAK4-iXZcEc Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: Ed Maste Content-Type: text/plain; charset=ISO-8859-1 Cc: Steve Kargl , Ian Lepore , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 23:21:24 -0000 On 10 April 2014 16:16, Ed Maste wrote: > On 8 April 2014 18:17, Ian Lepore wrote: >> >> Only when building the kernel. For userland we've got nothing. gdb >> aside, even addr2line doesn't work on userland binaries anymore. It >> used to be hard to do debugging for arm. Now it's impossible. > > The elftoolchain-based binutils replacements (nm, addr2line, etc.) > work well, although there are a few remaining features that need to be > implemented in some of them. They also inherently support (at least > some) cross-arch use cases. I'm hopeful that they'll see these > features added, and be imported, before too long. Right, so can we flip back to dwarf-2 and make that stuff continue to somewhat-work until the replacements are ready? :) -a From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 23:21:42 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16DAA314 for ; Thu, 10 Apr 2014 23:21:42 +0000 (UTC) Received: from eastrmfepo103.cox.net (eastrmfepo103.cox.net [68.230.241.215]) by mx1.freebsd.org (Postfix) with ESMTP id B2D0B11BE for ; Thu, 10 Apr 2014 23:21:41 +0000 (UTC) Received: from eastrmimpo210 ([68.230.241.225]) by eastrmfepo101.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140410214437.SYXL16123.eastrmfepo101.cox.net@eastrmimpo210> for ; Thu, 10 Apr 2014 17:44:37 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo210 with cox id oMkc1n00E41obj401MkcTS; Thu, 10 Apr 2014 17:44:36 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020203.534710C5.0004,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=aZC/a2Ut c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=N659UExz7-8A:10 a=kviXuzpPAAAA:8 a=6I5d2MoRAAAA:8 a=CrjhSTmFFmCxGrAho0YA:9 a=pILNOxqGKmIA:10 a=SV7veod9ZcQA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <534710F4.1040908@cox.net> Date: Thu, 10 Apr 2014 17:45:24 -0400 From: "John D. Hendrickson and Sara Darnell" User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: Warner Losh Subject: Re: ar and ranlib -D References: <86eh15usv2.fsf@nine.des.no> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: =?windows-1252?Q?Dag-Erling_Sm=F8rgr?= =?windows-1252?Q?av?= , Ed Maste , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: johnandsara2@cox.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 23:21:42 -0000 Warner Losh wrote: > On Apr 10, 2014, at 9:22 AM, Ed Maste wrote: > >> On 10 April 2014 11:06, Dag-Erling Smørgrav wrote: >>> The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which >>> defaults to -D. This ensures that all timestamps inside static >>> libraries in the base system are hardcoded to 0 (aka the epoch), which >>> is a huge step towards fully reproducible builds. Any objections? >> Looks good to me, I'm not sure why this didn't happen long ago. > > Once upon a time, ranlib didn’t like this too well and complained that > the index was older than the file. Then it was made a special case. These > days (and these days includes time since ~1995 or 2000), people > always rebuild the entire .a anyway, so the value of having a timestamp > in there is low, at best, so always doing this has become so boring > that i’m surprised this isn’t the default behavior. Given that we always > rebuild, though, this change is totally safe. > > Warner > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > i can't confirm a ar needs inner timestamps as an interface dependancy issue or that software depends on this format however it does make sense IF AND ONLY IF the timestamp are used for compilation order dependancy, and compilation is about mapping holes by dependancy, it's ok for releasetime AFTER compiling (NOT during compiling) but you all aren't saying you've done any research but i don't know if any kernels use it with lib version as a link to and extended interface or as a flag. i don't know if any existing software crashes if the value is 0. does install(1) strip this info at the final stage ? never mind no. these days people always build such and such a way. bull. ar is used by assemblers not just gcc(1) some people insert and remove asm libraries by hand. all what the job requires to get done. any argument "these days we will only allow". no. that's messing up what had already worked right ? ok well good luck From owner-freebsd-arch@FreeBSD.ORG Thu Apr 10 23:34:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F021F58A for ; Thu, 10 Apr 2014 23:34:12 +0000 (UTC) Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC9F012A7 for ; Thu, 10 Apr 2014 23:34:12 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id y10so4470631pdj.41 for ; Thu, 10 Apr 2014 16:34:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=qjcnLlvfWGNngxzTIzGpFyjABpP9h5i6jsMvKlkQPxA=; b=faaA71iXlc4MovqVmUOiD1glLU/tNek4hQ9RR5/4wlbj/1DrZDiIXW1YpKynAoQAQh V/zS2z0TYPc/hfUchHqx8599uCYT8YmW6OSdnitW0iDqcvwfVnUpW3qEHY0FSGbjRMzg do7YqsbOJtJOfv1qYLUpMiaydtH8OZpfX8wtdddk0k1GzMT7kl3/4UXW6t66rMUQrLI3 b+Xk/aFBuduRKXe7D+KqBZW1o9XMy+2JZ2aj849AgSfa7Svgr2kmDNnEJ6cjuhOrxi97 5Nj4ooRKsTFHPpRzIrycKU2ZkJEnmA26zJitLFSY1NJPfnL8x9uI4eorCRa7qiYOuwsx EOlA== X-Gm-Message-State: ALoCoQk4wWp90OHWLRmhy/N9hExC39BVIQBbhRDOr3FAsBfHXJf6z3ljiUZr3R6jLhcNmh3IHOsr X-Received: by 10.68.90.132 with SMTP id bw4mr22865215pbb.136.1397172846629; Thu, 10 Apr 2014 16:34:06 -0700 (PDT) Received: from [10.64.24.116] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id zv3sm26809419pab.20.2014.04.10.16.34.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Apr 2014 16:34:06 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Time for turning off gdb by default? Or worse... From: Warner Losh In-Reply-To: Date: Thu, 10 Apr 2014 17:34:03 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <03C737FA-45A4-4BCE-9F7C-B2B0889C71AC@bsdimp.com> References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> <1396995427.81853.449.camel@revolution.hippie.lan> To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: Ed Maste , Ian Lepore , Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Apr 2014 23:34:13 -0000 On Apr 10, 2014, at 5:21 PM, Adrian Chadd wrote: > On 10 April 2014 16:16, Ed Maste wrote: >> On 8 April 2014 18:17, Ian Lepore wrote: >>>=20 >>> Only when building the kernel. For userland we've got nothing. gdb >>> aside, even addr2line doesn't work on userland binaries anymore. It >>> used to be hard to do debugging for arm. Now it's impossible. >>=20 >> The elftoolchain-based binutils replacements (nm, addr2line, etc.) >> work well, although there are a few remaining features that need to = be >> implemented in some of them. They also inherently support (at least >> some) cross-arch use cases. I'm hopeful that they'll see these >> features added, and be imported, before too long. >=20 > Right, so can we flip back to dwarf-2 and make that stuff continue to > somewhat-work until the replacements are ready? :) You are asking the wrong question: would anybody hurt me if I just did = this? :) Warner From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 01:39:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 879253ED; Fri, 11 Apr 2014 01:39:04 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22DA11DBA; Fri, 11 Apr 2014 01:39:04 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id j15so4698378qaq.16 for ; Thu, 10 Apr 2014 18:39:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=C3GAlXJwY+3xC1XYfHI2TdVfQ8zlmJoTaOdnkHRdh2k=; b=YtyfJJOdE5gYmM6mzn8B2WAn0FCRf2JBM50i5VNSf8R+7KdwPbK7Z9zEShh1piCKDR cErmxvxdCvADhZcPTIQR4whfn9zbG9qq/Cs0XZcM8tLZtH15g0JH1fo95tOeU7LIOnX1 /bTpDAgMlQOPktCcRDTtVKPGZv+2luVoXliursb2N6xA3N5DX3S1ZIzL+d0ITp7d9Esi w0Ln7I6VspevY9ekWBkKXeI2uBoPC1u/6aMa9dRE5k05mghv9RmDkP2ilJqaQAc+/+sK PZ85mWQkC2x9JnryYvLiXZ0p3D5UEwFD0MNsIlIPB+6/eTCCDYNO7e/i7mZwLXI+Whwk KsRA== MIME-Version: 1.0 X-Received: by 10.224.166.210 with SMTP id n18mr25275404qay.6.1397180343158; Thu, 10 Apr 2014 18:39:03 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.50.206 with HTTP; Thu, 10 Apr 2014 18:39:03 -0700 (PDT) In-Reply-To: <03C737FA-45A4-4BCE-9F7C-B2B0889C71AC@bsdimp.com> References: <20140408212435.GA75404@troutmask.apl.washington.edu> <57ECB078-3D7A-4BE8-AA29-1ED7BB347DBD@bsdimp.com> <1396995427.81853.449.camel@revolution.hippie.lan> <03C737FA-45A4-4BCE-9F7C-B2B0889C71AC@bsdimp.com> Date: Thu, 10 Apr 2014 18:39:03 -0700 X-Google-Sender-Auth: LmDBq1iLNyftd_WV7UkJsGdiAEY Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Adrian Chadd To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Cc: Ed Maste , Ian Lepore , Steve Kargl , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 01:39:04 -0000 On 10 April 2014 16:34, Warner Losh wrote: >> Right, so can we flip back to dwarf-2 and make that stuff continue to >> somewhat-work until the replacements are ready? :) > > You are asking the wrong question: would anybody hurt me if I just did this? :) I'd guard you. :) -a From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 07:51:08 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1A5A1FE for ; Fri, 11 Apr 2014 07:51:08 +0000 (UTC) Received: from csmtp5.one.com (csmtp5.one.com [195.47.247.105]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84AD713B2 for ; Fri, 11 Apr 2014 07:51:08 +0000 (UTC) Received: from bigmac.router9fbd7c.com (unknown [176.222.238.90]) by csmtp5.one.com (Postfix) with ESMTPA id 2CBA4400000B5; Fri, 11 Apr 2014 07:51:00 +0000 (UTC) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Erik Cederstrand In-Reply-To: <86eh15usv2.fsf@nine.des.no> Date: Fri, 11 Apr 2014 09:51:00 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <8F5B6CDC-3680-4F3A-887B-B01F0E9C3E5D@cederstrand.dk> References: <86eh15usv2.fsf@nine.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 07:51:08 -0000 Den 10/04/2014 kl. 17.06 skrev Dag-Erling Sm=F8rgrav : > The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which > defaults to -D. This ensures that all timestamps inside static > libraries in the base system are hardcoded to 0 (aka the epoch), which > is a huge step towards fully reproducible builds. Any objections? I've used a similar patch for some time without problems. You should = grep the tree to check that ar and ranlib are always called with = ${ARFLAGS} and ${RANLIBFLAGS}. I seem to remember that some makefiles = just hardcode the flags. Erik= From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 07:59:00 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57F0C46B for ; Fri, 11 Apr 2014 07:59:00 +0000 (UTC) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1BB8F1469 for ; Fri, 11 Apr 2014 07:58:59 +0000 (UTC) Received: from bigmac.router9fbd7c.com (unknown [176.222.238.90]) by csmtp2.one.com (Postfix) with ESMTPA id 59362400146F2 for ; Fri, 11 Apr 2014 07:51:41 +0000 (UTC) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: ar and ranlib -D From: Erik Cederstrand In-Reply-To: <86eh15usv2.fsf@nine.des.no> Date: Fri, 11 Apr 2014 09:51:42 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <86eh15usv2.fsf@nine.des.no> X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 07:59:00 -0000 Den 10/04/2014 kl. 17.06 skrev Dag-Erling Sm=F8rgrav : > The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which > defaults to -D. This ensures that all timestamps inside static > libraries in the base system are hardcoded to 0 (aka the epoch), which > is a huge step towards fully reproducible builds. Any objections? I've used a similar patch for some time without problems. You should = grep the tree to check that ar and ranlib are always called with = ${ARFLAGS} and ${RANLIBFLAGS}. I seem to remember that some makefiles = just hardcode the flags. Erik= From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 18:27:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96755784; Fri, 11 Apr 2014 18:27:04 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 67A631CA0; Fri, 11 Apr 2014 18:27:04 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3BIQq1I003955 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 11 Apr 2014 11:26:55 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <534833E6.9070707@freebsd.org> Date: Sat, 12 Apr 2014 02:26:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: Time for turning off gdb by default? Or worse... References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:27:04 -0000 On 4/9/14, 9:03 PM, Ed Maste wrote: > On 8 April 2014 22:49, Julian Elischer wrote: >> also does lldb work with the xxgdb and ddd frontends? >> does it have its own frontend that makes it usable? > LLDB recently gained a curses-based UI. It seems pretty usable, > although I generally prefer to use a debugger's command-line > interface. > > I've put a screenshot of it here: > http://people.freebsd.org/~emaste/lldb/lldb-gui.png bleagh! compare with ddd.. http://img.brothersoft.com/screenshots/softimage/d/data_display_debugger_for_mac-203841-1231223624.jpeg https://www.gnu.org/software/ddd/ for more examples > > -Ed > From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 18:31:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ECB30959; Fri, 11 Apr 2014 18:31:21 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A42F61CD9; Fri, 11 Apr 2014 18:31:21 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3BIV9jR003967 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 11 Apr 2014 11:31:12 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <534834E6.1010206@freebsd.org> Date: Sat, 12 Apr 2014 02:31:02 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: John Baldwin , Warner Losh Subject: Re: Time for turning off gdb by default? Or worse... References: <201404091145.58792.jhb@freebsd.org> <674B7C0B-9235-4030-9A44-7F9984CA2F67@bsdimp.com> <201404101702.52622.jhb@freebsd.org> In-Reply-To: <201404101702.52622.jhb@freebsd.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:31:22 -0000 On 4/11/14, 5:02 AM, John Baldwin wrote: > On Thursday, April 10, 2014 4:37:17 pm Warner Losh wrote: >> OK. Here’s the summary of the thread: >> >> (1) gdb in tree is ancient >> (2) kgdb is quite useful, and only in tree >> (3) ports gdb rocks, but… >> (4) ports gdb exists only for a few architectures >> (5) Fixing ptrace will allow us to use a more-stock gdb >> >> Action items: >> >> (1) Create a wiki page with timeline to deactivation and removal. >> (2) Create milestones along the path for >> (a) kgdb + devel/gdb* >> (b) architectural coverage >> (c) ptrace fixes > I would actually invert these. I think (c) is the simplest to do > (in regards to the thread changes I mentioned) and I think it makes (b) > a lot easier to do. > >> (3) profit. > Otherwise, sounds good to me. > >> I’ve done these steps and documented them at https://wiki.freebsd.org/GdbRetirement to allow work to progress (or not) without repeating this discussion. Thanks > for everybody’s feedback. Feel free to comment on the wiki page or edit it for missing items (or testing you’ve done). >> At this point, I’m withdrawing the gdb disabled by default patches. > So one thing we kicked around on IRC is that I think it would be nice > to have some sort of place to collaborate on maintaining useful GPLv3 > toolchain bits. I don't think they belong in the main tree. However, > it might be nice to someday have another SVN repo that can be overlaid > into an existing src checkout (maybe using SVN external references?) > to allow GPLv3 gdb, etc. to be built as part of a world build. > > I don't know that we need an SVN repo on a FreeBSD.org machine right now, > but something would be nice. I have for some time, actively promoted the idea of "base ports vs regular ports" where base ports are not in the tree as such, but we maintain the port as part of the project, and a failure to build a base port is as much a reason to "stop ship" as failure to compile something in the tree.. From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 18:37:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AFCAE47 for ; Fri, 11 Apr 2014 18:37:43 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E828E1DBA for ; Fri, 11 Apr 2014 18:37:42 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3BIbb8Z004006 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 11 Apr 2014 11:37:40 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5348366A.1030001@freebsd.org> Date: Sat, 12 Apr 2014 02:37:30 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Warner Losh , freebsd-arch Subject: Re: Separating out building bootstrap and system compilers References: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> In-Reply-To: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:37:43 -0000 On 4/9/14, 11:25 AM, Warner Losh wrote: > I’d love to be able to say > > make buildworld WITHOUT_GCC=t WITHOUT_CLANG=t > > and get a working system out of it, without compilers. Too bad I can’t right now. > > Luckily, I worked up these patches. Here’s my proposed commit message. Please comment on the patch > (which can be found at http://people.freebsd.org/~imp/patch-queue/bootstrap) > > Separate out enabling building clang and/or gcc for the system and > building clang and/or gcc as the bootstrap compiler. Normally, the > default compiler is used. WITH_CLANG_BOOTSTRAP and/or > WITH_GCC_BOOTSTRAP will enable building these compilers as part > bootstrap phase. WITH/WITHOUT_CLANG_IS_CC controls which compiler is > used by default for the bootstrap phase, as well as which compiler is > installed as cc. buildworld now successfully completes building the > cross compiler with WITHOUT_CLANG=t and WITHOUT_GCC=t and produces a > built system with neither of these included. > > MK_CROSS_COMPILER will now force MK_CLANG_BOOTSTRAP=no and > MK_GCC_BOOTSTRAP=no. > > BOOTSTRAP_COMPILER was considered, but rejected, since pc98 needs both > clang and gcc to bootstrap still. It should be revisisted in the > future if this requirement goes away. Values should be gcc, clang or > none. > > Chances are good that MK_BINUTILS is a good candidate for similar > treatment. We likely need to fold Xxx causing things to magically not > happen into this scheme as well, but that may be a larger, more disruptive > change. > > Comments? for added credit add a top level arg that builds and installs all the bootstrap stuff (includes, libs, compilers, other tools) in a given destination.. I happen to need this. (ok, not need but it would be nice) at $JOB. Do it by hand at the moment. > > Warner > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 18:41:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6ABCA2B5; Fri, 11 Apr 2014 18:41:07 +0000 (UTC) Received: from mx1.fisglobal.com (mx1.fisglobal.com [199.200.24.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx1.fisglobal.com", Issuer "VeriSign Class 3 Secure Server CA - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B99851DEA; Fri, 11 Apr 2014 18:41:06 +0000 (UTC) Received: from smarthost.fisglobal.com ([10.132.206.193]) by ltcfislmsgpa04.fnfis.com (8.14.5/8.14.5) with ESMTP id s3BIf3uj024447 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Fri, 11 Apr 2014 13:41:04 -0500 Received: from THEMADHATTER (10.242.181.54) by smarthost.fisglobal.com (10.132.206.193) with Microsoft SMTP Server id 14.3.174.1; Fri, 11 Apr 2014 13:41:02 -0500 From: Sender: Devin Teske To: "'Julian Elischer'" , "'Ed Maste'" References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> <534833E6.9070707@freebsd.org> In-Reply-To: <534833E6.9070707@freebsd.org> Subject: RE: Time for turning off gdb by default? Or worse... Date: Fri, 11 Apr 2014 11:40:57 -0700 Message-ID: <10a201cf55b5$952a90b0$bf7fb210$@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AQCigVhaEdfHpHjbXGD3vvYF5FhSLgLb5L/3Ac+WvRMBOz3JqwI94lUCnSVDa8A= Content-Language: en-us X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.96, 1.0.14, 0.0.0000 definitions=2014-04-11_07:2014-04-11,2014-04-11,1970-01-01 signatures=0 Cc: 'freebsd-arch' X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 18:41:07 -0000 > -----Original Message----- > From: Julian Elischer [mailto:julian@freebsd.org] > Sent: Friday, April 11, 2014 11:27 AM > To: Ed Maste > Cc: freebsd-arch > Subject: Re: Time for turning off gdb by default? Or worse... > > On 4/9/14, 9:03 PM, Ed Maste wrote: > > On 8 April 2014 22:49, Julian Elischer wrote: > >> also does lldb work with the xxgdb and ddd frontends? > >> does it have its own frontend that makes it usable? > > LLDB recently gained a curses-based UI. It seems pretty usable, > > although I generally prefer to use a debugger's command-line > > interface. > > > > I've put a screenshot of it here: > > http://people.freebsd.org/~emaste/lldb/lldb-gui.png > > bleagh! > > compare with ddd.. > > http://img.brothersoft.com/screenshots/softimage/d/data_display_debugg > er_for_mac-203841-1231223624.jpeg > > https://www.gnu.org/software/ddd/ for more examples Beats the pants off the ol' Macsbug program ;) -- Devin _____________ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 20:27:36 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB052E6C for ; Fri, 11 Apr 2014 20:27:35 +0000 (UTC) Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B48961985 for ; Fri, 11 Apr 2014 20:27:35 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h18so1279738igc.14 for ; Fri, 11 Apr 2014 13:27:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=gk2yfztPbnzn9EXHgW8xIpYvVn2/BcCxtrL4Z4C80TA=; b=OU3ejdEsdvaQdPWHxd6Wmgc0d2rhTMYRjaPsjQBiW+9ZVb+uQjevlXNGDTJ/3e76pO HnyWRbuwYVypmenb3d7ABoa7WgXFQhszuASGv+ONAZOyxOLo4ZBePVH6xt+wMN/E/+Pv nI/i1YADP4O4yDaJAJvTLaKDNFGGlZEtmOXTs4DNMBdlXw2DMd7W/jK8L8uLZuH8kXwq wiq+9om6wqWKGpA1oqooZ2eVu6RHVrqLrXTf99BqJYPqVnQ1x9OAIRL4LmL5nDWUQq6K GSnQyZVgbeVbbHBgLxl5tM6MtrPHUgt1OcPuBnPvivx8/UnCBRRVbxSWST+KrwH+bO9v a+/g== X-Gm-Message-State: ALoCoQl+n91PHO+TvdFOrNMHpSFRz72azQiXba5puNfP4aAhK/Lw3RjkvoL2T7GkUmM7Ev6oyszV X-Received: by 10.50.143.34 with SMTP id sb2mr5974561igb.48.1397246528705; Fri, 11 Apr 2014 13:02:08 -0700 (PDT) Received: from [192.168.43.111] ([172.56.9.126]) by mx.google.com with ESMTPSA id j9sm7991000igu.10.2014.04.11.13.02.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 11 Apr 2014 13:02:08 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: Separating out building bootstrap and system compilers From: Warner Losh In-Reply-To: <5348366A.1030001@freebsd.org> Date: Fri, 11 Apr 2014 14:02:06 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <09D78C17-A4F6-4A79-96D4-413B937265F4@bsdimp.com> <5348366A.1030001@freebsd.org> To: Julian Elischer X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 20:27:36 -0000 On Apr 11, 2014, at 12:37 PM, Julian Elischer = wrote: > On 4/9/14, 11:25 AM, Warner Losh wrote: >> I=92d love to be able to say >>=20 >> make buildworld WITHOUT_GCC=3Dt WITHOUT_CLANG=3Dt >>=20 >> and get a working system out of it, without compilers. Too bad I = can=92t right now. >>=20 >> Luckily, I worked up these patches. Here=92s my proposed commit = message. Please comment on the patch >> (which can be found at = http://people.freebsd.org/~imp/patch-queue/bootstrap) >>=20 >> Separate out enabling building clang and/or gcc for the system and >> building clang and/or gcc as the bootstrap compiler. Normally, the >> default compiler is used. WITH_CLANG_BOOTSTRAP and/or >> WITH_GCC_BOOTSTRAP will enable building these compilers as part >> bootstrap phase. WITH/WITHOUT_CLANG_IS_CC controls which compiler is >> used by default for the bootstrap phase, as well as which compiler is >> installed as cc. buildworld now successfully completes building the >> cross compiler with WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt and = produces a >> built system with neither of these included. >>=20 >> MK_CROSS_COMPILER will now force MK_CLANG_BOOTSTRAP=3Dno and >> MK_GCC_BOOTSTRAP=3Dno. >>=20 >> BOOTSTRAP_COMPILER was considered, but rejected, since pc98 needs = both >> clang and gcc to bootstrap still. It should be revisisted in the >> future if this requirement goes away. Values should be gcc, clang or >> none. >>=20 >> Chances are good that MK_BINUTILS is a good candidate for similar >> treatment. We likely need to fold Xxx causing things to magically not >> happen into this scheme as well, but that may be a larger, more = disruptive >> change. >>=20 >> Comments? >=20 > for added credit add a top level arg that builds and installs all the = bootstrap stuff (includes, libs, compilers, other tools) in a given = destination.. > I happen to need this. (ok, not need but it would be nice) at $JOB. = Do it by hand at the moment. How does make xdev not fit your needs? Warner >>=20 >> Warner >>=20 >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Fri Apr 11 20:35:47 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4ABA9151; Fri, 11 Apr 2014 20:35:47 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EEEA11A44; Fri, 11 Apr 2014 20:35:46 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id j5so5965848qga.18 for ; Fri, 11 Apr 2014 13:35:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=vOIcaESle0rxwGK6/Jslsb7kdAVbupxVVikJdgHuoo8=; b=Q+MMWvloDtMO/RaidGDzMj6GCm4VL11YaduQg9IruWr9WE8T7/9Rtlq7jC1ZV0sZn8 uTrGRuMGycbmabJtFCwGai2VpZUa+JTxOWN6rH3Xx94lU8dvbNcXQt0Z8qJphIIMTZ5B 1kM7PTsjk+WZWacTGqdghBGU8CQREy+HNRgeD3lz3veiMx8Zn8RJgYOFqwV0vFAjhOLI HB6pCzudDBh6R7tjMo5HTV8SYsd1yA6qB8t0/F0t0cmo0Mmu/X1O7K3pwp5XOMUEBhfl p/zVwdA/VrlQFvP5NUtbs5hzOmXFmIIO1yACmBO6i7/wbl4ycu20/d8mUBagVLVn3F6c BNTQ== MIME-Version: 1.0 X-Received: by 10.140.89.11 with SMTP id u11mr4279152qgd.93.1397248546148; Fri, 11 Apr 2014 13:35:46 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Fri, 11 Apr 2014 13:35:46 -0700 (PDT) In-Reply-To: <534833E6.9070707@freebsd.org> References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> <534833E6.9070707@freebsd.org> Date: Fri, 11 Apr 2014 16:35:46 -0400 X-Google-Sender-Auth: K29VUs6HFigOL7Lwurdoahha_wI Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Ed Maste To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2014 20:35:47 -0000 On 11 April 2014 14:26, Julian Elischer > bleagh! > > compare with ddd.. > > http://img.brothersoft.com/screenshots/softimage/d/data_display_debugger_for_mac-203841-1231223624.jpeg > > https://www.gnu.org/software/ddd/ for more examples I'm not sure how that's relevant; we're not contemplating putting that in the base system. From owner-freebsd-arch@FreeBSD.ORG Sat Apr 12 00:49:46 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EE10C1A for ; Sat, 12 Apr 2014 00:49:46 +0000 (UTC) Received: from eastrmfepo203.cox.net (eastrmfepo203.cox.net [68.230.241.218]) by mx1.freebsd.org (Postfix) with ESMTP id 161651241 for ; Sat, 12 Apr 2014 00:49:45 +0000 (UTC) Received: from eastrmimpo305 ([68.230.241.237]) by eastrmfepo203.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140412004939.ZITK30677.eastrmfepo203.cox.net@eastrmimpo305> for ; Fri, 11 Apr 2014 20:49:39 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo305 with cox id oope1n00D41obj401opeul; Fri, 11 Apr 2014 20:49:39 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020209.53488DA3.003B,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=Sv5OHoy0 c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=N659UExz7-8A:10 a=kviXuzpPAAAA:8 a=6I5d2MoRAAAA:8 a=Q2wZHFncVIygqW3tLTsA:9 a=pILNOxqGKmIA:10 a=4vB-4DCPJfMA:10 a=SV7veod9ZcQA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53488DA7.5000307@cox.net> Date: Fri, 11 Apr 2014 20:49:43 -0400 From: "John D. Hendrickson and Sara Darnell" User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: Warner Losh Subject: Re: ar and ranlib -D References: <86eh15usv2.fsf@nine.des.no> <534710F4.1040908@cox.net> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Cc: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= , Ed Maste , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: johnandsara2@cox.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 00:49:46 -0000 Warner Losh wrote: > On Apr 10, 2014, at 3:45 PM, John D. Hendrickson and Sara Darnell wrote: > >> Warner Losh wrote: >>> On Apr 10, 2014, at 9:22 AM, Ed Maste wrote: >>>> On 10 April 2014 11:06, Dag-Erling Smørgrav wrote: >>>>> The attached patch adds -D to ARFLAGS and introduces RANLIBFLAGS which >>>>> defaults to -D. This ensures that all timestamps inside static >>>>> libraries in the base system are hardcoded to 0 (aka the epoch), which >>>>> is a huge step towards fully reproducible builds. Any objections? >>>> Looks good to me, I'm not sure why this didn't happen long ago. >>> Once upon a time, ranlib didn’t like this too well and complained that >>> the index was older than the file. Then it was made a special case. These >>> days (and these days includes time since ~1995 or 2000), people >>> always rebuild the entire .a anyway, so the value of having a timestamp >>> in there is low, at best, so always doing this has become so boring >>> that i’m surprised this isn’t the default behavior. Given that we always >>> rebuild, though, this change is totally safe. >>> Warner >>> _______________________________________________ >>> freebsd-arch@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >>> To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >> i can't confirm a ar needs inner timestamps as an interface dependancy issue or that software depends on this format >> >> however it does make sense IF AND ONLY IF the timestamp are used for compilation order dependancy, and compilation is about mapping holes by dependancy, it's ok for releasetime AFTER compiling (NOT during compiling) >> >> but you all aren't saying you've done any research >> >> but i don't know if any kernels use it with lib version as a link to and extended interface or as a flag. i don't know if any existing software crashes if the value is 0. >> >> does install(1) strip this info at the final stage ? never mind >> >> no. these days people always build such and such a way. bull. >> >> ar is used by assemblers not just gcc(1) >> >> some people insert and remove asm libraries by hand. all what the job requires to get done. >> >> any argument "these days we will only allow". no. that's messing up what had already worked right ? >> >> ok well good luck > > In the tree, we do none of these things, so the change is safe. The kernel doesn’t care, the user land .a files don’t care since we always rebuild them and never insert. > Outside the tree, people use bsd.lib.mk, which does things the way I say so the change is safe. > Outside of that, these changes make no difference, especially for the use cases you describe, so that is safe. > > What am I missing? > > Warner > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > i think your fine since BSD so far has an excellent original source tree to fall back on and no one will depend on the change per say i don't run BSD so i can't confirm your libtools even stores/reads time in ar's i was hoping you didn't think i meant compile literal (i meant when it's final link time resetting is fine, while it's being made too much is going on to say for sure who when needs a dependancy stamp before) i think your on track fine if your objective is release time sanity. i only meant to highlight things to think about i almost wish i hadn't well, good luck again From owner-freebsd-arch@FreeBSD.ORG Sat Apr 12 01:50:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A2A26BA; Sat, 12 Apr 2014 01:50:00 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A3611717; Sat, 12 Apr 2014 01:49:59 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3C1nqgL005039 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 11 Apr 2014 18:49:55 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53489BBA.2040802@freebsd.org> Date: Sat, 12 Apr 2014 09:49:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: Time for turning off gdb by default? Or worse... References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> <534833E6.9070707@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 01:50:00 -0000 On 4/12/14, 4:35 AM, Ed Maste wrote: > On 11 April 2014 14:26, Julian Elischer >> bleagh! >> >> compare with ddd.. >> >> http://img.brothersoft.com/screenshots/softimage/d/data_display_debugger_for_mac-203841-1231223624.jpeg >> >> https://www.gnu.org/software/ddd/ for more examples > I'm not sure how that's relevant; we're not contemplating putting that > in the base system. > I'm suggestng that it's another way where gdb has features that lldb doesn't. A plethora of front-ends. From owner-freebsd-arch@FreeBSD.ORG Sat Apr 12 02:14:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CF3AC45 for ; Sat, 12 Apr 2014 02:14:09 +0000 (UTC) Received: from mail-ve0-x229.google.com (mail-ve0-x229.google.com [IPv6:2607:f8b0:400c:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 036EB1935 for ; Sat, 12 Apr 2014 02:14:08 +0000 (UTC) Received: by mail-ve0-f169.google.com with SMTP id pa12so5729309veb.28 for ; Fri, 11 Apr 2014 19:14:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=otaiMu7Uicqnu1WRq3ecQkx5//rkQY9wwtBisbJTvpc=; b=M++STf1ijonuQ/KFOVAhY0qgGE4LWoKWK6xQ8OWxqSb14CIbyntXhaBrMKM+m8/C2X dTDS2AkTZsoy/4p3NF4L2KMXJ/vZ1YtSsql+nCxS7K8P92W9JXGfLsY+PgPJzhxsaQB+ 8xE7YuRVQ+Tmoxt0INmpBC7V2yBHPoYqV4fos= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=otaiMu7Uicqnu1WRq3ecQkx5//rkQY9wwtBisbJTvpc=; b=BDBoR13pKPpAExfSWI6e9nQ5D7EiXQaXWxyd2MBcIGzHKGCSCgOS4cQNjiwRuGDEfw pJ1eicZaQLFSBdRT0qk7Gkfy9jK6NGUd8/jlJURImguvMCfRJVTuF5v82G4byPUK8Efb R9dPOWbELtBHRxJEZ+4R8hqdY/oP9VACGxJDwzb+j27y2eeX5zhKqvVsa2Bpvq3G7eZN TDeJnVjOGgXFLnmBLwHdixyz1RLweftPtTzu2JVIeF3t7wWF5p3VVrTkkdLe8EC2HddX RG1E5mQi4PKOv4sI93JcC2q89/BeGiwWZJcc1MNBp5kuEuEq/B8YhR7I3N4MghZCIilV vq5A== X-Gm-Message-State: ALoCoQnsKXQmG3fhM0xkr3dJLjT+WdqbKpaqA2JsUjG6M5bovU7fc1tpJCfqQO6PsHN6sOuhv+Uq MIME-Version: 1.0 X-Received: by 10.220.198.197 with SMTP id ep5mr10405829vcb.21.1397268847710; Fri, 11 Apr 2014 19:14:07 -0700 (PDT) Received: by 10.220.30.69 with HTTP; Fri, 11 Apr 2014 19:14:07 -0700 (PDT) In-Reply-To: <53489BBA.2040802@freebsd.org> References: <20140408212629.GD21331@kib.kiev.ua> <5344B551.30200@freebsd.org> <534833E6.9070707@freebsd.org> <53489BBA.2040802@freebsd.org> Date: Fri, 11 Apr 2014 19:14:07 -0700 Message-ID: Subject: Re: Time for turning off gdb by default? Or worse... From: Peter Wemm To: Julian Elischer Content-Type: text/plain; charset=ISO-8859-1 Cc: Ed Maste , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Apr 2014 02:14:09 -0000 On Fri, Apr 11, 2014 at 6:49 PM, Julian Elischer wrote: > On 4/12/14, 4:35 AM, Ed Maste wrote: >> >> On 11 April 2014 14:26, Julian Elischer >>> >>> bleagh! >>> >>> compare with ddd.. >>> >>> >>> http://img.brothersoft.com/screenshots/softimage/d/data_display_debugger_for_mac-203841-1231223624.jpeg >>> >>> https://www.gnu.org/software/ddd/ for more examples >> >> I'm not sure how that's relevant; we're not contemplating putting that >> in the base system. >> > I'm suggestng that it's another way where gdb has features that lldb > doesn't. A plethora of front-ends. Maybe, but that's not the question here. The curses front end was a distraction. The issue was: Given that the in-tree gdb is woefully stale and increasingly difficult to use with our current toolchain, is it time to turn it off and put our weight behind the ports version? If so, what would we have to take care of or bring over to the port version before we could? The consensus seems to be that the loss of kgdb functionality would be a blocker. Front ends for debuggers are way out of scope of the original question. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 13:59:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71678704; Tue, 15 Apr 2014 13:59:22 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48F751D4C; Tue, 15 Apr 2014 13:59:22 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 65EA5B924; Tue, 15 Apr 2014 09:59:20 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Time for turning off gdb by default? Or worse... Date: Tue, 15 Apr 2014 09:31:24 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <53489BBA.2040802@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201404150931.24460.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 15 Apr 2014 09:59:20 -0400 (EDT) Cc: Ed Maste X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 13:59:22 -0000 On Friday, April 11, 2014 10:14:07 pm Peter Wemm wrote: > On Fri, Apr 11, 2014 at 6:49 PM, Julian Elischer wrote: > > On 4/12/14, 4:35 AM, Ed Maste wrote: > >> > >> On 11 April 2014 14:26, Julian Elischer > >>> > >>> bleagh! > >>> > >>> compare with ddd.. > >>> > >>> > >>> http://img.brothersoft.com/screenshots/softimage/d/data_display_debugger_for_mac-203841-1231223624.jpeg > >>> > >>> https://www.gnu.org/software/ddd/ for more examples > >> > >> I'm not sure how that's relevant; we're not contemplating putting that > >> in the base system. > >> > > I'm suggestng that it's another way where gdb has features that lldb > > doesn't. A plethora of front-ends. > > Maybe, but that's not the question here. The curses front end was a > distraction. > > The issue was: > Given that the in-tree gdb is woefully stale and increasingly > difficult to use with our current toolchain, is it time to turn it off > and put our weight behind the ports version? If so, what would we > have to take care of or bring over to the port version before we > could? > > The consensus seems to be that the loss of kgdb functionality would be > a blocker. > > Front ends for debuggers are way out of scope of the original question. It would be relevant if the answer were to ditch gdb entirely and depend on a kernel front-end for lldb for all kernel debugging (which is not an unreasonable assumption to make given what we've done with the compiler). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 20:50:23 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4407C11 for ; Tue, 15 Apr 2014 20:50:23 +0000 (UTC) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A61B1AAA for ; Tue, 15 Apr 2014 20:50:23 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id bj1so10112969pad.3 for ; Tue, 15 Apr 2014 13:50:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:content-type :content-transfer-encoding:subject:message-id:date:to:mime-version; bh=nZAijvXSX+W8CImG0Gd0f6edyTAG7L9nnlys1S0htas=; b=ge4WHrWt2truvQne3f52L06kgv0o0Zra2rsgKQ/wXCQpGs6HxsYu4uXmAac99wprLo n29Ltt++TH7+/fhf/nD/pPcefjwr0ZwYn8Q00q19ytbqiQumFAq28sBJDG0jzv3w22Zq HDor78wc3zva0dDJVfVIuDyJW2zi0iAR3vcVqJvn0XDxZes1lN6w8h3D/59TEUYhTC/E S2JS3wORLKTg8IgsNUGJImnwLTgRYaQy1FrHOeSrCP0+GU6IF5VLHIlKd1ep3UJjDTZx Xsof6kZ0X7ccMykmydFm/YukzNQLPNlyHtCnOEnwDT4FEZcGHSCp6RBy/6tkvD730UHV bCQg== X-Gm-Message-State: ALoCoQlIqrM9J+2h9kbljuAH24z2bP4pAv3AureB7OoWFeGHe1qTxV+1ceJmEwzs8GHsnRfwuatW X-Received: by 10.66.146.199 with SMTP id te7mr4313105pab.106.1397595017370; Tue, 15 Apr 2014 13:50:17 -0700 (PDT) Received: from macintosh-c42c033c0b73.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id hy3sm42306065pbc.31.2014.04.15.13.50.16 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Apr 2014 13:50:16 -0700 (PDT) Sender: Warner Losh From: Warner Losh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: EISA in GENERIC Message-Id: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> Date: Tue, 15 Apr 2014 14:50:14 -0600 To: "freebsd-arch@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 20:50:23 -0000 Greetings, one and all. The time has come to trim EISA from the generic i386 kernel. Please see http://people.freebsd.org/~imp/patch-queue/eisa for the proposed change. It introduces a MK_EISA too so one can control building the eisa-only modules as well as the eisa attachments in modules. There are those that say it is time to vote EISA off the island. = Perhaps, but that=92s a completely different discussion than the one I=92m = wanting to have now. The normal way that should be done is to remove it in 12 after disabling it in 11. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 20:59:48 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E6BBFFD for ; Tue, 15 Apr 2014 20:59:48 +0000 (UTC) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5AE5D1BB7 for ; Tue, 15 Apr 2014 20:59:43 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id s3FKxJ1e016986; Tue, 15 Apr 2014 15:59:19 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id s3FKxJI0016985; Tue, 15 Apr 2014 15:59:19 -0500 (CDT) (envelope-from brooks) Date: Tue, 15 Apr 2014 15:59:19 -0500 From: Brooks Davis To: Warner Losh Subject: Re: EISA in GENERIC Message-ID: <20140415205919.GA16560@lor.one-eyed-alien.net> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 20:59:48 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 15, 2014 at 02:50:14PM -0600, Warner Losh wrote: > Greetings, one and all. >=20 > The time has come to trim EISA from the generic i386 kernel. >=20 > Please see http://people.freebsd.org/~imp/patch-queue/eisa for > the proposed change. It introduces a MK_EISA too so one can > control building the eisa-only modules as well as the eisa attachments > in modules. The diff looks good to me. -- Brooks --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iKYEARECAGYFAlNNnaZfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDY1NUQ1MTlDMjZBNzgyRTcyNTI5OUJGMDVE OEU4QkU5RjIzODFBRDQACgkQXY6L6fI4GtQXNgCfRuwrgTKNAjPti/P4Vy0bWxuE eQkAn2+e/zC4ouQ4RdT+jiYm8kDQqVgi =IrX+ -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 21:40:49 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B80DD6E0 for ; Tue, 15 Apr 2014 21:40:49 +0000 (UTC) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "troutmask.apl.washington.edu", Issuer "troutmask.apl.washington.edu" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 95DF810A7 for ; Tue, 15 Apr 2014 21:40:49 +0000 (UTC) Received: from troutmask.apl.washington.edu (localhost.apl.washington.edu [127.0.0.1]) by troutmask.apl.washington.edu (8.14.8/8.14.8) with ESMTP id s3FLeggO073501 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 15 Apr 2014 14:40:42 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.14.8/8.14.8/Submit) id s3FLegbH073500; Tue, 15 Apr 2014 14:40:42 -0700 (PDT) (envelope-from sgk) Date: Tue, 15 Apr 2014 14:40:42 -0700 From: Steve Kargl To: Warner Losh Subject: Re: EISA in GENERIC Message-ID: <20140415214042.GA73405@troutmask.apl.washington.edu> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 21:40:49 -0000 On Tue, Apr 15, 2014 at 02:50:14PM -0600, Warner Losh wrote: > > The time has come to trim EISA from the generic i386 kernel. > > Please see http://people.freebsd.org/~imp/patch-queue/eisa for > the proposed change. It introduces a MK_EISA too so one can > control building the eisa-only modules as well as the eisa attachments > in modules. > > There are those that say it is time to vote EISA off the island. Perhaps, > but that?s a completely different discussion than the one I?m wanting > to have now. The normal way that should be done is to remove it in 12 > after disabling it in 11. > No problem with intent of patch. Do you need to make any changes for bt(4)? My first foray into EISA used at Buslogic BT-742A. 'man bt' does not show a dependence on 'device eisa', but bt(4) certainly supported EISA cards. -- Steve From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 22:41:53 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30703535 for ; Tue, 15 Apr 2014 22:41:53 +0000 (UTC) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70C5D169A for ; Tue, 15 Apr 2014 22:41:52 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id s3FMfYcq017384 for ; Tue, 15 Apr 2014 17:41:34 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id s3FMfYDF017383 for arch@freebsd.org; Tue, 15 Apr 2014 17:41:34 -0500 (CDT) (envelope-from brooks) Date: Tue, 15 Apr 2014 17:41:34 -0500 From: Brooks Davis To: arch@freebsd.org Subject: time to svn rm lib/libkse? Message-ID: <20140415224134.GA17146@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vkogqOf2sHV7VnPd" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:41:53 -0000 --vkogqOf2sHV7VnPd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I noticed to chance that we still include the source for libkse in the tree. It was unhooked from the build and kernel support removed in 2008 so I think it's due to be removed from HEAD and maybe 8, 9, and 10. -- Brooks --vkogqOf2sHV7VnPd Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iKYEARECAGYFAlNNtZ1fFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDY1NUQ1MTlDMjZBNzgyRTcyNTI5OUJGMDVE OEU4QkU5RjIzODFBRDQACgkQXY6L6fI4GtTbYgCgsmDmzzpEVdBwzpCGT2ce77gR d4IAoLFagxfau4lObtwvA3lFANwZn+ao =GWUY -----END PGP SIGNATURE----- --vkogqOf2sHV7VnPd-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 22:44:31 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94C906CB for ; Tue, 15 Apr 2014 22:44:31 +0000 (UTC) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64F2D16AD for ; Tue, 15 Apr 2014 22:44:31 +0000 (UTC) Received: by mail-pa0-f48.google.com with SMTP id hz1so10130084pad.21 for ; Tue, 15 Apr 2014 15:44:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=VK0Wk7kkej0WqQ9/Js+d93PPv/TLruN2XZ/AaBkjOXc=; b=SJL2gaabOzkwHFjYoun2tcYxi4G26rnbi6aTXs7WNu+19MW59UOdMtHtDZdboYUnQz vJlTiuX9aXCMo17Kbn8e5GlA/I0rYqtq/QbiBTBqK5HZr3opSI8l2Bh9zDF0XE9zWu/+ gm9GpVVbAigxonTHwttkTgaVijl1n1RVsA5SUZ1rcqbfUf/LpwbubHX4S8XmIHKUe4Zr zauVMSgCOxfax3Vz08ywndFu3qmWLeg7e9l2wh+7BWe7k1gUPTtZ8JVKFMPE9DRMhDtF eMLXsOKs/6Xmcwo4l9MqyffFLL0wcq5WkgA8FJoVPnqvXBC+iHOSU7Z9qMHiUeX8WKYE 4MUw== X-Gm-Message-State: ALoCoQky/81gG/ufER44WhTsGKih0LhDxkS60qK0T5oFi/fkrOX/I/MKrQxQVXK6hf6BJDxJmIgx X-Received: by 10.67.2.34 with SMTP id bl2mr4813706pad.58.1397601869370; Tue, 15 Apr 2014 15:44:29 -0700 (PDT) Received: from macintosh-c42c033c0b73.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ja8sm42686410pbd.3.2014.04.15.15.44.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Apr 2014 15:44:28 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: EISA in GENERIC From: Warner Losh In-Reply-To: <20140415214042.GA73405@troutmask.apl.washington.edu> Date: Tue, 15 Apr 2014 16:44:25 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8E01A77C-D7DE-4298-AA19-DCECF400812D@bsdimp.com> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> <20140415214042.GA73405@troutmask.apl.washington.edu> To: Steve Kargl X-Mailer: Apple Mail (2.1874) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:44:31 -0000 On Apr 15, 2014, at 3:40 PM, Steve Kargl = wrote: > On Tue, Apr 15, 2014 at 02:50:14PM -0600, Warner Losh wrote: >>=20 >> The time has come to trim EISA from the generic i386 kernel. >>=20 >> Please see http://people.freebsd.org/~imp/patch-queue/eisa for >> the proposed change. It introduces a MK_EISA too so one can >> control building the eisa-only modules as well as the eisa = attachments >> in modules. >>=20 >> There are those that say it is time to vote EISA off the island. = Perhaps, >> but that?s a completely different discussion than the one I?m wanting >> to have now. The normal way that should be done is to remove it in 12 >> after disabling it in 11. >>=20 >=20 > No problem with intent of patch. >=20 > Do you need to make any changes for bt(4)? My first foray into > EISA used at Buslogic BT-742A. 'man bt' does not show a dependence > on 'device eisa', but bt(4) certainly supported EISA cards. This is handled in the config system. When =91device eisa=92 is omitted, = the eisa attachment for bt is omitted. There=92s no bt or buslogic module, = so no change is needed to cope there. At this late date, it is doubtful if = a bt/buslogic module would be useful to create=85 Thanks for the review... Warner From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 22:44:47 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90EB2765; Tue, 15 Apr 2014 22:44:47 +0000 (UTC) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C25416AE; Tue, 15 Apr 2014 22:44:46 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.8/8.14.8/NETPLEX) with ESMTP id s3FMij6H010833; Tue, 15 Apr 2014 18:44:45 -0400 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Tue, 15 Apr 2014 18:44:45 -0400 (EDT) Date: Tue, 15 Apr 2014 18:44:45 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Brooks Davis Subject: Re: time to svn rm lib/libkse? In-Reply-To: <20140415224134.GA17146@lor.one-eyed-alien.net> Message-ID: References: <20140415224134.GA17146@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 22:44:47 -0000 On Tue, 15 Apr 2014, Brooks Davis wrote: > I noticed to chance that we still include the source for libkse in the > tree. It was unhooked from the build and kernel support removed in > 2008 so I think it's due to be removed from HEAD and maybe 8, 9, and 10. +1, no objection here. -- DE From owner-freebsd-arch@FreeBSD.ORG Tue Apr 15 23:11:58 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DF68DD21 for ; Tue, 15 Apr 2014 23:11:58 +0000 (UTC) Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B370C1947 for ; Tue, 15 Apr 2014 23:11:58 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id y10so10031652pdj.13 for ; Tue, 15 Apr 2014 16:11:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=W9hqzRfoNR7AyubXYtz4K3JzfykRxlxgXF3JtXgGQkk=; b=hoZ6kItXX7anbkgnZmhwmGvk1GJ4AqKdPAbVYH6OPC6kYO++TsIfSwWO0WQj95itgH ARnzo/TXgE4T3WEDTrLmG4U6qTS3ibWdlMqQkqGUYsxtP60OE4OaSQzTt5I1J3lnWwry 3oa084bo+DNkxiqdMz9fQ3HwSVuvTzxpUsKpDQNfXSUaxkZJyBZ2ul522EKc7ijroAtW fueeQP2G7onLhERJMINUIuzi46NtMOaX/NhYXy5jl6Y1ItEJQ5MvxXLXIEcSwiiuNUFz sN1H4T4JrFQcWWinfSYAtP8//mTVcz892+U/rowVx8HqhP47nxpXkob/1htZkL/FqHJO piEQ== X-Gm-Message-State: ALoCoQnnUikulGpXu+rKsYmj0MOAlf620s4cIYEHyAa2bYDFL9+yZA2M6WYhxSUAv7MYh3l0ju31 X-Received: by 10.66.233.169 with SMTP id tx9mr4790445pac.7.1397603512446; Tue, 15 Apr 2014 16:11:52 -0700 (PDT) Received: from macintosh-c42c033c0b73.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id vo1sm101647579pab.32.2014.04.15.16.11.50 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Apr 2014 16:11:51 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: time to svn rm lib/libkse? From: Warner Losh In-Reply-To: <20140415224134.GA17146@lor.one-eyed-alien.net> Date: Tue, 15 Apr 2014 17:11:43 -0600 Content-Transfer-Encoding: 7bit Message-Id: References: <20140415224134.GA17146@lor.one-eyed-alien.net> To: Brooks Davis X-Mailer: Apple Mail (2.1874) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Apr 2014 23:11:58 -0000 On Apr 15, 2014, at 4:41 PM, Brooks Davis wrote: > I noticed to chance that we still include the source for libkse in the > tree. It was unhooked from the build and kernel support removed in > 2008 so I think it's due to be removed from HEAD and maybe 8, 9, and 10. /* XXXKSE What to do */ I think that it is time to remove it. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 00:37:01 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28ED3938 for ; Wed, 16 Apr 2014 00:37:01 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E00651197 for ; Wed, 16 Apr 2014 00:37:00 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so10623016qgd.15 for ; Tue, 15 Apr 2014 17:37:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=UzhCa/gvddpXVAMpcmMIcProiUXRhhFV6eNhVuP9g0I=; b=YHixUdgwl/zinz07m6yMgPvd/c42MWjaznoTFrxxoLQEAHdxP9ReWWcCJTNkkDdKGc +UksRN0pzjNqqz4G9JbyfFChuEluOjvbsgDt6uYuDSZ2t9GN0Dz47BWmUnpwLCRRHipC zz1R4E3Q7kqCHgpXhVT2+v0NptJpjjUIHdJUHYLLWBya9QZ+gHskW0fO/tmMZisr2K7H c0tlhIoY6+hk8JffziI6lA+ySUzd7fx2wpgjUWf0koGzVuJ8pncnXgGM79rpcPvCRUxF G9Lg9YF6AFpiH8E0LHcE3OwTe2lHAaJZBvq+Eu3BIj772ixNzxgIX+cG8NVKS5QtuK+w pIgg== MIME-Version: 1.0 X-Received: by 10.229.81.71 with SMTP id w7mr321831qck.8.1397608620007; Tue, 15 Apr 2014 17:37:00 -0700 (PDT) Sender: carpeddiem@gmail.com Received: by 10.140.88.105 with HTTP; Tue, 15 Apr 2014 17:36:59 -0700 (PDT) In-Reply-To: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> Date: Tue, 15 Apr 2014 20:36:59 -0400 X-Google-Sender-Auth: 9kA9rrO5e-ornQbzWZJLf4fpQ6Y Message-ID: Subject: Re: EISA in GENERIC From: Ed Maste To: Warner Losh Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 00:37:01 -0000 On 15 April 2014 16:50, Warner Losh wrote: > Greetings, one and all. > > The time has come to trim EISA from the generic i386 kernel. > > Please see http://people.freebsd.org/~imp/patch-queue/eisa for > the proposed change. It introduces a MK_EISA too so one can > control building the eisa-only modules as well as the eisa attachments > in modules. Looks good to me. From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 00:49:37 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4E28CC2 for ; Wed, 16 Apr 2014 00:49:37 +0000 (UTC) Received: from mail.soaustin.net (pancho.soaustin.net [76.74.250.40]) by mx1.freebsd.org (Postfix) with ESMTP id A7BE3127F for ; Wed, 16 Apr 2014 00:49:37 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id C75965607F; Tue, 15 Apr 2014 19:49:36 -0500 (CDT) Date: Tue, 15 Apr 2014 19:49:36 -0500 From: Mark Linimon To: Warner Losh Subject: Re: EISA in GENERIC Message-ID: <20140416004936.GA30375@lonesome.com> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 00:49:37 -0000 On Tue, Apr 15, 2014 at 02:50:14PM -0600, Warner Losh wrote: > The time has come to trim EISA from the generic i386 kernel. Even a junk-collector like me has no EISA-capable machines left. Like, for 6 or 7 years. (might still have a SCSI controller though. If anyone wants it :-) ) mcl From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 02:28:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 773A3393 for ; Wed, 16 Apr 2014 02:28:00 +0000 (UTC) Received: from mail-pa0-f43.google.com (mail-pa0-f43.google.com [209.85.220.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 442BE1BDF for ; Wed, 16 Apr 2014 02:27:59 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id bj1so10359346pad.30 for ; Tue, 15 Apr 2014 19:27:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=IO7Qzihqw2f9ZUKkqDeyjMHi+t5mEeYsvTa2iWwg0vU=; b=ShshdjCSIY70EBoPxavorMDHRB+nwtyfLTw+UehymvCrzJlliBcZoFD+qJ8LKim+M0 ZGrAVdb2npuo0h+yf+xWT5EXw9bsSqV7PQCDwvtDpWswp2uyw+i1ogecvBCLiZlTH4aa ghZ7jlLC7f2pI/nuPc7K+KnLxeWapBTiYW82u0Dykz9tvRpaL85+4PZmMPA5+GY1rvm3 x+rTNGco4uBVmxD9scO0dtqGEyBmO5dDcuQyqrhfy4x3nFJN6Ys+SYSpT/Yk4dQzJgwT MxgVMoaV5NFbCsTqgA/ubcAyh6GjAbqdbAcsyX9QErPqaoHr97A4bmolv2pKnyxzT88R L6Iw== X-Gm-Message-State: ALoCoQmRVFqZ8BhV46c6QzAxG/k0FTLIp0zky0r/2g2L/DJFRD6jQtxC00sr2BT1fLxgosHAeE/J X-Received: by 10.68.201.226 with SMTP id kd2mr5499832pbc.157.1397615272911; Tue, 15 Apr 2014 19:27:52 -0700 (PDT) Received: from macintosh-c42c033c0b73.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id nx12sm103628711pab.6.2014.04.15.19.27.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Apr 2014 19:27:52 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140410034645.C058658097@chaos.jnpr.net> Date: Tue, 15 Apr 2014 20:27:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 02:28:00 -0000 Hi Simon, On Apr 9, 2014, at 9:46 PM, Simon J. Gerraty wrote: >> It sounds like we=3D92re in agreement on how to proceed, or very = nearly =3D >> so. I have some more >> comments below, but I think the next order of business would be = patches. =3D >> Do you have some ready to roll?=20 >=20 > No, not yet - I figured getting some level of consensus fist was > worthwhile. >=20 > My cunning plan was simply to remove the mechanism from bsd.own.mk, > and include the new file at approximately the original location. > Thus not necessarily any functional change. > The devil's in the details of course. >=20 > I'm obviously biased, but I think contrib/bmake/mk/options.mk is close > to what is desired - just the mechanism with no policy. I=92ve created a bsd.srcopt.mk and adjusted bsd.own.mk in a sample = patch, since it sounds like we=92re headed in the right direction. Please take a look = at it and let me know what you think. http://people.freebsd.org/~imp/patch-queue/bsd.srcopt.mk contains the patch and commit message (if I did the queue management = right) that I=92d like to proffer. If we can make this somehow more generic, I=92d be game, but we have = some fairly ugly interdependencies that may complicate a clean, simple = solution. >> If so, then we should iterate on them. If not, I can =3D >> find some time over >> the coming days/weeks. I=3D92d like to have this done before BSDcan, = but =3D >> if not we can >> chat then, assuming you are going. >=20 > Sounds good, yes I'll be there. >=20 >>> That's why I'd put such things in local.sys.mk or some other = makefile >>> that affects /usr/src but isn't install into /usr/share/mk/ >>=20 >> That=3D92s a bit of a departure over how we=3D92ve done things, but = one that =3D >> may make a good amount of sense.=20 >=20 > I've been distributing my *.mk files for a long time, and I prefer > people leave them as is, so I make it easy for them to customize = without > hacking. Adding .-include "local.*.mk" allows for a lot of > customization. > When I wrote dirdeps.mk et al it was with the understanding that = Juniper > would make them publicly available, so used the same model. >=20 > IIRC there was some discussion at dev summit or BSDCan last year about > having src/ look for src.conf, make.conf etc inside the tree rather = than > /etc. I know in our internal builds we want to ensure that our src/ = is > self contained. Yea, we can work out the policy mechanism for this. I was kinda hoping to be able to use this tool to fix the NO_CTF issue that we talked about earlier in this thread. I=92m not tied to what I=92ve posted here, but I = would like to reach resolution so I can fix some other things in the build = without much delay. >> Where would the src build pick this up from if =3D >> it isn=3D92t installed? >=20 > src/share/mk should work fine. OK. I haven=92t added it to the Makefile so it gets installed, and = things seem to build, but that=92s not a viable way to commit things. Please = comment on what I=92ve done and what you think the right way to proceed might = be. The easy path is, of course, just installing it :) >>> Wouldn't it be simpler to set MK_CTF=3D3Dno *before* including =3D >> bsd.own.mk ? >>=20 >> Well, kinda=3D85 Then the issue becomes, in what I think you are =3D >> proposing, what happens >> to the meta variables, or MK_foo that sets a lot of MK_bar. Assuming = we =3D >> move all of >> them to their own file, we have two sections. One that sets MK_xxx =3D >> variables based on >> WITH/WITHOUT, and a second one that sets them based on other MK_xxx =3D= >> variables. >> If I set MK_CTF=3D3Dno in my makefile, and it caused other MK_ flags = to be =3D >> set, then I=3D92d have >> to include something to take another run at setting those =3D >> meta-variables. >=20 > Not necessarily. The stuff in bsd.own.mk like: >=20 > .if ${MK_TOOLCHAIN} =3D=3D "no" > MK_BINUTILS:=3D no > MK_CLANG:=3D no > MK_GCC:=3D no > MK_GDB:=3D no > .endif >=20 > doesn't need to change, and for more elaborate stuff, simply being = able > to include *options.mk more than once may help quite a bit. That might be hard, since MK_FOO being set from the first time won=92t allow us to override them the second or further time. I opted for a = guard target in my rendition, but now that I think about it, I=92m not so sure = that=92s needed and multiple include would allow us to remove the current restriction that you can=92t toggle MK_META from =93no=94 to =93yes=94 = after we=92ve included the option.mk file. >>> Yes, that's essentially what I was proposing. >>> By extracting the mechanism to its own file, it can be re-used. >>=20 >> Do you have patches? I think I like it... >=20 > No, but could do so pretty quickly. > I need to resync projects/bmake and start writing my material for = BSDCan > ;-) I could look doing this as part of that. >=20 > The hardest bit is naming the new makefile ;-) > bsd.options.mk would be an obvious choice - though that conflicts with > ports usage. Yea, I chose bsd.srcopt.mk for my doodles. >>> Calling it bsd.options.mk is a conflict with ports. >>> Though including it as "bsd.options.mk" both in bsd.own.mk and in = the >>> relevant ports makefile, should probably mitigate that. >>=20 >> I thought ports used a different mechanism and defined special magic = so =3D >> the >> src tree mechanism was disabled. >=20 > The ports makefile is much more elaborate but I believe achieves = similar > results. The name conflict could probably be worked around if > bsd.options.mk is considered the best name choice. I opted for a different name to not conflict. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 11:15:59 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D05DB7B4 for ; Wed, 16 Apr 2014 11:15:59 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 91900111E for ; Wed, 16 Apr 2014 11:15:59 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id B18846303; Wed, 16 Apr 2014 11:15:58 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 42A693001F; Wed, 16 Apr 2014 13:15:56 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Warner Losh Subject: Re: EISA in GENERIC References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> Date: Wed, 16 Apr 2014 13:15:56 +0200 In-Reply-To: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> (Warner Losh's message of "Tue, 15 Apr 2014 14:50:14 -0600") Message-ID: <86bnw15xub.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 11:15:59 -0000 Warner Losh writes: > The time has come to trim EISA from the generic i386 kernel. Can we also remove ISA network adapters? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 15:58:07 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1642249B for ; Wed, 16 Apr 2014 15:58:07 +0000 (UTC) Received: from mail-ie0-f181.google.com (mail-ie0-f181.google.com [209.85.223.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4D6A1FD0 for ; Wed, 16 Apr 2014 15:58:06 +0000 (UTC) Received: by mail-ie0-f181.google.com with SMTP id tp5so10735486ieb.12 for ; Wed, 16 Apr 2014 08:58:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=jHpUZJJlCfoi3dylubvNAXA7xbxxnvlUo54l1/w9/44=; b=JGd1mxO3KvVSVny+BLIvnj3Irn+YpnE3724xgRQpBsVcOQPtLUdGkVQcXCh0TYraH+ B3xbefdKPzY3Jw90WK0oXVRNQRm+RSoimlCO6XNxN720Kl1SkfGG7ZGnxcm3riW5T/xp S1cL47KkMm5wUFK65hOUaof6EEaSQ7/uCRY7PTxA+AMqZi5r2/VgUG7mQf/AJGyYibkG EvuD5IgkYpdO/d1y6+m4xpCKK2HzfIN2fvZ+zg36h6/0VHAO8Q0oG+7iLtp/ToxEQEyg mTDyzYSOlLs3yHmnI8BWOgTv9lVins45Q6mbSm815zBHMMOkDBF0bHYJtAAFYLQeT39b ad4Q== X-Gm-Message-State: ALoCoQnnQUcuXfCCx1hPl6DVenyJx3S4H8LyS9g2wi5QsJMp8KROMC/nFF3ckvhbxT5Z80yvBsJD X-Received: by 10.43.151.7 with SMTP id kq7mr2514114icc.78.1397662374627; Wed, 16 Apr 2014 08:32:54 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id s8sm46808477ige.4.2014.04.16.08.32.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Apr 2014 08:32:54 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: EISA in GENERIC From: Warner Losh In-Reply-To: <86bnw15xub.fsf@nine.des.no> Date: Wed, 16 Apr 2014 09:32:52 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <1196BBD9-E5A1-490E-A81A-C2E90B574D3B@bsdimp.com> References: <957D23B4-264C-4AAB-945C-F82B9877FAC9@bsdimp.com> <86bnw15xub.fsf@nine.des.no> To: =?windows-1252?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1874) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 15:58:07 -0000 On Apr 16, 2014, at 5:15 AM, Dag-Erling Sm=F8rgrav wrote: > Warner Losh writes: >> The time has come to trim EISA from the generic i386 kernel. >=20 > Can we also remove ISA network adapters? That=92s a separate issue, along with ISA SCSI adapters. One that I=92d = like to talk about in a separate thread. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 16:36:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C26B54A1 for ; Wed, 16 Apr 2014 16:36:08 +0000 (UTC) Received: from mail.modirum.com (mail.modirum.com [31.185.27.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82C231465 for ; Wed, 16 Apr 2014 16:36:07 +0000 (UTC) Received: from 93-153-61-159.tmcz.cz ([93.153.61.159] helo=desktop.reztek) by mail.modirum.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WaSof-000IJN-L4 for freebsd-arch@freebsd.org; Wed, 16 Apr 2014 16:36:05 +0000 From: Matthew Rezny To: freebsd-arch@freebsd.org Subject: Re: EISA in GENERIC Date: Wed, 16 Apr 2014 18:36:02 +0200 Message-ID: <7524617.WO5TPN2b6E@desktop.reztek> Organization: RezTek, s.r.o. User-Agent: KMail/4.12.3 (FreeBSD/10.0-STABLE; KDE/4.12.3; amd64; ; ) In-Reply-To: <86bnw15xub.fsf@nine.des.no> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-SA-Authenticated: Yes X-SA-Exim-Connect-IP: 93.153.61.159 X-SA-Exim-Mail-From: matthew@reztek.cz X-SA-Exim-Scanned: No (on mail.modirum.com); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 16:36:08 -0000 > Warner Losh writes: > > The time has come to trim EISA from the generic i386 kernel. >=20 > Can we also remove ISA network adapters? >=20 > DES > -- > Dag-Erling Sm=F8rgrav - des at des.no Remove from GENERIC? Go ahead. Remove from src tree? No. I see no reason to need ISA NICs or storage support in GENERIC on amd64= and=20 little reason have it in GENERIC for i386. General ISA support needs to= remain=20 for sio/uart and lpt. Anyone using ISA network, storage, sound, etc is=20= probably ok building a custom kernel. My fist thought when I saw this thread was "I hope someone doesn't ask = to=20 remove ISA support". From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 16:56:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46C9CAA0 for ; Wed, 16 Apr 2014 16:56:52 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11598167E for ; Wed, 16 Apr 2014 16:56:51 +0000 (UTC) Received: by mail-ig0-f175.google.com with SMTP id ur14so1264267igb.14 for ; Wed, 16 Apr 2014 09:56:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=oh3aeOGiccJ/4SSl1DKSDjYqHLelv2F7WhQVSONqqAI=; b=CKrXqV6WzNB3Ffaw53nuJ+WRyBXZiJ2801f5LiDgEXw7bR+W2MVybi2xRBtqrcJvWp uiT2prIKLuIj7XLXeWQQIotaceB6b2i8tvKfu1+nQ/Myd8VPcyyv7wwAD3g5XtyKZNsD IBWRtMjgUHLJZlVBxGVbaRRzk1+R38rej0ggGBll7DpCBrAbdiK/KLRsPSraGWReJyEX DjSDXcRS7WiSgPDk8iyLjk5h27r+5SlBt5R8pczDp9ysvVrna35OXY1xg/RFvXKWoWMF IcTzc7AG+MS6BMYdqv6VzbyHj+EP/8ORGg9lh+zMK1zIUQJ5341ez3Eii+VJ0p0JcWI0 VPSg== X-Gm-Message-State: ALoCoQnCuXFNk9RjNrfYDOqaUCG9xvrDxZvv77Av8scPqxYVdYuxn/p7m+9uPNM3AdW/l7t55bsU X-Received: by 10.50.22.210 with SMTP id g18mr6069076igf.19.1397667405053; Wed, 16 Apr 2014 09:56:45 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id s1sm45747879igr.14.2014.04.16.09.56.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Apr 2014 09:56:44 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: EISA in GENERIC From: Warner Losh In-Reply-To: <7524617.WO5TPN2b6E@desktop.reztek> Date: Wed, 16 Apr 2014 10:56:42 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <421F815B-906C-4CD0-8D5B-349FBD5B6607@bsdimp.com> References: <7524617.WO5TPN2b6E@desktop.reztek> To: Matthew Rezny X-Mailer: Apple Mail (2.1874) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 16:56:52 -0000 On Apr 16, 2014, at 10:36 AM, Matthew Rezny wrote: >> Warner Losh writes: >>> The time has come to trim EISA from the generic i386 kernel. >>=20 >> Can we also remove ISA network adapters? >>=20 >> DES >> -- >> Dag-Erling Sm=F8rgrav - des at des.no >=20 > Remove from GENERIC? Go ahead. > Remove from src tree? No. >=20 > I see no reason to need ISA NICs or storage support in GENERIC on = amd64 and=20 > little reason have it in GENERIC for i386. General ISA support needs = to remain=20 > for sio/uart and lpt. Anyone using ISA network, storage, sound, etc is=20= > probably ok building a custom kernel. >=20 > My fist thought when I saw this thread was "I hope someone doesn't ask = to=20 > remove ISA support". ISA support simply cannot be removed from the tree, full stop. It is our = generic bus on x86, and many things that don=92t have a better attachment are = attached here. While some movement towards ACPI has happened here, we haven=92t done a good job of cutting the cord here because things work well enough for people to focus on other things. Most, but not all, ISA network, storage and sound drivers have loadable modules, so could easily be loaded at boot should someone wish to run = GENERIC and still have these devices supported. So there=92s a back-door to = getting a system recovered from modern boot media (assuming, of course, that these systems could boot off modern boot media). In addition, many of the ISA drivers are also PC Card drivers. I still = get many questions about PC Card devices and drivers. At least with PC Card, though, we could automatically load the right .ko 95% of the time. But = this gets us back to the no generic bus-level matching code infrastructure = that could be mined to get this data. USB does this with a set of scripts = that are kludgy in the sense they are only for USB. PCI could do it, but some drivers would need a lot of work. USB, PC Card and ISA PNP could do it, with some special marking of the arrays. Old-school ISA, where we have to have identify routines or hints can never be automatically loaded in response to plug and play data from the bus (but perhaps could be loaded based on the presence of hints). So it is for all these reasons I don=92t want to talk about ISA at this = time. It is a much messier situation, with ugly tendrils into weird places. EISA, on the other hand, is very well contained and very easy to turn on/off once the right build knobs are in place (since it already was close to being trivial to get rid of). Warner= From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 17:15:56 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C3F625D for ; Wed, 16 Apr 2014 17:15:56 +0000 (UTC) Received: from mail.modirum.com (mail.modirum.com [31.185.27.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DDB4E1867 for ; Wed, 16 Apr 2014 17:15:55 +0000 (UTC) Received: from 93-153-61-159.tmcz.cz ([93.153.61.159] helo=desktop.reztek) by mail.modirum.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1WaTRA-0009Xo-Q1 for freebsd-arch@freebsd.org; Wed, 16 Apr 2014 17:15:52 +0000 From: Matthew Rezny To: freebsd-arch@freebsd.org Subject: Re: EISA in GENERIC Date: Wed, 16 Apr 2014 19:15:44 +0200 Message-ID: <5384422.Lmm1UmY4VK@desktop.reztek> Organization: RezTek, s.r.o. User-Agent: KMail/4.12.3 (FreeBSD/10.0-STABLE; KDE/4.12.3; amd64; ; ) In-Reply-To: <421F815B-906C-4CD0-8D5B-349FBD5B6607@bsdimp.com> References: <7524617.WO5TPN2b6E@desktop.reztek> <421F815B-906C-4CD0-8D5B-349FBD5B6607@bsdimp.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-SA-Authenticated: Yes X-SA-Exim-Connect-IP: 93.153.61.159 X-SA-Exim-Mail-From: matthew@reztek.cz X-SA-Exim-Scanned: No (on mail.modirum.com); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 17:15:56 -0000 On Wednesday 16 April 2014 10:56:42 you wrote: > On Apr 16, 2014, at 10:36 AM, Matthew Rezny wrote= : > >> Warner Losh writes: > >>> The time has come to trim EISA from the generic i386 kernel. > >>=20 > >> Can we also remove ISA network adapters? > >>=20 > >> DES > >> -- > >> Dag-Erling Sm=C3=B8rgrav - des at des.no > >=20 > > Remove from GENERIC? Go ahead. > > Remove from src tree? No. > >=20 > > I see no reason to need ISA NICs or storage support in GENERIC on a= md64 > > and > > little reason have it in GENERIC for i386. General ISA support need= s to > > remain for sio/uart and lpt. Anyone using ISA network, storage, sou= nd, > > etc is probably ok building a custom kernel. > >=20 > > My fist thought when I saw this thread was "I hope someone doesn't = ask to > > remove ISA support". >=20 > ISA support simply cannot be removed from the tree, full stop. It is = our > generic bus on x86, and many things that don=E2=80=99t have a better = attachment are > attached here. While some movement towards ACPI has happened here, we= > haven=E2=80=99t done a good job of cutting the cord here because thin= gs work well > enough for people to focus on other things. >=20 > Most, but not all, ISA network, storage and sound drivers have loadab= le > modules, so could easily be loaded at boot should someone wish to run= > GENERIC and still have these devices supported. So there=E2=80=99s a = back-door to > getting a system recovered from modern boot media (assuming, of cours= e, > that these systems could boot off modern boot media). >=20 > In addition, many of the ISA drivers are also PC Card drivers. I stil= l get > many questions about PC Card devices and drivers. At least with PC Ca= rd, > though, we could automatically load the right .ko 95% of the time. Bu= t this > gets us back to the no generic bus-level matching code infrastructure= that > could be mined to get this data. USB does this with a set of scripts = that > are kludgy in the sense they are only for USB. PCI could do it, but s= ome > drivers would need a lot of work. USB, PC Card and ISA PNP could do i= t, > with some special marking of the arrays. Old-school ISA, where we hav= e > to have identify routines or hints can never be automatically loaded = in > response to plug and play data from the bus (but perhaps could be loa= ded > based on the presence of hints). >=20 > So it is for all these reasons I don=E2=80=99t want to talk about ISA= at this time. > It is a much messier situation, with ugly tendrils into weird places.= EISA, > on the other hand, is very well contained and very easy to turn on/of= f once > the right build knobs are in place (since it already was close to bei= ng > trivial to get rid of). >=20 > Warner I guess I was a little unclear. What I meant was I think it is fine if = we we=20 remove all ISA network and storage drivers from GENERIC, so long as the= y=20 remain in tree so they can be loaded as modules or compiled into custom= =20 kernels. I'm well aware that ISA provides essential functionality on i386, so we= can't=20 get rid of ISA bus support. I consider that topic off the table. I perh= aps=20 caused confusion in the last sentence. What I meant there was "I hope s= omeone=20 doesn't ask to remove all ISA device support from GENERIC", as in the s= erial=20 ports and floppy controller specifically. From owner-freebsd-arch@FreeBSD.ORG Wed Apr 16 17:38:15 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1663A939 for ; Wed, 16 Apr 2014 17:38:15 +0000 (UTC) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 55BCC1AB9 for ; Wed, 16 Apr 2014 17:38:14 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.7/8.14.7) with ESMTP id s3GHbuFi024627 for ; Wed, 16 Apr 2014 12:37:56 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.7/8.14.7/Submit) id s3GHbuhQ024626 for arch@freebsd.org; Wed, 16 Apr 2014 12:37:56 -0500 (CDT) (envelope-from brooks) Date: Wed, 16 Apr 2014 12:37:56 -0500 From: Brooks Davis To: arch@freebsd.org Subject: libncurses XOR libncursesw in base Message-ID: <20140416173756.GA24311@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/9DWx/yDrRhgMJTb" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2014 17:38:15 -0000 --/9DWx/yDrRhgMJTb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline What would it take to switch all the remaining users of libncurses to libncursesw? I'm not suggesting removing libncurses, just avoiding it in base. The relatively recent switch to require libncursesw for vi is a minor pain on small embedded systems. I had attempted to work around this with WITHOUT_NCURSESW since vi already had an option for /rescue support but that's been broken in two separate ways in the couple months since I merged it a couple months ago. I don't understand the issues with switching the libraries well enough to just do it, but would be happy to do the work if they are explained. -- Brooks --/9DWx/yDrRhgMJTb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iKYEARECAGYFAlNOv/RfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl bnBncC5maWZ0aGhvcnNlbWFuLm5ldDY1NUQ1MTlDMjZBNzgyRTcyNTI5OUJGMDVE OEU4QkU5RjIzODFBRDQACgkQXY6L6fI4GtSO7QCfZoRSRFzBkx9D38dsDR7LU+cp NHkAoM5POLr0Z+KtVSwEWOnP99CgVRlm =PzSW -----END PGP SIGNATURE----- --/9DWx/yDrRhgMJTb-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 06:34:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46DCECA9; Thu, 17 Apr 2014 06:34:55 +0000 (UTC) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe001.messaging.microsoft.com [216.32.180.11]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E670D1BCC; Thu, 17 Apr 2014 06:34:54 +0000 (UTC) Received: from mail229-va3-R.bigfish.com (10.7.14.226) by VA3EHSOBE002.bigfish.com (10.7.40.22) with Microsoft SMTP Server id 14.1.225.22; Thu, 17 Apr 2014 06:34:10 +0000 Received: from mail229-va3 (localhost [127.0.0.1]) by mail229-va3-R.bigfish.com (Postfix) with ESMTP id 279A1B6041F; Thu, 17 Apr 2014 06:34:10 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.10; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 3 X-BigFish: VPS3(zz1dbaI1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz17326ah8275dh1de097h186068h74efjz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail229-va3: transitioning domain of juniper.net does not designate 66.129.239.10 as permitted sender) client-ip=66.129.239.10; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail229-va3 (localhost.localdomain [127.0.0.1]) by mail229-va3 (MessageSwitch) id 1397716447695681_27985; Thu, 17 Apr 2014 06:34:07 +0000 (UTC) Received: from VA3EHSMHS019.bigfish.com (unknown [10.7.14.245]) by mail229-va3.bigfish.com (Postfix) with ESMTP id A6D2134007B; Thu, 17 Apr 2014 06:34:07 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.239.10) by VA3EHSMHS019.bigfish.com (10.7.99.29) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 17 Apr 2014 06:34:07 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Wed, 16 Apr 2014 23:34:49 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s3H6YbV60659; Wed, 16 Apr 2014 23:34:43 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 3BBF85809E; Wed, 16 Apr 2014 23:34:37 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> Comments: In-reply-to: Warner Losh message dated "Tue, 15 Apr 2014 20:27:49 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Wed, 16 Apr 2014 23:34:37 -0700 Message-ID: <20140417063437.3BBF85809E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 06:34:55 -0000 >> I'm obviously biased, but I think contrib/bmake/mk/options.mk is close >> to what is desired - just the mechanism with no policy. > >I=92ve created a bsd.srcopt.mk and adjusted bsd.own.mk in a sample patch, s= >ince >it sounds like we=92re headed in the right direction. Please take a look at= > it and >let me know what you think. Thanks >http://people.freebsd.org/~imp/patch-queue/bsd.srcopt.mk > >contains the patch and commit message (if I did the queue management right) >that I=92d like to proffer. I could be miss-reading this, but it looks like you moved the list of options to bsd.srcopt.mk ? That's not what I was suggesting, though I can see how it might be useful. I was specifically thinking of a makefile that does not define/know any options at all. You set a list of options and include it and it does the WITH[OUT]_* -> MK_* dance. Eg. contrib/bmake/mk/options.mk consumes OPTIONS_DEFAULT_* but does not set even a default list. If you include it with no options list, it does nothing - pure mechanism. Now it might be very useful to extract the list of src options from bsd.own.mk to something safer to include anytime. But it is still (I think) useful to separate the options processing from their definition. Eg. something like: bsd.own.mk .include defines a bunch of options .include sets MK_* based on WITH[OUT]_* for the list of options provided or per my comments below about installing, perhaps the list of options etc might be better in src.options.mk (ie something we don't install? I just thought of that so take with grain of salt) Regardless, it might be handy to see the full files, to get a clearer picture. >earlier in this thread. I=92m not tied to what I=92ve posted here, but I wo= >uld >like to reach resolution so I can fix some other things in the build without >much delay. Sure. > >>> Where would the src build pick this up from if =3D >>> it isn=3D92t installed? >> = > >> src/share/mk should work fine. > >OK. I haven=92t added it to the Makefile so it gets installed, and things >seem to build, but that=92s not a viable way to commit things. Please comme= >nt >on what I=92ve done and what you think the right way to proceed might be. >The easy path is, of course, just installing it :) Sorry, which file are you talking about? FWIW I think all bsd.*.mk should get installed. but I think it perfectly reasonable to declare that anything matching the pattern local.* or src.* doesn't get installed. Hopefully that shouldn't surprise anyone either. To restate that slightly differently, you can think of these things as a hierarchy: bsd.* mechanism for building stuff (not just /usr/src) src.* stuff just for building /usr/src local.* local tweaks to all the above >> Not necessarily. The stuff in bsd.own.mk like: >> = > >> .if ${MK_TOOLCHAIN} =3D=3D "no" >> MK_BINUTILS:=3D no >> MK_CLANG:=3D no >> MK_GCC:=3D no >> MK_GDB:=3D no >> .endif >> = > >> doesn't need to change, and for more elaborate stuff, simply being able >> to include *options.mk more than once may help quite a bit. > >That might be hard, since MK_FOO being set from the first time won=92t Unless MK_FOO is set on make's commandline, you can override it any time you like - whether that makes sense or whether it gives people headaches is another matter. Most of the current options seem to be more about whether certain features/apps etc should be built. Changing the value of such options once set doesn't make a lot of sense. eg. no point building libfoo without support for goop, and then building food with a need for it. The sorts of knobs more likely to be useful to tweak as we go, are those that affect how we do building (cf. configuring src), so MK_CTF, MK_META_MODE, and probably many others we could come up with if the mechanism were sufficiently flexible and easy to grok. >> No, but could do so pretty quickly. Sorry I lied... ;-) >I opted for a different name to not conflict. Probably best. Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 08:46:13 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83CB2B32; Thu, 17 Apr 2014 08:46:13 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 561291CE9; Thu, 17 Apr 2014 08:46:12 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s3H8jws8023647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 17 Apr 2014 01:46:03 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <534F94BA.5040001@freebsd.org> Date: Thu, 17 Apr 2014 16:45:46 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Warner Losh , Brooks Davis Subject: Re: time to svn rm lib/libkse? References: <20140415224134.GA17146@lor.one-eyed-alien.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 08:46:13 -0000 On 4/16/14, 7:11 AM, Warner Losh wrote: > On Apr 15, 2014, at 4:41 PM, Brooks Davis wrote: > >> I noticed to chance that we still include the source for libkse in the >> tree. It was unhooked from the build and kernel support removed in >> 2008 so I think it's due to be removed from HEAD and maybe 8, 9, and 10. > /* XXXKSE What to do */ > > I think that it is time to remove it. +1 > > Warner > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 16:31:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 70C228EF for ; Thu, 17 Apr 2014 16:31:02 +0000 (UTC) Received: from mail-pa0-f52.google.com (mail-pa0-f52.google.com [209.85.220.52]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C6DB14F7 for ; Thu, 17 Apr 2014 16:31:01 +0000 (UTC) Received: by mail-pa0-f52.google.com with SMTP id rd3so552429pab.11 for ; Thu, 17 Apr 2014 09:31:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=ASz+Kg+OE0uN39J7w5ST4jJ7DzlmFMriSQ2eGqpEfkU=; b=UT339FFFlHW9WpQensabXNps+J4MyIzKGgx8NT0e0xmrGxDWVR1AKpYwsmv7dZdcEI njqkGex6Rj4jVhpkbQK38sULD2pas7CVUsizvfAq8ZvoUBVaO0sFM7Y474crZnO7vkZy 0cq9lbas/HrOD9r5HkcNgbRK1msNV+4S51wD6Lz0wl8o2c7YRN9OFIJBSYyDumFPpqSj GEMPv49P5/hdRwnVGJPQw2QCxDYQLx3ssNuAis50qA7cHxqDnI6sTdGEMrKVlFXRCRd+ R1z8KKthQYM8E0xzOLhbZoIItJetGLDEc4s3kWQN5W/vXxtjbDV6nM0ltuFy3DtuX/C5 qRag== X-Gm-Message-State: ALoCoQmsIClqetWmx9ETzzFX6RwFlp8cxaQVESrxscjfbSGE24aD5rivHaTfHNDBC52I3sLy+H4l X-Received: by 10.68.226.197 with SMTP id ru5mr16615273pbc.77.1397752260994; Thu, 17 Apr 2014 09:31:00 -0700 (PDT) Received: from lgmac-ckapadia.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ff4sm128677125pad.24.2014.04.17.09.30.59 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 09:31:00 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140417063437.3BBF85809E@chaos.jnpr.net> Date: Thu, 17 Apr 2014 10:30:58 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 16:31:02 -0000 On Apr 17, 2014, at 12:34 AM, Simon J. Gerraty wrote: >=20 >>> I'm obviously biased, but I think contrib/bmake/mk/options.mk is = close >>> to what is desired - just the mechanism with no policy. >>=20 >> I=3D92ve created a bsd.srcopt.mk and adjusted bsd.own.mk in a sample = patch, s=3D >> ince >> it sounds like we=3D92re headed in the right direction. Please take a = look at=3D >> it and >> let me know what you think. >=20 > Thanks >=20 >> http://people.freebsd.org/~imp/patch-queue/bsd.srcopt.mk >>=20 >> contains the patch and commit message (if I did the queue management = right) >> that I=3D92d like to proffer. >=20 > I could be miss-reading this, but it looks like you moved the list of > options to bsd.srcopt.mk ?=20 > That's not what I was suggesting, though I can see how it might be = useful. This was step one: separate out the options processing from the = bsd.own.mk stuff. Having a few lines that are generic would be nice to include. I=92d like = to go ahead and commit step 1, even if we refine things a bit later to keep the change = sets manageable and under control=85 Is that reasonable? It will help other areas and we = don=92t need to do much more than settle on filenames. :) > I was specifically thinking of a makefile that does not define/know = any > options at all. You set a list of options and include it and > it does the WITH[OUT]_* -> MK_* dance. > Eg. contrib/bmake/mk/options.mk consumes OPTIONS_DEFAULT_* but does = not > set even a default list. If you include it with no options list, it > does nothing - pure mechanism. Yea, that was the notion of the next step. > Now it might be very useful to extract the list of src options from > bsd.own.mk to something safer to include anytime. > But it is still (I think) useful to separate the options processing > from their definition. Yea, this is just the first step. > Eg. something like: >=20 > bsd.own.mk > .include > defines a bunch of options=20 > .include > sets MK_* based on WITH[OUT]_* for the list of > options provided >=20 > or per my comments below about installing, perhaps the list of options > etc might be better in src.options.mk (ie something we don't install? > I just thought of that so take with grain of salt) Yea, I have some concerns about going that far. See below. > Regardless, it might be handy to see the full files, to get a clearer > picture.=20 >=20 >> earlier in this thread. I=3D92m not tied to what I=3D92ve posted = here, but I wo=3D >> uld >> like to reach resolution so I can fix some other things in the build = without >> much delay. >=20 > Sure. >=20 >>=20 >>>> Where would the src build pick this up from if =3D3D >>>> it isn=3D3D92t installed? >>> =3D >>=20 >>> src/share/mk should work fine. >>=20 >> OK. I haven=3D92t added it to the Makefile so it gets installed, and = things >> seem to build, but that=3D92s not a viable way to commit things. = Please comme=3D >> nt >> on what I=3D92ve done and what you think the right way to proceed = might be. >> The easy path is, of course, just installing it :) >=20 > Sorry, which file are you talking about?=20 My bsd.srcopts.mk was what I was talking about here needing to be added = to share/mk/Makefile. > FWIW I think all bsd.*.mk should get installed. > but I think it perfectly reasonable to declare that anything matching = the > pattern local.* or src.* doesn't get installed. > Hopefully that shouldn't surprise anyone either. IF we can achieve that separation, then great. > To restate that slightly differently, you can think of these things as = a > hierarchy: >=20 > bsd.* mechanism for building stuff (not just /usr/src) > src.* stuff just for building /usr/src > local.* local tweaks to all the above Yes, as long as the MK_FOO variables that are really src building stuff = don=92t infect the bsd.foo.mk files, this could work out. Keeping the infection = rate down might be hard, but not impossible to manage. Would we do a wholesale change of bsd.foo.mk to src.foo.mk for all the = Makefiles in the tree if we went down this path? Or would the src.XXX stuff just = be for the orchestration we do between the different directories and such and we=92d = continue to build them the way we always have? >>> Not necessarily. The stuff in bsd.own.mk like: >>> =3D >>=20 >>> .if ${MK_TOOLCHAIN} =3D3D=3D3D "no" >>> MK_BINUTILS:=3D3D no >>> MK_CLANG:=3D3D no >>> MK_GCC:=3D3D no >>> MK_GDB:=3D3D no >>> .endif >>> =3D >>=20 >>> doesn't need to change, and for more elaborate stuff, simply being = able >>> to include *options.mk more than once may help quite a bit. >>=20 >> That might be hard, since MK_FOO being set from the first time = won=3D92t >=20 > Unless MK_FOO is set on make's commandline, you can override it any = time > you like - whether that makes sense or whether it gives people = headaches > is another matter. >=20 > Most of the current options seem to be more about whether certain > features/apps etc should be built. Changing the value of such = options=20 > once set doesn't make a lot of sense. eg. no point building libfoo > without support for goop, and then building food with a need for it. Yea. That does make a certain amount of sense. > The sorts of knobs more likely to be useful to tweak as we go, are = those > that affect how we do building (cf. configuring src), so MK_CTF, > MK_META_MODE, and probably many others we could come up with if the > mechanism were sufficiently flexible and easy to grok. Yea, I=92m not too hung up on the edge cases, but want to make sure that = we know the limitations going into this. Warner >>> No, but could do so pretty quickly. >=20 > Sorry I lied... ;-) >=20 >> I opted for a different name to not conflict. >=20 > Probably best. >=20 > Thanks > --sjg >=20 From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 18:27:19 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE45EEAA; Thu, 17 Apr 2014 18:27:19 +0000 (UTC) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe001.messaging.microsoft.com [216.32.180.11]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 996BF122D; Thu, 17 Apr 2014 18:27:19 +0000 (UTC) Received: from mail73-va3-R.bigfish.com (10.7.14.230) by VA3EHSOBE010.bigfish.com (10.7.40.12) with Microsoft SMTP Server id 14.1.225.22; Thu, 17 Apr 2014 18:11:24 +0000 Received: from mail73-va3 (localhost [127.0.0.1]) by mail73-va3-R.bigfish.com (Postfix) with ESMTP id 423731C033B; Thu, 17 Apr 2014 18:11:24 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.10; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail73-va3: transitioning domain of juniper.net does not designate 66.129.239.10 as permitted sender) client-ip=66.129.239.10; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail73-va3 (localhost.localdomain [127.0.0.1]) by mail73-va3 (MessageSwitch) id 1397758282699575_6011; Thu, 17 Apr 2014 18:11:22 +0000 (UTC) Received: from VA3EHSMHS025.bigfish.com (unknown [10.7.14.237]) by mail73-va3.bigfish.com (Postfix) with ESMTP id A24654E0091; Thu, 17 Apr 2014 18:11:22 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.239.10) by VA3EHSMHS025.bigfish.com (10.7.99.35) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 17 Apr 2014 18:11:22 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 17 Apr 2014 11:12:04 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s3HIC2V33638; Thu, 17 Apr 2014 11:12:02 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 0EFEA5809E; Thu, 17 Apr 2014 11:11:59 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Thu, 17 Apr 2014 10:30:58 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 17 Apr 2014 11:11:59 -0700 Message-ID: <20140417181159.0EFEA5809E@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 18:27:20 -0000 On Thu, 17 Apr 2014 10:30:58 -0600, Warner Losh writes: >This was step one: separate out the options processing from the = >bsd.own.mk stuff. >Having a few lines that are generic would be nice to include. I=92d like = >to go ahead and >commit step 1, even if we refine things a bit later to keep the change = >sets manageable >and under control=85 Is that reasonable? It will help other areas and we = >don=92t need to >do much more than settle on filenames. :) Sure. Naming stuff is hard, and warrants early attention. On that front it occurred to me that (since it sets MK_*) bsd.mkopts.mk might be a good name for the pure mechanism makefile. So in the 3 level setup I mentioned you'd have: bsd.own.mk -> bsd.srcopts.mk -> bsd.mkopts.mk If that sounds ok, I think we can proceed to next step. >> Sorry, which file are you talking about?=20 > >My bsd.srcopts.mk was what I was talking about here needing to be added = >to share/mk/Makefile. Ah right. This is why I think separating the mechanism is good. The mechanism should definitely be installed - because it is very handy (assuming some of the restrictions are removed). The /usr/src "policy" is another matter (possibly bikeshed material ;-) >> FWIW I think all bsd.*.mk should get installed. >> but I think it perfectly reasonable to declare that anything matching = >the >> pattern local.* or src.* doesn't get installed. >> Hopefully that shouldn't surprise anyone either. > >IF we can achieve that separation, then great. Yep. >> To restate that slightly differently, you can think of these things as = >a >> hierarchy: >>=20 >> bsd.* mechanism for building stuff (not just /usr/src) >> src.* stuff just for building /usr/src >> local.* local tweaks to all the above > >Yes, as long as the MK_FOO variables that are really src building stuff = >don=92t >infect the bsd.foo.mk files, this could work out. Exactly. MK_* which are meaningful to bsd.*.mk need an option list in bsd.*.mk - bsd.own.mk being perhaps a natural location? For MK_* which are only actioned in makefiles like *bin/Makefile they can be configured via a src.* file since they do not need to be installed. By providing guidance and "obvious" places to list the two classes of knobs we can limit the infection risk. >Would we do a wholesale change of bsd.foo.mk to src.foo.mk for all the = >Makefiles I'm not sure that would be necessary. Having to support lots of active branches makes me leery of gratuitous changes ;-) If we had sys.mk do .-include "src.sys.mk" .-include "local.sys.mk" that would be sufficient to enable all sorts of cool stuff. Similarly, bsd.init.mk doing .-include "src.init.mk" .-include "local.init.mk" would allow for customization of things to be done after Makefile has been read. The above would be sufficient to allow src.* to influence bin/Makefile and lib/Makefile etc without the need to touch them. Other customization hooks would be cool too, bsd.sys.mk is a somewhat unfortunate name, but it would be handy to have another point for including {src,local}.*.mk at the end of the normal lib,prog,* mk files. Note that even though we don't install src.*.mk or local.*.mk, those hook points are still very useful to anyone building their own stuff with bsd.*.mk, since now they can customize their own builds without the need to hack bsd.*.mk Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 20:29:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31185817 for ; Thu, 17 Apr 2014 20:29:39 +0000 (UTC) Received: from mail-pa0-f43.google.com (mail-pa0-f43.google.com [209.85.220.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F05721FDB for ; Thu, 17 Apr 2014 20:29:38 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id bj1so746419pad.30 for ; Thu, 17 Apr 2014 13:29:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=VWA65BqCNHDwKNuzoNCCDQ8v/2QKxeYA1ilLEXug7i8=; b=cW5eK7nodJX6QeF+UXp+otiykkRcAey4RbNJpiKUCxJEu/SN4L7KaeV+6cygtO1dE8 e+AzYPunfo4+CTliFNBpsbwOri64NFTqL9wXj6Kmi5EU5rDuMCVbC9Oxo5cu4bmqYXC4 aDJbitF4m+JYcyswmPgECxAwCzmBj3imDMOEjYesd4ri1B3psUBIxhJSFA/vyxRsK8bt R8eJ6YgK770XFTSvfxEBPdW0geIf3RFQ3mmcQp9acJvaNyHTtpOYZ2AzPqcpKdEVK2ra 9uIR6C4N34azVu4+Z1aySzCm1xD6ti5N6F7j4kb6Rvwma3L419pZdTGimtfDwgikBMsC wIQg== X-Gm-Message-State: ALoCoQnSlZEXLu5Z0kaHblw/gewOOlj0vqIgnPfVakat3FQE6sVYnkrTzMtZ2Lq2S062yn2O9+W+ X-Received: by 10.68.143.231 with SMTP id sh7mr17963252pbb.7.1397766572259; Thu, 17 Apr 2014 13:29:32 -0700 (PDT) Received: from lgmac-ckapadia.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id yo9sm131082154pab.16.2014.04.17.13.29.30 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 13:29:31 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140417181159.0EFEA5809E@chaos.jnpr.net> Date: Thu, 17 Apr 2014 14:29:28 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> <20140417181159.0EFEA5809E@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 20:29:39 -0000 On Apr 17, 2014, at 12:11 PM, Simon J. Gerraty wrote: >=20 > On Thu, 17 Apr 2014 10:30:58 -0600, Warner Losh writes: >> This was step one: separate out the options processing from the =3D >> bsd.own.mk stuff. >> Having a few lines that are generic would be nice to include. I=3D92d = like =3D >> to go ahead and >> commit step 1, even if we refine things a bit later to keep the = change =3D >> sets manageable >> and under control=3D85 Is that reasonable? It will help other areas = and we =3D >> don=3D92t need to >> do much more than settle on filenames. :) >=20 > Sure. Naming stuff is hard, and warrants early attention. > On that front it occurred to me that (since it sets MK_*) > bsd.mkopts.mk might be a good name for the pure mechanism makefile. > So in the 3 level setup I mentioned you'd have: > bsd.own.mk -> bsd.srcopts.mk -> bsd.mkopts.mk Since we=92re planning a src.opts.mk at some point (to control /usr/src = builds), the bsd.srcopts.mk name seems to have a short shelf life. Maybe just = bsd.opts.mk? > If that sounds ok, I think we can proceed to next step. I=92d like to do the above tweak. There=92s on conflict with ports and = the name seems better. >>> Sorry, which file are you talking about?=3D20 >>=20 >> My bsd.srcopts.mk was what I was talking about here needing to be = added =3D >> to share/mk/Makefile. >=20 > Ah right. This is why I think separating the mechanism is good. > The mechanism should definitely be installed - because it is very = handy > (assuming some of the restrictions are removed). >=20 > The /usr/src "policy" is another matter (possibly bikeshed material = ;-) Yea, I think both of the above files, regardless of name, should be = installed. They just allow things to be configured, without providing a place to = configure them (well, without a place that=92s different than today=92s policy of = /etc/src.conf unless overridden). >>> FWIW I think all bsd.*.mk should get installed. >>> but I think it perfectly reasonable to declare that anything = matching =3D >> the >>> pattern local.* or src.* doesn't get installed. >>> Hopefully that shouldn't surprise anyone either. >>=20 >> IF we can achieve that separation, then great. >=20 > Yep. I=92ll start down that path once we have the above stuff done. >>> To restate that slightly differently, you can think of these things = as =3D >> a >>> hierarchy: >>> =3D20 >>> bsd.* mechanism for building stuff (not just /usr/src) >>> src.* stuff just for building /usr/src >>> local.* local tweaks to all the above >>=20 >> Yes, as long as the MK_FOO variables that are really src building = stuff =3D >> don=3D92t >> infect the bsd.foo.mk files, this could work out.=20 >=20 > Exactly. =20 >=20 > MK_* which are meaningful to bsd.*.mk need an option list in > bsd.*.mk - bsd.own.mk being perhaps a natural location? Yes. > For MK_* which are only actioned in makefiles like *bin/Makefile > they can be configured via a src.* file since they do not need to be > installed. I think so, yes. > By providing guidance and "obvious" places to list the two classes of > knobs we can limit the infection risk. Yea=85 There may be some contamination in user Makefiles today, but that should be minimal and will be easy to eradicate. >> Would we do a wholesale change of bsd.foo.mk to src.foo.mk for all = the =3D >> Makefiles >=20 > I'm not sure that would be necessary. > Having to support lots of active branches makes me leery of gratuitous > changes ;-) Yea. I=92d hoped you=92d say that. We build individual = programs/libraries with the bsd.*.mk files only, in general, although there may be some exceptions that are = carefully called out and those exceptions would only be to the extent of including the = src.opts.mk files. > If we had sys.mk do >=20 > .-include "src.sys.mk" > .-include "local.sys.mk" >=20 > that would be sufficient to enable all sorts of cool stuff. In non-posix mode, sure. This would be the natural place for the MK option setting. While it is early, it isn=92t too bad. It works great for buildworld scenarios, since we have a well-defined location for src.sys.mk. It works less well for the individual src Makefiles that might need to access variables set in src.sys.mk, which becomes unfindable w/o extra args on the command line, or requires knowledge of where the Makefile lives in the tree to reach over and snag it. > Similarly, bsd.init.mk doing >=20 > .-include "src.init.mk" > .-include "local.init.mk" >=20 > would allow for customization of things to be done after Makefile has > been read.=20 What would go there? Or would they be empty place holders? > The above would be sufficient to allow src.* to influence bin/Makefile > and lib/Makefile etc without the need to touch them. I both like and dislike that notion :) I more like than dislike it for = the bin/Makefile case (my only objection being it=20 > Other customization hooks would be cool too, bsd.sys.mk is a somewhat > unfortunate name, but it would be handy to have another point for > including {src,local}.*.mk at the end of the normal lib,prog,* mk = files. Also missing is a defined place for users to define WITH/WITHOUT_FOO. > Note that even though we don't install src.*.mk or local.*.mk, those > hook points are still very useful to anyone building their own stuff > with bsd.*.mk, since now they can customize their own builds without = the > need to hack bsd.*.mk=20 True, but it looks like it may make in-tree use in the non-build-world = case more complicated. There may be some resistance to that. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Apr 17 22:07:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 791892F9; Thu, 17 Apr 2014 22:07:01 +0000 (UTC) Received: from va3outboundpool.messaging.microsoft.com (va3ehsobe004.messaging.microsoft.com [216.32.180.14]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2274218AD; Thu, 17 Apr 2014 22:07:00 +0000 (UTC) Received: from mail86-va3-R.bigfish.com (10.7.14.230) by VA3EHSOBE006.bigfish.com (10.7.40.26) with Microsoft SMTP Server id 14.1.225.22; Thu, 17 Apr 2014 22:06:04 +0000 Received: from mail86-va3 (localhost [127.0.0.1]) by mail86-va3-R.bigfish.com (Postfix) with ESMTP id 91C94300396; Thu, 17 Apr 2014 22:06:04 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.10; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 2 X-BigFish: VPS2(zz1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz8275dh1de097hz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail86-va3: transitioning domain of juniper.net does not designate 66.129.239.10 as permitted sender) client-ip=66.129.239.10; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail86-va3 (localhost.localdomain [127.0.0.1]) by mail86-va3 (MessageSwitch) id 1397772362838083_15256; Thu, 17 Apr 2014 22:06:02 +0000 (UTC) Received: from VA3EHSMHS013.bigfish.com (unknown [10.7.14.240]) by mail86-va3.bigfish.com (Postfix) with ESMTP id C3D813600D8; Thu, 17 Apr 2014 22:06:02 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.239.10) by VA3EHSMHS013.bigfish.com (10.7.99.23) with Microsoft SMTP Server (TLS) id 14.16.227.3; Thu, 17 Apr 2014 22:06:01 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 17 Apr 2014 15:06:44 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s3HM6XV02584; Thu, 17 Apr 2014 15:06:38 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 242585809F; Thu, 17 Apr 2014 15:06:33 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> <20140417181159.0EFEA5809E@chaos.jnpr.net> <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Thu, 17 Apr 2014 14:29:28 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 17 Apr 2014 15:06:33 -0700 Message-ID: <20140417220633.242585809F@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org, sjg@juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Apr 2014 22:07:01 -0000 On Thu, 17 Apr 2014 14:29:28 -0600, Warner Losh writes: >> bsd.own.mk -> bsd.srcopts.mk -> bsd.mkopts.mk > >Since we=92re planning a src.opts.mk at some point (to control /usr/src bui= >lds), >the bsd.srcopts.mk name seems to have a short shelf life. Maybe just bsd.op= >ts.mk? Sure bsd.opts.mk sounds fine. >> .-include "src.sys.mk" >> .-include "local.sys.mk" >> = > >> that would be sufficient to enable all sorts of cool stuff. > >In non-posix mode, sure. This would be the natural place for >the MK option setting. While it is early, it isn=92t too bad. It works >great for buildworld scenarios, since we have a well-defined >location for src.sys.mk. It works less well for the individual src >Makefiles that might need to access variables set in src.sys.mk, >which becomes unfindable w/o extra args on the command line, >or requires knowledge of where the Makefile lives in the tree >to reach over and snag it. I've avoided that issue for ages by using a wrapper script to condition the env for the tree I'm working in, junos freebsd, netbsd etc. If all your trees have a share/mk, that you want to use you can just export MAKESYSPATH=".../share/mk" and bmake will DTRT. >> Similarly, bsd.init.mk doing >> = > >> .-include "src.init.mk" >> .-include "local.init.mk" >> = > >> would allow for customization of things to be done after Makefile has >> been read. = > >What would go there? Or would they be empty place holders? It depends. Some things need to be done after Makefile has been read (rather than before) - eg any variable ref on rhs of dependency rule needs to have a value at that time. Let's say I wanted to be able to build head in meta mode (rather than keep syncing to projects/bmake). If there were suitable local.*.mk hook points I could do that without affecting anyone else, until such time as I can convince others that its a brilliant idea ;-) Even if FreeBSD.org decide they want to build /usr/src in meta mode, that doesn't mean that every user of FreeBSD wants all their own projects to build that way. Some of the glue I had in local.*.mk could move to src.*.mk so that /usr/src could leverage it without impacting every user of bsd.*.mk That's just one example of course. >Also missing is a defined place for users to define WITH/WITHOUT_FOO. You can still support {src,local,make}.conf >True, but it looks like it may make in-tree use in the non-build-world case= > more complicated. There may be some resistance to that. I'm not sure it would impact one way or the other. It all boils down to whether /usr/share/mk, -m * or $MAKESYSPATH are used. I would figure anyone inclined to go adding local.*.mk to their tree, would be able to cope with setting MAKESYSPATH or whatever else it takes to do what they want. Thanks --sjg From owner-freebsd-arch@FreeBSD.ORG Fri Apr 18 02:57:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9DBB6F8 for ; Fri, 18 Apr 2014 02:57:35 +0000 (UTC) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 936F912A3 for ; Fri, 18 Apr 2014 02:57:35 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id bj1so1024028pad.31 for ; Thu, 17 Apr 2014 19:57:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=60N7mEFAojRh4wTpk2cjBxmas+TDVeMALr4X/9/cJhs=; b=Zf5h4vU5KfQ37+rGFxDeEnfrspCNr1KHlz91QrWcFvDLMzWWvSkkknvGebka10C9PU uO1bUBuY8eRHKtYJ3p7btNvKJykbTOWEIpkMozhSD1+ZegWw9uqKETEyh5qyg438qs9e JjsIy50SYpcMgUsJqxv/mNP0EpXBGxtdfNiMYE7gxa2Yx+JXWdn+CAIY1OG9vSUU4bB3 d/WOogo+ka5ljNd4YJV8H0ojZKCXA0NOgOrZ1WVf5vPc9D1UYgZaQopixJJOoWjvcp7j YGOJbhKPDi8Q/3IayeKYlYAuj7nNJTW5ksIJ6r58GrcK8evf/LcgXh8MlVZTr8XnGICk Er4w== X-Gm-Message-State: ALoCoQnid6qndENBaDqArn4Lc8KX4gWZWwefle0/eyFefbQmk/bwDulMH/qnkIVKsq+w1H60dvnC X-Received: by 10.66.124.137 with SMTP id mi9mr19412128pab.111.1397789854457; Thu, 17 Apr 2014 19:57:34 -0700 (PDT) Received: from lgmac-ckapadia.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ek2sm56628046pbd.30.2014.04.17.19.57.33 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 19:57:33 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <20140417220633.242585809F@chaos.jnpr.net> Date: Thu, 17 Apr 2014 20:57:31 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <18D76DD4-AA14-46CB-8D41-6176D08FA56F@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> <20140417181159.0EFEA5809E@chaos.jnpr.net> <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> <20140417220633.242585809F@chaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 02:57:35 -0000 On Apr 17, 2014, at 4:06 PM, Simon J. Gerraty wrote: >=20 > On Thu, 17 Apr 2014 14:29:28 -0600, Warner Losh writes: >>> bsd.own.mk -> bsd.srcopts.mk -> bsd.mkopts.mk >>=20 >> Since we=3D92re planning a src.opts.mk at some point (to control = /usr/src bui=3D >> lds), >> the bsd.srcopts.mk name seems to have a short shelf life. Maybe just = bsd.op=3D >> ts.mk? >=20 > Sure bsd.opts.mk sounds fine. Done. >>> .-include "src.sys.mk" >>> .-include "local.sys.mk" >>> =3D >>=20 >>> that would be sufficient to enable all sorts of cool stuff. >>=20 >> In non-posix mode, sure. This would be the natural place for >> the MK option setting. While it is early, it isn=3D92t too bad. It = works >> great for buildworld scenarios, since we have a well-defined >> location for src.sys.mk. It works less well for the individual src >> Makefiles that might need to access variables set in src.sys.mk, >> which becomes unfindable w/o extra args on the command line, >> or requires knowledge of where the Makefile lives in the tree >> to reach over and snag it. >=20 > I've avoided that issue for ages by using a wrapper script to = condition > the env for the tree I'm working in, junos freebsd, netbsd etc. > If all your trees have a share/mk, that you want to use you can just=20= > export MAKESYSPATH=3D".../share/mk" > and bmake will DTRT. Can that be set in a Makefile? >>> Similarly, bsd.init.mk doing >>> =3D >>=20 >>> .-include "src.init.mk" >>> .-include "local.init.mk" >>> =3D >>=20 >>> would allow for customization of things to be done after Makefile = has >>> been read. =3D >>=20 >> What would go there? Or would they be empty place holders? >=20 > It depends. Some things need to be done after Makefile has been read > (rather than before) - eg any variable ref on rhs of dependency rule > needs to have a value at that time. >=20 > Let's say I wanted to be able to build head in meta mode (rather than > keep syncing to projects/bmake). > If there were suitable local.*.mk hook points I could do that without > affecting anyone else, until such time as I can convince others that = its > a brilliant idea ;-) >=20 > Even if FreeBSD.org decide they want to build /usr/src in > meta mode, that doesn't mean that every user of FreeBSD wants all = their > own projects to build that way. Some of the glue I had in local.*.mk > could move to src.*.mk so that /usr/src could leverage it without > impacting every user of bsd.*.mk >=20 > That's just one example of course. OK. I=92m sold. >> Also missing is a defined place for users to define WITH/WITHOUT_FOO. >=20 > You can still support {src,local,make}.conf OK. Will keep it around. >> True, but it looks like it may make in-tree use in the = non-build-world case=3D >> more complicated. There may be some resistance to that. >=20 > I'm not sure it would impact one way or the other. > It all boils down to whether /usr/share/mk, -m * or $MAKESYSPATH > are used. >=20 > I would figure anyone inclined to go adding local.*.mk to their tree, > would be able to cope with setting MAKESYSPATH or whatever else it = takes > to do what they want. Yea, I agree for local.*.mk, but the case is less clear for src.*.mk = which is now required for the build=85 I should do a census on the number of = files this affects. I=92ll post patches as soon as I finish make universe. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Apr 18 03:04:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3D539E0 for ; Fri, 18 Apr 2014 03:04:42 +0000 (UTC) Received: from mail-pd0-f171.google.com (mail-pd0-f171.google.com [209.85.192.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 84B5B1410 for ; Fri, 18 Apr 2014 03:04:42 +0000 (UTC) Received: by mail-pd0-f171.google.com with SMTP id r10so1010932pdi.2 for ; Thu, 17 Apr 2014 20:04:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=FXn9eho4BOOk5pFKhwWpZOFYVDKteoeWbWOyYoCqQlw=; b=iBSJ5Hh2ELMTV361HtO/jYqmL5TEu/U6j/HdL5gzvZEa8xZqZjqRbVL7E9v8os8b4U KNev6C+aalXidK3fTmJPfFQx0bcWm+U2ylV6paDPsxGiHOzYXRm9mrN5bzVx9NP7EZXZ CvwMjoB2GjS+p15n0CETEM9lE2Lw0Ho72Fbk4A2ml/m06M2q+Dwq7t1kLd8dED6Vt7wU TPYQ2LnZAvGfp0JYnsBb+NqCL/L4QLBVxcaWkqVjjP0lxcDozaN043N3MaxjKv1lvDSW 89d6tHHwOoHQGS6tEQviRODAigm06jDQ3alD+xqJYtP3R1hFyUP3sRN0kNEElqGNSHMW wo9A== X-Gm-Message-State: ALoCoQm/KCp7vCibA/Ctc/2wzs1x4Xs7jSjzAseWf44O1UwplaGqMDH9MnllZ/u3kDix0wMqUfAC X-Received: by 10.66.227.104 with SMTP id rz8mr19585677pac.74.1397790276791; Thu, 17 Apr 2014 20:04:36 -0700 (PDT) Received: from lgmac-ckapadia.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ha2sm56685671pbb.8.2014.04.17.20.04.35 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 20:04:36 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: make WITH[OUT]_* more useful? From: Warner Losh In-Reply-To: <18D76DD4-AA14-46CB-8D41-6176D08FA56F@bsdimp.com> Date: Thu, 17 Apr 2014 21:04:34 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <230F4807-19EB-46B3-BC74-987E3C47E69F@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> <20140417181159.0EFEA5809E@chaos.jnpr.net> <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> <20140417220633.242585809F@chaos.jnpr.net> <18D76DD4-AA14-46CB-8D41-6176D08FA56F@bsdimp.com> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.1874) Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 03:04:42 -0000 On Apr 17, 2014, at 8:57 PM, Warner Losh wrote: >=20 > I=92ll post patches as soon as I finish make universe. >=20 Or not. My mercurial patch queue can be cloned from = http://people.freebsd.org/~imp/patch-queue if you want to check out what = I plan on committing, or offer comments before I commit. You can also = browse that URL and check out the individual patches. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Apr 18 07:01:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5554714F; Fri, 18 Apr 2014 07:01:13 +0000 (UTC) Received: from am1outboundpool.messaging.microsoft.com (am1ehsobe002.messaging.microsoft.com [213.199.154.205]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.global.frontbridge.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A41B41A6F; Fri, 18 Apr 2014 07:01:11 +0000 (UTC) Received: from mail40-am1-R.bigfish.com (10.3.201.234) by AM1EHSOBE015.bigfish.com (10.3.207.137) with Microsoft SMTP Server id 14.1.225.22; Fri, 18 Apr 2014 06:45:15 +0000 Received: from mail40-am1 (localhost [127.0.0.1]) by mail40-am1-R.bigfish.com (Postfix) with ESMTP id B595E6055D; Fri, 18 Apr 2014 06:45:14 +0000 (UTC) X-Forefront-Antispam-Report: CIP:66.129.239.15; KIP:(null); UIP:(null); IPV:NLI; H:P-EMF01-SAC.jnpr.net; RD:none; EFVD:NLI X-SpamScore: 4 X-BigFish: VPS4(zz1432Izz1f42h1ee6h1de0h1fdah2073h2146h1202h1e76h2189h1d1ah1d2ah21bch1fc6h208ch1082kzz74efjz31h2a8h839hd25hf0ah1288h12a5h12a9h12bdh12e5h137ah139eh13b6h1441h14ddh1504h1537h162dh1631h1758h1898h18e1h1946h19b5h1ad9h1b0ah1b2fh1b88h224fh1fb3h1d0ch1d2eh1d3fh1de2h1dfeh1dffh1e23h1fe8h1ff5h2218h2216h226dh22d0h24afh2327h2336h2381h2438h2461h2487h24ach24d7h2516h2545h255eh25cch25f6h2605h268bh26d3h1155h) Received-SPF: softfail (mail40-am1: transitioning domain of juniper.net does not designate 66.129.239.15 as permitted sender) client-ip=66.129.239.15; envelope-from=sjg@juniper.net; helo=P-EMF01-SAC.jnpr.net ; SAC.jnpr.net ; Received: from mail40-am1 (localhost.localdomain [127.0.0.1]) by mail40-am1 (MessageSwitch) id 1397803511792535_28958; Fri, 18 Apr 2014 06:45:11 +0000 (UTC) Received: from AM1EHSMHS006.bigfish.com (unknown [10.3.201.242]) by mail40-am1.bigfish.com (Postfix) with ESMTP id BEA63200079; Fri, 18 Apr 2014 06:45:11 +0000 (UTC) Received: from P-EMF01-SAC.jnpr.net (66.129.239.15) by AM1EHSMHS006.bigfish.com (10.3.207.106) with Microsoft SMTP Server (TLS) id 14.16.227.3; Fri, 18 Apr 2014 06:45:11 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF01-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 17 Apr 2014 23:45:53 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s3I6jqV07823; Thu, 17 Apr 2014 23:45:53 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id A6C9E5809F; Thu, 17 Apr 2014 23:45:52 -0700 (PDT) To: Warner Losh Subject: Re: make WITH[OUT]_* more useful? In-Reply-To: <18D76DD4-AA14-46CB-8D41-6176D08FA56F@bsdimp.com> References: <20140401051327.F20F958097@chaos.jnpr.net> <20140401064316.GQ99393@ivaldir.etoilebsd.net> <766EFF76-7B41-423F-B447-DBA25BD14470@bsdimp.com> <20140401162912.7A9D058097@chaos.jnpr.net> <20140401195249.8752258097@chaos.jnpr.net> <2349B38E-F6F2-4911-92F2-3D62FCBD73DA@bsdimp.com> <20140402212328.7CE8958097@chaos.jnpr.net> <5D9BD168-D7E5-477E-AEA3-47B24445C89F@bsdimp.com> <20140410034645.C058658097@chaos.jnpr.net> <20140417063437.3BBF85809E@chaos.jnpr.net> <06CC1B46-2BF2-439C-8F97-16EA8DD7805C@bsdimp.com> <20140417181159.0EFEA5809E@chaos.jnpr.net> <7C527DA8-ABC0-4D6A-AACB-64D89246B569@bsdimp.com> <20140417220633.242585809F@chaos.jnpr.net> <18D76DD4-AA14-46CB-8D41-6176D08FA56F@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Thu, 17 Apr 2014 20:57:31 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 17 Apr 2014 23:45:52 -0700 Message-ID: <20140418064552.A6C9E5809F@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-OriginatorOrg: juniper.net X-FOPE-CONNECTOR: Id%0$Dn%*$RO%0$TLS%0$FQDN%$TlsDn% Cc: Baptiste Daroussin , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2014 07:01:13 -0000 On Thu, 17 Apr 2014 20:57:31 -0600, Warner Losh writes: >> export MAKESYSPATH=3D".../share/mk" >> and bmake will DTRT. > >Can that be set in a Makefile? Yes, but the tricky bit is setting it in a makefile that will be found without that set. >> I would figure anyone inclined to go adding local.*.mk to their tree, >> would be able to cope with setting MAKESYSPATH or whatever else it takes >> to do what they want. > >Yea, I agree for local.*.mk, but the case is less clear for src.*.mk which = >is >now required for the build=85 I should do a census on the number of files t= >his affects. Yes, that would probably be useful data. >I=92ll post patches as soon as I finish make universe. Cool - thanks From owner-freebsd-arch@FreeBSD.ORG Fri May 2 22:09:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44AB6B19 for ; Fri, 2 May 2014 22:09:31 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3A8811BF for ; Fri, 2 May 2014 22:09:30 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id n15so277777wiw.8 for ; Fri, 02 May 2014 15:09:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=GjBTMx3DW0EDf0Cvy/FGAF6DuaQExa7GpV0nip0mu0c=; b=KO2d73/X1kynzBZVMG9z7YLNkvx/G+AWyvv0LDb2n2monZKhBoQwp1UYKXCIteDp8Q 7ut0uiqBiH5Q8jmhfkW2xYv5rVjsOcfA4Bq8pjQHNPyLTE093torOtscfRbUmv2v8DId rnZQEP7huNYIS16Wteb2vGa8VCz4L7UFqnXULOpSFxwADgeryRGtdcbNX9s5nIZCY3GF 0PPAHustoFhplqRL7eY6m2iYcXuJcg4N8tnVEUgPNmlUcLumxQzuT1sdUVaiZczGKkfp 2+9WONCR2iGKf7fULpRDnrcYR6x9uTU5UIbpf6NpdOaB/RMl7RfTJkwqfo1eeX/eWgV2 gtAQ== X-Received: by 10.180.81.40 with SMTP id w8mr4808805wix.45.1399068569086; Fri, 02 May 2014 15:09:29 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id by1sm4049107wjc.26.2014.05.02.15.09.27 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 02 May 2014 15:09:28 -0700 (PDT) Date: Sat, 3 May 2014 00:09:25 +0200 From: Mateusz Guzik To: freebsd-arch@freebsd.org Subject: safer alternative for pfind and pget? Message-ID: <20140502220925.GB28158@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 22:09:31 -0000 (I hope this list is ok.) Hello, currently when one wants to lookup a process there are only 2 functions to help: - pfind which provides no filtering whatsoever - pget which provides limited (mostly negative) filtering This leads to bugs since it places burden on all callers to make sure they filter out all states they cannot cope with (new, exiting, execing) and it is easy to miss some. So I would like to change this so that default function to use would provide the following semantics: with no flags it returns a fully constructed not-exiting process which the caller can debug Flags allow to ignore each restriction. It is unclear to me if this should replace pget or a new function should be created and all pget callers converted over time (and selected pfind callers). Thoughts? -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Fri May 2 22:36:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8994FEB6 for ; Fri, 2 May 2014 22:36:11 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6060513C5 for ; Fri, 2 May 2014 22:36:11 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s42Ma93D094529 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 2 May 2014 15:36:10 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s42Ma9aI094528; Fri, 2 May 2014 15:36:09 -0700 (PDT) (envelope-from jmg) Date: Fri, 2 May 2014 15:36:09 -0700 From: John-Mark Gurney To: Mateusz Guzik Subject: Re: safer alternative for pfind and pget? Message-ID: <20140502223609.GU43976@funkthat.com> Mail-Followup-To: Mateusz Guzik , freebsd-arch@freebsd.org References: <20140502220925.GB28158@dft-labs.eu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140502220925.GB28158@dft-labs.eu> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 02 May 2014 15:36:10 -0700 (PDT) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 May 2014 22:36:11 -0000 Mateusz Guzik wrote this message on Sat, May 03, 2014 at 00:09 +0200: > - pget which provides limited (mostly negative) filtering as pget is undocumented, please make it part of this project to document it.. > It is unclear to me if this should replace pget or a new function should > be created and all pget callers converted over time (and selected pfind > callers). It really sounds like pget should add a few flags, and then maybe add a few defines that mask all the flags that you want to make it easier... If you documented this, then I think half of the questions would be answered... Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Sat May 3 09:12:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F9BA2D3; Sat, 3 May 2014 09:12:22 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E826179F; Sat, 3 May 2014 09:12:21 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id 10so3900720lbg.23 for ; Sat, 03 May 2014 02:12:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=LQCXgg3KdxZT1+5o+s6w6/1++KYHTReWCAgjYzpWSgA=; b=yJdRxa84JsLc30rwkqB7xBUcvCEyW6fVVIf2lq0SOymQjFsMBCURGWzfE0LzIzkbLY F8+eFloqaaQJN9YTQTKHGVwc6uYTifMJFrm8p0+FA5czAWzYfsuKGXYBM6S/AKbINVQF 32HbLPEIRlcyHw5pFbj04cvNvZBNTd4/vfkO68csuZcmJbnpFctnIP1E5wPyvMBYqbfx uKcYPZ1Te3KnO2GclqJTQvdweQGqMzyUDBVenqPGBO2ZdyGOs4+LEHRsxBUR7NCztH0+ 5rUd3fxKe+SyDRpua07lA8WnZC25OxDVwlYgUjA9NbD1vA2lZ6aBECgZppIIhAWLYtCB pp/w== X-Received: by 10.152.8.7 with SMTP id n7mr2417326laa.22.1399108338875; Sat, 03 May 2014 02:12:18 -0700 (PDT) Received: from localhost ([178.150.115.244]) by mx.google.com with ESMTPSA id cr6sm1858816lbb.19.2014.05.03.02.12.16 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 May 2014 02:12:17 -0700 (PDT) Sender: Mikolaj Golub Date: Sat, 3 May 2014 12:12:14 +0300 From: Mikolaj Golub To: Mateusz Guzik , freebsd-arch@freebsd.org Subject: Re: safer alternative for pfind and pget? Message-ID: <20140503091213.GA3876@gmail.com> References: <20140502220925.GB28158@dft-labs.eu> <20140502223609.GU43976@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140502223609.GU43976@funkthat.com> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: John-Mark Gurney , Sergey Kandaurov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 09:12:22 -0000 On Fri, May 02, 2014 at 03:36:09PM -0700, John-Mark Gurney wrote: > Mateusz Guzik wrote this message on Sat, May 03, 2014 at 00:09 +0200: > > - pget which provides limited (mostly negative) filtering > > as pget is undocumented, please make it part of this project to > document it.. pluknet had a man page for pget long time ago, reviewed by kib. I thought it was committed. Don't know why it did not happen. -- Mikolaj Golub From owner-freebsd-arch@FreeBSD.ORG Sat May 3 09:59:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE389D8D; Sat, 3 May 2014 09:59:14 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 613FA1AC7; Sat, 3 May 2014 09:59:14 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id x12so3701426wgg.18 for ; Sat, 03 May 2014 02:59:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=kvF+QEiP65WRDJEvtTmJHvCV9SaQS1YWerThPprZep0=; b=vPubWx3toq+gUnT1o8gLMRZ8QwWhmdfK+ogtkXdRlqpKZd9awJb/3C/AwC5yLWCd3U rUrgCUDWjNU8YAPfItqJhU/qlD3yIgIu+09gPruyxqrLJ/jFHLb3cklJLAmuxP2G1+/t Zvqe5Mm9oWio8ubhD7HVs1+b2+KemxN7HC2iBKCBjtI+xPCiRWoEmyMCGafIeo2rNtyM TQDcQwqS4Lt9h6UiDASbTBRh4Erhfx2TUFtacpS5yT5+uOy9MftcexoyCcz5Rhgzzm3d GZqc6dWvg2mp1saicevOH6zgpnIJ0Yq10cJlyqylT+V4bOunFlKDuDvPPkSJ0BaTBFqD KvQw== MIME-Version: 1.0 X-Received: by 10.180.91.114 with SMTP id cd18mr7003633wib.28.1399111152533; Sat, 03 May 2014 02:59:12 -0700 (PDT) Sender: pluknet@gmail.com Received: by 10.216.169.193 with HTTP; Sat, 3 May 2014 02:59:12 -0700 (PDT) In-Reply-To: <20140503091213.GA3876@gmail.com> References: <20140502220925.GB28158@dft-labs.eu> <20140502223609.GU43976@funkthat.com> <20140503091213.GA3876@gmail.com> Date: Sat, 3 May 2014 13:59:12 +0400 X-Google-Sender-Auth: RwJseouEhikXMU7XfgU9IHdNFVc Message-ID: Subject: Re: safer alternative for pfind and pget? From: Sergey Kandaurov To: Mikolaj Golub Content-Type: multipart/mixed; boundary=f46d043749ad3f9ffb04f87bf39d Cc: John-Mark Gurney , Mateusz Guzik , FreeBSD Arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 09:59:15 -0000 --f46d043749ad3f9ffb04f87bf39d Content-Type: text/plain; charset=ISO-8859-1 On 3 May 2014 13:12, Mikolaj Golub wrote: > On Fri, May 02, 2014 at 03:36:09PM -0700, John-Mark Gurney wrote: >> Mateusz Guzik wrote this message on Sat, May 03, 2014 at 00:09 +0200: >> > - pget which provides limited (mostly negative) filtering >> >> as pget is undocumented, please make it part of this project to >> document it.. > > pluknet had a man page for pget long time ago, reviewed by kib. I > thought it was committed. Don't know why it did not happen. This is something I have for pget(9). If it is good enough, I'll commit it. -- wbr, pluknet --f46d043749ad3f9ffb04f87bf39d Content-Type: application/octet-stream; name="pget.9" Content-Disposition: attachment; filename="pget.9" Content-Transfer-Encoding: base64 X-Attachment-Id: f_huqqkeip0 LlwiIENvcHlyaWdodCAoYykgMjAxMSBTZXJnZXkgS2FuZGF1cm92Ci5cIiBBbGwgcmlnaHRzIHJl c2VydmVkLgouXCIKLlwiIFJlZGlzdHJpYnV0aW9uIGFuZCB1c2UgaW4gc291cmNlIGFuZCBiaW5h cnkgZm9ybXMsIHdpdGggb3Igd2l0aG91dAouXCIgbW9kaWZpY2F0aW9uLCBhcmUgcGVybWl0dGVk IHByb3ZpZGVkIHRoYXQgdGhlIGZvbGxvd2luZyBjb25kaXRpb25zCi5cIiBhcmUgbWV0OgouXCIg MS4gUmVkaXN0cmlidXRpb25zIG9mIHNvdXJjZSBjb2RlIG11c3QgcmV0YWluIHRoZSBhYm92ZSBj b3B5cmlnaHQKLlwiICAgIG5vdGljZSwgdGhpcyBsaXN0IG9mIGNvbmRpdGlvbnMgYW5kIHRoZSBm b2xsb3dpbmcgZGlzY2xhaW1lci4KLlwiIDIuIFJlZGlzdHJpYnV0aW9ucyBpbiBiaW5hcnkgZm9y bSBtdXN0IHJlcHJvZHVjZSB0aGUgYWJvdmUgY29weXJpZ2h0Ci5cIiAgICBub3RpY2UsIHRoaXMg bGlzdCBvZiBjb25kaXRpb25zIGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIgaW4gdGhlCi5c IiAgICBkb2N1bWVudGF0aW9uIGFuZC9vciBvdGhlciBtYXRlcmlhbHMgcHJvdmlkZWQgd2l0aCB0 aGUgZGlzdHJpYnV0aW9uLgouXCIKLlwiIFRISVMgU09GVFdBUkUgSVMgUFJPVklERUQgQlkgVEhF IEFVVEhPUiBBTkQgQ09OVFJJQlVUT1JTIGBgQVMgSVMnJyBBTkQKLlwiIEFOWSBFWFBSRVNTIE9S IElNUExJRUQgV0FSUkFOVElFUywgSU5DTFVESU5HLCBCVVQgTk9UIExJTUlURUQgVE8sIFRIRQou XCIgSU1QTElFRCBXQVJSQU5USUVTIE9GIE1FUkNIQU5UQUJJTElUWSBBTkQgRklUTkVTUyBGT1Ig QSBQQVJUSUNVTEFSIFBVUlBPU0UKLlwiIEFSRSBESVNDTEFJTUVELiAgSU4gTk8gRVZFTlQgU0hB TEwgVEhFIEFVVEhPUiBPUiBDT05UUklCVVRPUlMgQkUgTElBQkxFCi5cIiBGT1IgQU5ZIERJUkVD VCwgSU5ESVJFQ1QsIElOQ0lERU5UQUwsIFNQRUNJQUwsIEVYRU1QTEFSWSwgT1IgQ09OU0VRVUVO VElBTAouXCIgREFNQUdFUyAoSU5DTFVESU5HLCBCVVQgTk9UIExJTUlURUQgVE8sIFBST0NVUkVN RU5UIE9GIFNVQlNUSVRVVEUgR09PRFMKLlwiIE9SIFNFUlZJQ0VTOyBMT1NTIE9GIFVTRSwgREFU QSwgT1IgUFJPRklUUzsgT1IgQlVTSU5FU1MgSU5URVJSVVBUSU9OKQouXCIgSE9XRVZFUiBDQVVT RUQgQU5EIE9OIEFOWSBUSEVPUlkgT0YgTElBQklMSVRZLCBXSEVUSEVSIElOIENPTlRSQUNULCBT VFJJQ1QKLlwiIExJQUJJTElUWSwgT1IgVE9SVCAoSU5DTFVESU5HIE5FR0xJR0VOQ0UgT1IgT1RI RVJXSVNFKSBBUklTSU5HIElOIEFOWSBXQVkKLlwiIE9VVCBPRiBUSEUgVVNFIE9GIFRISVMgU09G VFdBUkUsIEVWRU4gSUYgQURWSVNFRCBPRiBUSEUgUE9TU0lCSUxJVFkgT0YKLlwiIFNVQ0ggREFN QUdFLgouXCIKLlwiICRGcmVlQlNEJAouXCIKLkRkIERlY2VtYmVyIDIxLCAyMDExCi5EdCBQR0VU IDkKLk9zCi5TaCBOQU1FCi5ObSBwZ2V0Ci5OZCBsb2NhdGUgYSBwcm9jZXNzIGJ5IG51bWJlcgou U2ggU1lOT1BTSVMKLkluIHN5cy9wYXJhbS5oCi5JbiBzeXMvcHJvYy5oCi5GdCBpbnQKLkZuIHBn ZXQgInBpZF90IHBpZCIgImludCBmbGFncyIgInN0cnVjdCBwcm9jICoqcHAiCi5TaCBERVNDUklQ VElPTgpUaGlzIGZ1bmN0aW9uCnRha2VzIGEKLkZhIHBpZAphcyBpdHMgYXJndW1lbnQgYW5kIGZp bGxzIGEgcG9pbnRlciB0byB0aGUKLlZ0IHByb2MKc3RydWN0dXJlIGluCi5GYSAqcHAgLgpUaGUg YWN0dWFsIG9wZXJhdGlvbiBpcyBwZXJmb3JtZWQgYnkgaW52b2tpbmcgdGhlCi5YciBwZmluZCA5 CmZ1bmN0aW9uLgpUaGUgZm91bmQgcHJvY2VzcyBpcyByZXR1cm5lZCBsb2NrZWQuCk9ubHkgZm9y Ci5EdiBQR0VUX0hPTEQKY2FzZSBpdCBpcyByZXR1cm5lZCB1bmxvY2tlZCAoYnV0IHJlZmVyZW5j ZWQpLgpUaGUKLkZuIHBnZXQKZnVuY3Rpb24gY2FuCnBlcmZvcm0gYWRkaXRpb25hbCBtYW5pcHVs YXRpb25zLCBkZXBlbmRpbmcgb24gYQouRmEgZmxhZ3MKYXJndW1lbnQuCi5QcApUaGUKLkZhIGZs YWdzCmFyZ3VtZW50IGlzIHRoZSBsb2dpY2FsIE9SIG9mIHNvbWUgc3Vic2V0IG9mOgouQmwgLXRh ZyAtd2lkdGggIi5EdiBQR0VUX05PVElORVhFQyIKLkl0IER2IFBHRVRfSE9MRApJZiBzZXQsIHRo ZSBmb3VuZCBwcm9jZXNzIHdpbGwgYmUgcmVmZXJlbmNlZCBhbmQgdW5sb2NrZWQuCi5JdCBEdiBQ R0VUX0NBTlNFRQpJZiBzZXQsIHRoZSBmb3VuZCBwcm9jZXNzIHdpbGwgYmUgY2hlY2tlZCBmb3Ig aXRzIHZpc2liaWxpdHkuClNlZQouWHIgcF9jYW5zZWUgOSAuCi5JdCBEdiBQR0VUX0NBTkRFQlVH CklmIHNldCwgdGhlIGZvdW5kIHByb2Nlc3Mgd2lsbCBiZSBjaGVja2VkIGZvciBpdHMgZGVidWdn YWJpbGl0eS4KU2VlCi5YciBwX2NhbmRlYnVnIDkgLgouSXQgRHYgUEdFVF9JU0NVUlJFTlQKSWYg c2V0LCB0aGUgZm91bmQgcHJvY2VzcyB3aWxsIGJlIGNoZWNrZWQgdGhhdCBpdCBtYXRjaGVzIHRo ZSBjdXJyZW50CnByb2Nlc3MgY29udGV4dC4KLkl0IER2IFBHRVRfTk9UV0VYSVQKSWYgc2V0LCB0 aGUgZm91bmQgcHJvY2VzcyB3aWxsIGJlIGNoZWNrZWQgdGhhdCBpdCBkb2VzIG5vdCBoYXZlIHRo ZSBwcm9jZXNzCmZsYWcKLkR2IFBfV0VYSVQKc2V0LgouSXQgRHYgUEdFVF9OT1RJTkVYRUMKSWYg c2V0LCB0aGUgZm91bmQgcHJvY2VzcyB3aWxsIGJlIGNoZWNrZWQgdGhhdCBpdCBkb2VzIG5vdCBo YXZlIHRoZSBwcm9jZXNzCmZsYWcKLkR2IFBfSU5FWEVDCnNldC4KLkl0IER2IFBHRVRfTk9USUQK SWYgc2V0LAouRmEgcGlkCmlzIG5vdCBhc3N1bWVkIGFzIGEgdGhyZWFkIGlkIGZvciB2YWx1ZXMg bGFyZ2VyIHRoYW4KLkR2IFBJRF9NQVggLgouSXQgRHYgUEdFVF9XQU5UUkVBRApBIHNob3J0aGFu ZCBmb3IKLlBxIER2IFBHRVRfSE9MRCB8IFBHRVRfQ0FOREVCVUcgfCBQR0VUX05PVFdFWElUIC4K LkVsCi5TaCBSRVRVUk4gVkFMVUVTCklmIHRoZSBwcm9jZXNzIGlzIGZvdW5kIGluIHRoZSBzcGVj aWZpZWQgd2F5LCB0aGVuIHplcm8gaXMgcmV0dXJuZWQsCm90aGVyd2lzZSBhbiBhcHByb3ByaWF0 ZSBlcnJvciBjb2RlIGlzIHJldHVybmVkLgouU2ggU0VFIEFMU08KLlhyIHBfY2FuZGVidWcgOSAs Ci5YciBwX2NhbnNlZSA5ICwKLlhyIHBmaW5kIDkK --f46d043749ad3f9ffb04f87bf39d-- From owner-freebsd-arch@FreeBSD.ORG Sat May 3 18:21:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C047DF1; Sat, 3 May 2014 18:21:05 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A100016AD; Sat, 3 May 2014 18:21:04 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.8/8.14.8) with ESMTP id s43IKsQ9095799; Sat, 3 May 2014 21:20:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s43IKsQ9095799 Received: (from kostik@localhost) by tom.home (8.14.8/8.14.8/Submit) id s43IKr0C095796; Sat, 3 May 2014 21:20:53 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 3 May 2014 21:20:53 +0300 From: Konstantin Belousov To: Sergey Kandaurov Subject: Re: safer alternative for pfind and pget? Message-ID: <20140503182053.GC74331@kib.kiev.ua> References: <20140502220925.GB28158@dft-labs.eu> <20140502223609.GU43976@funkthat.com> <20140503091213.GA3876@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jy6Sn24JjFx/iggw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: Mikolaj Golub , John-Mark Gurney , Mateusz Guzik , FreeBSD Arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 May 2014 18:21:05 -0000 --jy6Sn24JjFx/iggw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, May 03, 2014 at 01:59:12PM +0400, Sergey Kandaurov wrote: > On 3 May 2014 13:12, Mikolaj Golub wrote: > > On Fri, May 02, 2014 at 03:36:09PM -0700, John-Mark Gurney wrote: > >> Mateusz Guzik wrote this message on Sat, May 03, 2014 at 00:09 +0200: > >> > - pget which provides limited (mostly negative) filtering > >> > >> as pget is undocumented, please make it part of this project to > >> document it.. > > > > pluknet had a man page for pget long time ago, reviewed by kib. I > > thought it was committed. Don't know why it did not happen. >=20 > This is something I have for pget(9). > If it is good enough, I'll commit it. >=20 See some minor notes below. And yes, please commit this. > .\" Copyright (c) 2011 Sergey Kandaurov > .\" All rights reserved. > .\" > .\" Redistribution and use in source and binary forms, with or without > .\" modification, are permitted provided that the following conditions > .\" are met: > .\" 1. Redistributions of source code must retain the above copyright > .\" notice, this list of conditions and the following disclaimer. > .\" 2. Redistributions in binary form must reproduce the above copyright > .\" notice, this list of conditions and the following disclaimer in the > .\" documentation and/or other materials provided with the distributio= n. > .\" > .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PU= RPOSE > .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIAB= LE > .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUE= NTIAL > .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOO= DS > .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, S= TRICT > .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY= WAY > .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > .\" SUCH DAMAGE. > .\" > .\" $FreeBSD$ > .\" > .Dd December 21, 2011 > .Dt PGET 9 > .Os > .Sh NAME > .Nm pget > .Nd locate a process by number > .Sh SYNOPSIS > .In sys/param.h > .In sys/proc.h > .Ft int > .Fn pget "pid_t pid" "int flags" "struct proc **pp" > .Sh DESCRIPTION > This function > takes a > .Fa pid > as its argument, which can be either a process or thread id, In the later case, a process owning the specified thread is looked for. >and fills a pointer to the > .Vt proc > structure in > .Fa *pp . > The actual operation is performed by invoking the > .Xr pfind 9 > function. > The found process is returned locked. > Only for > .Dv PGET_HOLD > case it is returned unlocked (but referenced). s/referenced/held/ > The > .Fn pget > function can > perform additional manipulations, depending on a > .Fa flags > argument. > .Pp > The > .Fa flags > argument is the logical OR of some subset of: > .Bl -tag -width ".Dv PGET_NOTINEXEC" > .It Dv PGET_HOLD > If set, the found process will be referenced and unlocked. > .It Dv PGET_CANSEE > If set, the found process will be checked for its visibility. > See > .Xr p_cansee 9 . > .It Dv PGET_CANDEBUG > If set, the found process will be checked for its debuggability. > See > .Xr p_candebug 9 . > .It Dv PGET_ISCURRENT > If set, the found process will be checked that it matches the current > process context. > .It Dv PGET_NOTWEXIT > If set, the found process will be checked that it does not have the proce= ss > flag > .Dv P_WEXIT > set. > .It Dv PGET_NOTINEXEC > If set, the found process will be checked that it does not have the proce= ss > flag > .Dv P_INEXEC > set. > .It Dv PGET_NOTID > If set, > .Fa pid > is not assumed as a thread id for values larger than > .Dv PID_MAX . > .It Dv PGET_WANTREAD > A shorthand for > .Pq Dv PGET_HOLD | PGET_CANDEBUG | PGET_NOTWEXIT . > .El > .Sh RETURN VALUES > If the process is found in the specified way, then zero is returned, > otherwise an appropriate error code is returned. > .Sh SEE ALSO > .Xr p_candebug 9 , > .Xr p_cansee 9 , > .Xr pfind 9 --jy6Sn24JjFx/iggw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTZTOFAAoJEJDCuSvBvK1BaJMQAIeEuT9RHLqR6iHDkE3iLaSc zfiNxMXkz6j0J9o1nlBvXmY8Ra7EvwhGQydmhxh5LJUfQdRKbz98jhzxXdUsLEUc ThePYNCaEsM64pnF+OdL2X9BVkINj4Yaa8sAQVhD0Eot49xUhPcEw4StC7fx6qO5 uLBqxtlzcVnVUGyKJwLmi2BZn2Hgu8igsAs4i3zlzq6IIgbVvFLUSEdah0JyGw/2 y1r5CSNk57LovlYaMPIgPvth+n/StPJnP144XVgg+Cvsy6r7kbd4OwBx6MmsrcVq 7IFjVtRDRhYC3avj/3KyAgHwF/pb0YEGtKbVd5bJCR2pAkcrf4iUybjmGnhGy3R+ pgUktME+OI0iydVEpHbwrCvHNEAFmp93Sny7wYMzo0q6hz6wK2e7ehNskFQy0RD2 5u9MZHPcQbUh6nMV/lOhOGuyPJL1xV61cyfIiCvTmiqrw1xYnmGvR4WNeiHSQaek 0mUOUS79cFiAKF0rWzjrwOXiK8xPBclv+0aNk6FHsjI1B7qBM0Nh0gVEZlUfvlRP QiJ/Ni130ot6fhlamgjphrjAXEAnWcPvWZ1QaG6NzHPChES2daUY7+LC5Nn+eKAE cNV5cmeq2aGbPL8IRmn2rmTPCeAdbOvT3Q0fjY4GTA7UWfXmJlK+FsdoqXcgOIoK 7Pf5/1aFHW/9JKjkPLp4 =azoE -----END PGP SIGNATURE----- --jy6Sn24JjFx/iggw-- From owner-freebsd-arch@FreeBSD.ORG Sun May 4 08:13:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01C3A56C; Sun, 4 May 2014 08:13:50 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D99F1B71; Sun, 4 May 2014 08:13:49 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id 63so700435qgz.16 for ; Sun, 04 May 2014 01:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=fWJhD5u2ewfB2n1vGTcAeFYUcfWIC41/GoMYmOjdW9U=; b=c/ugLMUtCysjoH9+5cIjY/FnivGkuSuqp3Fquseh4YEyCdyHI2O7Es/HmNi3ECLYYf FdGy7rpHdW7AKKNNEE8qDZ9TYERsNCfa75vrhmXQ049g3/rQ4/Zo0UxcNa57vfFUgDGX FEsN/Cst3rXsSFRnn8KjkVSUr7bBcHpqcX+SXJ7I84xrgAhlgdRi/lOiH2eh3uvW5kqz AciMQJgwDdnL5BcuD1ZpeEr6B59pdS/Ix44o+PiNHSX1csZrBilRCl7Ks31JFPkpES1J wnnN+4u1E/3vP7/0uTZtBkGkxlUGF2PkfR3vQsxXWpld+AcWX1B5hz7dXjFOBC7C+Q4W KKfA== MIME-Version: 1.0 X-Received: by 10.224.15.202 with SMTP id l10mr5134262qaa.76.1399191228712; Sun, 04 May 2014 01:13:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 01:13:48 -0700 (PDT) In-Reply-To: References: <3F7430D7-3C0F-43E1-8EBD-8AA4F701497C@FreeBSD.org> <20140503155745.GA2457@La-Habana> <20140503192305.GA1847@La-Habana> <536592D1.7080403@freebsd.org> Date: Sun, 4 May 2014 01:13:48 -0700 X-Google-Sender-Auth: h1080AmssIna7L0KrecPaYd9USY Message-ID: Subject: Re: Leaving the Desktop Market From: Adrian Chadd To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Nathan Whitehorn , Jordan Hubbard , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 08:13:50 -0000 [snip] ok, how about this to start with: http://people.freebsd.org/~adrian/power/20140504-powerd-1.diff This defaults powerd to set minfreq to 50% of the hardware available maxfreq, unless minfreq is explicitly given on the command line. This should at least fix some of the really silly default decisions made by powerd during idle. This code sorely needs an update to the cpufreq_*() routines. That can come later. Does anyone have any issues? -a From owner-freebsd-arch@FreeBSD.ORG Sun May 4 08:15:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 578266FD; Sun, 4 May 2014 08:15:51 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E271E1B9E; Sun, 4 May 2014 08:15:50 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so5057434qgf.17 for ; Sun, 04 May 2014 01:15:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/7yUUlj/wOggeFEBIS29SA117U92VS4Y/5mFJsgceo0=; b=EBRlcVbgqanj/1+VHMRFYcqmAA1YP7X11XYTt6NVw+9J6+NbhR2NmhOCTdemiK0LD/ kaGY4It7N/7ttrZWKe7wGCfFShbDx+KtvARv3bRmbUhYKGZTBcDVPzhAxrwnyhH5rakb MQ2CwHUvUeFcyjqh1kGjuoXeWWG55CuzeieAscTYPXJCU1P9xrOTK3u0eJ16o8uRSW3p PxyAcKF7XIqqczIyb10JgEB0V3djya2ymIzT8O7HiQqWJnV/330HNbxnbfCl1BcL3u2u o4Y/JWL24iKW2fBVC32FzoifuWBD/ABSaoIxPV/HUYEae+z0AEP/fd0WOwHso6S0lrkR I0Ww== MIME-Version: 1.0 X-Received: by 10.140.104.195 with SMTP id a61mr15432136qgf.102.1399191350091; Sun, 04 May 2014 01:15:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 01:15:49 -0700 (PDT) In-Reply-To: References: <3F7430D7-3C0F-43E1-8EBD-8AA4F701497C@FreeBSD.org> <20140503155745.GA2457@La-Habana> <20140503192305.GA1847@La-Habana> <536592D1.7080403@freebsd.org> Date: Sun, 4 May 2014 01:15:49 -0700 X-Google-Sender-Auth: SbbZsYZ8dyQDuQCxjg7PNI3X1Hk Message-ID: Subject: Re: Leaving the Desktop Market From: Adrian Chadd To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Nathan Whitehorn , Jordan Hubbard , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 08:15:51 -0000 [snip].. and silly me, cpufreq_*() is a kernel interface. Boy that API would be nice in userland. -a From owner-freebsd-arch@FreeBSD.ORG Sun May 4 08:27:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC14FA86; Sun, 4 May 2014 08:27:39 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EC6C1C57; Sun, 4 May 2014 08:27:39 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id x13so6629258qcv.35 for ; Sun, 04 May 2014 01:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=SJOaulOTSv4UQdCH9/3BW8n8h7iEu4lgIdbH75efsY0=; b=TfQweivZppTF1+TdnafZrV4EIxav+d/SWv+UGP8ziU2u+WVl4tXmNfuE1uIOa5RPVN iEbxJNf6JEmlhqfpEkbnSA/EfYnuvvFdrr0n0iP13Q24X0/2p55eXAIhVByk1T/AE4O2 VkTTEQ87vV1Njty5/42Gake4Yt+fnmR/dMKbG2tQGk3dHedWFiI37dHLb1rVe2muEqrA TDK6xPQF9M3wVW0ScKgVO0TwObweBOpCgdxAF2Y8q/SQXKAbFjPhFSauNoX0TYoUB762 iyXxfcWRrdsQ6aBLnOITo6CL50Nv467O1z+5xPLZfVT0oNUdniav8H0TXyDOhLULGan6 PCRg== MIME-Version: 1.0 X-Received: by 10.140.96.51 with SMTP id j48mr33463775qge.24.1399192058524; Sun, 04 May 2014 01:27:38 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 01:27:38 -0700 (PDT) Date: Sun, 4 May 2014 01:27:38 -0700 X-Google-Sender-Auth: u8h749eWk6MBEQcd3bo2SBVr0wM Message-ID: Subject: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Kevin Oberman , "freebsd-arch@freebsd.org" , "freebsd-acpi@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 08:27:39 -0000 Hi, I'd like to propose flipping a few things: * Flipping the default lid state to S3. I think ACPI suspend/resume seems to work well enough these days and I've not met anyone lately who expects the default from their laptop to be "stay awake with the lid shut." * Save chip bugs that we should add workarounds for, we should be OK to enter lower sleep states when idling. Flipping this may expose some further crazy driver, platform or timer bugs, but they again likely should be fixed. what do people think? -a Index: etc/defaults/rc.conf =================================================================== --- etc/defaults/rc.conf (revision 265255) +++ etc/defaults/rc.conf (working copy) @@ -642,9 +642,9 @@ devfs_set_rulesets="" # A list of /mount/dev=ruleset_name settings to # apply (must be mounted already, i.e. fstab(5)) devfs_load_rulesets="YES" # Enable to always load the default rulesets -performance_cx_lowest="HIGH" # Online CPU idle state +performance_cx_lowest="Cmax" # Online CPU idle state performance_cpu_freq="NONE" # Online CPU frequency -economy_cx_lowest="HIGH" # Offline CPU idle state +economy_cx_lowest="Cmax" # Offline CPU idle state economy_cpu_freq="NONE" # Offline CPU frequency virecover_enable="YES" # Perform housekeeping for the vi(1) editor ugidfw_enable="NO" # Load mac_bsdextended(4) rules on boot Index: sys/dev/acpica/acpi.c =================================================================== --- sys/dev/acpica/acpi.c (revision 265255) +++ sys/dev/acpica/acpi.c (working copy) @@ -620,11 +620,12 @@ /* * Dispatch the default sleep state to devices. The lid switch is set - * to UNKNOWN by default to avoid surprising users. + * to S3 to mirror what everything else iBook and later does. */ sc->acpi_power_button_sx = acpi_sleep_states[ACPI_STATE_S5] ? ACPI_STATE_S5 : ACPI_STATE_UNKNOWN; - sc->acpi_lid_switch_sx = ACPI_STATE_UNKNOWN; + sc->acpi_lid_switch_sx = acpi_sleep_states[ACPI_STATE_S3] ? + ACPI_STATE_S3 : ACPI_STATE_UNKNOWN; sc->acpi_standby_sx = acpi_sleep_states[ACPI_STATE_S1] ? ACPI_STATE_S1 : ACPI_STATE_UNKNOWN; sc->acpi_suspend_sx = acpi_sleep_states[ACPI_STATE_S3] ? From owner-freebsd-arch@FreeBSD.ORG Sun May 4 09:28:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0EE79612; Sun, 4 May 2014 09:28:58 +0000 (UTC) Received: from mail.fer.hr (mail.fer.hr [161.53.72.233]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.fer.hr", Issuer "TERENA SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A48B11ED; Sun, 4 May 2014 09:28:57 +0000 (UTC) Received: from x23 (31.147.125.66) by MAIL.fer.hr (161.53.72.233) with Microsoft SMTP Server (TLS) id 14.2.342.3; Sun, 4 May 2014 11:27:42 +0200 Date: Sun, 4 May 2014 11:28:07 +0200 From: Marko Zec To: Adrian Chadd Subject: Re: Leaving the Desktop Market Message-ID: <20140504112807.1136d108@x23> In-Reply-To: References: <3F7430D7-3C0F-43E1-8EBD-8AA4F701497C@FreeBSD.org> <20140503155745.GA2457@La-Habana> <20140503192305.GA1847@La-Habana> <536592D1.7080403@freebsd.org> Organization: FER X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.19; amd64-portbld-freebsd9.1) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [31.147.125.66] Cc: Kevin Oberman , "freebsd-hackers@freebsd.org" , Nathan Whitehorn , Jordan Hubbard , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 09:28:58 -0000 On Sun, 4 May 2014 01:13:48 -0700 Adrian Chadd wrote: > [snip] > > ok, how about this to start with: > > http://people.freebsd.org/~adrian/power/20140504-powerd-1.diff > > This defaults powerd to set minfreq to 50% of the hardware available > maxfreq, unless minfreq is explicitly given on the command line. As already noted earlier in this thread, if you disable hint.p4tcc.0.disabled=1 hint.acpi_throttle.0.disabled=1 then the kernel does not even expose expose those silly minimum "frequencies", and your problem goes away without patching powerd. A more reasonable and simpler patch would be to disable the two offending throttling drivers by default, I really cannot see a single reason why do we need them at all, less why they are enabled. Marko > This should at least fix some of the really silly default decisions > made by powerd during idle. > > This code sorely needs an update to the cpufreq_*() routines. That can > come later. > > Does anyone have any issues? > > > > -a > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun May 4 15:55:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E915AE8; Sun, 4 May 2014 15:55:57 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B671411C4; Sun, 4 May 2014 15:55:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s44Ftk9m064089; Mon, 5 May 2014 01:55:46 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 5 May 2014 01:55:46 +1000 (EST) From: Ian Smith To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: Message-ID: <20140505011654.O11699@sola.nimnet.asn.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 15:55:57 -0000 On Sun, 4 May 2014 01:27:38 -0700, Adrian Chadd wrote: > Hi, > > I'd like to propose flipping a few things: > > * Flipping the default lid state to S3. I think ACPI suspend/resume > seems to work well enough these days and I've not met anyone lately > who expects the default from their laptop to be "stay awake with the > lid shut." Meet me; sitting here right now with the X200 lid closed, backlight off, idling but 'working' via ssh. Laptops running as (mostly) headless servers would normally run with lid down - been doing this for years, keeps the cockroaches out, mostly :) Anyway, this would be a very poor default on any machine that didn't resume completely well every time. 'Seems to work well enough' on a still fairly limited range of laptops, I gather. I've yet to get this X200 (on stable/9) to suspend without losing USB - after the _second_ suspend, unlike yours - and besides it's not a big deal to set it to S3 when desired. Maybe an rc.conf knob like 'suspend_on_lid="YES"' ono rather than requiring sysctl.conf setting? > * Save chip bugs that we should add workarounds for, we should be OK > to enter lower sleep states when idling. Flipping this may expose some > further crazy driver, platform or timer bugs, but they again likely > should be fixed. This seems likely less controversial, and should suit most, probably. An Nate Lawson said often many years ago, we really need some sort of profile mechanism to describe as a package different ACPI and power settings for different brands / models / usage cases, plugins if you like; then 'resumes_reliably' could condition stuff. But I digress .. 2c, and not assuming myself to be any sort of 'average user', Ian > what do people think? > > > -a > > > Index: etc/defaults/rc.conf > =================================================================== > --- etc/defaults/rc.conf (revision 265255) > +++ etc/defaults/rc.conf (working copy) > @@ -642,9 +642,9 @@ > devfs_set_rulesets="" # A list of /mount/dev=ruleset_name settings to > # apply (must be mounted already, i.e. fstab(5)) > devfs_load_rulesets="YES" # Enable to always load the default rulesets > -performance_cx_lowest="HIGH" # Online CPU idle state > +performance_cx_lowest="Cmax" # Online CPU idle state > performance_cpu_freq="NONE" # Online CPU frequency > -economy_cx_lowest="HIGH" # Offline CPU idle state > +economy_cx_lowest="Cmax" # Offline CPU idle state > economy_cpu_freq="NONE" # Offline CPU frequency > virecover_enable="YES" # Perform housekeeping for the vi(1) editor > ugidfw_enable="NO" # Load mac_bsdextended(4) rules on boot > Index: sys/dev/acpica/acpi.c > =================================================================== > --- sys/dev/acpica/acpi.c (revision 265255) > +++ sys/dev/acpica/acpi.c (working copy) > @@ -620,11 +620,12 @@ > > /* > * Dispatch the default sleep state to devices. The lid switch is set > - * to UNKNOWN by default to avoid surprising users. > + * to S3 to mirror what everything else iBook and later does. > */ > sc->acpi_power_button_sx = acpi_sleep_states[ACPI_STATE_S5] ? > ACPI_STATE_S5 : ACPI_STATE_UNKNOWN; > - sc->acpi_lid_switch_sx = ACPI_STATE_UNKNOWN; > + sc->acpi_lid_switch_sx = acpi_sleep_states[ACPI_STATE_S3] ? > + ACPI_STATE_S3 : ACPI_STATE_UNKNOWN; > sc->acpi_standby_sx = acpi_sleep_states[ACPI_STATE_S1] ? > ACPI_STATE_S1 : ACPI_STATE_UNKNOWN; > sc->acpi_suspend_sx = acpi_sleep_states[ACPI_STATE_S3] ? > _______________________________________________ > freebsd-acpi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-acpi > To unsubscribe, send any mail to "freebsd-acpi-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Sun May 4 17:36:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E667E5F; Sun, 4 May 2014 17:36:08 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2607:f2f8:a528::3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id 727DC1DBE; Sun, 4 May 2014 17:36:08 +0000 (UTC) Received: from [IPv6:2601:9:8280:426:bc6f:4b2:eb90:358c] (unknown [IPv6:2601:9:8280:426:bc6f:4b2:eb90:358c]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 9BAB739827; Sun, 4 May 2014 10:36:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=felyko.com; s=mail; t=1399224967; bh=50NS81vUSCzx6I37i5p+nnHnhNaQUeX53nEr1PxBZ8o=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=jt2cyyrL2UggOxI6aRiGGB2pQNzTCgFH4iWamCdwuJfZiHlbhBdGTDnx2bIlxv9Je V6HwLlLqiq9cfnDHKr98pkAS5sZhyeNpTnUTxs339vijkF8IDx3P5zKBB8HU3HLyGE KUW7WbexxhYKdSogNXasBpQhJZzwb5+GBskaFJOE= Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Rui Paulo In-Reply-To: Date: Sun, 4 May 2014 10:36:06 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Adrian Chadd X-Mailer: Apple Mail (2.1874) Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 17:36:08 -0000 On May 4, 2014, at 1:27, Adrian Chadd wrote: > * Flipping the default lid state to S3. I think ACPI suspend/resume > seems to work well enough these days and I've not met anyone lately > who expects the default from their laptop to be "stay awake with the > lid shut." The sysctl is really just a hack. We should have a much better = mechanism for integrating our ACPI with the X11 desktop environments. = GNOME/KDE/Mate don't understand our sysctl and get confused easily.=20 You can turn it on by default, but I'm sure ACPI suspend/resume is not = working well enough like you say. How many laptops have you tested? = For completeness, how many desktops? =20 There are bunch of ports kernel modules that will crash your system if = you suspend. VirtualBox is one of them. > * Save chip bugs that we should add workarounds for, we should be OK > to enter lower sleep states when idling. Flipping this may expose some > further crazy driver, platform or timer bugs, but they again likely > should be fixed. They are still not fixed. Some of these problems are not in FreeBSD = though and I don't expect us to be able to work around them. We should = try to identify systems where C3 has surprising effects and blacklist = them. -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Sun May 4 20:00:53 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C470115A; Sun, 4 May 2014 20:00:53 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 8629F1967; Sun, 4 May 2014 20:00:53 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 33BAA15B2; Sun, 4 May 2014 20:00:46 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.8/8.14.8) with ESMTP id s44K0iME002224; Sun, 4 May 2014 20:00:45 GMT (envelope-from phk@phk.freebsd.dk) To: Ian Smith Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-reply-to: <20140505011654.O11699@sola.nimnet.asn.au> From: "Poul-Henning Kamp" References: <20140505011654.O11699@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1 Date: Sun, 04 May 2014 20:00:44 +0000 Message-ID: <2223.1399233644@critter.freebsd.dk> Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 20:00:53 -0000 In message <20140505011654.O11699@sola.nimnet.asn.au>, Ian Smith writes: >On Sun, 4 May 2014 01:27:38 -0700, Adrian Chadd wrote: >'Seems to work well enough' on a still fairly limited range of laptops, >I gather. I'd say. I havn't seen suspend/resume work for ages on my T4xx laptops and as far as I recall it never worked on this T430s at all. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sun May 4 20:51:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8A60DD9; Sun, 4 May 2014 20:51:21 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9737B1D2C; Sun, 4 May 2014 20:51:21 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id i13so1648875qae.35 for ; Sun, 04 May 2014 13:51:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ofv7A+WQRti4Aj+Iqf5bgsjGskdBWIQvLkUsAnxnq+E=; b=xrHaN60NDzdd7s/eQVzLKEzHm4vHEYXrC4u3375NS2icF6SikjJePe3HSHVOctEDVO prrQqQpV8psTFFzjBqPx4yk0rE0ocfWH60rttm2nJWR0V6nqf/af+qy/UQqhnMOSVRRj CFMMVzJOr3E+qklq5aBSprtb2kHcSEgNBUt0/ZM/0i5VAIxjdkX2t6xZRFwKqrSsWAiZ Gjbvj6apcDFjvQUFgOsHv7DpsjZx15kXb+W1p13T2KiSO34nsqf+tPeqWwgt3OD519u7 lLZFfiPBYDRtWJs5Y+nv4ApMT0j5uxz9vKzyalgZQpU9sqp+rrkyhqIDvxHktHyY1cfx md/Q== MIME-Version: 1.0 X-Received: by 10.140.109.70 with SMTP id k64mr37307699qgf.92.1399236680704; Sun, 04 May 2014 13:51:20 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 13:51:20 -0700 (PDT) In-Reply-To: <2223.1399233644@critter.freebsd.dk> References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> Date: Sun, 4 May 2014 13:51:20 -0700 X-Google-Sender-Auth: pTQvF9ec-uJIPoOdX7WCkgV2oJ0 Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Poul-Henning Kamp Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Ian Smith , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 20:51:22 -0000 On 4 May 2014 13:00, Poul-Henning Kamp wrote: > In message <20140505011654.O11699@sola.nimnet.asn.au>, Ian Smith writes: >>On Sun, 4 May 2014 01:27:38 -0700, Adrian Chadd wrote: > >>'Seems to work well enough' on a still fairly limited range of laptops, >>I gather. > > I'd say. > > I havn't seen suspend/resume work for ages on my T4xx laptops and > as far as I recall it never worked on this T430s at all. I've tested it (-HEAD) on: * T43 * T60 * T60p * T400 * T500 * T420 * X220 I'm actively using the T60, T400 and X220 right now. I'd like to see it working on more laptops and I've worked with various people in the past to try and fix whichever resume issues I find. I'm happy leaving this as-is but at some point something has to be bitten and the bugs in the drivers / ACPI stuff need to be fixed. :) -a From owner-freebsd-arch@FreeBSD.ORG Sun May 4 22:50:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 499BC996; Sun, 4 May 2014 22:50:49 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C72A816AC; Sun, 4 May 2014 22:22:08 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 1041EB80B5; Mon, 5 May 2014 00:22:07 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id E555528497; Mon, 5 May 2014 00:22:06 +0200 (CEST) Date: Mon, 5 May 2014 00:22:06 +0200 From: Jilles Tjoelker To: Sergey Kandaurov Subject: Re: safer alternative for pfind and pget? Message-ID: <20140504222206.GB65318@stack.nl> References: <20140502220925.GB28158@dft-labs.eu> <20140502223609.GU43976@funkthat.com> <20140503091213.GA3876@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Mikolaj Golub , John-Mark Gurney , Mateusz Guzik , FreeBSD Arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 22:50:49 -0000 On Sat, May 03, 2014 at 01:59:12PM +0400, Sergey Kandaurov wrote: > On 3 May 2014 13:12, Mikolaj Golub wrote: > > On Fri, May 02, 2014 at 03:36:09PM -0700, John-Mark Gurney wrote: > >> Mateusz Guzik wrote this message on Sat, May 03, 2014 at 00:09 +0200: > >> > - pget which provides limited (mostly negative) filtering > >> as pget is undocumented, please make it part of this project to > >> document it.. > > pluknet had a man page for pget long time ago, reviewed by kib. I > > thought it was committed. Don't know why it did not happen. > This is something I have for pget(9). > If it is good enough, I'll commit it. Thanks for adding this man page. The description for PGET_NOTWEXIT is incomplete. It not only rejects processes with P_WEXIT set, but also zombies. In other words, if PGET_NOTWEXIT is not set, zombies are acceptable. This is really wrong since e.g. procstat -k should work on processes with P_WEXIT set but not on zombies, but it should probably be documented the way it is. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sun May 4 22:59:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0771B5A; Sun, 4 May 2014 22:59:26 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8234B1AE4; Sun, 4 May 2014 22:59:26 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s44MxMgI009250 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 4 May 2014 16:59:22 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s44MxLbp009247; Sun, 4 May 2014 16:59:22 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Sun, 4 May 2014 16:59:21 -0600 (MDT) From: Warren Block To: Rui Paulo Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Sun, 04 May 2014 16:59:22 -0600 (MDT) Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 May 2014 22:59:27 -0000 On Sun, 4 May 2014, Rui Paulo wrote: > On May 4, 2014, at 1:27, Adrian Chadd wrote: > >> * Flipping the default lid state to S3. I think ACPI suspend/resume >> seems to work well enough these days and I've not met anyone lately >> who expects the default from their laptop to be "stay awake with the >> lid shut." > > The sysctl is really just a hack. We should have a much better mechanism for integrating our ACPI with the X11 desktop environments. GNOME/KDE/Mate don't understand our sysctl and get confused easily. > > You can turn it on by default, but I'm sure ACPI suspend/resume is not working well enough like you say. How many laptops have you tested? For completeness, how many desktops? Suspend works for me. It's the resume part, on Dell and Acer systems at least, that does not work at all. > There are bunch of ports kernel modules that will crash your system if you suspend. VirtualBox is one of them. > >> * Save chip bugs that we should add workarounds for, we should be OK >> to enter lower sleep states when idling. Flipping this may expose some >> further crazy driver, platform or timer bugs, but they again likely >> should be fixed. > > They are still not fixed. Some of these problems are not in FreeBSD though and I don't expect us to be able to work around them. We should try to identify systems where C3 has surprising effects and blacklist them. If there were an easy-to-run test, many would be happy to report results. From owner-freebsd-arch@FreeBSD.ORG Mon May 5 00:25:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 961BC8BB; Mon, 5 May 2014 00:25:51 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 447EE1272; Mon, 5 May 2014 00:25:51 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id j5so60365qga.0 for ; Sun, 04 May 2014 17:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=irv2gEMJA82oBZO0fa6zl5ndmiu60OV+43+t46tO1zM=; b=oHWEkhXuG5Qz5ji/QwNo5pS+DvT5giIwwejWAwewLLPPsq8VBmv1F5bmoqoKLVyYGl HC5r5TKeNaKgGVJChhJIV+GsShhjrP5Evx8r4YzXawGrn/4gN0gdoNllYuJrwuJkUqaD Qkq7aIsf5Db1zK2R6QzionmVENGFohnbGre1XPPRKUk/3GOMg96jDI1jbG6G3CA/9p3c A8Nu/m1bvcKzwIqzbXfBRYpGPfNMBIPdXirOzfBdlgALQ+93V5vEzLSZ6McS4dGyvNdo sg/AImeqOXS9wJawdcw7YIBlLN8g+njcSovrg95gEcheEKC/fE1Sl2E4LuRpuRI5vbs0 X0Gg== MIME-Version: 1.0 X-Received: by 10.229.198.2 with SMTP id em2mr85033qcb.21.1399249550404; Sun, 04 May 2014 17:25:50 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 17:25:50 -0700 (PDT) In-Reply-To: References: Date: Sun, 4 May 2014 17:25:50 -0700 X-Google-Sender-Auth: 0PpzMHdxqj9hlUqdClQ-a1l-rJQ Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Warren Block Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 00:25:51 -0000 [snip] The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=Cmax" and then use stuff. The problem is that we're not getting anywhere near enough exposure to this kind of stuff because we don't have it on by default and we don't have an active QA group with ridiculous amounts of hardware. So, I'd like to flip it on and then start blacklisting devices that actively don't work in halt states above C1. We're never going to cross this bridge fully if we leave things at C1 out of fear. I'm only suggesting we do this on -HEAD. If it's too scary then we can always flip the default back to C1 for 11.0-RELEASE. -a From owner-freebsd-arch@FreeBSD.ORG Mon May 5 00:53:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7791EB11; Mon, 5 May 2014 00:53:24 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 23688149F; Mon, 5 May 2014 00:53:23 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.8/8.14.8) with ESMTP id s450rLG0009836 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 4 May 2014 18:53:21 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.8/8.14.8/Submit) with ESMTP id s450rLNq009833; Sun, 4 May 2014 18:53:21 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Sun, 4 May 2014 18:53:21 -0600 (MDT) From: Warren Block To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Sun, 04 May 2014 18:53:21 -0600 (MDT) Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 00:53:24 -0000 On Sun, 4 May 2014, Adrian Chadd wrote: > [snip] > > The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=Cmax" and then use stuff. It's that "use stuff" step that would preferably be automated. Is the failure mode a lockup, or could a program detect problems? The idea is to get lots of feedback, fast. > The problem is that we're not getting anywhere near enough exposure to > this kind of stuff because we don't have it on by default and we don't > have an active QA group with ridiculous amounts of hardware. > > So, I'd like to flip it on and then start blacklisting devices that > actively don't work in halt states above C1. We're never going to > cross this bridge fully if we leave things at C1 out of fear. > > I'm only suggesting we do this on -HEAD. If it's too scary then we can > always flip the default back to C1 for 11.0-RELEASE. Seems reasonable to me. From owner-freebsd-arch@FreeBSD.ORG Mon May 5 01:07:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1366F66; Mon, 5 May 2014 01:07:34 +0000 (UTC) Received: from mail-qa0-x229.google.com (mail-qa0-x229.google.com [IPv6:2607:f8b0:400d:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EC6B15A2; Mon, 5 May 2014 01:07:34 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id dc16so4844912qab.28 for ; Sun, 04 May 2014 18:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=C/RiJASl00e/cggkv6Iocpw27MfRAq7Xeem0vxC7+Oo=; b=v02bDV7KTQBIyS1gZ0vBCZCAUknQuz+icZWhX1i0lncUhst41mJN4bj17NKLwFTZIF yxD9SYBAvQOR9XpDP+UeN3sP1pf6RYDyDBuiipr0glAXklM/BiTHFVktpqAY6kAffsaI xdksE8JQQpNVxRA6hwl535c8ZxLHI1A85jl45ZPo/8rRQr6/GUCvCJIZxk8DIBaR209F lIZxWYlZe0JFDNMSqudh66ipGP/jhaE8fdmfsdNUs2ajRM8lSbqbidgN6wHqFc8/GWV0 u7Ir8cOy0WXhXlaqlt7P+rkPxCChvGzpKF0IsZ0tMUlnEc+bIdZqknQMwFMQpfd0jXox /rwg== MIME-Version: 1.0 X-Received: by 10.224.38.138 with SMTP id b10mr467403qae.98.1399252053166; Sun, 04 May 2014 18:07:33 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 18:07:33 -0700 (PDT) In-Reply-To: References: Date: Sun, 4 May 2014 18:07:33 -0700 X-Google-Sender-Auth: bYHhbVFb43FcK3XkSBMH78nm_KI Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Warren Block Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 01:07:35 -0000 On 4 May 2014 17:53, Warren Block wrote: > On Sun, 4 May 2014, Adrian Chadd wrote: > >> [snip] >> >> The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=Cmax" and then use >> stuff. > > > It's that "use stuff" step that would preferably be automated. Is the > failure mode a lockup, or could a program detect problems? The idea is to > get lots of feedback, fast. The lack of a general test suite is a bigger problem. :-) -a From owner-freebsd-arch@FreeBSD.ORG Mon May 5 05:18:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C973A28; Mon, 5 May 2014 05:18:15 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F6D21FB4; Mon, 5 May 2014 05:18:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s455I3ir092634; Mon, 5 May 2014 15:18:03 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 5 May 2014 15:18:03 +1000 (EST) From: Ian Smith To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: Message-ID: <20140505135809.Y11699@sola.nimnet.asn.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Warren Block , Kevin Oberman , "freebsd-acpi@freebsd.org" , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 05:18:15 -0000 On Sun, 4 May 2014 17:25:50 -0700, Adrian Chadd wrote: > [snip] > > The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=Cmax" and then use stuff. This doesn't work, on stable/9 at least, in that it only sets cpu.0 .. you need to set sysctl hw.acpi.cpu.cx_lowest - which rather surprised me, although that's what /etc/rc.d/power_profile has always set. I guess it's only cpu.0.freq that still sets all CPUs in sync. Further, rather than -economy_cx_lowest="HIGH" # Offline CPU idle state +economy_cx_lowest="Cmax" # Offline CPU idle state you could use "LOW" which already sets it to "Cmax", though on 8.2: lowest_value="`(sysctl -n dev.cpu.0.cx_supported | \ awk '{ print "C" split($0, a) }' -) 2> /dev/null`" I'm not sure what the point of setting cx_lowest to C8 is on a machine where lowest cx_supported is C3, but it seems to do no harm on mine. Where does C8 come from anyway? Is that the lowest on any known hardware, or the lowest the ACPI spec supports? root@x200:~ # sysctl -a | grep cx hw.acpi.cpu.cx_lowest: C3 dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.0.cx_lowest: C3 dev.cpu.0.cx_usage: 0.00% 0.79% 99.20% last 35us dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.1.cx_lowest: C3 dev.cpu.1.cx_usage: 0.08% 1.59% 98.32% last 62us root@x200:~ # sysctl dev.cpu.0.cx_lowest=Cmax dev.cpu.0.cx_lowest: C3 -> C8 root@x200:~ # sysctl -a | grep cx hw.acpi.cpu.cx_lowest: C3 dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.0.cx_lowest: C8 <<<< dev.cpu.0.cx_usage: 0.00% 5.66% 94.33% last 73us dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.1.cx_lowest: C3 <<<< dev.cpu.1.cx_usage: 0.07% 1.91% 98.01% last 2us root@x200:~ # sysctl dev.cpu.1.cx_lowest=C2 dev.cpu.1.cx_lowest: C3 -> C2 root@x200:~ # sysctl -a | grep cx hw.acpi.cpu.cx_lowest: C3 <<<< dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.0.cx_lowest: C8 <<<< dev.cpu.0.cx_usage: 0.00% 0.19% 99.80% last 474us dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.1.cx_lowest: C2 <<<< dev.cpu.1.cx_usage: 3.47% 96.52% 0.00% last 1us root@x200:~ # sysctl hw.acpi.cpu.cx_lowest=Cmax hw.acpi.cpu.cx_lowest: C3 -> C8 root@x200:~ # sysctl -a | grep cx hw.acpi.cpu.cx_lowest: C8 dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_usage: 0.00% 3.05% 96.94% last 351us dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 dev.cpu.1.cx_lowest: C8 dev.cpu.1.cx_usage: 0.00% 7.05% 92.94% last 2us I've long run with C3 in AC power mode without issue, but don't know if that's true for all machines; all yours and mine are Thinkpads! > The problem is that we're not getting anywhere near enough exposure to > this kind of stuff because we don't have it on by default and we don't > have an active QA group with ridiculous amounts of hardware. > > So, I'd like to flip it on and then start blacklisting devices that > actively don't work in halt states above C1. We're never going to > cross this bridge fully if we leave things at C1 out of fear. > > I'm only suggesting we do this on -HEAD. If it's too scary then we can > always flip the default back to C1 for 11.0-RELEASE. Yeah, I think this likely uncontroversial - unlike lid state S3 - and I don't recall hearing about machines that fail below C1 if lower states are shown as available. As you say, this might shake these out. But where would you put such a blacklist? Someting like USB quirks? cheers, Ian From owner-freebsd-arch@FreeBSD.ORG Mon May 5 05:22:14 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B855C15; Mon, 5 May 2014 05:22:14 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8C731FD9; Mon, 5 May 2014 05:22:13 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id i13so1903473qae.7 for ; Sun, 04 May 2014 22:22:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=RSwHUOUUlZI4ZpFAxg8ANDO8ZVVxL/FluJ6ckofBbX4=; b=HoT/CJb/EOS9URlzkA2qpJDpLQSgfS9I5RrR5khv0TOAUhr/9kJWaJKhQowBJ/Np4Z TfTiIGRDHxKCUHxK/zXUytXlgZmmRViraOqKx7IENias3tKPVqire8VEJcN+hMUga396 p06JNTh5jFAIGkgb5UbDnMHXAWb5WPY2glWuMCq9rJtE9749iCcAE2q3MTmFGj4OMVNs Ee/OtQR/STu7h8KrC8Yiwc+2hQGggTNUpUSXbBCX4mhYFqQrStUHnmsLApbtcimNCXAf JUja8GGIpQAU8Atx8EqVS1JREHpK2Ln5tQpMfYONATF4Oj86JrTRnSBfDep15xJKWSfl IU5Q== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr42105697qaf.26.1399267332980; Sun, 04 May 2014 22:22:12 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 4 May 2014 22:22:12 -0700 (PDT) In-Reply-To: <20140505135809.Y11699@sola.nimnet.asn.au> References: <20140505135809.Y11699@sola.nimnet.asn.au> Date: Sun, 4 May 2014 22:22:12 -0700 X-Google-Sender-Auth: u-RXGYRcF7mwBWtF_rrFP10VxR4 Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Ian Smith Content-Type: text/plain; charset=UTF-8 Cc: Warren Block , Kevin Oberman , "freebsd-acpi@freebsd.org" , Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 05:22:14 -0000 On 4 May 2014 22:18, Ian Smith wrote: > On Sun, 4 May 2014 17:25:50 -0700, Adrian Chadd wrote: > > [snip] > > > > The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=Cmax" and then use stuff. > > This doesn't work, on stable/9 at least, in that it only sets cpu.0 .. > you need to set sysctl hw.acpi.cpu.cx_lowest - which rather surprised > me, although that's what /etc/rc.d/power_profile has always set. I > guess it's only cpu.0.freq that still sets all CPUs in sync. Yeah. Sorry, my mistake. One should just do the rc.conf change and reboot. > Further, rather than > -economy_cx_lowest="HIGH" # Offline CPU idle state > +economy_cx_lowest="Cmax" # Offline CPU idle state > you could use "LOW" which already sets it to "Cmax", though on 8.2: > lowest_value="`(sysctl -n dev.cpu.0.cx_supported | \ > awk '{ print "C" split($0, a) }' -) 2> /dev/null`" > > I'm not sure what the point of setting cx_lowest to C8 is on a machine > where lowest cx_supported is C3, but it seems to do no harm on mine. Well, it's just convenient to set it to that. It's the same as Cmax. > Where does C8 come from anyway? Is that the lowest on any known > hardware, or the lowest the ACPI spec supports? Not sure. It's what's chosen when you use Cmax. :-) > root@x200:~ # sysctl -a | grep cx > hw.acpi.cpu.cx_lowest: C3 > dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.0.cx_lowest: C3 > dev.cpu.0.cx_usage: 0.00% 0.79% 99.20% last 35us > dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.1.cx_lowest: C3 > dev.cpu.1.cx_usage: 0.08% 1.59% 98.32% last 62us > > root@x200:~ # sysctl dev.cpu.0.cx_lowest=Cmax > dev.cpu.0.cx_lowest: C3 -> C8 > root@x200:~ # sysctl -a | grep cx > hw.acpi.cpu.cx_lowest: C3 > dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.0.cx_lowest: C8 <<<< > dev.cpu.0.cx_usage: 0.00% 5.66% 94.33% last 73us > dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.1.cx_lowest: C3 <<<< > dev.cpu.1.cx_usage: 0.07% 1.91% 98.01% last 2us > > root@x200:~ # sysctl dev.cpu.1.cx_lowest=C2 > dev.cpu.1.cx_lowest: C3 -> C2 > root@x200:~ # sysctl -a | grep cx > hw.acpi.cpu.cx_lowest: C3 <<<< > dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.0.cx_lowest: C8 <<<< > dev.cpu.0.cx_usage: 0.00% 0.19% 99.80% last 474us > dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.1.cx_lowest: C2 <<<< > dev.cpu.1.cx_usage: 3.47% 96.52% 0.00% last 1us > > root@x200:~ # sysctl hw.acpi.cpu.cx_lowest=Cmax > hw.acpi.cpu.cx_lowest: C3 -> C8 > root@x200:~ # sysctl -a | grep cx > hw.acpi.cpu.cx_lowest: C8 > dev.cpu.0.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.0.cx_lowest: C8 > dev.cpu.0.cx_usage: 0.00% 3.05% 96.94% last 351us > dev.cpu.1.cx_supported: C1/1/1 C2/2/1 C3/3/57 > dev.cpu.1.cx_lowest: C8 > dev.cpu.1.cx_usage: 0.00% 7.05% 92.94% last 2us > > I've long run with C3 in AC power mode without issue, but don't know if > that's true for all machines; all yours and mine are Thinkpads! I've only had problems on an older Atom. That I'll bring with me to bsdcan to finally show mav. Linux has had the same problem with this particular Atom and timekeeping. ;( > > The problem is that we're not getting anywhere near enough exposure to > > this kind of stuff because we don't have it on by default and we don't > > have an active QA group with ridiculous amounts of hardware. > > > > So, I'd like to flip it on and then start blacklisting devices that > > actively don't work in halt states above C1. We're never going to > > cross this bridge fully if we leave things at C1 out of fear. > > > > I'm only suggesting we do this on -HEAD. If it's too scary then we can > > always flip the default back to C1 for 11.0-RELEASE. > > Yeah, I think this likely uncontroversial - unlike lid state S3 - and I > don't recall hearing about machines that fail below C1 if lower states > are shown as available. As you say, this might shake these out. > > But where would you put such a blacklist? Someting like USB quirks? I'm not sure yet. Maybe make it userland and as part of the rc / power scripts. That way Cmax gets turned into C1. -a From owner-freebsd-arch@FreeBSD.ORG Mon May 5 05:45:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06FBFE6D; Mon, 5 May 2014 05:45:13 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id BEADA1164; Mon, 5 May 2014 05:45:12 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id D715715AF; Mon, 5 May 2014 05:45:10 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.8/8.14.8) with ESMTP id s455jAbt022968; Mon, 5 May 2014 05:45:10 GMT (envelope-from phk@phk.freebsd.dk) To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-reply-to: From: "Poul-Henning Kamp" References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 05 May 2014 05:45:10 +0000 Message-ID: <22967.1399268710@critter.freebsd.dk> Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Ian Smith , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 05:45:13 -0000 In message , Adrian Chadd writes: >> I havn't seen suspend/resume work for ages on my T4xx laptops and >> as far as I recall it never worked on this T430s at all. > >I've tested it (-HEAD) on: > >* T43 >* T60 >* T60p >* T400 >* T500 >* T420 >* X220 I'm building head as we speak, I'll test once it's done. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon May 5 06:17:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DBD9D16A; Mon, 5 May 2014 06:17:12 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E444135F; Mon, 5 May 2014 06:17:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s456H87B094601; Mon, 5 May 2014 16:17:08 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 5 May 2014 16:17:08 +1000 (EST) From: Ian Smith To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: Message-ID: <20140505153421.W11699@sola.nimnet.asn.au> References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Kevin Oberman , Poul-Henning Kamp , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 06:17:13 -0000 On Sun, 4 May 2014 13:51:20 -0700, Adrian Chadd wrote: > On 4 May 2014 13:00, Poul-Henning Kamp wrote: > > I havn't seen suspend/resume work for ages on my T4xx laptops and > > as far as I recall it never worked on this T430s at all. > > I've tested it (-HEAD) on: > > * T43 > * T60 > * T60p > * T400 > * T500 > * T420 > * X220 > > I'm actively using the T60, T400 and X220 right now. So, is the USB not working after $n resumes on T4xx and T2xx (at least) now fixed in HEAD? If so, it would be REALLY good to MFC whatever fixed it to 10 and hopefully to 9 before 9.3. I need to do more tests on stable/9; it didn't work with the offered workarounds on 9.2-RELEASE (leaving VESA out of kernel leaves me with no screen on resume in console mode, setting sysctl dev.[ue]hci.*.wake=1 fails to even start to resume) though if that works on 10 I'd give it a go, despite Darren Pilgrim's dvd1_to_memstick script failing to have pkg use any of the local DVD packages, which is why I went back to 9.2 I know you only like working on HEAD, but unless there's API/ABI reasons preventing MFC, it would be great to have 9.3 work in this respect .. my X200 is not useful for purpose if it won't suspend/resume 100% reliably, with USB, and I don't want to run 11 on it, it's needed for developing non-FreeBSD stuff (in freepascal, if that's not a dirty word here :) and for that it needs to be rock solid while travelling from place to place. > I'd like to see it working on more laptops and I've worked with > various people in the past to try and fix whichever resume issues I > find. Indeed; last I heard was last July with unsolved USB issues on your T400, and the mysterious but related 'CPU0: local APIC error 0x40' > I'm happy leaving this as-is but at some point something has to be > bitten and the bugs in the drivers / ACPI stuff need to be fixed. :) It's a bit disconcerting if fixes only ever make it into HEAD, for me. cheers, Ian (someone please sing out if I shouldn't be crossposting to -arch) From owner-freebsd-arch@FreeBSD.ORG Mon May 5 06:25:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F947264; Mon, 5 May 2014 06:25:23 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id F40DA13F0; Mon, 5 May 2014 06:25:22 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id CC23915AF; Mon, 5 May 2014 06:25:21 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.8/8.14.8) with ESMTP id s456PL1G085790; Mon, 5 May 2014 06:25:21 GMT (envelope-from phk@phk.freebsd.dk) To: Ian Smith Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-reply-to: <20140505153421.W11699@sola.nimnet.asn.au> From: "Poul-Henning Kamp" References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 05 May 2014 06:25:21 +0000 Message-ID: <85787.1399271121@critter.freebsd.dk> Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 06:25:23 -0000 In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith writes: >I need to do more tests on stable/9; it didn't work with the offered >workarounds on 9.2-RELEASE (leaving VESA out of kernel leaves me with no >screen on resume in console mode, setting sysctl dev.[ue]hci.*.wake=1 Do we have a canonical page with all the various workarounds one should attempt in order to get suspend/resume to work ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon May 5 07:00:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57C2C51D; Mon, 5 May 2014 07:00:44 +0000 (UTC) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BD8F3167E; Mon, 5 May 2014 07:00:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id s4570cAh096109; Mon, 5 May 2014 17:00:39 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Mon, 5 May 2014 17:00:38 +1000 (EST) From: Ian Smith To: Poul-Henning Kamp Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-Reply-To: <85787.1399271121@critter.freebsd.dk> Message-ID: <20140505163316.R11699@sola.nimnet.asn.au> References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 07:00:44 -0000 On Mon, 5 May 2014 06:25:21 +0000, Poul-Henning Kamp wrote: > In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith writes: > > >I need to do more tests on stable/9; it didn't work with the offered > >workarounds on 9.2-RELEASE (leaving VESA out of kernel leaves me with no > >screen on resume in console mode, setting sysctl dev.[ue]hci.*.wake=1 > > Do we have a canonical page with all the various workarounds one should > attempt in order to get suspend/resume to work ? Bits scattered all over the place. For the above there's: http://unethicalblogger.com/2013/12/03/scratchiest-neckbeard-freebsd-x200.html which refers to te tail of this thread in -usb (and -stable) http://lists.freebsd.org/pipermail/freebsd-usb/2013-July/thread.html#12242 which began with Adrian's post in http://lists.freebsd.org/pipermail/freebsd-usb/2013-June/thread.html Then there's the wiki page: https://wiki.freebsd.org/SuspendResume which leads to some more bits, lots more scattered through -acpi and -laptop over the time. There used to be lots of useful per-laptop snippets in the FLCL (freebsd laptop compatibility list) which sadly disappeared last year, but I just googled up an archive of it at http://archive.today/CVo46 latest from mid-2013. Ah sorry, spoke too soon, individual pages redirect to eg: http://laptop.bsdgroup.de/freebsd/index.html?action=list_laptop_mf&mfid=1 which is still down :( Ian From owner-freebsd-arch@FreeBSD.ORG Mon May 5 09:16:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06BAFC91; Mon, 5 May 2014 09:16:55 +0000 (UTC) Received: from smtprelay05.ispgateway.de (smtprelay05.ispgateway.de [80.67.31.94]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B3E7A1308; Mon, 5 May 2014 09:16:54 +0000 (UTC) Received: from [78.35.164.35] (helo=fabiankeil.de) by smtprelay05.ispgateway.de with esmtpsa (SSLv3:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1WhEeX-0004tS-0W; Mon, 05 May 2014 10:53:37 +0200 Date: Mon, 5 May 2014 10:53:45 +0200 From: Fabian Keil To: "freebsd-acpi@freebsd.org" Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Message-ID: <20140505105345.099e4503@fabiankeil.de> In-Reply-To: References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/gPHa/TvqAC+pMJ9jm174mKD"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 09:16:55 -0000 --Sig_/gPHa/TvqAC+pMJ9jm174mKD Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Warren Block wrote: > On Sun, 4 May 2014, Adrian Chadd wrote: > > The easy-to-run test is "sysctl dev.cpu.0.cx_lowest=3DCmax" and then us= e stuff. >=20 > It's that "use stuff" step that would preferably be automated. Is the=20 > failure mode a lockup, or could a program detect problems? The idea=20 > is to get lots of feedback, fast. One possible failure mode is that timers tick unreliably which can be detected automatically (in some cases), for example by using the DTrace script timestamp-test.d: http://www.fabiankeil.de/gehacktes/dtrace-timestamp-tests/ It should also be possible to let the kernel do this itself and log an error message and maybe optionally disable Cx states that cause problems: sysctl dev.cpu.0.cx_lowest=3DCmax-as-long-as-it-appears-to-be-working Fabian --Sig_/gPHa/TvqAC+pMJ9jm174mKD Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iEYEARECAAYFAlNnUZoACgkQBYqIVf93VJ3mbACfR/oMz44RrrpDrxX5dsJDQSxt cccAoMmC0JQIeUVcSqTgfZIwGheO7RoN =Qn19 -----END PGP SIGNATURE----- --Sig_/gPHa/TvqAC+pMJ9jm174mKD-- From owner-freebsd-arch@FreeBSD.ORG Mon May 5 09:54:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04FFC6E8; Mon, 5 May 2014 09:54:23 +0000 (UTC) Received: from mail.0x20.net (mail.0x20.net [217.69.76.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B2C7916BE; Mon, 5 May 2014 09:54:22 +0000 (UTC) Received: from e-new.0x20.net (mail.0x20.net [IPv6:2001:aa8:fffb:1::3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id AE5F06A600D; Mon, 5 May 2014 11:54:18 +0200 (CEST) Received: from e-new.0x20.net (localhost [127.0.0.1]) by e-new.0x20.net (8.14.7/8.14.7) with ESMTP id s459sIYu068431; Mon, 5 May 2014 11:54:18 +0200 (CEST) (envelope-from lars@e-new.0x20.net) Received: (from lars@localhost) by e-new.0x20.net (8.14.7/8.14.7/Submit) id s459sGWd067448; Mon, 5 May 2014 11:54:16 +0200 (CEST) (envelope-from lars) Date: Mon, 5 May 2014 11:54:16 +0200 From: Lars Engels To: Ian Smith Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Message-ID: <20140505095416.GF1451@e-new.0x20.net> References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <20140505153421.W11699@sola.nimnet.asn.au> X-Editor: VIM - Vi IMproved 7.4 X-Operation-System: FreeBSD 8.4-RELEASE-p4 User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Kevin Oberman , Adrian Chadd , "freebsd-arch@freebsd.org" , Poul-Henning Kamp , "freebsd-acpi@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 09:54:23 -0000 On Mon, May 05, 2014 at 04:17:08PM +1000, Ian Smith wrote: > On Sun, 4 May 2014 13:51:20 -0700, Adrian Chadd wrote: > > On 4 May 2014 13:00, Poul-Henning Kamp wrote: >=20 > > > I havn't seen suspend/resume work for ages on my T4xx laptops and > > > as far as I recall it never worked on this T430s at all. > >=20 > > I've tested it (-HEAD) on: > >=20 > > * T43 > > * T60 > > * T60p > > * T400 > > * T500 > > * T420 > > * X220 > >=20 > > I'm actively using the T60, T400 and X220 right now. I can also confirm that suspend/resume works out of the box on my X200 and= =20 on several T61 I used with 10.0-RELEASE, even without newcons the T61 woke up. > I know you only like working on HEAD, but unless there's API/ABI reasons= =20 > preventing MFC, it would be great to have 9.3 work in this respect .. my= =20 > X200 is not useful for purpose if it won't suspend/resume 100% reliably,= =20 > with USB, and I don't want to run 11 on it, it's needed for developing=20 > non-FreeBSD stuff (in freepascal, if that's not a dirty word here :) and= =20 > for that it needs to be rock solid while travelling from place to place. Tyler Croy got it working on his X200 which did not work for me: http://unethicalblogger.com/2013/12/03/scratchiest-neckbeard-freebsd-x200.h= tml I worked around around this with an el cheapo USB 3.0 PCI-Express card which fits nicely into the card slot and works after every resume. From owner-freebsd-arch@FreeBSD.ORG Mon May 5 16:26:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D1C33BE; Mon, 5 May 2014 16:26:09 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44EF610F; Mon, 5 May 2014 16:26:09 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A27A7B917; Mon, 5 May 2014 12:26:07 -0400 (EDT) From: John Baldwin To: freebsd-acpi@freebsd.org Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Date: Mon, 5 May 2014 11:09:39 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405051109.39345.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 05 May 2014 12:26:07 -0400 (EDT) Cc: Kevin Oberman , Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 16:26:09 -0000 On Sunday, May 04, 2014 4:27:38 am Adrian Chadd wrote: > Hi, > > I'd like to propose flipping a few things: > > * Flipping the default lid state to S3. I think ACPI suspend/resume > seems to work well enough these days and I've not met anyone lately > who expects the default from their laptop to be "stay awake with the > lid shut." > * Save chip bugs that we should add workarounds for, we should be OK > to enter lower sleep states when idling. Flipping this may expose some > further crazy driver, platform or timer bugs, but they again likely > should be fixed. > > what do people think? I think the lid switch thing is premature. Even on my X220 I use a devd hook to enable it only when i915drm is loaded as resume doesn't work until that is done. I think the Cmax thing OTOH is probably more appropriate. We have several things place that should "mostly" DTRT for picking the correct timers to use. The one case I know of recently were some somewhat older systems where the HPET wasn't reliable, but the system chose to use HPET instead of LAPIC becuase the LAPIC was known to stop during C1E, etc. In this case the user just stuck with plain old C1 and forced the LAPIC timer which worked fine. However, it is hard to identify those cases. On modern systems I would expect the LAPIC to work just fine, so this problem will become less and less important as time goes on. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon May 5 16:55:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34C3B679; Mon, 5 May 2014 16:55:31 +0000 (UTC) Received: from mail-qa0-x22d.google.com (mail-qa0-x22d.google.com [IPv6:2607:f8b0:400d:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C794C3B7; Mon, 5 May 2014 16:55:30 +0000 (UTC) Received: by mail-qa0-f45.google.com with SMTP id hw13so6991671qab.4 for ; Mon, 05 May 2014 09:55:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hJqYB5/TfZWf3v/eNWmemNQi5qdUZ01FyQLgHFnBJO4=; b=CLBbcxaVE4bMmTD/V+4f6ObYjtXVyMsrPW4HCUmFnGY14v/X7DCzzNn59SvVbvlevP HPIl2A+caO1lCJ5jLYIsnIXUcQJewc5TC2GbNJpJGDgfSDP6dZLvCBnsHWkOlx+pV+lD 1q4NPPzEQRa3xZhvjEISUegvSg7hW0saLIrGWGGqQhppgGiqoLNKkhy9zxDE8GTz3qG/ GTv5yns/mc7u6VxERepnwg70NDhIH3KKVnEeO4LrftJqgUQAwa4oY8BesQhD4hJ9khGM wAA7urNZCKCOw7hA2peJnbpxjH8R8WTF5Y+WYutMH46icLuuo1XzcxiaFRixV9AxDOu5 pMhg== MIME-Version: 1.0 X-Received: by 10.140.22.209 with SMTP id 75mr43396501qgn.4.1399308929590; Mon, 05 May 2014 09:55:29 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Mon, 5 May 2014 09:55:29 -0700 (PDT) In-Reply-To: <201405051109.39345.jhb@freebsd.org> References: <201405051109.39345.jhb@freebsd.org> Date: Mon, 5 May 2014 09:55:29 -0700 X-Google-Sender-Auth: qVhb-qF-sZLD18ukIF6f5h4v3NQ Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 16:55:31 -0000 On 5 May 2014 08:09, John Baldwin wrote: > On Sunday, May 04, 2014 4:27:38 am Adrian Chadd wrote: >> Hi, >> >> I'd like to propose flipping a few things: >> >> * Flipping the default lid state to S3. I think ACPI suspend/resume >> seems to work well enough these days and I've not met anyone lately >> who expects the default from their laptop to be "stay awake with the >> lid shut." >> * Save chip bugs that we should add workarounds for, we should be OK >> to enter lower sleep states when idling. Flipping this may expose some >> further crazy driver, platform or timer bugs, but they again likely >> should be fixed. >> >> what do people think? > > I think the lid switch thing is premature. Even on my X220 I use a devd > hook to enable it only when i915drm is loaded as resume doesn't work until > that is done. > > I think the Cmax thing OTOH is probably more appropriate. We have several > things place that should "mostly" DTRT for picking the correct timers to > use. The one case I know of recently were some somewhat older systems where > the HPET wasn't reliable, but the system chose to use HPET instead of LAPIC > becuase the LAPIC was known to stop during C1E, etc. In this case the user > just stuck with plain old C1 and forced the LAPIC timer which worked fine. > However, it is hard to identify those cases. On modern systems I would > expect the LAPIC to work just fine, so this problem will become less and > less important as time goes on. right. I'd rather we start finding more of these sooner rather than later. :-) -a From owner-freebsd-arch@FreeBSD.ORG Mon May 5 21:26:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A5830581; Mon, 5 May 2014 21:26:07 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7AFE08A3; Mon, 5 May 2014 21:26:07 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6EAA2B968; Mon, 5 May 2014 17:26:05 -0400 (EDT) From: John Baldwin To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Date: Mon, 5 May 2014 16:57:49 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <201405051109.39345.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201405051657.49992.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 05 May 2014 17:26:05 -0400 (EDT) Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2014 21:26:07 -0000 On Monday, May 05, 2014 12:55:29 pm Adrian Chadd wrote: > On 5 May 2014 08:09, John Baldwin wrote: > > On Sunday, May 04, 2014 4:27:38 am Adrian Chadd wrote: > >> Hi, > >> > >> I'd like to propose flipping a few things: > >> > >> * Flipping the default lid state to S3. I think ACPI suspend/resume > >> seems to work well enough these days and I've not met anyone lately > >> who expects the default from their laptop to be "stay awake with the > >> lid shut." > >> * Save chip bugs that we should add workarounds for, we should be OK > >> to enter lower sleep states when idling. Flipping this may expose some > >> further crazy driver, platform or timer bugs, but they again likely > >> should be fixed. > >> > >> what do people think? > > > > I think the lid switch thing is premature. Even on my X220 I use a devd > > hook to enable it only when i915drm is loaded as resume doesn't work until > > that is done. > > > > I think the Cmax thing OTOH is probably more appropriate. We have several > > things place that should "mostly" DTRT for picking the correct timers to > > use. The one case I know of recently were some somewhat older systems where > > the HPET wasn't reliable, but the system chose to use HPET instead of LAPIC > > becuase the LAPIC was known to stop during C1E, etc. In this case the user > > just stuck with plain old C1 and forced the LAPIC timer which worked fine. > > However, it is hard to identify those cases. On modern systems I would > > expect the LAPIC to work just fine, so this problem will become less and > > less important as time goes on. > > right. I'd rather we start finding more of these sooner rather than later. :-) The user in question found this on 9-stable with the existing defaults as the HPET was just plain broken on their system and that was unrelated to Cx states. (Rather, Cx states were only involved because worries about them are why the system chose to use HPET. Had Cx states been enabled by default, they would have had to disable those as well in addition to forcing LAPIC instead of HPET.) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue May 6 18:08:36 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBCFE3BA; Tue, 6 May 2014 18:08:36 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89CCE85; Tue, 6 May 2014 18:08:36 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so7198524qcx.41 for ; Tue, 06 May 2014 11:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ty3zJIZSWYLoxXq1SY1j4Sx2I0t57arOSXQn+vIOM4E=; b=GI1ifn8V0eqj/RHv0UxfJ4TjwONiO9kc2X7N+Y21iZ0zu/h4O2WPe3lbJQKu/pxag6 Deu35B2kDv+iAGJYqswxSsEP0tHocPC85PeNNjjob6+yBCAeCSM+vApJUhQZnvpd1e9n pa5mvmN+YrdK9zuqNKFFCgOOfVWbQllz10nFZlbdZI1Koc3N3mkfs28IV6ELl8paAoX/ UxUwJ1ivvJJZCpj5oXVhv09kNl7V1gcO1BEDYiF95jthO21cTl6STphOb9o3FZ9tXm31 BKdIJhyLCLOcf19QZeAjo1dxAjdkWUtKD6PWkcF+OrLKYxMw8sxdzSqG8q9zdUY+r4PU /Uhg== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr58210442qaf.26.1399399715673; Tue, 06 May 2014 11:08:35 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Tue, 6 May 2014 11:08:35 -0700 (PDT) In-Reply-To: <201405051657.49992.jhb@freebsd.org> References: <201405051109.39345.jhb@freebsd.org> <201405051657.49992.jhb@freebsd.org> Date: Tue, 6 May 2014 11:08:35 -0700 X-Google-Sender-Auth: gbhK2GqiysVLFEWdIbvq4S9Skxg Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 18:08:37 -0000 On 5 May 2014 13:57, John Baldwin wrote: > The user in question found this on 9-stable with the existing defaults as the > HPET was just plain broken on their system and that was unrelated to Cx states. > (Rather, Cx states were only involved because worries about them are why the > system chose to use HPET. Had Cx states been enabled by default, they would > have had to disable those as well in addition to forcing LAPIC instead of > HPET.) Hm. Sounds uncomfortable. How does Windows run on systems like this? Do the windows drivers just disable HPET and use LAPIC or worse for timing, and just ignore anything lower than C1? I'm going to flip the switch to enable Cmax on defaults/rc.conf on -HEAD today. -a From owner-freebsd-arch@FreeBSD.ORG Tue May 6 18:50:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F286CBA for ; Tue, 6 May 2014 18:50:07 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx1.freebsd.org (Postfix) with ESMTP id 07002402 for ; Tue, 6 May 2014 18:50:06 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx36.phplist.com (Postfix) with ESMTP id 2095812070 for ; Tue, 6 May 2014 19:50:06 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=phplist.com; h=date:to :from:reply-to:subject:message-id:list-unsubscribe:mime-version :content-type:content-transfer-encoding; s=s0; bh=35Eu1lAvSqkpry 01KzNE1nWmQ5Y=; b=Vx8M3I+uMHowjcaB22kZ2CzL4QkXK+/EJLTKiY4WGybc1d qrngLhcnvXBpdXhETKSvAAKHIZvacGz+3wvDSLLtsTDayW4MQIDWFBzySiNrINl5 /MZOyQBkUviiKlTiraUNjBB9BTmpBUJQk8BVTIEJZ7+dULRuvcNkaoN1YEDf0= Received: from thomas.hosted.phplist.com (mimosa [184.173.18.3]) by mx36.phplist.com (Postfix) with ESMTP id 0EF431206E for ; Tue, 6 May 2014 19:50:06 +0100 (BST) Received: from evo-hl21-1.gameservers.net [62.212.73.211] by thomas.hosted.phplist.com with HTTP; Tue, 06 May 2014 18:50:05 +0000 Date: Tue, 6 May 2014 18:50:06 +0000 To: freebsd-arch@freebsd.org From: Silverjewelryworld Reply-To: Silverjewelryworld Subject: Goodbye from our Newsletter Message-ID: X-Priority: 3 X-Mailer: PHPMailer 5.2.5 (https://github.com/Synchro/PHPMailer/) X-phpList-version: 3.0.5-hosted X-MessageID: systemmessage X-ListMember: freebsd-arch@freebsd.org Precedence: bulk Bounces-To: phlistbounces-thomas@phplist.com MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 18:50:07 -0000 =20 Goodbye from our Newsletter, sorry to see you go. You have been unsubscribed from our newsletters. This is the last email you will receive from us. We have added you to o= ur "blacklist", which means that our newsletter system will refuse to send you any other email, without manual intervention by our administrator. If there is an error in this information, you can re-subscribe: please go to http://thomas.hosted.phplist.com/lists/?p=3Dsubscribe and follow the steps. Thank you =20 =20 From owner-freebsd-arch@FreeBSD.ORG Tue May 6 18:55:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A36BEFE for ; Tue, 6 May 2014 18:55:50 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx1.freebsd.org (Postfix) with ESMTP id E6B606B4 for ; Tue, 6 May 2014 18:55:49 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx36.phplist.com (Postfix) with ESMTP id 8720F1206D for ; Tue, 6 May 2014 19:49:10 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=phplist.com; h=date:to :from:reply-to:subject:message-id:list-unsubscribe:mime-version :content-type:content-transfer-encoding; s=s0; bh=35Eu1lAvSqkpry 01KzNE1nWmQ5Y=; b=oBTUK9oJ8lQKvPcRucJcHYxdcT2b5isvJCCEvUdOUdJCGh FDk8c8C500FNl/f/3/UugHVFKU0BJ7FwicsvQwID909SBKTLwNb0tfa4eBz70OeU MzBX1gESLS9IiLalJG2uJAT/5+5yiAR++jio6xIETWjFAxffEHMkZlL9grVVI= Received: from thomas.hosted.phplist.com (mimosa [184.173.18.3]) by mx36.phplist.com (Postfix) with ESMTP id 74B1412064 for ; Tue, 6 May 2014 19:49:10 +0100 (BST) Received: from evo-hl21-1.gameservers.net [62.212.73.211] by thomas.hosted.phplist.com with HTTP; Tue, 06 May 2014 18:49:09 +0000 Date: Tue, 6 May 2014 18:49:10 +0000 To: freebsd-arch@freebsd.org From: Silverjewelryworld Reply-To: Silverjewelryworld Subject: Goodbye from our Newsletter Message-ID: <98942bed9e79f0039f6320d58a890b16@hosted.phplist.com> X-Priority: 3 X-Mailer: PHPMailer 5.2.5 (https://github.com/Synchro/PHPMailer/) X-phpList-version: 3.0.5-hosted X-MessageID: systemmessage X-ListMember: freebsd-arch@freebsd.org Precedence: bulk Bounces-To: phlistbounces-thomas@phplist.com MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 18:55:50 -0000 =20 Goodbye from our Newsletter, sorry to see you go. You have been unsubscribed from our newsletters. This is the last email you will receive from us. We have added you to o= ur "blacklist", which means that our newsletter system will refuse to send you any other email, without manual intervention by our administrator. If there is an error in this information, you can re-subscribe: please go to http://thomas.hosted.phplist.com/lists/?p=3Dsubscribe and follow the steps. Thank you =20 =20 From owner-freebsd-arch@FreeBSD.ORG Tue May 6 20:56:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 820AEFE4; Tue, 6 May 2014 20:56:41 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 57CBA286; Tue, 6 May 2014 20:56:41 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 437B6B9CB; Tue, 6 May 2014 16:56:40 -0400 (EDT) From: John Baldwin To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Date: Tue, 6 May 2014 16:37:29 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <201405051657.49992.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201405061637.30037.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 06 May 2014 16:56:40 -0400 (EDT) Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 20:56:41 -0000 On Tuesday, May 06, 2014 2:08:35 pm Adrian Chadd wrote: > On 5 May 2014 13:57, John Baldwin wrote: > > > The user in question found this on 9-stable with the existing defaults as the > > HPET was just plain broken on their system and that was unrelated to Cx states. > > (Rather, Cx states were only involved because worries about them are why the > > system chose to use HPET. Had Cx states been enabled by default, they would > > have had to disable those as well in addition to forcing LAPIC instead of > > HPET.) > > Hm. Sounds uncomfortable. How does Windows run on systems like this? > Do the windows drivers just disable HPET and use LAPIC or worse for > timing, and just ignore anything lower than C1? I have no idea. Maybe they use the RTC. :-/ Maybe the HPET on these systems works if you use it "sparingly". I believe OS X might have only used the HPET to provide the "missing" LAPIC wakeups when entering Cx for example. (Our current eventtimer system wants to always use whichever timer it picks, not switch off between them.) -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue May 6 21:40:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C48AAF47 for ; Tue, 6 May 2014 21:40:39 +0000 (UTC) Received: from mail-ob0-x236.google.com (mail-ob0-x236.google.com [IPv6:2607:f8b0:4003:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92832886 for ; Tue, 6 May 2014 21:40:39 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wn1so120908obc.27 for ; Tue, 06 May 2014 14:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ZRrOwgnNIZepSF4RHLSRvxSYBbsUc9yjXmxI0cr9qFw=; b=eY+S+3Kv1tyrcoj5k+PmwukVlxDwYRiMo6lkFO7gffvsKt8/2tn1QuS9JOdSBxSB2s ldeWZ0KCjiS4UmN9/budNPskDf/Tv1V1yFQPKlNif4CEDaHUWkMrwFAp47g4OruDt6NN MDajNNaJoeJqkFIYmrWgONaz+zUwA6DrkzYT8tpfYyostEYkO7RPvkoPCRYP0tUqfQIZ B2+xmZfEf32UMNAS2bZEFwNf+NHjPK7HZRpDHXVgpZkLp+OwFEg0omulE6fyKYMMNCUm VmEDtMguaObifzi67ygVNA848Vlsq2t4II1yph0kKOSi5Vib0YLMftXGf9dJ9SlBvc9I 3j1g== MIME-Version: 1.0 X-Received: by 10.182.5.65 with SMTP id q1mr4260087obq.74.1399412438934; Tue, 06 May 2014 14:40:38 -0700 (PDT) Received: by 10.76.23.130 with HTTP; Tue, 6 May 2014 14:40:38 -0700 (PDT) Date: Tue, 6 May 2014 17:40:38 -0400 Message-ID: Subject: RFC: PCI SR-IOV Driver interface From: Ryan Stone To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 21:40:39 -0000 PCI Single Root I/O Virtualization (SR-IOV) is an optional part of the PCIe standard that provides hardware acceleration for the virtualization of PCIe devices. When SR-IOV is in use, a function in a PCI device (known as a Physical Function, or PF) will present multiple Virtual PCI Functions (VF) on the PCI bus. These VFs are fully independent PCI devices that have access to the resources of the PF. For example, on a network interface card, VFs could transmit and receive packets independent of the PF. I've been working on FreeBSD support for SR-IOV. Because the capabilities of the VFs are very much dependent on the hardware, the PF driver (which is just a normal PCI driver) has to do a lot of the work in configuring the VFs. The SR-IOV infrastructure needs to accept configuration requests, do the work to create the VFs (including creating the PCI layer device_t objects in the kernel), and then hand things off to the PF and VF drivers to for them to do the device-specific configuration. One of my goals in this project is to have a single unified tool for configuring SR-IOV. This quite complicated because of the various capabilities that different PCI devices can have. I don't want driver maintainers to have to extend the IOV infrastructure or the configuration tool to add support for new capabilities in new hardware, so the tool needs to flexible in the type of configuration that it accepts. On the other hand, I don't want to burden driver writers with the need to do complicated parsing of configuration in the driver itself. The approach that I've taken has been to port pjd@'s nv(3) interface into the kernel. I have a functional prototype of this now but it's not production ready yet. The driver will implement a method that returns a schema defining the names and types of the configuration parameters that it accepts. Parameters can be flagged as required, in which case the infrastructure will reject the configuration if it does not contain the parameter. Alternatively, the schema can define a default value for the parameter, in which case the infrastructure will add the parameter to the configuration if the user did not specify a value for it. The SR-IOV infrastructure will also validate that every parameter had the correct type as specified in the schema, as well as rejecting any configuration that contains a parameter that is not defined in the schema at all. The goal is, as much as possible, to free the driver writers from having to do complicated verification of the configuration. That was a big wall of text, so let's look at what the interface actually looks like. int pci_setup_iov(device_t dev); This function should be called by a PF driver during device_attach() to register itself as an SR-IOV-capable driver (perhaps pci_attach_iov() would be a better name). An error from it probably shouldn't be treated as fatal, but it does mean the SR-IOV functionality won't be available. int pci_cleanup_iov(device_t dev); This function should be called during device_detach(). It's safe to call if pci_setup_iov() failed. If it fails, the detach must be aborted (the most likely cause is active VFs that must be destroyed before the PF driver can detach). PF drivers implement the following method to advertise their configuration schema: METHOD void get_iov_config_schema { device_t dev; nvlist_t *pf_schema; nvlist_t *vf_schema; } The use of the nvlist_t in the interface is somewhat unfortunate. The problem is that now every driver that includes "pci_if.h" needs to have the typedef nvlist_t defined (and I *really* don't want to modify every PCI driver in the tree...). I have a somewhat hacky workaround for the problem in my tree right now but I thought that I would highlight the issue in case people had opinions on the issue. There are separate schemas for the PF and VF. The drivers are not expected to manipulate the nvlists directly. Instead the infrastructure provides some functions for defining the schema: #define IOV_SCHEMA_HASDEFAULT (1 << 0) #define IOV_SCHEMA_REQUIRED (1 << 1) void pci_iov_schema_add_binary(nvlist_t *schema, const char *name, const char *type, uint32_t flags, uint8_t * defaultVal, size_t len); void pci_iov_schema_add_bool(nvlist_t *schema, const char *name, uint32_t flags, int defaultVal); void pci_iov_schema_add_string(nvlist_t *schema, const char *name, uint32_t flags, const char *defaultVal); void pci_iov_schema_add_uint8(nvlist_t *schema, const char *name, uint32_t flags, uint8_t defaultVal); void pci_iov_schema_add_uint16(nvlist_t *schema, const char *name, uint32_t flags, uint16_t defaultVal); void pci_iov_schema_add_uint32(nvlist_t *schema, const char *name, uint32_t flags, uint32_t defaultVal); void pci_iov_schema_add_uint64(nvlist_t *schema, const char *name, uint32_t flags, uint64_t defaultVal); A sample usage of these functions (from the ixgbe PF driver that I have been working on): static void ixgbe_get_iov_schema(device_t dev, nvlist_t *pf, nvlist_t *vf) { uint8_t null_mac[ETHER_ADDR_LEN] = {0, 0, 0, 0, 0, 0}; pci_iov_schema_add_binary(vf, "mac-addr", "mac-addr", IOV_SCHEMA_HASDEFAULT, null_mac, sizeof(null_mac)); pci_iov_schema_add_uint16(vf, "vlan", 0, 0); pci_iov_schema_add_bool(vf, "spoof-check", IOV_SCHEMA_HASDEFAULT, 1); pci_iov_schema_add_bool(vf, "allow-set-mac", IOV_SCHEMA_HASDEFAULT, 0); } This says that: - the VF accepts a parameter called "mac-addr" using the "mac-addr" type. The default value of this is 6 0 bytes (00:00:00:00:00:00) - The VF accepts an optional uint16_t parameter parameter called vlan. I chose not to set a default value because a VF could be configured to sent untagged traffic. - The VF accepts a boolean parameter called spoof-check, with defaults to true. - The VF accepts a boolean parameter called allow-set-mac, with defaults to false. I have a default value here because the VF has either be permitted to set the mac or not, and it's better to explicitly document the default values in the schema rather than encode them implicitly in the driver (the schema will be viewable using the userland configuration tool) - The PF doesn't have any configuration parameters in this driver METHOD int init_iov { device_t dev; int num_vfs; const nvlist_t *config; }; This method is called by the SR-IOV infrastructure when a request to enable SR-IOV has been received. This is called before SR-IOV is actually enabled in the hardware. The driver should use this method to do global (non per-VF) configuration of the PF. The config nvlist contains configuration parameters from the PF schema. If this method returns an error, the request is aborted. METHOD int uninit_iov { device_t dev; }; This method is called when SR-IOV is disabled, or if an enable request fails after init_iov() was called and returned without an error. METHOD int add_vf { device_t dev; int vfnum; const nvlist_t *config; }; This method is called once per VF, after SR-IOV has been enabled but before the VFs have been probed. The driver should set any VF-specific configuration in the PF at this time. The config nvlist contains parameters specified in the VF schema. For example, from the ixgbe schema example that I gave above the driver could call nvlist_get_bool(config, "allow-set-mac") to get the value of the "allow-set-mac" parameter (and this call would be guaranteed to succeed because a default value was set). On the other hand the driver would have to test for the presence of the "vlan" parameter as it was neither set as a required parameter nor was it given a default value. However if the parameter is present, the driver may assume that it's a If you want an example of what a PF driver using this interface might look like, you can check out my (in-progress) ixgbe PF driver on github: https://github.com/rysto32/freebsd/blob/38518e85c1e50254c78cfa9e0cc9cd1a7d8b10cf/sys/dev/ixgbe/ixgbe.c (Note: As I start preparing this work for review I will be rebasing and editing history fairly extensively. Clone my private branches at your own risk. :) ) The majority of the changes to ixgbe.c have to do with configuring the hardware and not dealing with the SR-IOV infrastructure, so I hope that's an indication that I'm on the right path. At this point I'm not ready for the code to be reviewed (although any comments would be welcome). At this stage I'm looking for comments on the design and the interface before I become fully committed to this path. From owner-freebsd-arch@FreeBSD.ORG Tue May 6 22:15:51 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9A79D3; Tue, 6 May 2014 22:15:51 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C33EB99; Tue, 6 May 2014 22:15:50 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WhneP-000KW2-7n; Tue, 06 May 2014 22:15:49 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s46MFkFj026622; Tue, 6 May 2014 16:15:46 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+8kRswA9DWBxTt4D8kCWJI Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Ian Lepore To: John Baldwin In-Reply-To: <201405061637.30037.jhb@freebsd.org> References: <201405051657.49992.jhb@freebsd.org> <201405061637.30037.jhb@freebsd.org> Content-Type: text/plain; charset="us-ascii" Date: Tue, 06 May 2014 16:15:46 -0600 Message-ID: <1399414546.22079.286.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2014 22:15:51 -0000 On Tue, 2014-05-06 at 16:37 -0400, John Baldwin wrote: > On Tuesday, May 06, 2014 2:08:35 pm Adrian Chadd wrote: > > On 5 May 2014 13:57, John Baldwin wrote: > > > > > The user in question found this on 9-stable with the existing defaults as the > > > HPET was just plain broken on their system and that was unrelated to Cx states. > > > (Rather, Cx states were only involved because worries about them are why the > > > system chose to use HPET. Had Cx states been enabled by default, they would > > > have had to disable those as well in addition to forcing LAPIC instead of > > > HPET.) > > > > Hm. Sounds uncomfortable. How does Windows run on systems like this? > > Do the windows drivers just disable HPET and use LAPIC or worse for > > timing, and just ignore anything lower than C1? > > I have no idea. Maybe they use the RTC. :-/ Maybe the HPET on these systems > works if you use it "sparingly". I believe OS X might have only used the HPET > to provide the "missing" LAPIC wakeups when entering Cx for example. (Our current > eventtimer system wants to always use whichever timer it picks, not switch off > between them.) > The eventtimer code is happy to switch between timers on the fly, but iirc the only interface to that feature is sysctl. I found it fairly easy to add an in-kernel API for changing the frequency of the current timer on the fly. I think it would be just as easy to add a kernel call to change timers as well. -- Ian From owner-freebsd-arch@FreeBSD.ORG Wed May 7 04:48:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E69114A0 for ; Wed, 7 May 2014 04:48:26 +0000 (UTC) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id A5757F78 for ; Wed, 7 May 2014 04:48:26 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id E60BE1A2333; Wed, 7 May 2014 14:48:17 +1000 (EST) Date: Wed, 7 May 2014 14:48:14 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Ryan Stone Subject: Re: RFC: PCI SR-IOV Driver interface In-Reply-To: Message-ID: <20140507141652.K1349@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=U6SrU4bu c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=dNjAHJjcj_oA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=vBoLb_oq0QFVOIqYKz8A:9 a=CjuIK1q_8ugA:10 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 04:48:27 -0000 On Tue, 6 May 2014, Ryan Stone wrote: > ... > PF drivers implement the following method to advertise their > configuration schema: > > METHOD void get_iov_config_schema { > device_t dev; > nvlist_t *pf_schema; > nvlist_t *vf_schema; > } > > The use of the nvlist_t in the interface is somewhat unfortunate. The > problem is that now every driver that includes "pci_if.h" needs to > have the typedef nvlist_t defined (and I *really* don't want to modify > every PCI driver in the tree...). I have a somewhat hacky workaround > for the problem in my tree right now but I thought that I would > highlight the issue in case people had opinions on the issue. Hrmph. style(9) explicitly forbids typedefs like nvlist_t. It is just a pointer to a struct. But since it is just that, it is easy to declare it in several headers. The struct type is not needed. The user nv.h begins with massive namespace pollution: % #include % % #include % #include % #include % #include Why bother hiding the struct type when you expose all this? The API does use FILE. FILE exposes its struct too. The string 'FILE *'now occurs in 24 headers in /usr/include/*.h and in 31 headers in /usr/include/*/*.h :-(. stdio.h is referenced in about half of these. Such mistakes shouldn't be repeated in new APIs. % % #ifndef _NVLIST_T_DECLARED % #define _NVLIST_T_DECLARED % struct nvlist; Bogus forward declaration (not needed). % % typedef struct nvlist nvlist_t; % #endif The polluting symbols are of course undocumented in nv's man page (except indirectly, by having a prototype that uses FILE, and many that use va_list bool or uint64_t). Bruce From owner-freebsd-arch@FreeBSD.ORG Wed May 7 09:12:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06A53CF for ; Wed, 7 May 2014 09:12:24 +0000 (UTC) Received: from melamine.cuivre.fr.eu.org (houdart.cuivre.fr.eu.org [81.57.40.110]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B74C8C7E for ; Wed, 7 May 2014 09:12:23 +0000 (UTC) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id A00DEF093; Wed, 7 May 2014 11:12:13 +0200 (CEST) Date: Wed, 7 May 2014 11:12:13 +0200 From: Thomas Quinot To: freebsd-arch@freebsd.org Subject: Proposed extension to stat(1) Message-ID: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 09:12:24 -0000 I'm proposing the addition of a command line switch -H to stat(1), causing arguments to be interpreted as NFS file handles instead of file names. This comes in handy when investigating NFS load issues. (A further possible extension would be to optionally include information from statfs/fhstatfs, but that's next on my list :) ). Thomas. Index: usr.bin/stat/stat.1 =================================================================== --- usr.bin/stat/stat.1 (révision 265192) +++ usr.bin/stat/stat.1 (copie de travail) @@ -38,7 +38,7 @@ .Nd display file status .Sh SYNOPSIS .Nm -.Op Fl FLnq +.Op Fl FHLnq .Op Fl f Ar format | Fl l | r | s | x .Op Fl t Ar timefmt .Op Ar @@ -124,6 +124,12 @@ .Fl F implies .Fl l . +.It Fl H +Treat each argument as the hexadecimal representation of an NFS file handle, +and use +.Xr fhstat 2 +instead of +.Xr lstat 2 . .It Fl L Use .Xr stat 2 Index: usr.bin/stat/stat.c =================================================================== --- usr.bin/stat/stat.c (révision 265192) +++ usr.bin/stat/stat.c (copie de travail) @@ -50,6 +50,7 @@ #include #include #include +#include #include #include @@ -203,9 +204,10 @@ { struct stat st; int ch, rc, errs, am_readlink; - int lsF, fmtchar, usestat, fn, nonl, quiet; + int lsF, fmtchar, usestat, nfs_handle, fn, nonl, quiet; const char *statfmt, *options, *synopsis; char dname[sizeof _PATH_DEV + SPECNAMELEN] = _PATH_DEV; + fhandle_t fhnd; const char *file; am_readlink = 0; @@ -212,6 +214,7 @@ lsF = 0; fmtchar = '\0'; usestat = 0; + nfs_handle = 0; nonl = 0; quiet = 0; linkfail = 0; @@ -226,9 +229,9 @@ fmtchar = 'f'; quiet = 1; } else { - options = "f:FlLnqrst:x"; + options = "f:FHlLnqrst:x"; synopsis = "[-FLnq] [-f format | -l | -r | -s | -x] " - "[-t timefmt] [file ...]"; + "[-t timefmt] [file|handle ...]"; } while ((ch = getopt(argc, argv, options)) != -1) @@ -236,6 +239,9 @@ case 'F': lsF = 1; break; + case 'H': + nfs_handle = 1; + break; case 'L': usestat = 1; break; @@ -320,8 +326,33 @@ file = "(stdin)"; rc = fstat(STDIN_FILENO, &st); } else { + int j; + char *inval; + file = argv[0]; - if (usestat) { + if (nfs_handle) { + rc = 0; + bzero (&fhnd, sizeof fhnd); + j = MIN(2 * sizeof fhnd, strlen(file)); + if (j & 1) { + rc = -1; + } else { + while (j) { + ((char*) &fhnd)[j / 2 - 1] = strtol(&file[j - 2], &inval, 16); + if (inval != NULL) { + rc = -1; + break; + } + argv[0][j - 2] = '\0'; + j -= 2; + } + if (!rc) + rc = fhstat(&fhnd, &st); + else + errno = EINVAL; + } + + } else if (usestat) { /* * Try stat() and if it fails, fall back to * lstat() just in case we're examining a From owner-freebsd-arch@FreeBSD.ORG Wed May 7 18:34:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 28D43D6E; Wed, 7 May 2014 18:34:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 026F8C97; Wed, 7 May 2014 18:34:35 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id AE9BFB917; Wed, 7 May 2014 14:34:32 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Proposed extension to stat(1) Date: Wed, 7 May 2014 11:48:59 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> In-Reply-To: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405071148.59285.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 07 May 2014 14:34:32 -0400 (EDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 18:34:35 -0000 On Wednesday, May 07, 2014 5:12:13 am Thomas Quinot wrote: > I'm proposing the addition of a command line switch -H to stat(1), > causing arguments to be interpreted as NFS file handles instead of file > names. This comes in handy when investigating NFS load issues. > > (A further possible extension would be to optionally include information > from statfs/fhstatfs, but that's next on my list :) ). Cute. Sounds good to me. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed May 7 18:34:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B6EFD70 for ; Wed, 7 May 2014 18:34:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 448EFC98 for ; Wed, 7 May 2014 18:34:35 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2F385B964; Wed, 7 May 2014 14:34:34 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: RFC: PCI SR-IOV Driver interface Date: Wed, 7 May 2014 11:50:03 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405071150.03854.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 07 May 2014 14:34:34 -0400 (EDT) Cc: Ryan Stone X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 18:34:35 -0000 On Tuesday, May 06, 2014 5:40:38 pm Ryan Stone wrote: > PCI Single Root I/O Virtualization (SR-IOV) is an optional part of the > PCIe standard that provides hardware acceleration for the > virtualization of PCIe devices. When SR-IOV is in use, a function in a > PCI device (known as a Physical Function, or PF) will present multiple > Virtual PCI Functions (VF) on the PCI bus. These VFs are fully > independent PCI devices that have access to the resources of the PF. > For example, on a network interface card, VFs could transmit and > receive packets independent of the PF. [ Trimmed ] I think your design here sounds fine. I would just explode the typedef and use 'struct nvlist' or whatever it is directly to avoid the header pollution. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed May 7 18:34:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E540EB0; Wed, 7 May 2014 18:34:41 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14388C9E; Wed, 7 May 2014 18:34:41 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id EE815B964; Wed, 7 May 2014 14:34:39 -0400 (EDT) From: John Baldwin To: Ian Lepore Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax Date: Wed, 7 May 2014 12:26:34 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <201405061637.30037.jhb@freebsd.org> <1399414546.22079.286.camel@revolution.hippie.lan> In-Reply-To: <1399414546.22079.286.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405071226.34487.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 07 May 2014 14:34:40 -0400 (EDT) Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 18:34:41 -0000 On Tuesday, May 06, 2014 6:15:46 pm Ian Lepore wrote: > On Tue, 2014-05-06 at 16:37 -0400, John Baldwin wrote: > > On Tuesday, May 06, 2014 2:08:35 pm Adrian Chadd wrote: > > > On 5 May 2014 13:57, John Baldwin wrote: > > > > > > > The user in question found this on 9-stable with the existing defaults as the > > > > HPET was just plain broken on their system and that was unrelated to Cx states. > > > > (Rather, Cx states were only involved because worries about them are why the > > > > system chose to use HPET. Had Cx states been enabled by default, they would > > > > have had to disable those as well in addition to forcing LAPIC instead of > > > > HPET.) > > > > > > Hm. Sounds uncomfortable. How does Windows run on systems like this? > > > Do the windows drivers just disable HPET and use LAPIC or worse for > > > timing, and just ignore anything lower than C1? > > > > I have no idea. Maybe they use the RTC. :-/ Maybe the HPET on these systems > > works if you use it "sparingly". I believe OS X might have only used the HPET > > to provide the "missing" LAPIC wakeups when entering Cx for example. (Our current > > eventtimer system wants to always use whichever timer it picks, not switch off > > between them.) > > > > The eventtimer code is happy to switch between timers on the fly, but > iirc the only interface to that feature is sysctl. I found it fairly > easy to add an in-kernel API for changing the frequency of the current > timer on the fly. I think it would be just as easy to add a kernel > call to change timers as well. Ah. Well, if it's only a small slice of time where machines need this, I'd be fine with those machines just having to disable C1E and forcing use of LAPIC rather than adding a lot of goop to do LAPIC + HPET and then hoping the HPET works well enough. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed May 7 19:38:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E3035DF; Wed, 7 May 2014 19:38:57 +0000 (UTC) Received: from melamine.cuivre.fr.eu.org (houdart.cuivre.fr.eu.org [81.57.40.110]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4CB430B; Wed, 7 May 2014 19:38:56 +0000 (UTC) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id 009B7F4E7; Wed, 7 May 2014 21:38:53 +0200 (CEST) Date: Wed, 7 May 2014 21:38:53 +0200 From: Thomas Quinot To: John Baldwin , Eitan Adler Subject: Re: Proposed extension to stat(1) Message-ID: <20140507193853.GA53842@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201405071148.59285.jhb@freebsd.org> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 May 2014 19:38:57 -0000 * John Baldwin, 2014-05-07 : > Cute. Sounds good to me. * Eitan Adler, 2014-05-07 : > I like this idea :-) Thanks :-) Committed! Thomas. From owner-freebsd-arch@FreeBSD.ORG Thu May 8 07:44:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D65286D; Thu, 8 May 2014 07:44:23 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D15FAAD; Thu, 8 May 2014 07:44:23 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id ld10so2373262pab.31 for ; Thu, 08 May 2014 00:44:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=LTXs1u6npUSEPg3OGRlgi9hiUPDABLfIE2kmVSKTBV0=; b=QM+rM5vcr6vjfKYR5s+7bMmWyYdfcmx8vwppyxMxMFlH2iktxE8vl4thzhR1Vuj43j oqmqPjKTn3QiO9Tz+mgj0pBc5UsaiMbKfHu5HUFANxHS2HF+4+6pPobzjlgWGnLC3Xtt oO9xiVA4Fi2mpLLk4kiDXrtT96K3K0MYWvzCnyg1UBpCMVjt9XkwhFtaAPe195MZOfkR x05K341uicpv97GWRwcleH/0OXRgAWJHd6F0B7fa8uks8RUrY0fNkrW6tyV6D9P75LNm MwSiIVi6Q1SlvwhpX53iy8rtQBSgwV30iCbiIkAMTOeBCYDPJ3znTtxcJnSLSkXxjN1G AmPg== X-Received: by 10.66.150.169 with SMTP id uj9mr4616967pab.148.1399535062297; Thu, 08 May 2014 00:44:22 -0700 (PDT) Received: from localhost (c-76-21-78-151.hsd1.ca.comcast.net. [76.21.78.151]) by mx.google.com with ESMTPSA id gj9sm518389pbc.7.2014.05.08.00.44.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 May 2014 00:44:21 -0700 (PDT) Sender: Gleb Kurtsou Date: Thu, 8 May 2014 00:44:50 -0700 From: Gleb Kurtsou To: Thomas Quinot Subject: Re: Proposed extension to stat(1) Message-ID: <20140508074450.GA2678@reks> References: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 07:44:23 -0000 On (07/05/2014 11:12), Thomas Quinot wrote: > I'm proposing the addition of a command line switch -H to stat(1), > causing arguments to be interpreted as NFS file handles instead of file > names. This comes in handy when investigating NFS load issues. > > (A further possible extension would be to optionally include information > from statfs/fhstatfs, but that's next on my list :) ). > > Thomas. > > Index: usr.bin/stat/stat.1 > =================================================================== > --- usr.bin/stat/stat.1 (révision 265192) > +++ usr.bin/stat/stat.1 (copie de travail) > @@ -38,7 +38,7 @@ > .Nd display file status > .Sh SYNOPSIS > .Nm > -.Op Fl FLnq > +.Op Fl FHLnq > .Op Fl f Ar format | Fl l | r | s | x > .Op Fl t Ar timefmt > .Op Ar > @@ -124,6 +124,12 @@ > .Fl F > implies > .Fl l . > +.It Fl H > +Treat each argument as the hexadecimal representation of an NFS file handle, > +and use > +.Xr fhstat 2 > +instead of > +.Xr lstat 2 . Noting that it requires root may be helpful. 'each argument' confused me a lot. > .It Fl L > Use > .Xr stat 2 > Index: usr.bin/stat/stat.c > =================================================================== > --- usr.bin/stat/stat.c (révision 265192) > +++ usr.bin/stat/stat.c (copie de travail) > @@ -50,6 +50,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -203,9 +204,10 @@ > { > struct stat st; > int ch, rc, errs, am_readlink; > - int lsF, fmtchar, usestat, fn, nonl, quiet; > + int lsF, fmtchar, usestat, nfs_handle, fn, nonl, quiet; > const char *statfmt, *options, *synopsis; > char dname[sizeof _PATH_DEV + SPECNAMELEN] = _PATH_DEV; > + fhandle_t fhnd; whitespace? > const char *file; > > am_readlink = 0; > @@ -212,6 +214,7 @@ > lsF = 0; > fmtchar = '\0'; > usestat = 0; > + nfs_handle = 0; whitespace? > nonl = 0; > quiet = 0; > linkfail = 0; > @@ -226,9 +229,9 @@ > fmtchar = 'f'; > quiet = 1; > } else { > - options = "f:FlLnqrst:x"; > + options = "f:FHlLnqrst:x"; > synopsis = "[-FLnq] [-f format | -l | -r | -s | -x] " > - "[-t timefmt] [file ...]"; > + "[-t timefmt] [file|handle ...]"; > } > > while ((ch = getopt(argc, argv, options)) != -1) > @@ -236,6 +239,9 @@ > case 'F': > lsF = 1; > break; > + case 'H': > + nfs_handle = 1; > + break; Whitespace. > case 'L': > usestat = 1; > break; > @@ -320,8 +326,33 @@ > file = "(stdin)"; > rc = fstat(STDIN_FILENO, &st); > } else { > + int j; > + char *inval; > + > file = argv[0]; > - if (usestat) { > + if (nfs_handle) { > + rc = 0; > + bzero (&fhnd, sizeof fhnd); Style. Extra space after bzero. > + j = MIN(2 * sizeof fhnd, strlen(file)); sizeof style. > + if (j & 1) { > + rc = -1; > + } else { > + while (j) { > + ((char*) &fhnd)[j / 2 - 1] = strtol(&file[j - 2], &inval, 16); It's badly broken: - Result for excessive "01" vs "0102" will differ in strlen(file) > sizeof(fhnd) * 2 case. - Will "+1-2" be considered a valid file handle? - why do we care about strlen(file) != sizeof(fh) case? > + if (inval != NULL) { According to strtol(3) such error handling is insufficient. At least it won't work for strlen(file) > sizeof(fhnd) * 2 case. > + rc = -1; > + break; > + } > + argv[0][j - 2] = '\0'; First we initialize file = argv[0]. Then we set every third character in argv[0] to '\0'. Afterwards file is used to print error message or actual result. > + j -= 2; > + } > + if (!rc) style. rc is not bool. > + rc = fhstat(&fhnd, &st); > + else > + errno = EINVAL; > + } > + > + } else if (usestat) { > /* > * Try stat() and if it fails, fall back to > * lstat() just in case we're examining a From owner-freebsd-arch@FreeBSD.ORG Thu May 8 09:19:08 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58B548D7; Thu, 8 May 2014 09:19:08 +0000 (UTC) Received: from melamine.cuivre.fr.eu.org (houdart.cuivre.fr.eu.org [81.57.40.110]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D213D32C; Thu, 8 May 2014 09:19:07 +0000 (UTC) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id 64970FBD6; Thu, 8 May 2014 11:18:58 +0200 (CEST) Date: Thu, 8 May 2014 11:18:58 +0200 From: Thomas Quinot To: Gleb Kurtsou Subject: Re: Proposed extension to stat(1) Message-ID: <20140508091858.GA79282@melamine.cuivre.fr.eu.org> References: <20140507091213.GA55856@melamine.cuivre.fr.eu.org> <20140508074450.GA2678@reks> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="b5gNqxB1S1yM7hjW" Content-Disposition: inline In-Reply-To: <20140508074450.GA2678@reks> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.22 (2013-10-16) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 09:19:08 -0000 --b5gNqxB1S1yM7hjW Content-Type: multipart/mixed; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline * Gleb Kurtsou, 2014-05-08 : Thanks for the thorough review, Gleb! Much appreciated. > - why do we care about strlen(file) != sizeof(fh) case? tcpdump(1) may display handles with additional trailing bytes that are irrelevant to fhstat(2). It would be inconvenient to require the user to know about the exact length at which the information needs to be truncated. It is more helpful to silently discard the excess data. Attached is a diff vs the committed version (r265591) which I think addresses all of your remarks. Is there anything else that you think would need fixing? Thomas. --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename=d Content-Transfer-Encoding: quoted-printable Index: stat.1 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- stat.1 (r=E9vision 265665) +++ stat.1 (copie de travail) @@ -130,6 +130,7 @@ .Xr fhstat 2 instead of .Xr lstat 2 . +This requires root privileges. .It Fl L Use .Xr stat 2 Index: stat.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- stat.c (r=E9vision 265665) +++ stat.c (copie de travail) @@ -186,6 +186,7 @@ char *, size_t, /* a place to put the output */ int, int, int, int, /* the parsed format */ int, int); +int hex2byte(const char [2]); #if HAVE_STRUCT_STAT_ST_FLAGS char *xfflagstostr(unsigned long); #endif @@ -214,7 +215,7 @@ lsF =3D 0; fmtchar =3D '\0'; usestat =3D 0; - nfs_handle =3D 0; + nfs_handle =3D 0; nonl =3D 0; quiet =3D 0; linkfail =3D 0; @@ -327,32 +328,27 @@ rc =3D fstat(STDIN_FILENO, &st); } else { int j; - char *inval; =20 file =3D argv[0]; if (nfs_handle) { rc =3D 0; - bzero (&fhnd, sizeof fhnd); - j =3D MIN(2 * sizeof fhnd, strlen(file)); - if (j & 1) { + bzero(&fhnd, sizeof(fhnd)); + j =3D MIN(2 * sizeof(fhnd), strlen(file)); + if ((j & 1) !=3D 0) { rc =3D -1; } else { while (j) { - ((char*) &fhnd)[j / 2 - 1] =3D - strtol(&file[j - 2], - &inval, 16); - if (inval !=3D NULL) { - rc =3D -1; + rc =3D hex2byte(&file[j - 2]); + if (rc =3D=3D -1) break; - } - argv[0][j - 2] =3D '\0'; + ((char*) &fhnd)[j / 2 - 1] =3D rc; j -=3D 2; } - if (!rc) - rc =3D fhstat(&fhnd, &st); - else - errno =3D EINVAL; } + if (rc =3D=3D -1) + errno =3D EINVAL; + else + rc =3D fhstat(&fhnd, &st); =20 } else if (usestat) { /* @@ -1091,3 +1087,12 @@ =20 return (snprintf(buf, blen, lfmt, data)); } + + +#define hex2nibble(c) (c <=3D '9' ? c - '0' : toupper(c) - 'A' + 10) +int +hex2byte(const char c[2]) { + if (!(ishexnumber(c[0]) && ishexnumber(c[1]))) + return -1; + return (hex2nibble(c[0]) << 4) + hex2nibble(c[1]); +} --G4iJoqBmSsgzjUCe-- --b5gNqxB1S1yM7hjW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iD8DBQFTa0wAAE1UuDk9JGkRAqX+AJ90UpSfO78nDqCIxEBl6mlx5k2k7QCffj9a FZ2JO2LYd5wgup1vBvnJuqo= =MZL1 -----END PGP SIGNATURE----- --b5gNqxB1S1yM7hjW-- From owner-freebsd-arch@FreeBSD.ORG Thu May 8 11:41:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E599AD for ; Thu, 8 May 2014 11:41:21 +0000 (UTC) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 1F815202 for ; Thu, 8 May 2014 11:41:20 +0000 (UTC) Received: from nine.des.no (smtp.des.no [194.63.250.102]) by smtp-int.des.no (Postfix) with ESMTP id 463CEA4FF; Thu, 8 May 2014 11:41:20 +0000 (UTC) Received: by nine.des.no (Postfix, from userid 1001) id 607ECD32; Thu, 8 May 2014 13:41:05 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Bruce Evans Subject: Re: RFC: PCI SR-IOV Driver interface References: <20140507141652.K1349@besplex.bde.org> Date: Thu, 08 May 2014 13:41:05 +0200 In-Reply-To: <20140507141652.K1349@besplex.bde.org> (Bruce Evans's message of "Wed, 7 May 2014 14:48:14 +1000 (EST)") Message-ID: <86r444wl9q.fsf@nine.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 11:41:21 -0000 Bruce Evans writes: > Hrmph. style(9) explicitly forbids typedefs like nvlist_t. ...as does C, which reserves *_t, IIRC. Although I'm guilty of abusing that namespace myself. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu May 8 15:51:16 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 031DEE53 for ; Thu, 8 May 2014 15:51:16 +0000 (UTC) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id B6BF7FF2 for ; Thu, 8 May 2014 15:51:15 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id A6333422C9E; Fri, 9 May 2014 01:51:13 +1000 (EST) Date: Fri, 9 May 2014 01:51:12 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= Subject: Re: RFC: PCI SR-IOV Driver interface In-Reply-To: <86r444wl9q.fsf@nine.des.no> Message-ID: <20140509013938.H3451@besplex.bde.org> References: <20140507141652.K1349@besplex.bde.org> <86r444wl9q.fsf@nine.des.no> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=QIpRGG7L c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=dNjAHJjcj_oA:10 a=JzwRw_2MAAAA:8 a=nlC_4_pT8q9DhB4Ho9EA:9 a=cz2ZRIgtxKwA:10 a=wJWlkF7cXJYA:10 a=RLYB6GcqJLkgtVKBTukA:9 a=45ClL6m2LaAA:10 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2014 15:51:16 -0000 On Thu, 8 May 2014, [utf-8] Dag-Erling Sm=C3=B8rgrav wrote: > Bruce Evans writes: >> Hrmph. style(9) explicitly forbids typedefs like nvlist_t. > > ...as does C, which reserves *_t, IIRC. Although I'm guilty of abusing > that namespace myself. No, _t is not reserved in C, and is reserved for the implementation in POSIX. nv is an implementation. style(9) says not to use _t for typedefs, but that is for applications. Kernel code can be considered as both an implementation and an application. I normally consider it as an application, but wouldn't want to disallow using _t in it. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri May 9 03:43:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48BCB156; Fri, 9 May 2014 03:43:41 +0000 (UTC) Received: from mail-qg0-x22a.google.com (mail-qg0-x22a.google.com [IPv6:2607:f8b0:400d:c04::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9E2FBAE; Fri, 9 May 2014 03:43:40 +0000 (UTC) Received: by mail-qg0-f42.google.com with SMTP id q107so3956682qgd.1 for ; Thu, 08 May 2014 20:43:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ATMJLviVwmwdf45KoU4fojsPVik27wgy2hx46+60Obk=; b=d1vp0gGXFhdI7ZbFC8yUVaj4wrEKfeVJkOpRFLiF3YJZjLk+7IhrOEZg2bMbSt+uUc /Et1o/sy0BVM6CKmeUyDJfxfyhXP9EKs+usogHnL50Md4+cWy252POZHXPqpRvNR+y/S gCc4ispm8b6U8ciBw8AparMJiD0Ei8DDAE6fFw/1+XWI7gf87wVVFu2kQ2V8+0dm/ZL6 xypXhCkx9YMHDMW4G1wPjExGAJ/yCJN5e+4xqDT3bX9uKV428XrCkHjQ6O2IiV0AgHJx u5FWUAWf67cXclCh/ueshs+CsQEHrQmGR4uLiQNbWaefL8h90uj+YsgKSjJQ82MGAO6S 2A1Q== MIME-Version: 1.0 X-Received: by 10.140.96.51 with SMTP id j48mr10396376qge.24.1399607020007; Thu, 08 May 2014 20:43:40 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Thu, 8 May 2014 20:43:39 -0700 (PDT) In-Reply-To: References: <530508B7.7060102@FreeBSD.org> <201402191602.54465.jhb@freebsd.org> <201402201417.34148.jhb@freebsd.org> Date: Thu, 8 May 2014 20:43:39 -0700 X-Google-Sender-Auth: hSGWD_R_1S6yPq0xpmFKcVndWRI Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 03:43:41 -0000 Hi, I'd like to revisit this now. I'd like to commit this stuff as-is and then take some time to revisit the catch-all softclock from cpu0 swi. It's more complicated than it needs to be as it just assumes timeout_cpu == cpuid of cpu 0. So there's no easy way to slide in a new catch-all softclock. Once that's done I'd like to then experiment with turning on the pcpu tcp timer stuff and gluing that into the RSS CPU ID / netisr ID stuff. Thanks, -a On 20 February 2014 13:48, Adrian Chadd wrote: > On 20 February 2014 11:17, John Baldwin wrote: > >> (A further variant of this would be to divorce cpu0's swi from the >> catch-all softclock and let the catch-all softclock float, but bind >> all the per-cpu swis) > > I like this idea. If something (eg per-CPU TCP timers, if it's turned > on) makes a very specific decision about the CPU then it should be > fixed. Otherwise a lot of the underlying assumptions for things like > RSS just aren't guaranteed to hold. > > It could also perhaps extend to some abstract pool of CPUs later, if > we wanted to do things like one flowing swi per socket or whatnot when > we start booting on 1024 core boxes... > > -a From owner-freebsd-arch@FreeBSD.ORG Fri May 9 09:55:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D643FD9; Fri, 9 May 2014 09:55:32 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id C346DFF8; Fri, 9 May 2014 09:55:31 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.60.3]) by phk.freebsd.dk (Postfix) with ESMTP id 12B911598; Fri, 9 May 2014 09:55:30 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.8/8.14.8) with ESMTP id s499tSen007682; Fri, 9 May 2014 09:55:29 GMT (envelope-from phk@phk.freebsd.dk) To: Ian Smith Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-reply-to: <20140505163316.R11699@sola.nimnet.asn.au> From: "Poul-Henning Kamp" References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 09 May 2014 09:55:28 +0000 Message-ID: <7681.1399629328@critter.freebsd.dk> Cc: Kevin Oberman , Adrian Chadd , "freebsd-acpi@freebsd.org" , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 09:55:32 -0000 In message <20140505163316.R11699@sola.nimnet.asn.au>, Ian Smith writes: >On Mon, 5 May 2014 06:25:21 +0000, Poul-Henning Kamp wrote: > > In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith writes: > > > > Do we have a canonical page with all the various workarounds one should > > attempt in order to get suspend/resume to work ? > >Bits scattered all over the place. For the above there's: So based on various scattered hints, I tried booting the VT kernel, r265336, on my Thinkpad T430s and that seems to fix both Suspend/Resume and also console switching. Much appreciated! I'll keep an eye on any peripheral bogons as I used it now. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri May 9 10:26:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AD55F44B; Fri, 9 May 2014 10:26:01 +0000 (UTC) Received: from mail-qa0-x22a.google.com (mail-qa0-x22a.google.com [IPv6:2607:f8b0:400d:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59E482AA; Fri, 9 May 2014 10:26:01 +0000 (UTC) Received: by mail-qa0-f42.google.com with SMTP id j5so3898004qaq.29 for ; Fri, 09 May 2014 03:26:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=FNgIjzGxDKoqUBGyu+Z5GgpzaLrUONVlf0qh3gLiAOU=; b=s6A2jRD1Gd/16Ae3aUHqI6v1nCnu4EApZ2jegi1kv1S373+LYfv216r6LHQjxYLf99 khP8fQ5cg5lUMmNKZm5SJYEzh2BJt29QjSMMw/gyhpjhaD8ysG5YOaGtiPKYHM04iTmw xCReEfK96zWULQth1XnZOP6e3B6pIuFpPKbKttauJzXgXLDlyeBzYr1fpjgkZMhQiZsL cpVsXyjQWZggjicz+13uz9nMgvowlBI0HBDR1JSD/lsPKY5vP1hHOXzsw7AG0VC0uoQZ yVMG0iEbohC91zRgiXOB6Gj66guBLVvksIpk4nDDGJ1qpvIgDrj2GUvHW3KLKkdcCF+O PjZQ== MIME-Version: 1.0 X-Received: by 10.224.47.130 with SMTP id n2mr13007698qaf.26.1399631160040; Fri, 09 May 2014 03:26:00 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 03:25:59 -0700 (PDT) In-Reply-To: <7681.1399629328@critter.freebsd.dk> References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Date: Fri, 9 May 2014 03:25:59 -0700 X-Google-Sender-Auth: eiZxUpZlvPRLsB0N6qRpye8sFh4 Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Poul-Henning Kamp Content-Type: text/plain; charset=UTF-8 Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Ian Smith , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 10:26:01 -0000 Hi! On 9 May 2014 02:55, Poul-Henning Kamp wrote: > In message <20140505163316.R11699@sola.nimnet.asn.au>, Ian Smith writes: >>On Mon, 5 May 2014 06:25:21 +0000, Poul-Henning Kamp wrote: >> > In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith writes: >> > >> > Do we have a canonical page with all the various workarounds one should >> > attempt in order to get suspend/resume to work ? >> >>Bits scattered all over the place. For the above there's: > > So based on various scattered hints, I tried booting the VT kernel, > r265336, on my Thinkpad T430s and that seems to fix both Suspend/Resume > and also console switching. > > Much appreciated! > > I'll keep an eye on any peripheral bogons as I used it now. Woo! Would you mind populating http://wiki.freebsd.org/Laptops with your details? Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Fri May 9 18:08:36 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9640523F; Fri, 9 May 2014 18:08:36 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C945A8; Fri, 9 May 2014 18:08:36 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5D7BCB99B; Fri, 9 May 2014 14:08:35 -0400 (EDT) From: John Baldwin To: Adrian Chadd Subject: Re: [rfc] bind per-cpu timeout threads to each CPU Date: Fri, 9 May 2014 13:49:14 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <530508B7.7060102@FreeBSD.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201405091349.14381.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 09 May 2014 14:08:35 -0400 (EDT) Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 18:08:36 -0000 On Thursday, May 08, 2014 11:43:39 pm Adrian Chadd wrote: > Hi, > > I'd like to revisit this now. > > I'd like to commit this stuff as-is and then take some time to revisit > the catch-all softclock from cpu0 swi. It's more complicated than it > needs to be as it just assumes timeout_cpu == cpuid of cpu 0. So > there's no easy way to slide in a new catch-all softclock. > > Once that's done I'd like to then experiment with turning on the pcpu > tcp timer stuff and gluing that into the RSS CPU ID / netisr ID stuff. > > Thanks, To be clear, are you going to commit the change to bind all but CPU 0 to their CPU but let the "default" swi float for now? I think that is fine to commit, but I wouldn't want to bind the "default" swi for now. > -a > > > On 20 February 2014 13:48, Adrian Chadd wrote: > > On 20 February 2014 11:17, John Baldwin wrote: > > > >> (A further variant of this would be to divorce cpu0's swi from the > >> catch-all softclock and let the catch-all softclock float, but bind > >> all the per-cpu swis) > > > > I like this idea. If something (eg per-CPU TCP timers, if it's turned > > on) makes a very specific decision about the CPU then it should be > > fixed. Otherwise a lot of the underlying assumptions for things like > > RSS just aren't guaranteed to hold. > > > > It could also perhaps extend to some abstract pool of CPUs later, if > > we wanted to do things like one flowing swi per socket or whatnot when > > we start booting on 1024 core boxes... > > > > -a > -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri May 9 19:34:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 323D1928; Fri, 9 May 2014 19:34:00 +0000 (UTC) Received: from mail-qg0-x22d.google.com (mail-qg0-x22d.google.com [IPv6:2607:f8b0:400d:c04::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4F4CB18; Fri, 9 May 2014 19:33:59 +0000 (UTC) Received: by mail-qg0-f45.google.com with SMTP id z60so4905189qgd.4 for ; Fri, 09 May 2014 12:33:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Ii3i/oIW7SdAGZEjjWyXf5td+yfUZ2qVfppDK6uGZYY=; b=XeWBWeS+IEjaTfKoVOAOa5OZRgA6PK0C38ipGJETO9005Jo3rWC3+kqphiHtvJG10e hPmGepGYT5x0/cYPSW39zQ8/g/L1QXcu29/BHY3ogQctIkDmLj4LusrACPeBmgj4pOFQ ZyJn9KcuPTbNBE7jm0raro9JQ12Y9KAb0ScjwW80Gqfo2u9KT/Pq6Ppr7DEe5UhFTmWt hnlfMAMXyXCDCXs0fekXJ78NbLkDaE+JcHOCMtXEhjb2Hbq5BY7EzrIj38H6ALCT2j4a asOXVilgrJRztsy7oeahg2GZ3KSYLAPqMcEUJhhJLmXc8RRH2dNepixLkxzb2Lw+Fwsr 5P6w== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr18074687qaa.76.1399664038874; Fri, 09 May 2014 12:33:58 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 12:33:58 -0700 (PDT) In-Reply-To: <201405091349.14381.jhb@freebsd.org> References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> Date: Fri, 9 May 2014 12:33:58 -0700 X-Google-Sender-Auth: ABsMDqC0pDXKRt-5xIGqQQFDQCc Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 19:34:00 -0000 On 9 May 2014 10:49, John Baldwin wrote: > On Thursday, May 08, 2014 11:43:39 pm Adrian Chadd wrote: >> Hi, >> >> I'd like to revisit this now. >> >> I'd like to commit this stuff as-is and then take some time to revisit >> the catch-all softclock from cpu0 swi. It's more complicated than it >> needs to be as it just assumes timeout_cpu == cpuid of cpu 0. So >> there's no easy way to slide in a new catch-all softclock. >> >> Once that's done I'd like to then experiment with turning on the pcpu >> tcp timer stuff and gluing that into the RSS CPU ID / netisr ID stuff. >> >> Thanks, > > To be clear, are you going to commit the change to bind all but CPU 0 > to their CPU but let the "default" swi float for now? I think that is > fine to commit, but I wouldn't want to bind the "default" swi for now. I'd like to do it in the other order and bind everything, so things like the per-CPU TCP timer thing can be flipped on for RSS and actually be useful. I'm looking into what it'd take to create a separate default swi as well as a cpu-0 swi but as I said, it's pretty hairy there. How about i instead do the comprimise: * i'll pin all other swi's * default swi isn't pinned by default, but one can flip on a sysctl at boot time to pin it How's that sound? -a From owner-freebsd-arch@FreeBSD.ORG Fri May 9 19:50:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64141F14; Fri, 9 May 2014 19:50:40 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 24427C4C; Fri, 9 May 2014 19:50:39 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id D3478124A8; Sat, 10 May 2014 05:50:31 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUC09774 (AUTH peterg@ptree32.com.au); Sat, 10 May 2014 05:50:30 +1000 Message-ID: <536D3184.9070302@freebsd.org> Date: Fri, 09 May 2014 12:50:28 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Adrian Chadd , John Baldwin Subject: Re: [rfc] bind per-cpu timeout threads to each CPU References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 19:50:40 -0000 > How about i instead do the comprimise: > > * i'll pin all other swi's > * default swi isn't pinned by default, but one can flip on a sysctl at > boot time to pin it > > How's that sound? And also please a sysctl that disables any swi pinning. It is sometimes useful to change the default cpuset, for instance to allocate a subset of CPUs to some particular applications and not FreeBSD. Having kernel threads pinned prevents this from happening since they are in the default set. (Note that some network drivers are also culprits here, though disabling MSI-x in them is a workaround). later, Peter. From owner-freebsd-arch@FreeBSD.ORG Fri May 9 20:05:54 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FBEE805; Fri, 9 May 2014 20:05:54 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EFBD6DEE; Fri, 9 May 2014 20:05:53 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id e16so5066351qcx.41 for ; Fri, 09 May 2014 13:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=K4fROt0upQbhbNwC4VXqitoBtOrFfrEAiFREHIC5Bqg=; b=jwXoyclbhc+MbxwUTTrG4F3OVPcO6bER7Yr/BqIPcqpxn58O0Z313RhcPqQCXHwq8E 99dz8ihiHUMdTOeX5Gh2Fxul3/O9JyX8DZVCv6uX9W9vT8q/9NaKiQYc86L+tprp8CBi zBue6E0TV3cXkYa3o6w7aIRwtT2uU1Be0yhu9UJa8wK1PznqwjYKOuVtaBTZu5qZ36UA c3xV3vii73gT7xRRiT1WLQssfjSJSY3ASkrXYYcsnAFq4qYk4NZrlojGwpkkWWT0PjNT OWJ5DsOpOQuuZmPQwbITumkENLhX4f6nl+xLgf9up5yQm0HxQYu3EJyJkz1Sq2v2LO4z LpYA== MIME-Version: 1.0 X-Received: by 10.140.42.85 with SMTP id b79mr16760249qga.87.1399665952387; Fri, 09 May 2014 13:05:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 13:05:52 -0700 (PDT) In-Reply-To: <536D3184.9070302@freebsd.org> References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> <536D3184.9070302@freebsd.org> Date: Fri, 9 May 2014 13:05:52 -0700 X-Google-Sender-Auth: n8CryHSl9hlTy00b1sv-8jkOliA Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 20:05:54 -0000 On 9 May 2014 12:50, Peter Grehan wrote: >> How about i instead do the comprimise: >> >> * i'll pin all other swi's >> * default swi isn't pinned by default, but one can flip on a sysctl at >> boot time to pin it >> >> How's that sound? > > > And also please a sysctl that disables any swi pinning. > > It is sometimes useful to change the default cpuset, for instance to > allocate a subset of CPUs to some particular applications and not FreeBSD. > Having kernel threads pinned prevents this from happening since they are in > the default set. > > (Note that some network drivers are also culprits here, though disabling > MSI-x in them is a workaround). Yup. I've just done that. http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff Which workloads are you thinking about? Maybe we could introduce some higher level description of which CPU(s) at boot time to do "freebsd stuff" on, and then don't start things like pcpu swi's and NIC threads on those CPUs. Can you think of situations where we'd want to have per-cpu swi's even _running_ for CPUs that you want to dedicate to other things? There's nothing stopping you from scheduling a callout on a different target CPU. (It'd require some work, but it likely should be done in some form as part of overall RSS framework hacking.) -a From owner-freebsd-arch@FreeBSD.ORG Fri May 9 20:12:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 89EEAA2D; Fri, 9 May 2014 20:12:29 +0000 (UTC) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E793E9F; Fri, 9 May 2014 20:12:29 +0000 (UTC) Received: by mail-pa0-f46.google.com with SMTP id kq14so658345pab.19 for ; Fri, 09 May 2014 13:12:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=dLioNfn9FiVlEvybSPFIB3tkoFEK0Cb2/x+2/OcG0KY=; b=bSF7pBEaJKVTdAyXSiA6XL6qVZRoTI7QAGvjBlYfzTt+8rrAkqcg6iTiHqHrEcF0su Q6wcfVx7lPuBV6DACxK2mJp8KKv7uuvgcUNaT1NkWI699ypyuOYTuO0cVi2pIGKmnEgc GR+JwIBJlZZThaxsIwxBPi4Ce6IwQna9tJgSpzFrp9XsjO/Ss1/ihZB97S+ko5YjNEot Z219UHgWeBirSnYHrrTeULHVBVFEsXuTELTRV4SrLMFXoTUPmu8H/PinUuADOv1EcVFd A/TQjf6cnD601EKbMYm71IMKcYwSuaeEW5t4oiarwsw/S5nmD5ZTC8/NQAVPOPKY2T+v tkeg== MIME-Version: 1.0 X-Received: by 10.67.8.102 with SMTP id dj6mr24209393pad.10.1399666348963; Fri, 09 May 2014 13:12:28 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.73.34 with HTTP; Fri, 9 May 2014 13:12:28 -0700 (PDT) In-Reply-To: References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Date: Fri, 9 May 2014 13:12:28 -0700 X-Google-Sender-Auth: L9x_f_czLEgBkIM9ioseXVAWcpY Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Kevin Oberman To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" , Ian Smith , "freebsd-acpi@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 20:12:29 -0000 On Fri, May 9, 2014 at 3:25 AM, Adrian Chadd wrote: > Hi! > > On 9 May 2014 02:55, Poul-Henning Kamp wrote: > > In message <20140505163316.R11699@sola.nimnet.asn.au>, Ian Smith writes: > >>On Mon, 5 May 2014 06:25:21 +0000, Poul-Henning Kamp wrote: > >> > In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith > writes: > >> > > >> > Do we have a canonical page with all the various workarounds one > should > >> > attempt in order to get suspend/resume to work ? > >> > >>Bits scattered all over the place. For the above there's: > > > > So based on various scattered hints, I tried booting the VT kernel, > > r265336, on my Thinkpad T430s and that seems to fix both Suspend/Resume > > and also console switching. > > > > Much appreciated! > > > > I'll keep an eye on any peripheral bogons as I used it now. > > Woo! > > Would you mind populating http://wiki.freebsd.org/Laptops with your > details? > > Thanks! > > > -a > Excellent! This alone will save batteries and also lower the carbon footprint of FreeBSD servers! Just to clarify the various settings of *_cx_lowest in rc.conf, HIGH is obvious. At one time, LOW was also obvious, but then some vendors started shipping BIOS that "skipped" some C-states in different power conditions. E.g. C1, C2 and C3 when on Battery, but only C1 and C3 when on AC. This scenario was common on Sandybridge systems (like my T320). Skipping a state broke "LOW" as it only saw C1 when on AC. Thus, Cmax appeared. Cmax is simply C8. It is just easier ot remember then C8. The code was re-written to ignore "missing" C-states and try all possible C-states until C8 was reached. Why "LOW" was not just changed to deal with this I don't understand, but Cmax (or C8) is recommended to gain the maximum power savings from C-states. On AC power: dev.cpu.0.cx_supported: C1/1/1 C2/3/104 dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_usage: 8.86% 91.13% last 2685us On battery: dev.cpu.0.cx_supported: C1/1/1 C2/2/80 C3/3/109 dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_usage: 3.09% 0.74% 96.15% last 728us Note the supported list on AC? C2/3/104 The first part, "C2", is what the OS labels that second state. The next part, "3", is the ACPI number of this state. On AC, this system has no C-state 2, so FreeBSD call the ACPI state 3 "C2". Oh, the last number is the number of clock cycles required to get into/out of that state. so in my case, when on battery, my CPU goes ot C2 after being halted for 80 clock cycles and C3 after 109. I hope this makes sense to everyone. I'm not really sure that it does to me! -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-arch@FreeBSD.ORG Fri May 9 20:36:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B612B38D; Fri, 9 May 2014 20:36:03 +0000 (UTC) Received: from mail-qc0-x230.google.com (mail-qc0-x230.google.com [IPv6:2607:f8b0:400d:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A703CA; Fri, 9 May 2014 20:36:03 +0000 (UTC) Received: by mail-qc0-f176.google.com with SMTP id r5so5242998qcx.35 for ; Fri, 09 May 2014 13:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=nj3nn7zXA86NwjVh1fr7cUxbCzjDb/XQbOKNBgghR6o=; b=k8+mt1lNssLHLK9PMYYHSnBpbwdjQdF3RWJ3SAPvp/BWiadPsEUww3OTOVf5z3hKgp znqgnb/dV16VtxKDCnHNP6zi7rNQplPJqJ4OSQzyWt9B/mHfAxXg8djsRgEOwZ5Bv0w6 NuMdEGlbNeY2ptnthTEJJqH/4q+ZvaQymHILE3oZ99vGsUIX8dBFqnSI4nKZcv2k8MK2 kaOFgOvHHrvb7CpCsM95RbUoG9PWrxGWhNs031T9gUBIZnDO/AW2cKb62CGMA4Vlotkp vOCdudFlBsYwLnXZ7vcH70lb9PkB4hFB1T+vk6X0eR3H4a7gDnHtAI6GrcFtMJqIvjOc KB3Q== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr18507242qaa.76.1399667762540; Fri, 09 May 2014 13:36:02 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 13:36:02 -0700 (PDT) In-Reply-To: References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Date: Fri, 9 May 2014 13:36:02 -0700 X-Google-Sender-Auth: buWpk3LJagVjuFoJaUfq78cAcNg Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" , Ian Smith , "freebsd-acpi@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 20:36:03 -0000 cool! next; # pkg install intel-pcm # kldload cpuctl # pcm.x 1 See what it reports. -a On 9 May 2014 13:12, Kevin Oberman wrote: > On Fri, May 9, 2014 at 3:25 AM, Adrian Chadd wrote: >> >> Hi! >> >> On 9 May 2014 02:55, Poul-Henning Kamp wrote: >> > In message <20140505163316.R11699@sola.nimnet.asn.au>, Ian Smith writes: >> >>On Mon, 5 May 2014 06:25:21 +0000, Poul-Henning Kamp wrote: >> >> > In message <20140505153421.W11699@sola.nimnet.asn.au>, Ian Smith >> >> > writes: >> >> > >> >> > Do we have a canonical page with all the various workarounds one >> >> > should >> >> > attempt in order to get suspend/resume to work ? >> >> >> >>Bits scattered all over the place. For the above there's: >> > >> > So based on various scattered hints, I tried booting the VT kernel, >> > r265336, on my Thinkpad T430s and that seems to fix both Suspend/Resume >> > and also console switching. >> > >> > Much appreciated! >> > >> > I'll keep an eye on any peripheral bogons as I used it now. >> >> Woo! >> >> Would you mind populating http://wiki.freebsd.org/Laptops with your >> details? >> >> Thanks! >> >> >> -a > > > Excellent! This alone will save batteries and also lower the carbon > footprint of FreeBSD servers! > > Just to clarify the various settings of *_cx_lowest in rc.conf, HIGH is > obvious. At one time, LOW was also obvious, but then some vendors started > shipping BIOS that "skipped" some C-states in different power conditions. > E.g. C1, C2 and C3 when on Battery, but only C1 and C3 when on AC. This > scenario was common on Sandybridge systems (like my T320). Skipping a state > broke "LOW" as it only saw C1 when on AC. Thus, Cmax appeared. Cmax is > simply C8. It is just easier ot remember then C8. The code was re-written to > ignore "missing" C-states and try all possible C-states until C8 was > reached. > > Why "LOW" was not just changed to deal with this I don't understand, but > Cmax (or C8) is recommended to gain the maximum power savings from > C-states. > > On AC power: > dev.cpu.0.cx_supported: C1/1/1 C2/3/104 > dev.cpu.0.cx_lowest: C8 > dev.cpu.0.cx_usage: 8.86% 91.13% last 2685us > > On battery: > dev.cpu.0.cx_supported: C1/1/1 C2/2/80 C3/3/109 > dev.cpu.0.cx_lowest: C8 > dev.cpu.0.cx_usage: 3.09% 0.74% 96.15% last 728us > > Note the supported list on AC? > C2/3/104 The first part, "C2", is what the OS labels that second state. The > next part, "3", is the ACPI number of this state. On AC, this system has no > C-state 2, so FreeBSD call the ACPI state 3 "C2". Oh, the last number is the > number of clock cycles required to get into/out of that state. so in my > case, when on battery, my CPU goes ot C2 after being halted for 80 clock > cycles and C3 after 109. I hope this makes sense to everyone. I'm not really > sure that it does to me! > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com From owner-freebsd-arch@FreeBSD.ORG Fri May 9 21:05:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5616DAD; Fri, 9 May 2014 21:05:28 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9BB24399; Fri, 9 May 2014 21:05:28 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 905AFB976; Fri, 9 May 2014 17:05:26 -0400 (EDT) From: John Baldwin To: Peter Grehan Subject: Re: [rfc] bind per-cpu timeout threads to each CPU Date: Fri, 9 May 2014 16:51:49 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <530508B7.7060102@FreeBSD.org> <536D3184.9070302@freebsd.org> In-Reply-To: <536D3184.9070302@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201405091651.49818.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 09 May 2014 17:05:26 -0400 (EDT) Cc: Adrian Chadd , freebsd-current , Alexander Motin , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 21:05:28 -0000 On Friday, May 09, 2014 3:50:28 pm Peter Grehan wrote: > > How about i instead do the comprimise: > > > > * i'll pin all other swi's > > * default swi isn't pinned by default, but one can flip on a sysctl at > > boot time to pin it > > > > How's that sound? > > And also please a sysctl that disables any swi pinning. > > It is sometimes useful to change the default cpuset, for instance to > allocate a subset of CPUs to some particular applications and not > FreeBSD. Having kernel threads pinned prevents this from happening since > they are in the default set. > > (Note that some network drivers are also culprits here, though > disabling MSI-x in them is a workaround). I'd actually like a way to exempt certain kernel threads that are inherently per-CPU (such as queues for NIC drivers or per-CPU swi threads) from the default cpuset so that they don't break 'cpuset -l 0 -s 1'. Providing some sort of way to disable the pinning for now should be good, but I think I'd eventually prefer the former suggestion. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri May 9 21:33:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 21FE99E3; Fri, 9 May 2014 21:33:03 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id D4AE28E2; Fri, 9 May 2014 21:33:02 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.60.3]) by phk.freebsd.dk (Postfix) with ESMTP id CCC961598; Fri, 9 May 2014 21:33:01 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.8/8.14.8) with ESMTP id s49LX0gi010409; Fri, 9 May 2014 21:33:00 GMT (envelope-from phk@phk.freebsd.dk) To: Adrian Chadd Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax In-reply-to: From: "Poul-Henning Kamp" References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1 Date: Fri, 09 May 2014 21:33:00 +0000 Message-ID: <10408.1399671180@critter.freebsd.dk> Cc: Kevin Oberman , "freebsd-acpi@freebsd.org" , Ian Smith , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 21:33:03 -0000 In message , Adrian Chadd write s: >cool! > >next; > ># pkg install intel-pcm ># kldload cpuctl ># pcm.x 1 > >See what it reports. Lots of numbers. Am I looking for anything in particular ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri May 9 23:49:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D20B71D; Fri, 9 May 2014 23:49:50 +0000 (UTC) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id CF0812FD; Fri, 9 May 2014 23:49:49 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id CAF73124BB; Sat, 10 May 2014 09:49:47 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local ([64.245.0.210]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BUC14621 (AUTH peterg@ptree32.com.au); Sat, 10 May 2014 09:49:46 +1000 Message-ID: <536D6999.6080402@freebsd.org> Date: Fri, 09 May 2014 16:49:45 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: [rfc] bind per-cpu timeout threads to each CPU References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> <536D3184.9070302@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 May 2014 23:49:50 -0000 > Yup. I've just done that. > > http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff Thanks, that'll work. > Which workloads are you thinking about? Maybe we could introduce some > higher level description of which CPU(s) at boot time to do "freebsd > stuff" on, and then don't start things like pcpu swi's and NIC threads > on those CPUs. A classic case is partitioning cores into control and data plane groups. I'm sure there are lots more. What's nice about cpuset is that the choice and change can be dynamic, so long as there aren't pinned threads in the default set. An option to restrict FreeBSD pCPU threads to a subset could be useful. > Can you think of situations where we'd want to have per-cpu swi's even > _running_ for CPUs that you want to dedicate to other things? There's > nothing stopping you from scheduling a callout on a different target > CPU. At least for the uses I know, it's complete isolation from other processing, kernel threads included. The 'freebsd stuff' info you mentioned should be sufficient. later, Peter. From owner-freebsd-arch@FreeBSD.ORG Sat May 10 00:20:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AF39FD8; Sat, 10 May 2014 00:20:49 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9CC6772; Sat, 10 May 2014 00:20:48 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so5242078qgf.7 for ; Fri, 09 May 2014 17:20:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=z/NGzAbfzsqtTB0epJuYP9Viu9R+ee/zmLHUOYulrR0=; b=G10tBMUq29gxHndTCR2fkES5rT9NVrryTWKSnBLVFJY24yGgbivxL8JJ+fI8pkqtrj S0B8uOwsoGOsde6Yave0vmDLc91jpUJxjszVsQqE7jofEud3Af7U6eU4MQsx1tbUENBK NEhGMQB3K21X0po7QPhDtVPbmr/gxUjKg5csHeuIfuphAhTeAiIn9x8hJWRnJwTsGXSE bDTD5RYwgtwaWW7e7K2gnI1/9TmA+BSLKymxBTriBWwl9L81Q/CYcvJnFWx3/IipM1Zz f5RV8FzlN+PMD9lHsfnlMzfgqUWWNk5vNdTk3g3Ad72wFj9QsRv03sZZR7wxcULQWmPH 9zUg== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr19685052qaa.76.1399681248046; Fri, 09 May 2014 17:20:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 17:20:47 -0700 (PDT) In-Reply-To: <536D6999.6080402@freebsd.org> References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> <536D3184.9070302@freebsd.org> <536D6999.6080402@freebsd.org> Date: Fri, 9 May 2014 17:20:47 -0700 X-Google-Sender-Auth: QkuKd_5f0R3lKJzD650D5r2BSVc Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 00:20:49 -0000 On 9 May 2014 16:49, Peter Grehan wrote: >> Yup. I've just done that. >> >> http://people.freebsd.org/~adrian/norse/20140509-swi-pin-1.diff > > > Thanks, that'll work. > > >> Which workloads are you thinking about? Maybe we could introduce some >> higher level description of which CPU(s) at boot time to do "freebsd >> stuff" on, and then don't start things like pcpu swi's and NIC threads >> on those CPUs. > > > A classic case is partitioning cores into control and data plane groups. > I'm sure there are lots more. What's nice about cpuset is that the choice > and change can be dynamic, so long as there aren't pinned threads in the > default set. > > An option to restrict FreeBSD pCPU threads to a subset could be useful. Cool. >> Can you think of situations where we'd want to have per-cpu swi's even >> _running_ for CPUs that you want to dedicate to other things? There's >> nothing stopping you from scheduling a callout on a different target >> CPU. > At least for the uses I know, it's complete isolation from other > processing, kernel threads included. The 'freebsd stuff' info you mentioned > should be sufficient. Cool. I'll look at committing this stuff in the next few hours. It can always easily be undone. I'll revisit the "limit pcpu threads to cpuids " idea later. -a From owner-freebsd-arch@FreeBSD.ORG Sat May 10 00:54:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4F45DAC2; Sat, 10 May 2014 00:54:24 +0000 (UTC) Received: from mail-qc0-x229.google.com (mail-qc0-x229.google.com [IPv6:2607:f8b0:400d:c01::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE9A6DDA; Sat, 10 May 2014 00:54:23 +0000 (UTC) Received: by mail-qc0-f169.google.com with SMTP id e16so5480455qcx.28 for ; Fri, 09 May 2014 17:54:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=7mLHky1NeM8Eoa2WfbeMcKeXL6KSsRd2dQq8QnQD5aA=; b=wX8rqP8GqjDlxtMIr9+cWYT5fF+bCeFFLXw+BwXCd/io09lC+T7MsrRUR6k1sarC3B Az4dV6nsbLSUlv7njFljwQxuqtxlJ78hN8S89TkJQMv8p1kmkFuWr9/QH3z1YjN2bbOF b3DRcIYHuXtTZGhdeZjgLGI7VaNBkNVwtueVQ2uZfaOr9+7vD3ozkRR4OfBdmieyxMPf J7klQguoV3/FkA7lLG0XTADnJReBRXa6FGfmU3Az2aBmEzgLSC1RE+f0GoHG5PNsPxc4 HMfFFVvINewAeLq92q4VbYiixoBmqbfhWTqRDDq3oFow+D2c1hr4tcDc7KSdxCiZanq5 FEMg== MIME-Version: 1.0 X-Received: by 10.224.129.66 with SMTP id n2mr19432510qas.55.1399683263007; Fri, 09 May 2014 17:54:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 9 May 2014 17:54:22 -0700 (PDT) In-Reply-To: References: <530508B7.7060102@FreeBSD.org> <201405091349.14381.jhb@freebsd.org> <536D3184.9070302@freebsd.org> <536D6999.6080402@freebsd.org> Date: Fri, 9 May 2014 17:54:22 -0700 X-Google-Sender-Auth: VqfTPhKEPQOmMmFnqP-JGWL36V0 Message-ID: Subject: Re: [rfc] bind per-cpu timeout threads to each CPU From: Adrian Chadd To: Peter Grehan Content-Type: text/plain; charset=UTF-8 Cc: Alexander Motin , freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 May 2014 00:54:24 -0000 ok, I've committed it but I've left the default at "don't pin." That way the existing behaviour hasn't changed and it's easy to flip on to play with. Thanks for the feedback John / Peter! -a From owner-freebsd-arch@FreeBSD.ORG Mon May 12 00:40:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C49B389; Mon, 12 May 2014 00:40:32 +0000 (UTC) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60C4024E2; Mon, 12 May 2014 00:40:32 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id ey11so7156822pad.4 for ; Sun, 11 May 2014 17:40:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+Npo4i30kggNiHPQeEnMEUqZ08kj5ole2IZ/Htzgkis=; b=MfsXQsa2tBwvXJol3Dmjor2+PEJoY4M8YJvCYEG2hph1tBJ/+T//2215Pu93Suk40U wiFu0qECBRQrlM7NAMMEPwZaCqvF/1yRnv8qCQcwBGZ/1LujrXuqBcX56guPKkD44enu fEEid0xIlBgHJl4BajOjinuHSH6YbCWq09QaAfa4Yg+XNHkctLLLH6CaFqZmwgQwLJUj 1kX4WLoNgMjLRqgcdWpqNPoUnII1dunxF3Nib/ja6mnIxxsAItnA7gMrFLic1VjJNH7D GmG0GEgoKhztlRS4AxDIB5G9x/fU4jfjjlF1ZQL87fEBWZyFO/GyeQ4XQRaV1B3dmEQ4 AD4A== MIME-Version: 1.0 X-Received: by 10.66.244.176 with SMTP id xh16mr48869974pac.20.1399855232033; Sun, 11 May 2014 17:40:32 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.66.73.34 with HTTP; Sun, 11 May 2014 17:40:31 -0700 (PDT) In-Reply-To: References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Date: Sun, 11 May 2014 17:40:31 -0700 X-Google-Sender-Auth: 68Ocn8Ty8f-Am-VFC51U3IjUL-c Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Kevin Oberman To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" , Ian Smith , "freebsd-acpi@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 00:40:32 -0000 On Fri, May 9, 2014 at 1:36 PM, Adrian Chadd wrote: > cool! > > next; > > # pkg install intel-pcm > # kldload cpuctl > # pcm.x 1 > > See what it reports. > OK. Any documentation on what this is supposed to tell me? Some of it makes perfect sense and some baffles me. I see C-states of C1 and C6 when on AC and C1, C3, and C7 when on battery (and, of course, C0). FREQ vs. AFREQ look interesting, but I'm not sure I really understand the implications. The last few lines, from " PHYSICAL CORE IPC", are particularly mysterious to me. I can understand the words, but I think that they carry more significance than is obvious, at least to me. I'm not a hardware guy. -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com From owner-freebsd-arch@FreeBSD.ORG Mon May 12 15:14:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CDDC379F; Mon, 12 May 2014 15:14:09 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 752392FE3; Mon, 12 May 2014 15:14:09 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so7720160qge.22 for ; Mon, 12 May 2014 08:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=QIbCHbp0C1SqcWEd87wHH/HoozajtJh31AZaVbEZnzw=; b=zc2PoDOZ/e6WyPpkzJRw3k139d6DP6ugKZhGCQqlQaohFpSsd96cPnVnxwDOu0Z92D Vqh5Ech6pcCKdOXXHvOVww58+hdvpA87QEEMig10G86N7m9zx59++1mxaGOC8h8IDz9t 6MiAJniH3fZhokbs+6ukgDdRByptfBlw8A+Aa6f2rw3DgdKuG6j1BcJ53QBq4yWnZ0e/ eJp9AOF6fmu4E+UbET2qtA+YFgl5MlipmN19Dvj8S2C7gR2WpBGKInGQ6dc1eWG3AcU6 wLPoRfYmFLwur5dZeFK0LsJ4R29zmRfIZ6Tm0r4L9LgS2nwY+qSqyOcM84LNfNPpMgx2 Sszw== MIME-Version: 1.0 X-Received: by 10.224.38.138 with SMTP id b10mr12191463qae.98.1399907648569; Mon, 12 May 2014 08:14:08 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Mon, 12 May 2014 08:14:08 -0700 (PDT) In-Reply-To: References: <20140505011654.O11699@sola.nimnet.asn.au> <2223.1399233644@critter.freebsd.dk> <20140505153421.W11699@sola.nimnet.asn.au> <85787.1399271121@critter.freebsd.dk> <20140505163316.R11699@sola.nimnet.asn.au> <7681.1399629328@critter.freebsd.dk> Date: Mon, 12 May 2014 08:14:08 -0700 X-Google-Sender-Auth: pPW8a4SXPe8A-EAwASqWOsbjHUU Message-ID: Subject: Re: proposal: set default lid state to S3, performance/economy Cx states to Cmax From: Adrian Chadd To: Kevin Oberman Content-Type: text/plain; charset=UTF-8 Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" , Ian Smith , "freebsd-acpi@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2014 15:14:10 -0000 The documentation is uhm, what's on intel's (growing) blog post and responses: https://software.intel.com/en-us/articles/intel-performance-counter-monitor-a-better-way-to-measure-cpu-utilization Yes, we can / should add clearer documentation. I had to also go hunting in the source to figure some of it out. FREQ/AFREQ is just the current clock cycle counters / clock reference counter (TSC). Ie, a freq or afreq of 1.0 means the clock cycle counters == TSC counter. FREQ takes the sleep state into account (ie, only counts _running_ cycles.) AFREQ doesn't. So FREQ gives you a good indication of the running duty cycle versus the ideal maximum, and AFREQ tells you what frequency the core is running at versus the reference frequency. An AFREQ of < 1.0 means that the chip is underclocking that core. An AFREQ of > 1.0 means turboboost is on and it's overclocking that core. -a On 11 May 2014 17:40, Kevin Oberman wrote: > On Fri, May 9, 2014 at 1:36 PM, Adrian Chadd wrote: >> >> cool! >> >> next; >> >> # pkg install intel-pcm >> # kldload cpuctl >> # pcm.x 1 >> >> See what it reports. > > > OK. Any documentation on what this is supposed to tell me? Some of it makes > perfect sense and some baffles me. > > I see C-states of C1 and C6 when on AC and C1, C3, and C7 when on battery > (and, of course, C0). FREQ vs. AFREQ look interesting, but I'm not sure I > really understand the implications. The last few lines, from " PHYSICAL CORE > IPC", are particularly mysterious to me. I can understand the words, but I > think that they carry more significance than is obvious, at least to me. I'm > not a hardware guy. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com From owner-freebsd-arch@FreeBSD.ORG Thu May 15 04:43:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16DAE833; Thu, 15 May 2014 04:43:40 +0000 (UTC) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B86F32FAE; Thu, 15 May 2014 04:43:39 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id w7so917205qcr.34 for ; Wed, 14 May 2014 21:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=jhKEnfy+S3LIg3ExmQD+qvQG9QGh1+sT6wjto8q98/8=; b=ncf+edh1PaJ/r5WdUIrf7jqbfol5+jv+HsfNotJ/ogctKh/BN0EvL1HIX9eGeK+1EZ JaWBv1Yu7DADTixbB1usDWOEj3LqKw4IciZNA+pgd0N4K0JFmNeyqhB8bGmBR2d+CGVZ nSwIQHnpyw2QEzYtTVN4ecbS7mnExlY8gXzge/JcKPlkF33OBKyXNScdF3tiI6aFPdbv MJNAR03Llc8vRBo9LJCUoEEJ1Vfw/LD7Mx7SaARD6Fk3N3JwBIPN9m+vJUJnI9IDT0IO hV1Rr1qBtfFKf7+ECWBxrY541gveYsOv230wMn9EMbeoI60DaVaPPDybOoYWef9dJHu4 v9NA== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr9666950qaa.76.1400129018305; Wed, 14 May 2014 21:43:38 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Wed, 14 May 2014 21:43:38 -0700 (PDT) Date: Wed, 14 May 2014 21:43:38 -0700 X-Google-Sender-Auth: 615EBagYUG4pOjyHhtu_W3pPu-E Message-ID: Subject: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 04:43:40 -0000 Hi, here's a completely untested patch for discussion. I'm emailing it out mostly as a "is this a good idea" patch rather than a "it should just be committed" patch. The RSS stuff from Robert maps connections to pcbgroups based on the RSS hash, but it doesn't map the TCP timers the same way. So by default they're all on swi0 and the per CPU timers don't take the hash type or correctly choose the CPU. This patch: http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff does a few things: * it stores the hashtype in the inp as well as the flowid; * the rss code grows a new method that maps the flowid/hashtype to a cpuid * .. and the existing mbuf to cpuid method now uses this; * the tcp timer code now has an inline function that knows about RSS and defaults to the existing way of doing things if RSS isn't enabled. There's still a bunch of work left before all the lock contention compartmentalization and balancing of RSS is realised. I'm just chewing off the little corner bits that are easy to get done now. So, any comments? I'll give it a proper whirl on some 10G hardware in a few days but I thought I'd at least get the general idea out there for comment. Thanks, -a From owner-freebsd-arch@FreeBSD.ORG Thu May 15 05:26:27 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2EC5E1C; Thu, 15 May 2014 05:26:27 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E4EF226B; Thu, 15 May 2014 05:26:27 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so931392qgf.35 for ; Wed, 14 May 2014 22:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=CpDsdyDMfcRI9cGao7L4svj6N3YB5E4hD50EtYZNPBE=; b=zH7gVi035I+gHEmEMJgaflF690C8o3trLsbqTY4yJENwWOL2UeR1Hzr2ti5pfh3cMs 6GO05pP+fEGqd3ONZTKbFmjO5WQfqIfT4k59yPvlmEDiM8zIDUmbN+9XP+MPUHx2CynQ KyIw305O1mSY7nvNo6MGOp0URzAKQyDDUrUxPsnZ5Uz4qAUtNjsLMXUnB3JM/gnxY9+p xoI3tgfySuU4fB4kuDRaaVYdsloRiTDo8cCFtYlxQXm1mXt8c7nCg01jBt+c8nrl+rTl jRRoO9IqQ9p/ONgNWqUzJyZgrOVIJULXuIYBZRLGnE3zUGrPLKvEGylHM942ujeyABeV p2nQ== MIME-Version: 1.0 X-Received: by 10.224.51.2 with SMTP id b2mr9844840qag.49.1400131586528; Wed, 14 May 2014 22:26:26 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Wed, 14 May 2014 22:26:26 -0700 (PDT) In-Reply-To: References: Date: Wed, 14 May 2014 22:26:26 -0700 X-Google-Sender-Auth: g60kka5dSxI-4RNDAXWB8t4jZCs Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 05:26:27 -0000 .. and i've done a little more than no testing, so far so good. http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-2.diff This adds IP_FLOWID, IP_FLOWTYPE and IP_RSSCPUID to fetch the socket flowid, flowtype and cpuid from the inp. It's mostly for debugging for now. Thanks, -a On 14 May 2014 21:43, Adrian Chadd wrote: > Hi, > > here's a completely untested patch for discussion. I'm emailing it out > mostly as a "is this a good idea" patch rather than a "it should just > be committed" patch. > > The RSS stuff from Robert maps connections to pcbgroups based on the > RSS hash, but it doesn't map the TCP timers the same way. So by > default they're all on swi0 and the per CPU timers don't take the hash > type or correctly choose the CPU. > > This patch: > > http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff > > does a few things: > > * it stores the hashtype in the inp as well as the flowid; > * the rss code grows a new method that maps the flowid/hashtype to a cpuid > * .. and the existing mbuf to cpuid method now uses this; > * the tcp timer code now has an inline function that knows about RSS > and defaults to the existing way of doing things if RSS isn't enabled. > > There's still a bunch of work left before all the lock contention > compartmentalization and balancing of RSS is realised. I'm just > chewing off the little corner bits that are easy to get done now. > > So, any comments? I'll give it a proper whirl on some 10G hardware in > a few days but I thought I'd at least get the general idea out there > for comment. > > Thanks, > > > > > -a From owner-freebsd-arch@FreeBSD.ORG Thu May 15 13:36:52 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC3DB183; Thu, 15 May 2014 13:36:52 +0000 (UTC) Received: from mail-oa0-x22d.google.com (mail-oa0-x22d.google.com [IPv6:2607:f8b0:4003:c02::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5294728EE; Thu, 15 May 2014 13:36:52 +0000 (UTC) Received: by mail-oa0-f45.google.com with SMTP id l6so1242572oag.4 for ; Thu, 15 May 2014 06:36:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=9dXpo/Gq0pksI6O0kXDlpJ7jsTBDvXKgM/j4HLh8zZ4=; b=hwnkFnDC1ifolGhTD5VYGU8Slj179x9PhqDDWatQtKjuuxl4LoAvfer3ZqirkMOSGD SObnPLICB4CQd3yF/NlmBFjiAws90spMYCguABnGTYERu2aPhpLJ+QYGYRNkj7BRM/SA Hf+FMt2CSGXu0zkzhU+M92sG0ok2OIFHAGnk+zeyuFHYlH28UQObZy0zH8Dxy47Epacy 7uk3Bb3fL+eun1gYKu9mfInV1XZtFgilSzGEHQ6a9dhhuHArcQOmFZGv0HENZtxemgPN kbHLC8jSvKsx5sRo7Y64UEKhtwDOXC7tkUfYIzz0mqz/oc5ubaaz7lwE2f/EVcq/uY4I 9m9w== MIME-Version: 1.0 X-Received: by 10.60.92.132 with SMTP id cm4mr10082336oeb.49.1400161011519; Thu, 15 May 2014 06:36:51 -0700 (PDT) Received: by 10.182.216.197 with HTTP; Thu, 15 May 2014 06:36:51 -0700 (PDT) Date: Thu, 15 May 2014 15:36:51 +0200 Message-ID: Subject: [RFC] remove dead code from link_elf.c From: Oliver Pinter To: current@freebsd.org, arch@freebsd.org Content-Type: multipart/mixed; boundary=047d7b33d3c6b8550504f9706375 Cc: gavin@freebsd.org, kib@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 13:36:52 -0000 --047d7b33d3c6b8550504f9706375 Content-Type: text/plain; charset=ISO-8859-1 Hi all! I found that the SPARSE_MAPPING used only in sys/kern/link_elf.c file, and no other place in kernel, nor in generated codes in /usr/obj/... op@pandora-d opBSD.git> git grep -i SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING sys/kern/link_elf.c:#ifdef SPARSE_MAPPING I proposed to remove the old/dead codes. Patch attached. Tested on amd64. --047d7b33d3c6b8550504f9706375 Content-Type: application/octet-stream; name="0001-link_elf-remove-SPARSE_MAPPING-and-related-code.patch" Content-Disposition: attachment; filename="0001-link_elf-remove-SPARSE_MAPPING-and-related-code.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: file0 RnJvbSAwZDBjNzBmNjI5YWUwYzRhYjBjOTkwYTMzZGYzN2UxYTc2NmE4NDIwIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBPbGl2ZXIgUGludGVyIDxvbGl2ZXIucG50ckBnbWFpbC5jb20+ CkRhdGU6IFRodSwgMTUgTWF5IDIwMTQgMTU6MTk6MjQgKzAyMDAKU3ViamVjdDogW1BBVENIXSBs aW5rX2VsZjogcmVtb3ZlIFNQQVJTRV9NQVBQSU5HIGFuZCByZWxhdGVkIGNvZGUKClNpZ25lZC1v ZmYtYnk6IE9saXZlciBQaW50ZXIgPG9saXZlci5wbnRyQGdtYWlsLmNvbT4KLS0tCiBzeXMva2Vy bi9saW5rX2VsZi5jIHwgNTIgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLQogMSBmaWxlIGNoYW5nZWQsIDUyIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdp dCBhL3N5cy9rZXJuL2xpbmtfZWxmLmMgYi9zeXMva2Vybi9saW5rX2VsZi5jCmluZGV4IDYzMWJh NzUuLjIyZmQ1ODQgMTAwNjQ0Ci0tLSBhL3N5cy9rZXJuL2xpbmtfZWxmLmMKKysrIGIvc3lzL2tl cm4vbGlua19lbGYuYwpAQCAtNTUsMTEgKzU1LDYgQEAgX19GQlNESUQoIiRGcmVlQlNEJCIpOwog CiAjaW5jbHVkZSA8dm0vdm0uaD4KICNpbmNsdWRlIDx2bS92bV9wYXJhbS5oPgotI2lmZGVmIFNQ QVJTRV9NQVBQSU5HCi0jaW5jbHVkZSA8dm0vdm1fb2JqZWN0Lmg+Ci0jaW5jbHVkZSA8dm0vdm1f a2Vybi5oPgotI2luY2x1ZGUgPHZtL3ZtX2V4dGVybi5oPgotI2VuZGlmCiAjaW5jbHVkZSA8dm0v cG1hcC5oPgogI2luY2x1ZGUgPHZtL3ZtX21hcC5oPgogCkBAIC03Nyw5ICs3Miw2IEBAIHR5cGVk ZWYgc3RydWN0IGVsZl9maWxlIHsKIAlzdHJ1Y3QgbGlua2VyX2ZpbGUgbGY7CQkvKiBDb21tb24g ZmllbGRzICovCiAJaW50CQlwcmVsb2FkZWQ7CS8qIFdhcyBmaWxlIHByZS1sb2FkZWQgKi8KIAlj YWRkcl90CQlhZGRyZXNzOwkvKiBSZWxvY2F0aW9uIGFkZHJlc3MgKi8KLSNpZmRlZiBTUEFSU0Vf TUFQUElORwotCXZtX29iamVjdF90CW9iamVjdDsJCS8qIFZNIG9iamVjdCB0byBob2xkIGZpbGUg cGFnZXMgKi8KLSNlbmRpZgogCUVsZl9EeW4JCSpkeW5hbWljOwkvKiBTeW1ib2wgdGFibGUgZXRj LiAqLwogCUVsZl9IYXNoZWx0CW5idWNrZXRzOwkvKiBEVF9IQVNIIGluZm8gKi8KIAlFbGZfSGFz aGVsdAluY2hhaW5zOwpAQCAtMzk0LDkgKzM4Niw2IEBAIGxpbmtfZWxmX2luaXQodm9pZCogYXJn KQogCWVmID0gKGVsZl9maWxlX3QpIGxpbmtlcl9rZXJuZWxfZmlsZTsKIAllZi0+cHJlbG9hZGVk ID0gMTsKIAllZi0+YWRkcmVzcyA9IDA7Ci0jaWZkZWYgU1BBUlNFX01BUFBJTkcKLQllZi0+b2Jq ZWN0ID0gMDsKLSNlbmRpZgogCWVmLT5keW5hbWljID0gZHA7CiAKIAlpZiAoZHAgIT0gTlVMTCkK QEAgLTY3MSw5ICs2NjAsNiBAQCBsaW5rX2VsZl9saW5rX3ByZWxvYWQobGlua2VyX2NsYXNzX3Qg Y2xzLAogCWVmLT5wcmVsb2FkZWQgPSAxOwogCWVmLT5tb2RwdHIgPSBtb2RwdHI7CiAJZWYtPmFk ZHJlc3MgPSAqKGNhZGRyX3QgKiliYXNlcHRyOwotI2lmZGVmIFNQQVJTRV9NQVBQSU5HCi0JZWYt Pm9iamVjdCA9IDA7Ci0jZW5kaWYKIAlkcCA9ICh2bV9vZmZzZXRfdCllZi0+YWRkcmVzcyArICoo dm1fb2Zmc2V0X3QgKilkeW5wdHI7CiAJZWYtPmR5bmFtaWMgPSAoRWxmX0R5biAqKWRwOwogCWxm LT5hZGRyZXNzID0gZWYtPmFkZHJlc3M7CkBAIC04ODMsMjQgKzg2OSw3IEBAIGxpbmtfZWxmX2xv YWRfZmlsZShsaW5rZXJfY2xhc3NfdCBjbHMsIGNvbnN0IGNoYXIqIGZpbGVuYW1lLAogCX0KIAog CWVmID0gKGVsZl9maWxlX3QpIGxmOwotI2lmZGVmIFNQQVJTRV9NQVBQSU5HCi0JZWYtPm9iamVj dCA9IHZtX29iamVjdF9hbGxvY2F0ZShPQkpUX0RFRkFVTFQsIG1hcHNpemUgPj4gUEFHRV9TSElG VCk7Ci0JaWYgKGVmLT5vYmplY3QgPT0gTlVMTCkgewotCQllcnJvciA9IEVOT01FTTsKLQkJZ290 byBvdXQ7Ci0JfQotCWVmLT5hZGRyZXNzID0gKGNhZGRyX3QpIHZtX21hcF9taW4oa2VybmVsX21h cCk7Ci0JZXJyb3IgPSB2bV9tYXBfZmluZChrZXJuZWxfbWFwLCBlZi0+b2JqZWN0LCAwLAotCSAg ICAodm1fb2Zmc2V0X3QgKikgJmVmLT5hZGRyZXNzLCBtYXBzaXplLCAwLCBWTUZTX09QVElNQUxf U1BBQ0UsCi0JICAgIFZNX1BST1RfQUxMLCBWTV9QUk9UX0FMTCwgMCk7Ci0JaWYgKGVycm9yICE9 IDApIHsKLQkJdm1fb2JqZWN0X2RlYWxsb2NhdGUoZWYtPm9iamVjdCk7Ci0JCWVmLT5vYmplY3Qg PSAwOwotCQlnb3RvIG91dDsKLQl9Ci0jZWxzZQogCWVmLT5hZGRyZXNzID0gbWFsbG9jKG1hcHNp emUsIE1fTElOS0VSLCBNX1dBSVRPSyk7Ci0jZW5kaWYKIAltYXBiYXNlID0gZWYtPmFkZHJlc3M7 CiAKIAkvKgpAQCAtOTE3LDE5ICs4ODYsNiBAQCBsaW5rX2VsZl9sb2FkX2ZpbGUobGlua2VyX2Ns YXNzX3QgY2xzLCBjb25zdCBjaGFyKiBmaWxlbmFtZSwKIAkJYnplcm8oc2VnYmFzZSArIHNlZ3Nb aV0tPnBfZmlsZXN6LAogCQkgICAgc2Vnc1tpXS0+cF9tZW1zeiAtIHNlZ3NbaV0tPnBfZmlsZXN6 KTsKIAotI2lmZGVmIFNQQVJTRV9NQVBQSU5HCi0JCS8qCi0JCSAqIFdpcmUgZG93biB0aGUgcGFn ZXMKLQkJICovCi0JCWVycm9yID0gdm1fbWFwX3dpcmUoa2VybmVsX21hcCwKLQkJICAgICh2bV9v ZmZzZXRfdCkgc2VnYmFzZSwKLQkJICAgICh2bV9vZmZzZXRfdCkgc2VnYmFzZSArIHNlZ3NbaV0t PnBfbWVtc3osCi0JCSAgICBWTV9NQVBfV0lSRV9TWVNURU18Vk1fTUFQX1dJUkVfTk9IT0xFUyk7 Ci0JCWlmIChlcnJvciAhPSBLRVJOX1NVQ0NFU1MpIHsKLQkJCWVycm9yID0gRU5PTUVNOwotCQkJ Z290byBvdXQ7Ci0JCX0KLSNlbmRpZgogCX0KIAogI2lmZGVmIEdQUk9GCkBAIC0xMDg1LDE2ICsx MDQxLDggQEAgbGlua19lbGZfdW5sb2FkX2ZpbGUobGlua2VyX2ZpbGVfdCBmaWxlKQogCQlyZXR1 cm47CiAJfQogCi0jaWZkZWYgU1BBUlNFX01BUFBJTkcKLQlpZiAoZWYtPm9iamVjdCAhPSBOVUxM KSB7Ci0JCXZtX21hcF9yZW1vdmUoa2VybmVsX21hcCwgKHZtX29mZnNldF90KSBlZi0+YWRkcmVz cywKLQkJICAgICh2bV9vZmZzZXRfdCkgZWYtPmFkZHJlc3MKLQkJICAgICsgKGVmLT5vYmplY3Qt PnNpemUgPDwgUEFHRV9TSElGVCkpOwotCX0KLSNlbHNlCiAJaWYgKGVmLT5hZGRyZXNzICE9IE5V TEwpCiAJCWZyZWUoZWYtPmFkZHJlc3MsIE1fTElOS0VSKTsKLSNlbmRpZgogCWlmIChlZi0+c3lt YmFzZSAhPSBOVUxMKQogCQlmcmVlKGVmLT5zeW1iYXNlLCBNX0xJTktFUik7CiAJaWYgKGVmLT5z dHJiYXNlICE9IE5VTEwpCi0tIAoxLjkuMgoK --047d7b33d3c6b8550504f9706375-- From owner-freebsd-arch@FreeBSD.ORG Thu May 15 13:50:13 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8857176C; Thu, 15 May 2014 13:50:13 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA61D29F4; Thu, 15 May 2014 13:50:12 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.8/8.14.8) with ESMTP id s4FDo3Kh040309; Thu, 15 May 2014 16:50:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s4FDo3Kh040309 Received: (from kostik@localhost) by tom.home (8.14.8/8.14.8/Submit) id s4FDo3sr040306; Thu, 15 May 2014 16:50:03 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 15 May 2014 16:50:03 +0300 From: Konstantin Belousov To: Oliver Pinter Subject: Re: [RFC] remove dead code from link_elf.c Message-ID: <20140515135003.GO74331@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wmfuW8osuO2pi9jF" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: arch@freebsd.org, gavin@freebsd.org, current@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 13:50:13 -0000 --wmfuW8osuO2pi9jF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, May 15, 2014 at 03:36:51PM +0200, Oliver Pinter wrote: > Hi all! >=20 > I found that the SPARSE_MAPPING used only in sys/kern/link_elf.c file, > and no other place in kernel, nor in generated codes in /usr/obj/... >=20 > op@pandora-d opBSD.git> git grep -i SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING > sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >=20 > I proposed to remove the old/dead codes. Patch attached. Tested on amd64. I agree that SPARSE_MAPPING could go away. But your testing is void, since link_elf.c is practically not used on amd64, which utilizes object file format for klds. --wmfuW8osuO2pi9jF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBAgAGBQJTdMYKAAoJEJDCuSvBvK1Bu34P/ikDUz/j+RAq/Xk/dMznoHFx 27A7ym8WOOA4EN/GnCMdOgN7CJNCFeBBLNpBtfafISNJlxUbjJAJVFkxv3WuR591 tivRbG+0o26+3cSe8jF6CB4p7zfpG9qJ3qvXyo3RMeuWxZrBTKUk4JOy3rgK08+j WdBFybCHqyZnskCQcJTs9Hbifhnnb+La/YBkAfK1vP5h8UPrh7GUY8ES9gntn44R okKQKXvGGyZtM652Gb/ViIFtTdmjRW2VBjNyhPSXwcC3DTJOKFW/2sc7wJP6IiKg 8tOr2UvD01wqDJ9QTsMtIgaKXhHdiUbi7CRxoKE6JWamCat8YJ+K93x/HA1DXj8m ZqoIQ7mSVY/8LumnyOktN+9Qc1TYUVcEpQrbooAXELaHrRkwHUCkdcBTmipSr9LN C873CAakAP4+w4ZkWRWcewXAG0zZanYjBuvRVaLfgknT8ehzKFrBgOCTm/jd9hR2 jznuMcoQkXtKKXlHNIcprOZXb2rJvb/qndETqITo35oBcB2Fu9do3qMDsjKDuH9n aFFXHVB84EQxwZ6TGVexioegewkrRiIlsx3Cj/Su+RvMGocTDrgcLMEKBoI6eGzd FGOE74PbVN3ZVDLcGWbfeUIYuHEgg0gzIfeKsbqAu5c7SeUD/8/D1bMYK2mWVA35 fvBB5dgSfpK07RMs8dkb =uqHx -----END PGP SIGNATURE----- --wmfuW8osuO2pi9jF-- From owner-freebsd-arch@FreeBSD.ORG Thu May 15 14:19:37 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 968606B7; Thu, 15 May 2014 14:19:37 +0000 (UTC) Received: from mail-oa0-x22c.google.com (mail-oa0-x22c.google.com [IPv6:2607:f8b0:4003:c02::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48F8B2CA9; Thu, 15 May 2014 14:19:37 +0000 (UTC) Received: by mail-oa0-f44.google.com with SMTP id o6so1314394oag.17 for ; Thu, 15 May 2014 07:19:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WRya+kgIcK7Ynia8q8YG9npSypMp4e19ubGHqfOsUUg=; b=KmKkJEbtDjfMWt0scwGERTNf42GD4/c9ci/ZwgCvJI1nZa5gEjf7uuh53Y+u0WuZcU CONJTknqjB3SZwDCaxZrTBdbgCzT6LuQE+CR8geqP2ZRftPq0EfuFy+QaTkSSus5AKoH EGdala49xxOXOn0q5xn0KR33HdUW1eL1I8gvGq/nUJgyVdRLpzKMyAGf9BoJ9WAYtLnR Q3I8Yy1fCUx+hlotxFyzeTN1+TD3RfHVSMxzmW98qHng3d3rEGXeDfXcMuUPIToW16qd lfSIkN/mI5lHovqlRUdSAA7jC7LLKA8UgUaCajNd4gLLJCx+MoLHtY509eEmiy4Z7+yf +Izg== MIME-Version: 1.0 X-Received: by 10.182.229.101 with SMTP id sp5mr10520801obc.52.1400163576643; Thu, 15 May 2014 07:19:36 -0700 (PDT) Received: by 10.182.216.197 with HTTP; Thu, 15 May 2014 07:19:36 -0700 (PDT) In-Reply-To: <20140515135003.GO74331@kib.kiev.ua> References: <20140515135003.GO74331@kib.kiev.ua> Date: Thu, 15 May 2014 16:19:36 +0200 Message-ID: Subject: Re: [RFC] remove dead code from link_elf.c From: Oliver Pinter To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: arch@freebsd.org, gavin@freebsd.org, current@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 14:19:37 -0000 On 5/15/14, Konstantin Belousov wrote: > On Thu, May 15, 2014 at 03:36:51PM +0200, Oliver Pinter wrote: >> Hi all! >> >> I found that the SPARSE_MAPPING used only in sys/kern/link_elf.c file, >> and no other place in kernel, nor in generated codes in /usr/obj/... >> >> op@pandora-d opBSD.git> git grep -i SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> sys/kern/link_elf.c:#ifdef SPARSE_MAPPING >> >> I proposed to remove the old/dead codes. Patch attached. Tested on amd64. > > I agree that SPARSE_MAPPING could go away. But your testing is void, > since link_elf.c is practically not used on amd64, which utilizes object > file format for klds. > Hi kib@ Thanks for the quick response. From owner-freebsd-arch@FreeBSD.ORG Thu May 15 15:16:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AEE742C for ; Thu, 15 May 2014 15:16:29 +0000 (UTC) Received: from mail-ie0-f171.google.com (mail-ie0-f171.google.com [209.85.223.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 397D32266 for ; Thu, 15 May 2014 15:16:28 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id rl12so1148788iec.2 for ; Thu, 15 May 2014 08:16:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:content-type :content-transfer-encoding:subject:message-id:date:to:mime-version; bh=2w1wS29CNDSAhjcHapOoKR0RvjKd4UPtEP5U/Rzwf68=; b=JRodGJWCoEtKMQD62/Mud+tghT6YaHyoKdJzM9WOG2kIRJ4X+al0G/TK8C6pcmPI1p ypnpbvRdXFvBLIVCaNbeLocCVi0lVxZ06By+eHwt0IfEnU5IAmWDg9m9UjUND+1ZTlAm rAowH1xl1VP7owIgnA2Nx1beeelm9r97Mipm01Cc/T/xW/GBzBVv+m2MNQPsx7Ys1Qre WbbyBOTSj5BPuCJU2FVfdONdblDRcEdz2yYM06gJcAh8u1wGRbeCTZNatj9y6k7HJuxY KvJHTDkSnEwbSSa5wTtaL1JO4D9BQ+1mREH5xL9xf9Y/MD3BOLq1f827HjW8bNTWhcF3 pyDg== X-Gm-Message-State: ALoCoQm1gRN4bLEPOjJ2f6VnTULmeAH/prOR2mxiMiOY+dGpzbOzU2NzXDnLjCEEZZJDQFVFbUWw X-Received: by 10.43.137.5 with SMTP id im5mr3249120icc.69.1400166987889; Thu, 15 May 2014 08:16:27 -0700 (PDT) Received: from [10.69.210.117] ([137.122.64.30]) by mx.google.com with ESMTPSA id e5sm13730685igl.20.2014.05.15.08.16.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 15 May 2014 08:16:27 -0700 (PDT) Sender: Julio Merino From: Julio Merino Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Make atf libraries private Message-Id: Date: Thu, 15 May 2014 11:16:25 -0400 To: freebsd-arch@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 May 2014 15:16:29 -0000 Hello, I'd like to make the libatf-c and libatf-c++ private, both in CURRENT = _and_ in stable/10. These libraries are not built by default yet. Only when WITH_TESTS=3Dyes = is defined the libraries are built and installed. Once installed, nothing outside of the base system should use them with = one exception. The exception are a bunch of atf-related ports (lutok, = kyua*) that can use these libraries if and only if the TESTS ports = option is enabled. So... I don't think moving the library impacts binary compatibility in = any significant way. Plan: bring atf port up to date, move libraries to the private = directory, make tests link to the libraries in the new location and add = the old libraries to the obsolete list. I will move the libraries sometime next week unless there are objections = to be discussed. Or submit earlier if somebody confirms this is OK. Thanks!= From owner-freebsd-arch@FreeBSD.ORG Fri May 16 04:27:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABA501CF; Fri, 16 May 2014 04:27:11 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0182.outbound.protection.outlook.com [207.46.163.182]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9CF824EF; Fri, 16 May 2014 04:27:09 +0000 (UTC) Received: from BY2PR05CA046.namprd05.prod.outlook.com (10.141.250.36) by BY2PR05MB111.namprd05.prod.outlook.com (10.242.38.26) with Microsoft SMTP Server (TLS) id 15.0.944.11; Fri, 16 May 2014 04:11:24 +0000 Received: from BN1AFFO11FD041.protection.gbl (2a01:111:f400:7c10::159) by BY2PR05CA046.outlook.office365.com (2a01:111:e400:2c5f::36) with Microsoft SMTP Server (TLS) id 15.0.944.11 via Frontend Transport; Fri, 16 May 2014 04:11:24 +0000 Received: from P-EMF03-SAC.jnpr.net (66.129.239.17) by BN1AFFO11FD041.mail.protection.outlook.com (10.58.52.252) with Microsoft SMTP Server (TLS) id 15.0.939.9 via Frontend Transport; Fri, 16 May 2014 04:11:23 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF03-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Thu, 15 May 2014 21:10:10 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.24.29.229]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s4G4A3L54478; Thu, 15 May 2014 21:10:08 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos.jnpr.net (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id E4469580A1; Thu, 15 May 2014 21:10:03 -0700 (PDT) To: Julio Merino Subject: Re: Make atf libraries private In-Reply-To: References: Comments: In-reply-to: Julio Merino message dated "Thu, 15 May 2014 11:16:25 -0400." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Thu, 15 May 2014 21:10:03 -0700 Message-ID: <20140516041003.E4469580A1@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:66.129.239.17; CTRY:US; IPV:NLI; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(6009001)(199002)(189002)(77156001)(88136002)(81156002)(86362001)(93916002)(92566001)(21056001)(87936001)(92726001)(87286001)(53416003)(79102001)(90896003)(46102001)(77982001)(64706001)(70486001)(62966002)(48376002)(47776003)(20776003)(81342001)(50466002)(89996001)(558084003)(33656001)(76482001)(85852003)(81542001)(80022001)(83072002)(4396001)(50226001)(44976005)(74662001)(31966008)(99396002)(83322001)(77096999)(6806004)(97736001)(102836001)(76176999)(74502001)(102176002)(68736004)(69596002)(84676001)(50986999)(42262001); DIR:OUT; SFP:; SCL:1; SRVR:BY2PR05MB111; H:P-EMF03-SAC.jnpr.net; FPR:; MLV:sfv; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Forefront-PRVS: 02135EB356 Received-SPF: SoftFail (: domain of transitioning juniper.net discourages use of 66.129.239.17 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.17) smtp.mailfrom=sjg@juniper.net; X-OriginatorOrg: juniper.net Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 04:27:11 -0000 >I'd like to make the libatf-c and libatf-c++ private, both in CURRENT _and_ in > stable/10. Can you briefly explain why From owner-freebsd-arch@FreeBSD.ORG Fri May 16 17:47:17 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 527A8FCB; Fri, 16 May 2014 17:47:17 +0000 (UTC) Received: from mail-qg0-x232.google.com (mail-qg0-x232.google.com [IPv6:2607:f8b0:400d:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03845276A; Fri, 16 May 2014 17:47:16 +0000 (UTC) Received: by mail-qg0-f50.google.com with SMTP id z60so4661956qgd.23 for ; Fri, 16 May 2014 10:47:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=3fn/7a/yqhgBNu5ZYOwoheSNoBzqrpBRWoHfCP6v9zQ=; b=D0pim9RdvganPv0jklB+MjLiRlBX0Wn5AOoAfof23uvaDU4PcGXMjCiPml4lffWdA5 GrVMWPlbYa0zgPlxidBHeDEVcKYZ0f05uPj+DvoDTOAgEQ7EbA2uiXafJi6yudkoCoaW kb7rhU2HdpvgGW+YQQCkeWisymPeKB895a6wP1ZW1Owb1fbgjwemh4jVvwF36+WKpe/r 6kb/JczjR5yVIHLlFU6ROc3XFVY7AxVBYeG4/Q/DSDTmHXn8vcLh/jjV8wmg+FhXW9dd fGQPsPD2MYprLNfgwnF8mleELKUCaq3G0b6pxv2/nqKpZn4N43/SPJGApvSz0hBFNSKH 8OUg== MIME-Version: 1.0 X-Received: by 10.224.37.10 with SMTP id v10mr21295121qad.98.1400262436155; Fri, 16 May 2014 10:47:16 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 16 May 2014 10:47:16 -0700 (PDT) In-Reply-To: References: Date: Fri, 16 May 2014 10:47:16 -0700 X-Google-Sender-Auth: sLjaowIUnoP9phseKygkcJhL_n4 Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 17:47:17 -0000 Ok, I've given this a whirl on a slightly larger system. There's no 10Gbit/sec in it yet, but it's stable under 64,000 sockets at 1Gbit/sec. I'm going to commit this over the next couple of days unless there are any objections. The defaults are still the same so it won't affect the rest of you. -a On 14 May 2014 22:26, Adrian Chadd wrote: > .. and i've done a little more than no testing, so far so good. > > http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-2.diff > > This adds IP_FLOWID, IP_FLOWTYPE and IP_RSSCPUID to fetch the socket > flowid, flowtype and cpuid from the inp. It's mostly for debugging for > now. > > Thanks, > > > > -a > > > On 14 May 2014 21:43, Adrian Chadd wrote: >> Hi, >> >> here's a completely untested patch for discussion. I'm emailing it out >> mostly as a "is this a good idea" patch rather than a "it should just >> be committed" patch. >> >> The RSS stuff from Robert maps connections to pcbgroups based on the >> RSS hash, but it doesn't map the TCP timers the same way. So by >> default they're all on swi0 and the per CPU timers don't take the hash >> type or correctly choose the CPU. >> >> This patch: >> >> http://people.freebsd.org/~adrian/norse/20140514-tcp-rss-timers-1.diff >> >> does a few things: >> >> * it stores the hashtype in the inp as well as the flowid; >> * the rss code grows a new method that maps the flowid/hashtype to a cpuid >> * .. and the existing mbuf to cpuid method now uses this; >> * the tcp timer code now has an inline function that knows about RSS >> and defaults to the existing way of doing things if RSS isn't enabled. >> >> There's still a bunch of work left before all the lock contention >> compartmentalization and balancing of RSS is realised. I'm just >> chewing off the little corner bits that are easy to get done now. >> >> So, any comments? I'll give it a proper whirl on some 10G hardware in >> a few days but I thought I'd at least get the general idea out there >> for comment. >> >> Thanks, >> >> >> >> >> -a From owner-freebsd-arch@FreeBSD.ORG Fri May 16 21:31:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BD4B790 for ; Fri, 16 May 2014 21:31:06 +0000 (UTC) Received: from mail-qg0-f46.google.com (mail-qg0-f46.google.com [209.85.192.46]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9972B12 for ; Fri, 16 May 2014 21:31:06 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id q108so5043868qgd.5 for ; Fri, 16 May 2014 14:30:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-type; bh=xzBKRxk0LsUYE1Rbf4eZNGrV8r0ZbPbMjD7+nu34roE=; b=Gddokdy9AKCFthlzX4Pg7lgKyjgoHzIROb6LtYhk4bEPzHUnVzkvr+7DXx160ZVx8D vMicRPxwlXPaar85UTgj5Bhf1qIN7OXVejn2xJbMb7Uv3VuT8Rl4pxqoBVUf198SF9uo HDecfvNKEUt3JOt4sFjkDXjD/nm7ej1sUR223YS8j59/9q54xF6P4RsDpQD8ezsTNlvi iJDMfzJLtu4o9fUF7/0ZI8fw2LIxdjnVxEZsSFLaMx7tj4G0jwC2ZyWOHw0kTNhKNV/5 +ucWkzLFhcWud7jpvEX+HvpcsPZAaaQ+w5SBdDhgT4Mil8FMnq4DCR8yKDXkY8Wr/ifV aSxg== X-Gm-Message-State: ALoCoQlkUyU07KnRQAH0QCizkpLuEjchg7xNgRHIUzkozkj9b3vKAUoz/DhZZcMIgaZULYXzNxkM X-Received: by 10.224.80.195 with SMTP id u3mr27302709qak.37.1400275859698; Fri, 16 May 2014 14:30:59 -0700 (PDT) MIME-Version: 1.0 Sender: jmmv@meroh.net Received: by 10.96.83.99 with HTTP; Fri, 16 May 2014 14:30:38 -0700 (PDT) X-Originating-IP: [137.122.64.53] In-Reply-To: <20140516041003.E4469580A1@chaos.jnpr.net> References: <20140516041003.E4469580A1@chaos.jnpr.net> From: Julio Merino Date: Fri, 16 May 2014 17:30:38 -0400 X-Google-Sender-Auth: ODXH0onMe_DDHni8qH1ALyLTUsM Message-ID: Subject: Re: Make atf libraries private To: "Simon J. Gerraty" Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2014 21:31:06 -0000 On Fri, May 16, 2014 at 12:10 AM, Simon J. Gerraty wrote: >>I'd like to make the libatf-c and libatf-c++ private, both in CURRENT _and_ in >> stable/10. > > Can you briefly explain why Oh sure. Basically because I do not want the potentially-ancient ATF libraries in base to leak to external components, and because I do not want to have to worry about maintaining compatibility with these ancient versions when the time comes. (This will start happening as soon as FreeBSD 10 ages, for example.) At some point, ports that install tests (lutok, kyua, shtk, ...) *will* require a newer version of atf. Being able to just update devel/atf within ports and get the newer versions of the other ports working is trivial. Having to coordinate the ports updates with newer FreeBSD releases just so that those ports can be updated will be a nightmare. It seems nicer to me to restrict the ATF libraries in base to what FreeBSD needs them for: i.e. the ATF-based test programs inside the tree. Anything else can just install the external package from ports OR it can explicitly link against the private library -- clearly knowing what the compatibility-related consequences of doing so are. (As an aside, it seems that FreeBSD, as a project, wants to move as many libraries into private as possible partly for the reasons mentioned above. This came up during the devsummit. I'm wondering if this was documented anywhere...) From owner-freebsd-arch@FreeBSD.ORG Sat May 17 00:13:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A2451A9; Sat, 17 May 2014 00:13:44 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE52527EA; Sat, 17 May 2014 00:13:43 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so5284361qge.8 for ; Fri, 16 May 2014 17:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=ez2nZuSIk5nTYc25Bj2BvHaR2z32pWp5UE37c+DXWBI=; b=aMucMvGfD4VztmyRoCcKu0LthF5ELKwivUULVC+tVs8mfkAy67CD6VqtmDt+tpXD5v 9t1kJB/3OClln4gt7HJl8CVdwVB1x4BUhnMujbISjDlc6FHxIZab1tL33kbilkbc+xdJ GD2q/rp3U7Jp5V6wHMXLQTCUQFqi1VMAmb27aq/a+gBZv7hoaikyQUmeabA9rIVHrA/s 7tqjCQ98g9Y59vQJQ+0LHI50eRpQUyw1P7DnGuP8fFFeI1erjBy9FnxQtoPc8usjD7PC Oe2x23pil3fWcvFwNeKcYN8+14tcflhj2cjD5nHi54WXQXb6Q8jh+vrkOCUERnnvrLim ekoQ== MIME-Version: 1.0 X-Received: by 10.140.22.209 with SMTP id 75mr28925779qgn.4.1400285622901; Fri, 16 May 2014 17:13:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Fri, 16 May 2014 17:13:42 -0700 (PDT) In-Reply-To: References: Date: Fri, 16 May 2014 17:13:42 -0700 X-Google-Sender-Auth: wYZxX76B31db0QZ5ydIri50MoGA Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 May 2014 00:13:44 -0000 On 16 May 2014 10:47, Adrian Chadd wrote: > Ok, I've given this a whirl on a slightly larger system. There's no > 10Gbit/sec in it yet, but it's stable under 64,000 sockets at > 1Gbit/sec. > > I'm going to commit this over the next couple of days unless there are > any objections. The defaults are still the same so it won't affect the > rest of you. http://people.freebsd.org/~adrian/norse/20140517-tcp-rss-timers-3.diff I've committed the in.h IP_* fields for now, just to get that out of the way. This patch fixes the inp_cpu lookup code to actually be right. :-) -a From owner-freebsd-arch@FreeBSD.ORG Sun May 18 13:49:25 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1878D323 for ; Sun, 18 May 2014 13:49:25 +0000 (UTC) Received: from magneto.web.itd.umich.edu (magneto.web.itd.umich.edu [141.211.145.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7F372B69 for ; Sun, 18 May 2014 13:49:24 +0000 (UTC) Received: (from www@localhost) by magneto.web.itd.umich.edu (8.12.11.20060308/8.12.9) id s4IDnMC1027622; Sun, 18 May 2014 09:49:22 -0400 Date: Sun, 18 May 2014 09:49:22 -0400 To: freebsd-arch@freebsd.org From: =?UTF-8?Q?Robert_Hartwick?= Subject: =?UTF-8?Q?Top_of_the_day_to_you=21=21=21?= Message-ID: <737121d48450bd8770c2112271e003a4@cases.soe.umich.edu> X-Priority: 3 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="us-ascii" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 13:49:25 -0000 Top of the day to you!!! I have an offer involving large sum of money which= I intend to invest in your country.I need a partner to handle the receivin= g of the funds and investment procedure until I am discharged from the mili= tary.can you get back to me with your email address so that i send you the = full details? thank you.I'm Sgt.Robert Hartwick of the US army. From owner-freebsd-arch@FreeBSD.ORG Sun May 18 23:44:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 960862B8; Sun, 18 May 2014 23:44:24 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A30F2774; Sun, 18 May 2014 23:44:24 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id a108so7529598qge.36 for ; Sun, 18 May 2014 16:44:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=adycisgBehZMJaGd7jW/wmspl8OHPL4fhmk6fxrpFBM=; b=ix3fyDLVicTJn3vfd/V4+JYshmwuN1Nj5W0YYFpUmrzW+Kf/Tw4Ogi34Qsc5RdHH9G UA4ee3QpRnwvGEm8QBQxX/JWgbK+oeiNIk062dUgxZ2retGwlR4OB/F0LvYZ+kR/nV7P wl6fk9uVthG7bVPwv+KNBxzumc9LEBbYai/CPJxrLXFATy4RQMqDSHZYUUQi4LQfkUxc 6izNA/bwoYRMW9DOxhqXNntV/M3LcUkkb2PSsNMWYP/1kiquF3LOIjPN+1co8gvxeQFD gjtGoJQBWoC40BpaxJQoLHcF4wRnBSlUUDLUWgFt/JQMtH4NTVrjzs2vLc1JXSSbLGGY o/9w== MIME-Version: 1.0 X-Received: by 10.224.137.1 with SMTP id u1mr42220050qat.73.1400456663494; Sun, 18 May 2014 16:44:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 18 May 2014 16:44:23 -0700 (PDT) Date: Sun, 18 May 2014 16:44:23 -0700 X-Google-Sender-Auth: wWW3g-T1NLhWgsHe8cBIux6F_Vs Message-ID: Subject: [rfc] teach netstat about flowid/flowtype From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 May 2014 23:44:24 -0000 Hi! My next hack - adding flowid/flowtype to netstat. It's netstat -R. It's like -x, but listing RSS/flowid stuff. http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff This is useful for both RSS and non-RSS debugging - NICs that populate the mbuf flowid will start to show flowid's up in netstat. It doesn't currently show the rss CPUID as that isn't cached in the inpcb. I'll worry about adding that later on. Any issues with me just committing this? I promise I'll email doc@ to get it documented. Thanks, -a From owner-freebsd-arch@FreeBSD.ORG Mon May 19 01:38:47 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D142099A; Mon, 19 May 2014 01:38:47 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A11002F3E; Mon, 19 May 2014 01:38:47 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-232-70.lns20.per1.internode.on.net [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s4J1cgcW057102 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 18 May 2014 18:38:45 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5379609E.6000702@freebsd.org> Date: Mon, 19 May 2014 09:38:38 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Adrian Chadd , "freebsd-arch@freebsd.org" , FreeBSD Net Subject: Re: [rfc] teach netstat about flowid/flowtype References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 01:38:47 -0000 On 5/19/14, 7:44 AM, Adrian Chadd wrote: > Hi! > > My next hack - adding flowid/flowtype to netstat. It's netstat -R. > It's like -x, but listing RSS/flowid stuff. > > http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff > > This is useful for both RSS and non-RSS debugging - NICs that populate > the mbuf flowid will start to show flowid's up in netstat. > > It doesn't currently show the rss CPUID as that isn't cached in the > inpcb. I'll worry about adding that later on. > > Any issues with me just committing this? I promise I'll email doc@ to > get it documented. If you don't I'll get "you-know-who" to nag you.. I need a good Italian meal some time.. > > Thanks, > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Mon May 19 05:12:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EAA344B5; Mon, 19 May 2014 05:12:34 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE862FD1; Mon, 19 May 2014 05:12:34 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so7877734qgf.7 for ; Sun, 18 May 2014 22:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=XGbmWea/pHBL1Y2cyazgKn0kbT41OrKge0U6tO7daH0=; b=ZXN0eAbCtBDBCbn+tNan9oJ813Me2RxryZedKH9+zNc+I1YmGE9Me+DzOZw6mslL23 vs3WTou1tvaHjsVBik6uOLZ5+lQF8coNOKBPcRziNT4XlbVaqO8i6XJXFjYZbAHAfH2x WXfd/JEva/LX1fOjV7B+alpf8XFz5KPIEBJGJ0Cs15cPONzz+CzSv5UzJr2273JUNadn OY6sT0vmtC57O22vsBbpiicLqJBZAyAm5zuyhNtdG2JifjDy8kfUvzvpJtMJnpPT3yx9 ob39u1EYxgj7BW6LUdpnCndofGh3B2o8oEi/TPyBQXtrGvNAw4u1QRCdC0M2hWWyS932 Nbag== MIME-Version: 1.0 X-Received: by 10.224.37.10 with SMTP id v10mr39749773qad.98.1400476353722; Sun, 18 May 2014 22:12:33 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 18 May 2014 22:12:33 -0700 (PDT) In-Reply-To: <5379609E.6000702@freebsd.org> References: <5379609E.6000702@freebsd.org> Date: Sun, 18 May 2014 22:12:33 -0700 X-Google-Sender-Auth: yfWCFhb7l-MyLK3lD725Nu8VIgU Message-ID: Subject: Re: [rfc] teach netstat about flowid/flowtype From: Adrian Chadd To: Julian Elischer Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2014 05:12:35 -0000 *laugh* On 18 May 2014 18:38, Julian Elischer wrote: > On 5/19/14, 7:44 AM, Adrian Chadd wrote: >> >> Hi! >> >> My next hack - adding flowid/flowtype to netstat. It's netstat -R. >> It's like -x, but listing RSS/flowid stuff. >> >> http://people.freebsd.org/~adrian/norse/20140518-netstat-rss-1.diff >> >> This is useful for both RSS and non-RSS debugging - NICs that populate >> the mbuf flowid will start to show flowid's up in netstat. >> >> It doesn't currently show the rss CPUID as that isn't cached in the >> inpcb. I'll worry about adding that later on. >> >> Any issues with me just committing this? I promise I'll email doc@ to >> get it documented. > > If you don't I'll get "you-know-who" to nag you.. I need a good Italian meal > some time.. You should just visit her. She can show you what your "oh that's a crystal set" introduction spawned into. :-) -a From owner-freebsd-arch@FreeBSD.ORG Tue May 20 11:24:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7AB25BE for ; Tue, 20 May 2014 11:24:21 +0000 (UTC) Received: from mail-ig0-x248.google.com (mail-ig0-x248.google.com [IPv6:2607:f8b0:4001:c05::248]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9689B2A15 for ; Tue, 20 May 2014 11:24:21 +0000 (UTC) Received: by mail-ig0-f200.google.com with SMTP id uy17so1857370igb.3 for ; Tue, 20 May 2014 04:24:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:message-id:date:subject:from:to:content-type; bh=X7sE7kGO4iu3IB9kNjfr7TBihns+qb5AO8Zs/B3Y/kk=; b=Zxbj8IyMh80i1NS+Sp+mbtoEjbZ/eLnZi8AzUgz3lpBVmTGaJPDT36+wOxfiJuTDrt X84+BBsm84mgf95fJkZq2/0v/ZrqBQG256qLsJBwcf6k7prdKcDSCe6pjT04WQi3vn8m seKoTFO+K7hxyiAnn3bXSUSfwQ95Gr++Enzg7QiIKhIm9nUmuN9RY23/TKbzhqCJZ8iu MStx3enczw8UXgG3uHDYnggAURgbA3H1Ro9qePQvUe9duWLqu40p6TuHKF7ViP9isH5o yLBRWIfuD8d7nP6SZB9ATeYqBLu2wyNfbl9TzpYm9qc3ep7VBbDVT2DgGqI7g52e775/ 6Lzw== MIME-Version: 1.0 X-Received: by 10.42.233.133 with SMTP id jy5mr5171851icb.32.1400585061113; Tue, 20 May 2014 04:24:21 -0700 (PDT) Message-ID: <047d7bacc0d20ba34c04f9d31f0e@google.com> Date: Tue, 20 May 2014 11:24:21 +0000 Subject: www.freebsd.org From: Angel To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2014 11:24:22 -0000 Hi I found your contact over the web and wanted to send you a quick note. With Search Engine Optimization and Web Development, I can help your business achieve better ranking on prominent search engines like Google, Bing & others to generate more leads/sales. This may look like one of those spurious foreign emails you get in your inbox every day that promise big but delivers nothing. Just to be upfront we are happy to discuss your requirements. So, let me know if you are interested in receiving further information/quote with no strings attached from our team of SEO & Web experts. Best regards, Angel Web Expert / Specialist *HubScope* SEO Melbourne | Sydney | Perth | Brisbane | Adelaide & Hobart Disclaimer: We respect your privacy and want to make sure you are aware of a few things. By replying to this email, you authorize our affiliates that can help with your project to call you at the number you provided, and you understand that they may use automated phone technology to call you. At no time are you required to make a purchase. From owner-freebsd-arch@FreeBSD.ORG Wed May 21 06:52:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9369FB72; Wed, 21 May 2014 06:52:29 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36A8921E0; Wed, 21 May 2014 06:52:29 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so2547897qgf.31 for ; Tue, 20 May 2014 23:52:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=RDG+x4DCmBYG/qjveJA5meUavBAeO68jrQV9pVfwBYg=; b=APF4heMJRPHurqhsZBMtKiRbIdTMWLp3RE73X+0AZuvAKNHXvzae547eTn8mJVf/gg IU1IHBDJfnELq9jnko2eL+XdUwhCTecnnf/LPfum0brDFBPOLRzegD9/xAM+ig19mXLX 8jQhudkOLWRBbXwA8gwItQ5XvG2Tpqw7gAHaYb3u7EdzOg1u4yfx+2pz6RZ8Wj2lOgEU sr1KIcHbuYu9SJv+CVoDgIWDIKTNyBATTCzjNTjyXlkF5OCCX2s55/+Gh3KtZ/baft+b ylSzO4pyZDweSOKO4htHuYG62kaodA5Z1eAh5taCjnEZvMR2MaNIwICKV81STuuGZFHh Z5mA== MIME-Version: 1.0 X-Received: by 10.224.129.66 with SMTP id n2mr65395338qas.55.1400655148371; Tue, 20 May 2014 23:52:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Tue, 20 May 2014 23:52:28 -0700 (PDT) Date: Tue, 20 May 2014 23:52:28 -0700 X-Google-Sender-Auth: c_9tztid7o7-85D9wxCKnQecXU8 Message-ID: Subject: [rfc] add non-contiguous CPU ID support to in_rss.c From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" , Robert Watson Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2014 06:52:29 -0000 Hi Robert, This patch uses CPU_FIRST() and CPU_NEXT() to iterate over the CPU IDs. Think this is alright? -a Index: sys/netinet/in_rss.c =================================================================== --- sys/netinet/in_rss.c (revision 266429) +++ sys/netinet/in_rss.c (working copy) @@ -176,6 +176,7 @@ rss_init(__unused void *arg) { u_int i; + u_int cpuid; /* * Validate tunables, coerce to sensible values. @@ -245,11 +246,12 @@ /* * Set up initial CPU assignments: round-robin by default. - * - * XXXRW: Need a mapping to non-contiguous IDs here. */ - for (i = 0; i < rss_buckets; i++) - rss_table[i].rte_cpu = i % rss_ncpus; + cpuid = CPU_FIRST(); + for (i = 0; i < rss_buckets; i++) { + rss_table[i].rte_cpu = cpuid; + cpuid = CPU_NEXT(cpuid); + } /* * Randomize rrs_key. From owner-freebsd-arch@FreeBSD.ORG Mon May 26 14:48:59 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 23DD379F for ; Mon, 26 May 2014 14:48:59 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8A662A5F for ; Mon, 26 May 2014 14:48:58 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WowCv-000Krv-66 for freebsd-arch@FreeBSD.org; Mon, 26 May 2014 14:48:57 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s4QEmsbb050997 for ; Mon, 26 May 2014 08:48:54 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX18Figx81p/turnOiPN8j1qV Subject: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds From: Ian Lepore To: freebsd-arch Content-Type: multipart/mixed; boundary="=-iwxWkDDku4c4XVLMNASv" Date: Mon, 26 May 2014 08:48:53 -0600 Message-ID: <1401115733.1152.339.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 14:48:59 -0000 --=-iwxWkDDku4c4XVLMNASv Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Based on a request from Justin Gibbs and suggestions from Jilles and Warner on how to accomplish it, I've cobbled together a mechanism for fine grained build order controls on parallel subdir builds. This augments the coarse .WAIT mechanism, which is still useful if you've got a situation where "almost everything depends on A and B". Because the parallel subdir mechanism uses non-obvious mangling of target names, which should probably remain a private detail of the implementation, it's not easy to do things like "libfoo: libbar", so instead the new mechanism lets you set a variable that lists dependencies: SUBDIR_DEPEND_libfoo= libgroodah libpouet Note that while I'm using libraries as an example here, it really has nothing to do with the generated library files. This is really saying "build in directory libfoo after building in the libgroodah and libpouet directories." Is this kind of mechanism acceptable to everyone? I've updated lib/Makefile with dependency information based on the old almost-accurate comment block and by combing through lib/* makefiles looking for LDADD dependencies to other libraries within lib/*. This passed a universe run for me with -j12, but that takes about 5 hours so I can't run it through a lot of different -j numbers very easily. -- Ian --=-iwxWkDDku4c4XVLMNASv Content-Disposition: inline; filename="subdir_depend.diff" Content-Type: text/x-patch; name="subdir_depend.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: share/mk/bsd.subdir.mk =================================================================== --- share/mk/bsd.subdir.mk (revision 266650) +++ share/mk/bsd.subdir.mk (working copy) @@ -80,7 +80,11 @@ __subdir_targets= __subdir_targets+= .WAIT .else __subdir_targets+= ${__target}_subdir_${__dir} -${__target}_subdir_${__dir}: .MAKE +__deps= +.for __dep in ${SUBDIR_DEPEND_${__dir}} +__deps+= ${__target}_subdir_${__dep} +.endfor +${__target}_subdir_${__dir}: .MAKE ${__deps} @${_+_}set -e; \ if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ Index: lib/Makefile =================================================================== --- lib/Makefile (revision 266650) +++ lib/Makefile (working copy) @@ -3,82 +3,43 @@ .include -# To satisfy shared library or ELF linkage when only the libraries being -# built are visible: -# -# csu must be built before all shared libaries for ELF. -# libc must be built before all other shared libraries. -# libbsm must be built before libauditd. -# libcom_err must be built before libpam. -# libcrypt must be built before libpam. -# libkvm must be built before libdevstat. -# libldns must be built before libunbound. -# msun must be built before libg++ and libstdc++. -# libmd must be built before libatm, libopie, libradius, and libtacplus. -# ncurses must be built before libdialog, libedit and libreadline. -# libnetgraph must be built before libbsnmp/modules/snmp_netgraph. -# libopie must be built before libpam. -# libradius must be built before libpam. -# librpcsvc must be built before libpam. -# libsbuf must be built before libcam. -# libtacplus must be built before libpam. -# libutil must be built before libpam. -# libypclnt must be built before libpam. -# libgssapi must be built before librpcsec_gss -# -# Otherwise, the SUBDIR list should be in alphabetical order. -# -# Except it appears bind needs to be compiled last +# The SUBDIR_ORDERED list is a small set of libraries which are used by many +# of the other libraries. These are built first with a .WAIT between them +# and the main list to avoid needing a SUBDIR_DEPEND line on every library +# naming just these few items. SUBDIR_ORDERED= ${_csu} \ .WAIT \ libc \ libc_nonshared \ - .WAIT \ - msun \ - .WAIT \ - libbsm \ - libauditd \ - libutil \ - libpjdlog \ - libnv \ - ${_libcapsicum} \ libcompiler_rt \ - libcrypt \ + ${_libcplusplus} \ + ${_libcxxrt} \ libelf \ - ${_libiconv_modules} \ - libkvm \ - ${_libldns} \ - libmd \ - ncurses \ - ${_libnetgraph} \ - libradius \ - librpcsvc \ - libsbuf \ - libtacplus \ - ${_libypclnt} \ - ${_libcxxrt} \ - ${_libcplusplus} + msun -.if ${MK_KERBEROS_SUPPORT} != "no" -SUBDIR_ORDERED+= libcom_err -.endif +# The main list; please keep these sorted alphabetically. SUBDIR= ${SUBDIR_ORDERED} \ .WAIT \ libalias \ libarchive \ ${_libatm} \ + libauditd \ libbegemot \ libblocksruntime \ ${_libbluetooth} \ ${_libbsnmp} \ libbsdstat \ + libbsm \ libbz2 \ libcalendar \ libcam \ + ${_libcapsicum} \ ${_libcasper} \ + ${_libcom_err} \ libcompat \ + libcrypt \ libdevinfo \ libdevstat \ libdwarf \ @@ -91,26 +52,36 @@ SUBDIR= ${SUBDIR_ORDERED} \ ${_libgpib} \ ${_libgssapi} \ ${_librpcsec_gss} \ + ${_libiconv_modules} \ libipsec \ libjail \ libkiconv \ + libkvm \ + ${_libldns} \ liblzma \ libmagic \ libmandoc \ libmemstat \ + libmd \ ${_libmilter} \ ${_libmp} \ ${_libnandfs} \ libnetbsd \ + ${_libnetgraph} \ ${_libngatm} \ + libnv \ libopie \ libpam \ libpcap \ + libpjdlog \ ${_libpmc} \ ${_libproc} \ libprocstat \ + libradius \ + librpcsvc \ librt \ ${_librtld_db} \ + libsbuf \ ${_libsdp} \ ${_libsm} \ ${_libsmb} \ @@ -119,6 +90,7 @@ SUBDIR= ${SUBDIR_ORDERED} \ libstand \ libstdbuf \ libstdthreads \ + libtacplus \ ${_libtelnet} \ ${_libthr} \ libthread_db \ @@ -129,15 +101,52 @@ SUBDIR= ${SUBDIR_ORDERED} \ ${_libunbound} \ ${_libusbhid} \ ${_libusb} \ + libutil \ ${_libvgl} \ ${_libvmmapi} \ libwrap \ liby \ + ${_libypclnt} \ libz \ + ncurses \ ${_atf} \ ${_clang} \ ${_tests} +# Inter-library dependencies. When the makefile for a library contains LDADD +# libraries, those libraries should be listed as build order dependencies here. + +SUBDIR_DEPEND_libarchive= libz libbz2 libexpat liblzma libmb +SUBDIR_DEPEND_libatm= libmd +SUBDIR_DEPEND_libauditdm= libbsm +SUBDIR_DEPEND_libbsnmp= ${_libnetgraph} +SUBDIR_DEPEND_libc++= libcxxrt +SUBDIR_DEPEND_libc= libcompiler_rt +SUBDIR_DEPEND_libcam= libsbuf +SUBDIR_DEPEND_libcapsicum= libnv +SUBDIR_DEPEND_libcasper= libcapsicum libnv libpjdlog +SUBDIR_DEPEND_libdevstat= libkvm +SUBDIR_DEPEND_libdevstat= libkvm +SUBDIR_DEPEND_libdiaglog= ncurses +SUBDIR_DEPEND_libedit= ncurses +SUBDIR_DEPEND_libg++= msun +SUBDIR_DEPEND_libgeom= libexpat libsbuf +SUBDIR_DEPEND_liblibrpcsec_gss= libgssapi +SUBDIR_DEPEND_libmagic= libz +SUBDIR_DEPEND_libmemstat= libkvm +SUBDIR_DEPEND_libopie= libmd +SUBDIR_DEPEND_libpam= libcrypt libopie libradius librpcsvc libtacplus libutil ${_libypclnt} ${_libcom_err} +SUBDIR_DEPEND_libpjdlog= libutil +SUBDIR_DEPEND_libprocstat= libkvm libutil +SUBDIR_DEPEND_libradius= libmd +SUBDIR_DEPEND_libreadline= ncurses +SUBDIR_DEPEND_libsmb= libkiconv +SUBDIR_DEPEND_libstdc++= msun +SUBDIR_DEPEND_libtacplus= libmb +SUBDIR_DEPEND_libtacplus= libmd +SUBDIR_DEPEND_libulog= libmd +SUBDIR_DEPEND_libunbound= ${_libldns} + .if exists(${.CURDIR}/csu/${MACHINE_ARCH}-elf) _csu=csu/${MACHINE_ARCH}-elf .elif exists(${.CURDIR}/csu/${MACHINE_ARCH}) @@ -185,6 +194,10 @@ _librpcsec_gss= librpcsec_gss _libiconv_modules= libiconv_modules .endif +.if ${MK_KERBEROS_SUPPORT} != "no" +_libcom_err= libcom_err +.endif + .if ${MK_LDNS} != "no" _libldns= libldns .endif --=-iwxWkDDku4c4XVLMNASv-- From owner-freebsd-arch@FreeBSD.ORG Wed May 28 04:59:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80D8EEDD; Wed, 28 May 2014 04:59:18 +0000 (UTC) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E81E2A36; Wed, 28 May 2014 04:59:18 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id wm4so10399581obc.12 for ; Tue, 27 May 2014 21:59:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=KRji66ieKoJcTjoYQAX5qP3GConv1LlID7fxVRcALVM=; b=MsBPrevDMh8CQhI3dU3k0Syzjowcm35iJI6rhXS78aPCbdENVpQqOi+/o0S5Jc7qaA GBBEcmZDRtrhH+WWzzJOFQHwk33KyqXnJ6uSFEA2vmB1F9wfb8SlNi/CU8wlu5wKgaRQ iqmAGuNy+0CISAuOxdz5FFGIohqZXZ72INRpOfta5xPR2ucMLrbY6lfSJSnFpZG2LWf/ QdJ55fA7QATfzyRfOjWfROni73FcyPBYS/Hwrzu/apcsNkmykNpvTOwU+8Z/NxhOZaf5 dL/L04xotScgrY53L0T4vqiXfLHQh1sH4hG1KPRCvQrGBCgsD2alOlgin86YGNfL4UTS ofmA== X-Received: by 10.60.161.6 with SMTP id xo6mr9214546oeb.78.1401253157568; Tue, 27 May 2014 21:59:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.123.178 with HTTP; Tue, 27 May 2014 21:58:46 -0700 (PDT) In-Reply-To: <1401115733.1152.339.camel@revolution.hippie.lan> References: <1401115733.1152.339.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Wed, 28 May 2014 12:58:46 +0800 Message-ID: Subject: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds To: Ian Lepore Content-Type: multipart/mixed; boundary=089e0117784db310c304fa6eac79 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 04:59:18 -0000 --089e0117784db310c304fa6eac79 Content-Type: text/plain; charset=UTF-8 It failed cleandir at libmb with -j4. Test script attached and log snippet below. Tested with: - HW: i5-3450 CPU w/ 8GB memory - /usr/obj & src mounted on tmpfs. src uses ~1GB without .svn dir. /usr/obj uses another ~2GB for buildworld, not including buildkernel. If memory is not constrained I think it is easier to use tmpfs to uncover parallel timing/race issues hidden by slower I/O. -Jia-shiun. --- 8< ---------------------------------------- ===> cddl/usr.bin/ctfdump (cleandir) --- bin.cleandir__D --- --- clean --- rm -f rcp rcp.o util.o rcp.1.gz rcp.1.cat.gz --- lib.cleandir__D --- --- cleanobj --- --- libexec.cleandir__D --- --- clean --- rm -f rpc.sprayd sprayd.o rpc.sprayd.8.gz rpc.sprayd.8.cat.gz --- bin.cleandir__D --- --- cleandepend --- rm -f .depend GPATH GRTAGS GSYMS GTAGS --- lib.cleandir__D --- make[3]: make[3]: don't know how to make cleandir_subdir_libmb. Stop make[3]: stopped in /mnt/src/lib *** [lib.cleandir__D] Error code 2 make[2]: stopped in /mnt/src --- libexec.cleandir__D --- --- cleandepend --- rm -f .depend GPATH GRTAGS GSYMS GTAGS --- bin.cleandir__D --- --- cleanobj --- A failure has been detected in another branch of the parallel make make[4]: stopped in /mnt/src/bin/rcp --- libexec.cleandir__D --- A failure has been detected in another branch of the parallel make make[4]: stopped in /mnt/src/libexec/rpc.sprayd --- bin.cleandir__D --- *** [cleandir_subdir_rcp] Error code 2 make[3]: stopped in /mnt/src/bin 1 error make[3]: stopped in /mnt/src/bin --- libexec.cleandir__D --- *** [_sub.cleandir] Error code 2 make[3]: stopped in /mnt/src/libexec 1 error make[3]: stopped in /mnt/src/libexec *** [libexec.cleandir__D] Error code 2 make[2]: stopped in /mnt/src --- bin.cleandir__D --- *** [bin.cleandir__D] Error code 2 make[2]: stopped in /mnt/src --- cddl.cleandir__D --- A failure has been detected in another branch of the parallel make make[5]: stopped in /mnt/src/cddl/usr.bin/ctfdump *** [_sub.cleandir] Error code 2 make[4]: stopped in /mnt/src/cddl/usr.bin 1 error make[4]: stopped in /mnt/src/cddl/usr.bin *** [_sub.cleandir] Error code 2 make[3]: stopped in /mnt/src/cddl 1 error make[3]: stopped in /mnt/src/cddl *** [cddl.cleandir__D] Error code 2 make[2]: stopped in /mnt/src 4 errors make[2]: stopped in /mnt/src *** [_cleanobj] Error code 2 make[1]: stopped in /mnt/src 1 error make[1]: stopped in /mnt/src *** [buildworld] Error code 2 make: stopped in /mnt/src make: stopped in /mnt/src --089e0117784db310c304fa6eac79 Content-Type: text/plain; charset=US-ASCII; name="run.sh.txt" Content-Disposition: attachment; filename="run.sh.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hvq5w5v61 IyEvYmluL3NoCgplY2hvID09PT09PT0gPj4gcGNtLmxvZwpwY20ueCAxIC1jc3YgPj4gcGNtLmxv ZyAmClBJRD0kIQoKZWNobyBwaWQgJFBJRAoKZGF0ZSA+PiBidWlsZC5sb2cKCm1ha2UgLWo0IGJ1 aWxkd29ybGQgPj4gYnVpbGQubG9nICYKUFBJRD0kIQplY2hvIHdhaXRpbmcuLi4Kd2FpdCAkUFBJ RAoKZGF0ZSA+PiBidWlsZC5sb2cKCmtpbGwgJFBJRAoKZWNobyBkb25lCgo= --089e0117784db310c304fa6eac79-- From owner-freebsd-arch@FreeBSD.ORG Wed May 28 13:21:44 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 915CADE4 for ; Wed, 28 May 2014 13:21:44 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F994266D for ; Wed, 28 May 2014 13:21:44 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WpcxB-000C41-DN; Wed, 28 May 2014 12:27:33 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s4SCRUd6001143; Wed, 28 May 2014 06:27:30 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/cIT/cXfoWOjPyf1IsoL/z Subject: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds From: Ian Lepore To: Jia-Shiun Li In-Reply-To: References: <1401115733.1152.339.camel@revolution.hippie.lan> Content-Type: multipart/mixed; boundary="=-W4vrQItYiX5TYo1HaM5E" Date: Wed, 28 May 2014 06:27:30 -0600 Message-ID: <1401280050.1152.377.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 13:21:44 -0000 --=-W4vrQItYiX5TYo1HaM5E Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, 2014-05-28 at 12:58 +0800, Jia-Shiun Li wrote: > It failed cleandir at libmb with -j4. Test script attached and log > snippet below. > > Tested with: > - HW: i5-3450 CPU w/ 8GB memory > - /usr/obj & src mounted on tmpfs. > > src uses ~1GB without .svn dir. /usr/obj uses another ~2GB for > buildworld, not including buildkernel. If memory is not constrained I > think it is easier to use tmpfs to uncover parallel timing/race issues > hidden by slower I/O. > > > -Jia-shiun. Doh! There was a typo, libmb should have been libmd. More importantly, it shows that my testing didn't test anything at all. I think I applied the patch in one sandbox and then ran make universe in a different one. Here's an updated patch (this time I'll try to test it correctly myself too). Thanks for testing. -- Ian --=-W4vrQItYiX5TYo1HaM5E Content-Disposition: attachment; filename="subdir_depend.diff" Content-Type: text/x-patch; name="subdir_depend.diff"; charset="us-ascii" Content-Transfer-Encoding: 7bit Index: share/mk/bsd.subdir.mk =================================================================== --- share/mk/bsd.subdir.mk (revision 266650) +++ share/mk/bsd.subdir.mk (working copy) @@ -80,7 +80,11 @@ __subdir_targets= __subdir_targets+= .WAIT .else __subdir_targets+= ${__target}_subdir_${__dir} -${__target}_subdir_${__dir}: .MAKE +__deps= +.for __dep in ${SUBDIR_DEPEND_${__dir}} +__deps+= ${__target}_subdir_${__dep} +.endfor +${__target}_subdir_${__dir}: .MAKE ${__deps} @${_+_}set -e; \ if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ Index: lib/Makefile =================================================================== --- lib/Makefile (revision 266650) +++ lib/Makefile (working copy) @@ -3,82 +3,43 @@ .include -# To satisfy shared library or ELF linkage when only the libraries being -# built are visible: -# -# csu must be built before all shared libaries for ELF. -# libc must be built before all other shared libraries. -# libbsm must be built before libauditd. -# libcom_err must be built before libpam. -# libcrypt must be built before libpam. -# libkvm must be built before libdevstat. -# libldns must be built before libunbound. -# msun must be built before libg++ and libstdc++. -# libmd must be built before libatm, libopie, libradius, and libtacplus. -# ncurses must be built before libdialog, libedit and libreadline. -# libnetgraph must be built before libbsnmp/modules/snmp_netgraph. -# libopie must be built before libpam. -# libradius must be built before libpam. -# librpcsvc must be built before libpam. -# libsbuf must be built before libcam. -# libtacplus must be built before libpam. -# libutil must be built before libpam. -# libypclnt must be built before libpam. -# libgssapi must be built before librpcsec_gss -# -# Otherwise, the SUBDIR list should be in alphabetical order. -# -# Except it appears bind needs to be compiled last +# The SUBDIR_ORDERED list is a small set of libraries which are used by many +# of the other libraries. These are built first with a .WAIT between them +# and the main list to avoid needing a SUBDIR_DEPEND line on every library +# naming just these few items. SUBDIR_ORDERED= ${_csu} \ .WAIT \ libc \ libc_nonshared \ - .WAIT \ - msun \ - .WAIT \ - libbsm \ - libauditd \ - libutil \ - libpjdlog \ - libnv \ - ${_libcapsicum} \ libcompiler_rt \ - libcrypt \ + ${_libcplusplus} \ + ${_libcxxrt} \ libelf \ - ${_libiconv_modules} \ - libkvm \ - ${_libldns} \ - libmd \ - ncurses \ - ${_libnetgraph} \ - libradius \ - librpcsvc \ - libsbuf \ - libtacplus \ - ${_libypclnt} \ - ${_libcxxrt} \ - ${_libcplusplus} + msun -.if ${MK_KERBEROS_SUPPORT} != "no" -SUBDIR_ORDERED+= libcom_err -.endif +# The main list; please keep these sorted alphabetically. SUBDIR= ${SUBDIR_ORDERED} \ .WAIT \ libalias \ libarchive \ ${_libatm} \ + libauditd \ libbegemot \ libblocksruntime \ ${_libbluetooth} \ ${_libbsnmp} \ libbsdstat \ + libbsm \ libbz2 \ libcalendar \ libcam \ + ${_libcapsicum} \ ${_libcasper} \ + ${_libcom_err} \ libcompat \ + libcrypt \ libdevinfo \ libdevstat \ libdwarf \ @@ -91,26 +52,36 @@ SUBDIR= ${SUBDIR_ORDERED} \ ${_libgpib} \ ${_libgssapi} \ ${_librpcsec_gss} \ + ${_libiconv_modules} \ libipsec \ libjail \ libkiconv \ + libkvm \ + ${_libldns} \ liblzma \ libmagic \ libmandoc \ libmemstat \ + libmd \ ${_libmilter} \ ${_libmp} \ ${_libnandfs} \ libnetbsd \ + ${_libnetgraph} \ ${_libngatm} \ + libnv \ libopie \ libpam \ libpcap \ + libpjdlog \ ${_libpmc} \ ${_libproc} \ libprocstat \ + libradius \ + librpcsvc \ librt \ ${_librtld_db} \ + libsbuf \ ${_libsdp} \ ${_libsm} \ ${_libsmb} \ @@ -119,6 +90,7 @@ SUBDIR= ${SUBDIR_ORDERED} \ libstand \ libstdbuf \ libstdthreads \ + libtacplus \ ${_libtelnet} \ ${_libthr} \ libthread_db \ @@ -129,15 +101,50 @@ SUBDIR= ${SUBDIR_ORDERED} \ ${_libunbound} \ ${_libusbhid} \ ${_libusb} \ + libutil \ ${_libvgl} \ ${_libvmmapi} \ libwrap \ liby \ + ${_libypclnt} \ libz \ + ncurses \ ${_atf} \ ${_clang} \ ${_tests} +# Inter-library dependencies. When the makefile for a library contains LDADD +# libraries, those libraries should be listed as build order dependencies here. + +SUBDIR_DEPEND_libarchive= libz libbz2 libexpat liblzma libmd +SUBDIR_DEPEND_libatm= libmd +SUBDIR_DEPEND_libauditdm= libbsm +SUBDIR_DEPEND_libbsnmp= ${_libnetgraph} +SUBDIR_DEPEND_libc++= libcxxrt +SUBDIR_DEPEND_libc= libcompiler_rt +SUBDIR_DEPEND_libcam= libsbuf +SUBDIR_DEPEND_libcapsicum= libnv +SUBDIR_DEPEND_libcasper= libcapsicum libnv libpjdlog +SUBDIR_DEPEND_libdevstat= libkvm +SUBDIR_DEPEND_libdiaglog= ncurses +SUBDIR_DEPEND_libedit= ncurses +SUBDIR_DEPEND_libg++= msun +SUBDIR_DEPEND_libgeom= libexpat libsbuf +SUBDIR_DEPEND_liblibrpcsec_gss= libgssapi +SUBDIR_DEPEND_libmagic= libz +SUBDIR_DEPEND_libmemstat= libkvm +SUBDIR_DEPEND_libopie= libmd +SUBDIR_DEPEND_libpam= libcrypt libopie libradius librpcsvc libtacplus libutil ${_libypclnt} ${_libcom_err} +SUBDIR_DEPEND_libpjdlog= libutil +SUBDIR_DEPEND_libprocstat= libkvm libutil +SUBDIR_DEPEND_libradius= libmd +SUBDIR_DEPEND_libreadline= ncurses +SUBDIR_DEPEND_libsmb= libkiconv +SUBDIR_DEPEND_libstdc++= msun +SUBDIR_DEPEND_libtacplus= libmd +SUBDIR_DEPEND_libulog= libmd +SUBDIR_DEPEND_libunbound= ${_libldns} + .if exists(${.CURDIR}/csu/${MACHINE_ARCH}-elf) _csu=csu/${MACHINE_ARCH}-elf .elif exists(${.CURDIR}/csu/${MACHINE_ARCH}) @@ -185,6 +192,10 @@ _librpcsec_gss= librpcsec_gss _libiconv_modules= libiconv_modules .endif +.if ${MK_KERBEROS_SUPPORT} != "no" +_libcom_err= libcom_err +.endif + .if ${MK_LDNS} != "no" _libldns= libldns .endif --=-W4vrQItYiX5TYo1HaM5E-- From owner-freebsd-arch@FreeBSD.ORG Wed May 28 16:34:10 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F0F9B5E; Wed, 28 May 2014 16:34:10 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46F5329FE; Wed, 28 May 2014 16:34:09 +0000 (UTC) Received: from jammyc-sslvpn-nc.jnpr.net ([66.129.239.12]) (authenticated bits=0) by mail.xcllnt.net (8.14.8/8.14.8) with ESMTP id s4SGY7vd030740 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 28 May 2014 09:34:08 -0700 (PDT) (envelope-from marcel@xcllnt.net) From: Marcel Moolenaar Content-Type: multipart/signed; boundary="Apple-Mail=_97E6AFBE-6D5F-4B52-B449-4178F1B82A5C"; protocol="application/pgp-signature"; micalg=pgp-sha1 Subject: Roadmap for ifnet(9) for FreeBSD 11 Date: Wed, 28 May 2014 09:34:01 -0700 Message-Id: To: "freebsd-arch@FreeBSD.org Arch" Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) X-Mailer: Apple Mail (2.1878.2) Cc: Anuranjan Shukla , Gleb Smirnoff X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 16:34:10 -0000 --Apple-Mail=_97E6AFBE-6D5F-4B52-B449-4178F1B82A5C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 All, The ifnet structure represents a network interface. Right now it is known to all NIC drivers as well as to all protocols. This means that whenever we change the structure, we need at minimum recompile all drivers, but usually also change them. This severely slow downs the development in this area and also makes it hard, if not, impossible to merge things back to stable branches. There were at least 3 efforts on fixing this: 1) Juniper=92s JUNOS is a FreeBSD based operating system that has its own (alternative) network stack, but that leverages the network drivers from FreeBSD. Juniper mechanically changed all ifnet dereferences to to accessor methods. This could have been incorporated as early as 2011, but lacked good follow through. Marcel Moolenaar was prime contact for this. 2) Andre Oppermann was sponsored in 2013 by the FreeBSD Foundation to make ifnet(9) opaque. This is not complete as of the time of this writing. 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but that always has been delayed due to 1) and 2). To outline what we would like to achieve: o FreeBSD wants to have an opaque ifnet(9). The benefits of which are: - quicker development of networking improvements w/o breaking ABI/API - splitting up ifnet into multiple structures or otherwise make ifnet of variable size. o Juniper wants to continue using stock FreeBSD drivers against its own network stack. The suggested roadmap is as follows: Since Juniper already has a working patch that achieves the objective of making ifnet an opaque type by replacing ifnet pointer dereferences with function calls in an almost mechanical fashion, we propose to merge that first. The patch doesn't break old-style access to struct ifnet, which means that unconverted and converted drivers coexist. This gives us time to convert drivers. This also gives Juniper an important rendezvous point between their and our repos. The following drivers have been converted by Juniper: o fxp o em and lem o bxe. Additional drivers will be converted by the community. Once all drivers are converted, no network driver needs or uses =93struct ifnet=94. At that point the definition can be removed from headers included by drivers or changed entirely. Schedule: late May 2014 Juniper adds accessor methods. Access via void *, that inside if.c is re-casted to (struct ifnet *). This =93ugly=94 moment is required to keep unconverted drivers compilable and will go away shortly after. 1st week June 2014 Juniper commits few converted drivers. June - July All drivers are being converted. Gleb volunteers, but this should be done by as many hands as possible. August 2014 '#define if_t void *' is removed. 'typedef if_t struct ifnet *' is added. actual struct ifnet is hidden. 2014/2015 Better APIs developed and improved, actual ifnet representation becomes variable sized. New networking stack improvements are already unblocked at this moment. The complete patch as well as the conversion script can be found here: http://people.freebsd.org/~marcel/Juniper/201405/drvapi.diff http://people.freebsd.org/~marcel/Juniper/201405/convert_drvapi.sh --=20 Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_97E6AFBE-6D5F-4B52-B449-4178F1B82A5C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlOGD/kACgkQpgWlLWHuifYsmACfYnkzaY5ZNlk2YQ6U3ah6gXvB oewAni4UFiGB7oEip8AbVXnnKSDrCNFa =glkP -----END PGP SIGNATURE----- --Apple-Mail=_97E6AFBE-6D5F-4B52-B449-4178F1B82A5C-- From owner-freebsd-arch@FreeBSD.ORG Thu May 29 03:48:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F25912B; Thu, 29 May 2014 03:48:15 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2001:470:1:2d5:26:3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id 44B172512; Thu, 29 May 2014 03:48:15 +0000 (UTC) Received: from [IPv6:2601:9:8280:426:20f6:fc7d:1197:e2f0] (unknown [IPv6:2601:9:8280:426:20f6:fc7d:1197:e2f0]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 01EBD34A9E4; Wed, 28 May 2014 20:48:13 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 From: Rui Paulo In-Reply-To: Date: Wed, 28 May 2014 20:48:11 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Marcel Moolenaar X-Mailer: Apple Mail (2.1878.2) Cc: Anuranjan Shukla , Gleb Smirnoff , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 03:48:15 -0000 On May 28, 2014, at 9:34, Marcel Moolenaar wrote: > All, >=20 > The ifnet structure represents a network interface. Right now it > is known to all NIC drivers as well as to all protocols. This > means that whenever we change the structure, we need at minimum > recompile all drivers, but usually also change them. This severely > slow downs the development in this area and also makes it hard, if > not, impossible to merge things back to stable branches. >=20 > There were at least 3 efforts on fixing this: >=20 > 1) Juniper=92s JUNOS is a FreeBSD based operating system that has > its own (alternative) network stack, but that leverages the > network drivers from FreeBSD. Juniper mechanically changed all > ifnet dereferences to to accessor methods. This could have > been incorporated as early as 2011, but lacked good follow > through. Marcel Moolenaar was prime contact for this. >=20 > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD > Foundation to make ifnet(9) opaque. This is not complete as of > the time of this writing. >=20 > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but > that always has been delayed due to 1) and 2). This is indeed needed, but it would be nice to understand what would = happen if the community has comments about your patch. Will Juniper be = able to integrate back those comments? For example, I think the type = "if_t" should be "ifnet_t". Another comment I have is: why do you have = to cast if_t to (struct ifnet *) in all the accessor methods? It would = be better to create a private header typedef'ing if_t to struct ifnet, = avoiding the copy & paste casting. =20 -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Thu May 29 04:04:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD3ED452; Thu, 29 May 2014 04:04:28 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5859F2676; Thu, 29 May 2014 04:04:27 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s4T44Ptg087664 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 29 May 2014 08:04:25 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s4T44PtP087663; Thu, 29 May 2014 08:04:25 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 29 May 2014 08:04:25 +0400 From: Gleb Smirnoff To: Rui Paulo Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Message-ID: <20140529040425.GT50679@glebius.int.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: Anuranjan Shukla , "freebsd-arch@FreeBSD.org Arch" , Marcel Moolenaar X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 04:04:28 -0000 On Wed, May 28, 2014 at 08:48:11PM -0700, Rui Paulo wrote: R> On May 28, 2014, at 9:34, Marcel Moolenaar wrote: R> R> > All, R> > R> > The ifnet structure represents a network interface. Right now it R> > is known to all NIC drivers as well as to all protocols. This R> > means that whenever we change the structure, we need at minimum R> > recompile all drivers, but usually also change them. This severely R> > slow downs the development in this area and also makes it hard, if R> > not, impossible to merge things back to stable branches. R> > R> > There were at least 3 efforts on fixing this: R> > R> > 1) Juniper’s JUNOS is a FreeBSD based operating system that has R> > its own (alternative) network stack, but that leverages the R> > network drivers from FreeBSD. Juniper mechanically changed all R> > ifnet dereferences to to accessor methods. This could have R> > been incorporated as early as 2011, but lacked good follow R> > through. Marcel Moolenaar was prime contact for this. R> > R> > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD R> > Foundation to make ifnet(9) opaque. This is not complete as of R> > the time of this writing. R> > R> > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but R> > that always has been delayed due to 1) and 2). R> R> This is indeed needed, but it would be nice to understand what would happen if the community has comments about your patch. Will Juniper be able to integrate back those comments? For example, I think the type "if_t" should be "ifnet_t". Another comment I have is: why do you have to cast if_t to (struct ifnet *) in all the accessor methods? It would be better to create a private header typedef'ing if_t to struct ifnet, avoiding the copy & paste casting. Because for now patch supports compiling unconverted drivers. This requires to pass 'void *' as argument. When all drivers are converted, this will be removed. In if_var.h we would have: struct ifnet; typedef struct ifnet * if_t; All casts to (struct ifnet *) will go away. Speaking of the 'if_t' vs "ifnet_t'. The pros of if_t are: - The functions in API all begin with 'if_', so we are inline with that. - Once a driver is converted 'grep ifnet if_driver.c' will return 0, but will return 1 on unconverted driver. That's nice. - We had a really nice comment in if_var.h that originates from 4.4BSD: /* * Structure defining a network interface. * * (Would like to call this struct ``if'', but C isn't PL/1.) */ Originally the idea was to have 'struct if', but that is impossible. So the 'net' prefix was added as a workaround, that we do not need to carry now. And yes, the 'net' is a tautology. All stuff resides in the sys/net directory, so no need to repeat 'net' again. In the kernel the word 'interface' applies only to network interfaces, no need to repeat 'net' extra time. - It is shorter. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Thu May 29 05:38:27 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90719285; Thu, 29 May 2014 05:38:27 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id 16D512E3D; Thu, 29 May 2014 05:38:26 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id s4T5b16Z033344; Thu, 29 May 2014 00:37:01 -0500 (CDT) (envelope-from mike@karels.net) Message-Id: <201405290537.s4T5b16Z033344@mail.karels.net> To: Gleb Smirnoff From: Mike Karels Reply-to: mike@karels.net Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 In-reply-to: Your message of Thu, 29 May 2014 08:04:25 +0400. <20140529040425.GT50679@glebius.int.ru> Date: Thu, 29 May 2014 00:37:01 -0500 Cc: Marcel Moolenaar , Anuranjan Shukla , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 05:38:27 -0000 Marcel and others, is there more to the roadmap than making the ifnet easier to change? Could you outline a bit more of the roadmap? I know that Juniper has more levels in the hierarchy of interface data structures. What are you proposing that we change after this step? I'll also repeat the general part of Rui's question: R> This is indeed needed, but it would be nice to understand what would happen if the community has comments about your patch. Will Juniper be able to integrate back those comments? Mike From owner-freebsd-arch@FreeBSD.ORG Thu May 29 05:45:41 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34E3F3BE; Thu, 29 May 2014 05:45:41 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D68C92EDF; Thu, 29 May 2014 05:45:40 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id i50so20797126qgf.7 for ; Wed, 28 May 2014 22:45:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=Mo7arnjoVsz4fJ3URHK+mq9NBtJeoibAZBwL4rckS98=; b=oH8vDCuq3OP7Ftc0CqPwoRDHJny2H647ONYNbvT0AWm2J+yzoAAFbLyc2jaznk4SwM pWf3M/pZ62JUAYdnFJJXcBbxIAlGx1iq/5wYXye5G7SVOwfm9nUmvOx7EjTmqqMm8FI7 58Bv8JErTxPo9U85v8I6bAx8tUe8+PXKo1N+pxDpttHx7QyNNQBEhSTUB3d23mX1zrS7 dRaj9Gd3dGFvWMSYL0rMA8Zm/s8a1MCkjTRPo6JwoOJzH64rhq7LHsyok4t49wqw4XgW ZqhwNbphYSz+1AngQt0A5bk0NZricdv7bGaJBJVOTL/q29ZmLwghe3oQX8AmOTl+or1N NjUA== MIME-Version: 1.0 X-Received: by 10.224.16.199 with SMTP id p7mr6600737qaa.76.1401342339706; Wed, 28 May 2014 22:45:39 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Wed, 28 May 2014 22:45:39 -0700 (PDT) In-Reply-To: References: Date: Wed, 28 May 2014 22:45:39 -0700 X-Google-Sender-Auth: eJdjCt64-H8Ok814Veh0UEHC9WQ Message-ID: Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 From: Adrian Chadd To: Marcel Moolenaar Content-Type: text/plain; charset=UTF-8 Cc: Anuranjan Shukla , Gleb Smirnoff , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 05:45:41 -0000 Hi! On 28 May 2014 09:34, Marcel Moolenaar wrote: > All, > Since Juniper already has a working patch that achieves the > objective of making ifnet an opaque type by replacing ifnet > pointer dereferences with function calls in an almost mechanical > fashion, we propose to merge that first. > The patch doesn't break old-style access to struct ifnet, which > means that unconverted and converted drivers coexist. This gives > us time to convert drivers. This also gives Juniper an important > rendezvous point between their and our repos. Hi! Converting all the ifnet dereferencing and converting it to accessor functions/macros seems like a no-brainer pass. You can do this without necessarily converting them to void *. That can be a separate pass. i wonder about the immediate benefits of making it fully opaque versus "mostly opaque-ready for you." Would we get less formalish checking coverage with static analysis tools and the compiler? -a From owner-freebsd-arch@FreeBSD.ORG Thu May 29 06:35:01 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 69737DC8; Thu, 29 May 2014 06:35:01 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (dns-bn1lp0143.outbound.protection.outlook.com [207.46.163.143]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60EDB22B1; Thu, 29 May 2014 06:34:59 +0000 (UTC) Received: from CO2PR05MB730.namprd05.prod.outlook.com (10.141.228.15) by CO2PR05MB729.namprd05.prod.outlook.com (10.141.228.12) with Microsoft SMTP Server (TLS) id 15.0.949.11; Thu, 29 May 2014 06:34:56 +0000 Received: from CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) by CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) with mapi id 15.00.0949.001; Thu, 29 May 2014 06:34:56 +0000 From: Anuranjan Shukla To: "mike@karels.net" , Gleb Smirnoff Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Thread-Topic: Roadmap for ifnet(9) for FreeBSD 11 Thread-Index: AQHPewADX4+rVOOldkOoqmC4MWB2pZtWpNgA Date: Thu, 29 May 2014 06:34:55 +0000 Message-ID: References: <20140529040425.GT50679@glebius.int.ru> <201405290537.s4T5b16Z033344@mail.karels.net> In-Reply-To: <201405290537.s4T5b16Z033344@mail.karels.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [66.129.239.10] x-forefront-prvs: 022649CC2C x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(428001)(51704005)(479174003)(35774003)(199002)(189002)(24454002)(26614003)(377454003)(83322001)(50986999)(85852003)(83506001)(19580395003)(83072002)(99396002)(4396001)(19580405001)(92566001)(46102001)(54356999)(77982001)(101416001)(77096999)(99286001)(2656002)(64706001)(20776003)(86362001)(79102001)(74502001)(74662001)(21056001)(76482001)(81342001)(81542001)(66066001)(80022001)(76176999)(92726001)(31966008)(87936001)(36756003); DIR:OUT; SFP:; SCL:1; SRVR:CO2PR05MB729; H:CO2PR05MB730.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: juniper.net does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anshukla@juniper.net; Content-Type: text/plain; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net Cc: Marcel Moolenaar , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 06:35:01 -0000 On Rui=B9s question, if I understand it right, yes we=B9ll work on the patc= h based on comments and feedback. As is obvious, Juniper=B9s network stack ha= s its own set of the drvapi functions that do very different things in some cases. One intent in the submission is to agree upon the API itself as a first step, because further along we are in our production/shipping cycles with this change the harder it=B9ll be to pull off fundamental changes ther= e. On 5/28/14, 10:37 PM, "Mike Karels" wrote: >Marcel and others, is there more to the roadmap than making the ifnet >easier >to change? Could you outline a bit more of the roadmap? I know that >Juniper >has more levels in the hierarchy of interface data structures. What are >you >proposing that we change after this step? > >I'll also repeat the general part of Rui's question: > >R> This is indeed needed, but it would be nice to understand what would >happen if the community has comments about your patch. Will Juniper be >able to integrate back those comments? > > Mike From owner-freebsd-arch@FreeBSD.ORG Thu May 29 10:21:01 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D3864FE for ; Thu, 29 May 2014 10:21:01 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CBC927DF for ; Thu, 29 May 2014 10:20:56 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.8/8.14.8) with ESMTP id s4TAKsq7089736 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 29 May 2014 14:20:54 +0400 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.8/8.14.8/Submit) id s4TAKsFd089735 for arch@FreeBSD.org; Thu, 29 May 2014 14:20:54 +0400 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 29 May 2014 14:20:54 +0400 From: Gleb Smirnoff To: arch@FreeBSD.org Subject: [CFT/review] new sendfile(2) Message-ID: <20140529102054.GX50679@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="IuJpT0rwbUevm2bB" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 10:21:01 -0000 --IuJpT0rwbUevm2bB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello! At Netflix and Nginx we are experimenting with improving FreeBSD wrt sending large amounts of static data via HTTP. One of the approaches we are experimenting with is new sendfile(2) implementation, that doesn't block on the I/O done from the file descriptor. The problem with classic sendfile(2) is that if the the request length is large enough, and file data is not cached in VM, then sendfile(2) syscall would not return until it fills socket buffer with data. With modern internet socket buffers can be up to 1 Mb, thus time taken by the syscall raises by order of magnitude. All the time, the nginx worker is blocked in syscall and doesn't process data from other clients. The best current practice to mitigate that is known as "sendfile(2) + aio_read(2)". This is special mode of nginx operation on FreeBSD. The sendfile(2) call is issued with SF_NODISKIO flag, that forbids the syscall to perform disk I/O, and send only data that is cached by VM. If sendfile(2) reports that I/O needs to be done (but forbidden), then nginx would do aio_read() of a chunk of the file. The data read is cached by VM, as side affect. Then sendfile() is called again. Now for the new sendfile. The core idea is that sendfile() schedules the I/O, but doesn't wait for it to complete. It returns immediately to the process, and I/O completion is processed in kernel context. Unlike aio(4), no additional threads in kernel are created. The new sendfile is a drop-in replacement for the old one. Applications (like nginx) doesn't need recompile, neither configuration change. The SF_NODISKIO is ignored. The patch for review is available at: https://phabric.freebsd.org/D102 And for those who prefer email attachments, it is also attached. The patch has 3 logically separate changes in itself: 1) Split of socket buffer sb_cc field into sb_acc and sb_ccc. Where sb_acc stands for "available character count" and sb_ccc is "claimed character count". This allows us to write a data to a socket, that is not ready yet. The data sits in the socket, consumes its space, and keeps itself in the right order with earlier or later writes to socket. But it can be send only after it is marked as ready. This change is split across many files. 2) A new vnode operation: VOP_GETPAGES_ASYNC(). This one lives in sys/vm. 3) Actual implementation of new sendfile(2). This one lives in kern/uipc_syscalls.c At Netflix, we already see improvements with new sendfile(2). We can send more data utilizing same amount of CPU, and we can push closer to 0% idle, without experiencing short lags. However, we have somewhat modified VM subsystem, that behaves optimal for our task, but suboptimal for average FreeBSD system. I'd like someone from community to try the new sendfile(2) at other setup and see how does it serve for you. To be the early tester you need to checkout projects/sendfile branch and build kernel from it. The world from head/ would run fine with it. svn co http://svn.freebsd.org/base/projects/sendfile cd sendfile ... build kernel ... Limitations: - No testing were done on serving files on NFS. - No testing were done on serving files on ZFS. -- Totus tuus, Glebius. --IuJpT0rwbUevm2bB Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="project-sendfile.diff" Index: sys/dev/ti/if_ti.c =================================================================== --- sys/dev/ti/if_ti.c (.../head) (revision 266804) +++ sys/dev/ti/if_ti.c (.../projects/sendfile) (revision 266807) @@ -1629,7 +1629,7 @@ ti_newbuf_jumbo(struct ti_softc *sc, int idx, stru m[i]->m_data = (void *)sf_buf_kva(sf[i]); m[i]->m_len = PAGE_SIZE; MEXTADD(m[i], sf_buf_kva(sf[i]), PAGE_SIZE, - sf_buf_mext, (void*)sf_buf_kva(sf[i]), sf[i], + sf_mext_free, (void*)sf_buf_kva(sf[i]), sf[i], 0, EXT_DISPOSABLE); m[i]->m_next = m[i+1]; } @@ -1694,7 +1694,7 @@ nobufs: if (m[i]) m_freem(m[i]); if (sf[i]) - sf_buf_mext((void *)sf_buf_kva(sf[i]), sf[i]); + sf_mext_free((void *)sf_buf_kva(sf[i]), sf[i]); } return (ENOBUFS); } Index: sys/dev/cxgbe/tom/t4_cpl_io.c =================================================================== --- sys/dev/cxgbe/tom/t4_cpl_io.c (.../head) (revision 266804) +++ sys/dev/cxgbe/tom/t4_cpl_io.c (.../projects/sendfile) (revision 266807) @@ -338,11 +338,11 @@ t4_rcvd(struct toedev *tod, struct tcpcb *tp) INP_WLOCK_ASSERT(inp); SOCKBUF_LOCK(sb); - KASSERT(toep->sb_cc >= sb->sb_cc, + KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", - __func__, sb, sb->sb_cc, toep->sb_cc)); - toep->rx_credits += toep->sb_cc - sb->sb_cc; - toep->sb_cc = sb->sb_cc; + __func__, sb, sbused(sb), toep->sb_cc)); + toep->rx_credits += toep->sb_cc - sbused(sb); + toep->sb_cc = sbused(sb); credits = toep->rx_credits; SOCKBUF_UNLOCK(sb); @@ -863,15 +863,15 @@ do_peer_close(struct sge_iq *iq, const struct rss_ tp->rcv_nxt = be32toh(cpl->rcv_nxt); toep->ddp_flags &= ~(DDP_BUF0_ACTIVE | DDP_BUF1_ACTIVE); - KASSERT(toep->sb_cc >= sb->sb_cc, + KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", - __func__, sb, sb->sb_cc, toep->sb_cc)); - toep->rx_credits += toep->sb_cc - sb->sb_cc; + __func__, sb, sbused(sb), toep->sb_cc)); + toep->rx_credits += toep->sb_cc - sbused(sb); #ifdef USE_DDP_RX_FLOW_CONTROL toep->rx_credits -= m->m_len; /* adjust for F_RX_FC_DDP */ #endif - sbappendstream_locked(sb, m); - toep->sb_cc = sb->sb_cc; + sbappendstream_locked(sb, m, 0); + toep->sb_cc = sbused(sb); } socantrcvmore_locked(so); /* unlocks the sockbuf */ @@ -1281,12 +1281,12 @@ do_rx_data(struct sge_iq *iq, const struct rss_hea } } - KASSERT(toep->sb_cc >= sb->sb_cc, + KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", - __func__, sb, sb->sb_cc, toep->sb_cc)); - toep->rx_credits += toep->sb_cc - sb->sb_cc; - sbappendstream_locked(sb, m); - toep->sb_cc = sb->sb_cc; + __func__, sb, sbused(sb), toep->sb_cc)); + toep->rx_credits += toep->sb_cc - sbused(sb); + sbappendstream_locked(sb, m, 0); + toep->sb_cc = sbused(sb); sorwakeup_locked(so); SOCKBUF_UNLOCK_ASSERT(sb); Index: sys/dev/cxgbe/tom/t4_ddp.c =================================================================== --- sys/dev/cxgbe/tom/t4_ddp.c (.../head) (revision 266804) +++ sys/dev/cxgbe/tom/t4_ddp.c (.../projects/sendfile) (revision 266807) @@ -224,15 +224,15 @@ insert_ddp_data(struct toepcb *toep, uint32_t n) tp->rcv_wnd -= n; #endif - KASSERT(toep->sb_cc >= sb->sb_cc, + KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", - __func__, sb, sb->sb_cc, toep->sb_cc)); - toep->rx_credits += toep->sb_cc - sb->sb_cc; + __func__, sb, sbused(sb), toep->sb_cc)); + toep->rx_credits += toep->sb_cc - sbused(sb); #ifdef USE_DDP_RX_FLOW_CONTROL toep->rx_credits -= n; /* adjust for F_RX_FC_DDP */ #endif - sbappendstream_locked(sb, m); - toep->sb_cc = sb->sb_cc; + sbappendstream_locked(sb, m, 0); + toep->sb_cc = sbused(sb); } /* SET_TCB_FIELD sent as a ULP command looks like this */ @@ -459,15 +459,15 @@ handle_ddp_data(struct toepcb *toep, __be32 ddp_re else discourage_ddp(toep); - KASSERT(toep->sb_cc >= sb->sb_cc, + KASSERT(toep->sb_cc >= sbused(sb), ("%s: sb %p has more data (%d) than last time (%d).", - __func__, sb, sb->sb_cc, toep->sb_cc)); - toep->rx_credits += toep->sb_cc - sb->sb_cc; + __func__, sb, sbused(sb), toep->sb_cc)); + toep->rx_credits += toep->sb_cc - sbused(sb); #ifdef USE_DDP_RX_FLOW_CONTROL toep->rx_credits -= len; /* adjust for F_RX_FC_DDP */ #endif - sbappendstream_locked(sb, m); - toep->sb_cc = sb->sb_cc; + sbappendstream_locked(sb, m, 0); + toep->sb_cc = sbused(sb); wakeup: KASSERT(toep->ddp_flags & db_flag, ("%s: DDP buffer not active. toep %p, ddp_flags 0x%x, report 0x%x", @@ -897,7 +897,7 @@ handle_ddp(struct socket *so, struct uio *uio, int #endif /* XXX: too eager to disable DDP, could handle NBIO better than this. */ - if (sb->sb_cc >= uio->uio_resid || uio->uio_resid < sc->tt.ddp_thres || + if (sbused(sb) >= uio->uio_resid || uio->uio_resid < sc->tt.ddp_thres || uio->uio_resid > MAX_DDP_BUFFER_SIZE || uio->uio_iovcnt > 1 || so->so_state & SS_NBIO || flags & (MSG_DONTWAIT | MSG_NBIO) || error || so->so_error || sb->sb_state & SBS_CANTRCVMORE) @@ -935,7 +935,7 @@ handle_ddp(struct socket *so, struct uio *uio, int * payload. */ ddp_flags = select_ddp_flags(so, flags, db_idx); - wr = mk_update_tcb_for_ddp(sc, toep, db_idx, sb->sb_cc, ddp_flags); + wr = mk_update_tcb_for_ddp(sc, toep, db_idx, sbused(sb), ddp_flags); if (wr == NULL) { /* * Just unhold the pages. The DDP buffer's software state is @@ -960,8 +960,9 @@ handle_ddp(struct socket *so, struct uio *uio, int */ rc = sbwait(sb); while (toep->ddp_flags & buf_flag) { + /* XXXGL: shouldn't here be sbwait() call? */ sb->sb_flags |= SB_WAIT; - msleep(&sb->sb_cc, &sb->sb_mtx, PSOCK , "sbwait", 0); + msleep(&sb->sb_acc, &sb->sb_mtx, PSOCK , "sbwait", 0); } unwire_ddp_buffer(db); return (rc); @@ -1123,8 +1124,8 @@ restart: /* uio should be just as it was at entry */ KASSERT(oresid == uio->uio_resid, - ("%s: oresid = %d, uio_resid = %zd, sb_cc = %d", - __func__, oresid, uio->uio_resid, sb->sb_cc)); + ("%s: oresid = %d, uio_resid = %zd, sbused = %d", + __func__, oresid, uio->uio_resid, sbused(sb))); error = handle_ddp(so, uio, flags, 0); ddp_handled = 1; @@ -1134,7 +1135,7 @@ restart: /* Abort if socket has reported problems. */ if (so->so_error) { - if (sb->sb_cc > 0) + if (sbused(sb)) goto deliver; if (oresid > uio->uio_resid) goto out; @@ -1146,7 +1147,7 @@ restart: /* Door is closed. Deliver what is left, if any. */ if (sb->sb_state & SBS_CANTRCVMORE) { - if (sb->sb_cc > 0) + if (sbused(sb)) goto deliver; else goto out; @@ -1153,7 +1154,7 @@ restart: } /* Socket buffer is empty and we shall not block. */ - if (sb->sb_cc == 0 && + if (sbused(sb) == 0 && ((so->so_state & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)))) { error = EAGAIN; goto out; @@ -1160,18 +1161,18 @@ restart: } /* Socket buffer got some data that we shall deliver now. */ - if (sb->sb_cc > 0 && !(flags & MSG_WAITALL) && + if (sbused(sb) && !(flags & MSG_WAITALL) && ((sb->sb_flags & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)) || - sb->sb_cc >= sb->sb_lowat || - sb->sb_cc >= uio->uio_resid || - sb->sb_cc >= sb->sb_hiwat) ) { + sbused(sb) >= sb->sb_lowat || + sbused(sb) >= uio->uio_resid || + sbused(sb) >= sb->sb_hiwat) ) { goto deliver; } /* On MSG_WAITALL we must wait until all data or error arrives. */ if ((flags & MSG_WAITALL) && - (sb->sb_cc >= uio->uio_resid || sb->sb_cc >= sb->sb_lowat)) + (sbused(sb) >= uio->uio_resid || sbused(sb) >= sb->sb_lowat)) goto deliver; /* @@ -1190,7 +1191,7 @@ restart: deliver: SOCKBUF_LOCK_ASSERT(&so->so_rcv); - KASSERT(sb->sb_cc > 0, ("%s: sockbuf empty", __func__)); + KASSERT(sbused(sb) > 0, ("%s: sockbuf empty", __func__)); KASSERT(sb->sb_mb != NULL, ("%s: sb_mb == NULL", __func__)); if (sb->sb_flags & SB_DDP_INDICATE && !ddp_handled) @@ -1201,7 +1202,7 @@ deliver: uio->uio_td->td_ru.ru_msgrcv++; /* Fill uio until full or current end of socket buffer is reached. */ - len = min(uio->uio_resid, sb->sb_cc); + len = min(uio->uio_resid, sbused(sb)); if (mp0 != NULL) { /* Dequeue as many mbufs as possible. */ if (!(flags & MSG_PEEK) && len >= sb->sb_mb->m_len) { Index: sys/dev/cxgbe/iw_cxgbe/cm.c =================================================================== --- sys/dev/cxgbe/iw_cxgbe/cm.c (.../head) (revision 266804) +++ sys/dev/cxgbe/iw_cxgbe/cm.c (.../projects/sendfile) (revision 266807) @@ -585,8 +585,8 @@ process_data(struct c4iw_ep *ep) { struct sockaddr_in *local, *remote; - CTR5(KTR_IW_CXGBE, "%s: so %p, ep %p, state %s, sb_cc %d", __func__, - ep->com.so, ep, states[ep->com.state], ep->com.so->so_rcv.sb_cc); + CTR5(KTR_IW_CXGBE, "%s: so %p, ep %p, state %s, sbused %d", __func__, + ep->com.so, ep, states[ep->com.state], sbused(&ep->com.so->so_rcv)); switch (state_read(&ep->com)) { case MPA_REQ_SENT: @@ -602,11 +602,11 @@ process_data(struct c4iw_ep *ep) process_mpa_request(ep); break; default: - if (ep->com.so->so_rcv.sb_cc) - log(LOG_ERR, "%s: Unexpected streaming data. " - "ep %p, state %d, so %p, so_state 0x%x, sb_cc %u\n", + if (sbused(&ep->com.so->so_rcv)) + log(LOG_ERR, "%s: Unexpected streaming data. ep %p, " + "state %d, so %p, so_state 0x%x, sbused %u\n", __func__, ep, state_read(&ep->com), ep->com.so, - ep->com.so->so_state, ep->com.so->so_rcv.sb_cc); + ep->com.so->so_state, sbused(&ep->com.so->so_rcv)); break; } } Index: sys/dev/iscsi/icl.c =================================================================== --- sys/dev/iscsi/icl.c (.../head) (revision 266804) +++ sys/dev/iscsi/icl.c (.../projects/sendfile) (revision 266807) @@ -758,7 +758,7 @@ icl_receive_thread(void *arg) * is enough data received to read the PDU. */ SOCKBUF_LOCK(&so->so_rcv); - available = so->so_rcv.sb_cc; + available = sbavail(&so->so_rcv); if (available < ic->ic_receive_len) { so->so_rcv.sb_lowat = ic->ic_receive_len; cv_wait(&ic->ic_receive_cv, &so->so_rcv.sb_mtx); Index: sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c =================================================================== --- sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c (.../head) (revision 266804) +++ sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c (.../projects/sendfile) (revision 266807) @@ -445,8 +445,8 @@ t3_push_frames(struct socket *so, int req_completi * Autosize the send buffer. */ if (snd->sb_flags & SB_AUTOSIZE && VNET(tcp_do_autosndbuf)) { - if (snd->sb_cc >= (snd->sb_hiwat / 8 * 7) && - snd->sb_cc < VNET(tcp_autosndbuf_max)) { + if (sbused(snd) >= (snd->sb_hiwat / 8 * 7) && + sbused(snd) < VNET(tcp_autosndbuf_max)) { if (!sbreserve_locked(snd, min(snd->sb_hiwat + VNET(tcp_autosndbuf_inc), VNET(tcp_autosndbuf_max)), so, curthread)) @@ -597,10 +597,10 @@ t3_rcvd(struct toedev *tod, struct tcpcb *tp) INP_WLOCK_ASSERT(inp); SOCKBUF_LOCK(so_rcv); - KASSERT(toep->tp_enqueued >= so_rcv->sb_cc, - ("%s: so_rcv->sb_cc > enqueued", __func__)); - toep->tp_rx_credits += toep->tp_enqueued - so_rcv->sb_cc; - toep->tp_enqueued = so_rcv->sb_cc; + KASSERT(toep->tp_enqueued >= sbused(so_rcv), + ("%s: sbused(so_rcv) > enqueued", __func__)); + toep->tp_rx_credits += toep->tp_enqueued - sbused(so_rcv); + toep->tp_enqueued = sbused(so_rcv); SOCKBUF_UNLOCK(so_rcv); must_send = toep->tp_rx_credits + 16384 >= tp->rcv_wnd; @@ -1199,7 +1199,7 @@ do_rx_data(struct sge_qset *qs, struct rsp_desc *r } toep->tp_enqueued += m->m_pkthdr.len; - sbappendstream_locked(so_rcv, m); + sbappendstream_locked(so_rcv, m, 0); sorwakeup_locked(so); SOCKBUF_UNLOCK_ASSERT(so_rcv); @@ -1768,7 +1768,7 @@ wr_ack(struct toepcb *toep, struct mbuf *m) so_sowwakeup_locked(so); } - if (snd->sb_sndptroff < snd->sb_cc) + if (snd->sb_sndptroff < sbused(snd)) t3_push_frames(so, 0); out_free: Index: sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_cm.c =================================================================== --- sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_cm.c (.../head) (revision 266804) +++ sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_cm.c (.../projects/sendfile) (revision 266807) @@ -1515,11 +1515,11 @@ process_data(struct iwch_ep *ep) process_mpa_request(ep); break; default: - if (ep->com.so->so_rcv.sb_cc) + if (sbavail(&ep->com.so->so_rcv)) printf("%s Unexpected streaming data." " ep %p state %d so %p so_state %x so_rcv.sb_cc %u so_rcv.sb_mb %p\n", __FUNCTION__, ep, state_read(&ep->com), ep->com.so, ep->com.so->so_state, - ep->com.so->so_rcv.sb_cc, ep->com.so->so_rcv.sb_mb); + sbavail(&ep->com.so->so_rcv), ep->com.so->so_rcv.sb_mb); break; } return; Index: sys/kern/uipc_debug.c =================================================================== --- sys/kern/uipc_debug.c (.../head) (revision 266804) +++ sys/kern/uipc_debug.c (.../projects/sendfile) (revision 266807) @@ -403,7 +403,8 @@ db_print_sockbuf(struct sockbuf *sb, const char *s db_printf("sb_sndptroff: %u\n", sb->sb_sndptroff); db_print_indent(indent); - db_printf("sb_cc: %u ", sb->sb_cc); + db_printf("sb_acc: %u ", sb->sb_acc); + db_printf("sb_ccc: %u ", sb->sb_ccc); db_printf("sb_hiwat: %u ", sb->sb_hiwat); db_printf("sb_mbcnt: %u ", sb->sb_mbcnt); db_printf("sb_mbmax: %u\n", sb->sb_mbmax); Index: sys/kern/uipc_mbuf.c =================================================================== --- sys/kern/uipc_mbuf.c (.../head) (revision 266804) +++ sys/kern/uipc_mbuf.c (.../projects/sendfile) (revision 266807) @@ -389,7 +389,7 @@ mb_dupcl(struct mbuf *n, struct mbuf *m) * cleaned too. */ void -m_demote(struct mbuf *m0, int all) +m_demote(struct mbuf *m0, int all, int flags) { struct mbuf *m; @@ -405,7 +405,7 @@ void m_freem(m->m_nextpkt); m->m_nextpkt = NULL; } - m->m_flags = m->m_flags & (M_EXT|M_RDONLY|M_NOFREE); + m->m_flags = m->m_flags & (M_EXT | M_RDONLY | M_NOFREE | flags); } } Index: sys/kern/sys_socket.c =================================================================== --- sys/kern/sys_socket.c (.../head) (revision 266804) +++ sys/kern/sys_socket.c (.../projects/sendfile) (revision 266807) @@ -167,20 +167,17 @@ soo_ioctl(struct file *fp, u_long cmd, void *data, case FIONREAD: /* Unlocked read. */ - *(int *)data = so->so_rcv.sb_cc; + *(int *)data = sbavail(&so->so_rcv); break; case FIONWRITE: /* Unlocked read. */ - *(int *)data = so->so_snd.sb_cc; + *(int *)data = sbavail(&so->so_snd); break; case FIONSPACE: - if ((so->so_snd.sb_hiwat < so->so_snd.sb_cc) || - (so->so_snd.sb_mbmax < so->so_snd.sb_mbcnt)) - *(int *)data = 0; - else - *(int *)data = sbspace(&so->so_snd); + /* Unlocked read. */ + *(int *)data = sbspace(&so->so_snd); break; case FIOSETOWN: @@ -246,6 +243,7 @@ soo_stat(struct file *fp, struct stat *ub, struct struct thread *td) { struct socket *so = fp->f_data; + struct sockbuf *sb; #ifdef MAC int error; #endif @@ -261,15 +259,18 @@ soo_stat(struct file *fp, struct stat *ub, struct * If SBS_CANTRCVMORE is set, but there's still data left in the * receive buffer, the socket is still readable. */ - SOCKBUF_LOCK(&so->so_rcv); - if ((so->so_rcv.sb_state & SBS_CANTRCVMORE) == 0 || - so->so_rcv.sb_cc != 0) + sb = &so->so_rcv; + SOCKBUF_LOCK(sb); + if ((sb->sb_state & SBS_CANTRCVMORE) == 0 || sbavail(sb)) ub->st_mode |= S_IRUSR | S_IRGRP | S_IROTH; - ub->st_size = so->so_rcv.sb_cc - so->so_rcv.sb_ctl; - SOCKBUF_UNLOCK(&so->so_rcv); - /* Unlocked read. */ - if ((so->so_snd.sb_state & SBS_CANTSENDMORE) == 0) + ub->st_size = sbavail(sb) - sb->sb_ctl; + SOCKBUF_UNLOCK(sb); + + sb = &so->so_snd; + SOCKBUF_LOCK(sb); + if ((sb->sb_state & SBS_CANTSENDMORE) == 0) ub->st_mode |= S_IWUSR | S_IWGRP | S_IWOTH; + SOCKBUF_UNLOCK(sb); ub->st_uid = so->so_cred->cr_uid; ub->st_gid = so->so_cred->cr_gid; return (*so->so_proto->pr_usrreqs->pru_sense)(so, ub); Index: sys/kern/uipc_usrreq.c =================================================================== --- sys/kern/uipc_usrreq.c (.../head) (revision 266804) +++ sys/kern/uipc_usrreq.c (.../projects/sendfile) (revision 266807) @@ -790,11 +790,10 @@ uipc_rcvd(struct socket *so, int flags) u_int mbcnt, sbcc; unp = sotounpcb(so); - KASSERT(unp != NULL, ("uipc_rcvd: unp == NULL")); + KASSERT(unp != NULL, ("%s: unp == NULL", __func__)); + KASSERT(so->so_type == SOCK_STREAM || so->so_type == SOCK_SEQPACKET, + ("%s: socktype %d", __func__, so->so_type)); - if (so->so_type != SOCK_STREAM && so->so_type != SOCK_SEQPACKET) - panic("uipc_rcvd socktype %d", so->so_type); - /* * Adjust backpressure on sender and wakeup any waiting to write. * @@ -807,7 +806,7 @@ uipc_rcvd(struct socket *so, int flags) */ SOCKBUF_LOCK(&so->so_rcv); mbcnt = so->so_rcv.sb_mbcnt; - sbcc = so->so_rcv.sb_cc; + sbcc = sbavail(&so->so_rcv); SOCKBUF_UNLOCK(&so->so_rcv); /* * There is a benign race condition at this point. If we're planning to @@ -843,7 +842,10 @@ uipc_send(struct socket *so, int flags, struct mbu int error = 0; unp = sotounpcb(so); - KASSERT(unp != NULL, ("uipc_send: unp == NULL")); + KASSERT(unp != NULL, ("%s: unp == NULL", __func__)); + KASSERT(so->so_type == SOCK_STREAM || so->so_type == SOCK_DGRAM || + so->so_type == SOCK_SEQPACKET, + ("%s: socktype %d", __func__, so->so_type)); if (flags & PRUS_OOB) { error = EOPNOTSUPP; @@ -994,7 +996,7 @@ uipc_send(struct socket *so, int flags, struct mbu } mbcnt = so2->so_rcv.sb_mbcnt; - sbcc = so2->so_rcv.sb_cc; + sbcc = sbavail(&so2->so_rcv); sorwakeup_locked(so2); /* @@ -1011,9 +1013,6 @@ uipc_send(struct socket *so, int flags, struct mbu UNP_PCB_UNLOCK(unp2); m = NULL; break; - - default: - panic("uipc_send unknown socktype"); } /* Index: sys/kern/vfs_default.c =================================================================== --- sys/kern/vfs_default.c (.../head) (revision 266804) +++ sys/kern/vfs_default.c (.../projects/sendfile) (revision 266807) @@ -111,6 +111,7 @@ struct vop_vector default_vnodeops = { .vop_close = VOP_NULL, .vop_fsync = VOP_NULL, .vop_getpages = vop_stdgetpages, + .vop_getpages_async = vop_stdgetpages_async, .vop_getwritemount = vop_stdgetwritemount, .vop_inactive = VOP_NULL, .vop_ioctl = VOP_ENOTTY, @@ -726,10 +727,19 @@ vop_stdgetpages(ap) { return vnode_pager_generic_getpages(ap->a_vp, ap->a_m, - ap->a_count, ap->a_reqpage); + ap->a_count, ap->a_reqpage, NULL, NULL); } +/* XXX Needs good comment and a manpage. */ int +vop_stdgetpages_async(struct vop_getpages_async_args *ap) +{ + + return vnode_pager_generic_getpages(ap->a_vp, ap->a_m, + ap->a_count, ap->a_reqpage, ap->a_vop_getpages_iodone, ap->a_arg); +} + +int vop_stdkqfilter(struct vop_kqfilter_args *ap) { return vfs_kqfilter(ap); Index: sys/kern/uipc_socket.c =================================================================== --- sys/kern/uipc_socket.c (.../head) (revision 266804) +++ sys/kern/uipc_socket.c (.../projects/sendfile) (revision 266807) @@ -1459,12 +1459,12 @@ restart: * 2. MSG_DONTWAIT is not set */ if (m == NULL || (((flags & MSG_DONTWAIT) == 0 && - so->so_rcv.sb_cc < uio->uio_resid) && - so->so_rcv.sb_cc < so->so_rcv.sb_lowat && + sbavail(&so->so_rcv) < uio->uio_resid) && + sbavail(&so->so_rcv) < so->so_rcv.sb_lowat && m->m_nextpkt == NULL && (pr->pr_flags & PR_ATOMIC) == 0)) { - KASSERT(m != NULL || !so->so_rcv.sb_cc, - ("receive: m == %p so->so_rcv.sb_cc == %u", - m, so->so_rcv.sb_cc)); + KASSERT(m != NULL || !sbavail(&so->so_rcv), + ("receive: m == %p sbavail == %u", + m, sbavail(&so->so_rcv))); if (so->so_error) { if (m != NULL) goto dontblock; @@ -1746,9 +1746,7 @@ dontblock: SOCKBUF_LOCK(&so->so_rcv); } } - m->m_data += len; - m->m_len -= len; - so->so_rcv.sb_cc -= len; + sbmtrim(&so->so_rcv, m, len); } } SOCKBUF_LOCK_ASSERT(&so->so_rcv); @@ -1913,7 +1911,7 @@ restart: /* Abort if socket has reported problems. */ if (so->so_error) { - if (sb->sb_cc > 0) + if (sbavail(sb) > 0) goto deliver; if (oresid > uio->uio_resid) goto out; @@ -1925,7 +1923,7 @@ restart: /* Door is closed. Deliver what is left, if any. */ if (sb->sb_state & SBS_CANTRCVMORE) { - if (sb->sb_cc > 0) + if (sbavail(sb) > 0) goto deliver; else goto out; @@ -1932,7 +1930,7 @@ restart: } /* Socket buffer is empty and we shall not block. */ - if (sb->sb_cc == 0 && + if (sbavail(sb) == 0 && ((so->so_state & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)))) { error = EAGAIN; goto out; @@ -1939,18 +1937,18 @@ restart: } /* Socket buffer got some data that we shall deliver now. */ - if (sb->sb_cc > 0 && !(flags & MSG_WAITALL) && + if (sbavail(sb) > 0 && !(flags & MSG_WAITALL) && ((sb->sb_flags & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)) || - sb->sb_cc >= sb->sb_lowat || - sb->sb_cc >= uio->uio_resid || - sb->sb_cc >= sb->sb_hiwat) ) { + sbavail(sb) >= sb->sb_lowat || + sbavail(sb) >= uio->uio_resid || + sbavail(sb) >= sb->sb_hiwat) ) { goto deliver; } /* On MSG_WAITALL we must wait until all data or error arrives. */ if ((flags & MSG_WAITALL) && - (sb->sb_cc >= uio->uio_resid || sb->sb_cc >= sb->sb_hiwat)) + (sbavail(sb) >= uio->uio_resid || sbavail(sb) >= sb->sb_hiwat)) goto deliver; /* @@ -1964,7 +1962,7 @@ restart: deliver: SOCKBUF_LOCK_ASSERT(&so->so_rcv); - KASSERT(sb->sb_cc > 0, ("%s: sockbuf empty", __func__)); + KASSERT(sbavail(sb) > 0, ("%s: sockbuf empty", __func__)); KASSERT(sb->sb_mb != NULL, ("%s: sb_mb == NULL", __func__)); /* Statistics. */ @@ -1972,7 +1970,7 @@ deliver: uio->uio_td->td_ru.ru_msgrcv++; /* Fill uio until full or current end of socket buffer is reached. */ - len = min(uio->uio_resid, sb->sb_cc); + len = min(uio->uio_resid, sbavail(sb)); if (mp0 != NULL) { /* Dequeue as many mbufs as possible. */ if (!(flags & MSG_PEEK) && len >= sb->sb_mb->m_len) { @@ -1983,6 +1981,8 @@ deliver: for (m = sb->sb_mb; m != NULL && m->m_len <= len; m = m->m_next) { + KASSERT(!(m->m_flags & M_NOTAVAIL), + ("%s: m %p not available", __func__, m)); len -= m->m_len; uio->uio_resid -= m->m_len; sbfree(sb, m); @@ -2107,9 +2107,9 @@ soreceive_dgram(struct socket *so, struct sockaddr */ SOCKBUF_LOCK(&so->so_rcv); while ((m = so->so_rcv.sb_mb) == NULL) { - KASSERT(so->so_rcv.sb_cc == 0, - ("soreceive_dgram: sb_mb NULL but sb_cc %u", - so->so_rcv.sb_cc)); + KASSERT(sbavail(&so->so_rcv) == 0, + ("soreceive_dgram: sb_mb NULL but sbavail %u", + sbavail(&so->so_rcv))); if (so->so_error) { error = so->so_error; so->so_error = 0; @@ -3157,7 +3157,7 @@ filt_soread(struct knote *kn, long hint) so = kn->kn_fp->f_data; SOCKBUF_LOCK_ASSERT(&so->so_rcv); - kn->kn_data = so->so_rcv.sb_cc - so->so_rcv.sb_ctl; + kn->kn_data = sbavail(&so->so_rcv) - so->so_rcv.sb_ctl; if (so->so_rcv.sb_state & SBS_CANTRCVMORE) { kn->kn_flags |= EV_EOF; kn->kn_fflags = so->so_error; @@ -3167,7 +3167,7 @@ filt_soread(struct knote *kn, long hint) else if (kn->kn_sfflags & NOTE_LOWAT) return (kn->kn_data >= kn->kn_sdata); else - return (so->so_rcv.sb_cc >= so->so_rcv.sb_lowat); + return (sbavail(&so->so_rcv) >= so->so_rcv.sb_lowat); } static void @@ -3350,7 +3350,7 @@ soisdisconnected(struct socket *so) sorwakeup_locked(so); SOCKBUF_LOCK(&so->so_snd); so->so_snd.sb_state |= SBS_CANTSENDMORE; - sbdrop_locked(&so->so_snd, so->so_snd.sb_cc); + sbdrop_locked(&so->so_snd, sbused(&so->so_snd)); sowwakeup_locked(so); wakeup(&so->so_timeo); } Index: sys/kern/vnode_if.src =================================================================== --- sys/kern/vnode_if.src (.../head) (revision 266804) +++ sys/kern/vnode_if.src (.../projects/sendfile) (revision 266807) @@ -477,6 +477,19 @@ vop_getpages { }; +%% getpages_async vp L L L + +vop_getpages_async { + IN struct vnode *vp; + IN vm_page_t *m; + IN int count; + IN int reqpage; + IN vm_ooffset_t offset; + IN void (*vop_getpages_iodone)(void *); + IN void *arg; +}; + + %% putpages vp L L L vop_putpages { Index: sys/kern/uipc_sockbuf.c =================================================================== --- sys/kern/uipc_sockbuf.c (.../head) (revision 266804) +++ sys/kern/uipc_sockbuf.c (.../projects/sendfile) (revision 266807) @@ -68,7 +68,152 @@ static u_long sb_efficiency = 8; /* parameter for static struct mbuf *sbcut_internal(struct sockbuf *sb, int len); static void sbflush_internal(struct sockbuf *sb); +static void +sb_shift_nrdy(struct sockbuf *sb, struct mbuf *m) +{ + + SOCKBUF_LOCK_ASSERT(sb); + KASSERT(m->m_flags & M_NOTREADY, ("%s: m %p !M_NOTREADY", __func__, m)); + + m = m->m_next; + while (m != NULL && !(m->m_flags & M_NOTREADY)) { + m->m_flags &= ~M_BLOCKED; + sb->sb_acc += m->m_len; + m = m->m_next; + } + + sb->sb_fnrdy = m; +} + +int +sbready(struct sockbuf *sb, struct mbuf *m, int count) +{ + u_int blocker; + + SOCKBUF_LOCK(sb); + + if (sb->sb_state & SBS_CANTSENDMORE) { + SOCKBUF_UNLOCK(sb); + return (ENOTCONN); + } + + KASSERT(sb->sb_fnrdy != NULL, ("%s: sb %p NULL fnrdy", __func__, sb)); + + blocker = (sb->sb_fnrdy == m) ? M_BLOCKED : 0; + + for (int i = 0; i < count; i++, m = m->m_next) { + KASSERT(m->m_flags & M_NOTREADY, + ("%s: m %p !M_NOTREADY", __func__, m)); + m->m_flags &= ~(M_NOTREADY | blocker); + if (blocker) + sb->sb_acc += m->m_len; + } + + if (!blocker) { + SOCKBUF_UNLOCK(sb); + return (EWOULDBLOCK); + } + + /* This one was blocking all the queue. */ + for (; m && (m->m_flags & M_NOTREADY) == 0; m = m->m_next) { + KASSERT(m->m_flags & M_BLOCKED, + ("%s: m %p !M_BLOCKED", __func__, m)); + m->m_flags &= ~M_BLOCKED; + sb->sb_acc += m->m_len; + } + + sb->sb_fnrdy = m; + + SOCKBUF_UNLOCK(sb); + + return (0); +} + /* + * Adjust sockbuf state reflecting allocation of m. + */ +void +sballoc(struct sockbuf *sb, struct mbuf *m) +{ + + SOCKBUF_LOCK_ASSERT(sb); + + sb->sb_ccc += m->m_len; + + if (sb->sb_fnrdy == NULL) { + if (m->m_flags & M_NOTREADY) + sb->sb_fnrdy = m; + else + sb->sb_acc += m->m_len; + } else + m->m_flags |= M_BLOCKED; + + if (m->m_type != MT_DATA && m->m_type != MT_OOBDATA) + sb->sb_ctl += m->m_len; + + sb->sb_mbcnt += MSIZE; + sb->sb_mcnt += 1; + + if (m->m_flags & M_EXT) { + sb->sb_mbcnt += m->m_ext.ext_size; + sb->sb_ccnt += 1; + } +} + +/* + * Adjust sockbuf state reflecting freeing of m. + */ +void +sbfree(struct sockbuf *sb, struct mbuf *m) +{ + +#if 0 /* XXX: not yet: soclose() call path comes here w/o lock. */ + SOCKBUF_LOCK_ASSERT(sb); +#endif + + sb->sb_ccc -= m->m_len; + + if (!(m->m_flags & M_NOTAVAIL)) + sb->sb_acc -= m->m_len; + + if (sb->sb_fnrdy == m) + sb_shift_nrdy(sb, m); + + if (m->m_type != MT_DATA && m->m_type != MT_OOBDATA) + sb->sb_ctl -= m->m_len; + + sb->sb_mbcnt -= MSIZE; + sb->sb_mcnt -= 1; + if (m->m_flags & M_EXT) { + sb->sb_mbcnt -= m->m_ext.ext_size; + sb->sb_ccnt -= 1; + } + + if (sb->sb_sndptr == m) { + sb->sb_sndptr = NULL; + sb->sb_sndptroff = 0; + } + if (sb->sb_sndptroff != 0) + sb->sb_sndptroff -= m->m_len; +} + +/* + * Trim some amount of data from (first?) mbuf in buffer. + */ +void +sbmtrim(struct sockbuf *sb, struct mbuf *m, int len) +{ + + SOCKBUF_LOCK_ASSERT(sb); + KASSERT(len < m->m_len, ("%s: m %p len %d", __func__, m, len)); + + m->m_data += len; + m->m_len -= len; + sb->sb_acc -= len; + sb->sb_ccc -= len; +} + +/* * Socantsendmore indicates that no more data will be sent on the socket; it * would normally be applied to a socket when the user informs the system * that no more data is to be sent, by the protocol code (in case @@ -127,7 +272,7 @@ sbwait(struct sockbuf *sb) SOCKBUF_LOCK_ASSERT(sb); sb->sb_flags |= SB_WAIT; - return (msleep_sbt(&sb->sb_cc, &sb->sb_mtx, + return (msleep_sbt(&sb->sb_acc, &sb->sb_mtx, (sb->sb_flags & SB_NOINTR) ? PSOCK : PSOCK | PCATCH, "sbwait", sb->sb_timeo, 0, 0)); } @@ -184,7 +329,7 @@ sowakeup(struct socket *so, struct sockbuf *sb) sb->sb_flags &= ~SB_SEL; if (sb->sb_flags & SB_WAIT) { sb->sb_flags &= ~SB_WAIT; - wakeup(&sb->sb_cc); + wakeup(&sb->sb_acc); } KNOTE_LOCKED(&sb->sb_sel.si_note, 0); if (sb->sb_upcall != NULL) { @@ -519,7 +664,7 @@ sbappend(struct sockbuf *sb, struct mbuf *m) * that is, a stream protocol (such as TCP). */ void -sbappendstream_locked(struct sockbuf *sb, struct mbuf *m) +sbappendstream_locked(struct sockbuf *sb, struct mbuf *m, int flags) { SOCKBUF_LOCK_ASSERT(sb); @@ -529,8 +674,8 @@ void SBLASTMBUFCHK(sb); /* Remove all packet headers and mbuf tags to get a pure data chain. */ - m_demote(m, 1); - + m_demote(m, 1, flags & PRUS_NOTREADY ? M_NOTREADY : 0); + sbcompress(sb, m, sb->sb_mbtail); sb->sb_lastrecord = sb->sb_mb; @@ -543,38 +688,59 @@ void * that is, a stream protocol (such as TCP). */ void -sbappendstream(struct sockbuf *sb, struct mbuf *m) +sbappendstream(struct sockbuf *sb, struct mbuf *m, int flags) { SOCKBUF_LOCK(sb); - sbappendstream_locked(sb, m); + sbappendstream_locked(sb, m, flags); SOCKBUF_UNLOCK(sb); } #ifdef SOCKBUF_DEBUG void -sbcheck(struct sockbuf *sb) +sbcheck(struct sockbuf *sb, const char *file, int line) { - struct mbuf *m; - struct mbuf *n = 0; - u_long len = 0, mbcnt = 0; + struct mbuf *m, *n, *fnrdy; + u_long acc, ccc, mbcnt; SOCKBUF_LOCK_ASSERT(sb); + acc = ccc = mbcnt = 0; + fnrdy = NULL; + for (m = sb->sb_mb; m; m = n) { n = m->m_nextpkt; for (; m; m = m->m_next) { - len += m->m_len; + if ((m->m_flags & M_NOTREADY) && fnrdy == NULL) { + if (m != sb->sb_fnrdy) { + printf("sb %p: fnrdy %p != m %p\n", + sb, sb->sb_fnrdy, m); + goto fail; + } + fnrdy = m; + } + if (fnrdy) { + if (!(m->m_flags & M_NOTAVAIL)) { + printf("sb %p: fnrdy %p, m %p is avail\n", + sb, sb->sb_fnrdy, m); + goto fail; + } + } else + acc += m->m_len; + ccc += m->m_len; mbcnt += MSIZE; if (m->m_flags & M_EXT) /*XXX*/ /* pretty sure this is bogus */ mbcnt += m->m_ext.ext_size; } } - if (len != sb->sb_cc || mbcnt != sb->sb_mbcnt) { - printf("cc %ld != %u || mbcnt %ld != %u\n", len, sb->sb_cc, - mbcnt, sb->sb_mbcnt); - panic("sbcheck"); + if (acc != sb->sb_acc || ccc != sb->sb_ccc || mbcnt != sb->sb_mbcnt) { + printf("acc %ld/%u ccc %ld/%u mbcnt %ld/%u\n", + acc, sb->sb_acc, ccc, sb->sb_ccc, mbcnt, sb->sb_mbcnt); + goto fail; } + return; +fail: + panic("%s from %s:%u", __func__, file, line); } #endif @@ -800,6 +966,7 @@ sbcompress(struct sockbuf *sb, struct mbuf *m, str if (n && (n->m_flags & M_EOR) == 0 && M_WRITABLE(n) && ((sb->sb_flags & SB_NOCOALESCE) == 0) && + !(m->m_flags & M_NOTREADY) && m->m_len <= MCLBYTES / 4 && /* XXX: Don't copy too much */ m->m_len <= M_TRAILINGSPACE(n) && n->m_type == m->m_type) { @@ -806,7 +973,9 @@ sbcompress(struct sockbuf *sb, struct mbuf *m, str bcopy(mtod(m, caddr_t), mtod(n, caddr_t) + n->m_len, (unsigned)m->m_len); n->m_len += m->m_len; - sb->sb_cc += m->m_len; + sb->sb_ccc += m->m_len; + if (sb->sb_fnrdy == NULL) + sb->sb_acc += m->m_len; if (m->m_type != MT_DATA && m->m_type != MT_OOBDATA) /* XXX: Probably don't need.*/ sb->sb_ctl += m->m_len; @@ -843,13 +1012,13 @@ sbflush_internal(struct sockbuf *sb) * Don't call sbcut(sb, 0) if the leading mbuf is non-empty: * we would loop forever. Panic instead. */ - if (!sb->sb_cc && (sb->sb_mb == NULL || sb->sb_mb->m_len)) + if (sb->sb_ccc == 0 && (sb->sb_mb == NULL || sb->sb_mb->m_len)) break; - m_freem(sbcut_internal(sb, (int)sb->sb_cc)); + m_freem(sbcut_internal(sb, (int)sb->sb_ccc)); } - if (sb->sb_cc || sb->sb_mb || sb->sb_mbcnt) - panic("sbflush_internal: cc %u || mb %p || mbcnt %u", - sb->sb_cc, (void *)sb->sb_mb, sb->sb_mbcnt); + KASSERT(sb->sb_ccc == 0 && sb->sb_mb == 0 && sb->sb_mbcnt == 0, + ("%s: ccc %u mb %p mbcnt %u", __func__, + sb->sb_ccc, (void *)sb->sb_mb, sb->sb_mbcnt)); } void @@ -891,7 +1060,9 @@ sbcut_internal(struct sockbuf *sb, int len) if (m->m_len > len) { m->m_len -= len; m->m_data += len; - sb->sb_cc -= len; + sb->sb_ccc -= len; + if (!(m->m_flags & M_NOTAVAIL)) + sb->sb_acc -= len; if (sb->sb_sndptroff != 0) sb->sb_sndptroff -= len; if (m->m_type != MT_DATA && m->m_type != MT_OOBDATA) @@ -977,8 +1148,8 @@ sbsndptr(struct sockbuf *sb, u_int off, u_int len, struct mbuf *m, *ret; KASSERT(sb->sb_mb != NULL, ("%s: sb_mb is NULL", __func__)); - KASSERT(off + len <= sb->sb_cc, ("%s: beyond sb", __func__)); - KASSERT(sb->sb_sndptroff <= sb->sb_cc, ("%s: sndptroff broken", __func__)); + KASSERT(off + len <= sb->sb_acc, ("%s: beyond sb", __func__)); + KASSERT(sb->sb_sndptroff <= sb->sb_acc, ("%s: sndptroff broken", __func__)); /* * Is off below stored offset? Happens on retransmits. @@ -1091,7 +1262,7 @@ void sbtoxsockbuf(struct sockbuf *sb, struct xsockbuf *xsb) { - xsb->sb_cc = sb->sb_cc; + xsb->sb_cc = sb->sb_ccc; xsb->sb_hiwat = sb->sb_hiwat; xsb->sb_mbcnt = sb->sb_mbcnt; xsb->sb_mcnt = sb->sb_mcnt; Index: sys/kern/uipc_syscalls.c =================================================================== --- sys/kern/uipc_syscalls.c (.../head) (revision 266804) +++ sys/kern/uipc_syscalls.c (.../projects/sendfile) (revision 266807) @@ -132,9 +132,10 @@ static int filt_sfsync(struct knote *kn, long hint */ static SYSCTL_NODE(_kern_ipc, OID_AUTO, sendfile, CTLFLAG_RW, 0, "sendfile(2) tunables"); -static int sfreadahead = 1; + +static int sfreadahead = 0; SYSCTL_INT(_kern_ipc_sendfile, OID_AUTO, readahead, CTLFLAG_RW, - &sfreadahead, 0, "Number of sendfile(2) read-ahead MAXBSIZE blocks"); + &sfreadahead, 0, "Read this more pages than socket buffer can accept"); #ifdef SFSYNC_DEBUG static int sf_sync_debug = 0; @@ -1988,7 +1989,7 @@ filt_sfsync(struct knote *kn, long hint) * Detach mapped page and release resources back to the system. */ int -sf_buf_mext(struct mbuf *mb, void *addr, void *args) +sf_mext_free(struct mbuf *mb, void *addr, void *args) { vm_page_t m; struct sendfile_sync *sfs; @@ -2009,13 +2010,42 @@ int sfs = addr; sf_sync_deref(sfs); } - /* - * sfs may be invalid at this point, don't use it! - */ return (EXT_FREE_OK); } /* + * Same as above, but forces the page to be detached from the object + * and go into free pool. + */ +static int +sf_mext_free_nocache(struct mbuf *mb, void *addr, void *args) +{ + vm_page_t m; + struct sendfile_sync *sfs; + + m = sf_buf_page(args); + sf_buf_free(args); + vm_page_lock(m); + vm_page_unwire(m, 0); + if (m->wire_count == 0) { + vm_object_t obj; + + if ((obj = m->object) == NULL) + vm_page_free(m); + else if (!vm_page_xbusied(m) && VM_OBJECT_TRYWLOCK(obj)) { + vm_page_free(m); + VM_OBJECT_WUNLOCK(obj); + } + } + vm_page_unlock(m); + if (addr != NULL) { + sfs = addr; + sf_sync_deref(sfs); + } + return (EXT_FREE_OK); +} + +/* * Called to remove a reference to a sf_sync object. * * This is generally done during the mbuf free path to signify @@ -2608,106 +2638,181 @@ freebsd4_sendfile(struct thread *td, struct freebs } #endif /* COMPAT_FREEBSD4 */ + /* + * How much data to put into page i of n. + * Only first and last pages are special. + */ +static inline off_t +xfsize(int i, int n, off_t off, off_t len) +{ + + if (i == 0) + return (omin(PAGE_SIZE - (off & PAGE_MASK), len)); + + if (i == n - 1 && ((off + len) & PAGE_MASK) > 0) + return ((off + len) & PAGE_MASK); + + return (PAGE_SIZE); +} + +/* + * Offset within object for i page. + */ +static inline vm_offset_t +vmoff(int i, off_t off) +{ + + if (i == 0) + return ((vm_offset_t)off); + + return (trunc_page(off + i * PAGE_SIZE)); +} + +/* + * Pretend as if we don't have enough space, subtract xfsize() of + * all pages that failed. + */ +static inline void +fixspace(int old, int new, off_t off, int *space) +{ + + KASSERT(old > new, ("%s: old %d new %d", __func__, old, new)); + + /* Subtract last one. */ + *space -= xfsize(old - 1, old, off, *space); + old--; + + if (new == old) + /* There was only one page. */ + return; + + /* Subtract first one. */ + if (new == 0) { + *space -= xfsize(0, old, off, *space); + new++; + } + + /* Rest of pages are full sized. */ + *space -= (old - new) * PAGE_SIZE; + + KASSERT(*space >= 0, ("%s: space went backwards", __func__)); +} + +struct sf_io { + u_int nios; + int npages; + struct file *sock_fp; + struct mbuf *m; + vm_page_t pa[]; +}; + +static void +sf_io_done(void *arg) +{ + struct sf_io *sfio = arg; + struct socket *so; + + if (!refcount_release(&sfio->nios)) + return; + + so = sfio->sock_fp->f_data; + + if (sbready(&so->so_snd, sfio->m, sfio->npages) == 0) { + struct mbuf *m; + + m = m_get(M_NOWAIT, MT_DATA); + if (m == NULL) { + panic("XXXGL"); + } + m->m_len = 0; + CURVNET_SET(so->so_vnet); + /* XXXGL: curthread */ + (void )(so->so_proto->pr_usrreqs->pru_send) + (so, 0, m, NULL, NULL, curthread); + CURVNET_RESTORE(); + } + + /* XXXGL: curthread */ + fdrop(sfio->sock_fp, curthread); + free(sfio, M_TEMP); +} + static int -sendfile_readpage(vm_object_t obj, struct vnode *vp, int nd, - off_t off, int xfsize, int bsize, struct thread *td, vm_page_t *res) +sendfile_swapin(vm_object_t obj, struct sf_io *sfio, off_t off, off_t len, + int npages, int rhpages) { - vm_page_t m; - vm_pindex_t pindex; - ssize_t resid; - int error, readahead, rv; + vm_page_t *pa = sfio->pa; + int nios; - pindex = OFF_TO_IDX(off); + nios = 0; VM_OBJECT_WLOCK(obj); - m = vm_page_grab(obj, pindex, (vp != NULL ? VM_ALLOC_NOBUSY | - VM_ALLOC_IGN_SBUSY : 0) | VM_ALLOC_WIRED | VM_ALLOC_NORMAL); + for (int i = 0; i < npages; i++) + pa[i] = vm_page_grab(obj, OFF_TO_IDX(vmoff(i, off)), + VM_ALLOC_WIRED | VM_ALLOC_NORMAL); - /* - * Check if page is valid for what we need, otherwise initiate I/O. - * - * The non-zero nd argument prevents disk I/O, instead we - * return the caller what he specified in nd. In particular, - * if we already turned some pages into mbufs, nd == EAGAIN - * and the main function send them the pages before we come - * here again and block. - */ - if (m->valid != 0 && vm_page_is_valid(m, off & PAGE_MASK, xfsize)) { - if (vp == NULL) - vm_page_xunbusy(m); - VM_OBJECT_WUNLOCK(obj); - *res = m; - return (0); - } else if (nd != 0) { - if (vp == NULL) - vm_page_xunbusy(m); - error = nd; - goto free_page; - } + for (int i = 0; i < npages;) { + int j, a, count, rv; - /* - * Get the page from backing store. - */ - error = 0; - if (vp != NULL) { - VM_OBJECT_WUNLOCK(obj); - readahead = sfreadahead * MAXBSIZE; + if (vm_page_is_valid(pa[i], vmoff(i, off) & PAGE_MASK, + xfsize(i, npages, off, len))) { + vm_page_xunbusy(pa[i]); + i++; + continue; + } - /* - * Use vn_rdwr() instead of the pager interface for - * the vnode, to allow the read-ahead. - * - * XXXMAC: Because we don't have fp->f_cred here, we - * pass in NOCRED. This is probably wrong, but is - * consistent with our original implementation. - */ - error = vn_rdwr(UIO_READ, vp, NULL, readahead, trunc_page(off), - UIO_NOCOPY, IO_NODELOCKED | IO_VMIO | ((readahead / - bsize) << IO_SEQSHIFT), td->td_ucred, NOCRED, &resid, td); - SFSTAT_INC(sf_iocnt); - VM_OBJECT_WLOCK(obj); - } else { - if (vm_pager_has_page(obj, pindex, NULL, NULL)) { - rv = vm_pager_get_pages(obj, &m, 1, 0); - SFSTAT_INC(sf_iocnt); - m = vm_page_lookup(obj, pindex); - if (m == NULL) - error = EIO; - else if (rv != VM_PAGER_OK) { - vm_page_lock(m); - vm_page_free(m); - vm_page_unlock(m); - m = NULL; - error = EIO; + for (j = i + 1; j < npages; j++) + if (vm_page_is_valid(pa[j], vmoff(j, off) & PAGE_MASK, + xfsize(j, npages, off, len))) + break; + + while (!vm_pager_has_page(obj, OFF_TO_IDX(vmoff(i, off)), + NULL, &a) && i < j) { + pmap_zero_page(pa[i]); + pa[i]->valid = VM_PAGE_BITS_ALL; + pa[i]->dirty = 0; + vm_page_xunbusy(pa[i]); + i++; + } + if (i == j) + continue; + + count = min(a + 1, npages + rhpages - i); + for (j = npages; j < i + count; j++) { + pa[j] = vm_page_grab(obj, OFF_TO_IDX(vmoff(j, off)), + VM_ALLOC_NORMAL | VM_ALLOC_NOWAIT); + if (pa[j] == NULL) { + count = j - i; + break; } - } else { - pmap_zero_page(m); - m->valid = VM_PAGE_BITS_ALL; - m->dirty = 0; + if (pa[j]->valid) { + vm_page_xunbusy(pa[j]); + count = j - i; + break; + } } - if (m != NULL) - vm_page_xunbusy(m); + + refcount_acquire(&sfio->nios); + rv = vm_pager_get_pages_async(obj, pa + i, count, 0, + &sf_io_done, sfio); + + KASSERT(rv == VM_PAGER_OK, ("%s: pager fail obj %p page %p", + __func__, obj, pa[i])); + + SFSTAT_INC(sf_iocnt); + nios++; + + for (j = i; j < i + count && j < npages; j++) + KASSERT(pa[j] == vm_page_lookup(obj, + OFF_TO_IDX(vmoff(j, off))), + ("pa[j] %p lookup %p\n", pa[j], + vm_page_lookup(obj, OFF_TO_IDX(vmoff(j, off))))); + + i += count; } - if (error == 0) { - *res = m; - } else if (m != NULL) { -free_page: - vm_page_lock(m); - vm_page_unwire(m, 0); - /* - * See if anyone else might know about this page. If - * not and it is not valid, then free it. - */ - if (m->wire_count == 0 && m->valid == 0 && !vm_page_busied(m)) - vm_page_free(m); - vm_page_unlock(m); - } - KASSERT(error != 0 || (m->wire_count > 0 && - vm_page_is_valid(m, off & PAGE_MASK, xfsize)), - ("wrong page state m %p off %#jx xfsize %d", m, (uintmax_t)off, - xfsize)); VM_OBJECT_WUNLOCK(obj); - return (error); + + return (nios); } static int @@ -2814,41 +2919,26 @@ vn_sendfile(struct file *fp, int sockfd, struct ui struct vnode *vp; struct vm_object *obj; struct socket *so; - struct mbuf *m; + struct mbuf *m, *mh, *mhtail; struct sf_buf *sf; - struct vm_page *pg; struct shmfd *shmfd; struct vattr va; - off_t off, xfsize, fsbytes, sbytes, rem, obj_size; - int error, bsize, nd, hdrlen, mnw; + off_t off, sbytes, rem, obj_size; + int error, serror, bsize, hdrlen; - pg = NULL; obj = NULL; so = NULL; - m = NULL; - fsbytes = sbytes = 0; - hdrlen = mnw = 0; - rem = nbytes; - obj_size = 0; + m = mh = NULL; + sbytes = 0; error = sendfile_getobj(td, fp, &obj, &vp, &shmfd, &obj_size, &bsize); if (error != 0) return (error); - if (rem == 0) - rem = obj_size; error = kern_sendfile_getsock(td, sockfd, &sock_fp, &so); if (error != 0) goto out; - /* - * Do not wait on memory allocations but return ENOMEM for - * caller to retry later. - * XXX: Experimental. - */ - if (flags & SF_MNOWAIT) - mnw = 1; - #ifdef MAC error = mac_socket_check_send(td->td_ucred, so); if (error != 0) @@ -2856,31 +2946,27 @@ vn_sendfile(struct file *fp, int sockfd, struct ui #endif /* If headers are specified copy them into mbufs. */ - if (hdr_uio != NULL) { + if (hdr_uio != NULL && hdr_uio->uio_resid > 0) { hdr_uio->uio_td = td; hdr_uio->uio_rw = UIO_WRITE; - if (hdr_uio->uio_resid > 0) { - /* - * In FBSD < 5.0 the nbytes to send also included - * the header. If compat is specified subtract the - * header size from nbytes. - */ - if (kflags & SFK_COMPAT) { - if (nbytes > hdr_uio->uio_resid) - nbytes -= hdr_uio->uio_resid; - else - nbytes = 0; - } - m = m_uiotombuf(hdr_uio, (mnw ? M_NOWAIT : M_WAITOK), - 0, 0, 0); - if (m == NULL) { - error = mnw ? EAGAIN : ENOBUFS; - goto out; - } - hdrlen = m_length(m, NULL); + /* + * In FBSD < 5.0 the nbytes to send also included + * the header. If compat is specified subtract the + * header size from nbytes. + */ + if (kflags & SFK_COMPAT) { + if (nbytes > hdr_uio->uio_resid) + nbytes -= hdr_uio->uio_resid; + else + nbytes = 0; } - } + mh = m_uiotombuf(hdr_uio, M_WAITOK, 0, 0, 0); + hdrlen = m_length(mh, &mhtail); + } else + hdrlen = 0; + rem = nbytes ? omin(nbytes, obj_size - offset) : obj_size - offset; + /* * Protect against multiple writers to the socket. * @@ -2900,21 +2986,13 @@ vn_sendfile(struct file *fp, int sockfd, struct ui * The outer loop checks the state and available space of the socket * and takes care of the overall progress. */ - for (off = offset; ; ) { + for (off = offset; rem > 0; ) { + struct sf_io *sfio; + vm_page_t *pa; struct mbuf *mtail; - int loopbytes; - int space; - int done; + int nios, space, npages, rhpages; - if ((nbytes != 0 && nbytes == fsbytes) || - (nbytes == 0 && obj_size == fsbytes)) - break; - mtail = NULL; - loopbytes = 0; - space = 0; - done = 0; - /* * Check the socket state for ongoing connection, * no errors and space in socket buffer. @@ -2990,53 +3068,44 @@ retry_space: VOP_UNLOCK(vp, 0); goto done; } - obj_size = va.va_size; + if (va.va_size != obj_size) { + if (nbytes == 0) + rem += va.va_size - obj_size; + else if (offset + nbytes > va.va_size) + rem -= (offset + nbytes - va.va_size); + obj_size = va.va_size; + } } + if (space > rem) + space = rem; + + if (off & PAGE_MASK) + npages = 1 + howmany(space - + (PAGE_SIZE - (off & PAGE_MASK)), PAGE_SIZE); + else + npages = howmany(space, PAGE_SIZE); + + rhpages = SF_READAHEAD(flags) ? + SF_READAHEAD(flags) : sfreadahead; + rhpages = min(howmany(obj_size - (off & ~PAGE_MASK) - + (npages * PAGE_SIZE), PAGE_SIZE), rhpages); + + sfio = malloc(sizeof(struct sf_io) + + (rhpages + npages) * sizeof(vm_page_t), M_TEMP, M_WAITOK); + refcount_init(&sfio->nios, 1); + + nios = sendfile_swapin(obj, sfio, off, space, npages, rhpages); + /* * Loop and construct maximum sized mbuf chain to be bulk * dumped into socket buffer. */ - while (space > loopbytes) { - vm_offset_t pgoff; + pa = sfio->pa; + for (int i = 0; i < npages; i++) { struct mbuf *m0; /* - * Calculate the amount to transfer. - * Not to exceed a page, the EOF, - * or the passed in nbytes. - */ - pgoff = (vm_offset_t)(off & PAGE_MASK); - rem = obj_size - offset; - if (nbytes != 0) - rem = omin(rem, nbytes); - rem -= fsbytes + loopbytes; - xfsize = omin(PAGE_SIZE - pgoff, rem); - xfsize = omin(space - loopbytes, xfsize); - if (xfsize <= 0) { - done = 1; /* all data sent */ - break; - } - - /* - * Attempt to look up the page. Allocate - * if not found or wait and loop if busy. - */ - if (m != NULL) - nd = EAGAIN; /* send what we already got */ - else if ((flags & SF_NODISKIO) != 0) - nd = EBUSY; - else - nd = 0; - error = sendfile_readpage(obj, vp, nd, off, - xfsize, bsize, td, &pg); - if (error != 0) { - if (error == EAGAIN) - error = 0; /* not a real error */ - break; - } - - /* * Get a sendfile buf. When allocating the * first buffer for mbuf chain, we usually * wait as long as necessary, but this wait @@ -3045,17 +3114,18 @@ retry_space: * threads might exhaust the buffers and then * deadlock. */ - sf = sf_buf_alloc(pg, (mnw || m != NULL) ? SFB_NOWAIT : - SFB_CATCH); + sf = sf_buf_alloc(pa[i], + m != NULL ? SFB_NOWAIT : SFB_CATCH); if (sf == NULL) { SFSTAT_INC(sf_allocfail); - vm_page_lock(pg); - vm_page_unwire(pg, 0); - KASSERT(pg->object != NULL, - ("%s: object disappeared", __func__)); - vm_page_unlock(pg); + for (int j = i; j < npages; j++) { + vm_page_lock(pa[j]); + vm_page_unwire(pa[j], 0); + vm_page_unlock(pa[j]); + } if (m == NULL) - error = (mnw ? EAGAIN : EINTR); + error = ENOBUFS; + fixspace(npages, i, off, &space); break; } @@ -3063,36 +3133,26 @@ retry_space: * Get an mbuf and set it up as having * external storage. */ - m0 = m_get((mnw ? M_NOWAIT : M_WAITOK), MT_DATA); - if (m0 == NULL) { - error = (mnw ? EAGAIN : ENOBUFS); - (void)sf_buf_mext(NULL, NULL, sf); - break; - } - if (m_extadd(m0, (caddr_t )sf_buf_kva(sf), PAGE_SIZE, - sf_buf_mext, sfs, sf, M_RDONLY, EXT_SFBUF, - (mnw ? M_NOWAIT : M_WAITOK)) != 0) { - error = (mnw ? EAGAIN : ENOBUFS); - (void)sf_buf_mext(NULL, NULL, sf); - m_freem(m0); - break; - } - m0->m_data = (char *)sf_buf_kva(sf) + pgoff; - m0->m_len = xfsize; + m0 = m_get(M_WAITOK, MT_DATA); + (void )m_extadd(m0, (caddr_t )sf_buf_kva(sf), PAGE_SIZE, + (flags & SF_NOCACHE) ? sf_mext_free_nocache : + sf_mext_free, sfs, sf, M_RDONLY, EXT_SFBUF, + M_WAITOK); + m0->m_data = (char *)sf_buf_kva(sf) + + (vmoff(i, off) & PAGE_MASK); + m0->m_len = xfsize(i, npages, off, space); + m0->m_flags |= M_NOTREADY; + if (i == 0) + sfio->m = m0; + /* Append to mbuf chain. */ if (mtail != NULL) mtail->m_next = m0; - else if (m != NULL) - m_last(m)->m_next = m0; else m = m0; mtail = m0; - /* Keep track of bits processed. */ - loopbytes += xfsize; - off += xfsize; - /* * XXX eventually this should be a sfsync * method call! @@ -3104,47 +3164,51 @@ retry_space: if (vp != NULL) VOP_UNLOCK(vp, 0); + /* Keep track of bytes processed. */ + off += space; + rem -= space; + + /* Prepend header, if any. */ + if (hdrlen) { + mhtail->m_next = m; + m = mh; + mh = NULL; + } + + if (error) { + free(sfio, M_TEMP); + goto done; + } + /* Add the buffer chain to the socket buffer. */ - if (m != NULL) { - int mlen, err; + KASSERT(m_length(m, NULL) == space + hdrlen, + ("%s: mlen %u space %d hdrlen %d", + __func__, m_length(m, NULL), space, hdrlen)); - mlen = m_length(m, NULL); - SOCKBUF_LOCK(&so->so_snd); - if (so->so_snd.sb_state & SBS_CANTSENDMORE) { - error = EPIPE; - SOCKBUF_UNLOCK(&so->so_snd); - goto done; - } - SOCKBUF_UNLOCK(&so->so_snd); - CURVNET_SET(so->so_vnet); - /* Avoid error aliasing. */ - err = (*so->so_proto->pr_usrreqs->pru_send) - (so, 0, m, NULL, NULL, td); - CURVNET_RESTORE(); - if (err == 0) { - /* - * We need two counters to get the - * file offset and nbytes to send - * right: - * - sbytes contains the total amount - * of bytes sent, including headers. - * - fsbytes contains the total amount - * of bytes sent from the file. - */ - sbytes += mlen; - fsbytes += mlen; - if (hdrlen) { - fsbytes -= hdrlen; - hdrlen = 0; - } - } else if (error == 0) - error = err; - m = NULL; /* pru_send always consumes */ + CURVNET_SET(so->so_vnet); + if (nios == 0) { + free(sfio, M_TEMP); + serror = (*so->so_proto->pr_usrreqs->pru_send) + (so, 0, m, NULL, NULL, td); + } else { + sfio->sock_fp = sock_fp; + sfio->npages = npages; + fhold(sock_fp); + serror = (*so->so_proto->pr_usrreqs->pru_send) + (so, PRUS_NOTREADY, m, NULL, NULL, td); + sf_io_done(sfio); } + CURVNET_RESTORE(); - /* Quit outer loop on error or when we're done. */ - if (done) - break; + if (serror == 0) { + sbytes += space + hdrlen; + if (hdrlen) + hdrlen = 0; + } else if (error == 0) + error = serror; + m = NULL; /* pru_send always consumes */ + + /* Quit outer loop on error. */ if (error != 0) goto done; } @@ -3179,6 +3243,8 @@ out: fdrop(sock_fp, td); if (m) m_freem(m); + if (mh) + m_freem(mh); if (error == ERESTART) error = EINTR; Index: sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c =================================================================== --- sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c (.../head) (revision 266804) +++ sys/netgraph/bluetooth/socket/ng_btsocket_l2cap.c (.../projects/sendfile) (revision 266807) @@ -1127,9 +1127,8 @@ ng_btsocket_l2cap_process_l2ca_write_rsp(struct ng /* * Check if we have more data to send */ - sbdroprecord(&pcb->so->so_snd); - if (pcb->so->so_snd.sb_cc > 0) { + if (sbavail(&pcb->so->so_snd) > 0) { if (ng_btsocket_l2cap_send2(pcb) == 0) ng_btsocket_l2cap_timeout(pcb); else @@ -2510,7 +2509,7 @@ ng_btsocket_l2cap_send2(ng_btsocket_l2cap_pcb_p pc mtx_assert(&pcb->pcb_mtx, MA_OWNED); - if (pcb->so->so_snd.sb_cc == 0) + if (sbavail(&pcb->so->so_snd) == 0) return (EINVAL); /* XXX */ m = m_dup(pcb->so->so_snd.sb_mb, M_NOWAIT); Index: sys/netgraph/bluetooth/socket/ng_btsocket_rfcomm.c =================================================================== --- sys/netgraph/bluetooth/socket/ng_btsocket_rfcomm.c (.../head) (revision 266804) +++ sys/netgraph/bluetooth/socket/ng_btsocket_rfcomm.c (.../projects/sendfile) (revision 266807) @@ -3274,7 +3274,7 @@ ng_btsocket_rfcomm_pcb_send(ng_btsocket_rfcomm_pcb } for (error = 0, sent = 0; sent < limit; sent ++) { - length = min(pcb->mtu, pcb->so->so_snd.sb_cc); + length = min(pcb->mtu, sbavail(&pcb->so->so_snd)); if (length == 0) break; Index: sys/netgraph/bluetooth/socket/ng_btsocket_sco.c =================================================================== --- sys/netgraph/bluetooth/socket/ng_btsocket_sco.c (.../head) (revision 266804) +++ sys/netgraph/bluetooth/socket/ng_btsocket_sco.c (.../projects/sendfile) (revision 266807) @@ -906,7 +906,7 @@ ng_btsocket_sco_default_msg_input(struct ng_mesg * sbdroprecord(&pcb->so->so_snd); /* Send more if we have any */ - if (pcb->so->so_snd.sb_cc > 0) + if (sbavail(&pcb->so->so_snd) > 0) if (ng_btsocket_sco_send2(pcb) == 0) ng_btsocket_sco_timeout(pcb); @@ -1744,7 +1744,7 @@ ng_btsocket_sco_send2(ng_btsocket_sco_pcb_p pcb) mtx_assert(&pcb->pcb_mtx, MA_OWNED); while (pcb->rt->pending < pcb->rt->num_pkts && - pcb->so->so_snd.sb_cc > 0) { + sbavail(&pcb->so->so_snd) > 0) { /* Get a copy of the first packet on send queue */ m = m_dup(pcb->so->so_snd.sb_mb, M_NOWAIT); if (m == NULL) { Index: sys/ofed/drivers/infiniband/ulp/sdp/sdp_main.c =================================================================== --- sys/ofed/drivers/infiniband/ulp/sdp/sdp_main.c (.../head) (revision 266804) +++ sys/ofed/drivers/infiniband/ulp/sdp/sdp_main.c (.../projects/sendfile) (revision 266807) @@ -746,7 +746,7 @@ sdp_start_disconnect(struct sdp_sock *ssk) ("sdp_start_disconnect: sdp_drop() returned NULL")); } else { soisdisconnecting(so); - unread = so->so_rcv.sb_cc; + unread = sbused(&so->so_rcv); sbflush(&so->so_rcv); sdp_usrclosed(ssk); if (!(ssk->flags & SDP_DROPPED)) { @@ -888,7 +888,7 @@ sdp_append(struct sdp_sock *ssk, struct sockbuf *s m_adj(mb, SDP_HEAD_SIZE); n->m_pkthdr.len += mb->m_pkthdr.len; n->m_flags |= mb->m_flags & (M_PUSH | M_URG); - m_demote(mb, 1); + m_demote(mb, 1, 0); sbcompress(sb, mb, sb->sb_mbtail); return; } @@ -1258,7 +1258,7 @@ sdp_sorecv(struct socket *so, struct sockaddr **ps /* We will never ever get anything unless we are connected. */ if (!(so->so_state & (SS_ISCONNECTED|SS_ISDISCONNECTED))) { /* When disconnecting there may be still some data left. */ - if (sb->sb_cc > 0) + if (sbavail(sb)) goto deliver; if (!(so->so_state & SS_ISDISCONNECTED)) error = ENOTCONN; @@ -1266,7 +1266,7 @@ sdp_sorecv(struct socket *so, struct sockaddr **ps } /* Socket buffer is empty and we shall not block. */ - if (sb->sb_cc == 0 && + if (sbavail(sb) == 0 && ((so->so_state & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)))) { error = EAGAIN; goto out; @@ -1277,7 +1277,7 @@ restart: /* Abort if socket has reported problems. */ if (so->so_error) { - if (sb->sb_cc > 0) + if (sbavail(sb)) goto deliver; if (oresid > uio->uio_resid) goto out; @@ -1289,7 +1289,7 @@ restart: /* Door is closed. Deliver what is left, if any. */ if (sb->sb_state & SBS_CANTRCVMORE) { - if (sb->sb_cc > 0) + if (sbavail(sb)) goto deliver; else goto out; @@ -1296,18 +1296,18 @@ restart: } /* Socket buffer got some data that we shall deliver now. */ - if (sb->sb_cc > 0 && !(flags & MSG_WAITALL) && + if (sbavail(sb) && !(flags & MSG_WAITALL) && ((so->so_state & SS_NBIO) || (flags & (MSG_DONTWAIT|MSG_NBIO)) || - sb->sb_cc >= sb->sb_lowat || - sb->sb_cc >= uio->uio_resid || - sb->sb_cc >= sb->sb_hiwat) ) { + sbavail(sb) >= sb->sb_lowat || + sbavail(sb) >= uio->uio_resid || + sbavail(sb) >= sb->sb_hiwat) ) { goto deliver; } /* On MSG_WAITALL we must wait until all data or error arrives. */ if ((flags & MSG_WAITALL) && - (sb->sb_cc >= uio->uio_resid || sb->sb_cc >= sb->sb_lowat)) + (sbavail(sb) >= uio->uio_resid || sbavail(sb) >= sb->sb_lowat)) goto deliver; /* @@ -1321,7 +1321,7 @@ restart: deliver: SOCKBUF_LOCK_ASSERT(&so->so_rcv); - KASSERT(sb->sb_cc > 0, ("%s: sockbuf empty", __func__)); + KASSERT(sbavail(sb), ("%s: sockbuf empty", __func__)); KASSERT(sb->sb_mb != NULL, ("%s: sb_mb == NULL", __func__)); /* Statistics. */ @@ -1329,7 +1329,7 @@ deliver: uio->uio_td->td_ru.ru_msgrcv++; /* Fill uio until full or current end of socket buffer is reached. */ - len = min(uio->uio_resid, sb->sb_cc); + len = min(uio->uio_resid, sbavail(sb)); if (mp0 != NULL) { /* Dequeue as many mbufs as possible. */ if (!(flags & MSG_PEEK) && len >= sb->sb_mb->m_len) { @@ -1509,7 +1509,7 @@ sdp_urg(struct sdp_sock *ssk, struct mbuf *mb) if (so == NULL) return; - so->so_oobmark = so->so_rcv.sb_cc + mb->m_pkthdr.len - 1; + so->so_oobmark = sbused(&so->so_rcv) + mb->m_pkthdr.len - 1; sohasoutofband(so); ssk->oobflags &= ~(SDP_HAVEOOB | SDP_HADOOB); if (!(so->so_options & SO_OOBINLINE)) { Index: sys/ofed/drivers/infiniband/ulp/sdp/sdp_rx.c =================================================================== --- sys/ofed/drivers/infiniband/ulp/sdp/sdp_rx.c (.../head) (revision 266804) +++ sys/ofed/drivers/infiniband/ulp/sdp/sdp_rx.c (.../projects/sendfile) (revision 266807) @@ -183,7 +183,7 @@ sdp_post_recvs_needed(struct sdp_sock *ssk) * Compute bytes in the receive queue and socket buffer. */ bytes_in_process = (posted - SDP_MIN_TX_CREDITS) * buffer_size; - bytes_in_process += ssk->socket->so_rcv.sb_cc; + bytes_in_process += sbused(&ssk->socket->so_rcv); return bytes_in_process < max_bytes; } Index: sys/sys/socket.h =================================================================== --- sys/sys/socket.h (.../head) (revision 266804) +++ sys/sys/socket.h (.../projects/sendfile) (revision 266807) @@ -602,12 +602,15 @@ struct sf_hdtr_all { * Sendfile-specific flag(s) */ #define SF_NODISKIO 0x00000001 -#define SF_MNOWAIT 0x00000002 +#define SF_MNOWAIT 0x00000002 /* unused since 11.0 */ #define SF_SYNC 0x00000004 #define SF_KQUEUE 0x00000008 +#define SF_NOCACHE 0x00000010 +#define SF_FLAGS(rh, flags) (((rh) << 16) | (flags)) #ifdef _KERNEL #define SFK_COMPAT 0x00000001 +#define SF_READAHEAD(flags) ((flags) >> 16) #endif /* _KERNEL */ #endif /* __BSD_VISIBLE */ Index: sys/sys/sockbuf.h =================================================================== --- sys/sys/sockbuf.h (.../head) (revision 266804) +++ sys/sys/sockbuf.h (.../projects/sendfile) (revision 266807) @@ -89,8 +89,13 @@ struct sockbuf { struct mbuf *sb_lastrecord; /* (c/d) first mbuf of last * record in socket buffer */ struct mbuf *sb_sndptr; /* (c/d) pointer into mbuf chain */ + struct mbuf *sb_fnrdy; /* (c/d) pointer to first not ready buffer */ +#if 0 + struct mbuf *sb_lnrdy; /* (c/d) pointer to last not ready buffer */ +#endif u_int sb_sndptroff; /* (c/d) byte offset of ptr into chain */ - u_int sb_cc; /* (c/d) actual chars in buffer */ + u_int sb_acc; /* (c/d) available chars in buffer */ + u_int sb_ccc; /* (c/d) claimed chars in buffer */ u_int sb_hiwat; /* (c/d) max actual char count */ u_int sb_mbcnt; /* (c/d) chars of mbufs used */ u_int sb_mcnt; /* (c/d) number of mbufs in buffer */ @@ -120,10 +125,17 @@ struct sockbuf { #define SOCKBUF_LOCK_ASSERT(_sb) mtx_assert(SOCKBUF_MTX(_sb), MA_OWNED) #define SOCKBUF_UNLOCK_ASSERT(_sb) mtx_assert(SOCKBUF_MTX(_sb), MA_NOTOWNED) +/* + * Socket buffer private mbuf(9) flags. + */ +#define M_NOTREADY M_PROTO1 /* m_data not populated yet */ +#define M_BLOCKED M_PROTO2 /* M_NOTREADY in front of m */ +#define M_NOTAVAIL (M_NOTREADY | M_BLOCKED) + void sbappend(struct sockbuf *sb, struct mbuf *m); void sbappend_locked(struct sockbuf *sb, struct mbuf *m); -void sbappendstream(struct sockbuf *sb, struct mbuf *m); -void sbappendstream_locked(struct sockbuf *sb, struct mbuf *m); +void sbappendstream(struct sockbuf *sb, struct mbuf *m, int flags); +void sbappendstream_locked(struct sockbuf *sb, struct mbuf *m, int flags); int sbappendaddr(struct sockbuf *sb, const struct sockaddr *asa, struct mbuf *m0, struct mbuf *control); int sbappendaddr_locked(struct sockbuf *sb, const struct sockaddr *asa, @@ -136,7 +148,6 @@ int sbappendcontrol_locked(struct sockbuf *sb, str struct mbuf *control); void sbappendrecord(struct sockbuf *sb, struct mbuf *m0); void sbappendrecord_locked(struct sockbuf *sb, struct mbuf *m0); -void sbcheck(struct sockbuf *sb); void sbcompress(struct sockbuf *sb, struct mbuf *m, struct mbuf *n); struct mbuf * sbcreatecontrol(caddr_t p, int size, int type, int level); @@ -162,59 +173,54 @@ void sbtoxsockbuf(struct sockbuf *sb, struct xsock int sbwait(struct sockbuf *sb); int sblock(struct sockbuf *sb, int flags); void sbunlock(struct sockbuf *sb); +void sballoc(struct sockbuf *, struct mbuf *); +void sbfree(struct sockbuf *, struct mbuf *); +void sbmtrim(struct sockbuf *, struct mbuf *, int); +int sbready(struct sockbuf *, struct mbuf *, int); +static inline u_int +sbavail(struct sockbuf *sb) +{ + +#if 0 + SOCKBUF_LOCK_ASSERT(sb); +#endif + return (sb->sb_acc); +} + +static inline u_int +sbused(struct sockbuf *sb) +{ + +#if 0 + SOCKBUF_LOCK_ASSERT(sb); +#endif + return (sb->sb_ccc); +} + /* * How much space is there in a socket buffer (so->so_snd or so->so_rcv)? * This is problematical if the fields are unsigned, as the space might - * still be negative (cc > hiwat or mbcnt > mbmax). Should detect - * overflow and return 0. Should use "lmin" but it doesn't exist now. + * still be negative (ccc > hiwat or mbcnt > mbmax). */ -static __inline -long +static inline long sbspace(struct sockbuf *sb) { - long bleft; - long mleft; + long bleft, mleft; +#if 0 + SOCKBUF_LOCK_ASSERT(sb); +#endif + if (sb->sb_flags & SB_STOP) return(0); - bleft = sb->sb_hiwat - sb->sb_cc; + + bleft = sb->sb_hiwat - sb->sb_ccc; mleft = sb->sb_mbmax - sb->sb_mbcnt; - return((bleft < mleft) ? bleft : mleft); -} -/* adjust counters in sb reflecting allocation of m */ -#define sballoc(sb, m) { \ - (sb)->sb_cc += (m)->m_len; \ - if ((m)->m_type != MT_DATA && (m)->m_type != MT_OOBDATA) \ - (sb)->sb_ctl += (m)->m_len; \ - (sb)->sb_mbcnt += MSIZE; \ - (sb)->sb_mcnt += 1; \ - if ((m)->m_flags & M_EXT) { \ - (sb)->sb_mbcnt += (m)->m_ext.ext_size; \ - (sb)->sb_ccnt += 1; \ - } \ + return ((bleft < mleft) ? bleft : mleft); } -/* adjust counters in sb reflecting freeing of m */ -#define sbfree(sb, m) { \ - (sb)->sb_cc -= (m)->m_len; \ - if ((m)->m_type != MT_DATA && (m)->m_type != MT_OOBDATA) \ - (sb)->sb_ctl -= (m)->m_len; \ - (sb)->sb_mbcnt -= MSIZE; \ - (sb)->sb_mcnt -= 1; \ - if ((m)->m_flags & M_EXT) { \ - (sb)->sb_mbcnt -= (m)->m_ext.ext_size; \ - (sb)->sb_ccnt -= 1; \ - } \ - if ((sb)->sb_sndptr == (m)) { \ - (sb)->sb_sndptr = NULL; \ - (sb)->sb_sndptroff = 0; \ - } \ - if ((sb)->sb_sndptroff != 0) \ - (sb)->sb_sndptroff -= (m)->m_len; \ -} - #define SB_EMPTY_FIXUP(sb) do { \ if ((sb)->sb_mb == NULL) { \ (sb)->sb_mbtail = NULL; \ @@ -224,13 +230,15 @@ sbspace(struct sockbuf *sb) #ifdef SOCKBUF_DEBUG void sblastrecordchk(struct sockbuf *, const char *, int); +void sblastmbufchk(struct sockbuf *, const char *, int); +void sbcheck(struct sockbuf *, const char *, int); #define SBLASTRECORDCHK(sb) sblastrecordchk((sb), __FILE__, __LINE__) - -void sblastmbufchk(struct sockbuf *, const char *, int); #define SBLASTMBUFCHK(sb) sblastmbufchk((sb), __FILE__, __LINE__) +#define SBCHECK(sb) sbcheck((sb), __FILE__, __LINE__) #else -#define SBLASTRECORDCHK(sb) /* nothing */ -#define SBLASTMBUFCHK(sb) /* nothing */ +#define SBLASTRECORDCHK(sb) do {} while (0) +#define SBLASTMBUFCHK(sb) do {} while (0) +#define SBCHECK(sb) do {} while (0) #endif /* SOCKBUF_DEBUG */ #endif /* _KERNEL */ Index: sys/sys/protosw.h =================================================================== --- sys/sys/protosw.h (.../head) (revision 266804) +++ sys/sys/protosw.h (.../projects/sendfile) (revision 266807) @@ -209,6 +209,7 @@ struct pr_usrreqs { #define PRUS_OOB 0x1 #define PRUS_EOF 0x2 #define PRUS_MORETOCOME 0x4 +#define PRUS_NOTREADY 0x8 int (*pru_sense)(struct socket *so, struct stat *sb); int (*pru_shutdown)(struct socket *so); int (*pru_flush)(struct socket *so, int direction); Index: sys/sys/sf_buf.h =================================================================== --- sys/sys/sf_buf.h (.../head) (revision 266804) +++ sys/sys/sf_buf.h (.../projects/sendfile) (revision 266807) @@ -52,7 +52,7 @@ struct sfstat { /* sendfile statistics */ #include #include #include -struct mbuf; /* for sf_buf_mext() */ +struct mbuf; /* for sf_mext_free() */ extern counter_u64_t sfstat[sizeof(struct sfstat) / sizeof(uint64_t)]; #define SFSTAT_ADD(name, val) \ @@ -61,6 +61,6 @@ extern counter_u64_t sfstat[sizeof(struct sfstat) #define SFSTAT_INC(name) SFSTAT_ADD(name, 1) #endif /* _KERNEL */ -int sf_buf_mext(struct mbuf *mb, void *addr, void *args); +int sf_mext_free(struct mbuf *mb, void *addr, void *args); #endif /* !_SYS_SF_BUF_H_ */ Index: sys/sys/vnode.h =================================================================== --- sys/sys/vnode.h (.../head) (revision 266804) +++ sys/sys/vnode.h (.../projects/sendfile) (revision 266807) @@ -719,6 +719,7 @@ int vop_stdbmap(struct vop_bmap_args *); int vop_stdfsync(struct vop_fsync_args *); int vop_stdgetwritemount(struct vop_getwritemount_args *); int vop_stdgetpages(struct vop_getpages_args *); +int vop_stdgetpages_async(struct vop_getpages_async_args *); int vop_stdinactive(struct vop_inactive_args *); int vop_stdislocked(struct vop_islocked_args *); int vop_stdkqfilter(struct vop_kqfilter_args *); Index: sys/sys/socketvar.h =================================================================== --- sys/sys/socketvar.h (.../head) (revision 266804) +++ sys/sys/socketvar.h (.../projects/sendfile) (revision 266807) @@ -205,7 +205,7 @@ struct xsocket { /* can we read something from so? */ #define soreadabledata(so) \ - ((so)->so_rcv.sb_cc >= (so)->so_rcv.sb_lowat || \ + (sbavail(&(so)->so_rcv) >= (so)->so_rcv.sb_lowat || \ !TAILQ_EMPTY(&(so)->so_comp) || (so)->so_error) #define soreadable(so) \ (soreadabledata(so) || ((so)->so_rcv.sb_state & SBS_CANTRCVMORE)) Index: sys/sys/mbuf.h =================================================================== --- sys/sys/mbuf.h (.../head) (revision 266804) +++ sys/sys/mbuf.h (.../projects/sendfile) (revision 266807) @@ -922,7 +922,7 @@ struct mbuf *m_copypacket(struct mbuf *, int); void m_copy_pkthdr(struct mbuf *, struct mbuf *); struct mbuf *m_copyup(struct mbuf *, int, int); struct mbuf *m_defrag(struct mbuf *, int); -void m_demote(struct mbuf *, int); +void m_demote(struct mbuf *, int, int); struct mbuf *m_devget(char *, int, int, struct ifnet *, void (*)(char *, caddr_t, u_int)); struct mbuf *m_dup(struct mbuf *, int); Index: sys/vm/vnode_pager.h =================================================================== --- sys/vm/vnode_pager.h (.../head) (revision 266804) +++ sys/vm/vnode_pager.h (.../projects/sendfile) (revision 266807) @@ -41,7 +41,7 @@ #ifdef _KERNEL int vnode_pager_generic_getpages(struct vnode *vp, vm_page_t *m, - int count, int reqpage); + int count, int reqpage, void (*iodone)(void *), void *arg); int vnode_pager_generic_putpages(struct vnode *vp, vm_page_t *m, int count, boolean_t sync, int *rtvals); Index: sys/vm/vm_pager.h =================================================================== --- sys/vm/vm_pager.h (.../head) (revision 266804) +++ sys/vm/vm_pager.h (.../projects/sendfile) (revision 266807) @@ -51,18 +51,21 @@ typedef vm_object_t pgo_alloc_t(void *, vm_ooffset struct ucred *); typedef void pgo_dealloc_t(vm_object_t); typedef int pgo_getpages_t(vm_object_t, vm_page_t *, int, int); +typedef int pgo_getpages_async_t(vm_object_t, vm_page_t *, int, int, + void(*)(void *), void *); typedef void pgo_putpages_t(vm_object_t, vm_page_t *, int, int, int *); typedef boolean_t pgo_haspage_t(vm_object_t, vm_pindex_t, int *, int *); typedef void pgo_pageunswapped_t(vm_page_t); struct pagerops { - pgo_init_t *pgo_init; /* Initialize pager. */ - pgo_alloc_t *pgo_alloc; /* Allocate pager. */ - pgo_dealloc_t *pgo_dealloc; /* Disassociate. */ - pgo_getpages_t *pgo_getpages; /* Get (read) page. */ - pgo_putpages_t *pgo_putpages; /* Put (write) page. */ - pgo_haspage_t *pgo_haspage; /* Does pager have page? */ - pgo_pageunswapped_t *pgo_pageunswapped; + pgo_init_t *pgo_init; /* Initialize pager. */ + pgo_alloc_t *pgo_alloc; /* Allocate pager. */ + pgo_dealloc_t *pgo_dealloc; /* Disassociate. */ + pgo_getpages_t *pgo_getpages; /* Get (read) page. */ + pgo_getpages_async_t *pgo_getpages_async; /* Get page asyncly. */ + pgo_putpages_t *pgo_putpages; /* Put (write) page. */ + pgo_haspage_t *pgo_haspage; /* Query page. */ + pgo_pageunswapped_t *pgo_pageunswapped; }; extern struct pagerops defaultpagerops; @@ -103,6 +106,8 @@ vm_object_t vm_pager_allocate(objtype_t, void *, v void vm_pager_bufferinit(void); void vm_pager_deallocate(vm_object_t); static __inline int vm_pager_get_pages(vm_object_t, vm_page_t *, int, int); +static __inline int vm_pager_get_pages_async(vm_object_t, vm_page_t *, int, + int, void(*)(void *), void *); static __inline boolean_t vm_pager_has_page(vm_object_t, vm_pindex_t, int *, int *); void vm_pager_init(void); vm_object_t vm_pager_object_lookup(struct pagerlst *, void *); @@ -131,6 +136,27 @@ vm_pager_get_pages( return (r); } +static __inline int +vm_pager_get_pages_async(vm_object_t object, vm_page_t *m, int count, + int reqpage, void (*iodone)(void *), void *arg) +{ + int r; + + VM_OBJECT_ASSERT_WLOCKED(object); + + if (*pagertab[object->type]->pgo_getpages_async == NULL) { + /* Emulate async operation. */ + r = vm_pager_get_pages(object, m, count, reqpage); + VM_OBJECT_WUNLOCK(object); + (iodone)(arg); + VM_OBJECT_WLOCK(object); + } else + r = (*pagertab[object->type]->pgo_getpages_async)(object, m, + count, reqpage, iodone, arg); + + return (r); +} + static __inline void vm_pager_put_pages( vm_object_t object, Index: sys/vm/vm_page.c =================================================================== --- sys/vm/vm_page.c (.../head) (revision 266804) +++ sys/vm/vm_page.c (.../projects/sendfile) (revision 266807) @@ -2689,6 +2689,8 @@ retrylookup: sleep = (allocflags & VM_ALLOC_IGN_SBUSY) != 0 ? vm_page_xbusied(m) : vm_page_busied(m); if (sleep) { + if (allocflags & VM_ALLOC_NOWAIT) + return (NULL); /* * Reference the page before unlocking and * sleeping so that the page daemon is less @@ -2716,6 +2718,8 @@ retrylookup: } m = vm_page_alloc(object, pindex, allocflags & ~VM_ALLOC_IGN_SBUSY); if (m == NULL) { + if (allocflags & VM_ALLOC_NOWAIT) + return (NULL); VM_OBJECT_WUNLOCK(object); VM_WAIT; VM_OBJECT_WLOCK(object); Index: sys/vm/vm_page.h =================================================================== --- sys/vm/vm_page.h (.../head) (revision 266804) +++ sys/vm/vm_page.h (.../projects/sendfile) (revision 266807) @@ -390,6 +390,7 @@ vm_page_t PHYS_TO_VM_PAGE(vm_paddr_t pa); #define VM_ALLOC_IGN_SBUSY 0x1000 /* vm_page_grab() only */ #define VM_ALLOC_NODUMP 0x2000 /* don't include in dump */ #define VM_ALLOC_SBUSY 0x4000 /* Shared busy the page */ +#define VM_ALLOC_NOWAIT 0x8000 /* Return NULL instead of sleeping */ #define VM_ALLOC_COUNT_SHIFT 16 #define VM_ALLOC_COUNT(count) ((count) << VM_ALLOC_COUNT_SHIFT) Index: sys/vm/vnode_pager.c =================================================================== --- sys/vm/vnode_pager.c (.../head) (revision 266804) +++ sys/vm/vnode_pager.c (.../projects/sendfile) (revision 266807) @@ -83,6 +83,8 @@ static int vnode_pager_input_smlfs(vm_object_t obj static int vnode_pager_input_old(vm_object_t object, vm_page_t m); static void vnode_pager_dealloc(vm_object_t); static int vnode_pager_getpages(vm_object_t, vm_page_t *, int, int); +static int vnode_pager_getpages_async(vm_object_t, vm_page_t *, int, int, + void(*)(void *), void *); static void vnode_pager_putpages(vm_object_t, vm_page_t *, int, boolean_t, int *); static boolean_t vnode_pager_haspage(vm_object_t, vm_pindex_t, int *, int *); static vm_object_t vnode_pager_alloc(void *, vm_ooffset_t, vm_prot_t, @@ -92,6 +94,7 @@ struct pagerops vnodepagerops = { .pgo_alloc = vnode_pager_alloc, .pgo_dealloc = vnode_pager_dealloc, .pgo_getpages = vnode_pager_getpages, + .pgo_getpages_async = vnode_pager_getpages_async, .pgo_putpages = vnode_pager_putpages, .pgo_haspage = vnode_pager_haspage, }; @@ -664,6 +667,40 @@ vnode_pager_getpages(vm_object_t object, vm_page_t return rtval; } +static int +vnode_pager_getpages_async(vm_object_t object, vm_page_t *m, int count, + int reqpage, void (*iodone)(void *), void *arg) +{ + int rtval; + struct vnode *vp; + int bytes = count * PAGE_SIZE; + + vp = object->handle; + VM_OBJECT_WUNLOCK(object); + rtval = VOP_GETPAGES_ASYNC(vp, m, bytes, reqpage, 0, iodone, arg); + KASSERT(rtval != EOPNOTSUPP, + ("vnode_pager: FS getpages_async not implemented\n")); + VM_OBJECT_WLOCK(object); + return rtval; +} + +struct getpages_softc { + vm_page_t *m; + struct buf *bp; + vm_object_t object; + vm_offset_t kva; + off_t foff; + int size; + int count; + int unmapped; + int reqpage; + void (*iodone)(void *); + void *arg; +}; + +int vnode_pager_generic_getpages_done(struct getpages_softc *); +void vnode_pager_generic_getpages_done_async(struct buf *); + /* * This is now called from local media FS's to operate against their * own vnodes if they fail to implement VOP_GETPAGES. @@ -670,11 +707,11 @@ vnode_pager_getpages(vm_object_t object, vm_page_t */ int vnode_pager_generic_getpages(struct vnode *vp, vm_page_t *m, int bytecount, - int reqpage) + int reqpage, void (*iodone)(void *), void *arg) { vm_object_t object; vm_offset_t kva; - off_t foff, tfoff, nextoff; + off_t foff; int i, j, size, bsize, first; daddr_t firstaddr, reqblock; struct bufobj *bo; @@ -684,6 +721,7 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ struct mount *mp; int count; int error; + int unmapped; object = vp->v_object; count = bytecount / PAGE_SIZE; @@ -891,8 +929,8 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ * requires mapped buffers. */ mp = vp->v_mount; - if (mp != NULL && (mp->mnt_kern_flag & MNTK_UNMAPPED_BUFS) != 0 && - unmapped_buf_allowed) { + unmapped = (mp != NULL && (mp->mnt_kern_flag & MNTK_UNMAPPED_BUFS)); + if (unmapped && unmapped_buf_allowed) { bp->b_data = unmapped_buf; bp->b_kvabase = unmapped_buf; bp->b_offset = 0; @@ -905,7 +943,6 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ /* build a minimal buffer header */ bp->b_iocmd = BIO_READ; - bp->b_iodone = bdone; KASSERT(bp->b_rcred == NOCRED, ("leaking read ucred")); KASSERT(bp->b_wcred == NOCRED, ("leaking write ucred")); bp->b_rcred = crhold(curthread->td_ucred); @@ -923,10 +960,88 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ /* do the input */ bp->b_iooffset = dbtob(bp->b_blkno); - bstrategy(bp); - bwait(bp, PVM, "vnread"); + if (iodone) { /* async */ + struct getpages_softc *sc; + sc = malloc(sizeof(*sc), M_TEMP, M_WAITOK); + + sc->m = m; + sc->bp = bp; + sc->object = object; + sc->foff = foff; + sc->size = size; + sc->count = count; + sc->unmapped = unmapped; + sc->reqpage = reqpage; + sc->kva = kva; + + sc->iodone = iodone; + sc->arg = arg; + + bp->b_iodone = vnode_pager_generic_getpages_done_async; + bp->b_caller1 = sc; + BUF_KERNPROC(bp); + bstrategy(bp); + /* Good bye! */ + } else { + struct getpages_softc sc; + + sc.m = m; + sc.bp = bp; + sc.object = object; + sc.foff = foff; + sc.size = size; + sc.count = count; + sc.unmapped = unmapped; + sc.reqpage = reqpage; + sc.kva = kva; + + bp->b_iodone = bdone; + bstrategy(bp); + bwait(bp, PVM, "vnread"); + error = vnode_pager_generic_getpages_done(&sc); + } + + return (error ? VM_PAGER_ERROR : VM_PAGER_OK); +} + +void +vnode_pager_generic_getpages_done_async(struct buf *bp) +{ + struct getpages_softc *sc = bp->b_caller1; + int error; + + error = vnode_pager_generic_getpages_done(sc); + + vm_page_xunbusy(sc->m[sc->reqpage]); + + sc->iodone(sc->arg); + + free(sc, M_TEMP); +} + +int +vnode_pager_generic_getpages_done(struct getpages_softc *sc) +{ + vm_object_t object; + vm_offset_t kva; + vm_page_t *m; + struct buf *bp; + off_t foff, tfoff, nextoff; + int i, size, count, unmapped, reqpage; + int error = 0; + + m = sc->m; + bp = sc->bp; + object = sc->object; + foff = sc->foff; + size = sc->size; + count = sc->count; + unmapped = sc->unmapped; + reqpage = sc->reqpage; + kva = sc->kva; + if ((bp->b_ioflags & BIO_ERROR) != 0) error = EIO; @@ -939,7 +1054,7 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ } if ((bp->b_flags & B_UNMAPPED) == 0) pmap_qremove(kva, count); - if (mp != NULL && (mp->mnt_kern_flag & MNTK_UNMAPPED_BUFS) != 0) { + if (unmapped) { bp->b_data = (caddr_t)kva; bp->b_kvabase = (caddr_t)kva; bp->b_flags &= ~B_UNMAPPED; @@ -995,7 +1110,8 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_ if (error) { printf("vnode_pager_getpages: I/O read error\n"); } - return (error ? VM_PAGER_ERROR : VM_PAGER_OK); + + return (error); } /* Index: sys/rpc/clnt_vc.c =================================================================== --- sys/rpc/clnt_vc.c (.../head) (revision 266804) +++ sys/rpc/clnt_vc.c (.../projects/sendfile) (revision 266807) @@ -860,7 +860,7 @@ clnt_vc_soupcall(struct socket *so, void *arg, int * error condition */ do_read = FALSE; - if (so->so_rcv.sb_cc >= sizeof(uint32_t) + if (sbavail(&so->so_rcv) >= sizeof(uint32_t) || (so->so_rcv.sb_state & SBS_CANTRCVMORE) || so->so_error) do_read = TRUE; @@ -913,7 +913,7 @@ clnt_vc_soupcall(struct socket *so, void *arg, int * buffered. */ do_read = FALSE; - if (so->so_rcv.sb_cc >= ct->ct_record_resid + if (sbavail(&so->so_rcv) >= ct->ct_record_resid || (so->so_rcv.sb_state & SBS_CANTRCVMORE) || so->so_error) do_read = TRUE; Index: sys/rpc/svc_vc.c =================================================================== --- sys/rpc/svc_vc.c (.../head) (revision 266804) +++ sys/rpc/svc_vc.c (.../projects/sendfile) (revision 266807) @@ -546,7 +546,7 @@ svc_vc_ack(SVCXPRT *xprt, uint32_t *ack) { *ack = atomic_load_acq_32(&xprt->xp_snt_cnt); - *ack -= xprt->xp_socket->so_snd.sb_cc; + *ack -= sbused(&xprt->xp_socket->so_snd); return (TRUE); } Index: sys/ufs/ffs/ffs_vnops.c =================================================================== --- sys/ufs/ffs/ffs_vnops.c (.../head) (revision 266804) +++ sys/ufs/ffs/ffs_vnops.c (.../projects/sendfile) (revision 266807) @@ -105,6 +105,7 @@ extern int ffs_rawread(struct vnode *vp, struct ui static vop_fsync_t ffs_fsync; static vop_lock1_t ffs_lock; static vop_getpages_t ffs_getpages; +static vop_getpages_async_t ffs_getpages_async; static vop_read_t ffs_read; static vop_write_t ffs_write; static int ffs_extread(struct vnode *vp, struct uio *uio, int ioflag); @@ -125,6 +126,7 @@ struct vop_vector ffs_vnodeops1 = { .vop_default = &ufs_vnodeops, .vop_fsync = ffs_fsync, .vop_getpages = ffs_getpages, + .vop_getpages_async = ffs_getpages_async, .vop_lock1 = ffs_lock, .vop_read = ffs_read, .vop_reallocblks = ffs_reallocblks, @@ -847,18 +849,16 @@ ffs_write(ap) } /* - * get page routine + * Get page routines. */ static int -ffs_getpages(ap) - struct vop_getpages_args *ap; +ffs_getpages_checkvalid(vm_page_t *m, int count, int reqpage) { - int i; vm_page_t mreq; int pcount; - pcount = round_page(ap->a_count) / PAGE_SIZE; - mreq = ap->a_m[ap->a_reqpage]; + pcount = round_page(count) / PAGE_SIZE; + mreq = m[reqpage]; /* * if ANY DEV_BSIZE blocks are valid on a large filesystem block, @@ -870,24 +870,48 @@ static int if (mreq->valid) { if (mreq->valid != VM_PAGE_BITS_ALL) vm_page_zero_invalid(mreq, TRUE); - for (i = 0; i < pcount; i++) { - if (i != ap->a_reqpage) { - vm_page_lock(ap->a_m[i]); - vm_page_free(ap->a_m[i]); - vm_page_unlock(ap->a_m[i]); + for (int i = 0; i < pcount; i++) { + if (i != reqpage) { + vm_page_lock(m[i]); + vm_page_free(m[i]); + vm_page_unlock(m[i]); } } VM_OBJECT_WUNLOCK(mreq->object); - return VM_PAGER_OK; + return (VM_PAGER_OK); } VM_OBJECT_WUNLOCK(mreq->object); - return vnode_pager_generic_getpages(ap->a_vp, ap->a_m, - ap->a_count, - ap->a_reqpage); + return (-1); } +static int +ffs_getpages(struct vop_getpages_args *ap) +{ + int rv; + rv = ffs_getpages_checkvalid(ap->a_m, ap->a_count, ap->a_reqpage); + if (rv == VM_PAGER_OK) + return (rv); + + return (vnode_pager_generic_getpages(ap->a_vp, ap->a_m, ap->a_count, + ap->a_reqpage, NULL, NULL)); +} + +static int +ffs_getpages_async(struct vop_getpages_async_args *ap) +{ + int rv; + + rv = ffs_getpages_checkvalid(ap->a_m, ap->a_count, ap->a_reqpage); + if (rv == VM_PAGER_OK) { + (ap->a_vop_getpages_iodone)(ap->a_arg); + return (rv); + } + return (vnode_pager_generic_getpages(ap->a_vp, ap->a_m, ap->a_count, + ap->a_reqpage, ap->a_vop_getpages_iodone, ap->a_arg)); +} + /* * Extended attribute area reading. */ Index: sys/tools/vnode_if.awk =================================================================== --- sys/tools/vnode_if.awk (.../head) (revision 266804) +++ sys/tools/vnode_if.awk (.../projects/sendfile) (revision 266807) @@ -254,16 +254,26 @@ while ((getline < srcfile) > 0) { if (sub(/;$/, "") < 1) die("Missing end-of-line ; in \"%s\".", $0); - # pick off variable name - if ((argp = match($0, /[A-Za-z0-9_]+$/)) < 1) - die("Missing var name \"a_foo\" in \"%s\".", $0); - args[numargs] = substr($0, argp); - $0 = substr($0, 1, argp - 1); - - # what is left must be type - # remove trailing space (if any) - sub(/ $/, ""); - types[numargs] = $0; + # pick off argument name + if ((argp = match($0, /[A-Za-z0-9_]+$/)) > 0) { + args[numargs] = substr($0, argp); + $0 = substr($0, 1, argp - 1); + sub(/ $/, ""); + delete fargs[numargs]; + types[numargs] = $0; + } else { # try to parse a function pointer argument + if ((argp = match($0, + /\(\*[A-Za-z0-9_]+\)\([A-Za-z0-9_*, ]+\)$/)) < 1) + die("Missing var name \"a_foo\" in \"%s\".", + $0); + args[numargs] = substr($0, argp + 2); + sub(/\).+/, "", args[numargs]); + fargs[numargs] = substr($0, argp); + sub(/^\([^)]+\)/, "", fargs[numargs]); + $0 = substr($0, 1, argp - 1); + sub(/ $/, ""); + types[numargs] = $0; + } } if (numargs > 4) ctrargs = 4; @@ -286,8 +296,13 @@ while ((getline < srcfile) > 0) { if (hfile) { # Print out the vop_F_args structure. printh("struct "name"_args {\n\tstruct vop_generic_args a_gen;"); - for (i = 0; i < numargs; ++i) - printh("\t" t_spc(types[i]) "a_" args[i] ";"); + for (i = 0; i < numargs; ++i) { + if (fargs[i]) { + printh("\t" t_spc(types[i]) "(*a_" args[i] \ + ")" fargs[i] ";"); + } else + printh("\t" t_spc(types[i]) "a_" args[i] ";"); + } printh("};"); printh(""); @@ -301,8 +316,14 @@ while ((getline < srcfile) > 0) { printh(""); printh("static __inline int " uname "("); for (i = 0; i < numargs; ++i) { - printh("\t" t_spc(types[i]) args[i] \ - (i < numargs - 1 ? "," : ")")); + if (fargs[i]) { + printh("\t" t_spc(types[i]) "(*" args[i] \ + ")" fargs[i] \ + (i < numargs - 1 ? "," : ")")); + } else { + printh("\t" t_spc(types[i]) args[i] \ + (i < numargs - 1 ? "," : ")")); + } } printh("{"); printh("\tstruct " name "_args a;"); Index: sys/netinet/tcp_reass.c =================================================================== --- sys/netinet/tcp_reass.c (.../head) (revision 266804) +++ sys/netinet/tcp_reass.c (.../projects/sendfile) (revision 266807) @@ -248,7 +248,7 @@ present: m_freem(mq); else { mq->m_nextpkt = NULL; - sbappendstream_locked(&so->so_rcv, mq); + sbappendstream_locked(&so->so_rcv, mq, 0); wakeup = 1; } } Index: sys/netinet/accf_http.c =================================================================== --- sys/netinet/accf_http.c (.../head) (revision 266804) +++ sys/netinet/accf_http.c (.../projects/sendfile) (revision 266807) @@ -92,7 +92,7 @@ sbfull(struct sockbuf *sb) "mbcnt(%ld) >= mbmax(%ld): %d", sb->sb_cc, sb->sb_hiwat, sb->sb_cc >= sb->sb_hiwat, sb->sb_mbcnt, sb->sb_mbmax, sb->sb_mbcnt >= sb->sb_mbmax); - return (sb->sb_cc >= sb->sb_hiwat || sb->sb_mbcnt >= sb->sb_mbmax); + return (sbused(sb) >= sb->sb_hiwat || sb->sb_mbcnt >= sb->sb_mbmax); } /* @@ -162,13 +162,14 @@ static int sohashttpget(struct socket *so, void *arg, int waitflag) { - if ((so->so_rcv.sb_state & SBS_CANTRCVMORE) == 0 && !sbfull(&so->so_rcv)) { + if ((so->so_rcv.sb_state & SBS_CANTRCVMORE) == 0 && + !sbfull(&so->so_rcv)) { struct mbuf *m; char *cmp; int cmplen, cc; m = so->so_rcv.sb_mb; - cc = so->so_rcv.sb_cc - 1; + cc = sbavail(&so->so_rcv) - 1; if (cc < 1) return (SU_OK); switch (*mtod(m, char *)) { @@ -215,7 +216,7 @@ soparsehttpvers(struct socket *so, void *arg, int goto fallout; m = so->so_rcv.sb_mb; - cc = so->so_rcv.sb_cc; + cc = sbavail(&so->so_rcv); inspaces = spaces = 0; for (m = so->so_rcv.sb_mb; m; m = n) { n = m->m_nextpkt; @@ -304,7 +305,7 @@ soishttpconnected(struct socket *so, void *arg, in * have NCHRS left */ copied = 0; - ccleft = so->so_rcv.sb_cc; + ccleft = sbavail(&so->so_rcv); if (ccleft < NCHRS) goto readmore; a = b = c = '\0'; Index: sys/netinet/sctp_os_bsd.h =================================================================== --- sys/netinet/sctp_os_bsd.h (.../head) (revision 266804) +++ sys/netinet/sctp_os_bsd.h (.../projects/sendfile) (revision 266807) @@ -405,7 +405,7 @@ typedef struct callout sctp_os_timer_t; #define SCTP_SOWAKEUP(so) wakeup(&(so)->so_timeo) /* clear the socket buffer state */ #define SCTP_SB_CLEAR(sb) \ - (sb).sb_cc = 0; \ + (sb).sb_ccc = 0; \ (sb).sb_mb = NULL; \ (sb).sb_mbcnt = 0; Index: sys/netinet/tcp_output.c =================================================================== --- sys/netinet/tcp_output.c (.../head) (revision 266804) +++ sys/netinet/tcp_output.c (.../projects/sendfile) (revision 266807) @@ -322,7 +322,7 @@ after_sack_rexmit: * to send then the probe will be the FIN * itself. */ - if (off < so->so_snd.sb_cc) + if (off < sbavail(&so->so_snd)) flags &= ~TH_FIN; sendwin = 1; } else { @@ -348,7 +348,8 @@ after_sack_rexmit: */ if (sack_rxmit == 0) { if (sack_bytes_rxmt == 0) - len = ((long)ulmin(so->so_snd.sb_cc, sendwin) - off); + len = ((long)ulmin(sbavail(&so->so_snd), sendwin) - + off); else { long cwin; @@ -357,8 +358,8 @@ after_sack_rexmit: * sending new data, having retransmitted all the * data possible in the scoreboard. */ - len = ((long)ulmin(so->so_snd.sb_cc, tp->snd_wnd) - - off); + len = ((long)ulmin(sbavail(&so->so_snd), tp->snd_wnd) - + off); /* * Don't remove this (len > 0) check ! * We explicitly check for len > 0 here (although it @@ -457,12 +458,15 @@ after_sack_rexmit: * TODO: Shrink send buffer during idle periods together * with congestion window. Requires another timer. Has to * wait for upcoming tcp timer rewrite. + * + * XXXGL: should there be used sbused() or sbavail()? */ if (V_tcp_do_autosndbuf && so->so_snd.sb_flags & SB_AUTOSIZE) { if ((tp->snd_wnd / 4 * 5) >= so->so_snd.sb_hiwat && - so->so_snd.sb_cc >= (so->so_snd.sb_hiwat / 8 * 7) && - so->so_snd.sb_cc < V_tcp_autosndbuf_max && - sendwin >= (so->so_snd.sb_cc - (tp->snd_nxt - tp->snd_una))) { + sbused(&so->so_snd) >= (so->so_snd.sb_hiwat / 8 * 7) && + sbused(&so->so_snd) < V_tcp_autosndbuf_max && + sendwin >= (sbused(&so->so_snd) - + (tp->snd_nxt - tp->snd_una))) { if (!sbreserve_locked(&so->so_snd, min(so->so_snd.sb_hiwat + V_tcp_autosndbuf_inc, V_tcp_autosndbuf_max), so, curthread)) @@ -499,10 +503,11 @@ after_sack_rexmit: tso = 1; if (sack_rxmit) { - if (SEQ_LT(p->rxmit + len, tp->snd_una + so->so_snd.sb_cc)) + if (SEQ_LT(p->rxmit + len, tp->snd_una + sbavail(&so->so_snd))) flags &= ~TH_FIN; } else { - if (SEQ_LT(tp->snd_nxt + len, tp->snd_una + so->so_snd.sb_cc)) + if (SEQ_LT(tp->snd_nxt + len, tp->snd_una + + sbavail(&so->so_snd))) flags &= ~TH_FIN; } @@ -532,7 +537,7 @@ after_sack_rexmit: */ if (!(tp->t_flags & TF_MORETOCOME) && /* normal case */ (idle || (tp->t_flags & TF_NODELAY)) && - len + off >= so->so_snd.sb_cc && + len + off >= sbavail(&so->so_snd) && (tp->t_flags & TF_NOPUSH) == 0) { goto send; } @@ -660,7 +665,7 @@ dontupdate: * if window is nonzero, transmit what we can, * otherwise force out a byte. */ - if (so->so_snd.sb_cc && !tcp_timer_active(tp, TT_REXMT) && + if (sbavail(&so->so_snd) && !tcp_timer_active(tp, TT_REXMT) && !tcp_timer_active(tp, TT_PERSIST)) { tp->t_rxtshift = 0; tcp_setpersist(tp); @@ -786,7 +791,7 @@ send: * fractional unless the send sockbuf can * be emptied. */ - if (sendalot && off + len < so->so_snd.sb_cc) { + if (sendalot && off + len < sbavail(&so->so_snd)) { len -= len % (tp->t_maxopd - optlen); sendalot = 1; } @@ -889,7 +894,7 @@ send: * give data to the user when a buffer fills or * a PUSH comes in.) */ - if (off + len == so->so_snd.sb_cc) + if (off + len == sbavail(&so->so_snd)) flags |= TH_PUSH; SOCKBUF_UNLOCK(&so->so_snd); } else { Index: sys/netinet/siftr.c =================================================================== --- sys/netinet/siftr.c (.../head) (revision 266804) +++ sys/netinet/siftr.c (.../projects/sendfile) (revision 266807) @@ -781,9 +781,9 @@ siftr_siftdata(struct pkt_node *pn, struct inpcb * pn->flags = tp->t_flags; pn->rxt_length = tp->t_rxtcur; pn->snd_buf_hiwater = inp->inp_socket->so_snd.sb_hiwat; - pn->snd_buf_cc = inp->inp_socket->so_snd.sb_cc; + pn->snd_buf_cc = sbused(&inp->inp_socket->so_snd); pn->rcv_buf_hiwater = inp->inp_socket->so_rcv.sb_hiwat; - pn->rcv_buf_cc = inp->inp_socket->so_rcv.sb_cc; + pn->rcv_buf_cc = sbused(&inp->inp_socket->so_rcv); pn->sent_inflight_bytes = tp->snd_max - tp->snd_una; pn->t_segqlen = tp->t_segqlen; Index: sys/netinet/sctp_indata.c =================================================================== --- sys/netinet/sctp_indata.c (.../head) (revision 266804) +++ sys/netinet/sctp_indata.c (.../projects/sendfile) (revision 266807) @@ -70,7 +70,7 @@ sctp_calc_rwnd(struct sctp_tcb *stcb, struct sctp_ /* * This is really set wrong with respect to a 1-2-m socket. Since - * the sb_cc is the count that everyone as put up. When we re-write + * the sb_ccc is the count that everyone as put up. When we re-write * sctp_soreceive then we will fix this so that ONLY this * associations data is taken into account. */ @@ -77,7 +77,7 @@ sctp_calc_rwnd(struct sctp_tcb *stcb, struct sctp_ if (stcb->sctp_socket == NULL) return (calc); - if (stcb->asoc.sb_cc == 0 && + if (stcb->asoc.sb_ccc == 0 && asoc->size_on_reasm_queue == 0 && asoc->size_on_all_streams == 0) { /* Full rwnd granted */ @@ -1358,7 +1358,7 @@ sctp_process_a_data_chunk(struct sctp_tcb *stcb, s * When we have NO room in the rwnd we check to make sure * the reader is doing its job... */ - if (stcb->sctp_socket->so_rcv.sb_cc) { + if (stcb->sctp_socket->so_rcv.sb_ccc) { /* some to read, wake-up */ #if defined(__APPLE__) || defined(SCTP_SO_LOCK_TESTING) struct socket *so; Index: sys/netinet/sctp_pcb.c =================================================================== --- sys/netinet/sctp_pcb.c (.../head) (revision 266804) +++ sys/netinet/sctp_pcb.c (.../projects/sendfile) (revision 266807) @@ -3328,7 +3328,7 @@ sctp_inpcb_free(struct sctp_inpcb *inp, int immedi if ((asoc->asoc.size_on_reasm_queue > 0) || (asoc->asoc.control_pdapi) || (asoc->asoc.size_on_all_streams > 0) || - (so && (so->so_rcv.sb_cc > 0))) { + (so && (so->so_rcv.sb_ccc > 0))) { /* Left with Data unread */ struct mbuf *op_err; @@ -3556,7 +3556,7 @@ sctp_inpcb_free(struct sctp_inpcb *inp, int immedi TAILQ_REMOVE(&inp->read_queue, sq, next); sctp_free_remote_addr(sq->whoFrom); if (so) - so->so_rcv.sb_cc -= sq->length; + so->so_rcv.sb_ccc -= sq->length; if (sq->data) { sctp_m_freem(sq->data); sq->data = NULL; @@ -4775,7 +4775,7 @@ sctp_free_assoc(struct sctp_inpcb *inp, struct sct inp->sctp_flags |= SCTP_PCB_FLAGS_WAS_CONNECTED; if (so) { SOCK_LOCK(so); - if (so->so_rcv.sb_cc == 0) { + if (so->so_rcv.sb_ccc == 0) { so->so_state &= ~(SS_ISCONNECTING | SS_ISDISCONNECTING | SS_ISCONFIRMING | Index: sys/netinet/sctp_pcb.h =================================================================== --- sys/netinet/sctp_pcb.h (.../head) (revision 266804) +++ sys/netinet/sctp_pcb.h (.../projects/sendfile) (revision 266807) @@ -369,7 +369,7 @@ struct sctp_inpcb { } ip_inp; - /* Socket buffer lock protects read_queue and of course sb_cc */ + /* Socket buffer lock protects read_queue and of course sb_ccc */ struct sctp_readhead read_queue; LIST_ENTRY(sctp_inpcb) sctp_list; /* lists all endpoints */ Index: sys/netinet/sctp_usrreq.c =================================================================== --- sys/netinet/sctp_usrreq.c (.../head) (revision 266804) +++ sys/netinet/sctp_usrreq.c (.../projects/sendfile) (revision 266807) @@ -586,7 +586,7 @@ sctp_must_try_again: if (((flags & SCTP_PCB_FLAGS_SOCKET_GONE) == 0) && (atomic_cmpset_int(&inp->sctp_flags, flags, (flags | SCTP_PCB_FLAGS_SOCKET_GONE | SCTP_PCB_FLAGS_CLOSE_IP)))) { if (((so->so_options & SO_LINGER) && (so->so_linger == 0)) || - (so->so_rcv.sb_cc > 0)) { + (so->so_rcv.sb_ccc > 0)) { #ifdef SCTP_LOG_CLOSING sctp_log_closing(inp, NULL, 13); #endif @@ -751,7 +751,7 @@ sctp_disconnect(struct socket *so) } if (((so->so_options & SO_LINGER) && (so->so_linger == 0)) || - (so->so_rcv.sb_cc > 0)) { + (so->so_rcv.sb_ccc > 0)) { if (SCTP_GET_STATE(asoc) != SCTP_STATE_COOKIE_WAIT) { /* Left with Data unread */ @@ -916,7 +916,7 @@ sctp_flush(struct socket *so, int how) inp->sctp_flags |= SCTP_PCB_FLAGS_SOCKET_CANT_READ; SCTP_INP_READ_UNLOCK(inp); SCTP_INP_WUNLOCK(inp); - so->so_rcv.sb_cc = 0; + so->so_rcv.sb_ccc = 0; so->so_rcv.sb_mbcnt = 0; so->so_rcv.sb_mb = NULL; } @@ -925,7 +925,7 @@ sctp_flush(struct socket *so, int how) * First make sure the sb will be happy, we don't use these * except maybe the count */ - so->so_snd.sb_cc = 0; + so->so_snd.sb_ccc = 0; so->so_snd.sb_mbcnt = 0; so->so_snd.sb_mb = NULL; Index: sys/netinet/sctp_structs.h =================================================================== --- sys/netinet/sctp_structs.h (.../head) (revision 266804) +++ sys/netinet/sctp_structs.h (.../projects/sendfile) (revision 266807) @@ -982,7 +982,7 @@ struct sctp_association { uint32_t total_output_queue_size; - uint32_t sb_cc; /* shadow of sb_cc */ + uint32_t sb_ccc; /* shadow of sb_ccc */ uint32_t sb_send_resv; /* amount reserved on a send */ uint32_t my_rwnd_control_len; /* shadow of sb_mbcnt used for rwnd * control */ Index: sys/netinet/tcp_input.c =================================================================== --- sys/netinet/tcp_input.c (.../head) (revision 266804) +++ sys/netinet/tcp_input.c (.../projects/sendfile) (revision 266807) @@ -1729,7 +1729,7 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, tcp_timer_activate(tp, TT_REXMT, tp->t_rxtcur); sowwakeup(so); - if (so->so_snd.sb_cc) + if (sbavail(&so->so_snd)) (void) tcp_output(tp); goto check_delack; } @@ -1837,7 +1837,7 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, newsize, so, NULL)) so->so_rcv.sb_flags &= ~SB_AUTOSIZE; m_adj(m, drop_hdrlen); /* delayed header drop */ - sbappendstream_locked(&so->so_rcv, m); + sbappendstream_locked(&so->so_rcv, m, 0); } /* NB: sorwakeup_locked() does an implicit unlock. */ sorwakeup_locked(so); @@ -2541,7 +2541,7 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, * Otherwise we would send pure ACKs. */ SOCKBUF_LOCK(&so->so_snd); - avail = so->so_snd.sb_cc - + avail = sbavail(&so->so_snd) - (tp->snd_nxt - tp->snd_una); SOCKBUF_UNLOCK(&so->so_snd); if (avail > 0) @@ -2676,10 +2676,10 @@ process_ACK: cc_ack_received(tp, th, CC_ACK); SOCKBUF_LOCK(&so->so_snd); - if (acked > so->so_snd.sb_cc) { - tp->snd_wnd -= so->so_snd.sb_cc; + if (acked > sbavail(&so->so_snd)) { + tp->snd_wnd -= sbavail(&so->so_snd); mfree = sbcut_locked(&so->so_snd, - (int)so->so_snd.sb_cc); + (int)sbavail(&so->so_snd)); ourfinisacked = 1; } else { mfree = sbcut_locked(&so->so_snd, acked); @@ -2805,7 +2805,7 @@ step6: * actually wanting to send this much urgent data. */ SOCKBUF_LOCK(&so->so_rcv); - if (th->th_urp + so->so_rcv.sb_cc > sb_max) { + if (th->th_urp + sbavail(&so->so_rcv) > sb_max) { th->th_urp = 0; /* XXX */ thflags &= ~TH_URG; /* XXX */ SOCKBUF_UNLOCK(&so->so_rcv); /* XXX */ @@ -2827,7 +2827,7 @@ step6: */ if (SEQ_GT(th->th_seq+th->th_urp, tp->rcv_up)) { tp->rcv_up = th->th_seq + th->th_urp; - so->so_oobmark = so->so_rcv.sb_cc + + so->so_oobmark = sbavail(&so->so_rcv) + (tp->rcv_up - tp->rcv_nxt) - 1; if (so->so_oobmark == 0) so->so_rcv.sb_state |= SBS_RCVATMARK; @@ -2897,7 +2897,7 @@ dodata: /* XXX */ if (so->so_rcv.sb_state & SBS_CANTRCVMORE) m_freem(m); else - sbappendstream_locked(&so->so_rcv, m); + sbappendstream_locked(&so->so_rcv, m, 0); /* NB: sorwakeup_locked() does an implicit unlock. */ sorwakeup_locked(so); } else { Index: sys/netinet/sctp_input.c =================================================================== --- sys/netinet/sctp_input.c (.../head) (revision 266804) +++ sys/netinet/sctp_input.c (.../projects/sendfile) (revision 266807) @@ -1042,7 +1042,7 @@ sctp_handle_shutdown_ack(struct sctp_shutdown_ack_ if (stcb->sctp_socket) { if ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL)) { - stcb->sctp_socket->so_snd.sb_cc = 0; + stcb->sctp_socket->so_snd.sb_ccc = 0; } sctp_ulp_notify(SCTP_NOTIFY_ASSOC_DOWN, stcb, 0, NULL, SCTP_SO_NOT_LOCKED); } Index: sys/netinet/sctp_var.h =================================================================== --- sys/netinet/sctp_var.h (.../head) (revision 266804) +++ sys/netinet/sctp_var.h (.../projects/sendfile) (revision 266807) @@ -82,9 +82,9 @@ extern struct pr_usrreqs sctp_usrreqs; #define sctp_maxspace(sb) (max((sb)->sb_hiwat,SCTP_MINIMAL_RWND)) -#define sctp_sbspace(asoc, sb) ((long) ((sctp_maxspace(sb) > (asoc)->sb_cc) ? (sctp_maxspace(sb) - (asoc)->sb_cc) : 0)) +#define sctp_sbspace(asoc, sb) ((long) ((sctp_maxspace(sb) > (asoc)->sb_ccc) ? (sctp_maxspace(sb) - (asoc)->sb_ccc) : 0)) -#define sctp_sbspace_failedmsgs(sb) ((long) ((sctp_maxspace(sb) > (sb)->sb_cc) ? (sctp_maxspace(sb) - (sb)->sb_cc) : 0)) +#define sctp_sbspace_failedmsgs(sb) ((long) ((sctp_maxspace(sb) > (sb)->sb_ccc) ? (sctp_maxspace(sb) - (sb)->sb_ccc) : 0)) #define sctp_sbspace_sub(a,b) ((a > b) ? (a - b) : 0) @@ -195,10 +195,10 @@ extern struct pr_usrreqs sctp_usrreqs; } #define sctp_sbfree(ctl, stcb, sb, m) { \ - SCTP_SAVE_ATOMIC_DECREMENT(&(sb)->sb_cc, SCTP_BUF_LEN((m))); \ + SCTP_SAVE_ATOMIC_DECREMENT(&(sb)->sb_ccc, SCTP_BUF_LEN((m))); \ SCTP_SAVE_ATOMIC_DECREMENT(&(sb)->sb_mbcnt, MSIZE); \ if (((ctl)->do_not_ref_stcb == 0) && stcb) {\ - SCTP_SAVE_ATOMIC_DECREMENT(&(stcb)->asoc.sb_cc, SCTP_BUF_LEN((m))); \ + SCTP_SAVE_ATOMIC_DECREMENT(&(stcb)->asoc.sb_ccc, SCTP_BUF_LEN((m))); \ SCTP_SAVE_ATOMIC_DECREMENT(&(stcb)->asoc.my_rwnd_control_len, MSIZE); \ } \ if (SCTP_BUF_TYPE(m) != MT_DATA && SCTP_BUF_TYPE(m) != MT_HEADER && \ @@ -207,10 +207,10 @@ extern struct pr_usrreqs sctp_usrreqs; } #define sctp_sballoc(stcb, sb, m) { \ - atomic_add_int(&(sb)->sb_cc,SCTP_BUF_LEN((m))); \ + atomic_add_int(&(sb)->sb_ccc,SCTP_BUF_LEN((m))); \ atomic_add_int(&(sb)->sb_mbcnt, MSIZE); \ if (stcb) { \ - atomic_add_int(&(stcb)->asoc.sb_cc,SCTP_BUF_LEN((m))); \ + atomic_add_int(&(stcb)->asoc.sb_ccc,SCTP_BUF_LEN((m))); \ atomic_add_int(&(stcb)->asoc.my_rwnd_control_len, MSIZE); \ } \ if (SCTP_BUF_TYPE(m) != MT_DATA && SCTP_BUF_TYPE(m) != MT_HEADER && \ Index: sys/netinet/sctp_output.c =================================================================== --- sys/netinet/sctp_output.c (.../head) (revision 266804) +++ sys/netinet/sctp_output.c (.../projects/sendfile) (revision 266807) @@ -7104,7 +7104,7 @@ one_more_time: if ((stcb->sctp_socket != NULL) && \ ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { - atomic_subtract_int(&stcb->sctp_socket->so_snd.sb_cc, sp->length); + atomic_subtract_int(&stcb->sctp_socket->so_snd.sb_ccc, sp->length); } if (sp->data) { sctp_m_freem(sp->data); @@ -11382,7 +11382,7 @@ jump_out: drp->current_onq = htonl(asoc->size_on_reasm_queue + asoc->size_on_all_streams + asoc->my_rwnd_control_len + - stcb->sctp_socket->so_rcv.sb_cc); + stcb->sctp_socket->so_rcv.sb_ccc); } else { /*- * If my rwnd is 0, possibly from mbuf depletion as well as Index: sys/netinet/tcp_usrreq.c =================================================================== --- sys/netinet/tcp_usrreq.c (.../head) (revision 266804) +++ sys/netinet/tcp_usrreq.c (.../projects/sendfile) (revision 266807) @@ -826,7 +826,7 @@ tcp_usr_send(struct socket *so, int flags, struct m_freem(control); /* empty control, just free it */ } if (!(flags & PRUS_OOB)) { - sbappendstream(&so->so_snd, m); + sbappendstream(&so->so_snd, m, flags); if (nam && tp->t_state < TCPS_SYN_SENT) { /* * Do implied connect if not yet connected, @@ -858,7 +858,8 @@ tcp_usr_send(struct socket *so, int flags, struct socantsendmore(so); tcp_usrclosed(tp); } - if (!(inp->inp_flags & INP_DROPPED)) { + if (!(inp->inp_flags & INP_DROPPED) && + !(flags & PRUS_NOTREADY)) { if (flags & PRUS_MORETOCOME) tp->t_flags |= TF_MORETOCOME; error = tcp_output(tp); @@ -884,7 +885,7 @@ tcp_usr_send(struct socket *so, int flags, struct * of data past the urgent section. * Otherwise, snd_up should be one lower. */ - sbappendstream_locked(&so->so_snd, m); + sbappendstream_locked(&so->so_snd, m, flags); SOCKBUF_UNLOCK(&so->so_snd); if (nam && tp->t_state < TCPS_SYN_SENT) { /* @@ -908,10 +909,12 @@ tcp_usr_send(struct socket *so, int flags, struct tp->snd_wnd = TTCP_CLIENT_SND_WND; tcp_mss(tp, -1); } - tp->snd_up = tp->snd_una + so->so_snd.sb_cc; - tp->t_flags |= TF_FORCEDATA; - error = tcp_output(tp); - tp->t_flags &= ~TF_FORCEDATA; + tp->snd_up = tp->snd_una + sbavail(&so->so_snd); + if (!(flags & PRUS_NOTREADY)) { + tp->t_flags |= TF_FORCEDATA; + error = tcp_output(tp); + tp->t_flags &= ~TF_FORCEDATA; + } } out: TCPDEBUG2((flags & PRUS_OOB) ? PRU_SENDOOB : Index: sys/netinet/accf_dns.c =================================================================== --- sys/netinet/accf_dns.c (.../head) (revision 266804) +++ sys/netinet/accf_dns.c (.../projects/sendfile) (revision 266807) @@ -75,7 +75,7 @@ sohasdns(struct socket *so, void *arg, int waitfla struct sockbuf *sb = &so->so_rcv; /* If the socket is full, we're ready. */ - if (sb->sb_cc >= sb->sb_hiwat || sb->sb_mbcnt >= sb->sb_mbmax) + if (sbused(sb) >= sb->sb_hiwat || sb->sb_mbcnt >= sb->sb_mbmax) goto ready; /* Check to see if we have a request. */ @@ -115,7 +115,7 @@ skippacket(struct sockbuf *sb) { unsigned long packlen; struct packet q, *p = &q; - if (sb->sb_cc < 2) + if (sbavail(sb) < 2) return DNS_WAIT; q.m = sb->sb_mb; @@ -122,7 +122,7 @@ skippacket(struct sockbuf *sb) { q.n = q.m->m_nextpkt; q.moff = 0; q.offset = 0; - q.len = sb->sb_cc; + q.len = sbavail(sb); GET16(p, packlen); if (packlen + 2 > q.len) Index: sys/netinet/sctputil.c =================================================================== --- sys/netinet/sctputil.c (.../head) (revision 266804) +++ sys/netinet/sctputil.c (.../projects/sendfile) (revision 266807) @@ -67,9 +67,9 @@ sctp_sblog(struct sockbuf *sb, struct sctp_tcb *st struct sctp_cwnd_log sctp_clog; sctp_clog.x.sb.stcb = stcb; - sctp_clog.x.sb.so_sbcc = sb->sb_cc; + sctp_clog.x.sb.so_sbcc = sb->sb_ccc; if (stcb) - sctp_clog.x.sb.stcb_sbcc = stcb->asoc.sb_cc; + sctp_clog.x.sb.stcb_sbcc = stcb->asoc.sb_ccc; else sctp_clog.x.sb.stcb_sbcc = 0; sctp_clog.x.sb.incr = incr; @@ -4356,7 +4356,7 @@ sctp_add_to_readq(struct sctp_inpcb *inp, { /* * Here we must place the control on the end of the socket read - * queue AND increment sb_cc so that select will work properly on + * queue AND increment sb_ccc so that select will work properly on * read. */ struct mbuf *m, *prev = NULL; @@ -4482,7 +4482,7 @@ sctp_append_to_readq(struct sctp_inpcb *inp, * the reassembly queue. * * If PDAPI this means we need to add m to the end of the data. - * Increase the length in the control AND increment the sb_cc. + * Increase the length in the control AND increment the sb_ccc. * Otherwise sb is NULL and all we need to do is put it at the end * of the mbuf chain. */ @@ -4694,10 +4694,10 @@ sctp_free_bufspace(struct sctp_tcb *stcb, struct s if (stcb->sctp_socket && (((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL)) || ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE)))) { - if (stcb->sctp_socket->so_snd.sb_cc >= tp1->book_size) { - stcb->sctp_socket->so_snd.sb_cc -= tp1->book_size; + if (stcb->sctp_socket->so_snd.sb_ccc >= tp1->book_size) { + stcb->sctp_socket->so_snd.sb_ccc -= tp1->book_size; } else { - stcb->sctp_socket->so_snd.sb_cc = 0; + stcb->sctp_socket->so_snd.sb_ccc = 0; } } @@ -5232,11 +5232,11 @@ sctp_sorecvmsg(struct socket *so, in_eeor_mode = sctp_is_feature_on(inp, SCTP_PCB_FLAGS_EXPLICIT_EOR); if (SCTP_BASE_SYSCTL(sctp_logging_level) & SCTP_RECV_RWND_LOGGING_ENABLE) { sctp_misc_ints(SCTP_SORECV_ENTER, - rwnd_req, in_eeor_mode, so->so_rcv.sb_cc, uio->uio_resid); + rwnd_req, in_eeor_mode, so->so_rcv.sb_ccc, uio->uio_resid); } if (SCTP_BASE_SYSCTL(sctp_logging_level) & SCTP_RECV_RWND_LOGGING_ENABLE) { sctp_misc_ints(SCTP_SORECV_ENTERPL, - rwnd_req, block_allowed, so->so_rcv.sb_cc, uio->uio_resid); + rwnd_req, block_allowed, so->so_rcv.sb_ccc, uio->uio_resid); } error = sblock(&so->so_rcv, (block_allowed ? SBL_WAIT : 0)); if (error) { @@ -5255,7 +5255,7 @@ restart_nosblocks: (inp->sctp_flags & SCTP_PCB_FLAGS_SOCKET_ALLGONE)) { goto out; } - if ((so->so_rcv.sb_state & SBS_CANTRCVMORE) && (so->so_rcv.sb_cc == 0)) { + if ((so->so_rcv.sb_state & SBS_CANTRCVMORE) && (so->so_rcv.sb_ccc == 0)) { if (so->so_error) { error = so->so_error; if ((in_flags & MSG_PEEK) == 0) @@ -5262,7 +5262,7 @@ restart_nosblocks: so->so_error = 0; goto out; } else { - if (so->so_rcv.sb_cc == 0) { + if (so->so_rcv.sb_ccc == 0) { /* indicate EOF */ error = 0; goto out; @@ -5269,9 +5269,9 @@ restart_nosblocks: } } } - if ((so->so_rcv.sb_cc <= held_length) && block_allowed) { + if ((so->so_rcv.sb_ccc <= held_length) && block_allowed) { /* we need to wait for data */ - if ((so->so_rcv.sb_cc == 0) && + if ((so->so_rcv.sb_ccc == 0) && ((inp->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || (inp->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { if ((inp->sctp_flags & SCTP_PCB_FLAGS_CONNECTED) == 0) { @@ -5307,7 +5307,7 @@ restart_nosblocks: } held_length = 0; goto restart_nosblocks; - } else if (so->so_rcv.sb_cc == 0) { + } else if (so->so_rcv.sb_ccc == 0) { if (so->so_error) { error = so->so_error; if ((in_flags & MSG_PEEK) == 0) @@ -5364,11 +5364,11 @@ restart_nosblocks: SCTP_INP_READ_LOCK(inp); } control = TAILQ_FIRST(&inp->read_queue); - if ((control == NULL) && (so->so_rcv.sb_cc != 0)) { + if ((control == NULL) && (so->so_rcv.sb_ccc != 0)) { #ifdef INVARIANTS panic("Huh, its non zero and nothing on control?"); #endif - so->so_rcv.sb_cc = 0; + so->so_rcv.sb_ccc = 0; } SCTP_INP_READ_UNLOCK(inp); hold_rlock = 0; @@ -5489,11 +5489,11 @@ restart_nosblocks: } /* * if we reach here, not suitable replacement is available - * fragment interleave is NOT on. So stuff the sb_cc + * fragment interleave is NOT on. So stuff the sb_ccc * into the our held count, and its time to sleep again. */ - held_length = so->so_rcv.sb_cc; - control->held_length = so->so_rcv.sb_cc; + held_length = so->so_rcv.sb_ccc; + control->held_length = so->so_rcv.sb_ccc; goto restart; } /* Clear the held length since there is something to read */ @@ -5790,10 +5790,10 @@ get_more_data: if (SCTP_BASE_SYSCTL(sctp_logging_level) & SCTP_SB_LOGGING_ENABLE) { sctp_sblog(&so->so_rcv, control->do_not_ref_stcb ? NULL : stcb, SCTP_LOG_SBFREE, cp_len); } - atomic_subtract_int(&so->so_rcv.sb_cc, cp_len); + atomic_subtract_int(&so->so_rcv.sb_ccc, cp_len); if ((control->do_not_ref_stcb == 0) && stcb) { - atomic_subtract_int(&stcb->asoc.sb_cc, cp_len); + atomic_subtract_int(&stcb->asoc.sb_ccc, cp_len); } copied_so_far += cp_len; freed_so_far += cp_len; @@ -5938,7 +5938,7 @@ wait_some_more: (sctp_is_feature_on(inp, SCTP_PCB_FLAGS_FRAG_INTERLEAVE))) { goto release; } - if (so->so_rcv.sb_cc <= control->held_length) { + if (so->so_rcv.sb_ccc <= control->held_length) { error = sbwait(&so->so_rcv); if (error) { goto release; @@ -5965,8 +5965,8 @@ wait_some_more: } goto done_with_control; } - if (so->so_rcv.sb_cc > held_length) { - control->held_length = so->so_rcv.sb_cc; + if (so->so_rcv.sb_ccc > held_length) { + control->held_length = so->so_rcv.sb_ccc; held_length = 0; } goto wait_some_more; @@ -6113,13 +6113,13 @@ out: freed_so_far, ((uio) ? (slen - uio->uio_resid) : slen), stcb->asoc.my_rwnd, - so->so_rcv.sb_cc); + so->so_rcv.sb_ccc); } else { sctp_misc_ints(SCTP_SORECV_DONE, freed_so_far, ((uio) ? (slen - uio->uio_resid) : slen), 0, - so->so_rcv.sb_cc); + so->so_rcv.sb_ccc); } } stage_left: Index: sys/netinet/sctputil.h =================================================================== --- sys/netinet/sctputil.h (.../head) (revision 266804) +++ sys/netinet/sctputil.h (.../projects/sendfile) (revision 266807) @@ -284,10 +284,10 @@ do { \ } \ if (stcb->sctp_socket && ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || \ (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { \ - if (stcb->sctp_socket->so_snd.sb_cc >= tp1->book_size) { \ - atomic_subtract_int(&((stcb)->sctp_socket->so_snd.sb_cc), tp1->book_size); \ + if (stcb->sctp_socket->so_snd.sb_ccc >= tp1->book_size) { \ + atomic_subtract_int(&((stcb)->sctp_socket->so_snd.sb_ccc), tp1->book_size); \ } else { \ - stcb->sctp_socket->so_snd.sb_cc = 0; \ + stcb->sctp_socket->so_snd.sb_ccc = 0; \ } \ } \ } \ @@ -305,10 +305,10 @@ do { \ } \ if (stcb->sctp_socket && ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || \ (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { \ - if (stcb->sctp_socket->so_snd.sb_cc >= sp->length) { \ - atomic_subtract_int(&stcb->sctp_socket->so_snd.sb_cc,sp->length); \ + if (stcb->sctp_socket->so_snd.sb_ccc >= sp->length) { \ + atomic_subtract_int(&stcb->sctp_socket->so_snd.sb_ccc,sp->length); \ } else { \ - stcb->sctp_socket->so_snd.sb_cc = 0; \ + stcb->sctp_socket->so_snd.sb_ccc = 0; \ } \ } \ } \ @@ -320,7 +320,7 @@ do { \ if ((stcb->sctp_socket != NULL) && \ ((stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_TCPTYPE) || \ (stcb->sctp_ep->sctp_flags & SCTP_PCB_FLAGS_IN_TCPPOOL))) { \ - atomic_add_int(&stcb->sctp_socket->so_snd.sb_cc,sz); \ + atomic_add_int(&stcb->sctp_socket->so_snd.sb_ccc,sz); \ } \ } while (0) Index: usr.bin/bluetooth/btsockstat/btsockstat.c =================================================================== --- usr.bin/bluetooth/btsockstat/btsockstat.c (.../head) (revision 266804) +++ usr.bin/bluetooth/btsockstat/btsockstat.c (.../projects/sendfile) (revision 266807) @@ -255,8 +255,8 @@ hcirawpr(kvm_t *kvmd, u_long addr) (unsigned long) pcb.so, (unsigned long) this, pcb.flags, - so.so_rcv.sb_cc, - so.so_snd.sb_cc, + so.so_rcv.sb_ccc, + so.so_snd.sb_ccc, pcb.addr.hci_node); } } /* hcirawpr */ @@ -303,8 +303,8 @@ l2caprawpr(kvm_t *kvmd, u_long addr) "%-8lx %-8lx %6d %6d %-17.17s\n", (unsigned long) pcb.so, (unsigned long) this, - so.so_rcv.sb_cc, - so.so_snd.sb_cc, + so.so_rcv.sb_ccc, + so.so_snd.sb_ccc, bdaddrpr(&pcb.src, NULL, 0)); } } /* l2caprawpr */ @@ -361,8 +361,8 @@ l2cappr(kvm_t *kvmd, u_long addr) fprintf(stdout, "%-8lx %6d %6d %-17.17s/%-5d %-17.17s %-5d %s\n", (unsigned long) this, - so.so_rcv.sb_cc, - so.so_snd.sb_cc, + so.so_rcv.sb_ccc, + so.so_snd.sb_ccc, bdaddrpr(&pcb.src, local, sizeof(local)), pcb.psm, bdaddrpr(&pcb.dst, remote, sizeof(remote)), @@ -467,8 +467,8 @@ rfcommpr(kvm_t *kvmd, u_long addr) fprintf(stdout, "%-8lx %6d %6d %-17.17s %-17.17s %-4d %-4d %s\n", (unsigned long) this, - so.so_rcv.sb_cc, - so.so_snd.sb_cc, + so.so_rcv.sb_ccc, + so.so_snd.sb_ccc, bdaddrpr(&pcb.src, local, sizeof(local)), bdaddrpr(&pcb.dst, remote, sizeof(remote)), pcb.channel, Index: usr.bin/systat/netstat.c =================================================================== --- usr.bin/systat/netstat.c (.../head) (revision 266804) +++ usr.bin/systat/netstat.c (.../projects/sendfile) (revision 266807) @@ -333,8 +333,8 @@ enter_kvm(struct inpcb *inp, struct socket *so, in struct netinfo *p; if ((p = enter(inp, state, proto)) != NULL) { - p->ni_rcvcc = so->so_rcv.sb_cc; - p->ni_sndcc = so->so_snd.sb_cc; + p->ni_rcvcc = so->so_rcv.sb_ccc; + p->ni_sndcc = so->so_snd.sb_ccc; } } Index: usr.bin/netstat/netgraph.c =================================================================== --- usr.bin/netstat/netgraph.c (.../head) (revision 266804) +++ usr.bin/netstat/netgraph.c (.../projects/sendfile) (revision 266807) @@ -119,7 +119,7 @@ netgraphprotopr(u_long off, const char *name, int if (Aflag) printf("%8lx ", (u_long) this); printf("%-5.5s %6u %6u ", - name, sockb.so_rcv.sb_cc, sockb.so_snd.sb_cc); + name, sockb.so_rcv.sb_ccc, sockb.so_snd.sb_ccc); /* Get info on associated node */ if (ngpcb.node_id == 0 || csock == -1) Index: usr.bin/netstat/unix.c =================================================================== --- usr.bin/netstat/unix.c (.../head) (revision 266804) +++ usr.bin/netstat/unix.c (.../projects/sendfile) (revision 266807) @@ -287,7 +287,8 @@ unixdomainpr(struct xunpcb *xunp, struct xsocket * } else { printf("%8lx %-6.6s %6u %6u %8lx %8lx %8lx %8lx", (long)so->so_pcb, socktype[so->so_type], so->so_rcv.sb_cc, - so->so_snd.sb_cc, (long)unp->unp_vnode, (long)unp->unp_conn, + so->so_snd.sb_cc, (long)unp->unp_vnode, + (long)unp->unp_conn, (long)LIST_FIRST(&unp->unp_refs), (long)LIST_NEXT(unp, unp_reflink)); } Index: usr.bin/netstat/inet.c =================================================================== --- usr.bin/netstat/inet.c (.../head) (revision 266804) +++ usr.bin/netstat/inet.c (.../projects/sendfile) (revision 266807) @@ -137,7 +137,7 @@ pcblist_sysctl(int proto, const char *name, char * static void sbtoxsockbuf(struct sockbuf *sb, struct xsockbuf *xsb) { - xsb->sb_cc = sb->sb_cc; + xsb->sb_cc = sb->sb_ccc; xsb->sb_hiwat = sb->sb_hiwat; xsb->sb_mbcnt = sb->sb_mbcnt; xsb->sb_mcnt = sb->sb_mcnt; @@ -479,7 +479,8 @@ protopr(u_long off, const char *name, int af1, int printf("%6u %6u %6u ", tp->t_sndrexmitpack, tp->t_rcvoopack, tp->t_sndzerowin); } else { - printf("%6u %6u ", so->so_rcv.sb_cc, so->so_snd.sb_cc); + printf("%6u %6u ", + so->so_rcv.sb_cc, so->so_snd.sb_cc); } if (numeric_port) { if (inp->inp_vflag & INP_IPV4) { --IuJpT0rwbUevm2bB-- From owner-freebsd-arch@FreeBSD.ORG Thu May 29 16:31:17 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 76305C04 for ; Thu, 29 May 2014 16:31:17 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 372742BED for ; Thu, 29 May 2014 16:31:17 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id i50so1720957qgf.31 for ; Thu, 29 May 2014 09:31:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:content-type; bh=ICE6ybwNPn/bXcSijyW20inGpC7x47OzB1ZEb/kMH80=; b=a3M1iWW3DuhBQMNiTCLcBP6OAlVSkS7bReww2bnWmNkPDsuwU7S9vFsId/4mZZzS9U u+tgTHwSlOpZhJ1/FIrmlPRf6BDr0/YW7gMoCVtixuzMtrNgT2Hpnb5armvBFf05lb6v hbbCnrzqwgLrEVhPxnuWZh1wMVwbBOQrAsJ4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-type; bh=ICE6ybwNPn/bXcSijyW20inGpC7x47OzB1ZEb/kMH80=; b=c/jokZuzoj0uapBIz0/7LbWZPaqxf+yh3h7Ipw4pTXviASqyqfYgHtsNr+iVWUs5Ug INCXKX4BlULNoKX/i1L8wfwZmlZxF/G+3YBjB2dqHPaE5Vo29VyGMCFpDmZxahP0wV+z /vrztzjYfiFB33DADoFADM6H7ZEDnjmCTmMq+FW23bRyEGLmdGhaN0q5FDQCxcmX4QVL 7oGneA28AJt5DRNjIjLJarpmWSbt1hi65T8cELuvQSG7H8Xv7YPige2fGDVJ7w6W4vr7 hFGMGgBVr/YwOe8OG5tx0rXj6jrA6MlI79M90dJ0OMgsXxPztC4JeMCZgQvTgXGq65xd bTlg== X-Gm-Message-State: ALoCoQmj41uuq4Glft6VdRWGOHDZ71bGQdDlyyQ9952APm+J5JwsCWAOFdzbBtDOP5WDk8VgwlWH X-Received: by 10.140.94.225 with SMTP id g88mr10857933qge.101.1401381076298; Thu, 29 May 2014 09:31:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.147.135 with HTTP; Thu, 29 May 2014 09:30:46 -0700 (PDT) From: Eitan Adler Date: Thu, 29 May 2014 09:30:46 -0700 Message-ID: Subject: system data diagram To: "freebsd-arch@freebsd.org" , current@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 16:31:17 -0000 Hey all, http://www.brendangregg.com/Perf/linuxperftools.png is a neat diagram. Who wants to make one for FreeBSD? It'll even make its way to our website & documentation :) -- Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Thu May 29 23:37:15 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9467256A; Thu, 29 May 2014 23:37:15 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3909D2481; Thu, 29 May 2014 23:37:14 +0000 (UTC) Received: from jammyc-sslvpn-nc.jnpr.net ([66.129.239.12]) (authenticated bits=0) by mail.xcllnt.net (8.14.8/8.14.8) with ESMTP id s4TNb3Np037418 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 29 May 2014 16:37:05 -0700 (PDT) (envelope-from marcel@xcllnt.net) Content-Type: multipart/signed; boundary="Apple-Mail=_DF94C62A-7CEC-4128-B1FC-DF30062557C1"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 From: Marcel Moolenaar In-Reply-To: Date: Thu, 29 May 2014 16:36:58 -0700 Message-Id: References: To: Rui Paulo X-Mailer: Apple Mail (2.1878.2) Cc: Anuranjan Shukla , Gleb Smirnoff , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 23:37:15 -0000 --Apple-Mail=_DF94C62A-7CEC-4128-B1FC-DF30062557C1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On May 28, 2014, at 8:48 PM, Rui Paulo wrote: > On May 28, 2014, atThere were at least 3 efforts on fixing this: >>=20 >>=20 >> 1) Juniper=92s JUNOS is a FreeBSD based operating system that has >> its own (alternative) network stack, but that leverages the >> network drivers from FreeBSD. Juniper mechanically changed all >> ifnet dereferences to to accessor methods. This could have >> been incorporated as early as 2011, but lacked good follow >> through. Marcel Moolenaar was prime contact for this. >>=20 >> 2) Andre Oppermann was sponsored in 2013 by the FreeBSD >> Foundation to make ifnet(9) opaque. This is not complete as of >> the time of this writing. >>=20 >> 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but >> that always has been delayed due to 1) and 2). >=20 > This is indeed needed, but it would be nice to understand what would = happen if the community has comments about your patch. Will Juniper be = able to integrate back those comments? For example, I think the type = "if_t" should be "ifnet_t". Another comment I have is: why do you have = to cast if_t to (struct ifnet *) in all the accessor methods? It would = be better to create a private header typedef'ing if_t to struct ifnet, = avoiding the copy & paste casting. Both Gleb and Anu have answered some parts. I'll answer the integration part: We (=3D Juniper) will deal with it. There are too many cogs in the wheel for me to have a clear picture, so I'm not going to worry about it. I think the FreeBSD community should worry even less :-) The important thing to realize is that the patch has a history and a context that is not seen or understood by the FreeBSD community. As long as everyone focusses on the important aspects and not get lost in the minutiae, I'm confident we're all going to be happy with the end result, even if the interim isn't looking so good on first sight. HTH, --=20 Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_DF94C62A-7CEC-4128-B1FC-DF30062557C1 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlOHxJoACgkQpgWlLWHuifYjZgCdFCQK0QSnWdgN6GE82Pp4+XTz k+YAn3M6FNL+f9mFnB57Ihkn2SGyqI/K =yLD9 -----END PGP SIGNATURE----- --Apple-Mail=_DF94C62A-7CEC-4128-B1FC-DF30062557C1-- From owner-freebsd-arch@FreeBSD.ORG Thu May 29 23:56:40 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 496F97E2; Thu, 29 May 2014 23:56:40 +0000 (UTC) Received: from mail.xcllnt.net (mail.xcllnt.net [50.0.150.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E6FFA25F1; Thu, 29 May 2014 23:56:39 +0000 (UTC) Received: from jammyc-sslvpn-nc.jnpr.net ([66.129.239.12]) (authenticated bits=0) by mail.xcllnt.net (8.14.8/8.14.8) with ESMTP id s4TNuYFj037457 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Thu, 29 May 2014 16:56:37 -0700 (PDT) (envelope-from marcel@xcllnt.net) Content-Type: multipart/signed; boundary="Apple-Mail=_3D0D03CB-400A-486D-829B-A80560601996"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 From: Marcel Moolenaar In-Reply-To: <201405290537.s4T5b16Z033344@mail.karels.net> Date: Thu, 29 May 2014 16:56:28 -0700 Message-Id: <2A25AF90-69B9-4F81-A4DB-8289981BA8D2@xcllnt.net> References: <201405290537.s4T5b16Z033344@mail.karels.net> To: mike@karels.net X-Mailer: Apple Mail (2.1878.2) Cc: Anuranjan Shukla , Gleb Smirnoff , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 May 2014 23:56:40 -0000 --Apple-Mail=_3D0D03CB-400A-486D-829B-A80560601996 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On May 28, 2014, at 10:37 PM, Mike Karels wrote: > Marcel and others, is there more to the roadmap than making the ifnet = easier > to change? Could you outline a bit more of the roadmap? I know that = Juniper > has more levels in the hierarchy of interface data structures. What = are you > proposing that we change after this step? The community has shown an interest in the layering that is there in JUNOS and Juniper is willing and able to share details. However, after this first step I think Juniper should work with the community to get the module support of the network stack into FreeBSD. Being able to build the network stack as a module and having NIC drivers go through an API is good to have when doing core restructuring. Not only does it allow multiple approaches to be tried in parallel, it's easy to try new things and "switch" back to something that works. Things that can be done in parallel are mbuf changes, ifnet changes, better handling/support for Q-in-Q, better interfaces for network configuration, etc... HTH, --=20 Marcel Moolenaar marcel@xcllnt.net --Apple-Mail=_3D0D03CB-400A-486D-829B-A80560601996 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlOHySwACgkQpgWlLWHuifaIrwCbBWZIFjeBmF/9IzoyHnyr99rn DqUAn0ZSLMpbtNp/pf0osdCO4AUkDOW2 =H2yP -----END PGP SIGNATURE----- --Apple-Mail=_3D0D03CB-400A-486D-829B-A80560601996-- From owner-freebsd-arch@FreeBSD.ORG Fri May 30 05:54:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 866461DB; Fri, 30 May 2014 05:54:49 +0000 (UTC) Received: from mail-oa0-x231.google.com (mail-oa0-x231.google.com [IPv6:2607:f8b0:4003:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45AE72195; Fri, 30 May 2014 05:54:49 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id eb12so1389962oac.36 for ; Thu, 29 May 2014 22:54:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=PO7PiTqYVxgt6ZpmB7N5BkKabOcaCRKanhlBlJv0NPA=; b=WgwG+NTYe0QS2mcSvIXQT5A9H/hd/ct1R79QH8uo1OB55ow7G/PAXjnKKvBJJzl0Yn 0Mm+zJf3U2rodifVEHt+v5ImHz3jtxjpbPMev1quKTXN1QGlZECIM1hDqlBxXU/K6X7g ZdVmS+j40ivbV47lhL9fOtVYmddLCZSzzz4u3/RJMkNPw8HtFC2bvDUiwfzJ3N1AVA+n KPxJJBQYgavME/kokoBjfkdfzM1sYfiGxnfMnRY5Btn9JBtf6OrcBsqAIh50VnQRhSPf fYtW791FRojF+t7QcfSG0jUE5uMKxynJhQq2RkMViM1rI5ylPnHnFsaQVyOg8tzJ27d5 Z/OQ== X-Received: by 10.60.54.228 with SMTP id m4mr14022213oep.29.1401429288579; Thu, 29 May 2014 22:54:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.123.178 with HTTP; Thu, 29 May 2014 22:54:18 -0700 (PDT) In-Reply-To: <1401280050.1152.377.camel@revolution.hippie.lan> References: <1401115733.1152.339.camel@revolution.hippie.lan> <1401280050.1152.377.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Fri, 30 May 2014 13:54:18 +0800 Message-ID: Subject: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 05:54:49 -0000 Updated patch works fine for me surviving -j64 buildworld on quad-core. Compile time did not differ much, ~35 mins. I 'd guess that's because build has already been well paralleled inside each subdirs for ~4 jobs. It would be even better if someone can stress it on multicore system, say 12+ cores, to explore higher hw concurrency to see the difference. Looks other targets using .for loops can be translated to this finer dependency rules as well? e.g. _bootstrap-tools, probably already on your todo list? -Jia-Shiun. On Wed, May 28, 2014 at 8:27 PM, Ian Lepore wrote: > On Wed, 2014-05-28 at 12:58 +0800, Jia-Shiun Li wrote: >> It failed cleandir at libmb with -j4. Test script attached and log >> snippet below. >> >> Tested with: >> - HW: i5-3450 CPU w/ 8GB memory >> - /usr/obj & src mounted on tmpfs. >> >> src uses ~1GB without .svn dir. /usr/obj uses another ~2GB for >> buildworld, not including buildkernel. If memory is not constrained I >> think it is easier to use tmpfs to uncover parallel timing/race issues >> hidden by slower I/O. >> >> >> -Jia-shiun. > > Doh! There was a typo, libmb should have been libmd. More importantly, > it shows that my testing didn't test anything at all. I think I applied > the patch in one sandbox and then ran make universe in a different one. > Here's an updated patch (this time I'll try to test it correctly myself > too). > > Thanks for testing. > > -- Ian > From owner-freebsd-arch@FreeBSD.ORG Fri May 30 08:58:21 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B59E9E7B for ; Fri, 30 May 2014 08:58:21 +0000 (UTC) Received: from relay02.pair.com (relay02.pair.com [209.68.5.16]) by mx1.freebsd.org (Postfix) with SMTP id 509092F94 for ; Fri, 30 May 2014 08:58:20 +0000 (UTC) Received: (qmail 97335 invoked from network); 30 May 2014 08:51:38 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay02.pair.com with SMTP; 30 May 2014 08:51:38 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id s4U8pcGN011993; Fri, 30 May 2014 10:51:38 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id s4U8pcLi011992; Fri, 30 May 2014 10:51:38 +0200 (CEST) (envelope-from pho) Date: Fri, 30 May 2014 10:51:37 +0200 From: Peter Holm To: Gleb Smirnoff Subject: Re: [CFT/review] new sendfile(2) Message-ID: <20140530085137.GA11895@x2.osted.lan> References: <20140529102054.GX50679@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140529102054.GX50679@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 08:58:21 -0000 On Thu, May 29, 2014 at 02:20:54PM +0400, Gleb Smirnoff wrote: > Hello! > > At Netflix and Nginx we are experimenting with improving FreeBSD > wrt sending large amounts of static data via HTTP. > > One of the approaches we are experimenting with is new sendfile(2) > implementation, that doesn't block on the I/O done from the file > descriptor. > > The problem with classic sendfile(2) is that if the the request > length is large enough, and file data is not cached in VM, then > sendfile(2) syscall would not return until it fills socket buffer > with data. With modern internet socket buffers can be up to 1 Mb, > thus time taken by the syscall raises by order of magnitude. All > the time, the nginx worker is blocked in syscall and doesn't > process data from other clients. The best current practice to > mitigate that is known as "sendfile(2) + aio_read(2)". This is > special mode of nginx operation on FreeBSD. The sendfile(2) call > is issued with SF_NODISKIO flag, that forbids the syscall to > perform disk I/O, and send only data that is cached by VM. If > sendfile(2) reports that I/O needs to be done (but forbidden), then > nginx would do aio_read() of a chunk of the file. The data read > is cached by VM, as side affect. Then sendfile() is called again. > > Now for the new sendfile. The core idea is that sendfile() > schedules the I/O, but doesn't wait for it to complete. It > returns immediately to the process, and I/O completion is > processed in kernel context. Unlike aio(4), no additional > threads in kernel are created. The new sendfile is a drop-in > replacement for the old one. Applications (like nginx) doesn't > need recompile, neither configuration change. The SF_NODISKIO is > ignored. > > The patch for review is available at: > > https://phabric.freebsd.org/D102 > > And for those who prefer email attachments, it is also attached. > The patch has 3 logically separate changes in itself: > > 1) Split of socket buffer sb_cc field into sb_acc and sb_ccc. Where > sb_acc stands for "available character count" and sb_ccc is "claimed > character count". This allows us to write a data to a socket, that is > not ready yet. The data sits in the socket, consumes its space, and > keeps itself in the right order with earlier or later writes to socket. > But it can be send only after it is marked as ready. This change is > split across many files. > > 2) A new vnode operation: VOP_GETPAGES_ASYNC(). This one lives in sys/vm. > > 3) Actual implementation of new sendfile(2). This one lives in > kern/uipc_syscalls.c > > > > At Netflix, we already see improvements with new sendfile(2). > We can send more data utilizing same amount of CPU, and we can > push closer to 0% idle, without experiencing short lags. > > However, we have somewhat modified VM subsystem, that behaves > optimal for our task, but suboptimal for average FreeBSD system. > I'd like someone from community to try the new sendfile(2) at > other setup and see how does it serve for you. > > To be the early tester you need to checkout projects/sendfile > branch and build kernel from it. The world from head/ would > run fine with it. > > svn co http://svn.freebsd.org/base/projects/sendfile > cd sendfile > ... build kernel ... > > Limitations: > - No testing were done on serving files on NFS. > - No testing were done on serving files on ZFS. > I got this: panic: sbready: sb 0xfffff801834219e8 NULL fnrdy http://people.freebsd.org/~pho/stress/log/gleb007.txt - Peter From owner-freebsd-arch@FreeBSD.ORG Sat May 31 14:04:27 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4E8BFD for ; Sat, 31 May 2014 14:04:27 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 977CE2C07 for ; Sat, 31 May 2014 14:04:27 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WqjtZ-000GoO-QQ; Sat, 31 May 2014 14:04:25 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s4VE4NZL001203; Sat, 31 May 2014 08:04:23 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1+TAo89zN1rqdHOHDNRuHlW Subject: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds From: Ian Lepore To: Jia-Shiun Li In-Reply-To: References: <1401115733.1152.339.camel@revolution.hippie.lan> <1401280050.1152.377.camel@revolution.hippie.lan> Content-Type: text/plain; charset="us-ascii" Date: Sat, 31 May 2014 08:04:23 -0600 Message-ID: <1401545063.20883.41.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 May 2014 14:04:27 -0000 On Fri, 2014-05-30 at 13:54 +0800, Jia-Shiun Li wrote: > Updated patch works fine for me surviving -j64 buildworld on > quad-core. Compile time did not differ much, ~35 mins. I 'd guess > that's because build has already been well paralleled inside each > subdirs for ~4 jobs. > > It would be even better if someone can stress it on multicore system, > say 12+ cores, to explore higher hw concurrency to see the difference. > > Looks other targets using .for loops can be translated to this finer > dependency rules as well? e.g. _bootstrap-tools, probably already on > your todo list? > > > -Jia-Shiun. Yes, I wouldn't expect much build time difference because this doesn't really increase the parallelism other than for a couple of the early-build libraries. It does add new dependencies that weren't expressed before and were apparently just accidentally not causing problems for anyone. The parallelism in the bootstrap stuff in Makefile.inc1 is done with a different-but-similar mechanism, and it would be nice to fix that to just use bsd.subdir.mk instead of almost-duplicating it inline. That's on a longer-term to-do list for me. -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Jun 1 15:21:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50D45E3A; Sun, 1 Jun 2014 15:21:57 +0000 (UTC) Received: from mail-qg0-x236.google.com (mail-qg0-x236.google.com [IPv6:2607:f8b0:400d:c04::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0422E2531; Sun, 1 Jun 2014 15:21:56 +0000 (UTC) Received: by mail-qg0-f54.google.com with SMTP id q108so9045256qgd.41 for ; Sun, 01 Jun 2014 08:21:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=mm0FbfX7Iq8bd9dkoPFO8NwZ+MHsXpC5vrNfjC2R/QA=; b=ePoxpmcG8Y5YokOrUBcgUPmDOFrYwd5eGtImVUxBW82eDVcIFqTOtqK0vzOyHtFpJi 1qBpY2iV1x1/Zi0HcKUa/k2k2Yln7HLVYMqVqkc32l4RFtK1eUC6a+A3duG0aom7pXNS pTB7meL2dstePdL8uLH6mMFC8K52VUSoWV99Mh+DCftETfkgjYBmYkKqMYbJMrOsyNrY 6Y6wBJ4YqhaKTDzoOIgCYdkchyR8gKw1O0jx9M2sXkChsv/3shHdYla2rwQLf9PUGooC b8kHamUGwgfw2fz+8DpIFMUqJBCUMSIcgUprfQ2lVX6YsnlcEhMv+wRa01vFcqDqEEHK 2NUg== MIME-Version: 1.0 X-Received: by 10.140.35.212 with SMTP id n78mr38284529qgn.87.1401636115836; Sun, 01 Jun 2014 08:21:55 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.191.201 with HTTP; Sun, 1 Jun 2014 08:21:55 -0700 (PDT) Date: Sun, 1 Jun 2014 08:21:55 -0700 X-Google-Sender-Auth: 339m2wbqWM-d-pP9UDSMZlHRUuI Message-ID: Subject: [rfc] TCP timewait and credential handling - why would we get a TW with no credential? From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jun 2014 15:21:57 -0000 Hi, I've been seeing this panic under high very short-term connection TCP throughput tests (30,000 connections a second) with SO_REUSEPORT: Current language: auto; currently minimal (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) frame 11 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 644 crfree(tw->tw_cred); (kgdb) print tw $1 = (struct tcptw *) 0xfffff801348b5780 (kgdb) print tw->tw_cred $2 = (struct ucred *) 0x0 (kgdb) and: #10 0xffffffff808d017e in crfree (cr=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_prot.c:1837 #11 0xffffffff80a6bdf1 in tcp_twclose (tw=0xfffff801348b5780, reuse=0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:644 #12 0xffffffff80a6bc53 in tcp_twcheck (inp=, to=, th=, m=, tlen=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_timewait.c:425 #13 0xffffffff80a5cf3f in tcp_input (m=, off0=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/tcp_input.c:949 #14 0xffffffff809fa4e7 in ip_input (m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/netinet/ip_input.c:729 #15 0xffffffff80998441 in netisr_dispatch_src (proto=, source=, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #16 0xffffffff80990173 in ether_demux (ifp=, m=0xfffff8004831e000) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:775 #17 0xffffffff80990dce in ether_nh_input (m=) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:582 #18 0xffffffff80998441 in netisr_dispatch_src (proto=, source=, m=0xffffffff) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/netisr.c:972 #19 0xffffffff809903c6 in ether_input (ifp=, m=0x0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/net/if_ethersubr.c:683 #20 0xffffffff804e9d67 in igb_rxeof (count=99) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:4882 #21 0xffffffff804ea47f in igb_msix_que (arg=0xfffff8000d47e670) at /usr/home/adrian/git/github/erikarn/freebsd/sys/dev/e1000/if_igb.c:1626 #22 0xffffffff808aef03 in intr_event_execute_handlers (p=, ie=0xfffff8000f76cb00) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1263 #23 0xffffffff808af866 in ithread_loop (arg=0xfffff8000f908d20) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_intr.c:1276 #24 0xffffffff808acc41 in fork_exit (callout=0xffffffff808af7d0 , arg=0xfffff8000f908d20, frame=0xfffffe1046c2cac0) at /usr/home/adrian/git/github/erikarn/freebsd/sys/kern/kern_fork.c:977 #25 0xffffffff80c939ae in fork_trampoline () at /usr/home/adrian/git/github/erikarn/freebsd/sys/amd64/amd64/exception.S:605 #26 0x0000000000000000 in ?? () ... there's only one path to deleting the credentials and that's via tcp_twclose() -> tcp_tw_2msl_stop(). Has anyone seen this kind of problem before? -a From owner-freebsd-arch@FreeBSD.ORG Mon Jun 2 17:58:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7449E377; Mon, 2 Jun 2014 17:58:44 +0000 (UTC) Received: from mail-qg0-x234.google.com (mail-qg0-x234.google.com [IPv6:2607:f8b0:400d:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1764F226B; Mon, 2 Jun 2014 17:58:44 +0000 (UTC) Received: by mail-qg0-f52.google.com with SMTP id a108so11524547qge.11 for ; Mon, 02 Jun 2014 10:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=3tt+uyI2NXmmucKC1cVTbGdHlNhGfeqhbMRHLSXYiNM=; b=SdjdM1B1T5aHMGSKNXS9OSpqu5CZOku7lQCYtjmjZPG+VlX+MAuF9gURSND+i/z+XG 6pn0GoZrDSUdKDDqSt7z2x2vNhOqx6asq3Oh1G7nzFBEAn0YA4YjVPjOwVbKYXuEnzAf jrdElAi8Qe2ex4KWY3oyHWrM43hmZi/p7GlKZpKsk46ITQaA/dKFK6eaeep8QL7y9YVd cK8io1hU1oAfIELiEihFK5sb2eRSJr2SSfUOYxz9kyM3FO+sE23nX+rruAeuNXDIimS6 Z5EDd2+pj2ZU3B8Aydl3wbodyN2gQLSv6VAPnR5wYVuRgGo/8L3EikjA1CORcb2s7ThH jlJQ== MIME-Version: 1.0 X-Received: by 10.140.104.195 with SMTP id a61mr47227330qgf.102.1401731922954; Mon, 02 Jun 2014 10:58:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Mon, 2 Jun 2014 10:58:42 -0700 (PDT) Date: Mon, 2 Jun 2014 10:58:42 -0700 X-Google-Sender-Auth: OsEMorLcNE3AcXOPoVqJeh-p4Zw Message-ID: Subject: Seeing ENOTCAPABLE from write FDs in kqueue? From: Adrian Chadd To: Robert Watson , Pawel Jakub Dawidek , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 17:58:44 -0000 Hi guys/all, I've been tinkering with my RSS stuff where I have multiple listen sockets, one per worker thread, all terminating high throughput TCP transactions. It's all HTTP; I'm using libevent, evhttp and a very small amount of glue code to achieve this. libevent craps out from time to time because occasionally one of the events in kqueue returns ENOTCAPABLE. ENOTCAPABLE: ident=3D1686, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1324, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1740, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1628, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D1199, filter=3D-2, flags=3D0x00004000, fflags=3D0x0000= 0000, data=3D93 ENOTCAPABLE: ident=3D818, filter=3D-2, flags=3D0x00004000, fflags=3D0x00000= 000, data=3D93 ENOTCAPABLE: ident=3D389, filter=3D-2, flags=3D0x00004000, fflags=3D0x00000= 000, data=3D93 It's happening on the various data FDs; not on the listen socket. What I'm seeing from ktrace: 27770 rss-http CAP operation requires , descriptor holds 27770 rss-http CAP operation requires , descriptor holds <> 27770 rss-http CAP operation requires , descriptor holds <> 27770 rss-http CAP operation requires , descriptor holds <> 27770 rss-http CAP operation requires , descriptor holds <> 27770 rss-http CAP operation requires , descriptor holds <> .. so, why exactly would I be seeing this? Is this some capability race where we have an FD setup but no capability yet assigned when it's added into the kqueue set? It's happening under high throughput (> 30,000 TCP sessions a second.) Where would I continue debugging this? Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Mon Jun 2 22:38:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 275CFFC0; Mon, 2 Jun 2014 22:38:03 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F3F2E2C55; Mon, 2 Jun 2014 22:38:02 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s52Mc1dg026987 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Jun 2014 15:38:02 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s52Mc1rV026986; Mon, 2 Jun 2014 15:38:01 -0700 (PDT) (envelope-from jmg) Date: Mon, 2 Jun 2014 15:38:01 -0700 From: John-Mark Gurney To: Gleb Smirnoff Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Message-ID: <20140602223801.GC43976@funkthat.com> Mail-Followup-To: Gleb Smirnoff , Rui Paulo , Anuranjan Shukla , "freebsd-arch@FreeBSD.org Arch" , Marcel Moolenaar References: <20140529040425.GT50679@glebius.int.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140529040425.GT50679@glebius.int.ru> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 02 Jun 2014 15:38:02 -0700 (PDT) Cc: Marcel Moolenaar , Anuranjan Shukla , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 22:38:03 -0000 Gleb Smirnoff wrote this message on Thu, May 29, 2014 at 08:04 +0400: > On Wed, May 28, 2014 at 08:48:11PM -0700, Rui Paulo wrote: > R> On May 28, 2014, at 9:34, Marcel Moolenaar wrote: > R> > R> > All, > R> > > R> > The ifnet structure represents a network interface. Right now it > R> > is known to all NIC drivers as well as to all protocols. This > R> > means that whenever we change the structure, we need at minimum > R> > recompile all drivers, but usually also change them. This severely > R> > slow downs the development in this area and also makes it hard, if > R> > not, impossible to merge things back to stable branches. > R> > > R> > There were at least 3 efforts on fixing this: > R> > > R> > 1) Juniper???s JUNOS is a FreeBSD based operating system that has > R> > its own (alternative) network stack, but that leverages the > R> > network drivers from FreeBSD. Juniper mechanically changed all > R> > ifnet dereferences to to accessor methods. This could have > R> > been incorporated as early as 2011, but lacked good follow > R> > through. Marcel Moolenaar was prime contact for this. > R> > > R> > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD > R> > Foundation to make ifnet(9) opaque. This is not complete as of > R> > the time of this writing. > R> > > R> > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but > R> > that always has been delayed due to 1) and 2). > R> > R> This is indeed needed, but it would be nice to understand what would happen if the community has comments about your patch. Will Juniper be able to integrate back those comments? For example, I think the type "if_t" should be "ifnet_t". Another comment I have is: why do you have to cast if_t to (struct ifnet *) in all the accessor methods? It would be better to create a private header typedef'ing if_t to struct ifnet, avoiding the copy & paste casting. > > Because for now patch supports compiling unconverted drivers. This requires > to pass 'void *' as argument. When all drivers are converted, this will be > removed. In if_var.h we would have: > > struct ifnet; > typedef struct ifnet * if_t; > > All casts to (struct ifnet *) will go away. When? The schedule did not have a time frame on when this will happen.. Will it be this year? Next year? Next decade? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Mon Jun 2 22:50:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFCAE3EF; Mon, 2 Jun 2014 22:50:35 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1lp0145.outbound.protection.outlook.com [207.46.163.145]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47F8F2D40; Mon, 2 Jun 2014 22:50:34 +0000 (UTC) Received: from CO2PR05MB731.namprd05.prod.outlook.com (10.141.228.21) by CO2PR05MB569.namprd05.prod.outlook.com (10.141.197.12) with Microsoft SMTP Server (TLS) id 15.0.944.11; Mon, 2 Jun 2014 22:50:26 +0000 Received: from CO2PR05MB730.namprd05.prod.outlook.com (10.141.228.15) by CO2PR05MB731.namprd05.prod.outlook.com (10.141.228.21) with Microsoft SMTP Server (TLS) id 15.0.949.11; Mon, 2 Jun 2014 22:50:25 +0000 Received: from CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) by CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) with mapi id 15.00.0949.001; Mon, 2 Jun 2014 22:50:25 +0000 From: Anuranjan Shukla To: John-Mark Gurney , Gleb Smirnoff Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Thread-Topic: Roadmap for ifnet(9) for FreeBSD 11 Thread-Index: AQHPepLaX4+rVOOldkOoqmC4MWB2pZtW7HmAgAAEiYCAB4B2gP//jhsA Date: Mon, 2 Jun 2014 22:50:25 +0000 Message-ID: References: <20140529040425.GT50679@glebius.int.ru> <20140602223801.GC43976@funkthat.com> In-Reply-To: <20140602223801.GC43976@funkthat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 x-originating-ip: [66.129.239.10] x-microsoft-antispam: BL:0; ACTION:Default; RISK:Low; SCL:0; SPMLVL:NotSpam; PCL:0; RULEID: x-forefront-prvs: 0230B09AC4 x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(428001)(24454002)(377454003)(479174003)(51704005)(35774003)(199002)(189002)(46102001)(83072002)(19580405001)(81542001)(77982001)(79102001)(4396001)(81342001)(99396002)(76482001)(92566001)(19580395003)(85852003)(21056001)(31966008)(99286001)(92726001)(76176999)(87936001)(80022001)(20776003)(66066001)(54356999)(77096999)(50986999)(86362001)(101416001)(83506001)(83322001)(2656002)(74502001)(36756003)(74662001); DIR:OUT; SFP:; SCL:1; SRVR:CO2PR05MB731; H:CO2PR05MB730.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: juniper.net does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anshukla@juniper.net; Content-Type: text/plain; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BL:0; ACTION:Default; RISK:Low; SCL:0; SPMLVL:NotSpam; PCL:0; RULEID: X-OriginatorOrg: juniper.net Cc: Marcel Moolenaar , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 22:50:36 -0000 Hi John, The first email in this thread from Marcel has the schedule: On 5/28/14, 9:34 AM, "Marcel Moolenaar" wrote: <=8Bsnip=8B> Schedule: late May 2014 Juniper adds accessor methods. Access via void *, that inside if.c is re-casted to (struct ifnet *). This =B3ugly=B2 moment is required to keep unconverted drivers compilable and will go away shortly after. 1st week June 2014 Juniper commits few converted drivers. June - July All drivers are being converted. Gleb volunteers, but this should be done by as many hands as possible. August 2014 '#define if_t void *' is removed. 'typedef if_t struct ifnet *' is added. actual struct ifnet is hidden. 2014/2015 Better APIs developed and improved, actual ifnet representation becomes variable sized. New networking stack improvements are already unblocked at this moment. On 6/2/14, 3:38 PM, "John-Mark Gurney" wrote: >Gleb Smirnoff wrote this message on Thu, May 29, 2014 at 08:04 +0400: >> On Wed, May 28, 2014 at 08:48:11PM -0700, Rui Paulo wrote: >> R> On May 28, 2014, at 9:34, Marcel Moolenaar wrote: >> R>=20 >> R> > All, >> R> >=20 >> R> > The ifnet structure represents a network interface. Right now it >> R> > is known to all NIC drivers as well as to all protocols. This >> R> > means that whenever we change the structure, we need at minimum >> R> > recompile all drivers, but usually also change them. This severely >> R> > slow downs the development in this area and also makes it hard, if >> R> > not, impossible to merge things back to stable branches. >> R> >=20 >> R> > There were at least 3 efforts on fixing this: >> R> >=20 >> R> > 1) Juniper???s JUNOS is a FreeBSD based operating system that has >> R> > its own (alternative) network stack, but that leverages the >> R> > network drivers from FreeBSD. Juniper mechanically changed all >> R> > ifnet dereferences to to accessor methods. This could have >> R> > been incorporated as early as 2011, but lacked good follow >> R> > through. Marcel Moolenaar was prime contact for this. >> R> >=20 >> R> > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD >> R> > Foundation to make ifnet(9) opaque. This is not complete as of >> R> > the time of this writing. >> R> >=20 >> R> > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but >> R> > that always has been delayed due to 1) and 2). >> R>=20 >> R> This is indeed needed, but it would be nice to understand what would >>happen if the community has comments about your patch. Will Juniper be >>able to integrate back those comments? For example, I think the type >>"if_t" should be "ifnet_t". Another comment I have is: why do you have >>to cast if_t to (struct ifnet *) in all the accessor methods? It would >>be better to create a private header typedef'ing if_t to struct ifnet, >>avoiding the copy & paste casting. >>=20 >> Because for now patch supports compiling unconverted drivers. This >>requires >> to pass 'void *' as argument. When all drivers are converted, this will >>be >> removed. In if_var.h we would have: >>=20 >> struct ifnet; >> typedef struct ifnet * if_t; >>=20 >> All casts to (struct ifnet *) will go away. > >When? The schedule did not have a time frame on when this will happen.. >Will it be this year? Next year? Next decade? > >--=20 > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Mon Jun 2 23:07:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9733184E; Mon, 2 Jun 2014 23:07:34 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D5012EB1; Mon, 2 Jun 2014 23:07:34 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s52N7XLO027432 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Jun 2014 16:07:33 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s52N7X2d027431; Mon, 2 Jun 2014 16:07:33 -0700 (PDT) (envelope-from jmg) Date: Mon, 2 Jun 2014 16:07:33 -0700 From: John-Mark Gurney To: Anuranjan Shukla Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Message-ID: <20140602230733.GD43976@funkthat.com> Mail-Followup-To: Anuranjan Shukla , Gleb Smirnoff , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" , Marcel Moolenaar References: <20140529040425.GT50679@glebius.int.ru> <20140602223801.GC43976@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 02 Jun 2014 16:07:33 -0700 (PDT) Cc: Marcel Moolenaar , Gleb Smirnoff , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 23:07:34 -0000 Anuranjan Shukla wrote this message on Mon, Jun 02, 2014 at 22:50 +0000: > The first email in this thread from Marcel has the schedule: > > On 5/28/14, 9:34 AM, "Marcel Moolenaar" wrote: > > > Schedule: > > late May 2014 Juniper adds accessor methods. Access via > void *, that inside if.c is re-casted to > (struct ifnet *). This ³ugly² moment is > required to keep unconverted drivers > compilable and will go away shortly after. So, shortly... My definition of shortly is three months... So, will that happen? There are an awful lot of drivers to convert and test, in a short amount of time... The reason I'm asking is that often people say, oh, it'll happen shortly w/ things like docs, code fixes, etc, and then 2 years go by and nothing happens... I know that converting ALL drivers is not a simple matter considering how many different drivers we have in the tree, so shortly seems optimistic to me... > On 6/2/14, 3:38 PM, "John-Mark Gurney" wrote: > > >Gleb Smirnoff wrote this message on Thu, May 29, 2014 at 08:04 +0400: > >> On Wed, May 28, 2014 at 08:48:11PM -0700, Rui Paulo wrote: > >> R> On May 28, 2014, at 9:34, Marcel Moolenaar wrote: > >> R> > >> R> > All, > >> R> > > >> R> > The ifnet structure represents a network interface. Right now it > >> R> > is known to all NIC drivers as well as to all protocols. This > >> R> > means that whenever we change the structure, we need at minimum > >> R> > recompile all drivers, but usually also change them. This severely > >> R> > slow downs the development in this area and also makes it hard, if > >> R> > not, impossible to merge things back to stable branches. > >> R> > > >> R> > There were at least 3 efforts on fixing this: > >> R> > > >> R> > 1) Juniper???s JUNOS is a FreeBSD based operating system that has > >> R> > its own (alternative) network stack, but that leverages the > >> R> > network drivers from FreeBSD. Juniper mechanically changed all > >> R> > ifnet dereferences to to accessor methods. This could have > >> R> > been incorporated as early as 2011, but lacked good follow > >> R> > through. Marcel Moolenaar was prime contact for this. > >> R> > > >> R> > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD > >> R> > Foundation to make ifnet(9) opaque. This is not complete as of > >> R> > the time of this writing. > >> R> > > >> R> > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but > >> R> > that always has been delayed due to 1) and 2). > >> R> > >> R> This is indeed needed, but it would be nice to understand what would > >>happen if the community has comments about your patch. Will Juniper be > >>able to integrate back those comments? For example, I think the type > >>"if_t" should be "ifnet_t". Another comment I have is: why do you have > >>to cast if_t to (struct ifnet *) in all the accessor methods? It would > >>be better to create a private header typedef'ing if_t to struct ifnet, > >>avoiding the copy & paste casting. > >> > >> Because for now patch supports compiling unconverted drivers. This > >>requires > >> to pass 'void *' as argument. When all drivers are converted, this will > >>be > >> removed. In if_var.h we would have: > >> > >> struct ifnet; > >> typedef struct ifnet * if_t; > >> > >> All casts to (struct ifnet *) will go away. > > > >When? The schedule did not have a time frame on when this will happen.. > >Will it be this year? Next year? Next decade? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Mon Jun 2 23:10:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 820FA902; Mon, 2 Jun 2014 23:10:11 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 57CDA2EBE; Mon, 2 Jun 2014 23:10:10 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s52NA96m027499 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 2 Jun 2014 16:10:10 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s52NA9ji027498; Mon, 2 Jun 2014 16:10:09 -0700 (PDT) (envelope-from jmg) Date: Mon, 2 Jun 2014 16:10:09 -0700 From: John-Mark Gurney To: Anuranjan Shukla Subject: Re: Roadmap for ifnet(9) for FreeBSD 11 Message-ID: <20140602231009.GE43976@funkthat.com> Mail-Followup-To: Anuranjan Shukla , Gleb Smirnoff , Marcel Moolenaar , Rui Paulo , "freebsd-arch@FreeBSD.org Arch" References: <20140529040425.GT50679@glebius.int.ru> <20140602223801.GC43976@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Mon, 02 Jun 2014 16:10:10 -0700 (PDT) Cc: "freebsd-arch@FreeBSD.org Arch" , Gleb Smirnoff , Rui Paulo , Marcel Moolenaar X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jun 2014 23:10:11 -0000 Anuranjan Shukla wrote this message on Mon, Jun 02, 2014 at 22:50 +0000: > Hi John, > The first email in this thread from Marcel has the schedule: Weird, I read the schedule twice and still missed this: > August 2014 '#define if_t void *' is removed. > 'typedef if_t struct ifnet *' is added. > actual struct ifnet is hidden. My apologies... > On 6/2/14, 3:38 PM, "John-Mark Gurney" wrote: > > >Gleb Smirnoff wrote this message on Thu, May 29, 2014 at 08:04 +0400: > >> On Wed, May 28, 2014 at 08:48:11PM -0700, Rui Paulo wrote: > >> R> On May 28, 2014, at 9:34, Marcel Moolenaar wrote: > >> R> > >> R> > All, > >> R> > > >> R> > The ifnet structure represents a network interface. Right now it > >> R> > is known to all NIC drivers as well as to all protocols. This > >> R> > means that whenever we change the structure, we need at minimum > >> R> > recompile all drivers, but usually also change them. This severely > >> R> > slow downs the development in this area and also makes it hard, if > >> R> > not, impossible to merge things back to stable branches. > >> R> > > >> R> > There were at least 3 efforts on fixing this: > >> R> > > >> R> > 1) Juniper???s JUNOS is a FreeBSD based operating system that has > >> R> > its own (alternative) network stack, but that leverages the > >> R> > network drivers from FreeBSD. Juniper mechanically changed all > >> R> > ifnet dereferences to to accessor methods. This could have > >> R> > been incorporated as early as 2011, but lacked good follow > >> R> > through. Marcel Moolenaar was prime contact for this. > >> R> > > >> R> > 2) Andre Oppermann was sponsored in 2013 by the FreeBSD > >> R> > Foundation to make ifnet(9) opaque. This is not complete as of > >> R> > the time of this writing. > >> R> > > >> R> > 3) Gleb Smirnoff also planned to work on opaque ifnet(9), but > >> R> > that always has been delayed due to 1) and 2). > >> R> > >> R> This is indeed needed, but it would be nice to understand what would > >>happen if the community has comments about your patch. Will Juniper be > >>able to integrate back those comments? For example, I think the type > >>"if_t" should be "ifnet_t". Another comment I have is: why do you have > >>to cast if_t to (struct ifnet *) in all the accessor methods? It would > >>be better to create a private header typedef'ing if_t to struct ifnet, > >>avoiding the copy & paste casting. > >> > >> Because for now patch supports compiling unconverted drivers. This > >>requires > >> to pass 'void *' as argument. When all drivers are converted, this will > >>be > >> removed. In if_var.h we would have: > >> > >> struct ifnet; > >> typedef struct ifnet * if_t; > >> > >> All casts to (struct ifnet *) will go away. > > > >When? The schedule did not have a time frame on when this will happen.. > >Will it be this year? Next year? Next decade? > > > >-- > > John-Mark Gurney Voice: +1 415 225 5579 > > > > "All that I will do, has been done, All that I have, has not." > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Thu Jun 5 07:54:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DA28581 for ; Thu, 5 Jun 2014 07:54:26 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx1.freebsd.org (Postfix) with ESMTP id 67D222AEB for ; Thu, 5 Jun 2014 07:54:26 +0000 (UTC) Received: from mx36.phplist.com (mx36.phplist.com [50.23.59.119]) by mx36.phplist.com (Postfix) with ESMTP id 30FCC1206A for ; Thu, 5 Jun 2014 08:48:14 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=phplist.com; h=date:to :from:reply-to:subject:message-id:list-unsubscribe:mime-version :content-type:content-transfer-encoding; s=s0; bh=35Eu1lAvSqkpry 01KzNE1nWmQ5Y=; b=D7fnMZVTHZXY8bbyEnuqcSmWzS/RBO5iQ6b4ZYe5igi0hN wQP9NVBtYezCnOY/jCKM1L6+V/PANjcwi6AipzEQPCB0DXDx8RrdkzPZZd1T9Y8t mldQQGDEAufvRY7VAMbJRvir/+ExCn5d2PEpA0tsbNuHjFvBs0jvpIxNq5KVg= Received: from thomas.hosted.phplist.com (mimosa [184.173.18.3]) by mx36.phplist.com (Postfix) with ESMTP id 2258412068 for ; Thu, 5 Jun 2014 08:48:14 +0100 (BST) Received: from 195-154-185-238.rev.poneytelecom.eu [195.154.185.238] by thomas.hosted.phplist.com with HTTP; Thu, 05 Jun 2014 07:48:13 +0000 Date: Thu, 5 Jun 2014 07:48:14 +0000 To: freebsd-arch@freebsd.org From: Silverjewelryworld Reply-To: Silverjewelryworld Subject: Goodbye from our Newsletter Message-ID: X-Priority: 3 X-Mailer: PHPMailer 5.2.5 (https://github.com/Synchro/PHPMailer/) X-phpList-version: 3.0.5-hosted X-MessageID: systemmessage X-ListMember: freebsd-arch@freebsd.org Precedence: bulk Bounces-To: phlistbounces-thomas@phplist.com MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jun 2014 07:54:26 -0000 =20 Goodbye from our Newsletter, sorry to see you go. You have been unsubscribed from our newsletters. This is the last email you will receive from us. We have added you to o= ur "blacklist", which means that our newsletter system will refuse to send you any other email, without manual intervention by our administrator. If there is an error in this information, you can re-subscribe: please go to http://thomas.hosted.phplist.com/lists/?p=3Dsubscribe and follow the steps. Thank you =20 =20 From owner-freebsd-arch@FreeBSD.ORG Sat Jun 7 01:16:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75EE5624; Sat, 7 Jun 2014 01:16:20 +0000 (UTC) Received: from mail-qg0-x234.google.com (mail-qg0-x234.google.com [IPv6:2607:f8b0:400d:c04::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15E47251F; Sat, 7 Jun 2014 01:16:20 +0000 (UTC) Received: by mail-qg0-f52.google.com with SMTP id a108so6105032qge.11 for ; Fri, 06 Jun 2014 18:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type:content-transfer-encoding; bh=QP345DZG4URYtqtQyrmPwJB9IgIGioA+EI0e1no4sPw=; b=AtL/AiB3WHZUMhPha+ACcrdZFpm5FKwdB0TXcVqOfj8ONaybyd5lGm+c0TmBPHHWwg EQsiG0k7KaXKYY0+rS2g/AGBn0KzFvptQZh9ZBCNulDVaf/Wn2P6XJcowckANtloOvAK X5t5hDkBMCFnuUqbJ7lkHYRNpzVYR+wDXw02NdTzPbuSdHLvuRU7wFrOMqAaPQmchKCe KkHebkSg6EPrZF7UkrR7ZjLT3WXCzRu3zzA8EGZBA2CKjATT8MDTVNeaVRTzl+K0c5Zq NfdqKQq5W3olYYmR3wjahMkI4cVk7gkEUEGTfxZikFgneunNPLpBEBdiZULMcY/BIMvj m5xw== MIME-Version: 1.0 X-Received: by 10.229.57.129 with SMTP id c1mr14201796qch.7.1402103779207; Fri, 06 Jun 2014 18:16:19 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Fri, 6 Jun 2014 18:16:19 -0700 (PDT) In-Reply-To: References: Date: Fri, 6 Jun 2014 18:16:19 -0700 X-Google-Sender-Auth: TQNFhzv309iVcoy_ZokfndYNyyk Message-ID: Subject: Re: Seeing ENOTCAPABLE from write FDs in kqueue? From: Adrian Chadd To: Robert Watson , Pawel Jakub Dawidek , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jun 2014 01:16:20 -0000 ping? -a On 2 June 2014 10:58, Adrian Chadd wrote: > Hi guys/all, > > I've been tinkering with my RSS stuff where I have multiple listen > sockets, one per worker thread, all terminating high throughput TCP > transactions. It's all HTTP; I'm using libevent, evhttp and a very > small amount of glue code to achieve this. > > libevent craps out from time to time because occasionally one of the > events in kqueue returns ENOTCAPABLE. > > ENOTCAPABLE: ident=3D1686, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1324, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1740, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1628, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D1199, filter=3D-2, flags=3D0x00004000, fflags=3D0x00= 000000, data=3D93 > > ENOTCAPABLE: ident=3D818, filter=3D-2, flags=3D0x00004000, fflags=3D0x000= 00000, data=3D93 > > ENOTCAPABLE: ident=3D389, filter=3D-2, flags=3D0x00004000, fflags=3D0x000= 00000, data=3D93 > > It's happening on the various data FDs; not on the listen socket. > > What I'm seeing from ktrace: > > 27770 rss-http CAP operation requires , descriptor holds > > > 27770 rss-http CAP operation requires , descriptor holds <> > > 27770 rss-http CAP operation requires , descriptor holds <> > > 27770 rss-http CAP operation requires , descriptor holds <> > > 27770 rss-http CAP operation requires , descriptor holds <> > > 27770 rss-http CAP operation requires , descriptor holds <> > > .. so, why exactly would I be seeing this? Is this some capability > race where we have an FD setup but no capability yet assigned when > it's added into the kqueue set? > > It's happening under high throughput (> 30,000 TCP sessions a second.) > > Where would I continue debugging this? > > Thanks! > > > -a From owner-freebsd-arch@FreeBSD.ORG Sun Jun 8 22:00:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02DBE974; Sun, 8 Jun 2014 22:00:07 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23B7B2CA0; Sun, 8 Jun 2014 22:00:06 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id k14so3973477wgh.18 for ; Sun, 08 Jun 2014 15:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=YItOmMOBFQoj6QJKgpABX4aouSJ493PYBvMA+iKYx60=; b=ivue3Fh3yzI33NcYfjY/1NiQnhRZ9vD9y8cQ+Gc6pBS1UQNhp0qIBHLNwJ+54GigI2 JQTIIfYkwiLlZwaISYwe/8fYLvS/OF/ZzCuv3xII0/Cvbj9Axzhqnkhb5yNhuCDS3JoR o57kX5s/ISgZKEWJ0h8vW7H93WjrbFl28Ew+UG95v6x3ciOA4X6tlH6jNNvq4yRqa1f7 fcLEOqReOrMYjLF+SLv88JoqZG1XFvN9luQ98gBAyrct4wCQoBGOkx7SFfjZ102dZdi/ J4CZTPevtkx6wfqV9MSFD8nr+FJvm92LnIqdngd3Gnj9c5RVsRMgyYHGQA3qVmpO9WUE t+Gw== X-Received: by 10.194.87.200 with SMTP id ba8mr26337847wjb.28.1402264804463; Sun, 08 Jun 2014 15:00:04 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id ft17sm22223760wjc.14.2014.06.08.15.00.02 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 08 Jun 2014 15:00:03 -0700 (PDT) Date: Mon, 9 Jun 2014 00:00:00 +0200 From: Mateusz Guzik To: Adrian Chadd Subject: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) Message-ID: <20140608220000.GA5030@dft-labs.eu> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 22:00:07 -0000 Current support for capabilities is racy in respect to somewhat misbehaving programs. When fp is being installed under given fd number it is available before capabilities are copied. This is because fget_unlocked only gets fp atomatically, but caps can be modified irrespectively to that. This has 2 problems: - if the code is trying to access fd before the syscall returns, it may get ENOTCAPABLE (minor) - if the code is racing with dup2 it can access given fp in a way prevented by not-yet-installed caps (needs fixing) In url below I provide two reproducers: http://people.freebsd.org/~mjg/patches/caps-race/ One will open+close in 1 thread + read in 7 threads trying to get ENOTCAPABLE. Second one will create a fd with stripped caps and dup2 capped/uncapped fd in 1 thread + read in 7 threads trying to succeed. I propose to fix the problem by introducing sequence counters. I tested the patch below succesfully with my reproducers. Note that the patch is somewhat incomplete (ioctls are not handled) and has stuff in wrong places, I can do necessary tidy ups if it is agreed the approach taken is ok. This was not benchmarked yet, but should still beat shared locking :) diff --git a/sys/kern/kern_descrip.c b/sys/kern/kern_descrip.c index 88b26af..25e4a98 100644 --- a/sys/kern/kern_descrip.c +++ b/sys/kern/kern_descrip.c @@ -131,6 +131,51 @@ static int fill_socket_info(struct socket *so, struct kinfo_file *kif); static int fill_vnode_info(struct vnode *vp, struct kinfo_file *kif); static int getmaxfd(struct proc *p); + +static inline void +seq_begin(seq_t *seqp) +{ + + MPASS((*seqp & 1) == 0); + (*seqp)++; + wmb(); +} + +static inline void +seq_end(seq_t *seqp) +{ + + wmb(); + (*seqp)++; + MPASS((*seqp & 1) == 0); +} + +static inline seq_t +seq_read(seq_t *seqp) +{ + seq_t ret; + + ret = *seqp; + rmb(); + + return (ret); +} + +static inline bool +seq_in_modify(seq_t seq) +{ + + return (seq & 1); +} + +static inline bool +seq_changed(seq_t seq1, seq_t seq2) +{ + + MPASS(!seq_in_modify(seq1)); + return (seq1 != seq2); +} + /* * Each process has: * @@ -301,9 +346,11 @@ fdfree(struct filedesc *fdp, int fd) struct filedescent *fde; fde = &fdp->fd_ofiles[fd]; + seq_begin(&fde->fde_seq); filecaps_free(&fde->fde_caps); - bzero(fde, sizeof(*fde)); + bzero(fde_change(fde), fde_change_size); fdunused(fdp, fd); + seq_end(&fde->fde_seq); } /* @@ -878,8 +925,9 @@ do_dup(struct thread *td, int flags, int old, int new, /* * Duplicate the source descriptor. */ + seq_begin(&newfde->fde_seq); filecaps_free(&newfde->fde_caps); - *newfde = *oldfde; + memcpy(fde_change(newfde), fde_change(oldfde), fde_change_size); filecaps_copy(&oldfde->fde_caps, &newfde->fde_caps); if ((flags & DUP_CLOEXEC) != 0) newfde->fde_flags = oldfde->fde_flags | UF_EXCLOSE; @@ -887,6 +935,7 @@ do_dup(struct thread *td, int flags, int old, int new, newfde->fde_flags = oldfde->fde_flags & ~UF_EXCLOSE; if (new > fdp->fd_lastfile) fdp->fd_lastfile = new; + seq_end(&newfde->fde_seq); *retval = new; if (delfp != NULL) { @@ -1753,6 +1802,7 @@ finstall(struct thread *td, struct file *fp, int *fd, int flags, } fhold(fp); fde = &fdp->fd_ofiles[*fd]; + seq_begin(&fde->fde_seq); fde->fde_file = fp; if ((flags & O_CLOEXEC) != 0) fde->fde_flags |= UF_EXCLOSE; @@ -1760,6 +1810,7 @@ finstall(struct thread *td, struct file *fp, int *fd, int flags, filecaps_move(fcaps, &fde->fde_caps); else filecaps_fill(&fde->fde_caps); + seq_end(&fde->fde_seq); FILEDESC_XUNLOCK(fdp); return (0); } @@ -2313,6 +2364,8 @@ fget_unlocked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, cap_rights_t haverights; int error; #endif + struct filedescent fde; + int seq1, seq2; /* * Avoid reads reordering and then a first access to the @@ -2329,11 +2382,18 @@ fget_unlocked(struct filedesc *fdp, int fd, cap_rights_t *needrightsp, * due to preemption. */ for (;;) { - fp = fdp->fd_ofiles[fd].fde_file; + seq1 = seq_read(&fdp->fd_ofiles[fd].fde_seq); + if (seq_in_modify(seq1)) + continue; + fde = fdp->fd_ofiles[fd]; + seq2 = seq_read(&fdp->fd_ofiles[fd].fde_seq); + if (seq_changed(seq1, seq2)) + continue; + fp = fde.fde_file; if (fp == NULL) return (EBADF); #ifdef CAPABILITIES - haverights = *cap_rights(fdp, fd); + haverights = fde.fde_rights; if (needrightsp != NULL) { error = cap_check(&haverights, needrightsp); if (error != 0) diff --git a/sys/sys/filedesc.h b/sys/sys/filedesc.h index 516ef1b..84b38f2 100644 --- a/sys/sys/filedesc.h +++ b/sys/sys/filedesc.h @@ -42,6 +42,8 @@ #include +typedef uint32_t seq_t; + struct filecaps { cap_rights_t fc_rights; /* per-descriptor capability rights */ u_long *fc_ioctls; /* per-descriptor allowed ioctls */ @@ -50,6 +52,7 @@ struct filecaps { }; struct filedescent { + seq_t fde_seq; struct file *fde_file; /* file structure for open file */ struct filecaps fde_caps; /* per-descriptor rights */ uint8_t fde_flags; /* per-process open file flags */ @@ -58,6 +61,8 @@ struct filedescent { #define fde_fcntls fde_caps.fc_fcntls #define fde_ioctls fde_caps.fc_ioctls #define fde_nioctls fde_caps.fc_nioctls +#define fde_change(fde) ((char *)(fde) + sizeof(seq_t)) +#define fde_change_size (sizeof(struct filedescent) - sizeof(seq_t)) /* * This structure is used for the management of descriptors. It may be -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Jun 8 23:01:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A11A726C; Sun, 8 Jun 2014 23:01:05 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0ACC216F; Sun, 8 Jun 2014 23:01:04 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id bs8so3408470wib.0 for ; Sun, 08 Jun 2014 16:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=+jd8s9noM/r+xbXZ96bheCPPHxmwrp3C3DBbPJ/mNGI=; b=XBybRoTp+QUTYUeIDFJGpFvtky9hksvM4op8H0e2w1IidbkvxXD8BNOF7TpJpYWBiE vxoToSFWg7/hCz6Es/EjP3JdQNvEXpMN7XW1UWX0gr7FSXIYJ1QLry0tgr7ZxhR3XVUi R53YZ6rZKsHAQADYL/FEv0xGV/kKlOeXymQ3ARSMh5L3DOVYjgEtxIW2x6v6GjNZhhIC i4UNHrZbxCks+DyKKjop9zGbzYMatJiPIcAQ92R7pAr33+NvM+OygsGTfOLqSTEN07lQ qGESmquSB97SJrvhQ/a46feLhKY5M7g99ZKscMAcb+WcZGBOzIvADi25MLWZjol8pmL2 O2jg== X-Received: by 10.180.38.38 with SMTP id d6mr24009720wik.12.1402268462018; Sun, 08 Jun 2014 16:01:02 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id s10sm22386691wjs.29.2014.06.08.16.01.00 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 08 Jun 2014 16:01:01 -0700 (PDT) Date: Mon, 9 Jun 2014 01:00:59 +0200 From: Mateusz Guzik To: Adrian Chadd Subject: Re: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) Message-ID: <20140608230059.GB5030@dft-labs.eu> References: <20140608220000.GA5030@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140608220000.GA5030@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 23:01:05 -0000 On Mon, Jun 09, 2014 at 12:00:00AM +0200, Mateusz Guzik wrote: > Current support for capabilities is racy in respect to somewhat > misbehaving programs. > > When fp is being installed under given fd number it is available before > capabilities are copied. > > This is because fget_unlocked only gets fp atomatically, but caps can be > modified irrespectively to that. > > This has 2 problems: > - if the code is trying to access fd before the syscall returns, it may > get ENOTCAPABLE (minor) > - if the code is racing with dup2 it can access given fp in a way > prevented by not-yet-installed caps (needs fixing) > > In url below I provide two reproducers: > http://people.freebsd.org/~mjg/patches/caps-race/ > > One will open+close in 1 thread + read in 7 threads trying to get > ENOTCAPABLE. > > Second one will create a fd with stripped caps and dup2 capped/uncapped > fd in 1 thread + read in 7 threads trying to succeed. > > I propose to fix the problem by introducing sequence counters. > > I tested the patch below succesfully with my reproducers. > > Note that the patch is somewhat incomplete (ioctls are not handled) and > has stuff in wrong places, I can do necessary tidy ups if it is agreed > the approach taken is ok. > [..] > +static inline seq_t > +seq_read(seq_t *seqp) > +{ > + seq_t ret; > + > + ret = *seqp; > + rmb(); > + > + return (ret); > +} Doh, rmb() should be before read. As you can see my memory-barrier-fu is not too strong. > for (;;) { > - fp = fdp->fd_ofiles[fd].fde_file; > + seq1 = seq_read(&fdp->fd_ofiles[fd].fde_seq); > + if (seq_in_modify(seq1)) > + continue; ... and this should try to yield some cpu cycles (pause?) So, benchmarks: http://people.freebsd.org/~mjg/patches/caps-race/dup2-bench.c 1 thread does dup2 in a loop 7 threads fget() the same fd and fail quickly on a crappy 8-way machine I get the following: x with-barriers-ops-2 + without-barriers-ops +--------------------------------------------------------------------------------------------------------------------------------------+ | x + | | x + + ++ | | xxxx + ++++ | | xxxxxx + +++++ | | x xxxxxx ++ +++++ | | xx xxxxxx +++ +++++ | | xx xxxxxxxx x + +++ +++++ | | xxxxxxxxxxxx x + + +++++++++++ + | |xx xx xxxxxxxxxxxx x x x x +++ + +++++++++++ ++ + | |xx xxxxxxxxxxxxxxx x x xxx x xxx x x ++++++++ +++++++++++ +++ ++ + ++ +| | |_______MA________| |______A______| | +--------------------------------------------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 100 11735712 12597730 12040847 12072225 186613.68 + 100 13732889 14499008 14051852 14042094 145074.11 Difference at 95.0% confidence 1.96987e+06 +/- 46328.7 16.3174% +/- 0.383763% (Student's t, pooled s = 167139) It would be good if someone with some time and access to higher-cpu-count machine could test it better. -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Jun 8 23:44:33 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1221D9BA for ; Sun, 8 Jun 2014 23:44:33 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE8D42465 for ; Sun, 8 Jun 2014 23:44:32 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s58NiWRj077842 for ; Mon, 9 Jun 2014 00:44:32 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-arch@FreeBSD.org Subject: [Bug 121073] [kernel] [patch] run chroot as an unprivileged user Date: Sun, 08 Jun 2014 23:44:33 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eadler@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2014 23:44:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=121073 Eitan Adler changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd-arch@FreeBSD.org -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-arch@FreeBSD.ORG Tue Jun 10 13:57:57 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F5A2EB for ; Tue, 10 Jun 2014 13:57:57 +0000 (UTC) Received: from main.vmx.hu (178-248-200-106.giganet.hu [178.248.200.106]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4150828C9 for ; Tue, 10 Jun 2014 13:57:56 +0000 (UTC) Received: from csernovicsgyula by main.vmx.hu with local (Exim 4.80) (envelope-from ) id 1WuMYE-00007D-DL for arch@freebsd.org; Tue, 10 Jun 2014 15:57:22 +0200 To: arch@freebsd.org Subject: RFQ From: Chris Tellis Reply-To: betabattery@gmail.com MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Message-Id: Date: Tue, 10 Jun 2014 15:57:22 +0200 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 13:57:57 -0000 Hello, Please quote us for the below listed items. - Solar Panel Module 240/250w - Sealed Lead Acid Battery 12v 100-200aH - 11 SQF-2 Grundfos Solar Pump Also, advice on your acceptable methods of payment and stock availability. Thank You Chris Tellis Beta Battery From owner-freebsd-arch@FreeBSD.ORG Tue Jun 10 16:02:15 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F5963CB for ; Tue, 10 Jun 2014 16:02:15 +0000 (UTC) Received: from land.berklix.org (land.berklix.org [144.76.10.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9E4DA2591 for ; Tue, 10 Jun 2014 16:02:14 +0000 (UTC) Received: from mart.js.berklix.net (pD9FBE1BF.dip0.t-ipconnect.de [217.251.225.191]) (authenticated bits=128) by land.berklix.org (8.14.5/8.14.5) with ESMTP id s5AG0ukG083375 for ; Tue, 10 Jun 2014 16:00:57 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id s5AG23ME036588 for ; Tue, 10 Jun 2014 18:02:04 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.7/8.14.7) with ESMTP id s5AG1pOa013266 for ; Tue, 10 Jun 2014 18:02:03 +0200 (CEST) (envelope-from jhs@berklix.com) Message-Id: <201406101602.s5AG1pOa013266@fire.js.berklix.net> Subject: Re: RFQ From: "Julian H. Stacey" Organization: http://berklix.com BSD Unix Linux Consultants, Munich Germany User-agent: EXMH on FreeBSD http://berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Tue, 10 Jun 2014 15:57:22 +0200." Date: Tue, 10 Jun 2014 18:01:51 +0200 Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jun 2014 16:02:15 -0000 Hi, Reference: > From: Chris Tellis > Reply-to: betabattery@gmail.com > Date: Tue, 10 Jun 2014 15:57:22 +0200 Chris Tellis wrote: > Hello, > > Please quote us for the below listed items. > > - Solar Panel Module 240/250w > > - Sealed Lead Acid Battery 12v 100-200aH > > - 11 SQF-2 Grundfos Solar Pump > > Also, advice on your acceptable methods of payment and stock availability. > > Thank You > Chris Tellis > Beta Battery It's just another spammer to not reply to. I just got this as personal mail too. Cheers, Julian -- Julian Stacey, BSD Linux Unix C Sys Eng Consultant Munich http://berklix.com Interleave replies Below, like a play script. From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 18:53:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 857DD858; Wed, 11 Jun 2014 18:53:23 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8BFB2846; Wed, 11 Jun 2014 18:53:22 +0000 (UTC) Received: by mail-lb0-f178.google.com with SMTP id w7so92521lbi.23 for ; Wed, 11 Jun 2014 11:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:cc:content-type; bh=XyHpVN1uLydaUxAjuDK1fPZd5lVfXMXq0rawcIqqNa0=; b=g+TGWPFID1cESnUC0EorbQqEKVIWdYokb7HY0x2mOQmY5B7GFRpoHiWU8f5hOCFZ2y n3Oo+0na/tnxuCO2rJl9fUQjkEFovjXGzQsUd+qBDg5H/tQkjXQ7VY8M8ipTBcakwSBy hc17wJiM9ZFmG3xo6aKBFm/XvHzPnrWII8eoW7g4azYMOA3GdoqfvpF5H/QRJubWkalh sH2TCCkTXUkJ9xikvbzQsFpNEj6kTf/B3EUzxPi/QopgjQCYpTqAp6xZRYujZDfDdiqg KEo8nbLFgYfU4iE5uHYFl0gGo+r/2++BAoVEQNZK/DRgRywyM/rP2lhid/04abxLwJxE GOAg== MIME-Version: 1.0 X-Received: by 10.152.10.40 with SMTP id f8mr2691165lab.75.1402512800426; Wed, 11 Jun 2014 11:53:20 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 11:53:20 -0700 (PDT) Date: Wed, 11 Jun 2014 11:53:20 -0700 X-Google-Sender-Auth: 7iH_fC4dER1xYk2-3F0q1VaNaFw Message-ID: Subject: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Craig Rodrigues To: freebsd-arch Content-Type: text/plain; charset=ISO-8859-1 Cc: Glen Barber , freebsd-toolchain@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 18:53:23 -0000 Hi, Recently when trying to debug some coredumps in CURRENT from a userland process in the devel/libvirt port, I found that the gdb in base could not get a backtrace from the core file: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606.html I needed to add to my /etc/src.conf WITH_LLDB=yes and compile and install lldb, which *could* analyze the coredump. I realize that gdb in base is on its way out the door, and that clang and lldb are the new world order. However, one of the main blockers for removing gdb from base is that there is no equivalent to kgdb in the lldb world. Is anyone working on a kernel lldb? When is it expected to be ready? Will something like it be ready by end of 2014? Even though this may not be an issue in the stable/10 branch, it would be nice to have a kernel lldb debugger ready by the next 10.1R if possible. -- Craig From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 19:31:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9297F77D; Wed, 11 Jun 2014 19:31:20 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDDE2BE1; Wed, 11 Jun 2014 19:31:20 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::c5cc:f189:593d:8554] (unknown [IPv6:2001:7b8:3a7:0:c5cc:f189:593d:8554]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 1AF7C5C37; Wed, 11 Jun 2014 21:31:10 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_C95AB983-3A47-4638-B540-03A1056D9D33"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Dimitry Andric In-Reply-To: Date: Wed, 11 Jun 2014 21:30:55 +0200 Message-Id: References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1878.2) Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:31:20 -0000 --Apple-Mail=_C95AB983-3A47-4638-B540-03A1056D9D33 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 11 Jun 2014, at 20:53, Craig Rodrigues wrote: >=20 > Recently when trying to debug some coredumps in CURRENT from > a userland process in the devel/libvirt port, I found that the gdb in > base could not get a backtrace from the core file: >=20 > = http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606= .html Can you please post the output of the following? objdump -W /usr/local/sbin/libvirtd | head -Dimitry --Apple-Mail=_C95AB983-3A47-4638-B540-03A1056D9D33 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlOYrn0ACgkQsF6jCi4glqNUeACfQyrdsQXUv+fGLy6b6OjhSm/d lU4AoJelryqilH+pDy2GWLBckl45JJmc =c6iW -----END PGP SIGNATURE----- --Apple-Mail=_C95AB983-3A47-4638-B540-03A1056D9D33-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 19:34:19 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32E579C1 for ; Wed, 11 Jun 2014 19:34:19 +0000 (UTC) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F21E72C92 for ; Wed, 11 Jun 2014 19:34:18 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id ft15so132960pdb.39 for ; Wed, 11 Jun 2014 12:34:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=m93boXdfh/qmG2iPsPngp/4xeXwSKhcZgvcx8HhEFAU=; b=M/eb8wlBzubMEntl7vjeLe0246n0rdHG75f0Ho9+vB6C3g6GvLBFqDi4I2CuC4xnmn QuxyZsEDrIzJXgKRKQuI0l6g09IKo1SqDlf9D4e3elvZaxIZr8mBDpj7jUHYVyrgeKUV 79jAFudWvY2L/AxZ1DMNqvNm/Tgl+ks0eoEhlKfT7vvNrQ1SPQFkHAQkXYlQAiw22U/L qppO+XwP/6ifS8S7aETlwtgSfxIxvo6aA3jcXnps36bWl3f9xTsawjJiLTB12bXwDizK HMNL1FBIIN6DeJV/MFqtOv1h+n5J0iqMBS7E7dNLIGzfmzri2AovVluEVkyWvHRLrXlW QbGw== X-Gm-Message-State: ALoCoQnxsjU5DDiAeeKbsUAQo9EeWPuwjR8ZfrgKe4bcgmaBsf5KY2tlNPsbqK8FH12jar9YqTCF X-Received: by 10.66.240.130 with SMTP id wa2mr15768891pac.73.1402515252675; Wed, 11 Jun 2014 12:34:12 -0700 (PDT) Received: from lgmac-cvenus.corp.netflix.com (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id is5sm76471126pbb.8.2014.06.11.12.34.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 11 Jun 2014 12:34:12 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_90641765-7931-4A5F-ABC6-B3746065B63A"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Warner Losh In-Reply-To: Date: Wed, 11 Jun 2014 13:34:15 -0600 Message-Id: <33D37CFE-CA6B-4F89-9178-ACF2352CC6C5@bsdimp.com> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1878.2) Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:34:19 -0000 --Apple-Mail=_90641765-7931-4A5F-ABC6-B3746065B63A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Jun 11, 2014, at 12:53 PM, Craig Rodrigues = wrote: > Hi, >=20 > Recently when trying to debug some coredumps in CURRENT from > a userland process in the devel/libvirt port, I found that the gdb in > base could not get a backtrace from the core file: >=20 > = http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606= .html >=20 > I needed to add to my /etc/src.conf WITH_LLDB=3Dyes > and compile and install lldb, which *could* analyze the coredump. >=20 > I realize that gdb in base is on its way out the door, and that > clang and lldb are the new world order. However, one of the > main blockers for removing gdb from base is that > there is no equivalent to kgdb in the lldb world. >=20 > Is anyone working on a kernel lldb? When is it expected to be ready? > Will something like it be ready by end of 2014? >=20 > Even though this may not be an issue in the stable/10 branch, > it would be nice to have a kernel lldb debugger ready by > the next 10.1R if possible. Except for kgdb functionality, the gdb ports (both the 6.x and 7.x ones) = work much better than the gdb in the tree. This may be a good stop-gap = until lldb is ready. Warner --Apple-Mail=_90641765-7931-4A5F-ABC6-B3746065B63A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTmK83AAoJEGwc0Sh9sBEA3bgP/A6pwOewyvMql3Kt5Q88qLvW ESl2hCAa161UB2X6bB7F/EYCewjO8oF/csSArM/C1dkos2MW4lT+Rx5I9LgVsiSj oXjpRR9xxg3X7T+XXsfdHDYhzZbEFeQCGW5Z/T5lje92te4ZX7GKEUUa//aG+q5a j5XScZIQWxT8/xxWYsJIVFzoZ00HHd+/py3dKuKCoqscXwDybWCpLXdRzHIGUVJ9 t3Mu40e8cCibN7gSOoKefJkch89G+Y3uyqvE5u2fylhnkFRE6QO+rC7Muc6X29Ce 03F34kTU0AUCAXcCG1b2oQ89eKzdb+1cZ4iIg6X3L6AXjJeYhzGDK94oNgpInEbQ Z/FW8xcbONOHphanNOhdtp/iKFmXzZuBrjA+fwNDanu97+85dMNtnQ2Giy/mSQ7K uu1rQOHuMdfP4e0CzP8ped6c7lnartt4C/zl/XUumaKfMz+dRpKb/zwb/ZiFVQB8 Ztl4kfXVAqnOwajL9rdYlIyAh7DWIYfCrE0BP/FrgheHjerdec7NjrWSE+1rsVdG Rz27bTIA6ozi+0m+G+k5MpGo7fo42JCfFAIn0jTH37uhRfX4OMwI1H0Q/Lh79+wT BQhHiiawVxoYQKgt6T/lPBQaASvtK6a6krZrzw8LUjVPURvT3c+TcNvODqSh8YnW gS3P7Qpalel2fkpMNPWf =BOZX -----END PGP SIGNATURE----- --Apple-Mail=_90641765-7931-4A5F-ABC6-B3746065B63A-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 19:56:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CED2FF61; Wed, 11 Jun 2014 19:56:30 +0000 (UTC) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C62262ECD; Wed, 11 Jun 2014 19:56:29 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id c11so137067lbj.17 for ; Wed, 11 Jun 2014 12:56:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=FTxSneSce6s4taSh930jnBbfT0xTck8ic2udZd0dGvU=; b=MeL/VTIEis9Eq2PJyp7QFRjDc209pU469KuR1Jy6nEw78xRrh8TrjTExSRRIS78tyK oQInvdyF+8EAKRASPkEKhXnSI4xJbo9A1xfxw6gZiphC/9dBSXs/wuVvqFXVnirFAfuC R+sTCGrU1ZTBWlqkPmkAPDJLXjjjF2c823V0zUgi+p6BtKEFRudG30bX6gJkw0ErchK5 LT7wOsw8tl/MZe2pr49pppliJfqy5J2Nk9bVFwEanXeGthcmwTmsVGfnZ9mAB1i6QwLs AwNVSGkFtHC0o/LgFzyf2v5TqyPpVQGGjDHvJcWNfz6rYwtjhVtzw7Ac/Q8bpieiPYHc OmhQ== MIME-Version: 1.0 X-Received: by 10.112.130.229 with SMTP id oh5mr8276111lbb.49.1402516587680; Wed, 11 Jun 2014 12:56:27 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 12:56:27 -0700 (PDT) In-Reply-To: References: Date: Wed, 11 Jun 2014 12:56:27 -0700 X-Google-Sender-Auth: WLSqG3F7Dxwhn581ArLQWJePWi4 Message-ID: Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Craig Rodrigues To: Dimitry Andric Content-Type: text/plain; charset=ISO-8859-1 Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:56:30 -0000 On Wed, Jun 11, 2014 at 12:30 PM, Dimitry Andric wrote: > On 11 Jun 2014, at 20:53, Craig Rodrigues wrote: >> >> Recently when trying to debug some coredumps in CURRENT from >> a userland process in the devel/libvirt port, I found that the gdb in >> base could not get a backtrace from the core file: >> >> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606.html > > Can you please post the output of the following? > > objdump -W /usr/local/sbin/libvirtd | head > > -Dimitry > $ objdump -W /usr/local/sbin/libvirtd | head /usr/local/sbin/libvirtd: file format elf64-x86-64-freebsd The section .debug_aranges contains: Length: 92 Version: 2 Offset into .debug_info: 0 Pointer Size: 8 Segment Size: 0 -- Craig From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 19:58:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C3A2F113; Wed, 11 Jun 2014 19:58:37 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E767D2EEF; Wed, 11 Jun 2014 19:58:36 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id pn19so141529lab.20 for ; Wed, 11 Jun 2014 12:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=/67wby0THGpy9/P6JH1jFCi0I+O1ACr7SLJXnBn0mB8=; b=XZGzkSL0Ykvs9lAK4H1vYpv7xrLV/reEspox1OYWzRx54ioy6KnbbKy/LsVP+OQhNZ dmJyugtX5akrYAvYqLTCU6iwz7wjCtAl4wVNqaydNh1uH5GGj7Fm7DaHlxhykUBmnW5i vcPoh83nsjizm/fBxp67TZzTjkGXAd9uZ/mKVNuEiE56+kL3AkF6kZvPCKfeoFF4HETS Jqn527qORTcHUOhSEY4LTlKZIoldZVrs1UyX0UWnsoFAc9sYmnEACjX7ZEhF3cNVgM7G R5x7RI14fLgHqJC/p7uOyOeMn9OJyHUDrP7HvD9qI5QQGStvqg29/Oa5OLj3N01Tur89 ikjQ== MIME-Version: 1.0 X-Received: by 10.112.130.229 with SMTP id oh5mr8282626lbb.49.1402516714721; Wed, 11 Jun 2014 12:58:34 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 12:58:34 -0700 (PDT) In-Reply-To: <33D37CFE-CA6B-4F89-9178-ACF2352CC6C5@bsdimp.com> References: <33D37CFE-CA6B-4F89-9178-ACF2352CC6C5@bsdimp.com> Date: Wed, 11 Jun 2014 12:58:34 -0700 X-Google-Sender-Auth: RMDgtj5UX5QJDIGFsaEwwTi-HcI Message-ID: Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Craig Rodrigues To: Warner Losh Content-Type: text/plain; charset=ISO-8859-1 Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 19:58:37 -0000 On Wed, Jun 11, 2014 at 12:34 PM, Warner Losh wrote: > > Except for kgdb functionality, the gdb ports (both the 6.x and 7.x ones) work much better than the gdb in the tree. This may be a good stop-gap until lldb is ready. > > Warner > I installed devel/gdb which installs gdb762. When I tried to debug the corefile, I encountered the same problem which I saw in: http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606.html -- Craig From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 20:40:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 397865B1; Wed, 11 Jun 2014 20:40:07 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E6028237C; Wed, 11 Jun 2014 20:40:06 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::c5cc:f189:593d:8554] (unknown [IPv6:2001:7b8:3a7:0:c5cc:f189:593d:8554]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 398F05C37; Wed, 11 Jun 2014 22:40:03 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_B9FB59A4-79B1-4144-ACE0-291B0AFCB25A"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Dimitry Andric In-Reply-To: Date: Wed, 11 Jun 2014 22:39:49 +0200 Message-Id: <11764709-BB10-47C5-B537-5AC5A0A0E2E6@FreeBSD.org> References: To: Craig Rodrigues X-Mailer: Apple Mail (2.1878.2) Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 20:40:07 -0000 --Apple-Mail=_B9FB59A4-79B1-4144-ACE0-291B0AFCB25A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On 11 Jun 2014, at 21:56, Craig Rodrigues wrote: > On Wed, Jun 11, 2014 at 12:30 PM, Dimitry Andric = wrote: >> On 11 Jun 2014, at 20:53, Craig Rodrigues = wrote: >>>=20 >>> Recently when trying to debug some coredumps in CURRENT from >>> a userland process in the devel/libvirt port, I found that the gdb = in >>> base could not get a backtrace from the core file: >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002606= .html >>=20 >> Can you please post the output of the following? >>=20 >> objdump -W /usr/local/sbin/libvirtd | head >>=20 >> -Dimitry >>=20 >=20 > $ objdump -W /usr/local/sbin/libvirtd | head >=20 > /usr/local/sbin/libvirtd: file format elf64-x86-64-freebsd >=20 > The section .debug_aranges contains: >=20 > Length: 92 > Version: 2 > Offset into .debug_info: 0 > Pointer Size: 8 > Segment Size: 0 Ah sorry, your objdump prints sections in a different order than mine. I was mainly interested in the DWARF version of debug information; maybe the way you built libvirt has caused it to contain DWARF4 instead of DWARF2. Can you post the complete objdump output instead, or post just the part under "The section .debug_info contains"? That should look similar to: The section .debug_info contains: Compilation Unit @ offset 0x0: Length: 59 Version: 4 Abbrev Offset: 0 Pointer Size: 4 [...] but with possibly another "Version:" line. -Dimitry --Apple-Mail=_B9FB59A4-79B1-4144-ACE0-291B0AFCB25A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlOYvqEACgkQsF6jCi4glqPEAACg9iqYRyH9cjMBDhAw/1/3ZI8F OZkAoJVMjiuem2mQ9N8ySy5KGd30Z9Qh =kX7z -----END PGP SIGNATURE----- --Apple-Mail=_B9FB59A4-79B1-4144-ACE0-291B0AFCB25A-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 11 22:18:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B257CE8; Wed, 11 Jun 2014 22:18:22 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C83E2D99; Wed, 11 Jun 2014 22:18:21 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id u10so216548lbd.38 for ; Wed, 11 Jun 2014 15:18:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GPVm7jEMxHf0KYpgUxAsFjmZuAkx7ksf9kG3WmyQvwE=; b=k1N/3uANxAc+spNp5vDmdmFAxq+gd2IZw9VxFhDuLwcaHfJpjKLpaHtZVngimJ3sJh F8WWIkk70WdabItpKJxOLePxDQ/8QMkxXXoXehnRaCMLzLd5VsGchDH21Xg4yIUi2Idp rSxMIRo3Ay7KnNMeszQ8C77HUI/A5R7zu6vpyGAVwxo/bKKNAwL1h7/eZPw+cMD6Uzoj 0cdC+chbHVKnczdP0ReCt7uIcGcsTGVdTQaeXVLy82Fl6YlratoSiD9hNucXObgWGPfR rH+hquo+JhG44mvIWTvwophD55+tSCWoG5oBJIhcHyC3KB6xYBFdXStVtcAkiYaWE1Pr xxIw== MIME-Version: 1.0 X-Received: by 10.152.87.136 with SMTP id ay8mr14195621lab.42.1402525099077; Wed, 11 Jun 2014 15:18:19 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.67.73 with HTTP; Wed, 11 Jun 2014 15:18:19 -0700 (PDT) In-Reply-To: <11764709-BB10-47C5-B537-5AC5A0A0E2E6@FreeBSD.org> References: <11764709-BB10-47C5-B537-5AC5A0A0E2E6@FreeBSD.org> Date: Wed, 11 Jun 2014 15:18:19 -0700 X-Google-Sender-Auth: 4PGegnxkn4bzMBk71Nx3uVWNz9A Message-ID: Subject: Re: gdb in CURRENT cannot debug userland cores, when is kernel lldb coming? From: Craig Rodrigues To: Dimitry Andric Content-Type: text/plain; charset=ISO-8859-1 Cc: Glen Barber , freebsd-toolchain@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2014 22:18:22 -0000 On Wed, Jun 11, 2014 at 1:39 PM, Dimitry Andric wrote: > Ah sorry, your objdump prints sections in a different order than mine. > I was mainly interested in the DWARF version of debug information; maybe > the way you built libvirt has caused it to contain DWARF4 instead of > DWARF2. > > Can you post the complete objdump output instead, or post just the part > under "The section .debug_info contains"? That should look similar to: Here is the full output of objdump -W /usr/local/sbin/libvirtd: http://people.freebsd.org/~rodrigc/libvirtd_objdump.txt -- Craig From owner-freebsd-arch@FreeBSD.ORG Fri Jun 13 07:05:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 592118EE for ; Fri, 13 Jun 2014 07:05:02 +0000 (UTC) Received: from mail-qa0-x233.google.com (mail-qa0-x233.google.com [IPv6:2607:f8b0:400d:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1485C2901 for ; Fri, 13 Jun 2014 07:05:01 +0000 (UTC) Received: by mail-qa0-f51.google.com with SMTP id j7so1734695qaq.10 for ; Fri, 13 Jun 2014 00:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=CmA9wNtJ3TUUaSwethiOTDPH7HloN6NqlcxDwwUep78=; b=PW4XVo91s0nVhtlwSQchv3JGhzqkK2Mv5rIWOGGg4AHdfAH7xhbG5pRhv5xGQLpTPq 6767FyJF0LDVQkhkuqOV+InhiRODSFnnciRhWZRh0lsl+DKVNVEGWJnr5eWjH35fZFc0 tNizOPzAVZx98CjJtxlvzS+zjBJtydjX4FyoQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=CmA9wNtJ3TUUaSwethiOTDPH7HloN6NqlcxDwwUep78=; b=W4ac8mf09eM5xfyDdT1qrWs19tPih//la1R9ZU8xuMsN2DeBaz0kOrjH0rstL2VklG 68RXhqagMfWniD06zJ3zo9ROKuSBKkLfBq4/igoPPrztFkgI0ey+i8mwpgBQntCHq1LP GNDpcEwjm1tRPvOkRHjw4VIfks1dgoUcSFHGm7FfsBtpXNzetV0ik1E2aq302QARM1TD n3XvQcqmMLm8+EhhFtY5NAFYpltC6m7kAZhlU58X7l2AF7AfYWXHMTJdxvFfW/v9jqiG oA9QCYjBxRr2GLgSfI66DGWtubk6aszC0NpQWIc8DdgL3wtRY6MqzrEjZkgk3ZwTlfx6 mZUA== X-Gm-Message-State: ALoCoQlm6YoCWjL8EBC0Uvn60pwwd1oF/NjQ9kVbkDAEraD5b6F1ISB5HnFay9RvZuFUrFj3HpUI X-Received: by 10.224.135.132 with SMTP id n4mr920040qat.23.1402643101064; Fri, 13 Jun 2014 00:05:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.222.131 with HTTP; Fri, 13 Jun 2014 00:04:30 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Fri, 13 Jun 2014 00:04:30 -0700 Message-ID: Subject: Re: Improve cron(8) To: =?UTF-8?Q?Tomek_Wa=C5=82aszek?= , freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Jun 2014 07:05:02 -0000 +arch since hackers@ seems to be silent. On 11 June 2014 23:56, Tomek Wa=C5=82aszek wrote: > Hello, > I saw on the FreeBSD Ideas page topic about cron :). > I've started updating the 'original' FreeBSD cron from sources to vixi cr= on > 4.1. I think (well I hope :P) most of the features that were done in > FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately > some missing features at the moment: > - @every_second - this need to be done > - -s and -o, in vixi cron 4.1 daylight time switches are enabled by > default, at the moment there is no -s and -o options. So you need to remo= ve > '-s' from the cron rc script > > I've also added one feature from OpenBSD, crontab is poking cron using > unix-domain socket so we don't need to have suid on crontab. > > Path is in the attachment. I'm testing it on my FreeBSD box and it looks > good but anyway don't try it on production machines :). > > After the installation we have to do a few things: > - Add crontab group > - Change group to crontab on /var/cron/tabs > - Add sticky bit on /var/cron/tabs > - Add group write permissions on /var/cron/tabs > > This is still work in progress but if someone could have a look on this a= nd > give me some feedback it would be great. > > Regards, > Tomasz Walaszek > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " --=20 Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Sun Jun 15 20:47:41 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D1AE27B for ; Sun, 15 Jun 2014 20:47:41 +0000 (UTC) Received: from felyko.com (felyko.com [65.49.80.26]) by mx1.freebsd.org (Postfix) with ESMTP id 1D54320B4 for ; Sun, 15 Jun 2014 20:47:41 +0000 (UTC) Received: from [IPv6:2601:9:8280:426:588e:9675:19a:c250] (unknown [IPv6:2601:9:8280:426:588e:9675:19a:c250]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id EBCD434A9E4 for ; Sun, 15 Jun 2014 13:47:33 -0700 (PDT) From: Rui Paulo Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: RCU Message-Id: <3ED81E00-4333-4115-B149-D42AF6B68663@FreeBSD.org> Date: Sun, 15 Jun 2014 13:47:33 -0700 To: arch@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) X-Mailer: Apple Mail (2.1878.2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jun 2014 20:47:41 -0000 Hi, Has anyone implemented a Read-Copy-Update API on FreeBSD? -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Mon Jun 16 10:45:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3F58782D for ; Mon, 16 Jun 2014 10:45:49 +0000 (UTC) Received: from mail-wg0-x22f.google.com (mail-wg0-x22f.google.com [IPv6:2a00:1450:400c:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D09B621E5 for ; Mon, 16 Jun 2014 10:45:48 +0000 (UTC) Received: by mail-wg0-f47.google.com with SMTP id k14so5299394wgh.18 for ; Mon, 16 Jun 2014 03:45:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=kkY5vjl01e4BQi7M/0RXAuChb4S3uuq2fXZ59dgUPYc=; b=nQ3jLIwsarE0d+7h36bwL5gQXmMo8ZxYl5gC7YXtMEc6sbMeTJiJVLDfXDD4KL+zLZ 6F5NEspsTo/FAsNj6rN5VahadRCZBCdFPw57DpUuy0cTZqFNPWHUXvAPvLLhY7pqc2sf Tm0WsTUwKasKZ1+qxQfqiIewPwpTgiZUfUwqi9uQfG80N1FXVf5FaE9qI1L9SyS9X877 /houhzPd4v50pTFYhj0OA+P8fpIG3ID96glagNyhqgj1ZLVwB2Q4QEYUCWmDu9mFzq8L w+mDCvE25HzO1NNB5o4cRlRyYNhq37tlswXFOGamua0WPRBbJVQyb6TxUBRuxzN8B13K +HVA== MIME-Version: 1.0 X-Received: by 10.180.206.132 with SMTP id lo4mr26325719wic.46.1402915547086; Mon, 16 Jun 2014 03:45:47 -0700 (PDT) Received: by 10.194.221.168 with HTTP; Mon, 16 Jun 2014 03:45:47 -0700 (PDT) Date: Mon, 16 Jun 2014 16:15:47 +0530 Message-ID: Subject: Basic Question about Kernel Processes in FreeBSD. From: Dheeraj Kandula To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 10:45:49 -0000 Hey All, When I was reading through the FreeBSD kernel code came across the function kproc_create. This function creates a kernel process. Isn't it? But at some places in the code, there is mention that the address space is shared with proc0. My Question: Do all the kernel processes share the same address space. i.e. even though they are multiple processes, they share the same kernel address space. If so then why do we have kernel threads as threads are created in the first place to share the address space of a process so that they are light weight. Can someone shed some light on this. I am a bit confused about this. I though that processes doesn't exist in kernel context and only user processes existed. Dheeraj From owner-freebsd-arch@FreeBSD.ORG Mon Jun 16 12:19:42 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0C9EF759 for ; Mon, 16 Jun 2014 12:19:42 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E87682AA5 for ; Mon, 16 Jun 2014 12:19:41 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5GCJfhP092523 for ; Mon, 16 Jun 2014 13:19:41 +0100 (BST) (envelope-from bz-noreply@freebsd.org) From: bz-noreply@freebsd.org To: freebsd-arch@FreeBSD.org Subject: [Bug 121073] [kernel] [patch] run chroot as an unprivileged user Date: Mon, 16 Jun 2014 12:19:41 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rwatson@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 12:19:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=121073 Robert Watson changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rwatson@FreeBSD.org --- Comment #8 from Robert Watson --- A appreciate the desirability for the features implied by this change, but given the propensity for vulnerabilities relating to chroot() in the past, think we should take a very conservative approach to potentially adopting it. There's a particular concern with how it interacts with non-UNIX-ID-based models -- e.g., MAC, Capsicum, Audit, Jail, as well as a future fine-grained privilege model. Overall, I'd rate this proposed change as "extremely high risk; we will fix multiple vulnerabilities in it in the future," and so that cost would need to be carefully weighed against presumed benefit -- a fine-grained privilege model in which PRIV_CHROOT is delegable to only specific users or roles would help mitigate that risk. I wonder if a more suitable name for the proposed P_NOSUGID would be P_NOCREDCHANGE, and I also wonder if it should be CR_NOCREDCHANGE. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 16 13:48:16 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9976BBEB for ; Mon, 16 Jun 2014 13:48:16 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D4E622FD for ; Mon, 16 Jun 2014 13:48:16 +0000 (UTC) Received: from Julian-MBP3.local (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5GDm6JQ063859 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 16 Jun 2014 06:48:08 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <539EF590.9050208@freebsd.org> Date: Mon, 16 Jun 2014 21:48:00 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Dheeraj Kandula , "freebsd-arch@freebsd.org" Subject: Re: Basic Question about Kernel Processes in FreeBSD. References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 13:48:16 -0000 On 6/16/14, 6:45 PM, Dheeraj Kandula wrote: > Hey All, > When I was reading through the FreeBSD kernel code came across the > function kproc_create. This function creates a kernel process. Isn't it? > > But at some places in the code, there is mention that the address space is > shared with proc0. read the man pages in section 9 man kproc_create and all the "see also" pointers. All kernel "processes" share the same address space.. that's why they are kernel processes :-) they do however have separate accounting and scheduling resources. there may be a number of kernel threads assigned to a simgle kernel process. We really only have kernel processes to help us 'group' the threads, since all kernel threads see the same address space. Origianlly we just had kernel threads (lots of them). Having kernel processes just allowed us to group them in 'top' for example. > > My Question: > Do all the kernel processes share the same address space. i.e. > even though they are multiple processes, they share the same kernel address > space. If so then why do we have kernel threads as threads are created in > the first place to share the address space of a process so that they are > light weight. > > Can someone shed some light on this. I am a bit confused about this. I > though that processes doesn't exist in kernel context and only user > processes existed. > > Dheeraj > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > From owner-freebsd-arch@FreeBSD.ORG Mon Jun 16 16:13:24 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A87F4E52 for ; Mon, 16 Jun 2014 16:13:24 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9032621EF for ; Mon, 16 Jun 2014 16:13:24 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5GGDOf5058078 for ; Mon, 16 Jun 2014 17:13:24 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-arch@FreeBSD.org Subject: [Bug 121073] [kernel] [patch] run chroot as an unprivileged user Date: Mon, 16 Jun 2014 16:13:24 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nwhitehorn@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 16:13:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=121073 --- Comment #9 from Nathan Whitehorn --- There are, I think, two potential security issues here: 1. Many pieces of software assume that if you chroot and drop privileges, no further chroot is possible. 2. There could be sneaky ways of obtaining privileges once no-new-privileges is set. (1) is pretty straightforward since we can just disallow unprivileged chroot after any other chroot. (2) is the complex one. Are there others? Some no-cred-change property for processes seems extremely useful from a security perspective and, if we have one we could trust, this patch becomes trivial. Would it make sense just to work on that first and come back to unprivileged chroot later? -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 16 17:00:46 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9DA6F1F for ; Mon, 16 Jun 2014 17:00:46 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 820DD26A3 for ; Mon, 16 Jun 2014 17:00:46 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D1E13B984; Mon, 16 Jun 2014 13:00:43 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Basic Question about Kernel Processes in FreeBSD. Date: Mon, 16 Jun 2014 09:42:35 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201406160942.35203.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 16 Jun 2014 13:00:43 -0400 (EDT) Cc: Dheeraj Kandula X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 17:00:46 -0000 On Monday, June 16, 2014 6:45:47 am Dheeraj Kandula wrote: > Hey All, > When I was reading through the FreeBSD kernel code came across the > function kproc_create. This function creates a kernel process. Isn't it? > > But at some places in the code, there is mention that the address space is > shared with proc0. > > My Question: > Do all the kernel processes share the same address space. i.e. > even though they are multiple processes, they share the same kernel address > space. If so then why do we have kernel threads as threads are created in > the first place to share the address space of a process so that they are > light weight. > > Can someone shed some light on this. I am a bit confused about this. I > though that processes doesn't exist in kernel context and only user > processes existed. Yes, they share a single address space. There are other reasons you might want to have separate processes. The aio kproc's actually "borrow" user address spaces while doing I/O on behalf of a user process for example. You might also want kprocs so that things show up as processes in top, etc. There might also be other properties (like a cpuset set id) that you might want to use with kthreads (but those only work for processes for example). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jun 17 07:18:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F3DC3EEB for ; Tue, 17 Jun 2014 07:18:36 +0000 (UTC) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5BFE2CF7 for ; Tue, 17 Jun 2014 07:18:36 +0000 (UTC) Received: by mail-pd0-f180.google.com with SMTP id fp1so1987832pdb.25 for ; Tue, 17 Jun 2014 00:18:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:message-id:date:from:user-agent :mime-version:to:subject:content-type; bh=zxOFt937E5uh0pCE53TaCy7LKEsnqJKgly3P7fIf9Ic=; b=Smj/XD2XuGL8Z7Cz9E5ZMWIrRFM9YPcp+KG6OVM3YGSqNfcDggNyUZkMUgXM7Mg3c6 kPndAItw+cBVPCUryGDUAZQi759HMKUlNwPTY7vlS2898if/jupduoOs0boQ/jXLD95p SWrcE1Y1T3lP87x3nGNnq7V/EQnhp1G/PWoO3++hMjJ5ucopQMZcjLs5+nuwe5MJpM7d aYxBWgE3g9JYHoNUYjTPG4xzlAtISPESMQVTaC+DIQLAS+PpX9Ue1fmuEkwMrq+M4dZX 40kG9VCNbWMF+r+k9MNScitMr/khA614TnRFL1F7HfNUJmw37WyYWRzwHsYQAA9mos5Z QeZg== X-Gm-Message-State: ALoCoQnz14+BArjcUh2F0ro6M4JyVRtNfB5V895zXkeUuiPT90avENSAK/OBkokSX/X3VvQ5p347 X-Received: by 10.68.193.193 with SMTP id hq1mr30061872pbc.107.1402989510498; Tue, 17 Jun 2014 00:18:30 -0700 (PDT) Received: from zont-osx.local (c-69-181-251-166.hsd1.ca.comcast.net. [69.181.251.166]) by mx.google.com with ESMTPSA id zx1sm22371055pbc.60.2014.06.17.00.18.29 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 17 Jun 2014 00:18:29 -0700 (PDT) Sender: Andrey Zonov Message-ID: <539FEBC1.5030501@FreeBSD.org> Date: Tue, 17 Jun 2014 00:18:25 -0700 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Subject: PoC: passive serialization X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="9C79jEh6Nhnolpe5kdFQIWCAcqc6O1Pl4" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 07:18:37 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9C79jEh6Nhnolpe5kdFQIWCAcqc6O1Pl4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, I'd like to introduce my implementation of passive serialization [1] (RCU-like algorithm) which based on expired patent #4809168 [2]. This algorithm allows one to access data structures in non-blocking lock-less manner, but it is not just read-write lock alternative it is something different. It is like delayed garbage collector or if you know what RCU is, passive serialization basically the same (RCU based on that). Unlike NetBSD's implementation [3] my version is light-weight, with no additional locks. One atomic operation per context switch is the only overhead. To demonstrate how it works I converted process limits to use that mechanism [4]. There is also simple test [5] if you interested in measurements. Just configure which type of lock do you want to use (mutex/rwlock/rmlock/psz), duration (LOOPS) and load (LENGTH) and run `make -s run' to get numbers. Any questions, comments or suggestions are welcome. [1] http://people.freebsd.org/~zont/patches/psz/0001-Implement-passive-serial= ization.patch [2] http://www.google.com/patents/US4809168 [3] http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/kern/subr_pserialize.c?rev=3D= 1.7&content-type=3Dtext/x-cvsweb-markup [4] http://people.freebsd.org/~zont/patches/psz/0002-Lock-less-limits.pat= ch [5] http://people.freebsd.org/~zont/patches/psz/lock_test/ --=20 Andrey Zonov --9C79jEh6Nhnolpe5kdFQIWCAcqc6O1Pl4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJTn+vEAAoJEBWLemxX/CvTD/0IAKF+iXe7dVrEyYiWDQD+P6Iu Fsa/W+eTVuql2/pQlSteIAgL0lR19uKi7OmUjneQU2jCARtm+KQSh9Kwvj1cx7Hq ULZWX1144Ihl5IrEhclcN9r4WlOLKxviG30c7PCgmysMwHhJahAul/5FpRehZ727 ip+Pd8cEdCDuW4szuyKtRWMBQ3wrCjsXdvWjxYF24VXyajhLUthLewfn2IcW4/LF 1n643CjAtmV6Pu+WCHlNP0LinfF0IqJnHtvzpml6Bs6Ah9P0K+Q/3PMmH1lQbqgI RF2yUwgL0v/22TBosZXG1UsyBM0pbXIu90ace59U/Y2gIA5Z7T7i02Fvr28Q8CI= =0XOP -----END PGP SIGNATURE----- --9C79jEh6Nhnolpe5kdFQIWCAcqc6O1Pl4-- From owner-freebsd-arch@FreeBSD.ORG Tue Jun 17 15:04:35 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 094D6CEE for ; Tue, 17 Jun 2014 15:04:35 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E4FA6293E for ; Tue, 17 Jun 2014 15:04:34 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s5HF4Yok053368 for ; Tue, 17 Jun 2014 16:04:34 +0100 (BST) (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-arch@FreeBSD.org Subject: [Bug 121073] [kernel] [patch] run chroot as an unprivileged user Date: Tue, 17 Jun 2014 15:04:34 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: rwatson@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 15:04:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=121073 --- Comment #10 from Robert Watson --- Just to follow up on Nathan and my conversation on IRC, things are made rather more complicated than one might hope by a gradual increase in the number of processes, over time, with credential changes. For example, Mac OS X's sandboxing system, based on our MAC Framework, frequently experiences domain transitions, and we could anticipate similar changes. It sounds like there is a net agreement that adopting a nice model for finer-grained, role-based privileges (e.g., the Solaris model) would have significant benefit to reducing the exposure in the event something did go wrong with unprivileged chroot -- and solve a number of other problems (e.g., unprivileged DTrace, better privilege-set abstractions for Jail), and is a worthy cause on the path to this sort of change. However, unprivileged chroot() will remain a sticky problem as programs of necessity place enormous trust in the UNIX filesystem namespace -- especially when it comes to features such as runtime linking, configuration files, etc, so if there is any form of 'call-gate'-style privilege escalation (e.g., setuid, setgid, TE MAC transitioning binaries), we could get ourselves into trouble. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-arch@FreeBSD.ORG Tue Jun 17 21:36:20 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A401E9C5; Tue, 17 Jun 2014 21:36:20 +0000 (UTC) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0236.outbound.protection.outlook.com [207.46.163.236]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49F2B2FA1; Tue, 17 Jun 2014 21:36:19 +0000 (UTC) Received: from CO2PR05MB731.namprd05.prod.outlook.com (10.141.228.21) by CO2PR05MB699.namprd05.prod.outlook.com (10.141.229.21) with Microsoft SMTP Server (TLS) id 15.0.949.11; Tue, 17 Jun 2014 21:36:13 +0000 Received: from CO2PR05MB730.namprd05.prod.outlook.com (10.141.228.15) by CO2PR05MB731.namprd05.prod.outlook.com (10.141.228.21) with Microsoft SMTP Server (TLS) id 15.0.949.11; Tue, 17 Jun 2014 21:36:12 +0000 Received: from CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) by CO2PR05MB730.namprd05.prod.outlook.com ([10.141.228.15]) with mapi id 15.00.0949.001; Tue, 17 Jun 2014 21:36:12 +0000 From: Anuranjan Shukla To: Rui Paulo , "arch@freebsd.org" Subject: Re: RCU Thread-Topic: RCU Thread-Index: AQHPiNsRHLIjS7WJ9U2cmJ7auSPwCJt1YT0A Date: Tue, 17 Jun 2014 21:36:11 +0000 Message-ID: References: <3ED81E00-4333-4115-B149-D42AF6B68663@FreeBSD.org> In-Reply-To: <3ED81E00-4333-4115-B149-D42AF6B68663@FreeBSD.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.2.140509 x-originating-ip: [66.129.239.14] x-microsoft-antispam: BL:0; ACTION:Default; RISK:Low; SCL:0; SPMLVL:NotSpam; PCL:0; RULEID: x-forefront-prvs: 0245702D7B x-forefront-antispam-report: SFV:NSPM; SFS:(6009001)(428001)(479174003)(51704005)(377454003)(199002)(189002)(24454002)(105586002)(66066001)(80022001)(83072002)(77982001)(79102001)(50986999)(54356999)(36756003)(85852003)(76176999)(101416001)(86362001)(64706001)(20776003)(92566001)(87936001)(92726001)(4396001)(46102001)(74502001)(81342001)(74662001)(85306003)(83506001)(2656002)(31966008)(81542001)(99396002)(76482001)(19580405001)(83322001)(19580395003)(21056001)(221733001)(99286002)(95666004); DIR:OUT; SFP:; SCL:1; SRVR:CO2PR05MB731; H:CO2PR05MB730.namprd05.prod.outlook.com; FPR:; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: juniper.net does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=anshukla@juniper.net; Content-Type: text/plain; charset="iso-8859-1" Content-ID: <4FD22CE0742C0B43B9F89D1A3BF81194@namprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Microsoft-Antispam: BL:0; ACTION:Default; RISK:Low; SCL:0; SPMLVL:NotSpam; PCL:0; RULEID: X-OriginatorOrg: juniper.net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 21:36:20 -0000 AFAIK man (9) rmlock is close to RCU, which is IBM patented and GPL=B9d for linux. Apologies if you already know this and meant to indicate something else. On 6/15/14, 1:47 PM, "Rui Paulo" wrote: >Hi, > >Has anyone implemented a Read-Copy-Update API on FreeBSD? > >-- >Rui Paulo > > > >_______________________________________________ >freebsd-arch@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-arch >To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Jun 17 21:57:58 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B9D58C6C for ; Tue, 17 Jun 2014 21:57:58 +0000 (UTC) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A5DB2247 for ; Tue, 17 Jun 2014 21:57:58 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id un15so3071098pbc.13 for ; Tue, 17 Jun 2014 14:57:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:message-id:date:from:user-agent :mime-version:to:subject:references:in-reply-to:content-type; bh=DzGVVkuhcX0sl7nHdphdMXAWJCvWt0v7dK0FJegCOgw=; b=H0/q/eKoxIVIKad/0xVzdeBLT6LjLTBa74kcJMOlCOVYNXV64AKvoRSC1WW1yl+N3M V/2BSAgUq8QrTd/YzlcFmbGZ3hFzV9rOY8YGS/da3wLuHJVc3SG8NI+2IAiVOTFlI+vh 7bS4StyWgTmdJqenLOe+Xh2Dv3RQIOOyEn8OctRZKMdkFVOZOXLBjyySsWUVv5GAZJqJ GYgSKNKqaXkdIOiEnL5V1u6T4GqX9CLCBWZ9rsUnPbDvhMMpYFujW9TmZjf3zdVMFvFg cQnuq6MaMFD9/dzGFRWQM+cgCXor9w8Uw28WysQGemLhMrv7XmK3nnYgmRBfclxvuY7q 2KeA== X-Gm-Message-State: ALoCoQloyJLOBs29arkJrgDUMZUv7C706rUhnFsKUGDf7Htvz2lViBcN4bIlEhIK6xQuELFR9KAe X-Received: by 10.66.142.199 with SMTP id ry7mr35875624pab.10.1403042272346; Tue, 17 Jun 2014 14:57:52 -0700 (PDT) Received: from zont-osx.swifttest.com ([207.141.67.177]) by mx.google.com with ESMTPSA id dd5sm25595979pbc.85.2014.06.17.14.57.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 17 Jun 2014 14:57:51 -0700 (PDT) Sender: Andrey Zonov Message-ID: <53A0B9DA.8060007@FreeBSD.org> Date: Tue, 17 Jun 2014 14:57:46 -0700 From: Andrey Zonov User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Rui Paulo , arch@freebsd.org Subject: Re: RCU References: <3ED81E00-4333-4115-B149-D42AF6B68663@FreeBSD.org> In-Reply-To: <3ED81E00-4333-4115-B149-D42AF6B68663@FreeBSD.org> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xm5f78CcIenftt9bCd7N9cJ25E7dhtxBj" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Jun 2014 21:57:58 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xm5f78CcIenftt9bCd7N9cJ25E7dhtxBj Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 6/15/14, 1:47 PM, Rui Paulo wrote: > Hi, >=20 > Has anyone implemented a Read-Copy-Update API on FreeBSD? >=20 http://lists.freebsd.org/pipermail/freebsd-arch/2014-June/015419.html --=20 Andrey Zonov --xm5f78CcIenftt9bCd7N9cJ25E7dhtxBj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJToLneAAoJEBWLemxX/CvTf68IAKDP8X4nzA5mhs/uK8mf8DfV 9TmM612FUSWsw+Py4iraJgGxyecFmZjvzFTfyZ49PFHfQ3E0AQKMjOyZye58oLgK m3uSsvy0K2ffTbAIrp+cocbzxmr6oNA3/mR+3iTp7SmsyvCdCMB+LbfLdck7dHuj FnQ+/8VGZtlBmqEqsQl+Ooos7+AWy3eQoFShqv7rrsWYhR34RZ4J3L0Tuw+850QH PVYfWBvnMNSuGn2wgL7SQUOVoM7S0ZN8rq8ldBiLA/7zofPNowzhdhotLWCJF7ST 2SB5IMQLkSCbJiIQJjGI8WIaKcJAlwa7gm78CnSFVlp+7DxVYBV2MAbm0HoVANA= =8c8L -----END PGP SIGNATURE----- --xm5f78CcIenftt9bCd7N9cJ25E7dhtxBj-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 18 20:33:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 075DEB70; Wed, 18 Jun 2014 20:33:21 +0000 (UTC) Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C49E2DA4; Wed, 18 Jun 2014 20:33:19 +0000 (UTC) Received: by mail-we0-f170.google.com with SMTP id w61so1456774wes.1 for ; Wed, 18 Jun 2014 13:33:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=o/yO/cApgqWEeUutc1X/l5yHt6vHI4Dilcx2vTxtJUY=; b=f6PcJdiUtsYaiQIDgZ2V8iNipKOs7mQhnDbp6Exh8FT9YvxrJu22XakAb4dBiVBUR8 TmRrumg0/XEfcdyvfgJtbj478PBBFDm7eDTQ0dkv3LRS+VfkULzktPXhQVvzImMSkqPg cXrFB6JzrcTRxz1R5P/Wo1xalgxplKYw5iWC0JYl5VQNaiqaBMoHmZk9rKDdaNV1fDI7 TsTAknT4N2ceq2JWGgNVWkmYMvneLxFlK/9zSUdpGEL/3roic/0/kHbzKzhIhuDHEWfx 0dNLlOBxrJTli8kjrvUR6cUZNTvn+9MIL5r0UfqRaOpY6RkldLo/qDXvi7Yt2S3ZdToY XsBg== X-Received: by 10.180.189.44 with SMTP id gf12mr674304wic.14.1403123598299; Wed, 18 Jun 2014 13:33:18 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id vp5sm4308893wjc.31.2014.06.18.13.33.16 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 18 Jun 2014 13:33:16 -0700 (PDT) Date: Wed, 18 Jun 2014 22:33:14 +0200 From: Mateusz Guzik To: Adrian Chadd Subject: Re: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) Message-ID: <20140618203314.GB7157@dft-labs.eu> References: <20140608220000.GA5030@dft-labs.eu> <20140608230059.GB5030@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140608230059.GB5030@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 20:33:21 -0000 On Mon, Jun 09, 2014 at 01:00:59AM +0200, Mateusz Guzik wrote: > So, benchmarks: > > http://people.freebsd.org/~mjg/patches/caps-race/dup2-bench.c > > 1 thread does dup2 in a loop > 7 threads fget() the same fd and fail quickly > [..] > Difference at 95.0% confidence > 1.96987e+06 +/- 46328.7 > 16.3174% +/- 0.383763% > (Student's t, pooled s = 167139) > > It would be good if someone with some time and access to higher-cpu-count > machine could test it better. > pjd suggested on irc allowing a pass with inconsistent filedescent struct and check it before we fail and before fp is accepted. I also went ahead and modified seq_* function a little to make them easier to use. This is with inconsistent struct allowed: http://people.freebsd.org/~mjg/patches/caps-race/caps-race-seq.diff And this spins to get the right one: http://people.freebsd.org/~mjg/patches/caps-race/caps-race-seqearly.diff Patches differ only in fget_unlocked function. Performnance is unfortunately still bad and an approach with inconsistent struct is the slowest. Results are reproducible well enough. Comments? Maybe we should try to pack this into one atomically replaced structure after all, but I don't have time to work on that. I checked with 2 benchmarks: 1 thread dup2 + 7 read: x pure-kernel-dup2-bench + seq-dup2-bench * seqearly-dup2-bench +-------------------------------------------------------------------------------------------------------------------------------------+ | + * | | + * | | + * | | + * x | | + * x | | ++ * x | | ++ * * x x | | +++ * * x x | | +++ * * ** * x xxx | | ++++ + ***** ** * x x xxxx xx | | + ++++++ + + ***** * **** xx xx xxxx xx | | ++++++++ +++ +****** * **** * xxxxxx xxxxxxx | | ++++++++ ++++++******* ** **** * x xxxxxx xxxxxxxx | | +++++++++ ++++++********* ******** * xx xxxxxxxxxxxxxxxx | |++ ++++++++++++++**+********** *********** x x x x xxx xxxxxxxxxxxxxxxxx x| | |___MA____| |____M_A_____| |_______A_M_____| | +-------------------------------------------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 100 13065813 14435053 14157439 14103416 196933.44 + 100 11071952 11615178 11351083 11374348 121160.27 Difference at 95.0% confidence -2.72907e+06 +/- 45319.1 -19.3504% +/- 0.321334% (Student's t, pooled s = 163497) * 100 11517495 12111074 11774410 11828854 159139.54 Difference at 95.0% confidence -2.27456e+06 +/- 49626.4 -16.1277% +/- 0.351875% (Student's t, pooled s = 179037) 8 threads read: x pure-kernel-read-pipe-bench + seq-read-pipe-bench * seqearly-read-pipe-bench +-------------------------------------------------------------------------------------------------------------------------------------+ | * | | * | | + * x | | + * x | | + ++ * xx | | + ++ + * x xx | | + ++++ + * * xx xx | | + ++++ + + * **** * xx xxx | | + ++++ +++ * **** * xxxxxxx | | ++++++ ++++ * **** *** xxxxxxx | | ++++++ ++++ * ****** * *** x x xxxxxxx | |+ ++++++++++++ * * ******* ** ***** x xx xxxxxxxxxxx | |++++++++++++++++ + ** * ******* ** * * ***** * x xx x xxxxxxxxxxxxxxxx| |++++++++++++++++ + ++ + ** ************ * *** ****** * xxxx xxxx xxxxxxxxxxxxxxxxxx| | |___MA_____| |____M__A_______| |_____A_M___| | +-------------------------------------------------------------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 100 20721201 21597446 21369536 21306748 206438.73 + 100 17272059 18247946 17509832 17547079 178174.14 Difference at 95.0% confidence -3.75967e+06 +/- 53448.4 -17.6454% +/- 0.250852% (Student's t, pooled s = 192825) * 100 18417084 19360802 18739503 18836380 269109.08 Difference at 95.0% confidence -2.47037e+06 +/- 66477.4 -11.5943% +/- 0.312002% (Student's t, pooled s = 239830) -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Wed Jun 18 23:13:20 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14621E32 for ; Wed, 18 Jun 2014 23:13:20 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DE0AC2BC0 for ; Wed, 18 Jun 2014 23:13:19 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5INDIZL041270 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 18 Jun 2014 16:13:19 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5INDItB041269 for arch@FreeBSD.org; Wed, 18 Jun 2014 16:13:18 -0700 (PDT) (envelope-from jmg) Date: Wed, 18 Jun 2014 16:13:18 -0700 From: John-Mark Gurney To: arch@FreeBSD.org Subject: conflict between netif and pccard_ether... Message-ID: <20140618231318.GH31367@funkthat.com> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 18 Jun 2014 16:13:19 -0700 (PDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 23:13:20 -0000 So, I recently was trying to figure out why wireless on my notebook wouldn't work.. I would boot up the machine w/o the wireless configured, uncomment the lines in rc.conf, and then run "service netif start"... Wireless would associate, but when disconnect... After some investigation, it turns out that two copies of wpa_supplicant are being launched... I believe one from netif, and another from pccard_ether launched by devd... The issue is that both netif and pccard_ether "claim" ownership of them. pccard_ether will ignore the ifconfig_ line if NOAUTO is specified. IMO, we need to make one or the other "own" configuring and launching the interface... There is also the issue that wpa_sup doesn't use proper locking on the pidfile and allows two copies to be launched... My thoughts is to convert it to pidfile to fix this issue the easiest... Comments? Suggestions? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Thu Jun 19 12:21:59 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 32806837 for ; Thu, 19 Jun 2014 12:21:59 +0000 (UTC) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id EB1D4297F for ; Thu, 19 Jun 2014 12:21:57 +0000 (UTC) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 8A7E02703BF1 for ; Thu, 19 Jun 2014 14:21:49 +0200 (CEST) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 72F1A2703BEC for ; Thu, 19 Jun 2014 14:21:49 +0200 (CEST) Date: Thu, 19 Jun 2014 14:21:49 +0200 (CEST) From: Emeric POUPON To: freebsd-arch@freebsd.org Message-ID: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> In-Reply-To: <961817567.135121.1403178816482.JavaMail.zimbra@arkoon-netasq.com> Subject: How to properly handle several fonctions provided by the Winbond SuperIO chip? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Thread-Topic: How to properly handle several fonctions provided by the Winbond SuperIO chip? Thread-Index: wf/3trm5wPfPwflEhpkBu4Rgz0DguA== X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 12:21:59 -0000 Hello, I have a design question about how to configure/control a Winbond Super IO device. Currently, only the Watchdog feature is properly handled in FreeBSD (see dev/wbwd), but I would like to control the GPIO that are managed by this SuperIO device. Making a complete separate isa driver seems not to be a good idea : - duplicated probe/attach routines. - concurrency accesses on the registers. Indeed this device provides an "extended mode" in order to be configured, and it also provides a "logical device" selection in order to access specific features (one logical device for the watchdog, another one for a GPIO port, etc.). As far as I understand, they solved the problem on Linux by : - using separate drivers - using a memory locked mechanism when entering/exiting the extended mode. However, on FreeBSD I would split the whole thing in three drivers: - wbsio (sio stands for SuperIO), the main driver: - identify/attach/probe routines on the isa bus. - provide primitives to enter/exit the extended mode, and hangle an internal mutex to give exclusive access on this mode. - provide primitives to select the logical device and read/write the internal registers - attach child devices "wbwd" and "gpio". - wbwd, - child of wbsio - register the watchdog event callback - use wbsio primitives to get the work done - wbgpio, - child of wbsio - implement gpio methods - add child devices "gpioc" and "gpiobus" - use wbsio primitives to get the work done What do you think? Is that the good way to proceed? Regards, Emeric POUPON From owner-freebsd-arch@FreeBSD.ORG Thu Jun 19 13:35:05 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6EC3D5D3; Thu, 19 Jun 2014 13:35:05 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 482C3211B; Thu, 19 Jun 2014 13:35:05 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3E851B943; Thu, 19 Jun 2014 09:35:03 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: conflict between netif and pccard_ether... Date: Thu, 19 Jun 2014 09:17:16 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140618231318.GH31367@funkthat.com> In-Reply-To: <20140618231318.GH31367@funkthat.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201406190917.16458.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 19 Jun 2014 09:35:03 -0400 (EDT) Cc: arch@freebsd.org, John-Mark Gurney X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 13:35:05 -0000 On Wednesday, June 18, 2014 7:13:18 pm John-Mark Gurney wrote: > So, I recently was trying to figure out why wireless on my notebook > wouldn't work.. I would boot up the machine w/o the wireless > configured, uncomment the lines in rc.conf, and then run > "service netif start"... > > Wireless would associate, but when disconnect... After some > investigation, it turns out that two copies of wpa_supplicant are > being launched... I believe one from netif, and another from > pccard_ether launched by devd... > > The issue is that both netif and pccard_ether "claim" ownership of > them. pccard_ether will ignore the ifconfig_ line if NOAUTO is > specified. IMO, we need to make one or the other "own" configuring > and launching the interface... > > There is also the issue that wpa_sup doesn't use proper locking on > the pidfile and allows two copies to be launched... My thoughts > is to convert it to pidfile to fix this issue the easiest... I think this is actually the best fix. dhclient handles this correctly for this reason. > Comments? Suggestions? We had a thread a few months ago about this very topic and I committed changes to the rc.d scripts that I then had to revert because it broke other use cases. Having the redundant start be harmless is the simplest way to handle this. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jun 19 13:35:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF9E55D4 for ; Thu, 19 Jun 2014 13:35:05 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 874C6211C for ; Thu, 19 Jun 2014 13:35:05 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 68DC7B9A1; Thu, 19 Jun 2014 09:35:04 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? Date: Thu, 19 Jun 2014 09:19:04 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> In-Reply-To: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201406190919.04443.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 19 Jun 2014 09:35:04 -0400 (EDT) Cc: Emeric POUPON X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 13:35:05 -0000 On Thursday, June 19, 2014 8:21:49 am Emeric POUPON wrote: > Hello, > > I have a design question about how to configure/control a Winbond Super IO device. > > Currently, only the Watchdog feature is properly handled in FreeBSD (see dev/wbwd), but I would like to control the GPIO that are managed by this SuperIO device. > > Making a complete separate isa driver seems not to be a good idea : > - duplicated probe/attach routines. > - concurrency accesses on the registers. Indeed this device provides an "extended mode" in order to be configured, and it also provides a "logical device" selection in order to access specific features (one logical device for the watchdog, another one for a GPIO port, etc.). > > As far as I understand, they solved the problem on Linux by : > - using separate drivers > - using a memory locked mechanism when entering/exiting the extended mode. > > However, on FreeBSD I would split the whole thing in three drivers: > - wbsio (sio stands for SuperIO), the main driver: > - identify/attach/probe routines on the isa bus. > - provide primitives to enter/exit the extended mode, and hangle an internal mutex to give exclusive access on this mode. > - provide primitives to select the logical device and read/write the internal registers > - attach child devices "wbwd" and "gpio". > - wbwd, > - child of wbsio > - register the watchdog event callback > - use wbsio primitives to get the work done > - wbgpio, > - child of wbsio > - implement gpio methods > - add child devices "gpioc" and "gpiobus" > - use wbsio primitives to get the work done > > What do you think? Is that the good way to proceed? This sounds correct to treat the raw device as a bus and the logical devices it provides as children. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jun 19 15:22:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC234381 for ; Thu, 19 Jun 2014 15:22:02 +0000 (UTC) Received: from work.netasq.com (gwlille.netasq.com [91.212.116.1]) by mx1.freebsd.org (Postfix) with ESMTP id B0DD22B9E for ; Thu, 19 Jun 2014 15:22:01 +0000 (UTC) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 47FC62703BD1 for ; Thu, 19 Jun 2014 17:22:00 +0200 (CEST) Received: from work.netasq.com (localhost [127.0.0.1]) by work.netasq.com (Postfix) with ESMTP id 2478C2703BAA for ; Thu, 19 Jun 2014 17:22:00 +0200 (CEST) Date: Thu, 19 Jun 2014 17:21:59 +0200 (CEST) From: Emeric POUPON To: freebsd-arch@freebsd.org Message-ID: <750618593.166408.1403191319583.JavaMail.zimbra@arkoon-netasq.com> In-Reply-To: <201406190919.04443.jhb@freebsd.org> References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <201406190919.04443.jhb@freebsd.org> Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Thread-Topic: How to properly handle several fonctions provided by the Winbond SuperIO chip? Thread-Index: P4k3/ykDgm4oworsjWni0BoYakx0hQ== X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jun 2014 15:22:03 -0000 Thanks for your answer! I was thinking about calling some parent device functions from the children= devices in order to perform IO accesses. But I imagine it would be "better" to expose a kind of bus interface from t= he main driver? However, I'm not sure the extra work induced is worth it. What do you think= ? Emeric Poupon ----- Mail original ----- De: "John Baldwin" =C3=80: freebsd-arch@freebsd.org Cc: "Emeric POUPON" Envoy=C3=A9: Jeudi 19 Juin 2014 15:19:04 Objet: Re: How to properly handle several fonctions provided by the Winbond= SuperIO chip? On Thursday, June 19, 2014 8:21:49 am Emeric POUPON wrote: > Hello, >=20 > I have a design question about how to configure/control a Winbond Super I= O device. >=20 > Currently, only the Watchdog feature is properly handled in FreeBSD (see = dev/wbwd), but I would like to control the GPIO that are managed by this Su= perIO device. >=20 > Making a complete separate isa driver seems not to be a good idea : > - duplicated probe/attach routines. > - concurrency accesses on the registers. Indeed this device provides an "= extended mode" in order to be configured, and it also provides a "logical d= evice"=20 selection in order to access specific features (one logical device for the = watchdog, another one for a GPIO port, etc.). >=20 > As far as I understand, they solved the problem on Linux by : > - using separate drivers > - using a memory locked mechanism when entering/exiting the extended mode= . >=20 > However, on FreeBSD I would split the whole thing in three drivers: > - wbsio (sio stands for SuperIO), the main driver: > - identify/attach/probe routines on the isa bus. > - provide primitives to enter/exit the extended mode, and hangle an int= ernal mutex to give exclusive access on this mode. > - provide primitives to select the logical device and read/write the in= ternal registers > - attach child devices "wbwd" and "gpio". > - wbwd,=20 > - child of wbsio > - register the watchdog event callback > - use wbsio primitives to get the work done > - wbgpio, > - child of wbsio > - implement gpio methods > - add child devices "gpioc" and "gpiobus" > - use wbsio primitives to get the work done >=20 > What do you think? Is that the good way to proceed? This sounds correct to treat the raw device as a bus and the logical device= s it provides as children. --=20 John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jun 20 01:04:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B784E4AB; Fri, 20 Jun 2014 01:04:31 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D5E422D8E; Fri, 20 Jun 2014 01:04:30 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id x13so2939889wgg.15 for ; Thu, 19 Jun 2014 18:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=RHqvaeRtQO5LWdaVMSfSeSgyXS3ly0fAFw6FLS4SZP0=; b=dWjSigQcarf6CHZ5b90PNYgGKYLjeeX2yydDZ/RaWoIZIVsyMB/A5X5r517vjTljF+ 23dfpy9vXyu+Csn3qfGsSdGDUraRr2h3I8MffBZBsEH8ucZsofWaJEzaz3KhZwSuT/2y iJUdOLWm5ySuQL2jDaeC8MiZYPg8VHeHhx9B8i/veP3ud91MqYB4Y4duEf/rB+kjuEJ3 IcsM5kEmc8WWBTKB2c8PBp4DvD3SkA0WFSPA+BErkMsrtXDSbwnvUqI3nH1x9CJ9Ki2X cy2yOZ7uqlDwjWMACJUDhNzZJPN6Yy4oc5leGySFz6EIE+gnJ1rUXrvyGipFoKJ3aXi3 XEfw== X-Received: by 10.194.8.102 with SMTP id q6mr120751wja.74.1403226268188; Thu, 19 Jun 2014 18:04:28 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id fn1sm16189115wib.18.2014.06.19.18.04.26 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 19 Jun 2014 18:04:27 -0700 (PDT) Date: Fri, 20 Jun 2014 03:04:24 +0200 From: Mateusz Guzik To: Adrian Chadd Subject: Re: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) Message-ID: <20140620010424.GA5830@dft-labs.eu> References: <20140608220000.GA5030@dft-labs.eu> <20140608230059.GB5030@dft-labs.eu> <20140618203314.GB7157@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140618203314.GB7157@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 01:04:31 -0000 Good news. While previous approached were giving ~16% worse performance in my microbenchmarks I found a way to make up for this (needs verification for correctness). fget_unlocked does: /* * Avoid reads reordering and then a first access to the * fdp->fd_ofiles table which could result in OOB operation. */ if (fd < 0 || fd >= atomic_load_acq_int(&fdp->fd_nfiles)) return (EBADF); Turns out replacing this read with mere "fdp->fd_nfiles" gives ~10% performance increase over regular kernel and almost brings kernel with sequence counters to the level of regular kernel. So how to get rid of the need for atomic_load_acq_int? Currently struct filedesc contains file table and its size. File table pointer update is one operation, size update is another. Thus the need for memory barriers. Now let's consider the following structure: struct fdtable { int f_size; struct filedesc f_files[0]; }; Pointer to this strucure in struct filedesc can be replaced atomically, and from what I'm told the only synchronization needed is data dependency barrier (which in case of amd64 means there is no need to do anything). In short, we can get ~10% better performance and then sacrifice the gain (and some more) to have capability races fixed and be slightly slower than now. Thoughts? The patch is a total hack so I'm not attaching it, but as far as performance goes you can just move fd_nfiles to be next to fd_ofiles and remove atomic_load_acq_int in fget_unlocked. fdgrowtable will need to have a write barrier before the pointer is atomically replaced. -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Fri Jun 20 02:05:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E5568CE; Fri, 20 Jun 2014 02:05:49 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 835A222EC; Fri, 20 Jun 2014 02:05:49 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id z60so2892051qgd.16 for ; Thu, 19 Jun 2014 19:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2sCosEdce7yW9P9BkJV2LTvyFd5qXbF0b4lNe4U/qMc=; b=MDV8pogH2mUeyR2K0NhDZgjDF4bxdCMyrOmxnWI43TQoOaRSKpXkdR23IGPwXKpksZ ZYkdNwaSngUVvZlGednj+qMMwRV466bvJcxqi9kpihco8whtNYHu5byIDFYLnp7By5HH uWdE3jOl1FFYxH51YvJZ6FsJG15gIsJLjZLbVD5LK8r5tRydXB0wBglb7sXpov2/EnOa nSdtjMIKJtqypjtQ7xvkEu+H/4ewhqNBOFpqndREpUoevsM6MrQYvn+1WoFnJvKyCDV3 YpvG5oY9zsYkQcJ6X2X5xQvV8ZfDVWQOfL8Nmb62swvqpX0Eyl8bH58IK62Ndf3y5WVp rLEw== MIME-Version: 1.0 X-Received: by 10.229.65.200 with SMTP id k8mr467345qci.4.1403229948626; Thu, 19 Jun 2014 19:05:48 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Thu, 19 Jun 2014 19:05:48 -0700 (PDT) In-Reply-To: <20140620010424.GA5830@dft-labs.eu> References: <20140608220000.GA5030@dft-labs.eu> <20140608230059.GB5030@dft-labs.eu> <20140618203314.GB7157@dft-labs.eu> <20140620010424.GA5830@dft-labs.eu> Date: Thu, 19 Jun 2014 19:05:48 -0700 X-Google-Sender-Auth: ENQkmYvS5_UsbiQuF0KqtP-0x6c Message-ID: Subject: Re: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) From: Adrian Chadd To: Mateusz Guzik Content-Type: text/plain; charset=UTF-8 Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 02:05:50 -0000 [snip] I'm increasingly wary of hand-rolled memory barrier / atomic using constructs like this. It's way, way too easy to shoot a foot off on an architecture that you don't have or know. So, if we're going down this rabbit hole further, I think we should first define all the places this stuff gets touched and try to come up with some behavioral description that we could try and link to some existing (non-patent-encumbered) no-lock based design pattern. So in your example, yes the pointer assignment is atomic, but there's no current guarantee that anything currently operating on that pointer has finished. That's what things like RCU address. It's cool that you've kept digging into this :-) -a From owner-freebsd-arch@FreeBSD.ORG Fri Jun 20 02:16:27 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51C31673; Fri, 20 Jun 2014 02:16:27 +0000 (UTC) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6585624D3; Fri, 20 Jun 2014 02:16:26 +0000 (UTC) Received: by mail-we0-f173.google.com with SMTP id t60so3117385wes.18 for ; Thu, 19 Jun 2014 19:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=mVPi5W03PviADKx5rapdjPxjDelFdBbDo62lt/OxF3c=; b=IY/WvByqgTS2rFshOHegoBEj1D4v51dMGlJM8lXJ3dMHzw/LsKiVk2KFA/oEONppLK wPaI+40Mfxcv6fQi+mPkUcW+dF4lF8/WQwBN17a+CBhZzu3gct6K/o0U9EUCat3BBjEh Kej1iljrTL72Y4ULnONn3uaxthTpSxB2zhofKxFkcJ/yOUPNOZkZ9YGa8JY7CPrfx6zp YSjbvrYOR0CDj3m6lihryD0+8RhegY1OJh6YoponQnPf7py9u/AF303x6szbBgSoLVRf KstXCbvLimgGEfTI8E5idx3zSfJ1kc5/3AZ2Gj/WbWgZ9VRlecdSeZg9mJYnP3szpeP8 YDFg== X-Received: by 10.180.75.71 with SMTP id a7mr393471wiw.27.1403230584646; Thu, 19 Jun 2014 19:16:24 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id gp6sm276021wib.12.2014.06.19.19.16.23 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Thu, 19 Jun 2014 19:16:23 -0700 (PDT) Date: Fri, 20 Jun 2014 04:16:21 +0200 From: Mateusz Guzik To: Adrian Chadd Subject: Re: capability races (was: Re: Seeing ENOTCAPABLE from write FDs in kqueue?) Message-ID: <20140620021621.GC5830@dft-labs.eu> References: <20140608220000.GA5030@dft-labs.eu> <20140608230059.GB5030@dft-labs.eu> <20140618203314.GB7157@dft-labs.eu> <20140620010424.GA5830@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Robert Watson , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jun 2014 02:16:27 -0000 On Thu, Jun 19, 2014 at 07:05:48PM -0700, Adrian Chadd wrote: > [snip] > > I'm increasingly wary of hand-rolled memory barrier / atomic using > constructs like this. It's way, way too easy to shoot a foot off on an > architecture that you don't have or know. > Sure, hence I'm lookin for someone with strong memory-barrier-fu. > So, if we're going down this rabbit hole further, I think we should > first define all the places this stuff gets touched and try to come up > with some behavioral description that we could try and link to some > existing (non-patent-encumbered) no-lock based design pattern. > > So in your example, yes the pointer assignment is atomic, but there's > no current guarantee that anything currently operating on that pointer > has finished. That's what things like RCU address. > But we don't need that guarantee in here. File tables are freed only on process exit specifically because we never know if all threads finished reading. File table pointer is refreshed and fp validated before is returned to the caller; if validation fails there is relookup. Having RCU-like solution would allow us to free old tables without the process exiting of course, but would not affect correctness. As a side note, we could easily free old tables in singlethreaded processes. Multithreaded would require actual work to make sure all threads are stopped. -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sat Jun 21 10:40:33 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6560074E; Sat, 21 Jun 2014 10:40:33 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 41AB92723; Sat, 21 Jun 2014 10:40:32 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5LAeWur092987 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 21 Jun 2014 03:40:32 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5LAeWDo092986; Sat, 21 Jun 2014 03:40:32 -0700 (PDT) (envelope-from jmg) Date: Sat, 21 Jun 2014 03:40:32 -0700 From: John-Mark Gurney To: John Baldwin Subject: Re: conflict between netif and pccard_ether... Message-ID: <20140621104031.GM31367@funkthat.com> Mail-Followup-To: John Baldwin , freebsd-arch@freebsd.org, rpaulo@FreeBSD.org References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="mhjHhnbe5PrRcwjY" Content-Disposition: inline In-Reply-To: <201406190917.16458.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 21 Jun 2014 03:40:32 -0700 (PDT) Cc: rpaulo@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 10:40:33 -0000 --mhjHhnbe5PrRcwjY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline John Baldwin wrote this message on Thu, Jun 19, 2014 at 09:17 -0400: > On Wednesday, June 18, 2014 7:13:18 pm John-Mark Gurney wrote: > > So, I recently was trying to figure out why wireless on my notebook > > wouldn't work.. I would boot up the machine w/o the wireless > > configured, uncomment the lines in rc.conf, and then run > > "service netif start"... > > > > Wireless would associate, but when disconnect... After some > > investigation, it turns out that two copies of wpa_supplicant are > > being launched... I believe one from netif, and another from > > pccard_ether launched by devd... > > > > The issue is that both netif and pccard_ether "claim" ownership of > > them. pccard_ether will ignore the ifconfig_ line if NOAUTO is > > specified. IMO, we need to make one or the other "own" configuring > > and launching the interface... > > > > There is also the issue that wpa_sup doesn't use proper locking on > > the pidfile and allows two copies to be launched... My thoughts > > is to convert it to pidfile to fix this issue the easiest... > > I think this is actually the best fix. dhclient handles this correctly for > this reason. > > > Comments? Suggestions? > > We had a thread a few months ago about this very topic and I committed changes > to the rc.d scripts that I then had to revert because it broke other use > cases. Having the redundant start be harmless is the simplest way to handle > this. Ok, I converted wpa_supplicant to use pidfile(3), and it successfully fixes the issue I had... I get the error that wpa_supplicant is already running... It looks like in my case, devd was beating out netif in launching wpa_sup... I've attached the patch... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --mhjHhnbe5PrRcwjY Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="wpa.diff" diff --git a/contrib/wpa/src/utils/os_unix.c b/contrib/wpa/src/utils/os_unix.c index 23a93be..7a779f6 100644 --- a/contrib/wpa/src/utils/os_unix.c +++ b/contrib/wpa/src/utils/os_unix.c @@ -153,23 +153,34 @@ static int os_daemon(int nochdir, int noclose) #endif /* __APPLE__ */ +#include +#include +#include + int os_daemonize(const char *pid_file) { #if defined(__uClinux__) || defined(__sun__) return -1; #else /* defined(__uClinux__) || defined(__sun__) */ + pid_t otherpid; + struct pidfh *pfh; + + pfh = pidfile_open(pid_file, 0600, &otherpid); + if (pfh == NULL) { + if (errno == EEXIST) { + errx(1, "Daemon already running, pid: %jd.", + (intmax_t)otherpid); + } + warn("Cannot open or create pidfile."); + } + if (os_daemon(0, 0)) { perror("daemon"); + pidfile_remove(pfh); return -1; } - if (pid_file) { - FILE *f = fopen(pid_file, "w"); - if (f) { - fprintf(f, "%u\n", getpid()); - fclose(f); - } - } + pidfile_write(pfh); return -0; #endif /* defined(__uClinux__) || defined(__sun__) */ diff --git a/usr.sbin/wpa/wpa_supplicant/Makefile b/usr.sbin/wpa/wpa_supplicant/Makefile index 11cccf3..673a04d 100644 --- a/usr.sbin/wpa/wpa_supplicant/Makefile +++ b/usr.sbin/wpa/wpa_supplicant/Makefile @@ -49,8 +49,8 @@ CFLAGS+=-DCONFIG_BACKEND_FILE \ -DCONFIG_GAS \ -DPKCS12_FUNCS #CFLAGS+= -g -DPADD+= ${LIBPCAP} -LDADD+= -lpcap +DPADD+= ${LIBPCAP} ${LIBUTIL} +LDADD+= -lpcap -lutil # User customizations to the wpa_supplicant build environment CFLAGS+=${WPA_SUPPLICANT_CFLAGS} --mhjHhnbe5PrRcwjY-- From owner-freebsd-arch@FreeBSD.ORG Sat Jun 21 14:36:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49EC0AC6; Sat, 21 Jun 2014 14:36:22 +0000 (UTC) Received: from mail-qa0-x22e.google.com (mail-qa0-x22e.google.com [IPv6:2607:f8b0:400d:c00::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE9BD27DD; Sat, 21 Jun 2014 14:36:21 +0000 (UTC) Received: by mail-qa0-f46.google.com with SMTP id i13so4079275qae.5 for ; Sat, 21 Jun 2014 07:36:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=iCBbmQj53+KCPrk51mu/9sauCvMYGORgx54ItgX9ku4=; b=Pyei78rTYl89lN3H6i37eEC1uahQpIugc3AcJt0gjQ1PcCU3Ppfm1YhPgj4zqiseJz uojSLlhC7OmsM6fpsyMuVe5hjvAVKNCJxMrp9zaJzm2sBE6f+Y2HeUD8jwSp+3Dx67K/ ulr4ISrov938rL8BylWHCL80eq7YKdYD/0ZjF9hNZ1wmzkcQDyqvVpJD3lmYLbc4rNWb HFaIDoIUTyU8u4z30B99Z6UudxO6/dPD43aamRGr4c5ENgl6zglfF8lpRTSRv9CMqdjN fTD79LmvnB6IGS+ZGGDJO+/+bF/MISsladfMd9G1Dh3o/Z6Oby17cxUo5KiB38fo91lk Uf1g== MIME-Version: 1.0 X-Received: by 10.140.35.212 with SMTP id n78mr14183241qgn.87.1403361380919; Sat, 21 Jun 2014 07:36:20 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sat, 21 Jun 2014 07:36:20 -0700 (PDT) In-Reply-To: <20140621104031.GM31367@funkthat.com> References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> Date: Sat, 21 Jun 2014 07:36:20 -0700 X-Google-Sender-Auth: SlUNOx-Pj1EtKLsMnV8dxwJclQ0 Message-ID: Subject: Re: conflict between netif and pccard_ether... From: Adrian Chadd To: John Baldwin , "freebsd-arch@freebsd.org" , Rui Paulo Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 14:36:22 -0000 Why'd you remove these: -DCONFIG_GAS \ -DPKCS12_FUNCS ? -a On 21 June 2014 03:40, John-Mark Gurney wrote: > John Baldwin wrote this message on Thu, Jun 19, 2014 at 09:17 -0400: >> On Wednesday, June 18, 2014 7:13:18 pm John-Mark Gurney wrote: >> > So, I recently was trying to figure out why wireless on my notebook >> > wouldn't work.. I would boot up the machine w/o the wireless >> > configured, uncomment the lines in rc.conf, and then run >> > "service netif start"... >> > >> > Wireless would associate, but when disconnect... After some >> > investigation, it turns out that two copies of wpa_supplicant are >> > being launched... I believe one from netif, and another from >> > pccard_ether launched by devd... >> > >> > The issue is that both netif and pccard_ether "claim" ownership of >> > them. pccard_ether will ignore the ifconfig_ line if NOAUTO is >> > specified. IMO, we need to make one or the other "own" configuring >> > and launching the interface... >> > >> > There is also the issue that wpa_sup doesn't use proper locking on >> > the pidfile and allows two copies to be launched... My thoughts >> > is to convert it to pidfile to fix this issue the easiest... >> >> I think this is actually the best fix. dhclient handles this correctly for >> this reason. >> >> > Comments? Suggestions? >> >> We had a thread a few months ago about this very topic and I committed changes >> to the rc.d scripts that I then had to revert because it broke other use >> cases. Having the redundant start be harmless is the simplest way to handle >> this. > > Ok, I converted wpa_supplicant to use pidfile(3), and it successfully > fixes the issue I had... I get the error that wpa_supplicant is already > running... It looks like in my case, devd was beating out netif in > launching wpa_sup... > > I've attached the patch... > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Jun 21 16:54:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B0C35F5; Sat, 21 Jun 2014 16:54:20 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2001:470:1:2d5:26:3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id E542721F3; Sat, 21 Jun 2014 16:54:19 +0000 (UTC) Received: from [IPv6:2601:9:8280:426:5de5:fb76:6a72:abdb] (unknown [IPv6:2601:9:8280:426:5de5:fb76:6a72:abdb]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id DD11134A9D8; Sat, 21 Jun 2014 09:54:18 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: conflict between netif and pccard_ether... From: Rui Paulo In-Reply-To: <20140621104031.GM31367@funkthat.com> Date: Sat, 21 Jun 2014 09:54:18 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <9318E9E1-DF4C-4126-BD93-AF215BE1720A@FreeBSD.org> References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.1878.2) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 16:54:20 -0000 On Jun 21, 2014, at 3:40, John-Mark Gurney wrote: > We would prefer to submit this to the vendor, so if you can, please make = sure this code compiles only on FreeBSD while leaving the rest = untouched. -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Sat Jun 21 23:18:55 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0E81B5B1; Sat, 21 Jun 2014 23:18:55 +0000 (UTC) Received: from mail.netbsd.org (mail.NetBSD.org [IPv6:2001:4f8:3:7::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D63412D7A; Sat, 21 Jun 2014 23:18:54 +0000 (UTC) Received: from ws (localhost [IPv6:::1]) by mail.netbsd.org (Postfix) with SMTP id 394A914A2D0; Sat, 21 Jun 2014 23:18:53 +0000 (UTC) Date: Sun, 22 Jun 2014 00:18:52 +0100 From: Mindaugas Rasiukevicius To: Andrey Zonov Subject: Re: PoC: passive serialization In-Reply-To: <539FEBC1.5030501@FreeBSD.org> References: <539FEBC1.5030501@FreeBSD.org> X-Mailer: mail(1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <20140621231853.394A914A2D0@mail.netbsd.org> Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2014 23:18:55 -0000 Andrey Zonov wrote: > Hi, > > I'd like to introduce my implementation of passive serialization [1] > (RCU-like algorithm) which based on expired patent #4809168 [2]. > > This algorithm allows one to access data structures in non-blocking > lock-less manner, but it is not just read-write lock alternative it is > something different. It is like delayed garbage collector or if you > know what RCU is, passive serialization basically the same (RCU based on > that). Unlike NetBSD's implementation [3] my version is light-weight, > with no additional locks. One atomic operation per context switch is > the only overhead. To demonstrate how it works I converted process > limits to use that mechanism [4]. There is also simple test [5] if you > interested in measurements. Just configure which type of lock do you > want to use (mutex/rwlock/rmlock/psz), duration (LOOPS) and load > (LENGTH) and run `make -s run' to get numbers. Just a note on passive serialization in NetBSD: there is a lot of space for optimisations, simplifications or improvements to that code, but it was a deliberate choice to avoid them. The goal was to carefully implement the logic described in the expired patent (or at least attempt to be as close as our interpretation skills allow us to be). Any deviation from that logic increases the risk of falling under some other technique, primarily RCU, covered by other patent. RCU is not a single patent. It is a portfolio of multiple patents covering various aspects of RCU. Some of the original RCU patents have expired or lapsed (see 5,608,893; 5,727,209; 6,219,690; 6,886,162; 5,442,758 is still unclear), however there are plenty of others covering optimisations or variations of RCU (e.g. 8,055,860 and 8,706,706) as well as or particular applications (e.g. 8,666,952). More are being filled. RCU-like techniques are a patent minefield. So, to clarify my point: be careful with simplifications. -- Mindaugas From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 00:34:59 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B84DA1FA; Sun, 22 Jun 2014 00:34:59 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 77D7B22C2; Sun, 22 Jun 2014 00:34:59 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5M0YrM0006447 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 21 Jun 2014 17:34:53 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5M0YqFE006446; Sat, 21 Jun 2014 17:34:52 -0700 (PDT) (envelope-from jmg) Date: Sat, 21 Jun 2014 17:34:52 -0700 From: John-Mark Gurney To: Rui Paulo Subject: Re: conflict between netif and pccard_ether... Message-ID: <20140622003452.GR31367@funkthat.com> Mail-Followup-To: Rui Paulo , John Baldwin , freebsd-arch@FreeBSD.org References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> <9318E9E1-DF4C-4126-BD93-AF215BE1720A@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="rwgQ89ZNnFUwFHTC" Content-Disposition: inline In-Reply-To: <9318E9E1-DF4C-4126-BD93-AF215BE1720A@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 21 Jun 2014 17:34:53 -0700 (PDT) Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 00:34:59 -0000 --rwgQ89ZNnFUwFHTC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Rui Paulo wrote this message on Sat, Jun 21, 2014 at 09:54 -0700: > On Jun 21, 2014, at 3:40, John-Mark Gurney wrote: > > > > > We would prefer to submit this to the vendor, so if you can, please make sure this code compiles only on FreeBSD while leaving the rest untouched. Whee diff -D! New patch attached... Will you submit it upstream? and when can we expect this to hit head? Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --rwgQ89ZNnFUwFHTC Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="os_unix.c.patch" --- os_unix.c.orig 2014-06-22 07:20:23.000000000 +0700 +++ os_unix.c 2014-06-22 07:32:52.000000000 +0700 @@ -153,16 +153,40 @@ #endif /* __APPLE__ */ +#ifdef __FreeBSD__ +#include +#include +#include +#endif /* __FreeBSD__ */ + int os_daemonize(const char *pid_file) { #if defined(__uClinux__) || defined(__sun__) return -1; #else /* defined(__uClinux__) || defined(__sun__) */ +#ifdef __FreeBSD__ + pid_t otherpid; + struct pidfh *pfh; + + pfh = pidfile_open(pid_file, 0600, &otherpid); + if (pfh == NULL) { + if (errno == EEXIST) { + errx(1, "Daemon already running, pid: %jd.", + (intmax_t)otherpid); + } + warn("Cannot open or create pidfile."); + } +#endif /* __FreeBSD__ */ + if (os_daemon(0, 0)) { perror("daemon"); +#ifdef __FreeBSD__ + pidfile_remove(pfh); +#endif /* __FreeBSD__ */ return -1; } +#ifndef __FreeBSD__ if (pid_file) { FILE *f = fopen(pid_file, "w"); if (f) { @@ -170,6 +194,9 @@ fclose(f); } } +#else /* __FreeBSD__ */ + pidfile_write(pfh); +#endif /* __FreeBSD__ */ return -0; #endif /* defined(__uClinux__) || defined(__sun__) */ --rwgQ89ZNnFUwFHTC-- From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 00:47:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D69492FC; Sun, 22 Jun 2014 00:47:00 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B13EB2366; Sun, 22 Jun 2014 00:47:00 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5M0kxn3006648 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 21 Jun 2014 17:47:00 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5M0kxGu006647; Sat, 21 Jun 2014 17:46:59 -0700 (PDT) (envelope-from jmg) Date: Sat, 21 Jun 2014 17:46:59 -0700 From: John-Mark Gurney To: Adrian Chadd Subject: Re: conflict between netif and pccard_ether... Message-ID: <20140622004659.GS31367@funkthat.com> Mail-Followup-To: Adrian Chadd , John Baldwin , "freebsd-arch@freebsd.org" , Rui Paulo References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 21 Jun 2014 17:47:00 -0700 (PDT) Cc: Rui Paulo , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 00:47:01 -0000 Adrian Chadd wrote this message on Sat, Jun 21, 2014 at 07:36 -0700: > Why'd you remove these: > > -DCONFIG_GAS \ > -DPKCS12_FUNCS Hun? I didn't... I think your mail client is broken and ate the white space at the begining of the line... > On 21 June 2014 03:40, John-Mark Gurney wrote: > > John Baldwin wrote this message on Thu, Jun 19, 2014 at 09:17 -0400: > >> On Wednesday, June 18, 2014 7:13:18 pm John-Mark Gurney wrote: > >> > So, I recently was trying to figure out why wireless on my notebook > >> > wouldn't work.. I would boot up the machine w/o the wireless > >> > configured, uncomment the lines in rc.conf, and then run > >> > "service netif start"... > >> > > >> > Wireless would associate, but when disconnect... After some > >> > investigation, it turns out that two copies of wpa_supplicant are > >> > being launched... I believe one from netif, and another from > >> > pccard_ether launched by devd... > >> > > >> > The issue is that both netif and pccard_ether "claim" ownership of > >> > them. pccard_ether will ignore the ifconfig_ line if NOAUTO is > >> > specified. IMO, we need to make one or the other "own" configuring > >> > and launching the interface... > >> > > >> > There is also the issue that wpa_sup doesn't use proper locking on > >> > the pidfile and allows two copies to be launched... My thoughts > >> > is to convert it to pidfile to fix this issue the easiest... > >> > >> I think this is actually the best fix. dhclient handles this correctly for > >> this reason. > >> > >> > Comments? Suggestions? > >> > >> We had a thread a few months ago about this very topic and I committed changes > >> to the rc.d scripts that I then had to revert because it broke other use > >> cases. Having the redundant start be harmless is the simplest way to handle > >> this. > > > > Ok, I converted wpa_supplicant to use pidfile(3), and it successfully > > fixes the issue I had... I get the error that wpa_supplicant is already > > running... It looks like in my case, devd was beating out netif in > > launching wpa_sup... > > > > I've attached the patch... > > > > -- > > John-Mark Gurney Voice: +1 415 225 5579 > > > > "All that I will do, has been done, All that I have, has not." > > > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 01:26:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05A527D2; Sun, 22 Jun 2014 01:26:03 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88C3425E3; Sun, 22 Jun 2014 01:26:02 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id o8so4862015qcw.17 for ; Sat, 21 Jun 2014 18:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=jl0eH4/JasXf9rVrI/3w9L8XWOHbJ/ZnJGTZiE/GDXo=; b=iVXTmWnTT0sPGvjT2ORn6fgxPG1+Cx2aBlHEPd6L8pZoWlYRtdkxcvL010y8A1xmv3 tLm9Wsupmh5TBe18XemDJQzXkP0gappkotJzUz4KRj7klttLRBzN1R2/1yOGD4NAQlJE b5CVH9zthyJ+wueFtjEIbsG5o2XYgvXdyYOEGeBu4Na1/JQ/Cm+Z+Hos5klfVeELpNDp 8P/pDWL/tqcJIFiZfeK3cy/VPTZyBpRp+mlZ7CYKQj98VlqfwVLyrn0h9V+DOwG9h9uT 3HJr3aUNVqtpWG4VWk9XuQCU6iE9k1btjEcxPTsGPgzOypIAEvfkmpIMnHqoSJW1ePmO t8lw== MIME-Version: 1.0 X-Received: by 10.140.80.49 with SMTP id b46mr15469723qgd.102.1403400361260; Sat, 21 Jun 2014 18:26:01 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sat, 21 Jun 2014 18:26:01 -0700 (PDT) In-Reply-To: <20140622004659.GS31367@funkthat.com> References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> <20140622004659.GS31367@funkthat.com> Date: Sat, 21 Jun 2014 18:26:01 -0700 X-Google-Sender-Auth: eFynlWewgauxSqUjiXEt-LXmmu0 Message-ID: Subject: Re: conflict between netif and pccard_ether... From: Adrian Chadd To: Adrian Chadd , John Baldwin , "freebsd-arch@freebsd.org" , Rui Paulo Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 01:26:03 -0000 On 21 June 2014 17:46, John-Mark Gurney wrote: > Adrian Chadd wrote this message on Sat, Jun 21, 2014 at 07:36 -0700: >> Why'd you remove these: >> >> -DCONFIG_GAS \ >> -DPKCS12_FUNCS > > Hun? I didn't... I think your mail client is broken and ate the > white space at the begining of the line... Cool. I blame gmail. -a From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 04:55:34 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F146EDC8 for ; Sun, 22 Jun 2014 04:55:34 +0000 (UTC) Received: from felyko.com (felyko.com [IPv6:2001:470:1:2d5:26:3:1337:ca7]) by mx1.freebsd.org (Postfix) with ESMTP id D141F2411 for ; Sun, 22 Jun 2014 04:55:34 +0000 (UTC) Received: from [IPv6:2601:9:8280:426:8c98:7005:171b:a7bb] (unknown [IPv6:2601:9:8280:426:8c98:7005:171b:a7bb]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by felyko.com (Postfix) with ESMTPSA id 98B2834A9D8; Sat, 21 Jun 2014 21:55:33 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: conflict between netif and pccard_ether... From: Rui Paulo In-Reply-To: <20140622003452.GR31367@funkthat.com> Date: Sat, 21 Jun 2014 21:55:33 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> <9318E9E1-DF4C-4126-BD93-AF215BE1720A@FreeBSD.org> <20140622003452.GR31367@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.1878.2) Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 04:55:35 -0000 On Jun 21, 2014, at 17:34, John-Mark Gurney wrote: > Rui Paulo wrote this message on Sat, Jun 21, 2014 at 09:54 -0700: >> On Jun 21, 2014, at 3:40, John-Mark Gurney wrote: >>=20 >>> >>=20 >> We would prefer to submit this to the vendor, so if you can, please = make sure this code compiles only on FreeBSD while leaving the rest = untouched. >=20 > Whee diff -D! >=20 > New patch attached... Will you submit it upstream? and when can we > expect this to hit head? The upstream repository is here: http://hostap.epitest.fi/cgit/hostap/ Is there any way I can convince you to produce a patch suitable for this = repository? We can them email it to Jouni. It will be a while before I can update the tools in FreeBSD, so go ahead = and commit it. Thanks, -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 05:28:21 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E94A3270; Sun, 22 Jun 2014 05:28:20 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BE53B25DF; Sun, 22 Jun 2014 05:28:20 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s5M5SI5k010006 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 21 Jun 2014 22:28:19 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s5M5SIEQ010005; Sat, 21 Jun 2014 22:28:18 -0700 (PDT) (envelope-from jmg) Date: Sat, 21 Jun 2014 22:28:18 -0700 From: John-Mark Gurney To: Rui Paulo Subject: Re: conflict between netif and pccard_ether... Message-ID: <20140622052818.GU31367@funkthat.com> Mail-Followup-To: Rui Paulo , freebsd-arch@FreeBSD.org References: <20140618231318.GH31367@funkthat.com> <201406190917.16458.jhb@freebsd.org> <20140621104031.GM31367@funkthat.com> <9318E9E1-DF4C-4126-BD93-AF215BE1720A@FreeBSD.org> <20140622003452.GR31367@funkthat.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="6K2R/cS9K4qvcBNq" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 21 Jun 2014 22:28:19 -0700 (PDT) Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 05:28:21 -0000 --6K2R/cS9K4qvcBNq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Rui Paulo wrote this message on Sat, Jun 21, 2014 at 21:55 -0700: > On Jun 21, 2014, at 17:34, John-Mark Gurney wrote: > > > Rui Paulo wrote this message on Sat, Jun 21, 2014 at 09:54 -0700: > >> On Jun 21, 2014, at 3:40, John-Mark Gurney wrote: > >> > >>> > >> > >> We would prefer to submit this to the vendor, so if you can, please make sure this code compiles only on FreeBSD while leaving the rest untouched. > > > > Whee diff -D! > > > > New patch attached... Will you submit it upstream? and when can we > > expect this to hit head? > > > The upstream repository is here: > > http://hostap.epitest.fi/cgit/hostap/ > > Is there any way I can convince you to produce a patch suitable for this repository? We can them email it to Jouni. Ok, it's attached, but I don't know how to properly test using that repo, so it hasn't been compiled or anything... I just took the diff that worked, and applied it to the repo, and there was some offset, but applied clealy otherwise... > It will be a while before I can update the tools in FreeBSD, so go ahead and commit it. I'll commit it shortly... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --6K2R/cS9K4qvcBNq Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="hostap.git.patch" diff --git a/src/utils/os_unix.c b/src/utils/os_unix.c index 008ec6b..5853375 100644 --- a/src/utils/os_unix.c +++ b/src/utils/os_unix.c @@ -190,16 +190,40 @@ static int os_daemon(int nochdir, int noclose) #endif /* __APPLE__ */ +#ifdef __FreeBSD__ +#include +#include +#include +#endif /* __FreeBSD__ */ + int os_daemonize(const char *pid_file) { #if defined(__uClinux__) || defined(__sun__) return -1; #else /* defined(__uClinux__) || defined(__sun__) */ +#ifdef __FreeBSD__ + pid_t otherpid; + struct pidfh *pfh; + + pfh = pidfile_open(pid_file, 0600, &otherpid); + if (pfh == NULL) { + if (errno == EEXIST) { + errx(1, "Daemon already running, pid: %jd.", + (intmax_t)otherpid); + } + warn("Cannot open or create pidfile."); + } +#endif /* __FreeBSD__ */ + if (os_daemon(0, 0)) { perror("daemon"); +#ifdef __FreeBSD__ + pidfile_remove(pfh); +#endif /* __FreeBSD__ */ return -1; } +#ifndef __FreeBSD__ if (pid_file) { FILE *f = fopen(pid_file, "w"); if (f) { @@ -207,6 +231,9 @@ int os_daemonize(const char *pid_file) fclose(f); } } +#else /* __FreeBSD__ */ + pidfile_write(pfh); +#endif /* __FreeBSD__ */ return -0; #endif /* defined(__uClinux__) || defined(__sun__) */ --6K2R/cS9K4qvcBNq-- From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 12:35:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 899988BC; Sun, 22 Jun 2014 12:35:03 +0000 (UTC) Received: from mail-we0-x22c.google.com (mail-we0-x22c.google.com [IPv6:2a00:1450:400c:c03::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F25622269; Sun, 22 Jun 2014 12:35:02 +0000 (UTC) Received: by mail-we0-f172.google.com with SMTP id u57so5709095wes.17 for ; Sun, 22 Jun 2014 05:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=IX/jmEMpxXZUhkgH87iebJV6jPgfS5yiyDtpOD2yrpM=; b=wgZMldyNHWqeinF3/BtpJkBl0Qn6OemMsGHuvELBS3QXD3C/38O7Dxn6Rc7YFHvWN2 UDHtIPwlvFmfGuY+orhE6SRekR0EQe95lN4Es6Tw2HOJsxsxjCjEPVw5UUkVBqafOz2+ 5FVlW9XFgxuiV+BwPnmwEAZSBNl/uheNv38hR/zgwZduh8zjRPaACTN8fB83Q8+bQvPc oS7wE8Ev3OigoUnkO5nA/oDbLp/xiCMgttkiEqSfHIAPIrVoiuwua92cIlJUaGMklr/f i/UU1uHKFvO/wcCvNXW4+2zgZDqY7A1YZYVrjoOdO7FNMeibxmDZm6gsjqW11EDvtxjf 0+xQ== X-Received: by 10.194.77.177 with SMTP id t17mr18769661wjw.55.1403440500871; Sun, 22 Jun 2014 05:35:00 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id ww4sm1324556wjc.4.2014.06.22.05.34.59 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sun, 22 Jun 2014 05:35:00 -0700 (PDT) Date: Sun, 22 Jun 2014 14:34:57 +0200 From: Mateusz Guzik To: Mindaugas Rasiukevicius Subject: Re: PoC: passive serialization Message-ID: <20140622123457.GA20525@dft-labs.eu> References: <539FEBC1.5030501@FreeBSD.org> <20140621231853.394A914A2D0@mail.netbsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140621231853.394A914A2D0@mail.netbsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Andrey Zonov , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 12:35:03 -0000 On Sun, Jun 22, 2014 at 12:18:52AM +0100, Mindaugas Rasiukevicius wrote: > Just a note on passive serialization in NetBSD: there is a lot of space for > optimisations, simplifications or improvements to that code, but it was a > deliberate choice to avoid them. The goal was to carefully implement the > logic described in the expired patent (or at least attempt to be as close as > our interpretation skills allow us to be). Any deviation from that logic > increases the risk of falling under some other technique, primarily RCU, > covered by other patent. > Cursory look suggests passive serialization got very limited use in NetBSD, I was also unable to find any benchmarks. I do understand and appreciate your cautious approach, but it also limits the potential. Do you have any materials which would show gains of using the concept as implemented in your code? Right now it is unclear if the implementation limited to this patent can yield benefits which would justify added complexity. Thanks, -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 18:52:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCE0CF91 for ; Sun, 22 Jun 2014 18:52:52 +0000 (UTC) Received: from eastrmfepo203.cox.net (eastrmfepo203.cox.net [68.230.241.218]) by mx1.freebsd.org (Postfix) with ESMTP id 7663A2CDB for ; Sun, 22 Jun 2014 18:52:52 +0000 (UTC) Received: from eastrmimpo210 ([68.230.241.225]) by eastrmfepo203.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140622185246.XLXM2658.eastrmfepo203.cox.net@eastrmimpo210> for ; Sun, 22 Jun 2014 14:52:46 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo210 with cox id HWsl1o00J41obj401WsmYa; Sun, 22 Jun 2014 14:52:46 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020207.53A725FE.004F,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=aZC/a2Ut c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=IkcTkHD0fZMA:10 a=kviXuzpPAAAA:8 a=pGLkceISAAAA:8 a=6I5d2MoRAAAA:8 a=x-eTLskepIULTrIbjOMA:9 a=QEXdDO2ut3YA:10 a=MSl-tDqOz04A:10 a=SV7veod9ZcQA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53A72666.8090101@cox.net> Date: Sun, 22 Jun 2014 14:54:30 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: Eitan Adler Subject: Re: Improve cron(8) References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org, =?UTF-8?B?VG9tZWsgV2HFgmFzemVr?= , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 18:52:53 -0000 Eitan Adler wrote: > +arch since hackers@ seems to be silent. > > On 11 June 2014 23:56, Tomek WaÅ‚aszek wrote: >> Hello, >> I saw on the FreeBSD Ideas page topic about cron :). >> I've started updating the 'original' FreeBSD cron from sources to vixi cron >> 4.1. I think (well I hope :P) most of the features that were done in >> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately >> some missing features at the moment: >> - @every_second - this need to be done >> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >> default, at the moment there is no -s and -o options. So you need to remove >> '-s' from the cron rc script >> >> I've also added one feature from OpenBSD, crontab is poking cron using >> unix-domain socket so we don't need to have suid on crontab. >> >> Path is in the attachment. I'm testing it on my FreeBSD box and it looks >> good but anyway don't try it on production machines :). >> >> After the installation we have to do a few things: >> - Add crontab group >> - Change group to crontab on /var/cron/tabs >> - Add sticky bit on /var/cron/tabs >> - Add group write permissions on /var/cron/tabs >> >> This is still work in progress but if someone could have a look on this and >> give me some feedback it would be great. >> >> Regards, >> Tomasz Walaszek >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > you should up the version number or start your own renamed application a person claiming to be paul vixie of cron is still living (doesn't look old enough to be a vax hacker?). your not paul vixie. people who use vixie cron don't want a "distro hacked" one. also there are many cron replacements with frills features already for people to dl and use. i don't know of any serious bug fixes to cron. it simple (well not for paul to write it but) and anyone still using cron likely uses it because it is simple and un-tampered with thanks but please leave very basic apps "cron, ls, sed, mail" so they are not a compatibility problem - because many systems or softwares rely on them like , say, they rely on awk From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 22:16:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BCD85EB7 for ; Sun, 22 Jun 2014 22:16:28 +0000 (UTC) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 780602B16 for ; Sun, 22 Jun 2014 22:16:28 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id w8so4900716qac.8 for ; Sun, 22 Jun 2014 15:16:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=qqlr/IbyP0iJjES3fu5K76tw8XCPcwuTBLXZozCqw9Q=; b=g935Y+ELYfDoTtAq7onZIp02rGd2cFF8PD7kOoxJ1vEVSyJCbdoyXFi/RfIoJ002IT 7bQEaoFGyB9unOpp5Vk7FSJd4/NTmSyx9xqSaxG3BQNCjBrNedpvdCFQEz4OFR7DwwAz puJOJ3vYwrHXKFpJ8rRzrFakZSuy+BgqaC9w8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=qqlr/IbyP0iJjES3fu5K76tw8XCPcwuTBLXZozCqw9Q=; b=htlObF/43lq6KNSQtiOssV4kw6Sn0Ma96NnWA/8Q7rfcnyXxtWgNUmrSbRr2h28IUW fh94v7hQLRqKt2vMilLd0mVdMBuHLTNmAQG3J0nnfl+ZIKMVtjNOUDxA8C6PYx3ShskP LbX8uIeacLxuWk26apra0q6hfxgsBtCunAXkFSr39vD0jKg5t6m88iLaJG4pPs0m8CnQ JZWlkJD4yYkFDsLXDeLGtlQDdBODFIw31l38pZRbRqKJbFpoxQkOpLrE5fTEBXi16YEf TX6ZDjRxI+CZGophD3oGuIr/orLJSUA7+e+oad1M1xfsxl41wrMB0KaUfb93mg96jy5s w58w== X-Gm-Message-State: ALoCoQm0ejJ0MYYRQhKgOyEV98xKdyiFY6RgrrQLDplHbb5r3np8kLDn0EJHomAEOxIVBOMkajR1 X-Received: by 10.224.7.6 with SMTP id b6mr27141563qab.45.1403475387608; Sun, 22 Jun 2014 15:16:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.222.131 with HTTP; Sun, 22 Jun 2014 15:15:57 -0700 (PDT) In-Reply-To: <53A72666.8090101@cox.net> References: <53A72666.8090101@cox.net> From: Eitan Adler Date: Sun, 22 Jun 2014 15:15:57 -0700 Message-ID: Subject: Re: Improve cron(8) To: johnandsara2@cox.net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, =?UTF-8?Q?Tomek_Wa=C5=82aszek?= , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 22:16:28 -0000 On 22 June 2014 11:54, John D. Hendrickson and Sara Darnell wrote: > Eitan Adler wrote: >> >> +arch since hackers@ seems to be silent. >> >> On 11 June 2014 23:56, Tomek Wa=C5=82aszek wrote: >>> >>> Hello, >>> I saw on the FreeBSD Ideas page topic about cron :). >>> I've started updating the 'original' FreeBSD cron from sources to vixi >>> cron >>> 4.1. I think (well I hope :P) most of the features that were done in >>> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately >>> some missing features at the moment: >>> - @every_second - this need to be done >>> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >>> default, at the moment there is no -s and -o options. So you need to >>> remove >>> '-s' from the cron rc script >>> >>> I've also added one feature from OpenBSD, crontab is poking cron using >>> unix-domain socket so we don't need to have suid on crontab. >>> >>> Path is in the attachment. I'm testing it on my FreeBSD box and it look= s >>> good but anyway don't try it on production machines :). >>> >>> After the installation we have to do a few things: >>> - Add crontab group >>> - Change group to crontab on /var/cron/tabs >>> - Add sticky bit on /var/cron/tabs >>> - Add group write permissions on /var/cron/tabs >>> >>> This is still work in progress but if someone could have a look on this >>> and >>> give me some feedback it would be great. >>> >>> Regards, >>> Tomasz Walaszek >>> >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to >>> "freebsd-hackers-unsubscribe@freebsd.org" >> >> >> >> > > you should up the version number or start your own renamed application ... I don't know how to react to this message. You just told a potential contributor not to contribute since he was not the founder of the project. That goes against everything that open source is about. --=20 Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Sun Jun 22 22:18:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D29BE290 for ; Sun, 22 Jun 2014 22:18:39 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CEA72B2D for ; Sun, 22 Jun 2014 22:18:39 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id m20so5456617qcx.27 for ; Sun, 22 Jun 2014 15:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; bh=YmLXjdCNmBRRAeNZ4W2XuG64is/1kS4GuHMg0UH/sYs=; b=AwoBpvlqYZbA5dtZbblzDk+qVmS3CihRdl7t3xeWgXYLB35JizM9WMm7vW3s3oJCZD IbN+LQOhlQtyQr8P59JaR3M+L3WLLbkTdwOMlT5/ZYEsHXo8YYUItvKNMqypRj+3dxhy Z7abKocdNoXHDXwZe3v1Y6PhiKILhwqgSAQgg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=YmLXjdCNmBRRAeNZ4W2XuG64is/1kS4GuHMg0UH/sYs=; b=GVUkpbWRrskBfLEW1eIzYZDFbSFWNARIrIwJES7cglnjO4QR8kUR00qgwqRPmD+fVN 841mVk2vjQ7mAnGqtKU70yPfCLooBPgL35wjKnZBhDiaGZlhtYviCJQsMjQR5qUiAl/J yA93OqV/ZUUznDRoAjUIH773NGIsjkKSpNf4jSABzBaWDS4duxKHIXigV20hXjXWOhhM d3oOE7+FSUXRDeu+/IJ+9NfRmFnEVMYpTnzosgWRL5jipH3w2ZFAwHaQ6XpINV8Bvxml sEELL+j+c+jH8ZlR5LyNeShDWNBX+wmxWc1N4aTFsz7vrzV8ZXTpLTs+P+6xHydjD/AD XEbw== X-Gm-Message-State: ALoCoQm16Ykb9HG337oebEoyg5iVPl9pyDR24U2RcBJVwaU0X+LDZ4s+IKTTuYalvVW5rcqLemo3 X-Received: by 10.224.7.6 with SMTP id b6mr27151058qab.45.1403475518730; Sun, 22 Jun 2014 15:18:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.96.222.131 with HTTP; Sun, 22 Jun 2014 15:18:08 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Sun, 22 Jun 2014 15:18:08 -0700 Message-ID: Subject: Re: Improve cron(8) To: =?UTF-8?Q?Tomek_Wa=C5=82aszek?= , freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2014 22:18:40 -0000 On 13 June 2014 00:04, Eitan Adler wrote: > +arch since hackers@ seems to be silent. > > On 11 June 2014 23:56, Tomek Wa=C5=82aszek wrote: >> Hello, >> I saw on the FreeBSD Ideas page topic about cron :). >> I've started updating the 'original' FreeBSD cron from sources to vixi c= ron >> 4.1. I think (well I hope :P) most of the features that were done in >> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately >> some missing features at the moment: >> - @every_second - this need to be done >> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >> default, at the moment there is no -s and -o options. So you need to rem= ove >> '-s' from the cron rc script >> >> I've also added one feature from OpenBSD, crontab is poking cron using >> unix-domain socket so we don't need to have suid on crontab. >> >> Path is in the attachment. I'm testing it on my FreeBSD box and it looks >> good but anyway don't try it on production machines :). >> >> After the installation we have to do a few things: >> - Add crontab group >> - Change group to crontab on /var/cron/tabs >> - Add sticky bit on /var/cron/tabs >> - Add group write permissions on /var/cron/tabs >> >> This is still work in progress but if someone could have a look on this = and >> give me some feedback it would be great. Tomek, Could you please file a bug with this patch? I'll take a look when I have time, but hopefully someone will get to it first. Also, can you please generate the patch against HEAD and not 10.0.0? --=20 Eitan Adler From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 03:00:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5CAF9202; Mon, 23 Jun 2014 03:00:22 +0000 (UTC) Received: from mail-qc0-x22b.google.com (mail-qc0-x22b.google.com [IPv6:2607:f8b0:400d:c01::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B14022B5; Mon, 23 Jun 2014 03:00:21 +0000 (UTC) Received: by mail-qc0-f171.google.com with SMTP id w7so5578182qcr.16 for ; Sun, 22 Jun 2014 20:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=XCcSW4ztIdE9MZwenbVb+Y5LBD8pkZnWuG35im2njtA=; b=n2vpMMVqKOwpp4yvNDKjYLnLFcc1e2LzACABLIzWr5ElDvdinO5iZe9VIo5iGxz2mJ TzwUnuotgmvZscyN2/zUyvkeoQV/zAJ5kKrm0pfBIMEK/aKRJBgHkXmGzzptloeX4TLx bJYO+L/GcUDxMpAt0Enjl+cBPv+XMR6Z3XOXfX5equ/O92/SP5iPNTTdjEH6H3wSYQi3 dMYmN6MJloj9iZS6v56uirfQvDr/Agho7Vtss08htEmkWAMorRjFqdqLHdfT4W86fwrL IiENC0Yq75b8IjDfgGl58GnKVLn8z+0DK8DnFo7bWmJ7RLKPpZSswZyhO4sFk2rUiePV AAEw== MIME-Version: 1.0 X-Received: by 10.140.22.134 with SMTP id 6mr26254949qgn.4.1403492421197; Sun, 22 Jun 2014 20:00:21 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.43.134 with HTTP; Sun, 22 Jun 2014 20:00:21 -0700 (PDT) In-Reply-To: References: <53A72666.8090101@cox.net> Date: Sun, 22 Jun 2014 20:00:21 -0700 X-Google-Sender-Auth: bFNvfp1gGuU3DGnmgC0fOd0vCFY Message-ID: Subject: Re: Improve cron(8) From: Adrian Chadd To: Eitan Adler Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" , =?UTF-8?Q?Tomek_Wa=C5=82aszek?= , johnandsara2@cox.net, "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 03:00:22 -0000 Actually, I know how to react to that. Don't dissuade someone from contributing. Discuss the changes and whether they're positive or negative. -a On 22 June 2014 15:15, Eitan Adler wrote: > On 22 June 2014 11:54, John D. Hendrickson and Sara Darnell > wrote: >> Eitan Adler wrote: >>> >>> +arch since hackers@ seems to be silent. >>> >>> On 11 June 2014 23:56, Tomek Wa=C5=82aszek wrote= : >>>> >>>> Hello, >>>> I saw on the FreeBSD Ideas page topic about cron :). >>>> I've started updating the 'original' FreeBSD cron from sources to vixi >>>> cron >>>> 4.1. I think (well I hope :P) most of the features that were done in >>>> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunatel= y >>>> some missing features at the moment: >>>> - @every_second - this need to be done >>>> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >>>> default, at the moment there is no -s and -o options. So you need to >>>> remove >>>> '-s' from the cron rc script >>>> >>>> I've also added one feature from OpenBSD, crontab is poking cron using >>>> unix-domain socket so we don't need to have suid on crontab. >>>> >>>> Path is in the attachment. I'm testing it on my FreeBSD box and it loo= ks >>>> good but anyway don't try it on production machines :). >>>> >>>> After the installation we have to do a few things: >>>> - Add crontab group >>>> - Change group to crontab on /var/cron/tabs >>>> - Add sticky bit on /var/cron/tabs >>>> - Add group write permissions on /var/cron/tabs >>>> >>>> This is still work in progress but if someone could have a look on thi= s >>>> and >>>> give me some feedback it would be great. >>>> >>>> Regards, >>>> Tomasz Walaszek >>>> >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org" >>> >>> >>> >>> >> >> you should up the version number or start your own renamed application > ... > > I don't know how to react to this message. You just told a potential > contributor not to contribute since he was not the founder of the > project. That goes against everything that open source is about. > > > -- > Eitan Adler > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 03:45:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 42681EE6 for ; Mon, 23 Jun 2014 03:45:35 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 16D152711 for ; Mon, 23 Jun 2014 03:45:34 +0000 (UTC) Received: from Julian-MBP3.local (etroy.elischer.org [121.45.232.70]) (authenticated bits=0) by vps1.elischer.org (8.14.8/8.14.8) with ESMTP id s5N3jVAk090586 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 22 Jun 2014 20:45:33 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53A7A2D6.30905@freebsd.org> Date: Mon, 23 Jun 2014 11:45:26 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: PoC: passive serialization References: <539FEBC1.5030501@FreeBSD.org> <20140621231853.394A914A2D0@mail.netbsd.org> In-Reply-To: <20140621231853.394A914A2D0@mail.netbsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 03:45:35 -0000 On 6/22/14, 7:18 AM, Mindaugas Rasiukevicius wrote: > > Just a note on passive serialization in NetBSD: there is a lot of space for > optimisations, simplifications or improvements to that code, but it was a > deliberate choice to avoid them. The goal was to carefully implement the > logic described in the expired patent (or at least attempt to be as close as > our interpretation skills allow us to be). Any deviation from that logic > increases the risk of falling under some other technique, primarily RCU, > covered by other patent. > hopefully the recent ruling on software patents may make the whole thing moot given enough impetus. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 03:59:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64085299 for ; Mon, 23 Jun 2014 03:59:30 +0000 (UTC) Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25BD827DC for ; Mon, 23 Jun 2014 03:59:29 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h15so2509596igd.8 for ; Sun, 22 Jun 2014 20:59:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=opS7iXEBMNhF8MSlbi8+QnECsSm//b1kVqdolgyQI58=; b=koGbZDSsXIW4HEJwkhTpGqJRCwjXC63v4dCcaCSs5lQQhsD4vRnIzR3SWhLPUBPoNP w9VPUfBQZYfAIOqAVczCOPWvG0+U5oDadmINEE+dp7FkNIsxlUZgvDAbXu6ktMIRurXr iDYb0lsEYECQTACJ7lQ9E2dHBJdCEa1MwlX9SzFxFNxKdBNG0npRo3Ron0jVtxU/KoWM YSah1Gv2Fnq2xicvU6GXINCctqJ6/La5p6iSu7KqV+QYGjGtBnukocrq1npDxXxmrFsG jSKX4cZ4eyK/6laujyCDeKwZ0F3XcgDasFnyl+L26e2vPE4nO3N9koezFgq/u+m0vk8o u1gA== X-Gm-Message-State: ALoCoQkXi5LPAwXvKFCyCeCYq4NfTvgZhOJ+x87M0fgGPda3bbugN8vCOmNDtCqsbrBrUDPMltKm X-Received: by 10.42.25.19 with SMTP id y19mr20102652icb.45.1403495963255; Sun, 22 Jun 2014 20:59:23 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id ie13sm28948606igb.10.2014.06.22.20.59.22 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 22 Jun 2014 20:59:22 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_7098E7B9-D15F-4923-A4A2-C23EE267BE53"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Improve cron(8) From: Warner Losh In-Reply-To: Date: Sun, 22 Jun 2014 21:59:22 -0600 Message-Id: <348D4AE4-FAFD-45A7-A3E8-C507FBF5450A@bsdimp.com> References: <53A72666.8090101@cox.net> To: Adrian Chadd X-Mailer: Apple Mail (2.1878.2) Cc: Eitan Adler , "freebsd-arch@freebsd.org" , =?utf-8?Q?Tomek_Wa=C5=82aszek?= , johnandsara2@cox.net, "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 03:59:30 -0000 --Apple-Mail=_7098E7B9-D15F-4923-A4A2-C23EE267BE53 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Yea, The Paul Vixie I know never would be a lot nicer about it. The version number and name are the least of our worries here. Warner On Jun 22, 2014, at 9:00 PM, Adrian Chadd wrote: > Actually, I know how to react to that. >=20 > Don't dissuade someone from contributing. Discuss the changes and > whether they're positive or negative. >=20 > -a >=20 >=20 > On 22 June 2014 15:15, Eitan Adler wrote: >> On 22 June 2014 11:54, John D. Hendrickson and Sara Darnell >> wrote: >>> Eitan Adler wrote: >>>>=20 >>>> +arch since hackers@ seems to be silent. >>>>=20 >>>> On 11 June 2014 23:56, Tomek Wa=C5=82aszek = wrote: >>>>>=20 >>>>> Hello, >>>>> I saw on the FreeBSD Ideas page topic about cron :). >>>>> I've started updating the 'original' FreeBSD cron from sources to = vixi >>>>> cron >>>>> 4.1. I think (well I hope :P) most of the features that were done = in >>>>> FreeBSD cron are now ported into vixi cron 4.1, there are = unfortunately >>>>> some missing features at the moment: >>>>> - @every_second - this need to be done >>>>> - -s and -o, in vixi cron 4.1 daylight time switches are enabled = by >>>>> default, at the moment there is no -s and -o options. So you need = to >>>>> remove >>>>> '-s' from the cron rc script >>>>>=20 >>>>> I've also added one feature from OpenBSD, crontab is poking cron = using >>>>> unix-domain socket so we don't need to have suid on crontab. >>>>>=20 >>>>> Path is in the attachment. I'm testing it on my FreeBSD box and it = looks >>>>> good but anyway don't try it on production machines :). >>>>>=20 >>>>> After the installation we have to do a few things: >>>>> - Add crontab group >>>>> - Change group to crontab on /var/cron/tabs >>>>> - Add sticky bit on /var/cron/tabs >>>>> - Add group write permissions on /var/cron/tabs >>>>>=20 >>>>> This is still work in progress but if someone could have a look on = this >>>>> and >>>>> give me some feedback it would be great. >>>>>=20 >>>>> Regards, >>>>> Tomasz Walaszek >>>>>=20 >>>>> _______________________________________________ >>>>> freebsd-hackers@freebsd.org mailing list >>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>>> To unsubscribe, send any mail to >>>>> "freebsd-hackers-unsubscribe@freebsd.org" >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>=20 >>> you should up the version number or start your own renamed = application >> ... >>=20 >> I don't know how to react to this message. You just told a potential >> contributor not to contribute since he was not the founder of the >> project. That goes against everything that open source is about. >>=20 >>=20 >> -- >> Eitan Adler >> _______________________________________________ >> freebsd-arch@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-arch >> To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" --Apple-Mail=_7098E7B9-D15F-4923-A4A2-C23EE267BE53 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTp6YaAAoJEGwc0Sh9sBEAUM8QANJjeClSdpspLpygE1EYHMMV UanuiqAiqCMaa4d/0+97iVl7olidWNjyZjW5gP830fWp0QQrUStgpmtioBr2Zqae 5qyRUOGgoPkXQB4wgsBVIdpusgHKM2Fm7ZOewb0h9jVJ5BX3YKlpKbDYJZje15Zy fs7wwmpJ2EUkXPxrJRgQMT3mCtpiW8AyWc3O0/s0DlDBlQD2oYZcPeJkbts+++1i RLF3UV/ifBt1jmwG5oCwDOBtHwPDu6aRhH/VutgP+KG0pTpDZ6IYoB56RGeoapfT SZ9hf71St3GHiaWyBGvutiTp9ggs3ebdUTHkpGyJe6UHVGwv3aEW8EogZidsjdis Ro5qmA9PW818aNfw1UrDz9PWmKdO7L7t9tAgUB+TE087kn/LxBFz44AjsbGcdbCg w1yajo7BoqHGE3k3KFl/7kuJ9idP8vonBZuNdx9+k0G0wcdcOWkGTBtaaqNWUOP4 3vjNGjyzk8ROQhhYbDQFDdPYK+jnu5Azhu28S9X3Wih9zNdXm53alkDMgTveIkzA MiSfqNKX2SImzgWGiJwcviJfg1YA+h7zIrJUv4X/LDNHSORovLGkp1xusjlyY/nU ACJJ0x3AtVEr65+2/W6FQXjRrNBLAMLdVlVrFi9CpeTv6USo4M3AYeHURgq59BGl rYbA4s9Z91QgGEUCb3U1 =0GpS -----END PGP SIGNATURE----- --Apple-Mail=_7098E7B9-D15F-4923-A4A2-C23EE267BE53-- From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 04:29:51 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC3888A8 for ; Mon, 23 Jun 2014 04:29:51 +0000 (UTC) Received: from mail-yk0-x234.google.com (mail-yk0-x234.google.com [IPv6:2607:f8b0:4002:c07::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7436D29FD for ; Mon, 23 Jun 2014 04:29:51 +0000 (UTC) Received: by mail-yk0-f180.google.com with SMTP id 131so4424691ykp.11 for ; Sun, 22 Jun 2014 21:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=5xkV4UZJTZqu4nx0kaZ0lA75G+owuGXa8rHzlmsL4xE=; b=CZ67+hQ3OJA8ZTQ2qdHf2/xCp7/DsPiZafmfFbj4SiQDkDKU/5wzeHv42hLORvOK3Q BpTHiGtWQm+AL3v0qEK40IVv1J9OQREydDL2BtXQHyHSReVJBc/9aC1M3S/zGNlCgFS+ DrPN40q/8IsvpmHGCw/c+o8Q5Tm16FFJMr5cyMD1gJuZllOsbgeZQDqY97ZvEoBKTBuz pSNGKvB37mXz3hmfZq9Lgeubg5UogseY0YW2CafPP8JcfAbfKzisFgZyi2lbkiO8QzMC 0Z6IFZdIr9Nuy/BCe8xHk/8NSMgm0seDQ1HRfyh2JVj5HalcEPZDsX63QPWjh0SBtfZC k2XQ== MIME-Version: 1.0 X-Received: by 10.236.103.135 with SMTP id f7mr32367872yhg.102.1403497790719; Sun, 22 Jun 2014 21:29:50 -0700 (PDT) Received: by 10.170.70.6 with HTTP; Sun, 22 Jun 2014 21:29:50 -0700 (PDT) Received: by 10.170.70.6 with HTTP; Sun, 22 Jun 2014 21:29:50 -0700 (PDT) Date: Mon, 23 Jun 2014 00:29:50 -0400 Message-ID: Subject: From: =?UTF-8?Q?Mathieu_S=C3=A9guin?= To: freebsd-arch@FreeBSD.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 04:29:51 -0000 4.4 From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 05:08:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DF77D25 for ; Mon, 23 Jun 2014 05:08:21 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00B772CAC for ; Mon, 23 Jun 2014 05:08:20 +0000 (UTC) Received: from [192.168.0.100] ([87.139.233.65]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0LdZAI-1WGDnd0JxC-00ijqD; Mon, 23 Jun 2014 07:08:11 +0200 Message-ID: <53A7B645.3000801@gmx.de> Date: Mon, 23 Jun 2014 07:08:21 +0200 From: olli hauer User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: =?UTF-8?B?VG9tZWsgV2HFgmFzemVr?= Subject: Re: Improve cron(8) References: In-Reply-To: X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:Gxd3hIkR5lQii2D2OuvOBZvxEPfatOb4U3ga0UC5hz0/YD1pTD2 /bCup11AZF1WtaKVsPJF4yhrzPdEZKPKAOeqNL7fJ7F/l0+Xyt1d2TLb8wnVDWeecifvc5+ AGnbSOsKZ+w1SMOwtrHGDJkeVwrbVBAcYYTt5apKZcOPxHhSnCoTtu+NYySESzGDkR50Iz5 At+WOy34j6vLJi0dDq1RA== Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 05:08:21 -0000 On 2014-06-13 09:04, Eitan Adler wrote: > +arch since hackers@ seems to be silent. > > On 11 June 2014 23:56, Tomek WaÅ‚aszek wrote: >> Hello, >> I saw on the FreeBSD Ideas page topic about cron :). >> I've started updating the 'original' FreeBSD cron from sources to vixi cron >> 4.1. I think (well I hope :P) most of the features that were done in >> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately >> some missing features at the moment: >> - @every_second - this need to be done >> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >> default, at the moment there is no -s and -o options. So you need to remove >> '-s' from the cron rc script >> >> I've also added one feature from OpenBSD, crontab is poking cron using >> unix-domain socket so we don't need to have suid on crontab. >> >> Path is in the attachment. I'm testing it on my FreeBSD box and it looks >> good but anyway don't try it on production machines :). >> >> After the installation we have to do a few things: >> - Add crontab group >> - Change group to crontab on /var/cron/tabs >> - Add sticky bit on /var/cron/tabs >> - Add group write permissions on /var/cron/tabs >> >> This is still work in progress but if someone could have a look on this and >> give me some feedback it would be great. >> >> Regards, >> Tomasz Walaszek >> Hi Thomaz, having only a quick look into the diff but see there are also changes from snprintf to sprintf :( see the following snippet ). Anyway it seems no one is looking into PR's against cron so I like the idea to start with updating and looking into the PR's for cron. - if (mailto) { - register char **env; - auto char mailcmd[MAX_COMMAND]; - auto char hostname[MAXHOSTNAMELEN]; - - (void) gethostname(hostname, MAXHOSTNAMELEN); - (void) snprintf(mailcmd, sizeof(mailcmd), - MAILARGS, MAILCMD); + if (mailto && safe_p(usernm, mailto)) { + char **env; + char mailcmd[MAX_COMMAND]; + char hostname[MAXHOSTNAMELEN]; + + gethostname(hostname, MAXHOSTNAMELEN); + if (strlens(MAILFMT, MAILARG, NULL) + 1 + >= sizeof mailcmd) { + fprintf(stderr, "mailcmd too long\n"); + (void) _exit(ERROR_EXIT); + } + (void)sprintf(mailcmd, MAILFMT, MAILARG); -- olli From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 06:08:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8EFC975; Mon, 23 Jun 2014 06:08:48 +0000 (UTC) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 7047B20E9; Mon, 23 Jun 2014 06:08:48 +0000 (UTC) Received: from critter.freebsd.dk (unknown [192.168.60.3]) by phk.freebsd.dk (Postfix) with ESMTP id 06CB11598; Mon, 23 Jun 2014 06:08:46 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.9/8.14.9) with ESMTP id s5N68kme025393; Mon, 23 Jun 2014 06:08:46 GMT (envelope-from phk@phk.freebsd.dk) To: Julian Elischer Subject: Re: PoC: passive serialization In-reply-to: <53A7A2D6.30905@freebsd.org> From: "Poul-Henning Kamp" References: <539FEBC1.5030501@FreeBSD.org> <20140621231853.394A914A2D0@mail.netbsd.org> <53A7A2D6.30905@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 23 Jun 2014 06:08:46 +0000 Message-ID: <25392.1403503726@critter.freebsd.dk> Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 06:08:48 -0000 In message <53A7A2D6.30905@freebsd.org>, Julian Elischer writes: >On 6/22/14, 7:18 AM, Mindaugas Rasiukevicius wrote: >> >> Just a note on passive serialization in NetBSD: there is a lot of space for >> optimisations, simplifications or improvements to that code, but it was a >> deliberate choice to avoid them. The goal was to carefully implement the >> logic described in the expired patent (or at least attempt to be as close as >> our interpretation skills allow us to be). Any deviation from that logic >> increases the risk of falling under some other technique, primarily RCU, >> covered by other patent. >> >hopefully the recent ruling on software patents may make the whole >thing moot given enough impetus. Probably not in this case. This bite from page 3 seems like the gate through which RCU and similar patents will go: "They do not, for example, purport to improve the functioning of the computer itself" But read for yourself: http://www.supremecourt.gov/opinions/13pdf/13-298_7lh8.pdf -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 07:19:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AB46B76C for ; Mon, 23 Jun 2014 07:19:24 +0000 (UTC) Received: from mail-oa0-x234.google.com (mail-oa0-x234.google.com [IPv6:2607:f8b0:4003:c02::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 74102260A for ; Mon, 23 Jun 2014 07:19:24 +0000 (UTC) Received: by mail-oa0-f52.google.com with SMTP id j17so9579896oag.25 for ; Mon, 23 Jun 2014 00:19:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GJcw8DFSms2dsEmtg67fMqYmZsf+LrHDCe2hc9gsGpU=; b=XzdbXMfjD8NdSFAzyh58PEWr81ERY/Uv6aNrNAKIPQjo8jngEHvveE0fB38AzMZ6Xo Sc0s5iDN4VVsdCYmRFFpgdLyHalRhbup5h1v+/2ZfMNCTVBYvwObv2HecgFUuWA+CUiy UKBHYauPSnNBGL8AFY1IPxJzZY8B2XRQrVUJ9CbUzv1GJtWaoknyp47GaVwJWyVKBrl9 JwoiyPdRcjehL6IXW5UYdyeMIdyiqCFDjWlTpcSeL4bEBuUnCS1zJpY8oFplmyqjESKW ruXxSSVDk2wsB+7V+DCubh3M6qDpkoXs8rerz/OEwgcAqy0TxaKphaBO9oHW5h3To8YS ukLw== MIME-Version: 1.0 X-Received: by 10.60.15.231 with SMTP id a7mr1355036oed.50.1403507963710; Mon, 23 Jun 2014 00:19:23 -0700 (PDT) Received: by 10.76.154.8 with HTTP; Mon, 23 Jun 2014 00:19:23 -0700 (PDT) In-Reply-To: <53A7B645.3000801@gmx.de> References: <53A7B645.3000801@gmx.de> Date: Mon, 23 Jun 2014 09:19:23 +0200 Message-ID: Subject: Re: Improve cron(8) From: =?UTF-8?Q?Tomek_Wa=C5=82aszek?= To: olli hauer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 07:19:24 -0000 2014-06-23 7:08 GMT+02:00 olli hauer : > On 2014-06-13 09:04, Eitan Adler wrote: > > +arch since hackers@ seems to be silent. > > > > On 11 June 2014 23:56, Tomek Wa=C5=82aszek wrote= : > >> Hello, > >> I saw on the FreeBSD Ideas page topic about cron :). > >> I've started updating the 'original' FreeBSD cron from sources to vixi > cron > >> 4.1. I think (well I hope :P) most of the features that were done in > >> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunatel= y > >> some missing features at the moment: > >> - @every_second - this need to be done > >> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by > >> default, at the moment there is no -s and -o options. So you need to > remove > >> '-s' from the cron rc script > >> > >> I've also added one feature from OpenBSD, crontab is poking cron using > >> unix-domain socket so we don't need to have suid on crontab. > >> > >> Path is in the attachment. I'm testing it on my FreeBSD box and it loo= ks > >> good but anyway don't try it on production machines :). > >> > >> After the installation we have to do a few things: > >> - Add crontab group > >> - Change group to crontab on /var/cron/tabs > >> - Add sticky bit on /var/cron/tabs > >> - Add group write permissions on /var/cron/tabs > >> > >> This is still work in progress but if someone could have a look on thi= s > and > >> give me some feedback it would be great. > >> > >> Regards, > >> Tomasz Walaszek > >> > > Hi Thomaz, > > having only a quick look into the diff but see there are also changes > from snprintf to sprintf :( see the following snippet ). > Anyway it seems no one is looking into PR's against cron so I like the > idea to start with updating and looking into the PR's for cron. > > - if (mailto) { > - register char **env; > - auto char mailcmd[MAX_COMMAND]; > - auto char hostname[MAXHOSTNAMELEN]; > - > - (void) gethostname(hostname, MAXHOSTNAMELEN); > - (void) snprintf(mailcmd, sizeof(mailcmd), > - MAILARGS, MAILCMD); > + if (mailto && safe_p(usernm, mailto)) { > + char **env; > + char mailcmd[MAX_COMMAND]; > + char hostname[MAXHOSTNAMELEN]; > + > + gethostname(hostname, MAXHOSTNAMELEN); > + if (strlens(MAILFMT, MAILARG, NULL) + 1 > + >=3D sizeof mailcmd) { > + fprintf(stderr, "mailcmd too long\n"); > + (void) _exit(ERROR_EXIT); > + } > + (void)sprintf(mailcmd, MAILFMT, MAILARG); > > -- > olli > Hi, Thanks for finding this sprintf, I will fix it. I'm quite sure :) that there are some parts that need to be fixed or rewritten. I will generate a patch against HEAD and submit a bug with the patch. Thanks guys for good words ! :) --=20 Best regards, Tomasz Wa=C5=82aszek From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 15:52:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B991499D for ; Mon, 23 Jun 2014 15:52:13 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9188C263E for ; Mon, 23 Jun 2014 15:52:13 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7F715B98E; Mon, 23 Jun 2014 11:52:12 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? Date: Mon, 23 Jun 2014 09:56:48 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <201406190919.04443.jhb@freebsd.org> <750618593.166408.1403191319583.JavaMail.zimbra@arkoon-netasq.com> In-Reply-To: <750618593.166408.1403191319583.JavaMail.zimbra@arkoon-netasq.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201406230956.48220.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 23 Jun 2014 11:52:12 -0400 (EDT) Cc: Emeric POUPON X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 15:52:13 -0000 On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: > Thanks for your answer! > > I was thinking about calling some parent device functions from the children devices in order to perform IO accesses. > But I imagine it would be "better" to expose a kind of bus interface from the main driver? > However, I'm not sure the extra work induced is worth it. What do you think? I think it's fine to have them call each other directly if they are going to all live in the same module. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Jun 23 17:39:09 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DC644CD for ; Mon, 23 Jun 2014 17:39:09 +0000 (UTC) Received: from mail-ie0-f176.google.com (mail-ie0-f176.google.com [209.85.223.176]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D57B720E8 for ; Mon, 23 Jun 2014 17:39:08 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id rd18so6029999iec.21 for ; Mon, 23 Jun 2014 10:39:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=hZav0meCyGZAJJkf5IHSOeuAM/iRawfZ4l4P0d+3xxE=; b=jxTfRmFUmXTcDFTSXN+Nm+EAeVc4WDjkI7X+yycgz7Gnp8fQVzBqy4R4yl8t9h8aUz zh5URvew5XoPf75CVNK1PylYbCAYRXN3FhJngpRRRlBV1LruFGpwGU+HUaDXAaXHoTvK StdX5DCmQ811Nkh80UmzzKHcbKACJ7FGPRmDKdIqUqNFOGgCMvRLTmDBQVexsL43hAdu lIRhTxbX0fx6mYuNFpfMCUC0o2piXspGeeJJSjAqBTAYBWwCFo9a5KU4muuwRf/w7E06 B2pYz4eNV91rdFICb3khPWO+Fa2fANBEQKfj2iXNff8WkhdBBtPXeUlPt0RAqS8AlKk3 hM6w== X-Gm-Message-State: ALoCoQmHN/mW1lEYlLGYJax3k6BVRM3badcRkL4Xy8QvGkUk1ZOCC6EFqY8WX9TBQEh/dZGAtelI X-Received: by 10.50.28.37 with SMTP id y5mr281430igg.5.1403545142303; Mon, 23 Jun 2014 10:39:02 -0700 (PDT) Received: from netflix-mac.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id n17sm24716905igk.19.2014.06.23.10.39.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 23 Jun 2014 10:39:01 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_0DF483F6-486F-4E8F-B911-A4D2911C4AAF"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? From: Warner Losh In-Reply-To: <201406230956.48220.jhb@freebsd.org> Date: Mon, 23 Jun 2014 11:38:59 -0600 Message-Id: <0C33F78B-60F2-41A5-92AD-C002EACE312E@bsdimp.com> References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <201406190919.04443.jhb@freebsd.org> <750618593.166408.1403191319583.JavaMail.zimbra@arkoon-netasq.com> <201406230956.48220.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1878.2) Cc: Emeric POUPON , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2014 17:39:09 -0000 --Apple-Mail=_0DF483F6-486F-4E8F-B911-A4D2911C4AAF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jun 23, 2014, at 7:56 AM, John Baldwin wrote: > On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: >> Thanks for your answer! >>=20 >> I was thinking about calling some parent device functions from the = children=20 > devices in order to perform IO accesses. >> But I imagine it would be "better" to expose a kind of bus interface = from=20 > the main driver? >> However, I'm not sure the extra work induced is worth it. What do you = think? >=20 > I think it's fine to have them call each other directly if they are = going to=20 > all live in the same module. =46rom a pure design point of view, having helpers might be slightly = better. However, given the specialized nature of SuperIO chips, you=92ll likely = get very little reuse of out the more complicated design, so John=92s suggestion = will be a big win. The only time you=92ll likely need to get the parent involved is for = =93shared access=94 situations (some SuperIO chips have banked registers, and you have to = have some kind of locking in place to access them, even if the locking is = done with a grab/ungrab bus paradigm). Warner --Apple-Mail=_0DF483F6-486F-4E8F-B911-A4D2911C4AAF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTqGY0AAoJEGwc0Sh9sBEApr8P/33x5pj34CPTKRIxyYxY2S4m Kc+8UpLfdtNJGfMrfkApjiS+QTDYpeRCuMs1y0yM8fFgleqEONdbxmIi2CQBHIRJ Lo22mNAQQc4HwwbgPZ+lI/uiktMhdIUyaSOqcYtZCEN+Ipy+ukiOevvZwhlRovv7 Zi6QbuafJ4Tp1sjH3WlBfjlD8ZF6mdw9hJ5GrGSr2/WQgH/9Blreb2F4xHhANibj +7Vldp9rx2ZHxRz6aSfg2JITzAs5KT9v5AoW/MnUHt9xYvSmVcQY35bIttVJ4ujm GYyilrToEAa7F5xRDYPrhg8AiyIWkL2UU4lCkVvMBQjJ45hXxXBZvRPwNasSXYMN 7a4fzBO+jaBRZkVN8RxZt6XTCTP7a3Qws2AdswL0vKi8k5r0ygQ5nR+c9/HDJzS2 jNyjOs0mUqIFjXCIY06D+NBbL72IYw5l+n/9xMA17qsqcg3VOqfu+fxlKzRaAhUV mwCagAXL0P1c5dj7pXyBVRIL9Z7bhZAfLv1R1KAREde7ioFKmDNeJIaIYCcrIHWX Jh6pSw1ukBGAHXzI9D3XhIQ40ivCDwYh5ZvSHye0XRKTvhusNOj2Kxda4R4t/Lym osJjZq5XjWWazdEb0P0b68N7QU2LsYAgg2erJNUjSJAhM5at9CAECw+9tMT2DA8c utLxlgFWR+V7LGHsUSJv =Wo6a -----END PGP SIGNATURE----- --Apple-Mail=_0DF483F6-486F-4E8F-B911-A4D2911C4AAF-- From owner-freebsd-arch@FreeBSD.ORG Tue Jun 24 02:09:44 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3643D83; Tue, 24 Jun 2014 02:09:44 +0000 (UTC) Received: from mail.netbsd.org (mail.NetBSD.org [IPv6:2001:4f8:3:7::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 758DF2EF7; Tue, 24 Jun 2014 02:09:44 +0000 (UTC) Received: from ws (localhost [IPv6:::1]) by mail.netbsd.org (Postfix) with SMTP id 9A2BF14A247; Tue, 24 Jun 2014 02:09:41 +0000 (UTC) Date: Tue, 24 Jun 2014 03:09:41 +0100 From: Mindaugas Rasiukevicius To: Mateusz Guzik Subject: Re: PoC: passive serialization In-Reply-To: <20140622123457.GA20525@dft-labs.eu> References: <539FEBC1.5030501@FreeBSD.org> <20140621231853.394A914A2D0@mail.netbsd.org> <20140622123457.GA20525@dft-labs.eu> X-Mailer: mail(1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <20140624020942.9A2BF14A247@mail.netbsd.org> Cc: Andrey Zonov , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2014 02:09:44 -0000 Mateusz Guzik wrote: > Cursory look suggests passive serialization got very limited use in > NetBSD, I was also unable to find any benchmarks. We have been avoiding RCU-like synchronisation in favour of alternative techniques. Passive serialisation was added relatively recently. Also, in many cases we can get away with a simpler way to ensure the quiescent state (c.f. xc_broadcast(9) calls will nullop in the NetBSD tree). As for benchmarks: well, benchmarks of what? The writer side? It is very expensive and slow. However, this is exactly what passive serialisation promises: extremely low cost on the reader side and very expensive writer side. The writer side can be improved by garbage collecting the items in a batch (and NetBSD interface allows the caller to do that). However, if the reader-writer ratio is not significantly dominated by the readers and the potential pressure on memory is not desirable, then you hit the limits of passive serialisation. At this point, if you are trying to address those limitations - you are basically (re)implementing RCU. At least, in a sense of heading towards the area covered by its patent portfolio. And that may be a bit tricky. > I do understand and appreciate your cautious approach, but it also > limits the potential. > > Do you have any materials which would show gains of using the concept as > implemented in your code? Right now it is unclear if the implementation > limited to this patent can yield benefits which would justify added > complexity. I think you miss the point. A lot of that machinery (three queues, extra bitmasks, etc) is either redundant or could be done a simpler/better way. It is there to implement the claims of the lapsed patent. There are no technical gains (quite the opposite). The gain is the reduced risk. It is up to you which approach do you want to take. I am just suggesting to put some thought on non-technical aspects of this problem as well. -- Mindaugas From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 10:31:15 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0223CA6E; Wed, 25 Jun 2014 10:31:15 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 474262649; Wed, 25 Jun 2014 10:31:14 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id hi2so7561196wib.11 for ; Wed, 25 Jun 2014 03:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=UpSq2UntmVbwBFYG74K85m43JdQtoq1j2s25GOrxfXM=; b=sS5fCptOKbvNJMVyWCZ3vAcagusc5IRhFz5B/d5AWRWrHzyfPHufFwOI/x+G/2Gw6w CZzNnxWjAnx5ONoq9CmjR0/bYbbKOFF/mogOioSmj7VuR8tTFgxw+VSmR0EzKBEjItyt Sigv7PkeqAHt42LAAK+fql3D+E8ZeTsbLFNhaQAGdZiGBtG0DeNWTNQK5dU111LbaXNm mXWX9Yzk6Nv/Zp7UTtrn68wXlY/455B5O7scPHidOy8qWUa4IAAayyMaqzvo+jkN8VG1 NDhAfAMNd8X0B8TSMRG8sO0m3e7N+1pjam8SAsDw7Nm8+sNvUWI2RUlXx0FjAQPdNenH JFNA== X-Received: by 10.180.21.200 with SMTP id x8mr5894281wie.67.1403692271909; Wed, 25 Jun 2014 03:31:11 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id m1sm10817620wib.20.2014.06.25.03.31.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 03:31:11 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 25 Jun 2014 12:31:08 +0200 From: Baptiste Daroussin To: hackers@FreeBSD.org, arch@FreeBSD.org Subject: [CFR] Remove texinfo from base Message-ID: <20140625103107.GB23976@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="P2/AsD9qFFw+Iv3c" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 10:31:15 -0000 --P2/AsD9qFFw+Iv3c Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi all, Here is a patch (6.5MB) http://people.freebsd.org/~bapt/remove-texinfo.diff That removes completly texinfo from base as well as bsd.info.mk The ports tree has received the necessary bits so it is now able to handle info pages without anything related to texinfo in base regards, Bapt --P2/AsD9qFFw+Iv3c Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlOqpOsACgkQ8kTtMUmk6EylcgCaAjE7kZp54dr245X9dN4rTRpJ 0vwAn2sH/dbRAq+HxTQOk4CqaNVJvgpV =Xo5Z -----END PGP SIGNATURE----- --P2/AsD9qFFw+Iv3c-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 10:45:48 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CBEBAEF3; Wed, 25 Jun 2014 10:45:48 +0000 (UTC) Received: from thyme.infocus-llc.com (server.infocus-llc.com [206.156.254.44]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "*.infocus-llc.com", Issuer "*.infocus-llc.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A31272791; Wed, 25 Jun 2014 10:45:48 +0000 (UTC) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id 85B1E37B537; Wed, 25 Jun 2014 05:45:41 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3gz1Gs07Bwz2jT; Wed, 25 Jun 2014 05:45:41 -0500 (CDT) Date: Wed, 25 Jun 2014 05:45:40 -0500 From: "Matthew D. Fuller" To: Baptiste Daroussin Subject: Re: [CFR] Remove texinfo from base Message-ID: <20140625104540.GE86779@over-yonder.net> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140625103107.GB23976@ivaldir.etoilebsd.net> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.23-fullermd.4 (2014-03-12) X-Virus-Scanned: clamav-milter 0.98.3 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: arch@FreeBSD.org, hackers@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 10:45:48 -0000 On Wed, Jun 25, 2014 at 12:31:08PM +0200 I heard the voice of Baptiste Daroussin, and lo! it spake thus: > > The ports tree has received the necessary bits so it is now able to > handle info pages without anything related to texinfo in base How many ports does that break? As somebody with /usr/local/bin/ before /usr/bin/ in his $PATH, I've run into a steady trickle of ports over the years that break when various ports shadow base utils, and texinfo is the prime offender. e.g., recently (and still brokenly) lang/guile as says. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 10:52:17 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7593A2B1; Wed, 25 Jun 2014 10:52:17 +0000 (UTC) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7EF82854; Wed, 25 Jun 2014 10:52:16 +0000 (UTC) Received: by mail-we0-f173.google.com with SMTP id t60so1818855wes.18 for ; Wed, 25 Jun 2014 03:52:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=shPQL8TfCwphfBH1LvBlOYuYdoxcvF56wtx6q8gOaE4=; b=VLVuoPi9OClS5Dxl3dZOqDNGtXfV1hHRY53xrV6x0g2GvMQ3c/rY3Veg54mS+6KsUn bf9XwbROAmcCn6rK4nXMLG0RlxAHXm1ZnBGULBPNq4zpaMFYCL9k6/HMuaHKzAF8qxFL bROXh4N6OB1YGlvz/welSjBVml1Nd6ZY9pQZv/HC4ZCGItpH47NARgSHwsVxJAX68zG8 tdwrK3DMKYoe7RFM/OKsJUANbHTRy6iA+umbpUqIjT9mfJcnSakSuRzqvg8lBKm3KXPP mk778pO8xEfI6LyVNv4qMCUhS8gPlZ2J115BY/iFgFS5gzycw+qIBC2uv9vS3NuBjZEE kP/Q== X-Received: by 10.180.108.208 with SMTP id hm16mr5812254wib.80.1403693533524; Wed, 25 Jun 2014 03:52:13 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id ey16sm52419206wid.14.2014.06.25.03.52.12 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 03:52:12 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 25 Jun 2014 12:52:10 +0200 From: Baptiste Daroussin To: "Matthew D. Fuller" Subject: Re: [CFR] Remove texinfo from base Message-ID: <20140625105209.GC23976@ivaldir.etoilebsd.net> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/If7Wv41KAW8wPhq" Content-Disposition: inline In-Reply-To: <20140625104540.GE86779@over-yonder.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: arch@FreeBSD.org, hackers@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 10:52:17 -0000 --/If7Wv41KAW8wPhq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 25, 2014 at 05:45:40AM -0500, Matthew D. Fuller wrote: > On Wed, Jun 25, 2014 at 12:31:08PM +0200 I heard the voice of > Baptiste Daroussin, and lo! it spake thus: > >=20 > > The ports tree has received the necessary bits so it is now able to > > handle info pages without anything related to texinfo in base >=20 > How many ports does that break? >=20 > As somebody with /usr/local/bin/ before /usr/bin/ in his $PATH, I've > run into a steady trickle of ports over the years that break when > various ports shadow base utils, and texinfo is the prime offender. > e.g., recently (and still brokenly) lang/guile as > says. I have just committed the support for this in ports, anyway breakage should= be reported, right now it seems fine on my exp-run regards, Bapt --/If7Wv41KAW8wPhq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlOqqdkACgkQ8kTtMUmk6EzExgCfdtdBM+SlTZ0utGoygQiLrxOx XOkAn3J2WzEByjBhmCFxdZl6ApCO+sVD =OrkU -----END PGP SIGNATURE----- --/If7Wv41KAW8wPhq-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 11:00:51 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 489247A2; Wed, 25 Jun 2014 11:00:51 +0000 (UTC) Received: from thyme.infocus-llc.com (server.infocus-llc.com [206.156.254.44]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "*.infocus-llc.com", Issuer "*.infocus-llc.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1EA112923; Wed, 25 Jun 2014 11:00:50 +0000 (UTC) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id DB00137B537; Wed, 25 Jun 2014 06:00:49 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3gz1cK38RRz2jy; Wed, 25 Jun 2014 06:00:49 -0500 (CDT) Date: Wed, 25 Jun 2014 06:00:49 -0500 From: "Matthew D. Fuller" To: Baptiste Daroussin Subject: Re: [CFR] Remove texinfo from base Message-ID: <20140625110049.GF86779@over-yonder.net> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> <20140625105209.GC23976@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20140625105209.GC23976@ivaldir.etoilebsd.net> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.23-fullermd.4 (2014-03-12) X-Virus-Scanned: clamav-milter 0.98.3 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: arch@FreeBSD.org, hackers@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 11:00:51 -0000 On Wed, Jun 25, 2014 at 12:52:10PM +0200 I heard the voice of Baptiste Daroussin, and lo! it spake thus: > > I have just committed the support for this in ports, anyway breakage > should be reported, right now it seems fine on my exp-run Oh, yes. Sorry, I did phrase that poorly. This shouldn't _break_ anything, but I suspect it will uncover existing-but-hidden breakage. Which is good. But does merit awareness that "hey, this will probably happen somewhere, so know this is a place to look when a build breaks". -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 15:20:51 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 269F27E8 for ; Wed, 25 Jun 2014 15:20:51 +0000 (UTC) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E77DF2335 for ; Wed, 25 Jun 2014 15:20:50 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id rd3so1858545pab.31 for ; Wed, 25 Jun 2014 08:20:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=M7hP+SGDgiPFqW8M5hirZvcgxceUx4drVrGMd6BL9hQ=; b=X/48K5yBOtItU4wlUKsl9yq/sm+6jqLJbQc/q4uziQMqQDSTNqYfNEo3tX9CX7H3V4 Q/frxwynx7zi5aJuAtGEf5Cvz2lRrB1dcToCS7SQHU5pRZXQkG9EpVTseMTGpwoYSUqn gQs/COgynhaYlrnVCQVt7jIOsZCSw2kPcMtlcBOTKQ352E45svfUot08tMXfkEYcI8zd gZmGT9ubFt1AdThgbcbazX4qsTTipF8FK5jU4Y5623NggR+dF/dcEbg7Z8LZR0RCmkmX JTbNfaqR6Qu1hFPkrDlXapqEKwmS6cne6N5HBeJn2wlAJ66pUWutMbO7tDPNc8VXs/9W kayQ== X-Gm-Message-State: ALoCoQntqrM6TA8qhOoxYPbwb7RWnWPx269PQX36rTStPLSbHG7numMxbn23nCIKU1w6Zu+9sIna X-Received: by 10.66.136.103 with SMTP id pz7mr12762161pab.140.1403709644172; Wed, 25 Jun 2014 08:20:44 -0700 (PDT) Received: from [192.168.9.241] (173-11-114-225-SFBA.hfc.comcastbusiness.net. [173.11.114.225]) by mx.google.com with ESMTPSA id ao4sm5744291pbc.51.2014.06.25.08.20.42 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 25 Jun 2014 08:20:43 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_5687031B-EF46-46D7-96D4-51D426A63E4F"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: [CFR] Remove texinfo from base From: Warner Losh In-Reply-To: <20140625110049.GF86779@over-yonder.net> Date: Wed, 25 Jun 2014 08:20:41 -0700 Message-Id: <40EA1066-4776-4B2E-988A-9900BB842862@bsdimp.com> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> <20140625105209.GC23976@ivaldir.etoilebsd.net> <20140625110049.GF86779@over-yonder.net> To: "Matthew D. Fuller" X-Mailer: Apple Mail (2.1878.2) Cc: arch@FreeBSD.org, Baptiste Daroussin , hackers@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 15:20:51 -0000 --Apple-Mail=_5687031B-EF46-46D7-96D4-51D426A63E4F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jun 25, 2014, at 4:00 AM, Matthew D. Fuller = wrote: > On Wed, Jun 25, 2014 at 12:52:10PM +0200 I heard the voice of > Baptiste Daroussin, and lo! it spake thus: >>=20 >> I have just committed the support for this in ports, anyway breakage >> should be reported, right now it seems fine on my exp-run >=20 > Oh, yes. Sorry, I did phrase that poorly. This shouldn't _break_ > anything, but I suspect it will uncover existing-but-hidden breakage. >=20 > Which is good. But does merit awareness that "hey, this will probably > happen somewhere, so know this is a place to look when a build > breaks". I know it will break certain nanobsd configurations that build ports = because dependencies there (at least for the ones I=92ve done) aren=92t well = handled. So I agree that this patch is missing, at the very least, an UPDATING entry = and a __FreeBSD_version bump. Warner --Apple-Mail=_5687031B-EF46-46D7-96D4-51D426A63E4F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTqujJAAoJEGwc0Sh9sBEA/d8QANmi7arakpMdBhg9r+8/8HpC V/BU93XnT4QTEBzF9YI8C6AW1HWrz/nfEGtt4gJc0exNASdQcBxDlycYA+DA4ibT jvaY3d5lkMPXFacUfLPaj8xLorlH5yyThAGNjI+LQAudMNw4eSDXFkUkB3+JL39W nXjLBdwjsCCdlA5gSBNjTQK00dOp5pf2s4Kzopzu75+m8fegq5djCrZ184dom8bV IZhSmiyMgsW6aeF3ym9kwRr4i/o4zCefhobzy1J7KCdeSZUy3LUHKOPlE+j9xFyJ /UUkzu8q/MwLk2VXwG3Xi6aDfG9bw9vJokE7AqwHap0+oZqtLce7XSTC1VUoljIQ VSfQzO+zVCZOc0Y8gqSJGr/vY3QQ9kak+6HqHuy4YE6sHENOw0fEwmZPXtsXMMYP K/SdJwwbftvZldWlBDj7Hp8K82E3LV5g30KOzKGxdz9C17/ftbFpFoYzZ2Q4yni5 pEcPh0XoUVlxUFunnuOIhB1bMDsYvRF/Zk9xMBTMzuO3+5XSz9XYqyw5K7ehjBrZ hVx/vkjIHUEe8Wrb57UH32LxH3/NkYIF/Ji9dsYSsu6wN+qtWAi4jhiF1YIi3AOy 8wNdytLXwngCPEi666gPOSto6P2aia1hwJtZZ+ML0Me1XGV8KVYi8KatbbrnMv0N 89UgvGT9MSH1b5O2SQTo =Si+q -----END PGP SIGNATURE----- --Apple-Mail=_5687031B-EF46-46D7-96D4-51D426A63E4F-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 15:48:07 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEA998DD; Wed, 25 Jun 2014 15:48:07 +0000 (UTC) Received: from mail-we0-x22a.google.com (mail-we0-x22a.google.com [IPv6:2a00:1450:400c:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 057C0268A; Wed, 25 Jun 2014 15:48:06 +0000 (UTC) Received: by mail-we0-f170.google.com with SMTP id w61so2235433wes.29 for ; Wed, 25 Jun 2014 08:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=WiFmbFJAs3w6DYqu5qzKD3oPHQNq46PIoE5tDT2m3DU=; b=hgGX77vNQA3nxHVkUvv9XmC5ouKJx0usIq/UrDMG5bimDWc84nCjk0XuEmtLZa3gGJ 1pwMDeEDW2Q2xTEiLJHTEG/fyfmuLOI9+Kd1SzmqXvbiVKE2pCGLd8NNnJnRza+SHQ+s RxmQKUWR2rnFTm/vDU6Z/QBT4V1UpKdnUO8cJCuFhiWzTwJRRLFyHNcpggj7VNBrNo3N lzXU3kh0MEixrIqip2dvR+HMhhBMgCOZYC2lwjMjoLvAOp4FZxDiJ9+42RgXH/d4cguE fLCx7GZN1DF757m7ll6qHy7n2CqP8RrhH3uQPHsOptiPI2rgz1RDQGzD/AzOeWvL2xJr dcPw== X-Received: by 10.180.89.233 with SMTP id br9mr11409645wib.14.1403711282215; Wed, 25 Jun 2014 08:48:02 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id hc4sm8206776wjc.38.2014.06.25.08.48.00 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 08:48:01 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 25 Jun 2014 17:47:58 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: [CFR] Remove texinfo from base Message-ID: <20140625154758.GD23976@ivaldir.etoilebsd.net> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> <20140625105209.GC23976@ivaldir.etoilebsd.net> <20140625110049.GF86779@over-yonder.net> <40EA1066-4776-4B2E-988A-9900BB842862@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gRZ38brEgCoUohoa" Content-Disposition: inline In-Reply-To: <40EA1066-4776-4B2E-988A-9900BB842862@bsdimp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: arch@FreeBSD.org, hackers@FreeBSD.org, "Matthew D. Fuller" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 15:48:07 -0000 --gRZ38brEgCoUohoa Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 25, 2014 at 08:20:41AM -0700, Warner Losh wrote: >=20 > On Jun 25, 2014, at 4:00 AM, Matthew D. Fuller = wrote: >=20 > > On Wed, Jun 25, 2014 at 12:52:10PM +0200 I heard the voice of > > Baptiste Daroussin, and lo! it spake thus: > >>=20 > >> I have just committed the support for this in ports, anyway breakage > >> should be reported, right now it seems fine on my exp-run > >=20 > > Oh, yes. Sorry, I did phrase that poorly. This shouldn't _break_ > > anything, but I suspect it will uncover existing-but-hidden breakage. > >=20 > > Which is good. But does merit awareness that "hey, this will probably > > happen somewhere, so know this is a place to look when a build > > breaks". >=20 > I know it will break certain nanobsd configurations that build ports beca= use > dependencies there (at least for the ones I=E2=80=99ve done) aren=E2=80= =99t well handled. So > I agree that this patch is missing, at the very least, an UPDATING entry = and > a __FreeBSD_version bump. If you build a port that needs texinfo the port framework will do what it n= eeds regards, Bapt --gRZ38brEgCoUohoa Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlOq7y4ACgkQ8kTtMUmk6EwnOACdEkWlY0xAybY/GW/Zk0lRmeQG DsQAoJjuVG3XCfqYdh1UKUYxwsFt1PNk =Lt+3 -----END PGP SIGNATURE----- --gRZ38brEgCoUohoa-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 16:09:41 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 02604198 for ; Wed, 25 Jun 2014 16:09:41 +0000 (UTC) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C234928A8 for ; Wed, 25 Jun 2014 16:09:40 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id un15so1878761pbc.41 for ; Wed, 25 Jun 2014 09:09:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=yDCMfHXhUvQKRbJXBamVUaNlNS12vi/GTBdFsifZw+E=; b=Vhj6E+/jxlHnbL32P+NkcLTx4j9Odn9FuLHMtSGDuDuI9RibizqKXf5yeCRVZHJ5Sr +k/gN++mxrtVPEoD4Q/25Bm4NmzYHHLkh9KMg7SofEh3a4nHe99d6e2eElP9nj8Jm7Wd jlh/7p70APHfDjTRc7929G5sJz/fe90dwOYfO0dkj1H/M8bQBh2T78VurGjw0ILDz5mr 7fU7H/4ScGrzdT4+aruGmuDb3U7GdnyqKK8Y1wFBezy20moC0sM0/qvAo0x9FQDQvu0C zbL0hRpf/KXDwkoD1YrsNBMAjZRjfPqAgIEvOE4rUhwAQrmI7LgWpXL+D/DKTh1E953w 8jMA== X-Gm-Message-State: ALoCoQk4AS8SpRqioXPmRsymMvYW9uHxm3QwIOVX8jbaCC72GM6kxgjmD29hyihgnKM2lhWNg/Hr X-Received: by 10.66.254.136 with SMTP id ai8mr12949843pad.37.1403712573988; Wed, 25 Jun 2014 09:09:33 -0700 (PDT) Received: from [192.168.9.241] (173-11-114-225-SFBA.hfc.comcastbusiness.net. [173.11.114.225]) by mx.google.com with ESMTPSA id vs10sm5919512pbc.38.2014.06.25.09.09.32 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 25 Jun 2014 09:09:33 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_788F477C-A05C-439A-9B5A-6DA1B3145878"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: [CFR] Remove texinfo from base From: Warner Losh In-Reply-To: <20140625154758.GD23976@ivaldir.etoilebsd.net> Date: Wed, 25 Jun 2014 09:09:30 -0700 Message-Id: References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> <20140625105209.GC23976@ivaldir.etoilebsd.net> <20140625110049.GF86779@over-yonder.net> <40EA1066-4776-4B2E-988A-9900BB842862@bsdimp.com> <20140625154758.GD23976@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1878.2) Cc: arch@FreeBSD.org, hackers@FreeBSD.org, "Matthew D. Fuller" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 16:09:41 -0000 --Apple-Mail=_788F477C-A05C-439A-9B5A-6DA1B3145878 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jun 25, 2014, at 8:47 AM, Baptiste Daroussin = wrote: > On Wed, Jun 25, 2014 at 08:20:41AM -0700, Warner Losh wrote: >>=20 >> On Jun 25, 2014, at 4:00 AM, Matthew D. Fuller = wrote: >>=20 >>> On Wed, Jun 25, 2014 at 12:52:10PM +0200 I heard the voice of >>> Baptiste Daroussin, and lo! it spake thus: >>>>=20 >>>> I have just committed the support for this in ports, anyway = breakage >>>> should be reported, right now it seems fine on my exp-run >>>=20 >>> Oh, yes. Sorry, I did phrase that poorly. This shouldn't _break_ >>> anything, but I suspect it will uncover existing-but-hidden = breakage. >>>=20 >>> Which is good. But does merit awareness that "hey, this will = probably >>> happen somewhere, so know this is a place to look when a build >>> breaks". >>=20 >> I know it will break certain nanobsd configurations that build ports = because >> dependencies there (at least for the ones I=92ve done) aren=92t well = handled. So >> I agree that this patch is missing, at the very least, an UPDATING = entry and >> a __FreeBSD_version bump. >=20 > If you build a port that needs texinfo the port framework will do what = it needs Except in environments that don=92t do dependencies quite right, or = where only a subset of ports tree has been imported and texinfo isn=92t part of that=85 But = people with them usually know, which is why UPDATING is needed. That=92s all. There=92s = nothing else for you to do. Warner --Apple-Mail=_788F477C-A05C-439A-9B5A-6DA1B3145878 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTqvQ6AAoJEGwc0Sh9sBEAX6EQAKIhsFfHmnnNxZgoWB+GFIuF IJwVq2TSL1+InLufCcQcxsWvHfqDEcR5oDTgdjMTdrf9S1PTiMtHYP8TYoPMivjN U1iWtJfDdMSEQkk5t449TU+m4q7YqV9FZh2rx2MSXQ2E8fRbX2HhUl1p+PZPcEwF coPV2caY04LYhgyoE08a/S7nsQ1jm1hO1KrWjzaftr7CPWuvxjmZEQv4sunyU9kB MhrE5m0rV31n40sAuayduzfRWEesbwwLLtM5FK167nuX0JAW5GSEKeKt16ar9eQR nHV/yHWPc8yVlm/cY5NGp20BW4TxAmRaKoi4Yx3JYg8jyKCHP+Ahery65j2I5Eht y1qQta/StmD2hXNsdVzQAD72K3829zuadN4fdyvxDhXW3ckglMoCiPfUtTt82Cvk xBaHb1pvKIbKfFx61XiVVwzWtYClY6VB7+jF6RjHTLQo3mmHfFgvdTPJMPi8Cmgg b2uB+9lEpDYG5O0YdwLlwG2m5FufU/BmTtDEcC4BMok+vL8q3mqU8YftHX7pLdQE l+tvXUdm5w63Kp0DnpSh11AsVtPcOSQEqXuoq5cxIwBwp9Fh3lPntgEzKze552l2 1EBGR2+2ieCbekqggcHUsa32k4U+iZ2f3Mf4oRFyQkNp18exAXPbBOE9S6Ce6nkd 1BhFyGNJBRbsf7S9D8sj =D/gg -----END PGP SIGNATURE----- --Apple-Mail=_788F477C-A05C-439A-9B5A-6DA1B3145878-- From owner-freebsd-arch@FreeBSD.ORG Wed Jun 25 16:25:54 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08D7B983; Wed, 25 Jun 2014 16:25:54 +0000 (UTC) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AAF42A67; Wed, 25 Jun 2014 16:25:53 +0000 (UTC) Received: by mail-we0-f173.google.com with SMTP id t60so2331197wes.18 for ; Wed, 25 Jun 2014 09:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=3xw6+y39K0WhHkhQGW4AzB0ZQeMV+eAeQ4akcZMEScM=; b=qGjd7JD9wlzWWojn/ZNcJK0S9IsV46o2JS7vlM9vjtg05Ui5QuMriZRbIvUpk5gYhr 5UFgfsqxse8nLEMRapoWFpDUVOS6oi6rys9naxW3F6l/viHeBrSuFHN6pDB7O8075jlV oTJuC5BmvgIWnSgB1YqkP5Lii7bSIMiTRSCjPxi35CeLTglabEy//Xn14mEQIhFzh9gM Z3KqCXtTT3ErSEmovR3RMeNVBs9QpXdE+218OEhxhlMzyiK40nyg9PFPEgw6gpQJEbKv ATXkDb9vNpUDhpnlzjFtZQxt5VtRjD8qQ9CTnyrgop83y3Zv3nz0AR6dFfqNhj2JADbR M5gw== X-Received: by 10.180.198.116 with SMTP id jb20mr11268788wic.59.1403713549519; Wed, 25 Jun 2014 09:25:49 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id n2sm6085525wjf.40.2014.06.25.09.25.48 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Jun 2014 09:25:48 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 25 Jun 2014 18:25:45 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: [CFR] Remove texinfo from base Message-ID: <20140625162545.GE23976@ivaldir.etoilebsd.net> References: <20140625103107.GB23976@ivaldir.etoilebsd.net> <20140625104540.GE86779@over-yonder.net> <20140625105209.GC23976@ivaldir.etoilebsd.net> <20140625110049.GF86779@over-yonder.net> <40EA1066-4776-4B2E-988A-9900BB842862@bsdimp.com> <20140625154758.GD23976@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Y46ssxGX9/CNNfN6" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: arch@FreeBSD.org, hackers@FreeBSD.org, "Matthew D. Fuller" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jun 2014 16:25:54 -0000 --Y46ssxGX9/CNNfN6 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 25, 2014 at 09:09:30AM -0700, Warner Losh wrote: >=20 > On Jun 25, 2014, at 8:47 AM, Baptiste Daroussin wrote: >=20 > > On Wed, Jun 25, 2014 at 08:20:41AM -0700, Warner Losh wrote: > >>=20 > >> On Jun 25, 2014, at 4:00 AM, Matthew D. Fuller wrote: > >>=20 > >>> On Wed, Jun 25, 2014 at 12:52:10PM +0200 I heard the voice of > >>> Baptiste Daroussin, and lo! it spake thus: > >>>>=20 > >>>> I have just committed the support for this in ports, anyway breakage > >>>> should be reported, right now it seems fine on my exp-run > >>>=20 > >>> Oh, yes. Sorry, I did phrase that poorly. This shouldn't _break_ > >>> anything, but I suspect it will uncover existing-but-hidden breakage. > >>>=20 > >>> Which is good. But does merit awareness that "hey, this will probably > >>> happen somewhere, so know this is a place to look when a build > >>> breaks". > >>=20 > >> I know it will break certain nanobsd configurations that build ports b= ecause > >> dependencies there (at least for the ones I=E2=80=99ve done) aren=E2= =80=99t well handled. So > >> I agree that this patch is missing, at the very least, an UPDATING ent= ry and > >> a __FreeBSD_version bump. > >=20 > > If you build a port that needs texinfo the port framework will do what = it needs >=20 > Except in environments that don=E2=80=99t do dependencies quite right, or= where only a subset > of ports tree has been imported and texinfo isn=E2=80=99t part of that=E2= =80=A6 But people with them > usually know, which is why UPDATING is needed. That=E2=80=99s all. There= =E2=80=99s nothing else for you > to do. Yes sure UPDATING was anyway intended I'm not sure about the __FreeBSD_vers= ion. regards, Bapt --Y46ssxGX9/CNNfN6 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlOq+AkACgkQ8kTtMUmk6Exx0gCfc9mwkKffv/zbBG+yPP8gy/Sd eG4An14wQFabTUU16j01sjo7u3eWNEV1 =tFgj -----END PGP SIGNATURE----- --Y46ssxGX9/CNNfN6-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 1 13:14:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2759E95; Tue, 1 Jul 2014 13:14:36 +0000 (UTC) Received: from mx11.netapp.com (mx11.netapp.com [216.240.18.76]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx11.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C84402FDA; Tue, 1 Jul 2014 13:14:36 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.01,581,1400050800"; d="asc'?scan'208";a="131820345" Received: from vmwexceht03-prd.hq.netapp.com ([10.106.76.241]) by mx11-out.netapp.com with ESMTP; 01 Jul 2014 06:14:27 -0700 Received: from HIOEXCMBX04-PRD.hq.netapp.com (10.122.105.37) by vmwexceht03-prd.hq.netapp.com (10.106.76.241) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 1 Jul 2014 06:14:27 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com (10.122.105.40) by hioexcmbx04-prd.hq.netapp.com (10.122.105.37) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 1 Jul 2014 06:14:26 -0700 Received: from HIOEXCMBX07-PRD.hq.netapp.com ([::1]) by hioexcmbx07-prd.hq.netapp.com ([fe80::8485:306a:80a6:ba8f%21]) with mapi id 15.00.0847.030; Tue, 1 Jul 2014 06:14:26 -0700 From: "Eggert, Lars" To: Adrian Chadd Subject: Re: [rfc] tcp timer update for RSS Thread-Topic: [rfc] tcp timer update for RSS Thread-Index: AQHPb/ho5JPKULu5Hk2S1uPuczXO0JuL8ikA Date: Tue, 1 Jul 2014 13:14:25 +0000 Message-ID: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-originating-ip: [10.122.56.79] Content-Type: multipart/signed; boundary="Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24"; protocol="application/pgp-signature"; micalg=pgp-sha1 MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 13:14:37 -0000 --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ sys/netinet/ip_output.c: case IP_RSSCPUID: kernel compilation with RSS currently fails, because IP_RSSCPUID is = still used in ip_output.c. Lars --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="signature.asc" Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- iQCVAwUBU7K0MNZcnpRveo1xAQIpfAP/YWb4ZhTqG9iXsgKBM86HupgtRrq69T3k uy84RJMlXNdpuDEzKTlSPxUplt0j+AoqDgEpafG+zAtS3UtBgoKqBa/WTq4tlpaf m7fQMC9V85gcUaa7hovKAtfz963IW/2eZbUnHZAH7PmRJ9p49/vSH24yn4FT4r9D Gt7gTydxWuc= =9sL0 -----END PGP SIGNATURE----- --Apple-Mail=_8203B49C-66AE-4058-9896-C7F7822CFF24-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 1 17:24:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80615117; Tue, 1 Jul 2014 17:24:44 +0000 (UTC) Received: from mail-qa0-x22f.google.com (mail-qa0-x22f.google.com [IPv6:2607:f8b0:400d:c00::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 309262988; Tue, 1 Jul 2014 17:24:44 +0000 (UTC) Received: by mail-qa0-f47.google.com with SMTP id hw13so8002386qab.34 for ; Tue, 01 Jul 2014 10:24:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=qplSQQFgTXSo2B44iwRAOTcCRAvLVaFLOuZacq1Qw5Q=; b=zB2aVxo9AZO5phXTzK53rkVosWfgQh6mrfbLcBPg22pezuleQUcO+ZRraGYWgJP4sR 7dg6XGkNjcNHju9X91YI10gxmD0TR3wXPIx1a1no9CEGlz3x1mKDukRBDB+QvkC4lHHz Rkz3evY+2T11vDo4DSZDQEI6Y3gwCdaJa2DwgzyCPFATE9lPFwY1H4hkzvs/kXH+/qdm ENpfP6AVrs6XAT4pRrUPnr0xJo3B4ZePbDaRTl44kBQ9CkHHfI1Zx4VQ5/t9tbEQ215A darDVFign55N/t82hL27MlDCXKSr0IVSCvaS4aWaoCKBP2hL3neGvulGqMOZdeF2I3iC bPNQ== MIME-Version: 1.0 X-Received: by 10.224.66.70 with SMTP id m6mr77949266qai.55.1404235483405; Tue, 01 Jul 2014 10:24:43 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 10:24:43 -0700 (PDT) In-Reply-To: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> Date: Tue, 1 Jul 2014 10:24:43 -0700 X-Google-Sender-Auth: U4HDKadrc0dhAYamzt81UpLEu08 Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 17:24:44 -0000 On 1 July 2014 06:14, Eggert, Lars wrote: > Hi, > > [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys > sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ > sys/netinet/ip_output.c: case IP_RSSCPUID: > > kernel compilation with RSS currently fails, because IP_RSSCPUID is still= used in ip_output.c. I thought I committed a fix to remove that! I'll go fix that soon. Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 1 17:28:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4292409; Tue, 1 Jul 2014 17:28:03 +0000 (UTC) Received: from mail-qg0-x231.google.com (mail-qg0-x231.google.com [IPv6:2607:f8b0:400d:c04::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7533729C3; Tue, 1 Jul 2014 17:28:03 +0000 (UTC) Received: by mail-qg0-f49.google.com with SMTP id f51so3593295qge.36 for ; Tue, 01 Jul 2014 10:28:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=qplSQQFgTXSo2B44iwRAOTcCRAvLVaFLOuZacq1Qw5Q=; b=Mb0czZr5rWmLi40iGX0w6f+Bz5o22Ix6mVVspx9ikZx3WqzsEDt06ugLNX7+6ANVkt IMMVsLEpH56CyHO/7mJnjObd8zsP/tSCfsKiJh7QHkoUFErOrwPOJxgkY74wHHGuF4IA 1lC74IUjt3rFjIq4iaAIqyF0yzBs7p0760JZBSd/KmPcNujxAxeYuCGxDbtO9eNSD8Zy AB5YhUcHk0xhCi35iYEf3vZGl8JgPLjOax66L0jWxj2pJFPRjZDC245Q2B8mz8y5zgb5 hvM/SJN3LVDIjCHPHteLHI/VXvxSADNxK2uxHRF4uxNnGz/24BnEfIREtJgkj5U4zfIq PWeg== MIME-Version: 1.0 X-Received: by 10.224.130.136 with SMTP id t8mr10887784qas.49.1404235676235; Tue, 01 Jul 2014 10:27:56 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 10:27:56 -0700 (PDT) In-Reply-To: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> References: <84BEBDAC-189F-431F-932D-C280649F5D65@netapp.com> Date: Tue, 1 Jul 2014 10:27:56 -0700 X-Google-Sender-Auth: 1YLq-_-5Urp2Vq4AEvYGmoQd-_A Message-ID: Subject: Re: [rfc] tcp timer update for RSS From: Adrian Chadd To: "Eggert, Lars" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 17:28:03 -0000 On 1 July 2014 06:14, Eggert, Lars wrote: > Hi, > > [elars@stanley:/home/elars/src] 1 =E2=8C=80 grep -r IP_RSSCPUID sys > sys/netinet/in.h:/* 71 - XXX was IP_RSSCPUID - can recycle whenever */ > sys/netinet/ip_output.c: case IP_RSSCPUID: > > kernel compilation with RSS currently fails, because IP_RSSCPUID is still= used in ip_output.c. I thought I committed a fix to remove that! I'll go fix that soon. Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 1 18:02:20 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B717A7B5; Tue, 1 Jul 2014 18:02:20 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C4752D3C; Tue, 1 Jul 2014 18:02:20 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id v10so7883558qac.41 for ; Tue, 01 Jul 2014 11:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=k29Cqf3KrMEaMdlBk4kIeGaY3fzC2J0rdhfzdWPmX5w=; b=QtawPZXlnexi02LUPnKYHm6PhWCS1Z2ChdfG3knFSLDgXvjTWV1J45B+pClaoExHCo hBY3Pg2nkKx1xbaPQhIucshb5lWZOsYyK9Ko4S+b7qOrA+EJrTRGuNz7Kjnn+TFPi2Oi ZamKzKTde3nAtYQPOsG/52OMmExs6gTTM1WNxvMRwcfQykKu/IhTIk5AqRdxkCgbjrkk PXecvf6Ycr0aVXVe+AOYy6i2PIc4YGOxl4VAT75oeR4O7VZojKxpxJeSZ3Y7hTdnkEGF DBFn1g03+AvP82sdpyWklK8TCmHGfPmHd0Fqahl/LhZuS+QpNWLL6F6ZpFvOcVF0rxNr JOXw== MIME-Version: 1.0 X-Received: by 10.140.34.195 with SMTP id l61mr71334451qgl.87.1404237739505; Tue, 01 Jul 2014 11:02:19 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 1 Jul 2014 11:02:19 -0700 (PDT) Date: Tue, 1 Jul 2014 11:02:19 -0700 X-Google-Sender-Auth: jit6IfaJ6F7YjVvAu1q_Vw0i7Nc Message-ID: Subject: [rfc] IP_BINDMULTI and IP_RSSBUCKETID support From: Adrian Chadd To: FreeBSD Net , "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jul 2014 18:02:20 -0000 Hi! This patch: http://people.freebsd.org/~adrian/rss/20140701-rss-1.diff Implements a few things: * It introduces IP_BINDMULTI, which allows TCP/UDP sockets to be bound to the same matching address:port. It's intended to be what Linux SO_REUSEADDR has become except I specifically wanted it to be a different option. * .. it only allows the PCBs to be created, it doesn't actually load balance between multiple sockets like this. That requires a bunch more work to happen in the general case. But! * IP_RSSBUCKETID maps an IPv4 TCP (and later UDP, and much later IPv6) socket into a specific RSS bucket rather than in the global PCB list. That way it only receives incoming connections that has to that RSS bucket ID. If userland queries the RSS setup (sysctl net.inet.rss) and creates a worker thread per RSS bucket _AND_ pins it to the correct CPU for the RSS bucket, all the transmit and receive side will stay on the same CPU. The lock contention is almost eliminated in that case. My plan: * get this into the tree and tested/debugged; * add IPv4 UDP extensions to RSS; * teach a couple of popular threaded UDP applications (eg memcached) about this; * bask in what hopefully will be a good set of benefits; * work on IP_BINDMULTI support so multiple threads/processes can listen in the same RSS bucket and get some load balancing; * add IPv6 support; * add RSS rebalancing support; * add multi-socket, multi-NIC RSS awareness. Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 00:45:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8F47D9; Fri, 4 Jul 2014 00:45:35 +0000 (UTC) Received: from mail-qa0-x22c.google.com (mail-qa0-x22c.google.com [IPv6:2607:f8b0:400d:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E8862767; Fri, 4 Jul 2014 00:45:35 +0000 (UTC) Received: by mail-qa0-f44.google.com with SMTP id hw13so819046qab.17 for ; Thu, 03 Jul 2014 17:45:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=0IVFqMKpoPAx1ZIQqD/bVB+sG5tnXnl/84kbGvm9uN4=; b=Go3PkBe+rbPKqPI8mDdt8PWAqB0qddb6oGaHeaJvNAL70yv/Jyom+ySWddvsDGBs0V uS5kC18Icky/dU9R9QRVW8Gaa/fNXLWOH6IiLJPau4380LfXofVy+JjroDL2yUSIpMLv 3zBUsbfFc4mDu6WX3Lk+oOqwQIo4+6YoDZ+OSPppoFQkhSu7/IhFCEW1bhLbvkoF9fMY EvXmSQHEjhPWHl3NxquhI7fAR84hQsMRuzxT+w7PP/AST7+HtMtmvcKxTUpJh6NOzYZO UNSe7LuAw205ct8mkoQsD36oL04NN6Gs4PTIJnSm3X0CJGhExl4ExjqWDKsSicPp/DF+ agaQ== MIME-Version: 1.0 X-Received: by 10.140.34.195 with SMTP id l61mr12384760qgl.87.1404434734621; Thu, 03 Jul 2014 17:45:34 -0700 (PDT) Received: by 10.224.202.193 with HTTP; Thu, 3 Jul 2014 17:45:34 -0700 (PDT) Date: Thu, 3 Jul 2014 17:45:34 -0700 Message-ID: Subject: EDEADLK from fcntl(F_SETFL) ? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 00:45:35 -0000 Hi! I've seen sqlite3 crap out due to "disk IO error". It looks like the F_SETFL path is returning EDEADLK when it shouldn't be - only the "wait" version of this should be. The kernel code looks to be: lf_setlock() -> lf_add_outgoing() -> lf_add_edge() -> graph_add_edge() -> EDEADLK .. and lf_setlock() will return an error from lf_add_outgoing() without checking if it's (a) EDEADLK, and (b) whether we're going to sleep or not. So, sqlite3 trips up on this. I'm sure other things do. What should the correct thing be? It looks like EWOULDBLOCK is the correct value to return for F_SETFL failing, not EDEADLK. What do those-who-know-POSIX-standards-better-than-I think? Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 02:15:53 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85653FDA; Fri, 4 Jul 2014 02:15:53 +0000 (UTC) Received: from mail-qg0-x230.google.com (mail-qg0-x230.google.com [IPv6:2607:f8b0:400d:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 38A6D2FD2; Fri, 4 Jul 2014 02:15:53 +0000 (UTC) Received: by mail-qg0-f48.google.com with SMTP id q108so978506qgd.35 for ; Thu, 03 Jul 2014 19:15:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=6Xf3dUxrEe1WEzLSgdxviXehCkjBjCGH+DjycS/q0Zw=; b=Pd0ZN9NMnuADpBTLMMH5XcBaywNps0tM75YXyDQBZUlHCwrqI9q5iN2LVaMfKFY7nf 9UEzurQPOLx+4ceWi4++ipTT1PoQ1YaJU4rMvRL5mCbTvRveXyZ7LIJpo/ONZy3iioSe md95TBnc4CErefHKjD5srRSL4Utwe3MzeTjxWykzi8+rviou5fXfzibxjXuM/kZpOKbV NEvjIk7oC4UpGN3dy9RSwXMYyle83NC/hhErblxWHlwg7DL4ild7xqZRFBoKo9ph3V7+ BG+Md6O9t86QBERGzTYTMfqDeTo3gM5E4Gph5azZ33cpO0ac9jAR88EiULqnQdml+dS0 nUXw== MIME-Version: 1.0 X-Received: by 10.140.34.195 with SMTP id l61mr12942350qgl.87.1404440152051; Thu, 03 Jul 2014 19:15:52 -0700 (PDT) Received: by 10.224.202.193 with HTTP; Thu, 3 Jul 2014 19:15:51 -0700 (PDT) In-Reply-To: References: Date: Thu, 3 Jul 2014 19:15:51 -0700 Message-ID: Subject: Re: EDEADLK from fcntl(F_SETFL) ? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , freebsd-current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 02:15:53 -0000 Hi, I'm currently testing this out. It seems to be working out alright. adrian@test3:~/work/freebsd % svn diff stable/10/src/sys/kern/ Index: stable/10/src/sys/kern/kern_lockf.c =================================================================== --- stable/10/src/sys/kern/kern_lockf.c (revision 267627) +++ stable/10/src/sys/kern/kern_lockf.c (working copy) @@ -1425,6 +1425,14 @@ if (lockf_debug & 1) lf_print("lf_setlock: deadlock", lock); #endif + + /* + * If the lock isn't waiting, return EAGAIN + * rather than EDEADLK. + */ + if (((lock->lf_flags & F_WAIT) == 0) && + (error == EDEADLK)) + error = EAGAIN; lf_free_lock(lock); goto out; } On 3 July 2014 17:45, Adrian Chadd wrote: > Hi! > > I've seen sqlite3 crap out due to "disk IO error". It looks like the > F_SETFL path is returning EDEADLK when it shouldn't be - only the > "wait" version of this should be. > > The kernel code looks to be: > > lf_setlock() -> lf_add_outgoing() -> lf_add_edge() -> graph_add_edge() > -> EDEADLK > > .. and lf_setlock() will return an error from lf_add_outgoing() > without checking if it's (a) EDEADLK, and (b) whether we're going to > sleep or not. > > So, sqlite3 trips up on this. I'm sure other things do. What should > the correct thing be? It looks like EWOULDBLOCK is the correct value > to return for F_SETFL failing, not EDEADLK. > > What do those-who-know-POSIX-standards-better-than-I think? > > Thanks! > > > > -a From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 04:15:23 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44D79B78 for ; Fri, 4 Jul 2014 04:15:23 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C09D29F6 for ; Fri, 4 Jul 2014 04:15:22 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s644FMYI000376 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 3 Jul 2014 21:15:22 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s644FMZ6000375 for arch@FreeBSD.org; Thu, 3 Jul 2014 21:15:22 -0700 (PDT) (envelope-from jmg) Date: Thu, 3 Jul 2014 21:15:22 -0700 From: John-Mark Gurney To: arch@FreeBSD.org Subject: callout(9) really this complicated? Message-ID: <20140704041521.GW45513@funkthat.com> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 03 Jul 2014 21:15:22 -0700 (PDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 04:15:23 -0000 So, I was going to look at using callout(9) for some time delayed actions... But upon reading the docs, a) the docs are inconsistent, and b) the docs only talk about requirements in other section... Is there a better interface? If so, can we mark callout(9) deprecated? If not, I have some questions... If you want callout_drain to work properly, you have to add extra code to both your callout, and around the usage of it... callout_drain does not drain the callout: However, the callout subsystem does guarantee that the callout will be fully stopped before callout_drain() returns. Yet other parse of the docs say that you can depend upon the callout being fully stopped.. I've sent email to Ian (iedowse) about why he added this statement... Second, the amount of work you have to do to make sure you drain seems pretty crazy as documented in Avoiding Race Conditions... It seems like if I have created a callout w/ callout_init_mtx, that I shouldn't have to do extra work to make it work correctly... When calling _callout_stop_safe as callout_drain, there is no assert that the lock is held, though it is documented as requiring it by: The function callout_drain() is identical to callout_stop() except that it will wait for the callout to be completed if it is already in progress. Maybe we need to add an additional statement here? and assert that it isn't locked? Also, I have tried to follow the code, but it's complicated, so I'm hoping that I can get some help here. Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 06:15:38 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D3356E9 for ; Fri, 4 Jul 2014 06:15:38 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1FB432300 for ; Fri, 4 Jul 2014 06:15:37 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 92B551FE02D for ; Fri, 4 Jul 2014 08:15:36 +0200 (CEST) Message-ID: <53B64694.9030100@selasky.org> Date: Fri, 04 Jul 2014 08:15:48 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: Re: callout(9) really this complicated? References: <20140704041521.GW45513@funkthat.com> In-Reply-To: <20140704041521.GW45513@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 06:15:38 -0000 On 07/04/14 06:15, John-Mark Gurney wrote: > So, I was going to look at using callout(9) for some time delayed > actions... But upon reading the docs, a) the docs are inconsistent, > and b) the docs only talk about requirements in other section... > > Is there a better interface? If so, can we mark callout(9) deprecated? > If not, I have some questions... > > If you want callout_drain to work properly, you have to add extra code > to both your callout, and around the usage of it... > > callout_drain does not drain the callout: > However, the callout subsystem does guarantee that the callout will be > fully stopped before callout_drain() returns. > > Yet other parse of the docs say that you can depend upon the callout > being fully stopped.. I've sent email to Ian (iedowse) about why he > added this statement... > > Second, the amount of work you have to do to make sure you drain > seems pretty crazy as documented in Avoiding Race Conditions... > > It seems like if I have created a callout w/ callout_init_mtx, > that I shouldn't have to do extra work to make it work correctly... > > When calling _callout_stop_safe as callout_drain, there is no assert > that the lock is held, though it is documented as requiring it by: > The function callout_drain() is identical to callout_stop() except that > it will wait for the callout to be completed if it is already in > progress. > > Maybe we need to add an additional statement here? and assert that it > isn't locked? > > Also, I have tried to follow the code, but it's complicated, so I'm > hoping that I can get some help here. > > Thanks. > Hi, Probably the documentation needs an update. The implementation is fine. Basically like with many other multi processor APIs in the kernel: start and stop operations needs to be locked, not because of callout internals, but because of your driver staying sync. drain is always called unlocked. --HPS From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 09:32:57 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 775E5397; Fri, 4 Jul 2014 09:32:57 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F1C35248C; Fri, 4 Jul 2014 09:32:56 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s649WpkQ040614 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 4 Jul 2014 12:32:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s649WpkQ040614 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s649WpXu040613; Fri, 4 Jul 2014 12:32:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 4 Jul 2014 12:32:51 +0300 From: Konstantin Belousov To: Adrian Chadd Subject: Re: EDEADLK from fcntl(F_SETFL) ? Message-ID: <20140704093251.GF93733@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZJM0Iyk369AH1/Ez" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-current , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 09:32:57 -0000 --ZJM0Iyk369AH1/Ez Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 03, 2014 at 07:15:51PM -0700, Adrian Chadd wrote: > Hi, >=20 > I'm currently testing this out. It seems to be working out alright. >=20 > adrian@test3:~/work/freebsd % svn diff stable/10/src/sys/kern/ >=20 > Index: stable/10/src/sys/kern/kern_lockf.c >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > --- stable/10/src/sys/kern/kern_lockf.c (revision 267627) >=20 > +++ stable/10/src/sys/kern/kern_lockf.c (working copy) >=20 > @@ -1425,6 +1425,14 @@ >=20 > if (lockf_debug & 1) >=20 > lf_print("lf_setlock: deadlock", lock); >=20 > #endif >=20 > + >=20 > + /* >=20 > + * If the lock isn't waiting, return EAGAIN >=20 > + * rather than EDEADLK. >=20 > + */ >=20 > + if (((lock->lf_flags & F_WAIT) =3D=3D 0) && >=20 > + (error =3D=3D EDEADLK)) >=20 > + error =3D EAGAIN; >=20 > lf_free_lock(lock); >=20 > goto out; >=20 > } >=20 > On 3 July 2014 17:45, Adrian Chadd wrote: > > Hi! > > > > I've seen sqlite3 crap out due to "disk IO error". It looks like the > > F_SETFL path is returning EDEADLK when it shouldn't be - only the > > "wait" version of this should be. > > > > The kernel code looks to be: > > > > lf_setlock() -> lf_add_outgoing() -> lf_add_edge() -> graph_add_edge() > > -> EDEADLK > > > > .. and lf_setlock() will return an error from lf_add_outgoing() > > without checking if it's (a) EDEADLK, and (b) whether we're going to > > sleep or not. > > > > So, sqlite3 trips up on this. I'm sure other things do. What should > > the correct thing be? It looks like EWOULDBLOCK is the correct value > > to return for F_SETFL failing, not EDEADLK. > > > > What do those-who-know-POSIX-standards-better-than-I think? I doubt that the patch is correct. If there is an issue in kernel, the patch only hides it. Note that lf_setlock() first calls lf_getblock() to verify that there is no contending lock on the range, and if there is a conflicting lock, the very first statement inside the if() checks for F_WAIT. Either you get a real deadlock, or there is a bug elsewere. --ZJM0Iyk369AH1/Ez Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTtnTCAAoJEJDCuSvBvK1B74IP/is4vn246sDcNmV3K1UdObpM WwEz7QUMdP5y4mssxFYtgquxqa+ynNqAxcDk8Dq5H9BKobjV6A5UT69rZcxQAsyC 8Py5zx5rXPEPuHuh+20hSe3h/hhoZaygnUOz1nNAAHxpamTctxwqZmPvzYEWoS0Q P6Ztoc6z1JxZsUcXERLECKMLnbSqpRa45mfdB59NwlJSei2tGvaN9zJSe8BbJGat p23s0ObsxFUVX7izDyaJFsZpmHA+o1z6YW48+Gs+eyE9wkEsyuPfkN+uTO9ZIiGL tES/jrZcNWUzMd+6Mdq99dRMHrzCAEU5BpqoDX9sUUN45BErtI/Ul18+7w1Yjb9e dBv1NnpO0Gy2emJS+05ecNMJArbKcvLoEjrleHyH6kob0t9l4G2HbR08Oy9CwxL0 uk6ugNgyw1l5TTgkJLP3ewbo+9ngSWyh/USsoOCTj7rMWIQBP/2iPykc/6+HW1Vw 9swbq822tIJI9HeuIZUp0Ssa9yz6TCokiA9Wv1AqXhJ/PaFb2I4I9j5Z47NbxpYG ofCHyPM2RBsRat6oJQF0dGksag36Pqrw37Twt38yu7fxuKpJS3GIs2RDrY1BgeZQ 7FywjcJzgx6TgRjSRoRK5APQnNC81Qo1fqdr0Btw40Jk/FgNw4+4+9g1CO5ARhui EHRXS4DEX/0svZmTar9D =ucwR -----END PGP SIGNATURE----- --ZJM0Iyk369AH1/Ez-- From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 15:57:54 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33CC0D7B for ; Fri, 4 Jul 2014 15:57:54 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 135FB279E for ; Fri, 4 Jul 2014 15:57:54 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s64FvrfR010572 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 4 Jul 2014 08:57:53 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s64FvqoE010571; Fri, 4 Jul 2014 08:57:52 -0700 (PDT) (envelope-from jmg) Date: Fri, 4 Jul 2014 08:57:52 -0700 From: John-Mark Gurney To: Hans Petter Selasky Subject: Re: callout(9) really this complicated? Message-ID: <20140704155752.GB45513@funkthat.com> Mail-Followup-To: Hans Petter Selasky , arch@freebsd.org References: <20140704041521.GW45513@funkthat.com> <53B64694.9030100@selasky.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53B64694.9030100@selasky.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Fri, 04 Jul 2014 08:57:53 -0700 (PDT) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 15:57:54 -0000 Hans Petter Selasky wrote this message on Fri, Jul 04, 2014 at 08:15 +0200: > On 07/04/14 06:15, John-Mark Gurney wrote: > >So, I was going to look at using callout(9) for some time delayed > >actions... But upon reading the docs, a) the docs are inconsistent, > >and b) the docs only talk about requirements in other section... > > > >Is there a better interface? If so, can we mark callout(9) deprecated? > >If not, I have some questions... > > > >If you want callout_drain to work properly, you have to add extra code > >to both your callout, and around the usage of it... > > > >callout_drain does not drain the callout: > > However, the callout subsystem does guarantee that the callout will > > be > > fully stopped before callout_drain() returns. > > > >Yet other parse of the docs say that you can depend upon the callout > >being fully stopped.. I've sent email to Ian (iedowse) about why he > >added this statement... > > > >Second, the amount of work you have to do to make sure you drain > >seems pretty crazy as documented in Avoiding Race Conditions... > > > >It seems like if I have created a callout w/ callout_init_mtx, > >that I shouldn't have to do extra work to make it work correctly... > > > >When calling _callout_stop_safe as callout_drain, there is no assert > >that the lock is held, though it is documented as requiring it by: > > The function callout_drain() is identical to callout_stop() except > > that > > it will wait for the callout to be completed if it is already in > > progress. > > > >Maybe we need to add an additional statement here? and assert that it > >isn't locked? > > > >Also, I have tried to follow the code, but it's complicated, so I'm > >hoping that I can get some help here. > Probably the documentation needs an update. The implementation is fine. Probably... But w/ bad docs, it makes you wonder about the implementation... > Basically like with many other multi processor APIs in the kernel: > > start and stop operations needs to be locked, not because of callout > internals, but because of your driver staying sync. That's what I'd assume, but as above, the docs say otherwise, and the docs are there to clarify your assumptions... > drain is always called unlocked. Then why the whole long section about avoiding races? Point #3 is the main one that I'm talking about... It seems like it'd be easier for callout to properly maintain the active flag (maybe set a flag to tell callout to do so), or not even maintain it since it really isn't used for anything important if you can munge it all you want... There aren't any statements about bad things happening if you call _deactivate before the callout is called... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Jul 4 18:03:59 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 967652E3 for ; Fri, 4 Jul 2014 18:03:59 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 84EB622F4 for ; Fri, 4 Jul 2014 18:03:59 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-76-21-10-192.hsd1.ca.comcast.net [76.21.10.192]) by elvis.mu.org (Postfix) with ESMTPSA id A931E1A3C49 for ; Fri, 4 Jul 2014 11:03:53 -0700 (PDT) Message-ID: <53B6EC87.1070700@freebsd.org> Date: Fri, 04 Jul 2014 11:03:51 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: Re: callout(9) really this complicated? References: <20140704041521.GW45513@funkthat.com> In-Reply-To: <20140704041521.GW45513@funkthat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jul 2014 18:03:59 -0000 On 7/3/14 9:15 PM, John-Mark Gurney wrote: > So, I was going to look at using callout(9) for some time delayed > actions... But upon reading the docs, a) the docs are inconsistent, > and b) the docs only talk about requirements in other section... > > Is there a better interface? If so, can we mark callout(9) deprecated? > If not, I have some questions... > > If you want callout_drain to work properly, you have to add extra code > to both your callout, and around the usage of it... > > callout_drain does not drain the callout: > However, the callout subsystem does guarantee that the callout will be > fully stopped before callout_drain() returns. > > Yet other parse of the docs say that you can depend upon the callout > being fully stopped.. I've sent email to Ian (iedowse) about why he > added this statement... > > Second, the amount of work you have to do to make sure you drain > seems pretty crazy as documented in Avoiding Race Conditions... > > It seems like if I have created a callout w/ callout_init_mtx, > that I shouldn't have to do extra work to make it work correctly... > > When calling _callout_stop_safe as callout_drain, there is no assert > that the lock is held, though it is documented as requiring it by: > The function callout_drain() is identical to callout_stop() except that > it will wait for the callout to be completed if it is already in > progress. > > Maybe we need to add an additional statement here? and assert that it > isn't locked? > > Also, I have tried to follow the code, but it's complicated, so I'm > hoping that I can get some help here. > > Thanks. > This isn't so bad, basically just think about there being another thread launching threads to service a routine, the logic to stop/drain is moderately complex but given a bit more time it should be relatively obvious what is needed to be done. As far as the docs regarding callout_drain, that makes sense, I think the docs are a bit overly verbose (perhaps someone was drinking too much coffee). In other words, with regards to callout_drain, what the docs mean is that when callout_drain() returns, there should be no further calls to the callout. Nothing more is meant. The important trick is NOT to hold a mutex or other lock during callout_drain() that the callout itself uses. This is relatively straightforward. -Alfred From owner-freebsd-arch@FreeBSD.ORG Sun Jul 6 23:07:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8729E9 for ; Sun, 6 Jul 2014 23:07:49 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9297F2E8B for ; Sun, 6 Jul 2014 23:07:49 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id B74EA19360B for ; Sun, 6 Jul 2014 23:07:48 +0000 (UTC) Subject: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: freebsd-arch@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Sun, 06 Jul 2014 16:07:57 -0700 Message-ID: <1404688077.1059.115.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 23:07:50 -0000 Objective: install an xcompile toolchain into a jail for use by poudriere during arm/mips/sparc/power ports pkgs builds. The build should be possible from a non-root user. As far as I can tell, the xdev target is completely busted for non-clang arch's right now as it tries to build clang no matter what I do. Its missing some pretty key documentation to making it work correctly, so a lot of my attempts have been "guess and check" with verbose make. ----------------------------------------------------------------------- Attempt #1: I have been trying non-root xdev builds: MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips -- dies because it tries to compile CLANG. ----------------------------------------------------------------------- Attempt #2: Apply a hack from Baptiste that isn't quite right, but at least gets farther, note the missing variable causing "//usr/mips-freebsd" http://people.freebsd.org/~sbruno/src.ops.mk.diff ===> gnu/usr.bin/cc/gcov (all) mtree populating //usr/mips-freebsd mkdir: //usr/mips-freebsd: Permission denied *** Error code 1 ----------------------------------------------------------------------- Attempt #3: Add XDTP MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips XDTP=/var/tmp/mips_cc Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's a bit farther but compile failure in locating critical include files. ===> gnu/lib/libstdc++ (obj,depend,all,install) In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or directory ----------------------------------------------------------------------- Attempt #4: Add the additional XDDESTDIR MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst -- Same results as attempt #3 ----------------------------------------------------------------------- Even attempting to do stuff for *clang* enabled architectures bails because its not respecting prefixes: MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6 -- bails because it tries to: ===> usr.bin/clang/tblgen (all) mtree populating //usr/armv6-freebsd mtree: etc/ntp: Permission denied _xi-cross-tools ===> xdev gnu/usr.bin/binutils (install) ===> gnu/usr.bin/binutils/libiberty (install) ===> gnu/usr.bin/binutils/libbfd (install) ===> gnu/usr.bin/binutils/libopcodes (install) ===> gnu/usr.bin/binutils/libbinutils (install) ===> gnu/usr.bin/binutils/addr2line (install) ===> gnu/usr.bin/binutils/as (install) ===> gnu/usr.bin/binutils/ld (install) install: //usr/armv6-freebsd/usr/bin/ld: Permission denied *** Error code 71 ----------------------------------------------------------------------- Adding XDTP and XDDESTDIR results in a little more progress but obvious failures to attempt and install things directly into my host system: MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6 XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc ===> secure/lib/libssh (install) ===> usr.bin/lex/lib (obj,depend,all,install) mkdir: ../../../../usr: Permission denied *** Error code 1 Stop. make[1]: stopped in /home/sbruno/bsd/fbsd_head *** Error code 1 Stop. make: stopped in /home/sbruno/bsd/fbsd_head From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 00:38:21 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 11CFCE88; Mon, 7 Jul 2014 00:38:21 +0000 (UTC) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DBB3525DF; Mon, 7 Jul 2014 00:38:20 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s670cCkb022030; Sun, 6 Jul 2014 17:38:16 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201407070038.s670cCkb022030@gw.catspoiler.org> Date: Sun, 6 Jul 2014 17:38:12 -0700 (PDT) From: Don Lewis Subject: [patch] axe RF_TIMESHARE? To: arch@FreeBSD.org, jhb@FreeBSD.org MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 00:38:21 -0000 When he was reviewing my previous batch of subr_rman.c patches, John Baldwin suggested getting rid of the RF_TIMESHARE feature. There is nothing in the tree that uses it, and he didn't think that anything ever used it. Getting rid of RF_TIMESHARE gets rid of quite a bit of complexity and unbloats the the size of the code on i386 by more than 500 bytes. Before: text data bss dec hex filename 9663 376 28 10067 2753 subr_rman.o After: text data bss dec hex filename 9128 376 28 9532 253c subr_rman.o The int_rman_activate_resource() and int_rman_deactivate_resource() functions become trivial and I elected to manually inline both. It looks to me like the special handling of RF_ACTIVE isn't needed in reserve_resource_bound() isn't needed and doesn't have to be deferred to the end of the function. The comments seemed to indicate that this code was iffy for RF_TIMESHARE in any case. Should we nuke RF_TIMESHARE? Should this change be MFC'ed, and if so, how far back? Comments on the attached patch? Index: sys/kern/subr_rman.c =================================================================== --- sys/kern/subr_rman.c (revision 268273) +++ sys/kern/subr_rman.c (working copy) @@ -109,9 +109,6 @@ struct rman_head rman_head; static struct mtx rman_mtx; /* mutex to protect rman_head */ -static int int_rman_activate_resource(struct rman *rm, struct resource_i *r, - struct resource_i **whohas); -static int int_rman_deactivate_resource(struct resource_i *r); static int int_rman_release_resource(struct rman *rm, struct resource_i *r); static __inline struct resource_i * @@ -321,7 +318,7 @@ /* Not supported for shared resources. */ r = rr->__r_i; - if (r->r_flags & (RF_TIMESHARE | RF_SHAREABLE)) + if (r->r_flags & RF_SHAREABLE) return (EINVAL); /* @@ -434,7 +431,7 @@ return (0); } -#define SHARE_TYPE(f) (f & (RF_SHAREABLE | RF_TIMESHARE | RF_PREFETCHABLE)) +#define SHARE_TYPE(f) (f & (RF_SHAREABLE | RF_PREFETCHABLE)) struct resource * rman_reserve_resource_bound(struct rman *rm, u_long start, u_long end, @@ -451,10 +448,9 @@ "length %#lx, flags %u, device %s\n", rm->rm_descr, start, end, count, flags, dev == NULL ? "" : device_get_nameunit(dev))); - KASSERT((flags & (RF_WANTED | RF_FIRSTSHARE)) == 0, + KASSERT((flags & RF_FIRSTSHARE) == 0, ("invalid flags %#x", flags)); - new_rflags = (flags & ~(RF_ACTIVE | RF_WANTED | RF_FIRSTSHARE)) | - RF_ALLOCATED; + new_rflags = (flags & ~RF_FIRSTSHARE) | RF_ALLOCATED; mtx_lock(rm->rm_mtx); @@ -600,7 +596,7 @@ * additional work, but this does not seem warranted.) */ DPRINTF(("no unshared regions found\n")); - if ((flags & (RF_SHAREABLE | RF_TIMESHARE)) == 0) + if ((flags & RF_SHAREABLE) == 0) goto out; for (s = r; s && s->r_end <= end; s = TAILQ_NEXT(s, r_link)) { @@ -635,25 +631,11 @@ goto out; } } - /* * We couldn't find anything. */ + out: - /* - * If the user specified RF_ACTIVE in flags, we attempt to atomically - * activate the resource. If this fails, we release the resource - * and indicate overall failure. (This behavior probably doesn't - * make sense for RF_TIMESHARE-type resources.) - */ - if (rv && (flags & RF_ACTIVE) != 0) { - struct resource_i *whohas; - if (int_rman_activate_resource(rm, rv, &whohas)) { - int_rman_release_resource(rm, rv); - rv = NULL; - } - } - mtx_unlock(rm->rm_mtx); return (rv == NULL ? NULL : &rv->r_r); } @@ -667,91 +649,17 @@ dev)); } -static int -int_rman_activate_resource(struct rman *rm, struct resource_i *r, - struct resource_i **whohas) -{ - struct resource_i *s; - int ok; - - /* - * If we are not timesharing, then there is nothing much to do. - * If we already have the resource, then there is nothing at all to do. - * If we are not on a sharing list with anybody else, then there is - * little to do. - */ - if ((r->r_flags & RF_TIMESHARE) == 0 - || (r->r_flags & RF_ACTIVE) != 0 - || r->r_sharehead == NULL) { - r->r_flags |= RF_ACTIVE; - return 0; - } - - ok = 1; - for (s = LIST_FIRST(r->r_sharehead); s && ok; - s = LIST_NEXT(s, r_sharelink)) { - if ((s->r_flags & RF_ACTIVE) != 0) { - ok = 0; - *whohas = s; - } - } - if (ok) { - r->r_flags |= RF_ACTIVE; - return 0; - } - return EBUSY; -} - int rman_activate_resource(struct resource *re) { - int rv; - struct resource_i *r, *whohas; + struct resource_i *r; struct rman *rm; r = re->__r_i; rm = r->r_rm; mtx_lock(rm->rm_mtx); - rv = int_rman_activate_resource(rm, r, &whohas); + r->r_flags |= RF_ACTIVE; mtx_unlock(rm->rm_mtx); - return rv; -} - -int -rman_await_resource(struct resource *re, int pri, int timo) -{ - int rv; - struct resource_i *r, *whohas; - struct rman *rm; - - r = re->__r_i; - rm = r->r_rm; - mtx_lock(rm->rm_mtx); - for (;;) { - rv = int_rman_activate_resource(rm, r, &whohas); - if (rv != EBUSY) - return (rv); /* returns with mutex held */ - - if (r->r_sharehead == NULL) - panic("rman_await_resource"); - whohas->r_flags |= RF_WANTED; - rv = msleep(r->r_sharehead, rm->rm_mtx, pri, "rmwait", timo); - if (rv) { - mtx_unlock(rm->rm_mtx); - return (rv); - } - } -} - -static int -int_rman_deactivate_resource(struct resource_i *r) -{ - - r->r_flags &= ~RF_ACTIVE; - if (r->r_flags & RF_WANTED) { - r->r_flags &= ~RF_WANTED; - wakeup(r->r_sharehead); - } return 0; } @@ -762,7 +670,7 @@ rm = r->__r_i->r_rm; mtx_lock(rm->rm_mtx); - int_rman_deactivate_resource(r->__r_i); + r->__r_i->r_flags &= ~RF_ACTIVE; mtx_unlock(rm->rm_mtx); return 0; } @@ -773,7 +681,7 @@ struct resource_i *s, *t; if (r->r_flags & RF_ACTIVE) - int_rman_deactivate_resource(r); + r->r_flags &= ~RF_ACTIVE; /* * Check for a sharing list first. If there is one, then we don't Index: sys/sys/rman.h =================================================================== --- sys/sys/rman.h (revision 268273) +++ sys/sys/rman.h (working copy) @@ -42,8 +42,8 @@ #define RF_ALLOCATED 0x0001 /* resource has been reserved */ #define RF_ACTIVE 0x0002 /* resource allocation has been activated */ #define RF_SHAREABLE 0x0004 /* resource permits contemporaneous sharing */ -#define RF_TIMESHARE 0x0008 /* resource permits time-division sharing */ -#define RF_WANTED 0x0010 /* somebody is waiting for this resource */ +#define RF_SPARE1 0x0008 +#define RF_SPARE2 0x0010 #define RF_FIRSTSHARE 0x0020 /* first in sharing list */ #define RF_PREFETCHABLE 0x0040 /* resource is prefetchable */ #define RF_OPTIONAL 0x0080 /* for bus_alloc_resources() */ From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 00:41:41 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B959FE8; Mon, 7 Jul 2014 00:41:41 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0DB42664; Mon, 7 Jul 2014 00:41:40 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id i50so3140189qgf.14 for ; Sun, 06 Jul 2014 17:41:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=5QRNXdRj4vQhk0iJ2vZ5+HuLklZSVsjeEjAtbQyrqiU=; b=ADSxc0BkZKCCb7fTxA8D5+hKIPu0PsTLyc7AyfnkaGo9V9zONh6Mub1EHbOR31U2Zx CZpreZ6X0rwqfs9ktMnt4axXyiCdn9Q9KqXU1wxXVESCP7r8yGPuS6rWry/V9+xYC1SS zADoUuXxUa1My9DFmWSxEonvl2BS7sOE4L9TDIQsfezH/Jp+WmERiK0geddjkGGEN+iC 1iiKzaBdyDFW3RSRICtrvjMETGSwbXDaySPlWekC4YNORQUlEVs9PpP1+pkUw6XVqi9J bAJwB5q0HxiGemc2Ipf4vVe0SiYKjd9xXYtXofB2bqk7jQBpKv97fT+8kQQDkestKrlx U0rQ== MIME-Version: 1.0 X-Received: by 10.140.80.49 with SMTP id b46mr39156849qgd.102.1404693700004; Sun, 06 Jul 2014 17:41:40 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Sun, 6 Jul 2014 17:41:39 -0700 (PDT) In-Reply-To: <201407070038.s670cCkb022030@gw.catspoiler.org> References: <201407070038.s670cCkb022030@gw.catspoiler.org> Date: Sun, 6 Jul 2014 17:41:39 -0700 X-Google-Sender-Auth: D7X8H5FHezVqbGJfj0RRCKdij4M Message-ID: Subject: Re: [patch] axe RF_TIMESHARE? From: Adrian Chadd To: Don Lewis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 00:41:41 -0000 Hi, What's it supposed to be used for? -a On 6 July 2014 17:38, Don Lewis wrote: > When he was reviewing my previous batch of subr_rman.c patches, John > Baldwin suggested getting rid of the RF_TIMESHARE feature. There is > nothing in the tree that uses it, and he didn't think that anything ever > used it. > > Getting rid of RF_TIMESHARE gets rid of quite a bit of complexity and > unbloats the the size of the code on i386 by more than 500 bytes. > > Before: > text data bss dec hex filename > 9663 376 28 10067 2753 subr_rman.o > > After: > text data bss dec hex filename > 9128 376 28 9532 253c subr_rman.o > > > The int_rman_activate_resource() and int_rman_deactivate_resource() > functions become trivial and I elected to manually inline both. > > It looks to me like the special handling of RF_ACTIVE isn't needed in > reserve_resource_bound() isn't needed and doesn't have to be deferred to > the end of the function. The comments seemed to indicate that this code > was iffy for RF_TIMESHARE in any case. > > Should we nuke RF_TIMESHARE? > > Should this change be MFC'ed, and if so, how far back? > > Comments on the attached patch? > > Index: sys/kern/subr_rman.c > =================================================================== > --- sys/kern/subr_rman.c (revision 268273) > +++ sys/kern/subr_rman.c (working copy) > @@ -109,9 +109,6 @@ > > struct rman_head rman_head; > static struct mtx rman_mtx; /* mutex to protect rman_head */ > -static int int_rman_activate_resource(struct rman *rm, struct resource_i *r, > - struct resource_i **whohas); > -static int int_rman_deactivate_resource(struct resource_i *r); > static int int_rman_release_resource(struct rman *rm, struct resource_i *r); > > static __inline struct resource_i * > @@ -321,7 +318,7 @@ > > /* Not supported for shared resources. */ > r = rr->__r_i; > - if (r->r_flags & (RF_TIMESHARE | RF_SHAREABLE)) > + if (r->r_flags & RF_SHAREABLE) > return (EINVAL); > > /* > @@ -434,7 +431,7 @@ > return (0); > } > > -#define SHARE_TYPE(f) (f & (RF_SHAREABLE | RF_TIMESHARE | RF_PREFETCHABLE)) > +#define SHARE_TYPE(f) (f & (RF_SHAREABLE | RF_PREFETCHABLE)) > > struct resource * > rman_reserve_resource_bound(struct rman *rm, u_long start, u_long end, > @@ -451,10 +448,9 @@ > "length %#lx, flags %u, device %s\n", rm->rm_descr, start, end, > count, flags, > dev == NULL ? "" : device_get_nameunit(dev))); > - KASSERT((flags & (RF_WANTED | RF_FIRSTSHARE)) == 0, > + KASSERT((flags & RF_FIRSTSHARE) == 0, > ("invalid flags %#x", flags)); > - new_rflags = (flags & ~(RF_ACTIVE | RF_WANTED | RF_FIRSTSHARE)) | > - RF_ALLOCATED; > + new_rflags = (flags & ~RF_FIRSTSHARE) | RF_ALLOCATED; > > mtx_lock(rm->rm_mtx); > > @@ -600,7 +596,7 @@ > * additional work, but this does not seem warranted.) > */ > DPRINTF(("no unshared regions found\n")); > - if ((flags & (RF_SHAREABLE | RF_TIMESHARE)) == 0) > + if ((flags & RF_SHAREABLE) == 0) > goto out; > > for (s = r; s && s->r_end <= end; s = TAILQ_NEXT(s, r_link)) { > @@ -635,25 +631,11 @@ > goto out; > } > } > - > /* > * We couldn't find anything. > */ > + > out: > - /* > - * If the user specified RF_ACTIVE in flags, we attempt to atomically > - * activate the resource. If this fails, we release the resource > - * and indicate overall failure. (This behavior probably doesn't > - * make sense for RF_TIMESHARE-type resources.) > - */ > - if (rv && (flags & RF_ACTIVE) != 0) { > - struct resource_i *whohas; > - if (int_rman_activate_resource(rm, rv, &whohas)) { > - int_rman_release_resource(rm, rv); > - rv = NULL; > - } > - } > - > mtx_unlock(rm->rm_mtx); > return (rv == NULL ? NULL : &rv->r_r); > } > @@ -667,91 +649,17 @@ > dev)); > } > > -static int > -int_rman_activate_resource(struct rman *rm, struct resource_i *r, > - struct resource_i **whohas) > -{ > - struct resource_i *s; > - int ok; > - > - /* > - * If we are not timesharing, then there is nothing much to do. > - * If we already have the resource, then there is nothing at all to do. > - * If we are not on a sharing list with anybody else, then there is > - * little to do. > - */ > - if ((r->r_flags & RF_TIMESHARE) == 0 > - || (r->r_flags & RF_ACTIVE) != 0 > - || r->r_sharehead == NULL) { > - r->r_flags |= RF_ACTIVE; > - return 0; > - } > - > - ok = 1; > - for (s = LIST_FIRST(r->r_sharehead); s && ok; > - s = LIST_NEXT(s, r_sharelink)) { > - if ((s->r_flags & RF_ACTIVE) != 0) { > - ok = 0; > - *whohas = s; > - } > - } > - if (ok) { > - r->r_flags |= RF_ACTIVE; > - return 0; > - } > - return EBUSY; > -} > - > int > rman_activate_resource(struct resource *re) > { > - int rv; > - struct resource_i *r, *whohas; > + struct resource_i *r; > struct rman *rm; > > r = re->__r_i; > rm = r->r_rm; > mtx_lock(rm->rm_mtx); > - rv = int_rman_activate_resource(rm, r, &whohas); > + r->r_flags |= RF_ACTIVE; > mtx_unlock(rm->rm_mtx); > - return rv; > -} > - > -int > -rman_await_resource(struct resource *re, int pri, int timo) > -{ > - int rv; > - struct resource_i *r, *whohas; > - struct rman *rm; > - > - r = re->__r_i; > - rm = r->r_rm; > - mtx_lock(rm->rm_mtx); > - for (;;) { > - rv = int_rman_activate_resource(rm, r, &whohas); > - if (rv != EBUSY) > - return (rv); /* returns with mutex held */ > - > - if (r->r_sharehead == NULL) > - panic("rman_await_resource"); > - whohas->r_flags |= RF_WANTED; > - rv = msleep(r->r_sharehead, rm->rm_mtx, pri, "rmwait", timo); > - if (rv) { > - mtx_unlock(rm->rm_mtx); > - return (rv); > - } > - } > -} > - > -static int > -int_rman_deactivate_resource(struct resource_i *r) > -{ > - > - r->r_flags &= ~RF_ACTIVE; > - if (r->r_flags & RF_WANTED) { > - r->r_flags &= ~RF_WANTED; > - wakeup(r->r_sharehead); > - } > return 0; > } > > @@ -762,7 +670,7 @@ > > rm = r->__r_i->r_rm; > mtx_lock(rm->rm_mtx); > - int_rman_deactivate_resource(r->__r_i); > + r->__r_i->r_flags &= ~RF_ACTIVE; > mtx_unlock(rm->rm_mtx); > return 0; > } > @@ -773,7 +681,7 @@ > struct resource_i *s, *t; > > if (r->r_flags & RF_ACTIVE) > - int_rman_deactivate_resource(r); > + r->r_flags &= ~RF_ACTIVE; > > /* > * Check for a sharing list first. If there is one, then we don't > Index: sys/sys/rman.h > =================================================================== > --- sys/sys/rman.h (revision 268273) > +++ sys/sys/rman.h (working copy) > @@ -42,8 +42,8 @@ > #define RF_ALLOCATED 0x0001 /* resource has been reserved */ > #define RF_ACTIVE 0x0002 /* resource allocation has been activated */ > #define RF_SHAREABLE 0x0004 /* resource permits contemporaneous sharing */ > -#define RF_TIMESHARE 0x0008 /* resource permits time-division sharing */ > -#define RF_WANTED 0x0010 /* somebody is waiting for this resource */ > +#define RF_SPARE1 0x0008 > +#define RF_SPARE2 0x0010 > #define RF_FIRSTSHARE 0x0020 /* first in sharing list */ > #define RF_PREFETCHABLE 0x0040 /* resource is prefetchable */ > #define RF_OPTIONAL 0x0080 /* for bus_alloc_resources() */ > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 00:47:41 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ABBB51A8; Mon, 7 Jul 2014 00:47:41 +0000 (UTC) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77B702685; Mon, 7 Jul 2014 00:47:41 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s670lWUv022054; Sun, 6 Jul 2014 17:47:36 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201407070047.s670lWUv022054@gw.catspoiler.org> Date: Sun, 6 Jul 2014 17:47:32 -0700 (PDT) From: Don Lewis Subject: Re: [patch] axe RF_TIMESHARE? To: adrian@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 00:47:41 -0000 On 6 Jul, Adrian Chadd wrote: > Hi, > > What's it supposed to be used for? My understanding it that it is supposed to be used to allow two more devices to claim the same resource, such as an I/O port range, but only one device can be active at a time. From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 01:29:23 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 860FC685; Mon, 7 Jul 2014 01:29:23 +0000 (UTC) Received: from mail-qa0-x236.google.com (mail-qa0-x236.google.com [IPv6:2607:f8b0:400d:c00::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26C43290D; Mon, 7 Jul 2014 01:29:23 +0000 (UTC) Received: by mail-qa0-f54.google.com with SMTP id v10so3016263qac.13 for ; Sun, 06 Jul 2014 18:29:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=it2f24oFWoSXxo4Dy0+qulPfOHINE7705R9fiLGShjs=; b=r9oMK+ZOoFmbQ4kWmbSkNiT6N9tbtIpelUNpZ/evqyQVgeSIwBgz/nWGqOl+glx+tr 0UsGpaJBwDCHqviXrlTVfDIk4WSJAmH4W28eUU/RVj7ozLafs52PWYwPKXZJRWZuwExg acaCVBVsZwWCh8swNPaCAvfNBqcd/FhPPODyXSDe11Kiq9XZ4L4+r0XwBpzKCNY/skxg wfu0/q3w0SogYhPPa4l74z3ZlYgJUNJZp0XMjD/nzzvjEZ6E4nQr9D6owkmNIfAo5M/s moz2GRXBvK90TnZoLfIr005LK1MC6M8GzsD4xR9TfmzrS3VlrPOnETMYajh+RyYN8knN +ISA== MIME-Version: 1.0 X-Received: by 10.140.80.49 with SMTP id b46mr39426950qgd.102.1404696562139; Sun, 06 Jul 2014 18:29:22 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Sun, 6 Jul 2014 18:29:22 -0700 (PDT) In-Reply-To: <201407070047.s670lWUv022054@gw.catspoiler.org> References: <201407070047.s670lWUv022054@gw.catspoiler.org> Date: Sun, 6 Jul 2014 18:29:22 -0700 X-Google-Sender-Auth: AwNLwYSdy0OCRyz-PQvzT7ks8pU Message-ID: Subject: Re: [patch] axe RF_TIMESHARE? From: Adrian Chadd To: Don Lewis Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 01:29:23 -0000 On 6 July 2014 17:47, Don Lewis wrote: > On 6 Jul, Adrian Chadd wrote: >> Hi, >> >> What's it supposed to be used for? > > My understanding it that it is supposed to be used to allow two more > devices to claim the same resource, such as an I/O port range, but only > one device can be active at a time. Interesting. I wonder what kinds of things would want to do this. There's a few interesting things like implementing the spibus using this instead of the current way of calling a bus lock method before doing any bus IO. -a From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 05:54:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16AE0175; Mon, 7 Jul 2014 05:54:28 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AB2452C7A; Mon, 7 Jul 2014 05:54:27 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s675sO2A051034; Mon, 7 Jul 2014 01:54:24 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s675sOdQ051033; Mon, 7 Jul 2014 01:54:24 -0400 (EDT) (envelope-from wollman) Date: Mon, 7 Jul 2014 01:54:24 -0400 (EDT) From: Garrett Wollman Message-Id: <201407070554.s675sOdQ051033@hergotha.csail.mit.edu> To: adrian@freebsd.org Subject: Re: [patch] axe RF_TIMESHARE? X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: References: <201407070047.s670lWUv022054@gw.catspoiler.org> Organization: none X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Mon, 07 Jul 2014 01:54:24 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on hergotha.csail.mit.edu Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 05:54:28 -0000 In article , adrian@freebsd.org writes: >On 6 July 2014 17:47, Don Lewis wrote: >> On 6 Jul, Adrian Chadd wrote: >>> Hi, >>> >>> What's it supposed to be used for? >> >> My understanding it that it is supposed to be used to allow two more >> devices to claim the same resource, such as an I/O port range, but only >> one device can be active at a time. > >Interesting. I wonder what kinds of things would want to do this. Well, when I included it the original rman implementation, I had a few applications in mind: - shared interrupts on the ISA bus - some weird stuff that people were doing on parallel ports - if you had some device with a really big shared-memory region that you might want to be able to turn on and off to allow other devices to use that address space It's been more than a decade since I wrote that code, and I think pretty much every conceivable use for it has long since been overtaken by events. In any modern situation where you might want to use this, you probably need the cooperation of higher-level code to make it work anyway, so you might as well implement it at that layer instead. So yes, please, do make it go away. Simpler and easier to understand is better. -GAWollman From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 15:46:14 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4358FE6B; Mon, 7 Jul 2014 15:46:14 +0000 (UTC) Received: from dmz-mailsec-scanner-7.mit.edu (dmz-mailsec-scanner-7.mit.edu [18.7.68.36]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C7043235B; Mon, 7 Jul 2014 15:46:13 +0000 (UTC) X-AuditID: 12074424-f79146d00000067c-0d-53bac0bd268d Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id 73.C4.01660.DB0CAB35; Mon, 7 Jul 2014 11:46:06 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id s67Fk5NW030906; Mon, 7 Jul 2014 11:46:05 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s67Fk3Sr031151 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 7 Jul 2014 11:46:04 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s67Fk3Kp003749; Mon, 7 Jul 2014 11:46:03 -0400 (EDT) Date: Mon, 7 Jul 2014 11:46:03 -0400 (EDT) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: sbruno@freebsd.org Subject: Re: Total confusion over toolchain/xdev behavior In-Reply-To: <1404688077.1059.115.camel@bruno> Message-ID: References: <1404688077.1059.115.camel@bruno> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrAIsWRmVeSWpSXmKPExsUixCmqrbvvwK5gg4WSFrOnT2Oy6Ok9wejA 5DHj03yWAMYoLpuU1JzMstQifbsErox1B7+yFazmqPjYOJWxgbGHvYuRg0NCwETi5H+dLkZO IFNM4sK99WxdjFwcQgKzmSS2vpvBBOFsYJT4sKKdEcI5yCTxZsJSZpAWIYF6iaMtX8BsFgEt ifnrvrGB2GwCahKP9zazQoxVlNh8ahJYjQjQimn/7jCCbGYWkJGY89obxBQWsJTYMj0DpIJT QE/iWvNrRhCbV8BRYt2WV2wQm3Ql9r2YwgRiiwroSKzeP4UFokZQ4uTMJ2A2M9CYc3+us01g FJqFJDULSWoBI9MqRtmU3Crd3MTMnOLUZN3i5MS8vNQiXXO93MwSvdSU0k2MoIBld1HZwdh8 SOkQowAHoxIP74FVO4OFWBPLiitzDzFKcjApifJ67N4VLMSXlJ9SmZFYnBFfVJqTWnyIUYKD WUmEd8VyoBxvSmJlVWpRPkxKmoNFSZz3rbVVsJBAemJJanZqakFqEUxWhoNDSYL34X6gRsGi 1PTUirTMnBKENBMHJ8hwHqDh90FqeIsLEnOLM9Mh8qcYFaXEec+DJARAEhmleXC9sITyilEc 6BVh3g8gVTzAZATX/QpoMBPQ4M/vd4AMLklESEk1MEpLxV1ZEr97z/V/czNYzKVq2lR68pMY Jdfc3v+x9uiC7mKXl9fOdb2dY3r4/DmW/4uUHjbs5qnsyfUNE9i59mRC0CSXrcWHP3h5OcUu XPs+cPYdV6PNzTeCvZ2ZcqRW8B2QVu7bt/GqRoG/y8Gm2Xff6n//mrTlW1o07x9PWf6Ly/pP R4dd6VJiKc5INNRiLipOBACjb+4pAwMAAA== Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 15:46:14 -0000 On Sun, 6 Jul 2014, Sean Bruno wrote: > ----------------------------------------------------------------------- > > Even attempting to do stuff for *clang* enabled architectures bails > because its not respecting prefixes: > MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6 > -- bails because it tries to: > ===> usr.bin/clang/tblgen (all) > mtree populating //usr/armv6-freebsd > mtree: etc/ntp: Permission denied > _xi-cross-tools > ===> xdev gnu/usr.bin/binutils (install) > ===> gnu/usr.bin/binutils/libiberty (install) > ===> gnu/usr.bin/binutils/libbfd (install) > ===> gnu/usr.bin/binutils/libopcodes (install) > ===> gnu/usr.bin/binutils/libbinutils (install) > ===> gnu/usr.bin/binutils/addr2line (install) > ===> gnu/usr.bin/binutils/as (install) > ===> gnu/usr.bin/binutils/ld (install) > install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > *** Error code 71 This seems to be "obviously a bug" that should not be too hard to make progress on, especially if you already have make debug logs. -Ben From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 18:14:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2EA6A62; Mon, 7 Jul 2014 18:14:11 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A30662106; Mon, 7 Jul 2014 18:14:10 +0000 (UTC) Received: from [10.11.98.51] (unknown [208.111.174.98]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 4694619360B; Mon, 7 Jul 2014 18:14:09 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Benjamin Kaduk In-Reply-To: References: <1404688077.1059.115.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Mon, 07 Jul 2014 11:14:08 -0700 Message-ID: <1404756848.1105.1.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 18:14:11 -0000 On Mon, 2014-07-07 at 11:46 -0400, Benjamin Kaduk wrote: > On Sun, 6 Jul 2014, Sean Bruno wrote: > > > ----------------------------------------------------------------------- > > > > Even attempting to do stuff for *clang* enabled architectures bails > > because its not respecting prefixes: > > MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6 > > -- bails because it tries to: > > ===> usr.bin/clang/tblgen (all) > > mtree populating //usr/armv6-freebsd > > mtree: etc/ntp: Permission denied > > _xi-cross-tools > > ===> xdev gnu/usr.bin/binutils (install) > > ===> gnu/usr.bin/binutils/libiberty (install) > > ===> gnu/usr.bin/binutils/libbfd (install) > > ===> gnu/usr.bin/binutils/libopcodes (install) > > ===> gnu/usr.bin/binutils/libbinutils (install) > > ===> gnu/usr.bin/binutils/addr2line (install) > > ===> gnu/usr.bin/binutils/as (install) > > ===> gnu/usr.bin/binutils/ld (install) > > install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > > *** Error code 71 > > This seems to be "obviously a bug" that should not be too hard to make > progress on, especially if you already have make debug logs. > > -Ben Sure. I agree its a bug, if my usage of the XDEV target is supposed to be supported. I don't know who owns this target, what its supposed to be used for, nor what its requirements are. All I know is that people tell me its supposed to meat my requirements from my original "Objective" statement. Is this even the right make target to use? sean From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 19:28:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96F937C8 for ; Mon, 7 Jul 2014 19:28:49 +0000 (UTC) Received: from mail-pd0-f181.google.com (mail-pd0-f181.google.com [209.85.192.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64A6E27FD for ; Mon, 7 Jul 2014 19:28:48 +0000 (UTC) Received: by mail-pd0-f181.google.com with SMTP id v10so5854320pde.12 for ; Mon, 07 Jul 2014 12:28:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=tcjenkrVz/cZquusZ2pV0P+XyPKdbP/xmRznhkFQ++k=; b=b0QupsMh8hAa2wjUIQPNONPzo548tsh1hmfB1knsaKp/CUUwKQKuausk+8o/naLNqN Z2a7jgu3/ABj8eYkAauPfTeH4NxC3dJzVy6679e8+s2E8ZCs2NR17wsdoxrogjFEVlnm DYj1VyTAh8gr5M+g43PJu5YTnucvQRx6qba03Hj96ByEPuDxr0YtjKC9qmpAUG5dHIJY Bbwzxzj/XP0SEM/1oxa9oErl4/vgcia3WmK8bHnI1NmiZzfoQwoPLn96ALiSG/PMiXm7 ShaoVsUWEsvOx0o52PqQKTt/91M9BGlYG2RWnTxM2qAJzjF0FU8SYYL0ZeUQXVE2tuY2 P5zg== X-Gm-Message-State: ALoCoQmWhZrrQT/8RtQPLlm/6k9eEjbOnU6Hr+SgQdt+5ooD/cTfRfWtIHkcySuFrukXAKVV+zav X-Received: by 10.70.125.133 with SMTP id mq5mr490424pdb.97.1404761327941; Mon, 07 Jul 2014 12:28:47 -0700 (PDT) Received: from [10.64.26.2] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id co3sm53415089pbb.89.2014.07.07.12.28.46 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 12:28:47 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_4C09DEC7-5515-4388-A003-7A4BF02D5D41"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404756848.1105.1.camel@bruno> Date: Mon, 7 Jul 2014 13:28:55 -0600 Message-Id: <9CE37432-1028-44FE-B0B7-224625EB9AB3@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404756848.1105.1.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: Benjamin Kaduk , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 19:28:49 -0000 --Apple-Mail=_4C09DEC7-5515-4388-A003-7A4BF02D5D41 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 12:14 PM, Sean Bruno wrote: > On Mon, 2014-07-07 at 11:46 -0400, Benjamin Kaduk wrote: >> On Sun, 6 Jul 2014, Sean Bruno wrote: >>=20 >>> = ----------------------------------------------------------------------- >>>=20 >>> Even attempting to do stuff for *clang* enabled architectures bails >>> because its not respecting prefixes: >>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm = XDEV_ARCH=3Darmv6 >>> -- bails because it tries to: >>> =3D=3D=3D> usr.bin/clang/tblgen (all) >>> mtree populating //usr/armv6-freebsd >>> mtree: etc/ntp: Permission denied >>> _xi-cross-tools >>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) >>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) >>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied >>> *** Error code 71 >>=20 >> This seems to be "obviously a bug" that should not be too hard to = make=20 >> progress on, especially if you already have make debug logs. >>=20 >> -Ben >=20 >=20 > Sure. I agree its a bug, if my usage of the XDEV target is supposed = to > be supported. I built the xdev target years ago. There was no notion of it supporting = root-less build, since FreeBSD src didn=92t support it at all. The folks = that added it to FreeBSD missed xdev. > I don't know who owns this target, what its supposed to be used for, = nor > what its requirements are. All I know is that people tell me its > supposed to meat my requirements from my original "Objective" = statement. Well, the folks that did clang didn=92t update this target. Nor did the = folks that did the cross build stuff. Nor the folks that did=85 Well, = you get the idea. It has been an unloved target for some time now, or at = least not a priority for people adding things to the system. I=92ve done = bits and pieces here and there over the years, but I=92m sure that it is = quite rough around the edges. I fear that your experience validates that = assessment. Having said that, though, the whole cross build ports thing is getting = renewed interest after a decade of dormancy (I did xdev maybe 10 years = ago to allow Timing Solutions to build some ports with the native arm = compiler). Lately bugs have been fixed. You are quite likely the first = person to try it without root ever. There will be bumps for that :). > Is this even the right make target to use? It is the only target we have that supports building outside the crazy = build environment we have. Just for grins, I did the following on a stock system (without your = change to src.opts.mk from Bapt) and not as root: % mkdir $HOME/D % make DESTDIR=3D$HOME/D XDEV=3Dmips XDEV_ARCH=3Dmips xdev = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt %=20 and it hit the stdc++ bits not being defined. Please see above with the = "clang folks didn=92t properly integrate=94 Or possibly some other = reason. However, when I remove DESTDIR and build normally as root, it works. If = I add DESTDIR to the root build, it works=85 So there=92s definitely = something wrong with the non-root build. Warner --Apple-Mail=_4C09DEC7-5515-4388-A003-7A4BF02D5D41 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTuvT3AAoJEGwc0Sh9sBEAV8cQALV+oco/VF26i5daXLr+wWIY SAbg+eg7o185CjkYiYs439cB+gtEufwCaUlkm/DC9ZYGnd4lpXEaQQtYJtvRDxku wKUienXshtht7c3wLT0YS9ZEZU/qrmzB3v8Jvv6BUZKD3E/dCrM6NrJFwWHuHzXz m7ddIhtTi02GrKhTCu+gvR8sc5y4LrYtq2JjDYXU2TyZbZdNtrglpEfZiT9z0omU I+z4ggO+F4UsjFimQYRAxviT+9PGzPaLWYw8zPiMRMo/yMgJD6fRuvWQ+pRSrDuo xoM+vcLUEKUFZhIMXeT/vsisQFfx8Mn4M90t1K+kVyzFW3KjssAVjS7crWHAHPnp 5UV0HldbBRKz/fuFLZeoZ40R818sYWp9Y5pJPyXcoaCf/AUJbkFsNOX2GXrA5y2E wZVi/xmH1whwdxWpvvcw7ks13z7NNHL7vTY+4ahL+/5LIQG6Rwj/jWVkI6OTE6N0 seIKpZAcEhOuau702gnlJEqMzzGpanfo+jMNReV+km4FIIvxGWiWUOflsFTax5H9 TJWXcTxLq3LI+MEqQ+jfWcaUy/bAGoMMDm0WY+A6Oi7U035hzaJibJBhMxsqLgLV oPPq4sKp9aS6uEpQ5iaXnbFhlWBn6H1tKihoJ+vyDNYxlr8PUTtJ/UX+10LIBMwC u6CN2rH96TiYh1wAJbxV =r4Fp -----END PGP SIGNATURE----- --Apple-Mail=_4C09DEC7-5515-4388-A003-7A4BF02D5D41-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 19:54:19 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DE5046E; Mon, 7 Jul 2014 19:54:19 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CDF122A93; Mon, 7 Jul 2014 19:54:18 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::a4f8:2408:fcb7:b36e] (unknown [IPv6:2001:7b8:3a7:0:a4f8:2408:fcb7:b36e]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id A26375C44; Mon, 7 Jul 2014 21:54:08 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_8418DFC3-A1B1-42F8-9EF2-9C20C1E31919"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: <9CE37432-1028-44FE-B0B7-224625EB9AB3@bsdimp.com> Date: Mon, 7 Jul 2014 21:54:00 +0200 Message-Id: <45D67C04-B2E0-4287-AB1E-887E6D49DF2E@FreeBSD.org> References: <1404688077.1059.115.camel@bruno> <1404756848.1105.1.camel@bruno> <9CE37432-1028-44FE-B0B7-224625EB9AB3@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1878.6) Cc: Benjamin Kaduk , sbruno@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 19:54:19 -0000 --Apple-Mail=_8418DFC3-A1B1-42F8-9EF2-9C20C1E31919 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 07 Jul 2014, at 21:28, Warner Losh wrote: ... > Just for grins, I did the following on a stock system (without your = change to src.opts.mk from Bapt) and not as root: >=20 > % mkdir $HOME/D > % make DESTDIR=3D$HOME/D XDEV=3Dmips XDEV_ARCH=3Dmips xdev = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt > %=20 >=20 > and it hit the stdc++ bits not being defined. Please see above with = the "clang folks didn=92t properly integrate=94 Or possibly some other = reason. Try adding WITH_GNUCXX=3Dt, which should build and install the correct = libstdc++ headers to ${WORLDTMP} during the early stages. I recently = mailed you and scottl@ a patch for doing just that, if gcc is the = bootstrap compiler... :) -Dimitry --Apple-Mail=_8418DFC3-A1B1-42F8-9EF2-9C20C1E31919 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlO6+twACgkQsF6jCi4glqMemwCeIoQwu2Ggx5ek09Lve8ZdWg1S VkkAn2WdekdbNIZ/PGJ5aTxipbS5qHJ0 =B3iM -----END PGP SIGNATURE----- --Apple-Mail=_8418DFC3-A1B1-42F8-9EF2-9C20C1E31919-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 20:32:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C596D434 for ; Mon, 7 Jul 2014 20:32:02 +0000 (UTC) Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CF602E54 for ; Mon, 7 Jul 2014 20:32:02 +0000 (UTC) Received: by mail-pd0-f172.google.com with SMTP id w10so5925995pde.17 for ; Mon, 07 Jul 2014 13:31:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=QllaIespxxWt+xqNCPjKkh+7wUqWovaWhzxqiMOdhpI=; b=EfgyVozyn+iVjWj6bW1ujKc/jckSl3clRGxaQg2sDlmPAXXCra1i0SlSbK5Glg3FpP B9DsnXAjXEM0HkpI2HPQm08OYB2EHNzhkWwS8ZdN8qyhGgLjeugi6JWpOxJ4pd4AoSWY AgFb+ZsQZJ9XS3ln2VEHi0zrBtHZOQdvy+n3TnxVFN/9UFNTcO15GyyNXPxYTJL+AEPf zw5KWRdCc3iQMbG3BApm9JtjifUdgo99PYgCE4RFQkz0gqxJou8H67cn1HQQreMYGrfH A3Jd0PeafUS5KRdQfwAm/r04aFS6HqXi2yAkpeuCbGPdvBZpuRn8H0+egTtJb5gIqG4c lKHw== X-Gm-Message-State: ALoCoQkW5xC63v1rD0QA7e+AuDPlutc9U0NoJ1hYdOi3Lw0Al23q9BtxUfJJvzwavQ0GJ9ibBsL8 X-Received: by 10.68.194.134 with SMTP id hw6mr30911802pbc.49.1404765115751; Mon, 07 Jul 2014 13:31:55 -0700 (PDT) Received: from [10.64.26.2] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id pz10sm53546052pbb.33.2014.07.07.13.31.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 13:31:55 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_095BECE5-D660-4FFC-BBD4-F437D355FFEA"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <45D67C04-B2E0-4287-AB1E-887E6D49DF2E@FreeBSD.org> Date: Mon, 7 Jul 2014 14:32:04 -0600 Message-Id: <5630BA56-B731-4C58-9B1C-4A04B37E1080@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404756848.1105.1.camel@bruno> <9CE37432-1028-44FE-B0B7-224625EB9AB3@bsdimp.com> <45D67C04-B2E0-4287-AB1E-887E6D49DF2E@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1878.6) Cc: Benjamin Kaduk , sbruno@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 20:32:02 -0000 --Apple-Mail=_095BECE5-D660-4FFC-BBD4-F437D355FFEA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 1:54 PM, Dimitry Andric wrote: > On 07 Jul 2014, at 21:28, Warner Losh wrote: > ... >> Just for grins, I did the following on a stock system (without your = change to src.opts.mk from Bapt) and not as root: >>=20 >> % mkdir $HOME/D >> % make DESTDIR=3D$HOME/D XDEV=3Dmips XDEV_ARCH=3Dmips xdev = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt >> %=20 >>=20 >> and it hit the stdc++ bits not being defined. Please see above with = the "clang folks didn=92t properly integrate=94 Or possibly some other = reason. >=20 > Try adding WITH_GNUCXX=3Dt, which should build and install the correct = libstdc++ headers to ${WORLDTMP} during the early stages. I recently = mailed you and scottl@ a patch for doing just that, if gcc is the = bootstrap compiler... :) Didn=92t seem to help. I thought of that after I sent my mail=85 The = right things are being built, they appear to be installed, but the paths = used don=92t seem to use the installed path :( Warner --Apple-Mail=_095BECE5-D660-4FFC-BBD4-F437D355FFEA Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTuwPEAAoJEGwc0Sh9sBEA8acP/jauDqsrC/J5s5Hy5BRc1Oky TMeBzvO6NH3Yazv5GWaH9sk2+pz1R/fhPY2uBiPyp06oHk5a4QoKzSrsHcH6JvIa 8YHuU3gBHW8mR/usx1V2Nzv55SC5l+iymO/Nji0QBTHtWBSpgzkPAycU3y3vcvG8 kdChGaZV4XU/bJVOClwtQvZEO1LPwNs1b81i7WH+it+MumnXs7yj8kTnDzAJmppT CbMnvYTSkCuZ4CiEyp+WgtEXnIP9jfporfP+AN+drgJEDJNf8l9Gs3jI+WX4L0E2 fqB4vHvlBA0MvtROFNyhkLvehQIp4xcisGA4XpmCwXydrGqpGn/twCga80EOMVAe qsu/COKCab9Iaunk8Tm49BSOSPLMhgYH7Hqpa0wpFuxOhggly3oVbPcZlEO/Od3X 73fmcJ5jI1D/0+89vpMduvUvZWOdZBvL8S94MJVZQlX69g+xJ/C4UzJJ6ndvnXf8 vziR8549/auY+GdGDNqMgTfWkLNtH22TKYiqKP2h3SkBunGOjfKQ+Mdc52GWH3O3 7ViMIqz9xF1Sjzhq2xFu8RdU132LGUZKyD6IjLz39GnZATZMgAi1ozcjIXVe83HS EoONA7CswA7jpoo/8291SjHfiqpyqRqmJ4HX/8BrdFRro/qqsbrXDADrSz1YKLgi dBvAo8Ep+OS5BNSFOi3H =MF6G -----END PGP SIGNATURE----- --Apple-Mail=_095BECE5-D660-4FFC-BBD4-F437D355FFEA-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 20:51:36 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FBE2C5A; Mon, 7 Jul 2014 20:51:36 +0000 (UTC) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43A742063; Mon, 7 Jul 2014 20:51:35 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1X4Fst-0002TX-0q; Mon, 07 Jul 2014 20:51:35 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s67KpXtF001473; Mon, 7 Jul 2014 14:51:33 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/4ilmMvxDznmBm6zX3Zgah Subject: Re: Total confusion over toolchain/xdev behavior From: Ian Lepore To: sbruno@FreeBSD.org In-Reply-To: <1404688077.1059.115.camel@bruno> References: <1404688077.1059.115.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Mon, 07 Jul 2014 14:51:32 -0600 Message-ID: <1404766292.65432.43.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 20:51:36 -0000 On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: > Objective: install an xcompile toolchain into a jail for use by > poudriere during arm/mips/sparc/power ports pkgs builds. The build > should be possible from a non-root user. > > As far as I can tell, the xdev target is completely busted for non-clang > arch's right now as it tries to build clang no matter what I do. Its > missing some pretty key documentation to making it work correctly, so a > lot of my attempts have been "guess and check" with verbose make. > > ----------------------------------------------------------------------- > Attempt #1: > I have been trying non-root xdev builds: > MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips > -- dies because it tries to compile CLANG. > ----------------------------------------------------------------------- > > Attempt #2: > Apply a hack from Baptiste that isn't quite right, but at least gets > farther, note the missing variable causing "//usr/mips-freebsd" > http://people.freebsd.org/~sbruno/src.ops.mk.diff > > ===> gnu/usr.bin/cc/gcov (all) > mtree populating //usr/mips-freebsd > mkdir: //usr/mips-freebsd: Permission denied > *** Error code 1 > ----------------------------------------------------------------------- > > Attempt #3: Add XDTP > MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips > XDTP=/var/tmp/mips_cc > > Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's > a bit farther but compile failure in locating critical include files. > > ===> gnu/lib/libstdc++ (obj,depend,all,install) > In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or > directory > ----------------------------------------------------------------------- > > Attempt #4: Add the additional XDDESTDIR > MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips > XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst > -- Same results as attempt #3 > ----------------------------------------------------------------------- > > Even attempting to do stuff for *clang* enabled architectures bails > because its not respecting prefixes: > MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6 > -- bails because it tries to: > ===> usr.bin/clang/tblgen (all) > mtree populating //usr/armv6-freebsd > mtree: etc/ntp: Permission denied > _xi-cross-tools > ===> xdev gnu/usr.bin/binutils (install) > ===> gnu/usr.bin/binutils/libiberty (install) > ===> gnu/usr.bin/binutils/libbfd (install) > ===> gnu/usr.bin/binutils/libopcodes (install) > ===> gnu/usr.bin/binutils/libbinutils (install) > ===> gnu/usr.bin/binutils/addr2line (install) > ===> gnu/usr.bin/binutils/as (install) > ===> gnu/usr.bin/binutils/ld (install) > install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > *** Error code 71 > > ----------------------------------------------------------------------- > Adding XDTP and XDDESTDIR results in a little more progress but obvious > failures to attempt and install things directly into my host system: > > MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6 > XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc > ===> secure/lib/libssh (install) > ===> usr.bin/lex/lib (obj,depend,all,install) > mkdir: ../../../../usr: Permission denied > *** Error code 1 > > Stop. > make[1]: stopped in /home/sbruno/bsd/fbsd_head > *** Error code 1 > > Stop. > make: stopped in /home/sbruno/bsd/fbsd_head It looks to me like the permission part of the problem is being caused by a lack of DESTDIR=. Without that, it's trying to install to /usr and you don't have permission for that. Maybe the confusion is because the xdev target inherently builds-and-installs, unlike most other targets that separate those two actions. -- Ian From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 23:27:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 292AC4E3 for ; Mon, 7 Jul 2014 23:27:18 +0000 (UTC) Received: from mail-pd0-f179.google.com (mail-pd0-f179.google.com [209.85.192.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E96F12CC2 for ; Mon, 7 Jul 2014 23:27:17 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so6138375pde.10 for ; Mon, 07 Jul 2014 16:27:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=ppjZPgBL4XU2CsXsG4v06NXxWl123TmKfBDK/AJ0Q50=; b=N+qbBDbyV0jbF0xQoQSq44gMx2SHYji4zpvK0W3estmr66u24l1RqKW7zxo+Qao/2r tzdhjSxqMDWG4eyBnMGn0rrVd7DOuMGfcfLFc8ndosZ3VlTvw3Qw9Dg7NGZkLVy3sazz CnoVnqOMq0bdK0cnkLnjDa1UxvSj2rXQhI1D1gtpuLKMmiyIAUE8kifPxbhjV/l78U8e FPXZfZ/EkEFnEYS+jEhGHKps44YxeWAW6yGjqpnly/QSCNYxun9Ybv7F1+77QtqELnEf NsP2hivS4esvInG3RD3I1g5HzV19hcTYzKp2ivEKsGGufm2KjCcZEX1JYi55alGmzNyo bJCw== X-Gm-Message-State: ALoCoQkJGMTZB98X57t2t0qAuwvz0Wpo2agXmuWRF/pDbM+nvmUuZXHIc/s994zjJM9qRCd7V+FC X-Received: by 10.70.92.49 with SMTP id cj17mr1518271pdb.53.1404775636780; Mon, 07 Jul 2014 16:27:16 -0700 (PDT) Received: from [10.64.26.2] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id mz8sm21793054pdb.62.2014.07.07.16.27.15 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 16:27:16 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_3B3E7ECB-F0D7-46C6-B81D-F7FF65D1BE03"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404766292.65432.43.camel@revolution.hippie.lan> Date: Mon, 7 Jul 2014 17:27:25 -0600 Message-Id: <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1878.6) Cc: sbruno@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 23:27:18 -0000 --Apple-Mail=_3B3E7ECB-F0D7-46C6-B81D-F7FF65D1BE03 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: > On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: >> Objective: install an xcompile toolchain into a jail for use by >> poudriere during arm/mips/sparc/power ports pkgs builds. The build >> should be possible from a non-root user. >>=20 >> As far as I can tell, the xdev target is completely busted for = non-clang >> arch's right now as it tries to build clang no matter what I do. Its >> missing some pretty key documentation to making it work correctly, so = a >> lot of my attempts have been "guess and check" with verbose make. >>=20 >> = ----------------------------------------------------------------------- >> Attempt #1: >> I have been trying non-root xdev builds: >> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips = XDEV_ARCH=3Dmips >> -- dies because it tries to compile CLANG. >> = ----------------------------------------------------------------------- >>=20 >> Attempt #2: >> Apply a hack from Baptiste that isn't quite right, but at least gets >> farther, note the missing variable causing "//usr/mips-freebsd" >> http://people.freebsd.org/~sbruno/src.ops.mk.diff >>=20 >> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) >> mtree populating //usr/mips-freebsd >> mkdir: //usr/mips-freebsd: Permission denied >> *** Error code 1 >> = ----------------------------------------------------------------------- >>=20 >> Attempt #3: Add XDTP >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips >> XDTP=3D/var/tmp/mips_cc >>=20 >> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch applied, = get's >> a bit farther but compile failure in locating critical include files. >>=20 >> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) >> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc >> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: >> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc >> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file = or >> directory >> = ----------------------------------------------------------------------- >>=20 >> Attempt #4: Add the additional XDDESTDIR >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips >> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst >> -- Same results as attempt #3 >> = ----------------------------------------------------------------------- >>=20 >> Even attempting to do stuff for *clang* enabled architectures bails >> because its not respecting prefixes: >> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm = XDEV_ARCH=3Darmv6 >> -- bails because it tries to: >> =3D=3D=3D> usr.bin/clang/tblgen (all) >> mtree populating //usr/armv6-freebsd >> mtree: etc/ntp: Permission denied >> _xi-cross-tools >> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) >> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) >> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) >> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) >> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) >> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) >> =3D=3D=3D> gnu/usr.bin/binutils/as (install) >> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) >> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied >> *** Error code 71 >>=20 >> = ----------------------------------------------------------------------- >> Adding XDTP and XDDESTDIR results in a little more progress but = obvious >> failures to attempt and install things directly into my host system: >>=20 >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Darmv6 >> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc >> =3D=3D=3D> secure/lib/libssh (install) >> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) >> mkdir: ../../../../usr: Permission denied >> *** Error code 1 >>=20 >> Stop. >> make[1]: stopped in /home/sbruno/bsd/fbsd_head >> *** Error code 1 >>=20 >> Stop. >> make: stopped in /home/sbruno/bsd/fbsd_head >=20 > It looks to me like the permission part of the problem is being caused > by a lack of DESTDIR=3D. Without that, it's trying to install to /usr = and > you don't have permission for that. Maybe the confusion is because = the > xdev target inherently builds-and-installs, unlike most other targets > that separate those two actions. OK. After some detective work, it looks like libstdc++ needs to be done = before libsupc++ is done. I=92ve added this dependency in r268377 and = was able to do a full xdev build with a clean obj dir: rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd mkdir $HOME/F make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt -j 20 Sean, can you see if this works for you now? And sorry about the = cumbersome options. Those are in line to get fixed, but I wanna fix the = build-in-tree issues first (which, alas, are harder than you=92d think). = I suspect that bapt=92s src.opts.mk diffs are likely good candidates to = be used committed too, but the above works w/o it. Also, I just built = the binaries, I didn=92t see if they worked. Warner --Apple-Mail=_3B3E7ECB-F0D7-46C6-B81D-F7FF65D1BE03 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTuyzdAAoJEGwc0Sh9sBEArLsP/id8fth4V1UeN+54B2oIDDVf bttt5/22zrONP2Jk5NZXPxQvyVIFnNs8d24u0hdhApqjl++hjF5oXfao7hv4qysh Pcwik+x9lmoORQE8rGRpZAJu3VxdDH2mEsokJVLq0H6cV0LT46K1mgqQloe91ok4 6JrjJKNMuuv9X+2nY8fqglRFAVOV8Jf82i0gqRBrU0blzH07prJZKpIvrjsKpSgy x+UYzFfP2vlq4NcQlXZeYIKmVW4JHmKQA8hkTobO1KJMrvaY+/XK1s+dn4nCOIjP VIXK01/cQmSII/sBDtobDwa4LHttMY9JHLxGbmF5pnbisRurwErYXJycMbfU89Gn AfVYmmCH0po34lxFotMF6MLcMgJfnUfn9FABUr9jGk01Xu8q2f/OiHJ64LaZlE3Y Rmcb5twJr3u9hhrC1fb4ILDL4vMP5C6f2l+Su7Rx6SFhVYmdRqbsx4rN6A6On2+U hVW4aHpTHQItjUvi5t8p2ya30A6D89qBVpgjpsIzZyrsQOxjZxv7HpxZSUvxOlpr EXSkNKDOT7/RV2C/c4NHjdvdQuu8xwCx0eewM0izdThSp1wgDSC15+43mR+BWPaG tehBPq3gtFJClP1Z7XuvqnFVXTZOpULtd4ST8I3ZycB1EHW7N6sVc8Uv2Klwt88A KORvWsnAR1Yt1IC1XIsu =LExT -----END PGP SIGNATURE----- --Apple-Mail=_3B3E7ECB-F0D7-46C6-B81D-F7FF65D1BE03-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 7 23:52:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 211227D7; Mon, 7 Jul 2014 23:52:43 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A8152EA6; Mon, 7 Jul 2014 23:52:42 +0000 (UTC) Received: by mail-wg0-f44.google.com with SMTP id k14so5128891wgh.27 for ; Mon, 07 Jul 2014 16:52:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=o14OXk5SFzKUqKZ9/2PaYZHX1FPAFRXCc6pLX/tCOp0=; b=gXrlPxaIpvIBLMK2pxRT4Pe1e/ezP5f+NrGNe7q66vssxX0NEliCLH3cutRb783VoW yVVYG9R1REFuhJwyj+OtOFtK+fEwJh0Na4VWMV3bIBif/XRpfNylBec2yqk81Z03oVy0 gKKT95+Xra1s8AQzndiSx3tUP9e0fiTL0e8x8Ppq5gq2Jh4/9WTpepQuqKgSl6X25aFs RraIwrfTI0huc691ogRzT/ylzuZXOthoniLl4UV5rpyldHFg4AjbOzZRwF3EW0D/n0NI Y8uCP3xMOffUuNxkxrU8qXI1c+IXMQt6168MxAxj7BC2q6I2w2B31cFgtotAmvIMDpOk JFTQ== X-Received: by 10.194.157.195 with SMTP id wo3mr23466wjb.130.1404777160508; Mon, 07 Jul 2014 16:52:40 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id lk7sm92500488wjb.24.2014.07.07.16.52.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Jul 2014 16:52:39 -0700 (PDT) Sender: Baptiste Daroussin Date: Tue, 8 Jul 2014 01:52:37 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: Total confusion over toolchain/xdev behavior Message-ID: <20140707235237.GG97203@ivaldir.etoilebsd.net> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CEUtFxTsmBsHRLs3" Content-Disposition: inline In-Reply-To: <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2014 23:52:43 -0000 --CEUtFxTsmBsHRLs3 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: >=20 > On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: >=20 > > On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: > >> Objective: install an xcompile toolchain into a jail for use by > >> poudriere during arm/mips/sparc/power ports pkgs builds. The build > >> should be possible from a non-root user. > >>=20 > >> As far as I can tell, the xdev target is completely busted for non-cla= ng > >> arch's right now as it tries to build clang no matter what I do. Its > >> missing some pretty key documentation to making it work correctly, so a > >> lot of my attempts have been "guess and check" with verbose make. > >>=20 > >> ----------------------------------------------------------------------- > >> Attempt #1: > >> I have been trying non-root xdev builds: > >> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips XDEV_ARCH=3Dm= ips > >> -- dies because it tries to compile CLANG. > >> ----------------------------------------------------------------------- > >>=20 > >> Attempt #2: > >> Apply a hack from Baptiste that isn't quite right, but at least gets > >> farther, note the missing variable causing "//usr/mips-freebsd" > >> http://people.freebsd.org/~sbruno/src.ops.mk.diff > >>=20 > >> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) > >> mtree populating //usr/mips-freebsd > >> mkdir: //usr/mips-freebsd: Permission denied > >> *** Error code 1 > >> ----------------------------------------------------------------------- > >>=20 > >> Attempt #3: Add XDTP > >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips > >> XDTP=3D/var/tmp/mips_cc > >>=20 > >> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch applied, g= et's > >> a bit farther but compile failure in locating critical include files. > >>=20 > >> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) > >> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc > >> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > >> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > >> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file = or > >> directory > >> ----------------------------------------------------------------------- > >>=20 > >> Attempt #4: Add the additional XDDESTDIR > >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips > >> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst > >> -- Same results as attempt #3 > >> ----------------------------------------------------------------------- > >>=20 > >> Even attempting to do stuff for *clang* enabled architectures bails > >> because its not respecting prefixes: > >> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm XDEV_ARCH=3Da= rmv6 > >> -- bails because it tries to: > >> =3D=3D=3D> usr.bin/clang/tblgen (all) > >> mtree populating //usr/armv6-freebsd > >> mtree: etc/ntp: Permission denied > >> _xi-cross-tools > >> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/as (install) > >> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) > >> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > >> *** Error code 71 > >>=20 > >> ----------------------------------------------------------------------- > >> Adding XDTP and XDDESTDIR results in a little more progress but obvious > >> failures to attempt and install things directly into my host system: > >>=20 > >> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Darmv6 > >> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc > >> =3D=3D=3D> secure/lib/libssh (install) > >> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > >> mkdir: ../../../../usr: Permission denied > >> *** Error code 1 > >>=20 > >> Stop. > >> make[1]: stopped in /home/sbruno/bsd/fbsd_head > >> *** Error code 1 > >>=20 > >> Stop. > >> make: stopped in /home/sbruno/bsd/fbsd_head > >=20 > > It looks to me like the permission part of the problem is being caused > > by a lack of DESTDIR=3D. Without that, it's trying to install to /usr = and > > you don't have permission for that. Maybe the confusion is because the > > xdev target inherently builds-and-installs, unlike most other targets > > that separate those two actions. >=20 > OK. After some detective work, it looks like libstdc++ needs to be done b= efore libsupc++ is done. I=E2=80=99ve added this dependency in r268377 and = was able to do a full xdev build with a clean obj dir: >=20 > rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd > mkdir $HOME/F > make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CLANG= =3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt WITH_GCC_BOOTSTRAP=3Dt WITH_G= NUCXX=3Dt -j 20 We can avoid most of the above by using a patch like the following: http://people.freebsd.org/~bapt/Makefile.inc1.diff Extending the same thing xi-cross-tools and xb-cross-tools (expect the WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho should.) regards, Bapt --CEUtFxTsmBsHRLs3 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlO7MsUACgkQ8kTtMUmk6EyJ8wCff7XMATfK4j4N77smru0Nl5D8 geIAoIARwHUNcwecZemz27EUiWIyH/rN =In69 -----END PGP SIGNATURE----- --CEUtFxTsmBsHRLs3-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 01:29:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E687EF41 for ; Tue, 8 Jul 2014 01:29:17 +0000 (UTC) Received: from mail-pd0-f178.google.com (mail-pd0-f178.google.com [209.85.192.178]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B37FC2776 for ; Tue, 8 Jul 2014 01:29:17 +0000 (UTC) Received: by mail-pd0-f178.google.com with SMTP id r10so6264411pdi.37 for ; Mon, 07 Jul 2014 18:29:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=nO77q6ENZHZ0jKsr2uRpgKL0yrO+QIm5Ezw0e8/ZiYA=; b=FlOaG5sIhZ5Ui93eW7rTHQTw9Jurmy/dIZqL0+xkwm4BFVyOs8M9SpY/jlbld0deGN jjX90A0aqcCxVbZOZvBIR7VWbluY6qcbcm6ZW8EsF5hY54Fc8LK4SnrguqpSX8lkky4V BMrRDZdyxUzhWTZ9GAF06x/g3pVUCWNP/15j17JzowIq8rQL6/Q34vMV5Kidqcr7W9Fk NNP7rRMhTBVU1cP8xPxDZk5yebvw4ZoL2c237TWWExNHDe5oOfAA66uAZtsWwHqTh9uT hfQQwErdBxKFJCNIyRyA5B5wArj892V8+cULWLTo+yn2Jh/dCjVY4UnF7bzORnfuuSfZ CR0A== X-Gm-Message-State: ALoCoQnXkHTBtumkeSmBptlH+TjhTY5qf7j2ZY1GIcySZSPYLOjf/LDgROSg2tyBegrhs6RZLouf X-Received: by 10.68.129.99 with SMTP id nv3mr622564pbb.128.1404782951124; Mon, 07 Jul 2014 18:29:11 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id oc3sm22043585pdb.45.2014.07.07.18.29.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 18:29:10 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_3B88BDAA-FCB4-4A1D-A678-B33E78EBBB59"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <20140707235237.GG97203@ivaldir.etoilebsd.net> Date: Mon, 7 Jul 2014 19:29:01 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1878.6) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 01:29:18 -0000 --Apple-Mail=_3B88BDAA-FCB4-4A1D-A678-B33E78EBBB59 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin wrote: > On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: >>=20 >> On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: >>=20 >>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: >>>> Objective: install an xcompile toolchain into a jail for use by >>>> poudriere during arm/mips/sparc/power ports pkgs builds. The build >>>> should be possible from a non-root user. >>>>=20 >>>> As far as I can tell, the xdev target is completely busted for = non-clang >>>> arch's right now as it tries to build clang no matter what I do. = Its >>>> missing some pretty key documentation to making it work correctly, = so a >>>> lot of my attempts have been "guess and check" with verbose make. >>>>=20 >>>> = ----------------------------------------------------------------------- >>>> Attempt #1: >>>> I have been trying non-root xdev builds: >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips = XDEV_ARCH=3Dmips >>>> -- dies because it tries to compile CLANG. >>>> = ----------------------------------------------------------------------- >>>>=20 >>>> Attempt #2: >>>> Apply a hack from Baptiste that isn't quite right, but at least = gets >>>> farther, note the missing variable causing "//usr/mips-freebsd" >>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff >>>>=20 >>>> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) >>>> mtree populating //usr/mips-freebsd >>>> mkdir: //usr/mips-freebsd: Permission denied >>>> *** Error code 1 >>>> = ----------------------------------------------------------------------- >>>>=20 >>>> Attempt #3: Add XDTP >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips= >>>> XDTP=3D/var/tmp/mips_cc >>>>=20 >>>> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch = applied, get's >>>> a bit farther but compile failure in locating critical include = files. >>>>=20 >>>> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) >>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc >>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: >>>> = /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc >>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such = file or >>>> directory >>>> = ----------------------------------------------------------------------- >>>>=20 >>>> Attempt #4: Add the additional XDDESTDIR >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips= >>>> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst >>>> -- Same results as attempt #3 >>>> = ----------------------------------------------------------------------- >>>>=20 >>>> Even attempting to do stuff for *clang* enabled architectures bails >>>> because its not respecting prefixes: >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm = XDEV_ARCH=3Darmv6 >>>> -- bails because it tries to: >>>> =3D=3D=3D> usr.bin/clang/tblgen (all) >>>> mtree populating //usr/armv6-freebsd >>>> mtree: etc/ntp: Permission denied >>>> _xi-cross-tools >>>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) >>>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) >>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied >>>> *** Error code 71 >>>>=20 >>>> = ----------------------------------------------------------------------- >>>> Adding XDTP and XDDESTDIR results in a little more progress but = obvious >>>> failures to attempt and install things directly into my host = system: >>>>=20 >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Darmv6= >>>> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc >>>> =3D=3D=3D> secure/lib/libssh (install) >>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) >>>> mkdir: ../../../../usr: Permission denied >>>> *** Error code 1 >>>>=20 >>>> Stop. >>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head >>>> *** Error code 1 >>>>=20 >>>> Stop. >>>> make: stopped in /home/sbruno/bsd/fbsd_head >>>=20 >>> It looks to me like the permission part of the problem is being = caused >>> by a lack of DESTDIR=3D. Without that, it's trying to install to = /usr and >>> you don't have permission for that. Maybe the confusion is because = the >>> xdev target inherently builds-and-installs, unlike most other = targets >>> that separate those two actions. >>=20 >> OK. After some detective work, it looks like libstdc++ needs to be = done before libsupc++ is done. I=92ve added this dependency in r268377 = and was able to do a full xdev build with a clean obj dir: >>=20 >> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd >> mkdir $HOME/F >> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt -j 20 >=20 > We can avoid most of the above by using a patch like the following: > http://people.freebsd.org/~bapt/Makefile.inc1.diff > Extending the same thing xi-cross-tools and xb-cross-tools (expect the > WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho = should.) The patch looks good to my eye. Did you want me to expand it, or do you = want to do the honors? About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd duck, = and that=92s likely the problem that should be fixed in a different way. It is really = an internal variable that should be set based on the actual compiler type (possibly = with an override for the odd-duck pair of clang and libstdc++ which may not be = worth supporting). It is telling us we=92re doing something horribly wrong and = we should listen to that rather than add another compiler-related kludge to the build = system. I=92ll work on that bit. Also an aside: The horribly long command line was to try to get to the = bottom of the breakage, not to promote it as the proper way of doing things. Warner --Apple-Mail=_3B88BDAA-FCB4-4A1D-A678-B33E78EBBB59 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTu0ldAAoJEGwc0Sh9sBEArKUQAMJiIvxp2FgiVk/StHUEUjnx HFXgtSfh3RwTyhaEsnRCz6iILGHNx893Or/23Yw5D7bL43rsDCuqEV3wZIZT5D5a rXj+wudptfOhfpqru4bdtmSmtAXxeMkbpG+xoQilYnsU8r5TimjGuIHia/ZRS7ub +DxW0dMZzbFaKjXDhgYHKSZks6sRUyP5axDYTqz0oRG1g4yBBLSGyhZvdyHFxADA b3T5fVXR/3V3Yv9oUKO8pFb2i0iUok/3IZP8D0vPu70EuWvtpbQBe2cz3ZqxOD/k vlJsjC/IVqDg2xglATp3RQfbo9GXmdCu65CAJobTOrwsOlsVV9BiQvWuv5xmxIUO yxoPrUS7OfQJDsh0+6qkSe+jVKSVr+IwXd7qn/qlE/PEi5QNEmggS9grmPXgA0+q oLnDw6y4x03jKFC29JC/g7gqrhdhQ+G0swmZKUB1YTJrAerHL37kFa97/CwoqESl Aw2/FVazZKh0wmRtR2bv2Crl8BB1NDxgBhTtkhhayVAghQyusQZcwx4KZynV6+nY Hd/opY3wgOGgzXzBeDHYnui20CzTky1/1zh3J9wBUOV0xmBW2wEEWwEzqsl2IKFy xFIp8A+MOmsulte3A+ZxDDByqAOgAiPs0a7tQSAwoaqHzmfaApvSOrxodI0eHSMx NnxZVRoV8Gbu+UJ9AcUq =RhbQ -----END PGP SIGNATURE----- --Apple-Mail=_3B88BDAA-FCB4-4A1D-A678-B33E78EBBB59-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 01:57:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63A14C7B for ; Tue, 8 Jul 2014 01:57:04 +0000 (UTC) Received: from mail-pa0-f43.google.com (mail-pa0-f43.google.com [209.85.220.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BCD42AF6 for ; Tue, 8 Jul 2014 01:57:03 +0000 (UTC) Received: by mail-pa0-f43.google.com with SMTP id lf10so6432136pab.16 for ; Mon, 07 Jul 2014 18:57:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=uy5Tc5F690aLq8doOzd2dbx6nwBCbr8iyvJpsTdCsmc=; b=GysDUWInb4p+7eIhbZb/UYTlOz/Dp2Vu/Nsxfdfadgbqlm9XX8JSDvuYDQkHSyaFDb nwPbxPpyV+7X7ZIyxiKds/2A4v3/m3GrBTrsq+ydZ199e7PL1Zb2aXfJzJqjU1wdtuQh IXAcxd70fAjSJg2liq/FHT3MMALJdZ9IIDXNg2XZP115TXBv6Tmspa0lVJ6aFm/LNKOw S+XC356OwFw2dQHUsakJrdcDQ0R8xqy7RHd+AEUFB86ucPSbYgB61Q33nDM0ViscTCYH xIjrHvA9/OSobQx9fSaDZOzWsTNbl50E3NcynjnRm1v06FTanRkyHET5XX7dIZjNldsX A5Bw== X-Gm-Message-State: ALoCoQmbiQ7rt/G2C2SNnPrEz+AeALix3RNfWyB8BJCApz+EOJiq6ENk8rlmuF52r8nvp+8HZTln X-Received: by 10.68.103.98 with SMTP id fv2mr32289236pbb.18.1404784623122; Mon, 07 Jul 2014 18:57:03 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id fs3sm65000271pac.44.2014.07.07.18.57.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 18:57:01 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_9DC84E5A-F6ED-458A-978E-453B3BE4FDA2"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: Date: Mon, 7 Jul 2014 19:56:58 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1878.6) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 01:57:04 -0000 --Apple-Mail=_9DC84E5A-F6ED-458A-978E-453B3BE4FDA2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >=20 > About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd duck, = and that=92s > likely the problem that should be fixed in a different way. It is = really an internal > variable that should be set based on the actual compiler type = (possibly with an > override for the odd-duck pair of clang and libstdc++ which may not be = worth > supporting). It is telling us we=92re doing something horribly wrong = and we should listen > to that rather than add another compiler-related kludge to the build = system. I=92ll work > on that bit. Perhaps http://people.freesbd.org/~imp/patch-queue/86gnucxx might be the best way to cope=85 Comments? Warner --Apple-Mail=_9DC84E5A-F6ED-458A-978E-453B3BE4FDA2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTu0/qAAoJEGwc0Sh9sBEA8h8QAKoAKRkn86PJ+OMPaeFu/f/n +CEgg980KrnefDKfEY6N3tMDCrftcQYGn+VhlY0XtH7LQrKC4Da9Ks65Ok0NPpgK fErIXM9ViiBaKeH0KrG01VqNVWyny4ZHYRYblYuqLXLQgG0E50SINCLApA9iN7VU 8LtwCJ63eLZHvlrV5OiVyQcr+pCWROHidgQbyp6pBTvqNnUPAe7TEccs5vughJnd rysjqHEK9m26zwANWuDzsHnyBhEY5ek5Dz7R57ATnepgukgvPP/ir05lhWbItjRS 2Rcvs5aL1Qho4TRqailWX+qI5fJz9yUN33t6MIM3sk8PZuTotqv6Eb+vfEDm6krH 3Km+8DBko3O5r9wzmvocw1+IS9+A7oTx7uIGDHKIRmz7BKme0SHhSTfNCIbW14ni 70/B9PIyy2FCfaUZr7OGu3r2gYPSgIoxUe898w86Ug7Q/WFxmtOKTE/+Yhn155J9 12+O7Ge3T7llM6voYb/tZUNS0fbk0ABLjsGtcjLWae0uvMJAXZ7G7ShjHJ8vd9Ue CuyQLe599cVwY90vN2Sfz+lMZsuOkVf7FCHc5AOGlVVVVwIYRlAluLWpQSChdaYH RHh+S5L7Prk63dN3PdyQU42j2APTRuj28hCpwvMdwqcpXB2SzEJ2EU9NwS29Wy8T q/aRudobDrOGXQUImSOZ =aoqs -----END PGP SIGNATURE----- --Apple-Mail=_9DC84E5A-F6ED-458A-978E-453B3BE4FDA2-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 02:59:29 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 530A4FED for ; Tue, 8 Jul 2014 02:59:29 +0000 (UTC) Received: from mail-pa0-f42.google.com (mail-pa0-f42.google.com [209.85.220.42]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 27C262FBC for ; Tue, 8 Jul 2014 02:59:28 +0000 (UTC) Received: by mail-pa0-f42.google.com with SMTP id lj1so6553231pab.29 for ; Mon, 07 Jul 2014 19:59:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id:references:to; bh=BBMcLrYemcAEO7nT6FknzMDKNht20xQSiMw0ad8P2OQ=; b=ICM4n9R+TY29dQj2h65uaF197zNwdixi5olbn0p+zc77B5pgKU4CFErTIdIFPvF9xl R9nZLpucE0+Jl4XHTY7xQA9S9T8OKc3UxXVbsWN9LM5XkMJHIGrOEWUoPhN/nXz4b+8J xgJYxnVJ1ycV1rIgyYSLL4RQTRq5TFBDU0uys84DE06YJproijTxCoAsom1hk701fv+J r01pt0lTn/7uH6Enw+WoISFaZw74vcJZ0lSakWK1X00kAFtNtCy7aL8b9gzWUHfGkZGw woK3+4M2QlOfLzNRCMEmftbhc/+sTfDms5Ha9sIaoO7H7H+y+kfVw0N0TFJ1YwbkrHPO R19Q== X-Gm-Message-State: ALoCoQmZ10PrmD+mISIWdz2GsxTYT8t9F9Cq62qqRvLFvepFtnZM9rvbJaBGuZxFGMblKRTr7Q9i X-Received: by 10.68.231.196 with SMTP id ti4mr32803894pbc.48.1404788361934; Mon, 07 Jul 2014 19:59:21 -0700 (PDT) Received: from [192.168.1.103] (c-24-6-220-224.hsd1.ca.comcast.net. [24.6.220.224]) by mx.google.com with ESMTPSA id vf9sm54033516pbc.94.2014.07.07.19.59.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 19:59:21 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Tim Kientzle In-Reply-To: <1404688077.1059.115.camel@bruno> Date: Mon, 7 Jul 2014 19:59:18 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> References: <1404688077.1059.115.camel@bruno> To: freebsd-arch@freebsd.org X-Mailer: Apple Mail (2.1878.2) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 02:59:29 -0000 On Jul 6, 2014, at 4:07 PM, Sean Bruno wrote: > Objective: install an xcompile toolchain into a jail for use by > poudriere during arm/mips/sparc/power ports pkgs builds. The build > should be possible from a non-root user. >=20 > As far as I can tell, the xdev target is completely busted for = non-clang > arch's right now as it tries to build clang no matter what I do. I think you can avoid that if you specify both: WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt > Its missing some pretty key documentation to making it work = correctly, so a > lot of my attempts have been "guess and check" with verbose make. Some of the complexity here might just be that =91xdev=92 is currently trying to do too much. I think I would find it simpler to use if it were a handful of separate targets: # Build and install a cross-GCC using the in-tree sources make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-gcc # Build and install a cross-clang using the in-tree sources make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-clang # Build and install cross-binutils using the in-tree sources make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-binutils # Build and install cross-libraries # Prerequisite: (xdev-gcc or xdev-clang) and xdev-binutils make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-libs # Have I missed anything? If I understand correctly, xdev-clang in many cases could install symlinks to the host clang without actually building anything. (If I=92m right about that, maybe we could consider installing those symlinks for every architecture as part of the default base system.) Tim From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 03:07:54 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 641083E3 for ; Tue, 8 Jul 2014 03:07:54 +0000 (UTC) Received: from mail-pd0-f182.google.com (mail-pd0-f182.google.com [209.85.192.182]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32CC52139 for ; Tue, 8 Jul 2014 03:07:53 +0000 (UTC) Received: by mail-pd0-f182.google.com with SMTP id y13so6327255pdi.41 for ; Mon, 07 Jul 2014 20:07:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=pqR3EWRI/C3/JnYGG6/4EPcEcS7wetxEnOaToxI9n1Y=; b=BvGHr5GUjamukyA/OZl/9dxhYK4D/zmitdsIeQwHHkSn3njp/hRIsMiHLp01Bx8+zn ZRxsDLM5LP7ZdD/CY4acrcCh0nNVmkFt9FVFMY7lo276zlB/sS41OhTnCJ7m4xUZQDTC Qj4wbHlEmmIXBY07iqNiLdwfCePG7cZLn0yXmU/ciM4Ph51xzOIGfgucP7AkIl+/Gj4r c76WCsJ+ZvKHY8vOHuasRylyE0IgbXRm02a2cTngxDWPqijC8FeLUVBt7IsV41cCYFIr ktR04tNlauqJWAlgMKqGQ+OWZY6CXPh2p9L/IN6RAu43GjIjLk/vLYNGHbjB6uLPB/ei zIFA== X-Gm-Message-State: ALoCoQnVCyQKnK3pXXdC+J86uI9yspq0IWKI2dqP1kInY9Z9EsAiiqqYadkoY97ZM9j6Pto/EE0Y X-Received: by 10.68.103.165 with SMTP id fx5mr6872756pbb.118.1404788872558; Mon, 07 Jul 2014 20:07:52 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id fx14sm22267858pdb.4.2014.07.07.20.07.51 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 20:07:51 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_DED1089C-C57B-47FE-88BE-4FE509337281"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> Date: Mon, 7 Jul 2014 21:07:50 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> To: Tim Kientzle X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 03:07:54 -0000 --Apple-Mail=_DED1089C-C57B-47FE-88BE-4FE509337281 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 8:59 PM, Tim Kientzle wrote: >=20 > On Jul 6, 2014, at 4:07 PM, Sean Bruno wrote: >=20 >> Objective: install an xcompile toolchain into a jail for use by >> poudriere during arm/mips/sparc/power ports pkgs builds. The build >> should be possible from a non-root user. >>=20 >> As far as I can tell, the xdev target is completely busted for = non-clang >> arch's right now as it tries to build clang no matter what I do. >=20 > I think you can avoid that if you specify both: > WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt >=20 >> Its missing some pretty key documentation to making it work = correctly, so a >> lot of my attempts have been "guess and check" with verbose make. >=20 > Some of the complexity here might just be that =91xdev=92 is currently > trying to do too much. I think I would find it simpler to use if > it were a handful of separate targets: >=20 > # Build and install a cross-GCC using the in-tree sources > make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-gcc >=20 > # Build and install a cross-clang using the in-tree sources > make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-clang >=20 > # Build and install cross-binutils using the in-tree sources > make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-binutils >=20 > # Build and install cross-libraries > # Prerequisite: (xdev-gcc or xdev-clang) and xdev-binutils > make XDEV=3Darm XDEV_ARCH=3Darmv6 xdev-libs Interesting ideas=85 Will have to ponder them again, but in the = earliest days I tried these and hit some turbulence=85 But as I said elsewhere on this = thread, that was maybe 10 years ago now and maybe things have changed :) > # Have I missed anything? >=20 > If I understand correctly, xdev-clang in many cases > could install symlinks to the host clang without > actually building anything. (If I=92m right about that, > maybe we could consider installing those symlinks > for every architecture as part of the default base system.) The driver program would need to know to use a different sys root (or = we=92d need to install things in the clang centric way) as well as which = architecture to use, which it might be able to guess based on the binary used to = invoke it (now that=92s a feature that has a checkered past, some good, some = down right awful). I=92ve been after the clang folks to justify bringing in = everything on every platform by providing an example cross build in the base, but so = far nothing has materialized... Warner --Apple-Mail=_DED1089C-C57B-47FE-88BE-4FE509337281 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTu2CGAAoJEGwc0Sh9sBEAUXsQALcUEOw9KLsRvBol/rhMLlpT 0d3TXLePRvRBSpYei0J0Gi1Zq6IrrElSR+gwJg6KJMzaUjEK4lJL0i2Ipt5QOpKA 7BLUe5Y3kuCguXKxVWmTMk6giNZ1eFE1E8uq/TJIIjDZ7eCIOhfsH3kXWiLgn2/m X0ezmiFuulqL+EBT72G1d8nLV2YwnomDoAaJusT2KHg1cObQWy+jaO3NHp0e/kS8 4fih1wAdJtoflyPg0cv0oXWUHsv3fJw3JfEu8tOWXLqoLvbMznjIKpNeMjt2b+jY 7eDhF/4WChqsQCaOGQi6dWnPz2BTYZM1SyIY/wb2amctiQbvpTfRnUADXN0Vfpob kfBTo8/WELy73VVWWzOWSxnq2ZLPxP3D616oz6QwaSX548hMR0vhxcXLLQN0IkgV G5ikihF6hFkOZRZhZSh/8LFJ+zw1YWkqMGZJEYGLBdsJTzV++arQO11SMUjZbmlf W3jj3gXPRHqSEymwJwJl3mWSRWZe7BU24Ir4MWP4VzL8z/P4Q9sk4/LyIqvxB+jT L2oCiMNV6GBTi37Nj8G2U9Nsl03R4QRumLyBkz5m8XFxPOBQKnD+VJTlWSDemehl l0ct3Wk7ZBd8eqi8S1xr+4qcFXHolJu/LM850K/QcSzaRmjXH1/W68zR/2snudnK vOIZVQmbYER+zXirVDF9 =SlPe -----END PGP SIGNATURE----- --Apple-Mail=_DED1089C-C57B-47FE-88BE-4FE509337281-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 03:19:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9A71574 for ; Tue, 8 Jul 2014 03:19:42 +0000 (UTC) Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8A912216 for ; Tue, 8 Jul 2014 03:19:42 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kq14so6489616pab.20 for ; Mon, 07 Jul 2014 20:19:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=91d2SnteREsP71pAXyezqiOqFs0RKbNTw/FsAPVfzzY=; b=etvLxRAZptm4c+AjlDZ55PyZ4p09FcpSaQh7NMEoyW5q+0CUcPFHnmLzbjwVgjBe2X Zkyh6Ccw3NTG22IrpVnpeWEXiFsVVjy/tx/dgAGek/K4hOQzwtVmIRA0Tviq/IlhCbRm qblAvbu1zDhz8ywu6LsEfWsXjELEY1mEkf/TdYX/t11vhQ1SMixj/Q94lZO6lydFwIfO XkUPsySL+qdTs9wlOfpUQVRsiwyPBN/Ikzqg6KqTeJ+3VK1kxOGJP5vl6Ry8dcPG1k7N zlt8M7xs3T6jvF2f9Mc8wid08FbqSMdJco+LRE5vi/Ej9N/arGXPDvDjWzi5YRpW9HZH eecg== X-Gm-Message-State: ALoCoQmrajjy+SQPfsEQxGLforJk2pXm8IBk9R1yYHAPBucRV9Xiza5noCNHpKQJEPwtly8GPGwC X-Received: by 10.67.16.67 with SMTP id fu3mr32449803pad.38.1404789576004; Mon, 07 Jul 2014 20:19:36 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ce9sm22296819pdb.49.2014.07.07.20.19.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 07 Jul 2014 20:19:35 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_78CDEAC4-A7AD-4B60-ACB0-8C4B880A5143"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> Date: Mon, 7 Jul 2014 21:19:33 -0600 Message-Id: <9356CE3E-1249-4318-8946-644FDD817CDF@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> To: Tim Kientzle X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 03:19:43 -0000 --Apple-Mail=_78CDEAC4-A7AD-4B60-ACB0-8C4B880A5143 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 7, 2014, at 8:59 PM, Tim Kientzle wrote: >=20 > On Jul 6, 2014, at 4:07 PM, Sean Bruno wrote: >=20 >> Objective: install an xcompile toolchain into a jail for use by >> poudriere during arm/mips/sparc/power ports pkgs builds. The build >> should be possible from a non-root user. I just re-read this=85 So why doesn=92t the following work: make buildworld TARGET=3Dfoo make installworld TARGET=3Dfoo DESTDIR=3Droot-of-jail = WITH_INSTALL_AS_USER=3Dt to generate a jail that you can use? Or are you trying to get a native = cross compiler into the jail? neither toolchain nor xdev will fit your needs, I fear... Or, in gcc terms, you=92d like to build host =3D=3D build =3D=3D = arch-of-the-silicon-of-the-mahine, target =3D=3D mips or something? Warner --Apple-Mail=_78CDEAC4-A7AD-4B60-ACB0-8C4B880A5143 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTu2NFAAoJEGwc0Sh9sBEADX0P+wWfS5pD2iQZyXnSMd3un68S /qvfEUlqZ19lLGCGte67ZRxbigeN8J7Bdd4kSKiN+r0oEUVK5L3ckW/y54A3YKi2 0tu0S/Horta+t9AvIAJGLo5zJx+BVDX4l59DpYUDzhBA/yA4Lb9uwuQ1A6mTKlBj dt9ougoG0EEBv5Yx5eJyV00MUbeUmoaA93mkVGRFYJvZRxGpZ4s4u6umX3XoI4Gt l9SGJyAOo8+DPIedsT1QXzOim9StN3vY6THMG9IvU9yUd2IikMWIwJHxsZficmHQ gQvGHze0/RsKxH5wCFrmmJZ1ZKgiud4Za33agC/Z+4+6Pb14zojDKJdG1YEa8FQ1 OWA0XcAMfRk8Fv8756agEKmidPqXCuNUAvtbTrOrzVosk5tsP6XwsoudW4R2YGeE OQgDV9bTXIh5hQKKefJ67WWiQpS8njLoUAdCz8PN5vcA0IQHfLsKgy1Qk6pCqg/h emwLqT0gOtT6xRv3p8wgf4N9HTLf0NLcWqsXwyAtaE3jkRyzox4XdcrM5zG9/pQS yJ5JUmk+vbGgr0EHbh7th8f05G5gnESsry6DuR0neTEtmUbtxyu/xEoz9SzA919N QoIl8FObaMNvZHrrtQw43jfANVmb2gltxhWqI9W0PusK9JOFeLckTq8kGwyCv80s /dOu1pSMapjGW6/BwqZU =69aU -----END PGP SIGNATURE----- --Apple-Mail=_78CDEAC4-A7AD-4B60-ACB0-8C4B880A5143-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 06:56:23 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 341E7221; Tue, 8 Jul 2014 06:56:23 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A733F21F5; Tue, 8 Jul 2014 06:56:22 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::b4fc:aef0:b041:7f3] (unknown [IPv6:2001:7b8:3a7:0:b4fc:aef0:b041:7f3]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 73D245C44; Tue, 8 Jul 2014 08:56:18 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_98D5528D-DD30-44B5-8F37-9BEC487F512A"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: Date: Tue, 8 Jul 2014 08:56:12 +0200 Message-Id: <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> To: Warner Losh X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 06:56:23 -0000 --Apple-Mail=_98D5528D-DD30-44B5-8F37-9BEC487F512A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 08 Jul 2014, at 03:56, Warner Losh wrote: >=20 > On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >>=20 >> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >> likely the problem that should be fixed in a different way. It is = really an internal >> variable that should be set based on the actual compiler type = (possibly with an >> override for the odd-duck pair of clang and libstdc++ which may not = be worth >> supporting). It is telling us we=92re doing something horribly wrong = and we should listen >> to that rather than add another compiler-related kludge to the build = system. I=92ll work >> on that bit. >=20 > Perhaps > http://people.freesbd.org/~imp/patch-queue/86gnucxx > might be the best way to cope=85 >=20 > Comments? This would make it impossible to build libstdc++ with clang, and why = remove MK_GNUCXX at all[1]? Maybe the option should be renamed to = MK_LIBSTDCXX or MK_LIBSTDCPLUSPLUS, since that is basically what it = does: enable or disable building libstdc++ and its dependent components. If the compiler is base gcc, you *must* build libstdc++, since it cannot = build libc++. But if you are using e.g. gcc 4.8 as an external = toolchain, you could just as easily disable libstdc++, and build libc++ = instead. I think both should be a user-selectable option. -Dimitry [1]: That is, unless somebody is planning on removing libstdc++ = altogether... but then g++ will also have to go. ;) --Apple-Mail=_98D5528D-DD30-44B5-8F37-9BEC487F512A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlO7lhAACgkQsF6jCi4glqPEXACgw/jzs5IZTiZ6qa4Ikc8ozTEN lkwAoMWS5UJShWpYmoICfewrfBfF4Xkl =qqwi -----END PGP SIGNATURE----- --Apple-Mail=_98D5528D-DD30-44B5-8F37-9BEC487F512A-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 12:35:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24FE16DF for ; Tue, 8 Jul 2014 12:35:06 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 037042DAB for ; Tue, 8 Jul 2014 12:35:05 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 046A619360B; Tue, 8 Jul 2014 12:35:03 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <9356CE3E-1249-4318-8946-644FDD817CDF@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> <9356CE3E-1249-4318-8946-644FDD817CDF@bsdimp.com> Content-Type: text/plain; charset="windows-1251" Date: Tue, 08 Jul 2014 05:35:02 -0700 Message-ID: <1404822903.1662.0.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit Cc: Tim Kientzle , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 12:35:06 -0000 On Mon, 2014-07-07 at 21:19 -0600, Warner Losh wrote: > On Jul 7, 2014, at 8:59 PM, Tim Kientzle wrote: > > > > > On Jul 6, 2014, at 4:07 PM, Sean Bruno wrote: > > > >> Objective: install an xcompile toolchain into a jail for use by > >> poudriere during arm/mips/sparc/power ports pkgs builds. The build > >> should be possible from a non-root user. > > I just re-read this… > > So why doesn’t the following work: > > make buildworld TARGET=foo > make installworld TARGET=foo DESTDIR=root-of-jail WITH_INSTALL_AS_USER=t > > to generate a jail that you can use? Or are you trying to get a native cross compiler > into the jail? neither toolchain nor xdev will fit your needs, I fear... > > Or, in gcc terms, you’d like to build host == build == arch-of-the-silicon-of-the-mahine, target == mips or something? > > Warner > I am trying to speed up xcompile build of ports. Right now, qemu bsd-user is handling the builds via a native gcc/clang which is super slow as everything is emulated. sean From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:03:53 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A038DED1; Tue, 8 Jul 2014 15:03:53 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 682962D4C; Tue, 8 Jul 2014 15:03:52 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id C7416193DD9; Tue, 8 Jul 2014 15:03:50 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> Content-Type: text/plain; charset="iso-8859-13" Date: Tue, 08 Jul 2014 08:03:49 -0700 Message-ID: <1404831829.1662.7.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit Cc: Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:03:53 -0000 > > > > It looks to me like the permission part of the problem is being caused > > by a lack of DESTDIR=. Without that, it's trying to install to /usr and > > you don't have permission for that. Maybe the confusion is because the > > xdev target inherently builds-and-installs, unlike most other targets > > that separate those two actions. > > OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. Iÿve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir: > > rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd > mkdir $HOME/F > make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20 > > Sean, can you see if this works for you now? And sorry about the cumbersome options. Those are in line to get fixed, but I wanna fix the build-in-tree issues first (which, alas, are harder than youÿd think). I suspect that baptÿs src.opts.mk diffs are likely good candidates to be used committed too, but the above works w/o it. Also, I just built the binaries, I didnÿt see if they worked. > > Warner TL;DR --> Can we just *dump* XDDESTDIR and use DESTDIR? mtree seems to still be busted. Updated this morning and did an attempted non-root xdev build with and without WITH_GCC_BOOTSTRAP. make xdev MAKEOBJDIRPREFIX=/var/tmp DESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t ... mtree populating /var/tmp/mips_cc//usr/mips-freebsd mkdir -p /var/tmp/mips_cc//usr/mips-freebsd mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.root.dist -p /var/tmp/mips_cc//usr/mips-freebsd >/dev/null mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.usr.dist -p /var/tmp/mips_cc//usr/mips-freebsd/usr >/dev/null mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.include.dist -p /var/tmp/mips_cc//usr/mips-freebsd/usr/include >/dev/null _xi-cross-tools ===> xdev gnu/usr.bin/binutils (install) ===> gnu/usr.bin/binutils/libiberty (install) ===> gnu/usr.bin/binutils/libbfd (install) ===> gnu/usr.bin/binutils/libopcodes (install) ===> gnu/usr.bin/binutils/libbinutils (install) ===> gnu/usr.bin/binutils/addr2line (install) sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 555 addr2line /var/tmp/mips_cc//usr/mips-freebsd/usr/bin/addr2line sh /home/sbruno/fbsd_head/tools/install.sh -T debug -o root -g wheel -m 444 addr2line.debug /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug install: /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug: No such file or directory *** Error code 71 ----------------------------------------------------------------------------------------- make xdev MAKEOBJDIRPREFIX=/var/tmp DESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t ... mtree populating /var/tmp/mips_cc mkdir -p /var/tmp/mips_cc mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.root.dist -p /var/tmp/mips_cc >/dev/null mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.usr.dist -p /var/tmp/mips_cc/usr >/dev/null mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.include.dist -p /var/tmp/mips_cc/usr/include >/dev/null _xi-cross-tools ===> xdev gnu/usr.bin/binutils (install) ===> gnu/usr.bin/binutils/libiberty (install) ===> gnu/usr.bin/binutils/libbfd (install) ===> gnu/usr.bin/binutils/libopcodes (install) ===> gnu/usr.bin/binutils/libbinutils (install) ===> gnu/usr.bin/binutils/addr2line (install) sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 555 addr2line /var/tmp/mips_cc/usr/bin/addr2line sh /home/sbruno/fbsd_head/tools/install.sh -T debug -o root -g wheel -m 444 addr2line.debug /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug install: /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug: No such file or directory *** Error code 71 ----------------------------------------------------------------------------------------- Doesn't look like the install target for addr2line is creating its directory tree? ~/fbsd_head % ls /var/tmp/mips_cc/usr/lib/ aout compat dtrace engines i18n private From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:05:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB09D310 for ; Tue, 8 Jul 2014 15:05:32 +0000 (UTC) Received: from mail-ie0-f174.google.com (mail-ie0-f174.google.com [209.85.223.174]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB36A2D69 for ; Tue, 8 Jul 2014 15:05:32 +0000 (UTC) Received: by mail-ie0-f174.google.com with SMTP id rd18so4887176iec.5 for ; Tue, 08 Jul 2014 08:05:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=TFZTGy1Jezg18THY+yD7wNGn7AtqViVZyq/4Qw+A8sc=; b=S5saikmNgldehBaEzIWW5rHLspyE22QDMvBvDpB3Y7p8oEvuf5LXjB3S9R0DhHwdPu tNzrxoXrpyxMDSfPnygPD5XVOX6xN0f9lNRaor2GhxOPi9GkF8mfW2XdLEAJO48BXi1j cSuWgCLnlWw1DSOF6CMojuOacuSJfkAOuJWEtYufVjbGPQ9cPzaHCFzKZN+ZGTNOAdG3 ESb9nUSeSEuHntGoDBfQUaZJWmUdaEVpHn9vXD3hv9wuqTPtxqdEs7SL7KtXWxepW8mT hKMiquFO6/zh0Whzn/gOzMlBoBWNbO23qoEbPKDekjR6cuxcNriBxVUmYwDcFOLOnJPW ztqQ== X-Gm-Message-State: ALoCoQka5R33xJf0g3fzfjou9e9vGLaJt5jxHvqcW63eDXhh3Gwu8hM+qmKTTCh1zWy0UCYuJcEz X-Received: by 10.43.48.8 with SMTP id uu8mr41672569icb.10.1404831932124; Tue, 08 Jul 2014 08:05:32 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id ci7sm6147907igb.11.2014.07.08.08.05.31 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 08:05:31 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_D96A7FC0-1DB7-4D00-9B23-05B9DCB3F329"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> Date: Tue, 8 Jul 2014 09:05:25 -0600 Message-Id: <24F7CB1F-80F4-4D5A-9B09-F1D47F029FD9@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:05:33 -0000 --Apple-Mail=_D96A7FC0-1DB7-4D00-9B23-05B9DCB3F329 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 12:56 AM, Dimitry Andric wrote: > On 08 Jul 2014, at 03:56, Warner Losh wrote: >=20 >>=20 >> On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >>>=20 >>> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >>> likely the problem that should be fixed in a different way. It is = really an internal >>> variable that should be set based on the actual compiler type = (possibly with an >>> override for the odd-duck pair of clang and libstdc++ which may not = be worth >>> supporting). It is telling us we=92re doing something horribly wrong = and we should listen >>> to that rather than add another compiler-related kludge to the build = system. I=92ll work >>> on that bit. >>=20 >> Perhaps >> http://people.freesbd.org/~imp/patch-queue/86gnucxx >> might be the best way to cope=85 >>=20 >> Comments? >=20 > This would make it impossible to build libstdc++ with clang, and why = remove MK_GNUCXX at all[1]? Because it is a silly option that=92s mostly an internal knob? We don=92t = need to support options that trip us up at every turn, and MK_GNUCXX has = been doing that since its introduction. > Maybe the option should be renamed to MK_LIBSTDCXX or = MK_LIBSTDCPLUSPLUS, since that is basically what it does: enable or = disable building libstdc++ and its dependent components. No. Absolutely not. I categorically refuse. This is the *WRONG* way to = do this. If it is to be an option at all, it has to (a) get set correctly when = the compiler changes (it doesn=92t today), (b) not override the user=92s = choice (which is impossible today). > If the compiler is base gcc, you *must* build libstdc++, since it = cannot build libc++. But if you are using e.g. gcc 4.8 as an external = toolchain, you could just as easily disable libstdc++, and build libc++ = instead. I think both should be a user-selectable option. It needs to start working, or it needs to be removed. IT is that simple. = I=92m tired of mopping up options that only half-assed work to support = configurations that aren=92t even close to official on the theory that = they might be someday used which in turn causes real problems for = supported configurations. So, if we can support it without having the user have to care about it = except in the bad-shit-crazy cases, I=92m all for that. Otherwise, it = has to go. What we have now just isn=92t cutting it. Warner --Apple-Mail=_D96A7FC0-1DB7-4D00-9B23-05B9DCB3F329 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvAi2AAoJEGwc0Sh9sBEAvyoP/36RLDtWg2A3YtTY9NwtNkVr UyXn3TyHAxbkiiaPsHG/W5OD6M5/kldKdvJe1NSp4XZFEvKVGHghhQpqy1I95RWc 6NJtXGN0fw+xiyzRYfDhp5EApzwXD8e+vaH16lRDAMKfwSHLCPfFQv1L/sHPgpNH dYT3Rtc0E+WZ0OGAhdfJEJ10/iACu9cRtjfAJjp6AhqsvxgoNgPUBo7G9Pp3V8EZ 2qRDvtwe1MzUaWCwDy1gPoS71zWInmmtemlsTTqGDXpv1RDoTBadHz1JVED5mdp/ S1vUyA0Zibi19J63y+L4gvthX0Y80xAQNI/UJYR87QL7xGzeNM/F9ACzEOwz07VL NPbZcg5OQV75IYLvxk1+dbwsFOOkngalk0u2gFaG08vmkvepkcB6pk9C6hlxcLme PYW00K5J2m5rD2adNvD7cx5x3Hca5p+rIwlkgU+ZuTr4nn2Stx187GZZNnt0/tzc hvYWYha9V45KPStKhucnRo05Rd4lpWmnPVSwkpZK5NACRLBhMGiqjFC5N6As5tQv 83oW98CuMKrjk97lOLWVi/JGqpKJw909N9chyZkPd+HYqtdEjPkcCxeobMJKzpQh QFo6ogkGmxwGqbKnK+v4+Y2TU/IZtFfemfA84sBkpUL6LzMDSGiKLI5jSsA+YQi5 cyGcjvCXW3GVxTxiXxbr =tZBs -----END PGP SIGNATURE----- --Apple-Mail=_D96A7FC0-1DB7-4D00-9B23-05B9DCB3F329-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:10:21 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEE335C0 for ; Tue, 8 Jul 2014 15:10:21 +0000 (UTC) Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B32952DB9 for ; Tue, 8 Jul 2014 15:10:21 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h15so778885igd.2 for ; Tue, 08 Jul 2014 08:10:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=X1pSkN/YrV7LWQnR3Y/RiyX/rr2spQLKsxJhWhws9N4=; b=DnbB3r+6JqTbhcdBudwXdii2y0gk1upAkYUREQxssYKXAzxD8BVmmJhRkbrGra7IlN UZvNenKnoCtqSAxpKGAFHnqVqCs9wcbRmpF//nv6x3xBT4k3nB5IB/5xD2rWaNf7i2dZ JeyP3X+caSNmaWvd0/CZPvPP9uRhmD6RqqB8mlejFYuDDW1xdSD3xKlnNQgligv7mgq9 OXVsg1+W39L01IagFs2ZT/xXfN5TH2RkKJKVeMAYgxTpy0JP3h+L7Pb3jOLyjiyt6tko gZxUUvALM+D1LivV8m6z+cg0Bz8l249NDzI1JmGmT4vQeS7w4wBv+dFS65ZDbXntGej0 +e1w== X-Gm-Message-State: ALoCoQlb+p4LytDRzzYR/nnEBOtYbOTdRWRrRjkptN+pPrYeLIMzI3Pq72MRpNTq0M+vI6cRl+5n X-Received: by 10.50.27.4 with SMTP id p4mr4726852igg.22.1404832219392; Tue, 08 Jul 2014 08:10:19 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id f19sm55481igt.13.2014.07.08.08.10.18 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 08:10:18 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_F0F1A90A-7237-4D8D-9FD9-64BB0DB94A73"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404822903.1662.0.camel@bruno> Date: Tue, 8 Jul 2014 09:10:13 -0600 Message-Id: <13698752-8370-404F-8919-969804CFE0CC@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <2788FB1E-9498-40C2-94DE-F4C73A5DDD30@kientzle.com> <9356CE3E-1249-4318-8946-644FDD817CDF@bsdimp.com> <1404822903.1662.0.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: Tim Kientzle , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:10:22 -0000 --Apple-Mail=_F0F1A90A-7237-4D8D-9FD9-64BB0DB94A73 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1251 On Jul 8, 2014, at 6:35 AM, Sean Bruno wrote: > On Mon, 2014-07-07 at 21:19 -0600, Warner Losh wrote: >> On Jul 7, 2014, at 8:59 PM, Tim Kientzle wrote: >>=20 >>>=20 >>> On Jul 6, 2014, at 4:07 PM, Sean Bruno = wrote: >>>=20 >>>> Objective: install an xcompile toolchain into a jail for use by >>>> poudriere during arm/mips/sparc/power ports pkgs builds. The build >>>> should be possible from a non-root user. >>=20 >> I just re-read this=85 >>=20 >> So why doesn=92t the following work: >>=20 >> make buildworld TARGET=3Dfoo >> make installworld TARGET=3Dfoo DESTDIR=3Droot-of-jail = WITH_INSTALL_AS_USER=3Dt >>=20 >> to generate a jail that you can use? Or are you trying to get a = native cross compiler >> into the jail? neither toolchain nor xdev will fit your needs, I = fear... >>=20 >> Or, in gcc terms, you=92d like to build host =3D=3D build =3D=3D = arch-of-the-silicon-of-the-mahine, target =3D=3D mips or something? >>=20 >> Warner >>=20 >=20 > I am trying to speed up xcompile build of ports. Right now, qemu > bsd-user is handling the builds via a native gcc/clang which is super > slow as everything is emulated. Yea, we got nothing that=92s for that use case. toolchain is for the = weird world that we build /usr/src in. xdev is for building mips on x86 = when the rest of the tree is x86. You=92re wanting native binaries when = the rest of the world is mips but that behave like the world is mips so = that when invoked in the typical ports way, it just works. I don=92t = think xdev will do that because the paths are messed up and you=92ll = need two copies of libraries, includes, etc. It expects to be installed = in /usr/$ARCH-freebsd/ and if it isn=92t, it gets cranky. Having said that, it sounds like a third way to build these may be = needed. One that expects to be installed in /usr/bin, is amd64 = executable, but produces mips binaries from /usr/lib, /lib, etc. Warner --Apple-Mail=_F0F1A90A-7237-4D8D-9FD9-64BB0DB94A73 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvAnVAAoJEGwc0Sh9sBEAtiMP/RE5QZZBzBRAS0XyUWad2GAR x9pniXdHb2IyKY0lgfCGjYODIV7LrGBowJWKmeSoxR2ZhYjVdlWT/z5o8gX4zaBd UO8TvbfLXyXXCQyaHaSkgA5u+s2ZF68vuqfpeD2Yy9SDpIH6moZgPsfEK47j3Jau /m0zzR8sI6/M1WcPumH2XiHs5N0EQT3uWcU/V1vZjEv10xjziFuTmmlUELZ9X5qT l7bwVmre6kQaa5iB5Bc73uO6MekHOw0BUM+90TLsvbSpYXyku9i80h4mT7/tqWNt ROpKQrjoEy8sTdA90PN2pPdZLM7qa/nHHfbjTHJD71xgK/7tzh8fxhTLdh7jA9W3 +ePQojpQJdHSFSiIIGdNv7BdJtuzNadWK+iB3WjlOuhrwfxo7sgzrnAvxi7V4au4 6uOvWJQFN0c0vaPvrI2ahSPRJucVuh8afKaUWfkOHWrg+o0C8qhCClYs40oSi+rf AC7opqGTR3a9+MS000yL53u385MrOKxxB1kHHPjzpM4NcxXkcso9alhM9Q7+yhl/ zAZhI/TbJ0nI0J+m4IH1/0RzEuz4mmaa1ga/3SDG0vcTS+5Lkhu6W2w7MJbUWA5D 7E+766WN45Fx+JjTcafNvfiJ6ZJh0YF8/MZiRw3AlKCftrhSimbJLp//NXsHLG8P hSBU5zn/XxQIsSo0hvMW =SWYX -----END PGP SIGNATURE----- --Apple-Mail=_F0F1A90A-7237-4D8D-9FD9-64BB0DB94A73-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:15:54 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4751AADC for ; Tue, 8 Jul 2014 15:15:54 +0000 (UTC) Received: from mail-ie0-f177.google.com (mail-ie0-f177.google.com [209.85.223.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 082202E6C for ; Tue, 8 Jul 2014 15:15:53 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id tr6so424482ieb.8 for ; Tue, 08 Jul 2014 08:15:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=dNM3AC5vFnM7gP2JDmdoC/cLpVaEtX+7uiw77gE1C9A=; b=JM08EtZUPjYHMawDKPzwa2TYOA8ZDbtbUalVP8q/ZHGVlcGV7NodT9UigCWlnsKgCg X2IRDqYSSE8qBunVjJ7HSLCKs7i9nx5YimxdtkFsirseh8qEp9o1OWUSpELL1TJAssKz XJVyc/0bYU3s1LIBWUmSvpOY5WxY1oqzb05mAbR0knmDByCc5DV3u1Y3k4QceZRGKEVk T0g7CmSy2v5Uyg1TR+7q2Ah7rXyOWDWHt2PAQNGJKpHv7u/51Dyvb41JCi5tcd12+zVz tOGzo+HA5hpiWp5mGlE5c4dV3I+g8ux/cl35o/UY+fT7Mz+TA5Har6xG/tY9hZYiKhgF NJpA== X-Gm-Message-State: ALoCoQkQl7nELxYah9M9X/xdee5HwcVerOaMcFGM1pTh6ocGpM3hRIC2Hm2//RlwnSE9ka0oNnpK X-Received: by 10.50.142.97 with SMTP id rv1mr4694720igb.13.1404832547125; Tue, 08 Jul 2014 08:15:47 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id b10sm6204756igf.20.2014.07.08.08.15.46 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 08:15:46 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_F30AA1CF-B2EC-44F3-B830-4B341A5FFC61"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404831829.1662.7.camel@bruno> Date: Tue, 8 Jul 2014 09:15:41 -0600 Message-Id: <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:15:54 -0000 --Apple-Mail=_F30AA1CF-B2EC-44F3-B830-4B341A5FFC61 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 9:03 AM, Sean Bruno wrote: >>>=20 >>>=20 >>> It looks to me like the permission part of the problem is being = caused >>> by a lack of DESTDIR=3D. Without that, it's trying to install to = /usr and >>> you don't have permission for that. Maybe the confusion is because = the >>> xdev target inherently builds-and-installs, unlike most other = targets >>> that separate those two actions. >>=20 >> OK. After some detective work, it looks like libstdc++ needs to be = done before libsupc++ is done. I=B4ve added this dependency in r268377 = and was able to do a full xdev build with a clean obj dir: >>=20 >> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd >> mkdir $HOME/F >> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt -j 20 >>=20 >> Sean, can you see if this works for you now? And sorry about the = cumbersome options. Those are in line to get fixed, but I wanna fix the = build-in-tree issues first (which, alas, are harder than you=B4d think). = I suspect that bapt=B4s src.opts.mk diffs are likely good candidates to = be used committed too, but the above works w/o it. Also, I just built = the binaries, I didn=B4t see if they worked. >>=20 >> Warner >=20 > TL;DR --> Can we just *dump* XDDESTDIR and use DESTDIR? mtree seems to > still be busted. >=20 > Updated this morning and did an attempted non-root xdev build with and > without WITH_GCC_BOOTSTRAP. =20 >=20 > make xdev MAKEOBJDIRPREFIX=3D/var/tmp DESTDIR=3D/var/tmp/mips_cc = XDEV=3Dmips > XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt > WITH_GCC_BOOTSTRAP=3Dt >=20 > ... > mtree populating /var/tmp/mips_cc//usr/mips-freebsd > mkdir -p /var/tmp/mips_cc//usr/mips-freebsd > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.root.dist > -p /var/tmp/mips_cc//usr/mips-freebsd >/dev/null > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.usr.dist > -p /var/tmp/mips_cc//usr/mips-freebsd/usr >/dev/null > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.include.dist > -p /var/tmp/mips_cc//usr/mips-freebsd/usr/include >/dev/null > _xi-cross-tools > =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 555 > addr2line /var/tmp/mips_cc//usr/mips-freebsd/usr/bin/addr2line > sh /home/sbruno/fbsd_head/tools/install.sh -T debug -o root -g wheel = -m > 444 > addr2line.debug = /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug > install: = /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug: = No such file or directory > *** Error code 71 .debug? We only generate those when MK_DEBUG_FILES=3Dyes. Try removing = WITH_DEBUG_FILES from your environment. It looks to be broken. > = --------------------------------------------------------------------------= --------------- >=20 > make xdev MAKEOBJDIRPREFIX=3D/var/tmp DESTDIR=3D/var/tmp/mips_cc = XDEV=3Dmips > XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt > WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt >=20 > ... > mtree populating /var/tmp/mips_cc > mkdir -p /var/tmp/mips_cc > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.root.dist > -p /var/tmp/mips_cc >/dev/null > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.usr.dist > -p /var/tmp/mips_cc/usr >/dev/null > mtree -deU -f /home/sbruno/fbsd_head/etc/mtree/BSD.include.dist > -p /var/tmp/mips_cc/usr/include >/dev/null > _xi-cross-tools > =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 555 > addr2line /var/tmp/mips_cc/usr/bin/addr2line > sh /home/sbruno/fbsd_head/tools/install.sh -T debug -o root -g wheel = -m > 444 > addr2line.debug /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug > install: /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug: No = such > file or directory > *** Error code 71 > = --------------------------------------------------------------------------= --------------- >=20 > Doesn't look like the install target for addr2line is creating its > directory tree? >=20 > ~/fbsd_head % ls /var/tmp/mips_cc/usr/lib/ > aout compat dtrace engines i18n private but what=92s in /var/tmp/mips_cc//usr/bin? Warner --Apple-Mail=_F30AA1CF-B2EC-44F3-B830-4B341A5FFC61 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvAsdAAoJEGwc0Sh9sBEAwFkQAOy4neq+1nByu0cgTR8JLpjV oqc1vMi5SfyPv10tGO5MaiznL6yp5fnyJ59gznwa+2Ryo5DnxtllPEmXu9UdI34j rfGB/AwarXxq5KM1fCOpTJvrG9YeT1b7EVBdOBOcc+K3dDDtg8rmOzT4JnZemfNl UUWwbMisEW1+EJVkgcaPDI3CpxXqq+dTxH8WaeV4C34q/Fy3g082RBSeG18Qxivb u/AvV5NUrdhqQojSBuoWL6gJ022D7I/2xs/nSWcPM4+TARa39uQijTu58fL9ukpK bTNuh4NrERH2wLJcmbPmuRJ+VvhWXg4t3yIiMPfFCpogRCuNhdGuYcQ39E2WDC65 dhxf6a+KkqsAF5WWz4jty+Vb6Wlg7aQCvVlq4e4Rjel/1/D6nFRzwAaTxXyUAOcj ZhihbihjH74rW2/xhbKDnq1K/Xg17wWjLJlZaflcZfQvo9Z+uY8IkegdCorAwJj8 JAKb7FPSUDM1WCtApc54xHJ/t5PkjrrjUcSTel5uUUkhUQA3poG0iZ48PlLPtV5a g8SvDDEGrUUGz1H28pT3+fCSALKvN/e2AVSWuKpMHc++6AiGvmxw+dTYfXzaPryZ IWhEvGX8XZzWXiNwLbueGnvlnXvu8uWs51m3Oieug4uccuF6aN6Rkx82+SgY64or sR2vBvoXbB3FFoJ+7DyV =XI/d -----END PGP SIGNATURE----- --Apple-Mail=_F30AA1CF-B2EC-44F3-B830-4B341A5FFC61-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:55:52 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A001573C; Tue, 8 Jul 2014 15:55:52 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D9D4225D; Tue, 8 Jul 2014 15:55:52 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 3051C193DD9; Tue, 8 Jul 2014 15:55:51 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> Content-Type: text/plain; charset="us-ascii" Date: Tue, 08 Jul 2014 08:55:50 -0700 Message-ID: <1404834950.1662.9.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:55:52 -0000 On Tue, 2014-07-08 at 09:15 -0600, Warner Losh wrote: > > 444 > > > addr2line.debug /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug > > > install: /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug: No such file or directory > > *** Error code 71 > > .debug? We only generate those when MK_DEBUG_FILES=yes. Try removing > WITH_DEBUG_FILES from your environment. It looks to be broken. > > Ok, yes. The default cluster config had debug set in make.conf. Removed that and this error does indeed dissappear. I'll add it to my notes for later. sean From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:57:10 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06DC78FE for ; Tue, 8 Jul 2014 15:57:10 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D92562286 for ; Tue, 8 Jul 2014 15:57:09 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id C524F193DD9; Tue, 8 Jul 2014 15:57:08 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> Content-Type: text/plain; charset="iso-8859-13" Date: Tue, 08 Jul 2014 08:57:08 -0700 Message-ID: <1404835028.1662.11.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:57:10 -0000 On Tue, 2014-07-08 at 09:15 -0600, Warner Losh wrote: > > > addr2line.debug /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug > > install: /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug: No > such > > file or directory > > *** Error code 71 > > > ----------------------------------------------------------------------------------------- > > > > Doesn't look like the install target for addr2line is creating its > > directory tree? > > > > ~/fbsd_head % ls /var/tmp/mips_cc/usr/lib/ > > aout compat dtrace engines i18n private > > but whatÿs in /var/tmp/mips_cc//usr/bin? > > Warner "stuff" that looks like things that I'm interested in. :-) dirty.ysv:/var/tmp/mips_cc/usr/bin % ls CC ar c++ cc g++ gcov ld objcopy ranlib size strip addr2line as c++filt cpp gcc gcpp nm objdump readelf strings From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:58:22 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 48B58A8B for ; Tue, 8 Jul 2014 15:58:22 +0000 (UTC) Received: from mail-ig0-f177.google.com (mail-ig0-f177.google.com [209.85.213.177]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0D1B2229F for ; Tue, 8 Jul 2014 15:58:21 +0000 (UTC) Received: by mail-ig0-f177.google.com with SMTP id r10so828467igi.16 for ; Tue, 08 Jul 2014 08:58:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=MHseGBiK7zPcpWgrviUKAHd3LRXU4u49Hqz+vA15RRM=; b=M8FwMAWEF+OY+zvzcEEDFbM/UdDanjUi94rIeuAZaRuETMR1ZaiKp86TyUuZTok+E5 1eDCwd8C8sQ4DxdNiok4ZqJ8HFXKEb9gy85GzhivrAxFyl1NNTUDMj/S13afzmweQS6W ikZyRxI/AtFuRH9rlyvU6T7bFhSgWMuG4Lno7ngbak/AEVsArdgBaLUwNKaGjqqiwJ4Q /WG1deoE9/k0ECL3NvIEo1VjjP5B6DG5f8BxLQ/ll65vkSIn0f3NA7GE6EUhmNTPzxZL 7VgZkpVd5+jdm53/NJQNvwizwC6vQ4lJxwLsQXrgAEa5hmNGL3IR4vpP2dPxTLt2CZSy 41UQ== X-Gm-Message-State: ALoCoQnF4pGVixVPA4yToZzCYDim21ezpvQIhEmkh4pbSq8DTE+EXZQh3kCCx8Z24Whs7VeW9aYy X-Received: by 10.50.27.4 with SMTP id p4mr5095843igg.22.1404835094822; Tue, 08 Jul 2014 08:58:14 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id 8sm6501418igr.16.2014.07.08.08.58.14 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 08:58:14 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_82A59BFA-40F7-4D56-AF63-DE86DEB85622"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404835028.1662.11.camel@bruno> Date: Tue, 8 Jul 2014 09:58:08 -0600 Message-Id: <8DD3DED6-66D2-4938-9766-7674202CC6AC@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> <1404835028.1662.11.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:58:22 -0000 --Apple-Mail=_82A59BFA-40F7-4D56-AF63-DE86DEB85622 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 9:57 AM, Sean Bruno wrote: > On Tue, 2014-07-08 at 09:15 -0600, Warner Losh wrote: >>>=20 >> addr2line.debug = /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug >>> install: /var/tmp/mips_cc/usr/lib/debug/usr/bin/addr2line.debug: No >> such >>> file or directory >>> *** Error code 71 >>>=20 >> = --------------------------------------------------------------------------= --------------- >>>=20 >>> Doesn't look like the install target for addr2line is creating its >>> directory tree? >>>=20 >>> ~/fbsd_head % ls /var/tmp/mips_cc/usr/lib/ >>> aout compat dtrace engines i18n private >>=20 >> but what=B4s in /var/tmp/mips_cc//usr/bin? >>=20 >> Warner=20 >=20 > "stuff" that looks like things that I'm interested in. :-) >=20 > dirty.ysv:/var/tmp/mips_cc/usr/bin % ls=20 > CC ar c++ cc =20 > g++ gcov ld objcopy ranlib > size strip > addr2line as c++filt cpp gcc > gcpp nm objdump readelf = strings Woot. That=92s what=92s supposed to be there :) Warner --Apple-Mail=_82A59BFA-40F7-4D56-AF63-DE86DEB85622 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvBURAAoJEGwc0Sh9sBEAzsMP/iMfSsIPLcTTXkr7xUDK7kPr 3oeQ9Ev99JJJKDhhzVwO7NaLjGmor+XgWkF/TKw+wR58uYAI1Nj3A3jqXG6Gar32 abUjTVImXEcsAlpNVuqyO5fdbihsJtTeAWFogL3PEDvkGMEJIXx8X6Ia/tnPC2aE OFK7IcmUXNqJPslIi8a2UF4Dn5ZZi/ZhCJjjHIBKjhAAi5E3O6sc/XOOhhKSbS2r QRBKQv6nMf/OTMy/7PpNHQzqNXGPkV/7gGQ86yDQ42gQxzh5jmyjXHz4OoIEFytj 1I4jmprSjXCC/cUQYTVQHVg9BHWZOfuf1KI7ZrllxsN1YNlkChWvyzjvbrWdFAk4 /gkM1YFFsxnPFMbsZoUbK4SrMVyQDpn2VerZmVEy3zUdYrcaFNzBZx3trkBwlW0g 7KOY4/YeuOXLRo96hFBMfPbN/VIIah0E5ADL0KiDZBzL7QBdwfUE6o/G5642h+Ra iQNX/C6efipwTCmR5zrpSDvD1L9YYJA0PxkrS8LfTWWiJIrKJt2LdixpYWfpPepz 269Ph0jTqgo18/wxdPHPvI9tzu47ByboNYUzcIZVuLSOl2s7dun1j5RY0icu8VV2 XwMoqyC5pERCGEG64tzDARmXYv4b722paDyXFjiqOnBIImezP/A9l2vOsGdu7KjP 6FGMIg2hzXF5Bc6rJbyz =i5mm -----END PGP SIGNATURE----- --Apple-Mail=_82A59BFA-40F7-4D56-AF63-DE86DEB85622-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 15:59:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FD3BB6B for ; Tue, 8 Jul 2014 15:59:50 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4423422C7 for ; Tue, 8 Jul 2014 15:59:50 +0000 (UTC) Received: by mail-ig0-f175.google.com with SMTP id h3so834667igd.8 for ; Tue, 08 Jul 2014 08:59:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=+E6dsq5QbdIEJ2PjKljZcqTM7+RBP1ERgvnvT0B+BxY=; b=a3xnHOVELmsUlk1t4n027IgKyp/i3uhQurFMouiwS1Qy1VMnlNM8LLGwLvmO5T4ABj zMI4MyDYpaydRNI7Ai6VVovpLm9AomrKmSFo/D1ah0mwTIZvVKowNsdZCCmw9e/93oYD UGbmcrdsk57JMxH2uUyZH5JLRKD5s7pXjukctl4dPSXE0wQbzZYAyKV7TMhTVA1108RB nNJ9hI6onSnPz3/sdZNLsnDz9JZ/UKYVqtMNjTk1T1DO2qpSf5rXgn42psg7/+qEpm5O e6TkyDW3SsasXa9ijSGNarE8mgLyjHam/cN8OjbuCznVu0jm6RIx7UY1JxberGf2yhMB eQ/w== X-Gm-Message-State: ALoCoQkw8jtuDsjXBNVtAmjX88Vw8LRXNn6juLgzWGgigg8pDrMc9+IPHJlJ4O1w9RhEarbBLpQw X-Received: by 10.50.164.201 with SMTP id ys9mr5075722igb.40.1404835189236; Tue, 08 Jul 2014 08:59:49 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id d4sm6549358igc.5.2014.07.08.08.59.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 08:59:48 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_9E0C6141-AA14-4A6B-BFF6-9257D83D71AF"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404834950.1662.9.camel@bruno> Date: Tue, 8 Jul 2014 09:59:23 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <2D708C4F-31DB-4F9D-BC02-2050AD8765CD@bsdimp.com> <1404834950.1662.9.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 15:59:50 -0000 --Apple-Mail=_9E0C6141-AA14-4A6B-BFF6-9257D83D71AF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 9:55 AM, Sean Bruno wrote: > On Tue, 2014-07-08 at 09:15 -0600, Warner Losh wrote: >>> 444 >>>=20 >> addr2line.debug = /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug >>>=20 >> install: = /var/tmp/mips_cc//usr/mips-freebsd/usr/lib/debug/usr/bin/addr2line.debug: = No such file or directory >>> *** Error code 71 >>=20 >> .debug? We only generate those when MK_DEBUG_FILES=3Dyes. Try = removing >> WITH_DEBUG_FILES from your environment. It looks to be broken. >>=20 >=20 > Ok, yes. The default cluster config had debug set in make.conf. > Removed that and this error does indeed dissappear. I'll add it to my > notes for later. Yea, xdev hasn=92t been exhaustively tested with all options. :( It = turns out there=92s a lot, and they break it in subtle ways. Warner --Apple-Mail=_9E0C6141-AA14-4A6B-BFF6-9257D83D71AF Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvBVbAAoJEGwc0Sh9sBEA+iMQAL7u7mdiTffR7uB0sMlrDbz6 JtAd6UZjyPwtS2C/TY9k6lJsyoH5FdwxzoKytHfLZxucqK+9oGyqiauTRkkGz/ji sa6bIf6UjhP+OaZZxN29yFflOGMgJKnuEsISGqAtuhr3UFFN2hOSIHtjvOg7VLCh EnH0PQfOwSZJ/YsaALQpv2574FVdkn/b6IrXlN8bLM99+SRWSEAq98eyVGtrpq6E WZbMT5an/U/74Z6KCWfe6if6aYm02sKRCreSpmRl7ROKVsb3dWfMLUM7b4atUqEf /KQdUOLO7gx7+uticjAAxE/4xNJyqODrCq1M8JF/jfN+OaX9lKEUVUFAsufuTg7Q QUQSmfKKte5o+zHLJIQsk1/KFU5PZv2XW0faRUtQzndq/9DY1YlUq3L3/w+RyyDf 7vfM9ILiWgrXzDcZlWwix4zOAxn+S1j1SX6nQNSqhw0A8xZwDNU0IQWiEH3LTFDQ 7KxqFZxOFEux/Fb1bRs8uSrFXDnrbL0zT1c6NXi0d/blTqNGB5ZEL6ytj2la5MaO gFrjDJf9NfB3+DJgiBum/mHL23GBvbMDm0W+4vrAYUeqyIyQ1ssK6YM0i14/ZrhT QmWqqSHbTECJPGXjBEnYCzAb8AdzzXbbIpmgCNYiQWJpPZb3aW//+JVQwYCTI16c NFSmRbi6bzy5yIClAmIT =g/fu -----END PGP SIGNATURE----- --Apple-Mail=_9E0C6141-AA14-4A6B-BFF6-9257D83D71AF-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 16:04:34 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3353ECE2 for ; Tue, 8 Jul 2014 16:04:34 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F13EA2390 for ; Tue, 8 Jul 2014 16:04:33 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 72AEF193DD9 for ; Tue, 8 Jul 2014 16:04:32 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: freebsd-arch In-Reply-To: <1404831829.1662.7.camel@bruno> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> Content-Type: text/plain; charset="iso-8859-7" Date: Tue, 08 Jul 2014 09:04:31 -0700 Message-ID: <1404835471.1662.13.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 16:04:34 -0000 On Tue, 2014-07-08 at 08:03 -0700, Sean Bruno wrote: > > Sean, can you see if this works for you now? And sorry about the > cumbersome options. Those are in line to get fixed, but I wanna fix > the build-in-tree issues first (which, alas, are harder than you¢d > think). I suspect that bapt¢s src.opts.mk diffs are likely good > candidates to be used committed too, but the above works w/o it. Also, > I just built the binaries, I didn¢t see if they worked. > > > > Warner Ok, it seems like we've come full circle. :-) I'm once again back at include failures, for libstdc++: make -s xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t ... ===> secure/lib/libssl (obj,depend,all,install) ===> gnu/lib/libstdc++ (obj,depend,all,install) In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or directory In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:38, from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/bits/functexcept.h:41:28: error: bits/c++config.h: No such file or directory In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/bitmap_allocator.h:39:37: error: utility: No such file or directory /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/bitmap_allocator.h:40:60: error: functional: No such file or directory In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:43, from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/concurrence.h:39:19: error: cstdlib: No such file or directory /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/ext/concurrence.h:41:24: error: bits/gthr.h: No such file or directory In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:38, from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/include/bits/functexcept.h:44: error: expected constructor, destructor, or type conversion before '(' token In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/libsupc++/new:45, from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:41, from /home/sbruno/fbsd_head/gnu/lib/libstdc ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/libsupc++/exception:40: error: '#pragma' is not allowed here /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc ++/libsupc++/exception:133: error: expected declaration before end of line *** Error code 1 Stop. make[4]: stopped in /home/sbruno/fbsd_head/gnu/lib/libstdc++ *** Error code 1 From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 16:23:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9642377E for ; Tue, 8 Jul 2014 16:23:58 +0000 (UTC) Received: from mail-ig0-f169.google.com (mail-ig0-f169.google.com [209.85.213.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5733825FA for ; Tue, 8 Jul 2014 16:23:58 +0000 (UTC) Received: by mail-ig0-f169.google.com with SMTP id r10so943123igi.4 for ; Tue, 08 Jul 2014 09:23:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=Xh3dMnHw9ujT2LvujEDFvTdxiT8oKRnmnNfgmtOw2r4=; b=KSXXV+dcCpFeAr7ZiYMsm2YNbhUbdD8VaVd6tTpOTwZrbscyaMhRKmv/fdnH2HUOuQ cLsmCOv1lltgqXXR3ck2eo9AsYm02KjkXC2YlxMHXFpuoyR+dDOP9N2Qs3Hu4ARD98xH NfYDTLLja42hWLFFKMwx75bwUjE7Q8sFqqz4S6v4vLqKgYpjMhEtc/vGwbJd51xJPvfG whXTuFdJTu/Rd1ZucLHJzjjZjWpKpyC07UkIgf5h2UuM9T7eMVPfn1kwkqoWlRR0p5vg MXr362dr68TJwpKmN6nwOh3ZHrG6N1PWAzzF6ifn5smtwDAYvHq+iJiNL5tI/Yoggg1E u9Qw== X-Gm-Message-State: ALoCoQmBNwKVSKPRKKVLDgfrxWRViX/k1+fport+U84nY4i9JwCSlpNiQalqcAGIHGzRe1yKjKen X-Received: by 10.50.61.148 with SMTP id p20mr5148453igr.44.1404836637331; Tue, 08 Jul 2014 09:23:57 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id e2sm6694195igi.12.2014.07.08.09.23.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 09:23:56 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_AD0EDF63-FC1B-4FF7-AD3C-FD1195B7881C"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: Date: Tue, 8 Jul 2014 10:23:51 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> To: Warner Losh X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , Dimitry Andric , sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 16:23:58 -0000 --Apple-Mail=_AD0EDF63-FC1B-4FF7-AD3C-FD1195B7881C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 9:04 AM, Warner Losh wrote: >=20 > On Jul 8, 2014, at 12:56 AM, Dimitry Andric wrote: >=20 >> On 08 Jul 2014, at 03:56, Warner Losh wrote: >>=20 >>>=20 >>> On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >>>>=20 >>>> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >>>> likely the problem that should be fixed in a different way. It is = really an internal >>>> variable that should be set based on the actual compiler type = (possibly with an >>>> override for the odd-duck pair of clang and libstdc++ which may not = be worth >>>> supporting). It is telling us we=92re doing something horribly = wrong and we should listen >>>> to that rather than add another compiler-related kludge to the = build system. I=92ll work >>>> on that bit. >>>=20 >>> Perhaps >>> http://people.freesbd.org/~imp/patch-queue/86gnucxx >>> might be the best way to cope=85 >>>=20 >>> Comments? >>=20 >> This would make it impossible to build libstdc++ with clang, and why = remove MK_GNUCXX at all[1]? >=20 > Because it is a silly option that=92s mostly an internal knob? We = don=92t need to support options that trip us up at every turn, and = MK_GNUCXX has been doing that since its introduction. Also, in the current tree it means different things in different places. = In some places it says to build libstdc++, in other places it says to = build g++ and friends. These are two different meanings, which also = confuse things. This is why I call it an odd duck, among other reasons = (it=92s default is the default for the platform, not correct for the = options actually used, it means different things, it controls things = with a bool that are better off being controlled by a choice value, = etc). Warner --Apple-Mail=_AD0EDF63-FC1B-4FF7-AD3C-FD1195B7881C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvBsXAAoJEGwc0Sh9sBEAldQQAL0hmArA2mNjUil+o+aJoJtt yFg+FyIVvRHB2JwXn6PmxFdPGvKsoAfle0Gurqi0hfmSMbgfotbBTVdxixhWtAzC yq66Qhz3IpE8iXqtbkR8Hf+px9qSjR160p1Oi20QdR1TVVMeTYzCk0BtcyFAPKNG HdRqUelQS/ynt55zZOG16ElE+Y55OQvAnTC43qpXUCGhy9KxXPjfzgUfVv6qFjHk RGDJ96lBwHs3WZVoMnk43TgTUQ6rx2Df2vQSnR90gKkNgwS0Em+UICO924bgcOoa CixJYWOGbFKwd56dxH3Uimiys9/sZtZfTFB/TDC8SHiMKYmwr+kvDsPB7lft/32l J+0gRSGXhtP9TdNqGmXCodx1oh2n7J7vUsQ32WXZIc/xgYSnhuxUJ5mq7ZJc7nay Mzugf1WMUqfWyn2FfSXwWwb4+xsd/N2IFxqhbMcJfnCAn6z/dVXkV+ixCPhXssvZ +eZiofn/2Bi/fbXR68FZQ9Ax/tgxPyHNZQCGE8ejtOilA70CJf24faR3ox95nhTM MfLN+YN6Z3v30J4anMxPnjSsjK2jh8WC+hr8JZaoMuiFm94kBNWBgMp8gkpfQNEJ 8C2J8nW4ukOrnKmnSDea2BtDvvu+Ub32gajWrf4ci6pPLQWyVOJf0J5CZCsg3LMH wsxqD68sm1JETgBIJzFV =doo0 -----END PGP SIGNATURE----- --Apple-Mail=_AD0EDF63-FC1B-4FF7-AD3C-FD1195B7881C-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 17:16:44 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CCC009D8 for ; Tue, 8 Jul 2014 17:16:44 +0000 (UTC) Received: from mail-ie0-f175.google.com (mail-ie0-f175.google.com [209.85.223.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9151B2AD9 for ; Tue, 8 Jul 2014 17:16:44 +0000 (UTC) Received: by mail-ie0-f175.google.com with SMTP id at20so2450257iec.20 for ; Tue, 08 Jul 2014 10:16:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=nHiDSwQnOBVa71B8+v3wYXD+UtHLH0cn6st21IzjSFQ=; b=CnK8LG5CEpd4fBGFJESKPJHVcfweqaSDDQUONV3FTz264cDW8+HdftEWi3pAkgwaP/ TcAGkaOCTpaxMMTMKilLEKzUjb2P3Zj8OffvdSIKkKBdquAeJzrqADrxDkBzrs1LwB5Y GntiO+rkkTKgU4K5oVdfczc/SDNtOz3PQXlB+r6RB8lv2GbeXJZVi4CAQOMMpaeVQMAK WrrO3rXNhPgMWaF+Ohtd+GUH3AyIST4EhWAbWHzXzNqgSWTZPppp8yoB1l7LGm8L9WdA /+En50ig3AeFk1Vy7AFPxldg4pnRyzInXi+1cJtXKUxTjUzYwEH0d2lTq36lvFJY0M+8 IBfg== X-Gm-Message-State: ALoCoQmEy6UgVezgyH5G+aK1isNd9gO6eye87HyYe4kIobLub5cgDcWQMV2dJBaX0rPu/DKZ5sKa X-Received: by 10.42.154.132 with SMTP id q4mr16457449icw.4.1404839798033; Tue, 08 Jul 2014 10:16:38 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id qo2sm7043931igb.15.2014.07.08.10.16.37 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 10:16:37 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_8329BE43-0487-4FB3-93BD-6CADD4FFA9A7"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404835471.1662.13.camel@bruno> Date: Tue, 8 Jul 2014 11:16:32 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 17:16:45 -0000 --Apple-Mail=_8329BE43-0487-4FB3-93BD-6CADD4FFA9A7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-7 On Jul 8, 2014, at 10:04 AM, Sean Bruno wrote: > On Tue, 2014-07-08 at 08:03 -0700, Sean Bruno wrote: >>> Sean, can you see if this works for you now? And sorry about the >> cumbersome options. Those are in line to get fixed, but I wanna fix >> the build-in-tree issues first (which, alas, are harder than you=A2d >> think). I suspect that bapt=A2s src.opts.mk diffs are likely good >> candidates to be used committed too, but the above works w/o it. = Also, >> I just built the binaries, I didn=A2t see if they worked. >>>=20 >>> Warner=20 >=20 > Ok, it seems like we've come full circle. :-) I'm once again back at > include failures, for libstdc++: >=20 > make -s xdev MAKEOBJDIRPREFIX=3D/var/tmp XDDESTDIR=3D/var/tmp/mips_cc > XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt = WITHOUT_CLANG_BOOTSTRAP=3Dt > WITH_GCC=3Dt WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt > ... > =3D=3D=3D> secure/lib/libssl (obj,depend,all,install) > =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file = or > directory > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:38, > from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/bits/functexcept.h:41:28: error: bits/c++config.h: No such > file or directory > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/bitmap_allocator.h:39:37: error: utility: No such file = or > directory > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/bitmap_allocator.h:40:60: error: functional: No such = file > or directory > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:43, > from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/concurrence.h:39:19: error: cstdlib: No such file or > directory > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/ext/concurrence.h:41:24: error: bits/gthr.h: No such file = or > directory > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:38, > from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/include/bits/functexcept.h:44: error: expected constructor, > destructor, or type conversion before '(' token > In file included from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/libsupc++/new:45, > from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/include/ext/bitmap_allocator.h:41, > from /home/sbruno/fbsd_head/gnu/lib/libstdc > ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/libsupc++/exception:40: error: '#pragma' is not allowed here > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > ++/libsupc++/exception:133: error: expected declaration before end of > line > *** Error code 1 >=20 > Stop. > make[4]: stopped in /home/sbruno/fbsd_head/gnu/lib/libstdc++ > *** Error code 1 Does it work if you add WITHOUT_CXX=3Dt to the mix? Warner --Apple-Mail=_8329BE43-0487-4FB3-93BD-6CADD4FFA9A7 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvCdwAAoJEGwc0Sh9sBEAJGwP/j+Fy15MxmOQl6rJXFvtFm0m TYmwu5DaaHRqGbWyzjSwDKmcur1BiXtF1VUS9xZoCrnB/7qHAQuB8LNDtRvyuo2T V5jb5eYxyYUMuKkiV9JnWNbILOrytq9RZ/9ISyQlDNkTaAUZyA+BLZE8BHrLBBYP VjGspFKYOOlFQZI6SBwlq+0iO7utuYpJTaJTDwz0y03GyYiZMpYhDHN1AkZHIgTN tNYxOr6jD16w+JM5jH0FGqU2vF5s9VgWN+KVtJqzTfJuQJ/blNnexZ0QB1EXQH/o J7CtZBwpsBTZILPEw77za1krv9ag4h8namlE3n4VthtH3jcLnko2QcIH43YiwJlH PlLAXhJlc17Zu6euokP/6pLx1YUJ73EzTlN9/66g86J4m9bd6H/kJf9wuF81JG7P 7OPK9z05Lds0QrXTQFqxkH6JcehscsIR2qoeRp2E9fE66gQhh8svrBOyvzR6RlmB zLNfb/6Wtdfje0akwkp2nfyP8EToCm8O4G5n9BzCBp6c8H7be+pEXl65wmV0WsqZ U/ROfOxsQrja2mQOpCD5isZPLO+dUr6Vl8zJ/imveoysEXIiCEW4QwyzWV1MN3rg EcHQpFJBLxZ10a/zLEFXlasS8DlUi5QS5wRRf6O5Za8OQGw6mQCz5m3DSqwxL6hY U1K8TqlmAVVXn/ifc3P7 =dgmW -----END PGP SIGNATURE----- --Apple-Mail=_8329BE43-0487-4FB3-93BD-6CADD4FFA9A7-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 17:37:06 2014 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82A4FECA; Tue, 8 Jul 2014 17:37:06 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9AB2C2CA9; Tue, 8 Jul 2014 17:37:05 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::1133:f181:93e3:e668] (unknown [IPv6:2001:7b8:3a7:0:1133:f181:93e3:e668]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 0AF795C44; Tue, 8 Jul 2014 19:37:01 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_CA779C27-4913-4607-9D9A-2F4D1800CFF6"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: Date: Tue, 8 Jul 2014 19:36:48 +0200 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> To: Warner Losh X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 17:37:06 -0000 --Apple-Mail=_CA779C27-4913-4607-9D9A-2F4D1800CFF6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 08 Jul 2014, at 18:23, Warner Losh wrote: >=20 > On Jul 8, 2014, at 9:04 AM, Warner Losh wrote: >=20 >>=20 >> On Jul 8, 2014, at 12:56 AM, Dimitry Andric wrote: >>=20 >>> On 08 Jul 2014, at 03:56, Warner Losh wrote: >>>=20 >>>>=20 >>>> On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >>>>>=20 >>>>> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >>>>> likely the problem that should be fixed in a different way. It is = really an internal >>>>> variable that should be set based on the actual compiler type = (possibly with an >>>>> override for the odd-duck pair of clang and libstdc++ which may = not be worth >>>>> supporting). It is telling us we=92re doing something horribly = wrong and we should listen >>>>> to that rather than add another compiler-related kludge to the = build system. I=92ll work >>>>> on that bit. >>>>=20 >>>> Perhaps >>>> http://people.freesbd.org/~imp/patch-queue/86gnucxx >>>> might be the best way to cope=85 >>>>=20 >>>> Comments? >>>=20 >>> This would make it impossible to build libstdc++ with clang, and why = remove MK_GNUCXX at all[1]? >>=20 >> Because it is a silly option that=92s mostly an internal knob? We = don=92t need to support options that trip us up at every turn, and = MK_GNUCXX has been doing that since its introduction. >=20 > Also, in the current tree it means different things in different = places. In some places it says to build libstdc++, in other places it = says to build g++ and friends. This was an oddity introduced by theraven@ in r255321, where he = introduced MK_GNUCXX, but I have no idea why he conflated the two. = Before this commit, I had a local patch which simply added an option = disabling or enabling libstdc++ and libsupc++ (and nothing else). I = should have committed that first... :) In any case, it would be nice to still have the option to enable = building libstdc++, whatever the used compiler is. -Dimitry --Apple-Mail=_CA779C27-4913-4607-9D9A-2F4D1800CFF6 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlO8LDcACgkQsF6jCi4glqNInQCfYZrYReFzJL/ORfNtyu9jN2uW YEoAnidn+G/GSflB3eNcOtCWxiQMEG9T =TOYl -----END PGP SIGNATURE----- --Apple-Mail=_CA779C27-4913-4607-9D9A-2F4D1800CFF6-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 18:05:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 31680674 for ; Tue, 8 Jul 2014 18:05:23 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0E8B22F5A for ; Tue, 8 Jul 2014 18:05:22 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 931D0193DD9; Tue, 8 Jul 2014 18:05:20 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Tue, 08 Jul 2014 11:05:19 -0700 Message-ID: <1404842719.1662.15.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 18:05:23 -0000 On Tue, 2014-07-08 at 11:16 -0600, Warner Losh wrote: > > ++/libsupc++/exception:40: error: '#pragma' is not allowed here > > /home/sbruno/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > > ++/libsupc++/exception:133: error: expected declaration before end > of > > line > > *** Error code 1 > > > > Stop. > > make[4]: stopped in /home/sbruno/fbsd_head/gnu/lib/libstdc++ > > *** Error code 1 > > Does it work if you add WITHOUT_CXX=t to the mix? > > Warner > > So, it did not error out there. But this popped up. make xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t WITHOUT_CXX=t .......... ===> lib/libpcap (all) building static pcap library ranlib -D libpcap.a cc -isystem /var/tmp/mips_cc/usr/include -L/var/tmp/mips_cc/usr/lib --sysroot=/var/tmp/mips_cc/ -B/var/tmp/mips_cc/usr/libexec -B/var/tmp/mips_cc/usr/bin -B/var/tmp/mips_cc/usr/lib -fpic -DPIC -O -pipe -G0 -DHAVE_CONFIG_H -Dyylval=pcapyylval -I/home/sbruno/fbsd_head/lib/libpcap -I. -D_U_="__attribute__((unused))" -DHAVE_SNPRINTF -DHAVE_VSNPRINTF -DINET6 -DHAVE_NET_PFVAR_H -I/home/sbruno/fbsd_head/lib/libpcap/../../contrib/libpcap -std=gnu99 -c scanner.c -o scanner.So building shared library libpcap.so.8 ===> lib/libpmc (all) ===> lib/libproc (all) building static proc library ranlib -D libproc.a make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a building shared library libproc.so.2 /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ *** Error code 1 Stop. make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc *** Error code 1 From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 18:27:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 44287AA5; Tue, 8 Jul 2014 18:27:26 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F12E4218C; Tue, 8 Jul 2014 18:27:25 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::1133:f181:93e3:e668] (unknown [IPv6:2001:7b8:3a7:0:1133:f181:93e3:e668]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 1ECE45C44; Tue, 8 Jul 2014 20:27:23 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_CC11B1B8-AFE9-4863-87F5-2B108B55FE6E"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: <1404842719.1662.15.camel@bruno> Date: Tue, 8 Jul 2014 20:27:02 +0200 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 18:27:26 -0000 --Apple-Mail=_CC11B1B8-AFE9-4863-87F5-2B108B55FE6E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 08 Jul 2014, at 20:05, Sean Bruno wrote: ... > =3D=3D=3D> lib/libproc (all) > building static proc library > ranlib -D libproc.a > make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: > ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a > building shared library libproc.so.2 > /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ > *** Error code 1 >=20 > Stop. > make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc > *** Error code 1 Yes, libproc and it dependencies should be disabled when MK_CXX=3Dno. = Alternatively, libproc's demangling support could be conditionally = compiled out in that case. -Dimitry --Apple-Mail=_CC11B1B8-AFE9-4863-87F5-2B108B55FE6E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlO8OAYACgkQsF6jCi4glqOu5wCgx6yXN6/T/uk1Hv4RcZd1hci3 4jcAoJiNoSY8I3LMcrmWCrcIbUjejGwr =9PTP -----END PGP SIGNATURE----- --Apple-Mail=_CC11B1B8-AFE9-4863-87F5-2B108B55FE6E-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 19:02:48 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CA776D8; Tue, 8 Jul 2014 19:02:48 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D07302532; Tue, 8 Jul 2014 19:02:47 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::1133:f181:93e3:e668] (unknown [IPv6:2001:7b8:3a7:0:1133:f181:93e3:e668]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 75A255C44; Tue, 8 Jul 2014 21:02:44 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_AA1F36B9-C83B-439A-9C45-69678C31634B"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Dimitry Andric In-Reply-To: Date: Tue, 8 Jul 2014 21:02:25 +0200 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 19:02:48 -0000 --Apple-Mail=_AA1F36B9-C83B-439A-9C45-69678C31634B Content-Type: multipart/mixed; boundary="Apple-Mail=_DDDFE94F-1E3D-46CD-A6C8-60223B9C4CC6" --Apple-Mail=_DDDFE94F-1E3D-46CD-A6C8-60223B9C4CC6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 08 Jul 2014, at 20:27, Dimitry Andric wrote: > On 08 Jul 2014, at 20:05, Sean Bruno wrote: > ... >> =3D=3D=3D> lib/libproc (all) >> building static proc library >> ranlib -D libproc.a >> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: >> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a >> building shared library libproc.so.2 >> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ >> *** Error code 1 >>=20 >> Stop. >> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc >> *** Error code 1 >=20 > Yes, libproc and it dependencies should be disabled when MK_CXX=3Dno. = Alternatively, libproc's demangling support could be conditionally = compiled out in that case. Now with a suggested patch. -Dimitry --Apple-Mail=_DDDFE94F-1E3D-46CD-A6C8-60223B9C4CC6 Content-Disposition: attachment; filename=libproc-no-cxx-1.diff Content-Type: application/octet-stream; name="libproc-no-cxx-1.diff" Content-Transfer-Encoding: 7bit Index: lib/libproc/Makefile =================================================================== --- lib/libproc/Makefile (revision 268420) +++ lib/libproc/Makefile (working copy) @@ -15,7 +15,9 @@ INCS= libproc.h CFLAGS+= -I${.CURDIR} -.if ${MK_LIBCPLUSPLUS} != "no" +.if ${MK_CXX} == "no" +CFLAGS+= -DNO_CXA_DEMANGLE +.elif ${MK_LIBCPLUSPLUS} != "no" LDADD+= -lcxxrt DPADD+= ${LIBCXXRT} .else Index: lib/libproc/proc_sym.c =================================================================== --- lib/libproc/proc_sym.c (revision 268420) +++ lib/libproc/proc_sym.c (working copy) @@ -46,7 +46,9 @@ #include "_libproc.h" +#ifndef NO_CXA_DEMANGLE extern char *__cxa_demangle(const char *, char *, size_t *, int *); +#endif /* NO_CXA_DEMANGLE */ static void proc_rdl2prmap(rd_loadobj_t *, prmap_t *); @@ -53,20 +55,25 @@ static void proc_rdl2prmap(rd_loadobj_t *, prmap_t static void demangle(const char *symbol, char *buf, size_t len) { +#ifndef NO_CXA_DEMANGLE char *dembuf; - size_t demlen = len; + size_t demlen; - dembuf = malloc(len); - if (!dembuf) - goto fail; - dembuf = __cxa_demangle(symbol, dembuf, &demlen, NULL); - if (!dembuf) - goto fail; - strlcpy(buf, dembuf, len); - free(dembuf); + if (symbol[0] == '_' && symbol[1] == 'Z' && symbol[2]) { + dembuf = malloc(len); + if (!dembuf) + goto fail; + demlen = len; + dembuf = __cxa_demangle(symbol, dembuf, &demlen, NULL); + if (!dembuf) + goto fail; + strlcpy(buf, dembuf, len); + free(dembuf); + } return; fail: +#endif /* NO_CXA_DEMANGLE */ strlcpy(buf, symbol, len); } @@ -297,10 +304,7 @@ proc_addr2sym(struct proc_handle *p, uintptr_t add if (addr >= rsym && addr < rsym + sym.st_size) { s = elf_strptr(e, dynsymstridx, sym.st_name); if (s) { - if (s[0] == '_' && s[1] == 'Z' && s[2]) - demangle(s, name, namesz); - else - strlcpy(name, s, namesz); + demangle(s, name, namesz); memcpy(symcopy, &sym, sizeof(sym)); /* * DTrace expects the st_value to contain @@ -335,10 +339,7 @@ symtab: if (addr >= rsym && addr < rsym + sym.st_size) { s = elf_strptr(e, symtabstridx, sym.st_name); if (s) { - if (s[0] == '_' && s[1] == 'Z' && s[2]) - demangle(s, name, namesz); - else - strlcpy(name, s, namesz); + demangle(s, name, namesz); memcpy(symcopy, &sym, sizeof(sym)); /* * DTrace expects the st_value to contain --Apple-Mail=_DDDFE94F-1E3D-46CD-A6C8-60223B9C4CC6 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_DDDFE94F-1E3D-46CD-A6C8-60223B9C4CC6-- --Apple-Mail=_AA1F36B9-C83B-439A-9C45-69678C31634B Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlO8QFIACgkQsF6jCi4glqPVOQCgmjvieOSAhQ4iipguo7D2/wpK EU4An3/eu3Dkh7aWX8CVekA4d6m2jdOc =0MkF -----END PGP SIGNATURE----- --Apple-Mail=_AA1F36B9-C83B-439A-9C45-69678C31634B-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 20:06:39 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 484B6703 for ; Tue, 8 Jul 2014 20:06:39 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 319312BD3 for ; Tue, 8 Jul 2014 20:06:39 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s68K6dY3081004 for ; Tue, 8 Jul 2014 20:06:39 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.9/8.14.9/Submit) id s68K6cVv081003 for arch@FreeBSD.org; Tue, 8 Jul 2014 20:06:38 GMT (envelope-from bdrewery) Received: (qmail 17027 invoked from network); 8 Jul 2014 15:06:35 -0500 Received: from unknown (HELO blah) (freebsd@shatow.net@67.182.131.225) by sweb.xzibition.com with ESMTPA; 8 Jul 2014 15:06:35 -0500 Message-ID: <53BC4F49.7000903@FreeBSD.org> Date: Tue, 08 Jul 2014 15:06:33 -0500 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: sys/proc.h inclusion of sys/time.h Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:06:39 -0000 In r34924 sys/proc.h was changed to only include sys/time.h if not building in kernel. However, as the comment next to time.h says itimerval is needed. struct proc { .. struct itimerval p_realtimer; /* (c) Alarm timer. */ This manifests when (hackishly) including sys/proc.h with _KERNEL defined: > In file included from /root/svn/base/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/print-pflog.c:37: > /usr/include/sys/proc.h:524:19: error: field has incomplete type 'struct itimerval' > struct itimerval p_realtimer; /* (c) Alarm timer. */ (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I am testing that is intended for upstreaming. Perhaps I can use kern.pid_max in __FreeBSD__ and other hacks on other platforms, I have not yet decided on this.) Should we move the inclusion of sys/time.h outside of this ifdef or just add a forward declaration for struct itimerval above struct proc like many others? -- Regards, Bryan Drewery From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 20:28:02 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C6BFD95; Tue, 8 Jul 2014 20:28:02 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12BF52DC2; Tue, 8 Jul 2014 20:28:01 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 1ACF1193DD9; Tue, 8 Jul 2014 20:28:00 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Dimitry Andric In-Reply-To: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> Content-Type: text/plain; charset="us-ascii" Date: Tue, 08 Jul 2014 13:27:58 -0700 Message-ID: <1404851278.1662.17.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:28:02 -0000 On Tue, 2014-07-08 at 21:02 +0200, Dimitry Andric wrote: > On 08 Jul 2014, at 20:27, Dimitry Andric wrote: > > On 08 Jul 2014, at 20:05, Sean Bruno wrote: > > ... > >> ===> lib/libproc (all) > >> building static proc library > >> ranlib -D libproc.a > >> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: > >> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a > >> building shared library libproc.so.2 > >> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ > >> *** Error code 1 > >> > >> Stop. > >> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc > >> *** Error code 1 > > > > Yes, libproc and it dependencies should be disabled when MK_CXX=no. Alternatively, libproc's demangling support could be conditionally compiled out in that case. > > Now with a suggested patch. > > -Dimitry Getting closer, now we're at the point where we have some kind of path/permission failure: dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=/var/tmp DESTDIR=/var/tmp/mips_cc XDDESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t WITHOUT_CXX=t ===> secure/lib/libssh (install) sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 libssh.a /var/tmp/mips_cc/usr/lib/private sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 444 libssh.so.5 /var/tmp/mips_cc/usr/lib/private sh /home/sbruno/fbsd_head/tools/install.sh -l s libssh.so.5 /var/tmp/mips_cc/usr/lib/private/libssh.so ===> usr.bin/lex/lib (obj,depend,all,install) sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 libln.a /var/tmp/mips_cc/usr/lib /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i in *; do ln -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd-$i; ln -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd11.0-$i; done mkdir: ../../../../usr: Permission denied *** Error code 1 Stop. make[1]: stopped in /home/sbruno/fbsd_head *** Error code 1 Stop. make: stopped in /home/sbruno/fbsd_head From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 20:52:27 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB0647C8 for ; Tue, 8 Jul 2014 20:52:27 +0000 (UTC) Received: from mail-pa0-f44.google.com (mail-pa0-f44.google.com [209.85.220.44]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B52A2048 for ; Tue, 8 Jul 2014 20:52:26 +0000 (UTC) Received: by mail-pa0-f44.google.com with SMTP id rd3so7979402pab.31 for ; Tue, 08 Jul 2014 13:52:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=Ygi5XG2O9yCvMaBEzbbndO03rc9YeUKCoSfPwqGvPtY=; b=VBTlK2Cbs+8jRXABjFxnQaalw8Cg6mHI8s0opgYr0oR8IRw5+KP5nVF/U37rR5H/em w9P4111Rq2VNc3Q4TCR8KYFHshHFVQRzChkHeo563fsUd3KuS73Vso4fiJyuq3E7eGzZ XvyHC8HRkpC1k8RXIof4EukIu2fCrZAczWhI+f1jQd+LmxO/7ZaTHfDBQ5aaQQelP07o XFmGFeH0smS4fG2zKWMqzwPsS7WDAS5JERtixltQN00lKCK0wmchffdYWgKzrHlTofGm KGyW0QGieNtquR66Gkydpr2i2zq3r23DTTS/sPypvjbZu9WSrhBYC4l9EH1DxicIuqFO wweQ== X-Gm-Message-State: ALoCoQmWeF2flWrNlsvkVeHG/aCCLDLdyDXapebB2//LQGFvF+jCG3MACDanibha3FD0J8jBIEhV X-Received: by 10.68.239.231 with SMTP id vv7mr37235534pbc.10.1404852746192; Tue, 08 Jul 2014 13:52:26 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id vy8sm43188975pbc.84.2014.07.08.13.52.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 13:52:25 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_81E8B3CF-17C0-480A-B843-FAF9043D41F8"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: sys/proc.h inclusion of sys/time.h From: Warner Losh In-Reply-To: <53BC4F49.7000903@FreeBSD.org> Date: Tue, 8 Jul 2014 14:52:24 -0600 Message-Id: <95BA9E14-C369-4AC3-8D8A-51F1D16720C7@bsdimp.com> References: <53BC4F49.7000903@FreeBSD.org> To: Bryan Drewery X-Mailer: Apple Mail (2.1878.6) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:52:28 -0000 --Apple-Mail=_81E8B3CF-17C0-480A-B843-FAF9043D41F8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 2:06 PM, Bryan Drewery wrote: > In r34924 sys/proc.h was changed to only include sys/time.h if not = building in kernel. >=20 > However, as the comment next to time.h says itimerval is needed. >=20 > struct proc { > .. > struct itimerval p_realtimer; /* (c) Alarm timer. */ >=20 > This manifests when (hackishly) including sys/proc.h with _KERNEL = defined: >=20 >> In file included from = /root/svn/base/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/print-pfl= og.c:37: >> /usr/include/sys/proc.h:524:19: error: field has incomplete type = 'struct itimerval' >> struct itimerval p_realtimer; /* (c) Alarm timer. */ >=20 > (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I = am testing that is intended for upstreaming. Perhaps I can use = kern.pid_max in __FreeBSD__ and other hacks on other platforms, I have = not yet decided on this.) >=20 > Should we move the inclusion of sys/time.h outside of this ifdef or = just add a forward declaration for struct itimerval above struct proc = like many others? In the kernel, usually we just include sys/time.h before including = sys/proc.h. :) Traditionally, we haven=92t included all the pre-reqs and = left it to users of these files to do so. Should we change it? Not = without a good reason, imho, but honestly I think any reason better than = =93I think it would be cool=94 is likely good enough these days since = n^2 I/O behavior of always including all prereq files isn=92t as big a = deal as it once was... We can=92t do the forward struct declaration because by the time we get = around to defining (not using) struct proc, its size must be known. Warner --Apple-Mail=_81E8B3CF-17C0-480A-B843-FAF9043D41F8 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvFoIAAoJEGwc0Sh9sBEAs+IP/3dhH3seZ9b1z0yXMvUGoSna rpSslDLjKbtTdPELjbSwv2EmsCXEwUZpO+Ca0m9KpRwlhiaDr4qDOJ3vKAMthNOD L/gP9+n/D2mxAPZtzjsM2XDF8ualHnk22oJzxS3uKYEx0WNP0rT6fmHylGh9vT5v LUBasU7lUQltshTUoIdmBRQ9ZfNNhRKCxCwxRa268KixDAiKrstwBl1Ulq7RFFkq /FlJIEj/hmmYPrrMkB4d+8AtacmQUN6rASi84QYNSiDghTjNZj/Ojps06mQ/iNBZ tODE1E1YiXkWj3uzj+Fgg5qCj0gq+Jfs4iGBbyMxz0dIvnSep6IcEeQsljzdPDEs SkV0RRcwyrXba85lhFsyKXt7dngoxQteKjcyIbUg5UN5VotRAlDXBTPljIb0uDCy iOHYin82H5VpW5/9ZJAHGc7s4VwozOkjeFItCekr3mx1S5RHXLciyhflllTdNmmP NNEEJvfGuMxqwcgiV31nv77m4TbTpztPo9p72sqL5P7SVtGEoRRDi38R437yzgwj DfQVY7RXdJ0mWE9Y0OHPdjyrccARXRr3s5Ap/EgjPgMlAHgHVVXy5hfSViqAo54R iPNKMmFszpS7NzTOvkI/4ffrLOXxI3SYowkpKKmQJIuHlUsSMjS0GeqOYTqqkSJr T8FWawk4An8klEumyMMy =ax4H -----END PGP SIGNATURE----- --Apple-Mail=_81E8B3CF-17C0-480A-B843-FAF9043D41F8-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 20:55:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04F8B9B5 for ; Tue, 8 Jul 2014 20:55:51 +0000 (UTC) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C16C8206E for ; Tue, 8 Jul 2014 20:55:50 +0000 (UTC) Received: by mail-pa0-f48.google.com with SMTP id et14so7939853pad.21 for ; Tue, 08 Jul 2014 13:55:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=NKnSnG63j02uplhmREhA37MEhY5L8pGZz9mNyD5i6Eg=; b=PxVAVW3heqSbF4emPGZNInSI2tylYIH+21WfAWWyjqF3TGqM5sK+/ebrhPZl0GaPQ3 dHpwkoi5d4JDsPRcahkI2y+1vlUqQprISNBNpclObDC9L8citCu7qlVDe9iYLT/HhtFz pJhaRavQ1Sbs/1ujuIxNq+ai/4o1KuwoJG5TycaXtEQCRlrRhDGSNe25uuFjHztlKMms YagAoGaeqOkY86lz0q4bUewG8EebAM2kiiMcqmFtNFN5nQ8ThbX03bUd6KWRPc9DRPgT t3mMVrE79HO3TJBzi81LBaVuvaqrP9mVaUbvoghNredIJ2gn83e6SNI9yR87SelJDUgH qf8A== X-Gm-Message-State: ALoCoQnp4eP+Tro/S2IpkT1ivbM37S0pdsSe2tluFjdamQPeS8dr7BDW4p9knqda54ABJp8kAomF X-Received: by 10.68.200.133 with SMTP id js5mr5275854pbc.138.1404852950130; Tue, 08 Jul 2014 13:55:50 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id fo2sm56696634pbb.53.2014.07.08.13.55.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 13:55:49 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_B840BFED-6653-4DC6-B431-CB867A95C5C2"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: Date: Tue, 8 Jul 2014 14:55:48 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <67272C53-1908-454A-8E74-14D9A2EA0828@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1878.6) Cc: Baptiste Daroussin , sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:55:51 -0000 --Apple-Mail=_B840BFED-6653-4DC6-B431-CB867A95C5C2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 11:36 AM, Dimitry Andric wrote: > On 08 Jul 2014, at 18:23, Warner Losh wrote: >>=20 >> On Jul 8, 2014, at 9:04 AM, Warner Losh wrote: >>=20 >>>=20 >>> On Jul 8, 2014, at 12:56 AM, Dimitry Andric wrote: >>>=20 >>>> On 08 Jul 2014, at 03:56, Warner Losh wrote: >>>>=20 >>>>>=20 >>>>> On Jul 7, 2014, at 7:29 PM, Warner Losh wrote: >>>>>>=20 >>>>>> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >>>>>> likely the problem that should be fixed in a different way. It is = really an internal >>>>>> variable that should be set based on the actual compiler type = (possibly with an >>>>>> override for the odd-duck pair of clang and libstdc++ which may = not be worth >>>>>> supporting). It is telling us we=92re doing something horribly = wrong and we should listen >>>>>> to that rather than add another compiler-related kludge to the = build system. I=92ll work >>>>>> on that bit. >>>>>=20 >>>>> Perhaps >>>>> http://people.freesbd.org/~imp/patch-queue/86gnucxx >>>>> might be the best way to cope=85 >>>>>=20 >>>>> Comments? >>>>=20 >>>> This would make it impossible to build libstdc++ with clang, and = why remove MK_GNUCXX at all[1]? >>>=20 >>> Because it is a silly option that=92s mostly an internal knob? We = don=92t need to support options that trip us up at every turn, and = MK_GNUCXX has been doing that since its introduction. >>=20 >> Also, in the current tree it means different things in different = places. In some places it says to build libstdc++, in other places it = says to build g++ and friends. >=20 > This was an oddity introduced by theraven@ in r255321, where he = introduced MK_GNUCXX, but I have no idea why he conflated the two. = Before this commit, I had a local patch which simply added an option = disabling or enabling libstdc++ and libsupc++ (and nothing else). I = should have committed that first... :) Wish you had :) I=92ve taken various stabs at this since your feedback = on my initial change w/o being happy with the result. Will keep trying. > In any case, it would be nice to still have the option to enable = building libstdc++, whatever the used compiler is. Agreed. I=92ll make sure that you can specify this. I just get picky = about names and meanings :) Is it a requirement to build both libstdc++ = and libcxxrt or can it be either/or? Warner --Apple-Mail=_B840BFED-6653-4DC6-B431-CB867A95C5C2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvFrUAAoJEGwc0Sh9sBEA5mQQALMPzrLYnZmPzIQ1wiFndsh+ JE5/vGPXlCpJ0TOile4H/WF44qmqFT+vRuj8Ckix1gw5tsXp3oVdj2uMZMD1Plzd /EdtuXkNZiyXqQCv2QL59O73P1XgYJUcA7spr8pEBnKlnHOWhE0zR4hmol0JTfwS 8s1HVv4Kb+EuY4Ccoot3cH2oZH0T/LYS1MCvt7YsV1rWYiJGX8DIGkzD3CF5g3xS YmfIXaYMnzBV180rjtW7zYPHzjJu1kqhYo8ommTakX3YgXJ5gRGfQzhgrAAkojIR sroK2O9EQl/w8u54L7Yr0gmicqyyAi3C5FLFimqrWmpJH5ZQbyXM+auLLWunO4jW ylPcWYfyCk4iKkV5StM9sM6+NiHlvwWsRiOz8BRQaIk0eeSAfzi0ePoTgSLEpVuA N8NZUVerj5uelNIpnuqabL+ZgsaeQ4R50k7qrxBVdR2gVwB/TBYbtcYdwidM3+Uc 08Pifh1RKGJaCAAlwSj59/+AHOuVb8yjlZIWInGeqOFSkskjHsOoXomaFTFdNnzf fbay7ZUcrOhdfN4zXDgAefex28nr+A9m6I87Yz7dVY0JSt1hbCjuxGbKc1WDQLw/ OJIXHbEcKIvzZCe8Y/JvOvgQAbntXAn1I1r6yVLEZQXP7dv/C50PVe8msvF3N0se nQqMwnwFWVDkxWPDRg/o =M157 -----END PGP SIGNATURE----- --Apple-Mail=_B840BFED-6653-4DC6-B431-CB867A95C5C2-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 20:56:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBFC6A64 for ; Tue, 8 Jul 2014 20:56:42 +0000 (UTC) Received: from mail-pd0-f179.google.com (mail-pd0-f179.google.com [209.85.192.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6D9D207C for ; Tue, 8 Jul 2014 20:56:42 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id w10so7784295pde.38 for ; Tue, 08 Jul 2014 13:56:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=HXsLKbRsbp3YpZDVwbLHLQ3FpvLiXW5xZoIilt5zvRk=; b=QffmUOyBfzI2qRxqm1AYcVSXh7oYnizi5YCWCplLQg7YY7iR6w4WRnVI0fxJs/Sxwm 3umcgR5GpkJ5Dm4eo0e03eRc/NY+uDGKTlztBgz2PLGe+jZO3soY2437RqsRdnjdx8an KH2i9fAJQivXBrlV0UACoKPk/zXyTyCH/2SUV49+QmJ8t5+G6p2qG4j1/JAXUimv9Mkv VuWAVjXwoW5fG68Iigow2D1gPGeijnU1CV50kHrx+94zzmNdPqXBZUTntfTKMTYVcxmS 9MT0RHBLCR2C8KfzenfYeiegf5jgUqoQY/ng7xRb5XoTPg34oCsl3GOwQmE0VbVoo4Ys 6NKQ== X-Gm-Message-State: ALoCoQlMr3weDcb6ziLch61dGHeXPi6PxI0D1RFzSEE4zXodXy4x/6qepmBK7vycAlI0em8RPnx/ X-Received: by 10.68.176.5 with SMTP id ce5mr27405732pbc.93.1404853001803; Tue, 08 Jul 2014 13:56:41 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id fo2sm56696634pbb.53.2014.07.08.13.56.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 13:56:40 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_09853F3B-C8D6-466A-9075-03517B758FD9"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: Date: Tue, 8 Jul 2014 14:56:41 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> To: Dimitry Andric X-Mailer: Apple Mail (2.1878.6) Cc: sbruno@freebsd.org, freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 20:56:43 -0000 --Apple-Mail=_09853F3B-C8D6-466A-9075-03517B758FD9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Jul 8, 2014, at 1:02 PM, Dimitry Andric wrote: > On 08 Jul 2014, at 20:27, Dimitry Andric wrote: >> On 08 Jul 2014, at 20:05, Sean Bruno wrote: >> ... >>> =3D=3D=3D> lib/libproc (all) >>> building static proc library >>> ranlib -D libproc.a >>> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: >>> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a >>> building shared library libproc.so.2 >>> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ >>> *** Error code 1 >>>=20 >>> Stop. >>> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc >>> *** Error code 1 >>=20 >> Yes, libproc and it dependencies should be disabled when MK_CXX=3Dno. = Alternatively, libproc's demangling support could be conditionally = compiled out in that case. >=20 > Now with a suggested patch. >=20 > -Dimitry > I like this patch. Warner --Apple-Mail=_09853F3B-C8D6-466A-9075-03517B758FD9 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvFsJAAoJEGwc0Sh9sBEAS9QP/Rnw8SzEm21M3jHRpQzJv/jH Lk3aSTxgKDfiPXEjk2BlHJl7G7QGJcvIAcktJtjJyW8cglkuG2P3MPZL2mEG1Dp9 xAf0Y8T5t25YWOJYrK5c//GqdPw+CaR6Ai7Z0ePhsTPHLqPEphL0+X1BNLMnEtmP Mu9nkxw2VRJOSab+7IQsQcxykPoJ0Kux/m8w3AJff/jTfMHj3+kPUqObXBuy0ReA xqJQso90OJnSW0UAXtY+JgM7mKHrppiNPmUn7Be77KerH2xltzVnMcxPUITkQ2m4 KAGWh8ZiE/EOgF4K2arGYsZMpYGueR8LqezeWHUqDvKVDLjDylrKbJ+BpBXblkMv oNy8hha9mSzbQMIaTAh1fbrjZ6Us7DNvff6xAgFNkwyhPqpNmWnqzAFl00JvgIdE ijL1a2BlwP44fOeySb11LwA8nSxA2NbEGzlzGZQ4O2cY68fnKuOfcuCWsbQcE4U+ wN049mDZ/zXuI56cAkyJqvL/y5RDR/jKasraOFVpN6aiLJPt++vhDcqbz2KxPNcl mJC8+FGTzVIg3dtjb3ymTpBCaYlealFw8lAwyvASpKZFDT0CrKJix8QUZr3RkhDN 31lPol1Fj0BxI7SozkoQ4I+dwpLHLi+iXB4F07MZp6DDnXU9sxqy64NqBgMUuKCM 781KtUamorYsRsgmj0Il =hRJm -----END PGP SIGNATURE----- --Apple-Mail=_09853F3B-C8D6-466A-9075-03517B758FD9-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 21:01:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1981FB43 for ; Tue, 8 Jul 2014 21:01:30 +0000 (UTC) Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3FD42111 for ; Tue, 8 Jul 2014 21:01:29 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so7946780pab.36 for ; Tue, 08 Jul 2014 14:01:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=KEvSUgr90DbdLb70b6FkiF3E0PEUiQfcbRzXVCZeMwY=; b=Jmu5wU7TtSU9cA2NyvVX4Z+ADA4U2M1yMxR4axhOE2UX50kRpCqgs0NmJ4zbHZKMoP t00yOx/BCNT7TWZoplQbjCBuM6YDTjiIkWK0XLId4HzQ+oIZQMmB8shRuQlBAz9eCswg JKF9bf12B0glXL4R1jN+8lUh1ZsdwVJX3Cw8I7TbRR4D8F3LyAEdZ0XYdkMwKCSpaa0p 3os6oNxz178fVUvCR8lOgAVZrY2vI0UCMtPUKbyTk++j8HzCavJV6dSgPEkIpuL2z7YH +0IrzrZ79LXdBCmNCuwiU5Sg35yt0r75bIugDAx3e2JBJA3PvFXnMGWxvfYo9rk/xRpt NAEw== X-Gm-Message-State: ALoCoQnrgFWzxcXzSp2eup128JEg18AnKzcE8eO0SFbN9sm2plXsb2JWkDDoca8SthyMMIYgO+cx X-Received: by 10.68.201.68 with SMTP id jy4mr37200511pbc.64.1404853289229; Tue, 08 Jul 2014 14:01:29 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id oo3sm25448891pdb.19.2014.07.08.14.01.27 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 14:01:28 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_3B0CBAFE-51C4-4997-BFF6-175121588EDD"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404851278.1662.17.camel@bruno> Date: Tue, 8 Jul 2014 15:01:27 -0600 Message-Id: <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: Dimitry Andric , freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 21:01:30 -0000 --Apple-Mail=_3B0CBAFE-51C4-4997-BFF6-175121588EDD Content-Type: multipart/mixed; boundary="Apple-Mail=_705A3852-0E5D-411D-854E-5B4046E8C6C8" --Apple-Mail=_705A3852-0E5D-411D-854E-5B4046E8C6C8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 2:27 PM, Sean Bruno wrote: > On Tue, 2014-07-08 at 21:02 +0200, Dimitry Andric wrote: >> On 08 Jul 2014, at 20:27, Dimitry Andric wrote: >>> On 08 Jul 2014, at 20:05, Sean Bruno wrote: >>> ... >>>> =3D=3D=3D> lib/libproc (all) >>>> building static proc library >>>> ranlib -D libproc.a >>>> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: >>>> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a >>>> building shared library libproc.so.2 >>>> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ >>>> *** Error code 1 >>>>=20 >>>> Stop. >>>> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc >>>> *** Error code 1 >>>=20 >>> Yes, libproc and it dependencies should be disabled when MK_CXX=3Dno. = Alternatively, libproc's demangling support could be conditionally = compiled out in that case. >>=20 >> Now with a suggested patch. >>=20 >> -Dimitry >=20 >=20 > Getting closer, now we're at the point where we have some kind of > path/permission failure: >=20 > dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=3D/var/tmp > DESTDIR=3D/var/tmp/mips_cc XDDESTDIR=3D/var/tmp/mips_cc XDEV=3Dmips > XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt > WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt WITHOUT_CXX=3Dt >=20 > =3D=3D=3D> secure/lib/libssh (install) > sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 > libssh.a /var/tmp/mips_cc/usr/lib/private > sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 444 > libssh.so.5 /var/tmp/mips_cc/usr/lib/private > sh /home/sbruno/fbsd_head/tools/install.sh -l s > libssh.so.5 /var/tmp/mips_cc/usr/lib/private/libssh.so > =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 > libln.a /var/tmp/mips_cc/usr/lib > /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a > /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a > cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i in = *; > do ln > -sf ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd-$i; ln -sf = ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd11.0-$i; done > mkdir: ../../../../usr: Permission denied Oh! I know that one=85 That=92s from _xi-links target (the last one!). You can safely ignore = it. Something like the following would also eliminate the warning. Just not = too sure about it. You may also need to define WITH_INSTALL_AS_USER=3Dt. Warner --Apple-Mail=_705A3852-0E5D-411D-854E-5B4046E8C6C8 Content-Disposition: attachment; filename=P Content-Type: application/octet-stream; name="P" Content-Transfer-Encoding: 7bit diff -r 514f7a93a473 Makefile.inc1 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1968,6 +1968,7 @@ xdev-install: xdev-build _xi-mtree _xi-c DESTDIR=${XDDESTDIR} _xi-links: +.if ${MK_INSTALL_AS_USER} == "no" ${_+_}cd ${XDDESTDIR}/usr/bin; \ mkdir -p ../../../../usr/bin; \ for i in *; do \ @@ -1977,6 +1978,9 @@ xdev-install: xdev-build _xi-mtree _xi-c ../../../../usr/bin/${XDDIR}${OSREL}-$$i; \ done .else + @true +.endif +.else xdev xdev-build xdev-install: @echo "*** Error: Both XDEV and XDEV_ARCH must be defined for \"${.TARGET}\" target" .endif --Apple-Mail=_705A3852-0E5D-411D-854E-5B4046E8C6C8 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_705A3852-0E5D-411D-854E-5B4046E8C6C8-- --Apple-Mail=_3B0CBAFE-51C4-4997-BFF6-175121588EDD Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvFwoAAoJEGwc0Sh9sBEAZ1YP/jzNXQ0wgnZdasrLms+3FE6u wEC201TipW7/ZWEa6388/81niaPQAn6UF/DFxBpl/8jL2J1psNR3UKuUnoZQdg1G YELB/snVdoko61okFjhaP9DgsVPoAGB6GoGHJby0KLuYEgpbpNw394JiUM3xfH+g 0Ue5nmWWIqRZyfcwCdTxTzrpWIVSHQ0s1eYm5y96OO/QK9klet4tIZE6I5URlQj9 Kne/eZjxj/0jNPL/CUKjxul83lE7FapKGhkFp6tLkwdI/IzuYHYcst/cxHsm1nZf yXLhzro9zk1OOD1W1vTIqSXyOckt11szwbeU+nfsHDRSag2QL+2nuI7kr/dkNMbr CZn2+uG8drlp3kafCyKOZ24dfYhh1RkU/AqfCygOcRD7khJDbfBnkdolQxNTCowr rOHrkEBkHyu+D32u992bSfwVZZ2kI+AXyhVccFkbLGQuW9kLZmyg7EPoa6bs1KoO iCOqTEs4Ud92ISTaaJQL4vRDTUPF0dOWB3zW4OSfRPplpv9sqnOV1nu+zcYFwkGR wR0jg3qPxB+QY8xxzijXAbz1XSfdUOuqhAXYBd2swt5XjPxpVTrHD+h1v5ZhDm4Z jAVF9ftFnHRizUDLbVD+Rf9++N1HGdVY/BDqcwEuY843mIolL4owvNKVpts8T9cx uisMi1eUWacwZQf0nu+N =OZ5T -----END PGP SIGNATURE----- --Apple-Mail=_3B0CBAFE-51C4-4997-BFF6-175121588EDD-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 21:07:31 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 29FF6F03; Tue, 8 Jul 2014 21:07:31 +0000 (UTC) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E3753216E; Tue, 8 Jul 2014 21:07:30 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 8C6BAB801F; Tue, 8 Jul 2014 23:07:27 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id 7B7B828497; Tue, 8 Jul 2014 23:07:27 +0200 (CEST) Date: Tue, 8 Jul 2014 23:07:27 +0200 From: Jilles Tjoelker To: Bryan Drewery Subject: Re: sys/proc.h inclusion of sys/time.h Message-ID: <20140708210727.GA63071@stack.nl> References: <53BC4F49.7000903@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53BC4F49.7000903@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 21:07:31 -0000 On Tue, Jul 08, 2014 at 03:06:33PM -0500, Bryan Drewery wrote: > In r34924 sys/proc.h was changed to only include sys/time.h if not > building in kernel. > [snip] > (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I > am testing that is intended for upstreaming. Perhaps I can use > kern.pid_max in __FreeBSD__ and other hacks on other platforms, I have > not yet decided on this.) The kern.pid_max sysctl is mostly intended for running FreeBSD 1.0 binaries, which have a 16-bit pid_t. Therefore, it is run-time adjustable and existing processes may have a pid higher than its value. Ideally, you do not need PID_MAX and NO_PID; try to use a variable of type pid_t only for a process ID and store flags elsewhere. There may be a problem if you need to read pid_t from an internal structure or message, though. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 21:16:31 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF4811D0 for ; Tue, 8 Jul 2014 21:16:31 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A64202250 for ; Tue, 8 Jul 2014 21:16:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s68LGVKA008217 for ; Tue, 8 Jul 2014 21:16:31 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.9/8.14.9/Submit) id s68LGVTB008216 for arch@FreeBSD.org; Tue, 8 Jul 2014 21:16:31 GMT (envelope-from bdrewery) Received: (qmail 83794 invoked from network); 8 Jul 2014 16:16:29 -0500 Received: from unknown (HELO blah) (freebsd@shatow.net@67.182.131.225) by sweb.xzibition.com with ESMTPA; 8 Jul 2014 16:16:29 -0500 Message-ID: <53BC5FAA.4090501@FreeBSD.org> Date: Tue, 08 Jul 2014 16:16:26 -0500 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: arch@FreeBSD.org Subject: pflog/tcpdump/PID_MAX/NO_PID [was: sys/proc.h inclusion of sys/time.h] References: <53BC4F49.7000903@FreeBSD.org> <20140708210727.GA63071@stack.nl> In-Reply-To: <20140708210727.GA63071@stack.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 21:16:31 -0000 On 7/8/14, 4:07 PM, Jilles Tjoelker wrote: > On Tue, Jul 08, 2014 at 03:06:33PM -0500, Bryan Drewery wrote: >> In r34924 sys/proc.h was changed to only include sys/time.h if not >> building in kernel. >> [snip] >> (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I >> am testing that is intended for upstreaming. Perhaps I can use >> kern.pid_max in __FreeBSD__ and other hacks on other platforms, I have >> not yet decided on this.) > The kern.pid_max sysctl is mostly intended for running FreeBSD 1.0 > binaries, which have a 16-bit pid_t. Therefore, it is run-time > adjustable and existing processes may have a pid higher than its value. > > Ideally, you do not need PID_MAX and NO_PID; try to use a variable of > type pid_t only for a process ID and store flags elsewhere. There may be > a problem if you need to read pid_t from an internal structure or > message, though. > It's needed in this case as pflog will store NO_PID if no pid is stored for logged packets. This is 100000 on FreeBSD, 32767 on OpenBSD, 30001 on NetBSD. I am updating tcpdump to display logged uid/pid. Note that our pf does not log the pid ever right now and always uses NO_PID. I am going to explore what is needed to fix this. From talks I've had it seems there is a broad consensus that pf should not be using NO_PID in this case, but I am doubtful this will be fixed in all implementations. My goal here is just fixing tcpdump in an upstreamable way. Us changing pf to not use NO_PID only complicates my goal. While I could use a __FreeBSD__ ifdef it would only be valid on FreeBSD 11, earlier releases would still need the check on NO_PID. My current mockup is: > Index: ../../contrib/tcpdump/print-pflog.c > =================================================================== > --- ../../contrib/tcpdump/print-pflog.c (revision 268114) > +++ ../../contrib/tcpdump/print-pflog.c (working copy) > @@ -32,6 +32,10 @@ > #error "No pf headers available" > #endif > #include > +#include > +#define _KERNEL > +#include > +#undef _KERNEL > #include > #include > #include > @@ -101,12 +105,19 @@ > printf("rule %u/", rulenr); > else > printf("rule %u.%s.%u/", rulenr, hdr->ruleset, subrulenr); > - > - printf("%s: %s %s on %s: ", > - tok2str(pf_reasons, "unkn(%u)", hdr->reason), > + printf("%s", tok2str(pf_reasons, "unkn(%u)", hdr->reason)); > + if (vflag) > + printf("[uid %u, pid %u]", (unsigned)hdr->rule_uid, > + (unsigned)hdr->rule_pid); > + printf(": %s %s on %s: ", > tok2str(pf_actions, "unkn(%u)", hdr->action), > tok2str(pf_directions, "unkn(%u)", hdr->dir), > hdr->ifname); > + if (vflag && hdr->uid != UID_MAX) > + printf("[uid %u] ", (unsigned)hdr->uid); > + if (vflag && hdr->pid != NO_PID) > + printf("[pid %u] ", (unsigned)hdr->pid); > + /* XXX: pid is not displayed as we always have NO_PID > currently. */ > } -- Regards, Bryan Drewery From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 21:24:38 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F0B32435 for ; Tue, 8 Jul 2014 21:24:38 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB6D02329 for ; Tue, 8 Jul 2014 21:24:38 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 5FA48193DD9; Tue, 8 Jul 2014 21:24:37 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> Content-Type: text/plain; charset="windows-1251" Date: Tue, 08 Jul 2014 14:24:36 -0700 Message-ID: <1404854676.1662.29.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 21:24:39 -0000 On Tue, 2014-07-08 at 15:01 -0600, Warner Losh wrote: > On Jul 8, 2014, at 2:27 PM, Sean Bruno wrote: > > > On Tue, 2014-07-08 at 21:02 +0200, Dimitry Andric wrote: > >> On 08 Jul 2014, at 20:27, Dimitry Andric wrote: > >>> On 08 Jul 2014, at 20:05, Sean Bruno wrote: > >>> ... > >>>> ===> lib/libproc (all) > >>>> building static proc library > >>>> ranlib -D libproc.a > >>>> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: > >>>> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a > >>>> building shared library libproc.so.2 > >>>> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ > >>>> *** Error code 1 > >>>> > >>>> Stop. > >>>> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc > >>>> *** Error code 1 > >>> > >>> Yes, libproc and it dependencies should be disabled when MK_CXX=no. Alternatively, libproc's demangling support could be conditionally compiled out in that case. > >> > >> Now with a suggested patch. > >> > >> -Dimitry > > > > > > Getting closer, now we're at the point where we have some kind of > > path/permission failure: > > > > dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=/var/tmp > > DESTDIR=/var/tmp/mips_cc XDDESTDIR=/var/tmp/mips_cc XDEV=mips > > XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t > > WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t WITHOUT_CXX=t > > > > ===> secure/lib/libssh (install) > > sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 > > libssh.a /var/tmp/mips_cc/usr/lib/private > > sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m 444 > > libssh.so.5 /var/tmp/mips_cc/usr/lib/private > > sh /home/sbruno/fbsd_head/tools/install.sh -l s > > libssh.so.5 /var/tmp/mips_cc/usr/lib/private/libssh.so > > ===> usr.bin/lex/lib (obj,depend,all,install) > > sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m 444 > > libln.a /var/tmp/mips_cc/usr/lib > > /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a > > /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a > > cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i in *; > > do ln > > -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd-$i; ln -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd11.0-$i; done > > mkdir: ../../../../usr: Permission denied > > Oh! I know that one… > > That’s from _xi-links target (the last one!). You can safely ignore it. > > Something like the following would also eliminate the warning. Just not too sure about it. You may also need to define WITH_INSTALL_AS_USER=t. > > Warner > Ah crap, is there were I need "XDTP" defined or something? ===> usr.bin/lex/lib (obj,depend,all,install) sh /home/sbruno/fbsd_head/tools/install.sh -C -o sbruno -g devel -m 444 libln.a /var/tmp/mips_cc/usr/lib /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i in *; do ln -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd-$i; ln -sf ../..//usr/mips-freebsd/usr/bin/$i ../../../../usr/bin/mips-freebsd11.0-$i; done mkdir: ../../../../usr: Permission denied *** Error code 1 Stop. make[1]: stopped in /home/sbruno/fbsd_head *** Error code 1 Stop. make: stopped in /home/sbruno/fbsd_head sbruno@dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=/var/tmp DESTDIR=/var/tmp/mips_cc XDDESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t WITHOUT_CXX=t WITH_INSTALL_AS_USER=t sean From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 21:52:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09DA0D56 for ; Tue, 8 Jul 2014 21:52:37 +0000 (UTC) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C71F225AF for ; Tue, 8 Jul 2014 21:52:36 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id bj1so8037457pad.37 for ; Tue, 08 Jul 2014 14:52:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=AlxbqCCv23fc3NiQb/HAQsNJUmObc6u+bnbfnvevpMU=; b=Z8YW7ulxg19yoKstmZbwNu4TolBRKJJMbEYpt0LVIYKeOixBvwNxdWE7ycyT5oas22 YfkNc/TxHri/0NFLirYdADhaML4jB1Ntn+tphlbpsn7TXybPiIct2IP2Wd8jOW3dfApX VCCR3C6GL1kpYsilw/rUdDqqAWflu47e+Nshw57y23wIljcfIVi9uyCIDzMl0DTnEzpk y0LqgQO+euIWH2zKxW5IbvVxcIWSGJkTO7VyevaSfVmzXRXLCtdeOiYPBQ285+N8TB4P CkADzY6ZLfB7OYK8h1HqUBqmwIW0Vx8h73SgxxKN/xZfYC2r/FOcx2QjT4LLFLgt/KI2 k7DQ== X-Gm-Message-State: ALoCoQkzYmgYqNg5T1UVTFQ3SzZLajtvhhS2TlKZkhwdEoVBhHXyFbu6d32nC/KWJlUt+nWGfFUs X-Received: by 10.70.133.7 with SMTP id oy7mr3683677pdb.153.1404856356171; Tue, 08 Jul 2014 14:52:36 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ir10sm56770335pbc.59.2014.07.08.14.52.34 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 14:52:35 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_7FA79901-DC76-41E9-81DE-1CB4EDAC0C19"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404854676.1662.29.camel@bruno> Date: Tue, 8 Jul 2014 15:52:34 -0600 Message-Id: <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 21:52:37 -0000 --Apple-Mail=_7FA79901-DC76-41E9-81DE-1CB4EDAC0C19 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1251 On Jul 8, 2014, at 3:24 PM, Sean Bruno wrote: > On Tue, 2014-07-08 at 15:01 -0600, Warner Losh wrote: >> On Jul 8, 2014, at 2:27 PM, Sean Bruno = wrote: >>=20 >>> On Tue, 2014-07-08 at 21:02 +0200, Dimitry Andric wrote: >>>> On 08 Jul 2014, at 20:27, Dimitry Andric wrote: >>>>> On 08 Jul 2014, at 20:05, Sean Bruno = wrote: >>>>> ... >>>>>> =3D=3D=3D> lib/libproc (all) >>>>>> building static proc library >>>>>> ranlib -D libproc.a >>>>>> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, 322: >>>>>> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a >>>>>> building shared library libproc.so.2 >>>>>> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ >>>>>> *** Error code 1 >>>>>>=20 >>>>>> Stop. >>>>>> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc >>>>>> *** Error code 1 >>>>>=20 >>>>> Yes, libproc and it dependencies should be disabled when = MK_CXX=3Dno. Alternatively, libproc's demangling support could be = conditionally compiled out in that case. >>>>=20 >>>> Now with a suggested patch. >>>>=20 >>>> -Dimitry >>>=20 >>>=20 >>> Getting closer, now we're at the point where we have some kind of >>> path/permission failure: >>>=20 >>> dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=3D/var/tmp >>> DESTDIR=3D/var/tmp/mips_cc XDDESTDIR=3D/var/tmp/mips_cc XDEV=3Dmips >>> XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt >>> WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt WITHOUT_CXX=3Dt >>>=20 >>> =3D=3D=3D> secure/lib/libssh (install) >>> sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m = 444 >>> libssh.a /var/tmp/mips_cc/usr/lib/private >>> sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m = 444 >>> libssh.so.5 /var/tmp/mips_cc/usr/lib/private >>> sh /home/sbruno/fbsd_head/tools/install.sh -l s >>> libssh.so.5 /var/tmp/mips_cc/usr/lib/private/libssh.so >>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) >>> sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m = 444 >>> libln.a /var/tmp/mips_cc/usr/lib >>> /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a >>> /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a >>> cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i = in *; >>> do ln >>> -sf ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd-$i; ln -sf = ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd11.0-$i; done >>> mkdir: ../../../../usr: Permission denied >>=20 >> Oh! I know that one=85 >>=20 >> That=92s from _xi-links target (the last one!). You can safely = ignore it. >>=20 >> Something like the following would also eliminate the warning. Just = not too sure about it. You may also need to define = WITH_INSTALL_AS_USER=3Dt. >>=20 >> Warner >>=20 > Ah crap, is there were I need "XDTP" defined or something? Maybe=85 But it looks like it is still trying to do the links, so I = must have messed up something in the patch=85 And the links look kinda sketchy to me like it is reaching outside the = /usr/mips-freebsd area directly into /usr/bin, which won=92t matter for = what you are trying to do=85 You don=92t need mips-freebsd-cc binaries = in the chroot=85 though you might want different symlinks from your = chroot=92s /usr/bin/cc to /usr/mips-freebsd-cc/usr/bin/cc, etc. Warner > =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > sh /home/sbruno/fbsd_head/tools/install.sh -C -o sbruno -g devel -m = 444 > libln.a /var/tmp/mips_cc/usr/lib > /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a > /var/tmp/mips_cc/usr/lib/libfl.a -> /var/tmp/mips_cc/usr/lib/libln.a > cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i in = *; > do ln > -sf ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd-$i; ln -sf = ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd11.0-$i; done > mkdir: ../../../../usr: Permission denied > *** Error code 1 >=20 > Stop. > make[1]: stopped in /home/sbruno/fbsd_head > *** Error code 1 >=20 > Stop. > make: stopped in /home/sbruno/fbsd_head > sbruno@dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=3D/var/tmp > DESTDIR=3D/var/tmp/mips_cc XDDESTDIR=3D/var/tmp/mips_cc XDEV=3Dmips > XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt > WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt WITHOUT_CXX=3Dt = WITH_INSTALL_AS_USER=3Dt Maybe=85 But this step is optional=85 And I got the patch backwards :( --Apple-Mail=_7FA79901-DC76-41E9-81DE-1CB4EDAC0C19 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvGgiAAoJEGwc0Sh9sBEAvz0QAJGqmUQ66TGs0fmoTn7pNbHH yWIxytstPYXgyvhvrEk9ndSo1nUf6HMeM1JBNoau0Y11GDKHg9waTljzGZw9eCXN LPSbn5TJm3l8DBLQZilJPJRRAv0h8qI4Dw7qalibKoZwBGHzHJKnaQ5O78D0qG1J +ALF5mIEJtZsn/XCtB0mOJS5L4I3aLI3UEfgDPxZTJtOYT112RQZNx2/SJV/Cf/x 9Adh9roK1hR2mMvKzuFzlqNhUynIhQfjIA9HK5Ff2VHJfTztcQxDDAtvOIAaCBRA GGANyRDjdZPvrdNh1PJGReGfbSPoaD2tcZNtlNkOjuUk4rpWqAtJb7pae3c9qO3f cQelyYlYvkj2CvbNxinarCZB5E/ulOsC2biKbryZU+sibLRoAmShLAsF5Pf4VDO1 Ozigbz1xSgOYf7V1V+AsdVshaUKagZdocwYzAg5wIKN90dRu1X27Y4tDqQ3JhRMO r5p7/TWc/ErO/rzW5yM0yiKfWK1TlZ3hV07jradzbL3U7TyLRybHaKhc0cuPZmv9 822bQI5CiFC6VOkHlFd1LxXGrrqu3uESa6SbMrSdK3hAhoUGjVkh6+uHaJEa5+HT GDhDRjmvbLibi0mMmF1ILpefsNIGlGnc1U72PvmU8A8KHn1+6xQuouy7afW2Ihch XYDYn9DC5ni7+LtWp8Z8 =SUpu -----END PGP SIGNATURE----- --Apple-Mail=_7FA79901-DC76-41E9-81DE-1CB4EDAC0C19-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 22:00:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AE5DBA4 for ; Tue, 8 Jul 2014 22:00:26 +0000 (UTC) Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F8DB2631 for ; Tue, 8 Jul 2014 22:00:26 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id lj1so7962485pab.22 for ; Tue, 08 Jul 2014 15:00:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=0bnkZxjoY36Cu1dgeg5bodmVtmoQUuBr8u+9aNVx33M=; b=WDa/JoEXcnunXvmaya0MJDxz/gs3jffRhTWjCM23+bdTpq6ti5Q6dh73S0Q5MH1XI6 afG8w0akh1+Q0sgDeN72qsBAG9WigS0Yw75kIQeGJW2xaEfVKEASZmGIaP0NwhZ+FEzs xZJ6s9lbjlwk45Y3fis8G5x07spsckuyqZUgBcr+Jnr0GUAT9YGj05dQ98LXG4+mqLj5 8G162BVtYo/JoGfCKwRqyQ5iA5c9n7YkvpDfqxGYy+T3n923mRyzyPIjJdB09ZjjTvgI G8Ens68N43cW5t1155zMY+4SWJUTj8lYJ3YLT+5cvonIcGMylzdoNNOpTEHK06eICGXW /JGg== X-Gm-Message-State: ALoCoQmEt9o9OfFHJggHR9+5z3It0dTAqAWHhPAARNkS4datfzW8Nw0K7dZKDJbCJA8qQk6mAIeF X-Received: by 10.66.152.35 with SMTP id uv3mr37256705pab.74.1404856825250; Tue, 08 Jul 2014 15:00:25 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id ba10sm25740847pdb.73.2014.07.08.15.00.23 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 15:00:24 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_03EB9CF8-4AE5-4572-9716-139A549B68AA"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> Date: Tue, 8 Jul 2014 16:00:24 -0600 Message-Id: <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 22:00:26 -0000 --Apple-Mail=_03EB9CF8-4AE5-4572-9716-139A549B68AA Content-Type: multipart/mixed; boundary="Apple-Mail=_BADE264D-389F-40B1-BD4F-9C195823F9DD" --Apple-Mail=_BADE264D-389F-40B1-BD4F-9C195823F9DD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1251 On Jul 8, 2014, at 3:52 PM, Warner Losh wrote: >=20 > On Jul 8, 2014, at 3:24 PM, Sean Bruno wrote: >=20 >> On Tue, 2014-07-08 at 15:01 -0600, Warner Losh wrote: >>> On Jul 8, 2014, at 2:27 PM, Sean Bruno = wrote: >>>=20 >>>> On Tue, 2014-07-08 at 21:02 +0200, Dimitry Andric wrote: >>>>> On 08 Jul 2014, at 20:27, Dimitry Andric wrote: >>>>>> On 08 Jul 2014, at 20:05, Sean Bruno = wrote: >>>>>> ... >>>>>>> =3D=3D=3D> lib/libproc (all) >>>>>>> building static proc library >>>>>>> ranlib -D libproc.a >>>>>>> make[5]: /var/tmp/home/sbruno/fbsd_head/lib/libproc/.depend, = 322: >>>>>>> ignoring stale .depend for /var/tmp/mips_cc/usr/lib/libstdc++.a >>>>>>> building shared library libproc.so.2 >>>>>>> /var/tmp/mips_cc/usr/bin/ld: cannot find -lsupc++ >>>>>>> *** Error code 1 >>>>>>>=20 >>>>>>> Stop. >>>>>>> make[5]: stopped in /home/sbruno/fbsd_head/lib/libproc >>>>>>> *** Error code 1 >>>>>>=20 >>>>>> Yes, libproc and it dependencies should be disabled when = MK_CXX=3Dno. Alternatively, libproc's demangling support could be = conditionally compiled out in that case. >>>>>=20 >>>>> Now with a suggested patch. >>>>>=20 >>>>> -Dimitry >>>>=20 >>>>=20 >>>> Getting closer, now we're at the point where we have some kind of >>>> path/permission failure: >>>>=20 >>>> dirty.ysv:~/fbsd_head % make xdev MAKEOBJDIRPREFIX=3D/var/tmp >>>> DESTDIR=3D/var/tmp/mips_cc XDDESTDIR=3D/var/tmp/mips_cc XDEV=3Dmips >>>> XDEV_ARCH=3Dmips WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt = WITH_GCC=3Dt >>>> WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt WITHOUT_CXX=3Dt >>>>=20 >>>> =3D=3D=3D> secure/lib/libssh (install) >>>> sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m = 444 >>>> libssh.a /var/tmp/mips_cc/usr/lib/private >>>> sh /home/sbruno/fbsd_head/tools/install.sh -s -o root -g wheel -m = 444 >>>> libssh.so.5 /var/tmp/mips_cc/usr/lib/private >>>> sh /home/sbruno/fbsd_head/tools/install.sh -l s >>>> libssh.so.5 /var/tmp/mips_cc/usr/lib/private/libssh.so >>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) >>>> sh /home/sbruno/fbsd_head/tools/install.sh -C -o root -g wheel -m = 444 >>>> libln.a /var/tmp/mips_cc/usr/lib >>>> /var/tmp/mips_cc/usr/lib/libl.a -> /var/tmp/mips_cc/usr/lib/libln.a >>>> /var/tmp/mips_cc/usr/lib/libfl.a -> = /var/tmp/mips_cc/usr/lib/libln.a >>>> cd /var/tmp/mips_cc/usr/bin; mkdir -p ../../../../usr/bin; for i = in *; >>>> do ln >>>> -sf ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd-$i; ln -sf = ../..//usr/mips-freebsd/usr/bin/$i = ../../../../usr/bin/mips-freebsd11.0-$i; done >>>> mkdir: ../../../../usr: Permission denied >>>=20 >>> Oh! I know that one=85 >>>=20 >>> That=92s from _xi-links target (the last one!). You can safely = ignore it. >>>=20 >>> Something like the following would also eliminate the warning. Just = not too sure about it. You may also need to define = WITH_INSTALL_AS_USER=3Dt. >>>=20 >>> Warner >>>=20 >> Ah crap, is there were I need "XDTP" defined or something? >=20 > Maybe=85 But it looks like it is still trying to do the links, so I = must have messed up something in the patch=85 >=20 > And the links look kinda sketchy to me like it is reaching outside the = /usr/mips-freebsd area directly into /usr/bin, which won=92t matter for = what you are trying to do=85 You don=92t need mips-freebsd-cc binaries = in the chroot=85 though you might want different symlinks from your = chroot=92s /usr/bin/cc to /usr/mips-freebsd-cc/usr/bin/cc, etc. >=20 > Warner Try this instead: Warner --Apple-Mail=_BADE264D-389F-40B1-BD4F-9C195823F9DD Content-Disposition: attachment; filename=xdev-links Content-Type: application/octet-stream; x-unix-mode=0664; name="xdev-links" Content-Transfer-Encoding: 7bit # HG changeset patch # Parent 44a4c24f827bc29eb0ea023259e10a33961c8948 diff -r 44a4c24f827b Makefile.inc1 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -1943,8 +1943,8 @@ xdev-build: _xb-worldtmp _xb-bootstrap-t mtree -deU -f ${.CURDIR}/etc/mtree/BSD.include.dist \ -p ${XDDESTDIR}/usr/include >/dev/null -.ORDER: xdev-build _xi-mtree _xi-cross-tools _xi-includes _xi-libraries _xi-links -xdev-install: xdev-build _xi-mtree _xi-cross-tools _xi-includes _xi-libraries _xi-links +.ORDER: xdev-build _xi-mtree _xi-cross-tools _xi-includes _xi-libraries xdev-links +xdev-install: xdev-build _xi-mtree _xi-cross-tools _xi-includes _xi-libraries _xi-cross-tools: @echo "_xi-cross-tools" @@ -1967,9 +1967,9 @@ xdev-install: xdev-build _xi-mtree _xi-c ${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 libraries \ DESTDIR=${XDDESTDIR} -_xi-links: +xdev-links: ${_+_}cd ${XDDESTDIR}/usr/bin; \ - mkdir -p ../../../../usr/bin; \ + mkdir -p ../../../../usr/bin; \ for i in *; do \ ln -sf ../../${XDTP}/usr/bin/$$i \ ../../../../usr/bin/${XDDIR}-$$i; \ --Apple-Mail=_BADE264D-389F-40B1-BD4F-9C195823F9DD Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=windows-1251 --Apple-Mail=_BADE264D-389F-40B1-BD4F-9C195823F9DD-- --Apple-Mail=_03EB9CF8-4AE5-4572-9716-139A549B68AA Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvGn4AAoJEGwc0Sh9sBEA9D0P/0Itccba11zeTKc0LUQF0nrv 5RYo5Hm/rruJ4ndLUaQLOtbJbDRVVz8e0dWpRNlgWrhUojZgdrY8wwPb18s/W/kP 2xqeoZp7nlxxgmewQr8wZjaIt9SOKqbYtUDnp5bSHvzzo3KyXUjFAUbNFG9A8RL/ Ek7LOW23kyW1bUIB+n2U1fDaqJFZvGYPdcTy+F31o2PdvyQ7KSidKNCC7eQmXbL/ HDqf4kLn5vVLNZ2GavA7zyFGz8aL6cx0/PKOQzy4sF5IMcGBIICkGw4zPqx7DQZn Xs7w+39vobeyTwv0Mm37cOpKG3g9kms68nqABJ/AAyGWuazBXGYJJhlw47nDsoDe /AjQlsHMa4cqdM8emDsgixodC597y+atRzEeuZo88b4aSRjqXgOdFWQB+hty+L3X naeGlMoJj6ZpmhTQfF44Igr3xoqjCLhLlH4bfM5WiMyRdv0tqxib5+Tiao0LYXpP 6puy6YdqpYm2f7RR8GO+u74HNJvspt3H1oOxSGS7FtwrrW2G2H/BOp+4Ss/XIdSv An58QdrOKyJZnB2EVjGZNZTEbCC1ypDgJc+iO8/dqVFu+bSy77c9J2IbLME2bA4d ocbPWgg7lyb8cDeheY4DteZWQPKDeDwXJCHQBL/rqDTmik0xJGARD76vJ1FMGO3Y mtQK/VNrWhyqYCnkQwbs =0QFv -----END PGP SIGNATURE----- --Apple-Mail=_03EB9CF8-4AE5-4572-9716-139A549B68AA-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 22:54:45 2014 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EB1EDA7; Tue, 8 Jul 2014 22:54:45 +0000 (UTC) Received: from gw.catspoiler.org (gw.catspoiler.org [75.1.14.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 207EE2EF7; Tue, 8 Jul 2014 22:54:44 +0000 (UTC) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.13.3/8.13.3) with ESMTP id s68MsaPS028312; Tue, 8 Jul 2014 15:54:40 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201407082254.s68MsaPS028312@gw.catspoiler.org> Date: Tue, 8 Jul 2014 15:54:36 -0700 (PDT) From: Don Lewis Subject: Re: [patch] axe RF_TIMESHARE? To: adrian@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 22:54:45 -0000 On 6 Jul, Adrian Chadd wrote: > On 6 July 2014 17:47, Don Lewis wrote: >> On 6 Jul, Adrian Chadd wrote: >>> Hi, >>> >>> What's it supposed to be used for? >> >> My understanding it that it is supposed to be used to allow two more >> devices to claim the same resource, such as an I/O port range, but only >> one device can be active at a time. > > Interesting. I wonder what kinds of things would want to do this. Not much of anything that I can think of, which is probably why this feature was never used. The closest thing that I can think of is ISA, as Garrett mentioned. The thing that came to mind for me when I started looking at this is ISA attached COM ports. COM1 and COM3 both want to use IRQ 4, and COM2 and COM4 both want to use IRQ 3. The problem is that ISA IRQs can't be shared between slots, so it might be nice if RF_TIMESHARE could be used to share IRQ 3 between a modem card configured as COM4 and the COM2 serial port and then let the user pick the device to enable through software. Unfortunately this won't work because 16550-compatible UARTs don't have a way of disabling their IRQ pin drivers, so this has to be done with jumpers ... > There's a few interesting things like implementing the spibus using > this instead of the current way of calling a bus lock method before > doing any bus IO. The RF_TIMESHARE feature isn't a good fit for this because the resource being managed would be the address space on the SPI bus. If you had two slave devices that had the same starting SPI address, then you could use RF_TIMESHARE, but only if the address blocks occupied by both devices were also the same size (if not the resource manager considers the devices non-shareable), and there was some external way of enabling only one of the devices to listen to the SPI bus at a time. From owner-freebsd-arch@FreeBSD.ORG Tue Jul 8 23:29:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EE72AC50; Tue, 8 Jul 2014 23:29:51 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1blp0183.outbound.protection.outlook.com [207.46.163.183]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 357AE21F9; Tue, 8 Jul 2014 23:29:50 +0000 (UTC) Received: from BLUPR05CA005.namprd05.prod.outlook.com (10.255.219.163) by CO2PR05MB731.namprd05.prod.outlook.com (10.141.228.21) with Microsoft SMTP Server (TLS) id 15.0.980.8; Tue, 8 Jul 2014 23:29:48 +0000 Received: from BY2FFO11FD036.protection.gbl (2a01:111:f400:7c0c::157) by BLUPR05CA005.outlook.office365.com (2a01:111:e400:83f::35) with Microsoft SMTP Server (TLS) id 15.0.980.8 via Frontend Transport; Tue, 8 Jul 2014 23:29:48 +0000 Received: from P-EMF02-SAC.jnpr.net (66.129.239.16) by BY2FFO11FD036.mail.protection.outlook.com (10.1.14.221) with Microsoft SMTP Server (TLS) id 15.0.980.11 via Frontend Transport; Tue, 8 Jul 2014 23:29:47 +0000 Received: from magenta.juniper.net (172.17.27.123) by P-EMF02-SAC.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.146.0; Tue, 8 Jul 2014 16:29:47 -0700 Received: from chaos.jnpr.net (chaos.jnpr.net [172.21.16.28]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id s68NTkn78841; Tue, 8 Jul 2014 16:29:46 -0700 (PDT) (envelope-from sjg@juniper.net) Received: from chaos (localhost [127.0.0.1]) by chaos.jnpr.net (Postfix) with ESMTP id 5E97A580A2; Tue, 8 Jul 2014 16:29:46 -0700 (PDT) To: Warner Losh Subject: Re: Total confusion over toolchain/xdev behavior In-Reply-To: <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> Comments: In-reply-to: Warner Losh message dated "Mon, 07 Jul 2014 17:27:25 -0600." From: "Simon J. Gerraty" X-Mailer: MH-E 7.82+cvs; nmh 1.3; GNU Emacs 22.3.1 Date: Tue, 8 Jul 2014 16:29:46 -0700 Message-ID: <20140708232946.5E97A580A2@chaos.jnpr.net> MIME-Version: 1.0 Content-Type: text/plain X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:66.129.239.16; CTRY:US; IPV:NLI; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(6009001)(199002)(189002)(107046002)(57986006)(86362001)(101356003)(92726001)(102836001)(80022001)(33656002)(106466001)(70486001)(92566001)(21056001)(76482001)(93916002)(104166001)(76506005)(46102001)(62966002)(110136001)(81156004)(76176999)(50466002)(99396002)(87936001)(81342001)(97736001)(83322001)(85852003)(95666004)(6806004)(50226001)(47776003)(48376002)(105596002)(77156001)(64706001)(20776003)(89996001)(69596002)(84676001)(88136002)(102176002)(31966008)(74662001)(50986999)(93546004)(74502001)(85306003)(79102001)(44976005)(558084003)(90896003)(81542001)(4396001)(77982001)(83072002)(68736004)(87286001)(42262001); DIR:OUT; SFP:; SCL:1; SRVR:CO2PR05MB731; H:P-EMF02-SAC.jnpr.net; FPR:; MLV:sfv; PTR:InfoDomainNonexistent; MX:1; LANG:en; X-Microsoft-Antispam: BCL:0;PCL:0;RULEID: X-Forefront-PRVS: 0266491E90 Received-SPF: SoftFail (: domain of transitioning juniper.net discourages use of 66.129.239.16 as permitted sender) Authentication-Results: spf=softfail (sender IP is 66.129.239.16) smtp.mailfrom=sjg@juniper.net; X-OriginatorOrg: juniper.net Cc: sbruno@freebsd.org, Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2014 23:29:52 -0000 >OK. After some detective work, it looks like libstdc++ needs to be done = >before libsupc++ is done. I=92ve added this dependency in r268377 and = Not sure how much effort it saves you, but you can look at Makefile.depend files in project/bmake to see who depends on whom From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 00:06:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 311D233A; Wed, 9 Jul 2014 00:06:39 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66ADC25A7; Wed, 9 Jul 2014 00:06:38 +0000 (UTC) Received: by mail-wg0-f50.google.com with SMTP id x13so5622713wgg.33 for ; Tue, 08 Jul 2014 17:06:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=z4beO/t85BkxN7bBRn4uwFcF4nWkGvTin/IDkDyk1R4=; b=Wt8YyZ8s+NypViLSgo8rIxWCjs6KmCbiTu9NuPsN0Z4XnTSG81wxbVxM82q48adg9z 5k1bmPlvysZdsIA9MTudvGVVVdwkKbwNgG85h0ucoJ5u47xoWWXUMPVsXYzS8YYmg0kC VDGFG8w0bJFVL9GIjwRWrr4soEHPuYfsteI6rzMZBE0Sw5ErchTvTxbj2+aLFN32qOMM GJJbETqMMbHatWfBiO86vMcG0GFXxIaR392ZXaA86ujfYWNPLBQgjhUB37KJyexQYuCD 7C1MpbiOGrLDzx/mVHmoh0EiuDl9LP+9bfGKoU/UGG16NkH9vGPKQipVdYc2cC7LTRvc rxyQ== X-Received: by 10.180.102.100 with SMTP id fn4mr7605559wib.22.1404864396596; Tue, 08 Jul 2014 17:06:36 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id de6sm18751401wjc.16.2014.07.08.17.06.34 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Jul 2014 17:06:35 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 9 Jul 2014 02:06:33 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: Total confusion over toolchain/xdev behavior Message-ID: <20140709000628.GA56040@ivaldir.etoilebsd.net> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="nFreZHaLTZJo0R7j" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 00:06:39 -0000 --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote: >=20 > On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin wrote: >=20 > > On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: > >>=20 > >> On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: > >>=20 > >>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: > >>>> Objective: install an xcompile toolchain into a jail for use by > >>>> poudriere during arm/mips/sparc/power ports pkgs builds. The build > >>>> should be possible from a non-root user. > >>>>=20 > >>>> As far as I can tell, the xdev target is completely busted for non-c= lang > >>>> arch's right now as it tries to build clang no matter what I do. Its > >>>> missing some pretty key documentation to making it work correctly, s= o a > >>>> lot of my attempts have been "guess and check" with verbose make. > >>>>=20 > >>>> --------------------------------------------------------------------= --- > >>>> Attempt #1: > >>>> I have been trying non-root xdev builds: > >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips XDEV_ARCH= =3Dmips > >>>> -- dies because it tries to compile CLANG. > >>>> --------------------------------------------------------------------= --- > >>>>=20 > >>>> Attempt #2: > >>>> Apply a hack from Baptiste that isn't quite right, but at least gets > >>>> farther, note the missing variable causing "//usr/mips-freebsd" > >>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff > >>>>=20 > >>>> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) > >>>> mtree populating //usr/mips-freebsd > >>>> mkdir: //usr/mips-freebsd: Permission denied > >>>> *** Error code 1 > >>>> --------------------------------------------------------------------= --- > >>>>=20 > >>>> Attempt #3: Add XDTP > >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips > >>>> XDTP=3D/var/tmp/mips_cc > >>>>=20 > >>>> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch applied,= get's > >>>> a bit farther but compile failure in locating critical include files. > >>>>=20 > >>>> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) > >>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc > >>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > >>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc > >>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such fil= e or > >>>> directory > >>>> --------------------------------------------------------------------= --- > >>>>=20 > >>>> Attempt #4: Add the additional XDDESTDIR > >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dmips > >>>> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst > >>>> -- Same results as attempt #3 > >>>> --------------------------------------------------------------------= --- > >>>>=20 > >>>> Even attempting to do stuff for *clang* enabled architectures bails > >>>> because its not respecting prefixes: > >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm XDEV_ARCH= =3Darmv6 > >>>> -- bails because it tries to: > >>>> =3D=3D=3D> usr.bin/clang/tblgen (all) > >>>> mtree populating //usr/armv6-freebsd > >>>> mtree: etc/ntp: Permission denied > >>>> _xi-cross-tools > >>>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) > >>>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) > >>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > >>>> *** Error code 71 > >>>>=20 > >>>> --------------------------------------------------------------------= --- > >>>> Adding XDTP and XDDESTDIR results in a little more progress but obvi= ous > >>>> failures to attempt and install things directly into my host system: > >>>>=20 > >>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Darmv6 > >>>> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc > >>>> =3D=3D=3D> secure/lib/libssh (install) > >>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > >>>> mkdir: ../../../../usr: Permission denied > >>>> *** Error code 1 > >>>>=20 > >>>> Stop. > >>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head > >>>> *** Error code 1 > >>>>=20 > >>>> Stop. > >>>> make: stopped in /home/sbruno/bsd/fbsd_head > >>>=20 > >>> It looks to me like the permission part of the problem is being caused > >>> by a lack of DESTDIR=3D. Without that, it's trying to install to /us= r and > >>> you don't have permission for that. Maybe the confusion is because t= he > >>> xdev target inherently builds-and-installs, unlike most other targets > >>> that separate those two actions. > >>=20 > >> OK. After some detective work, it looks like libstdc++ needs to be don= e before libsupc++ is done. I=E2=80=99ve added this dependency in r268377 a= nd was able to do a full xdev build with a clean obj dir: > >>=20 > >> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd > >> mkdir $HOME/F > >> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CLAN= G=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt WITH_GCC_BOOTSTRAP=3Dt WITH_= GNUCXX=3Dt -j 20 > >=20 > > We can avoid most of the above by using a patch like the following: > > http://people.freebsd.org/~bapt/Makefile.inc1.diff > > Extending the same thing xi-cross-tools and xb-cross-tools (expect the > > WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho shoul= d.) >=20 > The patch looks good to my eye. Did you want me to expand it, or do you w= ant to > do the honors? >=20 > About the rest=E2=80=A6 Yea, you may be right=E2=80=A6. MK_GNUCXX is an = odd duck, and that=E2=80=99s > likely the problem that should be fixed in a different way. It is really = an internal > variable that should be set based on the actual compiler type (possibly w= ith an > override for the odd-duck pair of clang and libstdc++ which may not be wo= rth > supporting). It is telling us we=E2=80=99re doing something horribly wron= g and we should listen > to that rather than add another compiler-related kludge to the build syst= em. I=E2=80=99ll work > on that bit. >=20 > Also an aside: The horribly long command line was to try to get to the bo= ttom of > the breakage, not to promote it as the proper way of doing things. I haven't committed because it isn't complete yet :) I have updated it to the following: http://people.freebsd.org/~bapt/Makefile.inc1.diff but it is not enough as I end up with: /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++ So it goes further but it is not yet enough regards, Bapt --nFreZHaLTZJo0R7j Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlO8h4QACgkQ8kTtMUmk6Ez+VgCeLY/j9EGjjCLGFlD8cW9fm459 ijUAoKMiogHH5CHO0T2ysw3gR525xpiM =Bbmx -----END PGP SIGNATURE----- --nFreZHaLTZJo0R7j-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 00:20:24 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 803ABCF0 for ; Wed, 9 Jul 2014 00:20:24 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5973126ED for ; Wed, 9 Jul 2014 00:20:23 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 7541A193DD9; Wed, 9 Jul 2014 00:20:22 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: Warner Losh In-Reply-To: <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> Content-Type: text/plain; charset="windows-1251" Date: Tue, 08 Jul 2014 17:20:20 -0700 Message-ID: <1404865220.1662.42.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 00:20:24 -0000 > >>> That’s from _xi-links target (the last one!). You can safely ignore it. > >>> > >>> Something like the following would also eliminate the warning. Just not too sure about it. You may also need to define WITH_INSTALL_AS_USER=t. > >>> > >>> Warner > >>> > >> Ah crap, is there were I need "XDTP" defined or something? > > > > Maybe… But it looks like it is still trying to do the links, so I must have messed up something in the patch… > > > > And the links look kinda sketchy to me like it is reaching outside the /usr/mips-freebsd area directly into /usr/bin, which won’t matter for what you are trying to do… You don’t need mips-freebsd-cc binaries in the chroot… though you might want different symlinks from your chroot’s /usr/bin/cc to /usr/mips-freebsd-cc/usr/bin/cc, etc. > > > > Warner > > Try this instead: > > Warner > Ok, thanks to you guys for bearing with me. It looks like I can get the xdev target to compile utilizing this patch + dim's libproc-no-cxx-1.diff and bapt's orginal patch to bsd.src.mk with the following command: make -s -j8 xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/mips_cc XDEV=mips XDEV_ARCH=mips WITHOUT_CXX=t I'm going to try a couple of other arch's and see what else lies within. sean From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 03:26:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DEBCB668 for ; Wed, 9 Jul 2014 03:26:13 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B40EF294C for ; Wed, 9 Jul 2014 03:26:12 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id B7F1F193DD9 for ; Wed, 9 Jul 2014 03:26:10 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: freebsd-arch In-Reply-To: <1404865220.1662.42.camel@bruno> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> <1404865220.1662.42.camel@bruno> Content-Type: text/plain; charset="windows-1251" Date: Tue, 08 Jul 2014 20:26:09 -0700 Message-ID: <1404876369.1662.45.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 03:26:14 -0000 On Tue, 2014-07-08 at 17:20 -0700, Sean Bruno wrote: > > >>> That’s from _xi-links target (the last one!). You can safely ignore it. > > >>> > > >>> Something like the following would also eliminate the warning. Just not too sure about it. You may also need to define WITH_INSTALL_AS_USER=t. > > >>> > > >>> Warner > > >>> > > >> Ah crap, is there were I need "XDTP" defined or something? > > > > > > Maybe… But it looks like it is still trying to do the links, so I must have messed up something in the patch… > > > > > > And the links look kinda sketchy to me like it is reaching outside the /usr/mips-freebsd area directly into /usr/bin, which won’t matter for what you are trying to do… You don’t need mips-freebsd-cc binaries in the chroot… though you might want different symlinks from your chroot’s /usr/bin/cc to /usr/mips-freebsd-cc/usr/bin/cc, etc. > > > > > > Warner > > > > Try this instead: > > > > Warner > > > > > Ok, thanks to you guys for bearing with me. It looks like I can get the > xdev target to compile utilizing this patch + dim's > libproc-no-cxx-1.diff and bapt's orginal patch to bsd.src.mk with the > following command: > > make -s -j8 xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/mips_cc > XDEV=mips XDEV_ARCH=mips WITHOUT_CXX=t > > I'm going to try a couple of other arch's and see what else lies within. > > sean > > _______________________________________________ Final results of the day: For non-clang arches, mips, mip64, sparc64: make -s -j8 xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/sparc64_cc XDEV=sparc64 XDEV_ARCH=sparc64 WITHOUT_CXX=t For clang arches armv6: make -s -j8 xdev MAKEOBJDIRPREFIX=/var/tmp XDDESTDIR=/var/tmp/armv6_cc XDEV=arm XDEV_ARCH=armv6 All four arches selected compile and put "stuff" and "things" into XDDESTDIR and build as nonroot using the combined patches from dim, imp and bapt. What do you folks want to do next with the Mk changes in these patches? sean From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 03:48:51 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 042B5BFB for ; Wed, 9 Jul 2014 03:48:51 +0000 (UTC) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4C912A8D for ; Wed, 9 Jul 2014 03:48:50 +0000 (UTC) Received: by mail-pa0-f54.google.com with SMTP id et14so8507181pad.13 for ; Tue, 08 Jul 2014 20:48:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=hiRX8fpTgk7x3YDfHdixkaw7cIjTnXLsFMo9DgG8qcs=; b=abY+GNKd/JEd2linzuC80k3hL0XTgU+QnmgGDqWA4T2tbTwTEgj/VYBuXmnkGbVVg6 3eIr8DY5MSlYN9rUrbQe5xQiadgUgbW89s1DYBNCANJu4vfveHf2ufuCRmK6aoIV8jUH b1mOYMDRaNPt4jK7JuF4ZLeHtZ9Np7R5gAlOzxXN5jrOAjRbZJRw6BWTP4Lbt6CiFNMQ qVEoFvFXpwXH7mU3sGNGHsoDwheD5wFiNMW9TbPzvdcNdXNmfTBs/xhzqbbWUBqAZrZV N2nZrxyTBFOE/oRdLAyZI1DWYcaTHcq6N613x4hg77HoqjxANCrbJ/W0iTZ9EaUKSyqY +RAg== X-Gm-Message-State: ALoCoQk1OW9Sp/rSHsR+B9JI3xrTGDKjF9cUM20KFk+CiqWG60UT+MaGYjRUjqm04NLqkDXS1NM/ X-Received: by 10.68.102.3 with SMTP id fk3mr28754573pbb.95.1404877724783; Tue, 08 Jul 2014 20:48:44 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id cz3sm57335340pbc.9.2014.07.08.20.48.43 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 20:48:44 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_221413DC-DF81-4821-9955-7146E1086A2A"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <20140709000628.GA56040@ivaldir.etoilebsd.net> Date: Tue, 8 Jul 2014 21:48:45 -0600 Message-Id: References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <20140709000628.GA56040@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Mailer: Apple Mail (2.1878.6) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 03:48:51 -0000 --Apple-Mail=_221413DC-DF81-4821-9955-7146E1086A2A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 8, 2014, at 6:06 PM, Baptiste Daroussin wrote: > On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote: >>=20 >> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin = wrote: >>=20 >>> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: >>>>=20 >>>> On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: >>>>=20 >>>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: >>>>>> Objective: install an xcompile toolchain into a jail for use by >>>>>> poudriere during arm/mips/sparc/power ports pkgs builds. The = build >>>>>> should be possible from a non-root user. >>>>>>=20 >>>>>> As far as I can tell, the xdev target is completely busted for = non-clang >>>>>> arch's right now as it tries to build clang no matter what I do. = Its >>>>>> missing some pretty key documentation to making it work = correctly, so a >>>>>> lot of my attempts have been "guess and check" with verbose make. >>>>>>=20 >>>>>> = ----------------------------------------------------------------------- >>>>>> Attempt #1: >>>>>> I have been trying non-root xdev builds: >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips = XDEV_ARCH=3Dmips >>>>>> -- dies because it tries to compile CLANG. >>>>>> = ----------------------------------------------------------------------- >>>>>>=20 >>>>>> Attempt #2: >>>>>> Apply a hack from Baptiste that isn't quite right, but at least = gets >>>>>> farther, note the missing variable causing "//usr/mips-freebsd" >>>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff >>>>>>=20 >>>>>> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) >>>>>> mtree populating //usr/mips-freebsd >>>>>> mkdir: //usr/mips-freebsd: Permission denied >>>>>> *** Error code 1 >>>>>> = ----------------------------------------------------------------------- >>>>>>=20 >>>>>> Attempt #3: Add XDTP >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips = XDEV_ARCH=3Dmips >>>>>> XDTP=3D/var/tmp/mips_cc >>>>>>=20 >>>>>> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch = applied, get's >>>>>> a bit farther but compile failure in locating critical include = files. >>>>>>=20 >>>>>> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) >>>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc >>>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: >>>>>> = /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc >>>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such = file or >>>>>> directory >>>>>> = ----------------------------------------------------------------------- >>>>>>=20 >>>>>> Attempt #4: Add the additional XDDESTDIR >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips = XDEV_ARCH=3Dmips >>>>>> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst >>>>>> -- Same results as attempt #3 >>>>>> = ----------------------------------------------------------------------- >>>>>>=20 >>>>>> Even attempting to do stuff for *clang* enabled architectures = bails >>>>>> because its not respecting prefixes: >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm = XDEV_ARCH=3Darmv6 >>>>>> -- bails because it tries to: >>>>>> =3D=3D=3D> usr.bin/clang/tblgen (all) >>>>>> mtree populating //usr/armv6-freebsd >>>>>> mtree: etc/ntp: Permission denied >>>>>> _xi-cross-tools >>>>>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) >>>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied >>>>>> *** Error code 71 >>>>>>=20 >>>>>> = ----------------------------------------------------------------------- >>>>>> Adding XDTP and XDDESTDIR results in a little more progress but = obvious >>>>>> failures to attempt and install things directly into my host = system: >>>>>>=20 >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm = XDEV_ARCH=3Darmv6 >>>>>> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc >>>>>> =3D=3D=3D> secure/lib/libssh (install) >>>>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) >>>>>> mkdir: ../../../../usr: Permission denied >>>>>> *** Error code 1 >>>>>>=20 >>>>>> Stop. >>>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head >>>>>> *** Error code 1 >>>>>>=20 >>>>>> Stop. >>>>>> make: stopped in /home/sbruno/bsd/fbsd_head >>>>>=20 >>>>> It looks to me like the permission part of the problem is being = caused >>>>> by a lack of DESTDIR=3D. Without that, it's trying to install to = /usr and >>>>> you don't have permission for that. Maybe the confusion is = because the >>>>> xdev target inherently builds-and-installs, unlike most other = targets >>>>> that separate those two actions. >>>>=20 >>>> OK. After some detective work, it looks like libstdc++ needs to be = done before libsupc++ is done. I=92ve added this dependency in r268377 = and was able to do a full xdev build with a clean obj dir: >>>>=20 >>>> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd >>>> mkdir $HOME/F >>>> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips = WITHOUT_CLANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt = WITH_GCC_BOOTSTRAP=3Dt WITH_GNUCXX=3Dt -j 20 >>>=20 >>> We can avoid most of the above by using a patch like the following: >>> http://people.freebsd.org/~bapt/Makefile.inc1.diff >>> Extending the same thing xi-cross-tools and xb-cross-tools (expect = the >>> WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho = should.) >>=20 >> The patch looks good to my eye. Did you want me to expand it, or do = you want to >> do the honors? >>=20 >> About the rest=85 Yea, you may be right=85. MK_GNUCXX is an odd = duck, and that=92s >> likely the problem that should be fixed in a different way. It is = really an internal >> variable that should be set based on the actual compiler type = (possibly with an >> override for the odd-duck pair of clang and libstdc++ which may not = be worth >> supporting). It is telling us we=92re doing something horribly wrong = and we should listen >> to that rather than add another compiler-related kludge to the build = system. I=92ll work >> on that bit. >>=20 >> Also an aside: The horribly long command line was to try to get to = the bottom of >> the breakage, not to promote it as the proper way of doing things. >=20 > I haven't committed because it isn't complete yet :) > I have updated it to the following: > http://people.freebsd.org/~bapt/Makefile.inc1.diff > but it is not enough as I end up with: >=20 > /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++ >=20 > So it goes further but it is not yet enough I actually think xdev and a couple others should move to Makefile, and = the sub targets stay in Makefile.inc and be invoked with the right = TARGET and TARGET_ARCH so that we get the right defaults. Warner --Apple-Mail=_221413DC-DF81-4821-9955-7146E1086A2A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvLudAAoJEGwc0Sh9sBEARYAP/0vvb3MK5tyNPJtDNF3PB4u1 yHbzhSyHiMtxjWQrcqK5byvV7ZUUWMCE37jByCUME8s5YRwkG5a6io6qK2GEtJiW f0EM7RJxVFqB4vHt2QO3YmcFQ70bJi/mZrUv2BWdEoGm6/3yUFjqM0EGEKXSf4Qc P1yXN2XlzoMVShGoFAoXTNVX4Yb/mGOxSWtWgiO39GMpLy+CP1Qq851ACD/eKAfm 18IOzeuIOiPjXiZNs3XzNhkadWnzrCxXi2o6yBNtKaVpIvHlSKLsdfF8tw//rGkS L5mw5Qynyc3wNIOxsX1/DSgJJy+YMzQjeDdT3XgR3So8ly8iv85ZAuRHOnrtQGAl 9vIa2POcOa0mE04tG9+wvg5XshO5B7rK/Fv6rfUJsW1LhAWLreYlz0PBvu/VwjgE oBxnoejfBjfEYeWwNgzYNEPR6+lB/9wr1hFPreCxmKYxmUCNRsvw6huj0cAXq7fi quFCG8roto2iEh/IKasSl3aeCcJvWPr6U0wPBZkoxBOScFeVNWtACv//b/8jSJa/ 12mauWXBeOGVxNV7QvVtFpgzWlQBuN6Lym1so6hbqMJzjMputDV1ESNj1jDWDvKz O3QL+UeU1YW1SPDY3Gh/DxGgBAQ6o1eboLH6abqKab8LxVAnakpXIkg05sj/OOYj oZFw6gv2jXv4z+Bwrq44 =isxN -----END PGP SIGNATURE----- --Apple-Mail=_221413DC-DF81-4821-9955-7146E1086A2A-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 03:50:03 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C945ECAE for ; Wed, 9 Jul 2014 03:50:03 +0000 (UTC) Received: from mail-pa0-f45.google.com (mail-pa0-f45.google.com [209.85.220.45]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 92ABD2A99 for ; Wed, 9 Jul 2014 03:50:03 +0000 (UTC) Received: by mail-pa0-f45.google.com with SMTP id rd3so8434142pab.18 for ; Tue, 08 Jul 2014 20:49:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=qe734N/iNy7GyFJ6utRVQHXVZRIOa/GL3wdodLCDfI0=; b=cWiePnZmpNSRZLmybyg7x67IOYbsQ4k2eYox97nOR2Wi54+ZzipPaIdxxfsshpyIS1 jooTlISKajZEgg4NTV+mLp568qkbQhDUQxwppcIn2ejFlFAGHZkwMRn/pq8WrZMM2aPv TJ94/kVuR1OyWm4fjd1dttdh18xkiwOBZSpLexbJ2LcATLpx6fipkvnidFqndESo1obg csu1MqrUmQcQoL2XW9Du+UXMmHjt3WD+vFMX4Qrdx9buoHqTJo9DLyHVObampmJ1WhMt uQidBlKyWu+smFWbnci7VOJOMxh44GnqrQ5rnuD7YQzvISTsh9G46y1W4qDZfivELqjb eVkA== X-Gm-Message-State: ALoCoQkpZbOuXplSJ+oPD0HTitGMFo2FC/pTb90J9LTrydK+GWVc3k3BseRT9ETFTYMU0KKEZGn1 X-Received: by 10.68.143.231 with SMTP id sh7mr3686250pbb.7.1404877797569; Tue, 08 Jul 2014 20:49:57 -0700 (PDT) Received: from [10.64.24.152] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id yv7sm210491393pac.33.2014.07.08.20.49.56 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 08 Jul 2014 20:49:56 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_9BD3EEEA-BFC7-497A-ADBE-425C5D696B27"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Total confusion over toolchain/xdev behavior From: Warner Losh In-Reply-To: <1404876369.1662.45.camel@bruno> Date: Tue, 8 Jul 2014 21:49:58 -0600 Message-Id: <96AB65AF-F86F-4E35-8F94-8BE6730B28F9@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> <1404865220.1662.42.camel@bruno> <1404876369.1662.45.camel@bruno> To: sbruno@freebsd.org X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 03:50:03 -0000 --Apple-Mail=_9BD3EEEA-BFC7-497A-ADBE-425C5D696B27 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1251 On Jul 8, 2014, at 9:26 PM, Sean Bruno wrote: > On Tue, 2014-07-08 at 17:20 -0700, Sean Bruno wrote: >>>>>> That=92s from _xi-links target (the last one!). You can safely = ignore it. >>>>>>=20 >>>>>> Something like the following would also eliminate the warning. = Just not too sure about it. You may also need to define = WITH_INSTALL_AS_USER=3Dt. >>>>>>=20 >>>>>> Warner >>>>>>=20 >>>>> Ah crap, is there were I need "XDTP" defined or something? >>>>=20 >>>> Maybe=85 But it looks like it is still trying to do the links, so = I must have messed up something in the patch=85 >>>>=20 >>>> And the links look kinda sketchy to me like it is reaching outside = the /usr/mips-freebsd area directly into /usr/bin, which won=92t matter = for what you are trying to do=85 You don=92t need mips-freebsd-cc = binaries in the chroot=85 though you might want different symlinks from = your chroot=92s /usr/bin/cc to /usr/mips-freebsd-cc/usr/bin/cc, etc. >>>>=20 >>>> Warner >>>=20 >>> Try this instead: >>>=20 >>> Warner >>>=20 >>=20 >>=20 >> Ok, thanks to you guys for bearing with me. It looks like I can get = the >> xdev target to compile utilizing this patch + dim's >> libproc-no-cxx-1.diff and bapt's orginal patch to bsd.src.mk with the >> following command: >>=20 >> make -s -j8 xdev MAKEOBJDIRPREFIX=3D/var/tmp = XDDESTDIR=3D/var/tmp/mips_cc >> XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CXX=3Dt >>=20 >> I'm going to try a couple of other arch's and see what else lies = within. >>=20 >> sean >>=20 >> _______________________________________________ >=20 >=20 > Final results of the day: >=20 > For non-clang arches, mips, mip64, sparc64: > make -s -j8 xdev MAKEOBJDIRPREFIX=3D/var/tmp = XDDESTDIR=3D/var/tmp/sparc64_cc > XDEV=3Dsparc64 XDEV_ARCH=3Dsparc64 WITHOUT_CXX=3Dt >=20 > For clang arches armv6: > make -s -j8 xdev MAKEOBJDIRPREFIX=3D/var/tmp = XDDESTDIR=3D/var/tmp/armv6_cc > XDEV=3Darm XDEV_ARCH=3Darmv6 >=20 >=20 > All four arches selected compile and put "stuff" and "things" into > XDDESTDIR and build as nonroot using the combined patches from dim, = imp > and bapt. =20 >=20 > What do you folks want to do next with the Mk changes in these = patches? Let=92s try to get the changes reviewed to make sure we=92re not = stepping on each other=85 Why don=92t you kick that off with phabric = reviews? Warner --Apple-Mail=_9BD3EEEA-BFC7-497A-ADBE-425C5D696B27 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIbBAEBCgAGBQJTvLvmAAoJEGwc0Sh9sBEAHFsP+OTrBJba4rB5qA7kbpMcyzkG Pt1/AVOrR8HuJ2bDyEOksnu2DNkBpO30sX1N6i74ZWGEvdD7Gz5Z+cvsg/N7aypV XCMTIjDW7AyNFRRcLW4GGMwDLsdihoO9eXElnnIsocSToA+HMojPHOJ0u1hpIZoq FjVuLA08qghpZiQCI8Q8HeeAlGY2KcJAt4Co+Ale0LV7tm7opdIwdtVJE7XiR4ca kKeAEW0cd6MpxoSBxxcuzVdqPSEmWiXJRtGX3UtXIJNMbnuYSzFHQx/6jYx8HkVt My8dQ4+gFYDTTfF91fLXOkoeL3+xQmXCmfl69DafAUYQsuzeOP5xu9MTHNAndxG3 yyDAcKncqzdLFwj4Dqx8E67/AKvYsxP4tEWGivVaFbHEzc4xyulrIGO9iJrMwwBM wgs1IiEeSic6GiRsElImTSD9N7NtlMKISnkhj3uGm90Nfbkv7dZKuQyjcg/ujyWx Xeip1H1o2hvMvjiWvtVJOrfQDn53fMlCwdsv+8h1OXM+jiWmLg4wYcLw7mtBMP+5 Ng4NYjZzAwVtg5AoRMNdlKUyI7gYHdkKfkZdHd05X2j5dKUbizeJbCcyll0y6TYf TT2gn75iSInxwtFQWUq8YWIsI3wc/D/UaV2lpQjOV7OxU05598G4E2OiBDLGC8Cs THM2hvFIjE79c5Vp9Y8= =3DAu -----END PGP SIGNATURE----- --Apple-Mail=_9BD3EEEA-BFC7-497A-ADBE-425C5D696B27-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 06:09:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D65254F9; Wed, 9 Jul 2014 06:09:42 +0000 (UTC) Received: from mail-we0-x229.google.com (mail-we0-x229.google.com [IPv6:2a00:1450:400c:c03::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15158252A; Wed, 9 Jul 2014 06:09:41 +0000 (UTC) Received: by mail-we0-f169.google.com with SMTP id t60so6959890wes.14 for ; Tue, 08 Jul 2014 23:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=26SiAmeApzXlUKiTNg1fGQGH506lHyTCR+mNZNPx8Ks=; b=l5mi8LjBF4uk4vSRbbC67iIwhlLH9x9mukF1CA5SsjFsRJO6wQlSRDpvJi+sCmQbo6 nvDDLZGgiGbceFk4GMvitVKHEcyR69lcZ0ganNtPGvCKxGScKOw9vtHktL8M1bWoHI9T KUMndrA7iQCBcYWXPrmB9jH0KztRKGWwNDyqc4FEYRRlkO1U8Es+McdW5joX4vWjyqRg ywYRnYGXRUOg7+WQiVGi9zlg//f8JFsTEjmr4vnEd3qf4BMtmlK551l0KfefeV3oYvXW 2bKPVFpGIR7XawGJHVkf8Nxr6Fe1m1pMWEayGwCAiGdpxkzWteqnByqn2Ar1JqbSQSfp 7qFw== X-Received: by 10.194.83.39 with SMTP id n7mr46132588wjy.58.1404886180156; Tue, 08 Jul 2014 23:09:40 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id au7sm44345670wjc.41.2014.07.08.23.09.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 Jul 2014 23:09:38 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 9 Jul 2014 08:09:36 +0200 From: Baptiste Daroussin To: Warner Losh Subject: Re: Total confusion over toolchain/xdev behavior Message-ID: <20140709060936.GB56040@ivaldir.etoilebsd.net> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <20140707235237.GG97203@ivaldir.etoilebsd.net> <20140709000628.GA56040@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8GpibOaaTibBMecb" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: sbruno@FreeBSD.org, Ian Lepore , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 06:09:43 -0000 --8GpibOaaTibBMecb Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 08, 2014 at 09:48:45PM -0600, Warner Losh wrote: >=20 > On Jul 8, 2014, at 6:06 PM, Baptiste Daroussin wrote: >=20 > > On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote: > >>=20 > >> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin wrot= e: > >>=20 > >>> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote: > >>>>=20 > >>>> On Jul 7, 2014, at 2:51 PM, Ian Lepore wrote: > >>>>=20 > >>>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote: > >>>>>> Objective: install an xcompile toolchain into a jail for use by > >>>>>> poudriere during arm/mips/sparc/power ports pkgs builds. The build > >>>>>> should be possible from a non-root user. > >>>>>>=20 > >>>>>> As far as I can tell, the xdev target is completely busted for non= -clang > >>>>>> arch's right now as it tries to build clang no matter what I do. = Its > >>>>>> missing some pretty key documentation to making it work correctly,= so a > >>>>>> lot of my attempts have been "guess and check" with verbose make. > >>>>>>=20 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>> Attempt #1: > >>>>>> I have been trying non-root xdev builds: > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j8 xdev XDEV=3Dmips XDEV_ARCH= =3Dmips > >>>>>> -- dies because it tries to compile CLANG. > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #2: > >>>>>> Apply a hack from Baptiste that isn't quite right, but at least ge= ts > >>>>>> farther, note the missing variable causing "//usr/mips-freebsd" > >>>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff > >>>>>>=20 > >>>>>> =3D=3D=3D> gnu/usr.bin/cc/gcov (all) > >>>>>> mtree populating //usr/mips-freebsd > >>>>>> mkdir: //usr/mips-freebsd: Permission denied > >>>>>> *** Error code 1 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #3: Add XDTP > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dm= ips > >>>>>> XDTP=3D/var/tmp/mips_cc > >>>>>>=20 > >>>>>> Try defining a XDTP=3D/var/tmp/mips_cc with the above patch applie= d, get's > >>>>>> a bit farther but compile failure in locating critical include fil= es. > >>>>>>=20 > >>>>>> =3D=3D=3D> gnu/lib/libstdc++ (obj,depend,all,install) > >>>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc > >>>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30: > >>>>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libs= tdc > >>>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such f= ile or > >>>>>> directory > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Attempt #4: Add the additional XDDESTDIR > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Dmips XDEV_ARCH=3Dm= ips > >>>>>> XDTP=3D/var/tmp/mips_cc XDESTDIR=3D/var/tmp/mips_dst > >>>>>> -- Same results as attempt #3 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>>=20 > >>>>>> Even attempting to do stuff for *clang* enabled architectures bails > >>>>>> because its not respecting prefixes: > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s -j 8 xdev XDEV=3Darm XDEV_ARCH= =3Darmv6 > >>>>>> -- bails because it tries to: > >>>>>> =3D=3D=3D> usr.bin/clang/tblgen (all) > >>>>>> mtree populating //usr/armv6-freebsd > >>>>>> mtree: etc/ntp: Permission denied > >>>>>> _xi-cross-tools > >>>>>> =3D=3D=3D> xdev gnu/usr.bin/binutils (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libiberty (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbfd (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libopcodes (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/libbinutils (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/addr2line (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/as (install) > >>>>>> =3D=3D=3D> gnu/usr.bin/binutils/ld (install) > >>>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied > >>>>>> *** Error code 71 > >>>>>>=20 > >>>>>> ------------------------------------------------------------------= ----- > >>>>>> Adding XDTP and XDDESTDIR results in a little more progress but ob= vious > >>>>>> failures to attempt and install things directly into my host syste= m: > >>>>>>=20 > >>>>>> MAKEOBJDIRPREFIX=3D/var/tmp make -s xdev XDEV=3Darm XDEV_ARCH=3Dar= mv6 > >>>>>> XDDESTDIR=3D/var/tmp/arm_cc XDTP=3D/var/tmp/armv6_cc > >>>>>> =3D=3D=3D> secure/lib/libssh (install) > >>>>>> =3D=3D=3D> usr.bin/lex/lib (obj,depend,all,install) > >>>>>> mkdir: ../../../../usr: Permission denied > >>>>>> *** Error code 1 > >>>>>>=20 > >>>>>> Stop. > >>>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head > >>>>>> *** Error code 1 > >>>>>>=20 > >>>>>> Stop. > >>>>>> make: stopped in /home/sbruno/bsd/fbsd_head > >>>>>=20 > >>>>> It looks to me like the permission part of the problem is being cau= sed > >>>>> by a lack of DESTDIR=3D. Without that, it's trying to install to /= usr and > >>>>> you don't have permission for that. Maybe the confusion is because= the > >>>>> xdev target inherently builds-and-installs, unlike most other targe= ts > >>>>> that separate those two actions. > >>>>=20 > >>>> OK. After some detective work, it looks like libstdc++ needs to be d= one before libsupc++ is done. I=E2=80=99ve added this dependency in r268377= and was able to do a full xdev build with a clean obj dir: > >>>>=20 > >>>> rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd > >>>> mkdir $HOME/F > >>>> make xdev DESTDIR=3D$HOME/F XDEV=3Dmips XDEV_ARCH=3Dmips WITHOUT_CL= ANG=3Dt WITHOUT_CLANG_BOOTSTRAP=3Dt WITH_GCC=3Dt WITH_GCC_BOOTSTRAP=3Dt WIT= H_GNUCXX=3Dt -j 20 > >>>=20 > >>> We can avoid most of the above by using a patch like the following: > >>> http://people.freebsd.org/~bapt/Makefile.inc1.diff > >>> Extending the same thing xi-cross-tools and xb-cross-tools (expect the > >>> WITH_GNUCXX=3Dt because it it not set in src.opts.mk when it imho sho= uld.) > >>=20 > >> The patch looks good to my eye. Did you want me to expand it, or do yo= u want to > >> do the honors? > >>=20 > >> About the rest=E2=80=A6 Yea, you may be right=E2=80=A6. MK_GNUCXX is = an odd duck, and that=E2=80=99s > >> likely the problem that should be fixed in a different way. It is real= ly an internal > >> variable that should be set based on the actual compiler type (possibl= y with an > >> override for the odd-duck pair of clang and libstdc++ which may not be= worth > >> supporting). It is telling us we=E2=80=99re doing something horribly w= rong and we should listen > >> to that rather than add another compiler-related kludge to the build s= ystem. I=E2=80=99ll work > >> on that bit. > >>=20 > >> Also an aside: The horribly long command line was to try to get to the= bottom of > >> the breakage, not to promote it as the proper way of doing things. > >=20 > > I haven't committed because it isn't complete yet :) > > I have updated it to the following: > > http://people.freebsd.org/~bapt/Makefile.inc1.diff > > but it is not enough as I end up with: > >=20 > > /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++ > >=20 > > So it goes further but it is not yet enough >=20 > I actually think xdev and a couple others should move to Makefile, and th= e sub targets stay in Makefile.inc and be invoked with the right TARGET and= TARGET_ARCH so that we get the right defaults. >=20 I 100% agree on that :) regards, Bapt --8GpibOaaTibBMecb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlO83KAACgkQ8kTtMUmk6EwIjgCZAU3x1kbCQgphxbxLxbMALlv9 KkIAnjXmGNFOxIrKLPNcLmuA+9ln1FC4 =6PPE -----END PGP SIGNATURE----- --8GpibOaaTibBMecb-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 10:11:40 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58C14BD1; Wed, 9 Jul 2014 10:11:40 +0000 (UTC) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 1D3472A2C; Wed, 9 Jul 2014 10:11:39 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id B5CF842560F; Wed, 9 Jul 2014 20:11:31 +1000 (EST) Date: Wed, 9 Jul 2014 20:11:31 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Bryan Drewery Subject: Re: sys/proc.h inclusion of sys/time.h In-Reply-To: <53BC4F49.7000903@FreeBSD.org> Message-ID: <20140709200949.E1201@besplex.bde.org> References: <53BC4F49.7000903@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=U6SrU4bu c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=z5S34nFNDvQA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=6I5d2MoRAAAA:8 a=G2AMV_VWqIVSfMJ6v8IA:9 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 10:11:40 -0000 > On Tue, 8 Jul 2014, Bryan Drewery wrote: > In r34924 sys/proc.h was changed to only include sys/time.h if not building > in kernel. > > However, as the comment next to time.h says itimerval is needed. > > struct proc { > .. > struct itimerval p_realtimer; /* (c) Alarm timer. */ > > This manifests when (hackishly) including sys/proc.h with _KERNEL defined: > >> In file included from >> /root/svn/base/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/print-pflog.c:37: >> /usr/include/sys/proc.h:524:19: error: field has incomplete type 'struct >> itimerval' >> struct itimerval p_realtimer; /* (c) Alarm timer. */ > > (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I am > testing that is intended for upstreaming. Perhaps I can use kern.pid_max in > __FreeBSD__ and other hacks on other platforms, I have not yet decided on > this.) > > Should we move the inclusion of sys/time.h outside of this ifdef or just add > a forward declaration for struct itimerval above struct proc like many > others? > > -- > Regards, > Bryan Drewery > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 10:31:48 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 144C0F34; Wed, 9 Jul 2014 10:31:48 +0000 (UTC) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id B47902BCC; Wed, 9 Jul 2014 10:31:46 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id D49DD784578; Wed, 9 Jul 2014 20:08:54 +1000 (EST) Date: Wed, 9 Jul 2014 20:08:52 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Don Lewis Subject: Re: [patch] axe RF_TIMESHARE? In-Reply-To: <201407082254.s68MsaPS028312@gw.catspoiler.org> Message-ID: <20140709200848.Q1201@besplex.bde.org> References: <201407082254.s68MsaPS028312@gw.catspoiler.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=QIpRGG7L c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=IvhbSYBuX7IA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=6I5d2MoRAAAA:8 a=23CMgkCoh3DYdGOiiSMA:9 a=Vik_ctyhMKAFVRZW:21 a=NjSdpzuRAPzGTONZ:21 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 Cc: arch@freebsd.org, adrian@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 10:31:48 -0000 yOn Tue, 8 Jul 2014, Don Lewis wrote: > On 6 Jul, Adrian Chadd wrote: >> On 6 July 2014 17:47, Don Lewis wrote: >>> On 6 Jul, Adrian Chadd wrote: >>>> Hi, >>>> >>>> What's it supposed to be used for? >>> >>> My understanding it that it is supposed to be used to allow two more >>> devices to claim the same resource, such as an I/O port range, but only >>> one device can be active at a time. >> >> Interesting. I wonder what kinds of things would want to do this. > > Not much of anything that I can think of, which is probably why this > feature was never used. The closest thing that I can think of is ISA, > as Garrett mentioned. The thing that came to mind for me when I started > looking at this is ISA attached COM ports. COM1 and COM3 both want to > use IRQ 4, and COM2 and COM4 both want to use IRQ 3. The problem is > that ISA IRQs can't be shared between slots, so it might be nice if > RF_TIMESHARE could be used to share IRQ 3 between a modem card > configured as COM4 and the COM2 serial port and then let the user pick > the device to enable through software. Unfortunately this won't work > because 16550-compatible UARTs don't have a way of disabling their IRQ > pin drivers, so this has to be done with jumpers ... No, xx50-compatible UARTs have precisely such a way of disabling their IRQ pin drivers. It is the the OUT2 pin in the modem control register. This is a general purpose pin, but in compatible systems it used to gate the IRQ pin. Its name is misspelled MCR_IENABLE in sio and MCR_IE in uart (this hard-codes its special connection in its very name, as would be correct if this were hard-wired in the xx50 itself. It probably is actually hard-wired in ASICs implementing xx50's). Correct initialization of the OUT2 pin is difficult, and is not done except in old versions of sio, only on some configurations. It is necessary to turn off the OUT2 pin for all devices that might be driving an interrupt before trying to use the interrupt. For this, it is first necessary to know all these devices. Then become the owner or otherwise lock all the devices. Then turn off OUT2 on all except the one where interrupts will be used. Then keep owning all the others so as to keep OUT2 off on them. This is needed for warm boots from another OS that used the interrupts in a different way. Sharing always worked in Linux, so rebooting while Linux is using COM3 with IRQ4 to FreeBSD configured to only use COM1 with IRQ4 will do it. At least old BIOSes don't initialize all COM ports in all cases after a warm boot, perhaps because they have the same problem as FreeBSD with knowing the complete list. Cold boots always work right for unknown xx50 devices, since the hardware reset resets the device to a known good statet. The working versions depended on config(8) being used to configure the complete lists (except it had some internal hard-coded lists, especially for the COM_ESP case). This was broken by unapproved commits for new-bus. New-bus makes it harder for drivers to hack on the lists directly, and the direct hacking on the lists just broke. New-bus doesn't provide the multiple stages of probe/attach needed to do this AFAIK. and provides even less control over the ordering than old bus. Probe shouldn't write or gain ownership of anything. That means no writing to OUT2 for even the device being probed at probe time, and no owning the device from probe to attach so as to keep OUT2 off. So the drive should claim all the devices it can without checking if their interrupts work. Then attach all devices it can, and turn off their OUT2's but still not check if interrupts work. Finally, enable interrupts for just one, provided it has been determined that enough have been claimed. The final step could be done by a second attach pass or just when devices are first really used. The latter goes well with switching on OUT2 only for one being used -- turn off OUT2 on last close and let first open gain control of the interrupt. Device interrupts can be left on for opens that _don't_ gain control of the interrupt (these interrupts are gated off by OUT2), but of i/o to actually work on these opens polled mode must be used. RF_TIMESHARE doesn't simplify this significantly. Ownership of the interrupt should be claimed somewhere so that the interrupt resource is available at first open time. That can probably be done using a super-device or some hack to own the interrupt by the first device that can share it. Then you don't need a flag for it. It seems too hard (bloated) to generalize RF_TIMESHARE so that all of the OUT2 complications can be handled at the new-bus level. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 10:48:21 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7FA8C349; Wed, 9 Jul 2014 10:48:21 +0000 (UTC) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id 450722CE1; Wed, 9 Jul 2014 10:48:20 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 2CA76D65DDE; Wed, 9 Jul 2014 20:48:13 +1000 (EST) Date: Wed, 9 Jul 2014 20:48:12 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Bryan Drewery Subject: Re: sys/proc.h inclusion of sys/time.h In-Reply-To: <53BC4F49.7000903@FreeBSD.org> Message-ID: <20140709201148.W1201@besplex.bde.org> References: <53BC4F49.7000903@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=QIpRGG7L c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=z5S34nFNDvQA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=8txysXnHcTpGYzNcLocA:9 a=CjuIK1q_8ugA:10 Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 10:48:21 -0000 On Tue, 8 Jul 2014, Bryan Drewery wrote: > In r34924 sys/proc.h was changed to only include sys/time.h if not building > in kernel. That should give enough namespace pollution for your purposes. Several other non-kernel abuses depend on it. The ifdef to not do it in the kernel is to depend on the standard namespace pollution that sys/time.h is included in sys/param.h. > However, as the comment next to time.h says itimerval is needed. I don't like comments like that, since they will rot as depenendcies on the pollution grow. I must have written it since I hoped to remove the sys/time.h pollution on sys/param.h soon. > struct proc { > .. > struct itimerval p_realtimer; /* (c) Alarm timer. */ > > This manifests when (hackishly) including sys/proc.h with _KERNEL defined: > >> In file included from >> /root/svn/base/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/print-pflog.c:37: >> /usr/include/sys/proc.h:524:19: error: field has incomplete type 'struct >> itimerval' >> struct itimerval p_realtimer; /* (c) Alarm timer. */ > > (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I am > testing that is intended for upstreaming. Perhaps I can use kern.pid_max in > __FreeBSD__ and other hacks on other platforms, I have not yet decided on > this.) Ah, you were chummy with the implementation, but not chummy enough to know all the details of the kernel environment that must be duplicated to use the hack of defining _KERNEL. It seems to be necessary to include sys/param.h and define _KERNEL before that. There may be collateral pollution and further chumminess to avoid problems with it. Hrmph, PID_MAX actually is a parameter, so in belongs in sys/param.h unlike most of the things there. I thought it was in POSIX. POSIX actually considered and rejected it since it is incompatible with opaque pid_t. > Should we move the inclusion of sys/time.h outside of this ifdef or just add > a forward declaration for struct itimerval above struct proc like many > others? Moving it would be a step backwards. Elsewhere in the file, I tried hard to keep the rusage struct out of it. My version has a struct rusage_ext with all times it it uint64_t except for one struct bintime. This one struct bintime gives a much more critical dependency on sys/time.h than the rotted comment says. -current instead just includes sys/resource.h. This gives lots of pollution, but not sys/time.h since sys/resource.h has been de-polluted to include only sys/_timeval.h to declare the struct timeval's that it uses. Forward declarations only work for incomplete types. Lots of little include files like sys/_timeval.h can be used to reduce pollution. I don't like this much since using just a few of these wastes more time than including one huge polluted file; it just gives less pollution. sys/_timeval.h and sys/_timespec are still clean, but sys/timespec.h has rotted into 2 layers of nesting plus internal pollution. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 11:06:35 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 814D569D; Wed, 9 Jul 2014 11:06:35 +0000 (UTC) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 018252EAD; Wed, 9 Jul 2014 11:06:34 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id BEA66780D56; Wed, 9 Jul 2014 21:06:31 +1000 (EST) Date: Wed, 9 Jul 2014 21:06:31 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Bruce Evans Subject: Re: sys/proc.h inclusion of sys/time.h In-Reply-To: <20140709201148.W1201@besplex.bde.org> Message-ID: <20140709205126.V1201@besplex.bde.org> References: <53BC4F49.7000903@FreeBSD.org> <20140709201148.W1201@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=U6SrU4bu c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=z5S34nFNDvQA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=M9_dusmLnZ1afHhWExwA:9 a=CjuIK1q_8ugA:10 Cc: arch@freebsd.org, Bryan Drewery X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 11:06:35 -0000 On Wed, 9 Jul 2014, Bruce Evans wrote: > On Tue, 8 Jul 2014, Bryan Drewery wrote: > >> In r34924 sys/proc.h was changed to only include sys/time.h if not building >> in kernel. > ... >> struct proc { >> .. >> struct itimerval p_realtimer; /* (c) Alarm timer. */ >> >> This manifests when (hackishly) including sys/proc.h with _KERNEL defined: >> >>> In file included from >>> /root/svn/base/usr.sbin/tcpdump/tcpdump/../../../contrib/tcpdump/print-pflog.c:37: >>> /usr/include/sys/proc.h:524:19: error: field has incomplete type 'struct >>> itimerval' >>> struct itimerval p_realtimer; /* (c) Alarm timer. */ >> >> (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I am >> testing that is intended for upstreaming. Perhaps I can use kern.pid_max in >> __FreeBSD__ and other hacks on other platforms, I have not yet decided on >> this.) > > Ah, you were chummy with the implementation, but not chummy enough to > know all the details of the kernel environment that must be duplicated > to use the hack of defining _KERNEL. It seems to be necessary to include > sys/param.h and define _KERNEL before that. There may be collateral > pollution and further chumminess to avoid problems with it. PS: in some old cleanups, proc.h caused similar problems but I was able to avoid them by not including it at all. It is less needed for abuse outside the kernel than most kernel headers. % diff -u2 ./kvm.c~ ./kvm.c % --- ./kvm.c~ Wed Jun 9 06:13:50 2004 % +++ ./kvm.c Wed Jun 9 06:13:51 2004 % @@ -46,10 +46,10 @@ % % #include % -#include % -#include These cleanups are mainly to combine with ones for sys/user.h. Here I mainly just wanted to sort it. but sys/proc.h dependend on the sys/time.h pollution in it. % +#include % #include % +#include % #include % #include % -#include % +#include % % #include % diff -u2 ./kvm_file.c~ ./kvm_file.c % --- ./kvm_file.c~ Fri Aug 1 21:47:20 2003 % +++ ./kvm_file.c Fri Aug 1 21:47:53 2003 % @@ -48,23 +48,14 @@ % */ % % -#include % -#include % -#include % +#include % #define _KERNEL % #include % #undef _KERNEL sys/param.h turns out to be mustly unnecessary here. Defining _KERNEL before including it was not needed partly because including it at all is not needed, except possibly for other things that are not needed. % -#include % -#include % -#include % -#include % - % -#include % -#include % - % #include % % +#include % #include % -#include % -#include % +#include % +#include % % #include "kvm_private.h" % diff -u2 ./kvm_i386.c~ ./kvm_i386.c % --- ./kvm_i386.c~ Thu Oct 11 20:18:31 2001 % +++ ./kvm_i386.c Sun Sep 8 01:05:29 2002 % @@ -52,5 +52,4 @@ % #include % #include % -#include % #include % #include % @@ -60,4 +59,5 @@ % % #include % +#include % #include % PS2: grepping for PID_MAX shows that ps used to use BSD_PID_MAX (defined as 99999. That was supposed to be a copy of the kernel PID_MAX, perhaps to avoid the namespace problem. It now uses kern.pid_max with a fallback to 99999. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 15:08:32 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6F2730C for ; Wed, 9 Jul 2014 15:08:32 +0000 (UTC) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE75A28B7 for ; Wed, 9 Jul 2014 15:08:32 +0000 (UTC) Received: from [192.168.200.204] (c-50-131-5-126.hsd1.ca.comcast.net [50.131.5.126]) (using SSLv3 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 5EC8F193DD9 for ; Wed, 9 Jul 2014 15:08:25 +0000 (UTC) Subject: Re: Total confusion over toolchain/xdev behavior From: Sean Bruno Reply-To: sbruno@freebsd.org To: freebsd-arch In-Reply-To: <96AB65AF-F86F-4E35-8F94-8BE6730B28F9@bsdimp.com> References: <1404688077.1059.115.camel@bruno> <1404766292.65432.43.camel@revolution.hippie.lan> <20B72004-1499-4F99-A7C7-13173C50C7C6@bsdimp.com> <1404831829.1662.7.camel@bruno> <1404835471.1662.13.camel@bruno> <1404842719.1662.15.camel@bruno> <1404851278.1662.17.camel@bruno> <7CB79988-8221-4F00-AB79-FB24EB3CEF66@bsdimp.com> <1404854676.1662.29.camel@bruno> <9733B60C-5EDA-44A5-9D36-E62433DB8949@bsdimp.com> <98B42676-50A8-4034-995A-ACA9DCE83094@bsdimp.com> <1404865220.1662.42.camel@bruno> <1404876369.1662.45.camel@bruno> <96AB65AF-F86F-4E35-8F94-8BE6730B28F9@bsdimp.com> Content-Type: text/plain; charset="windows-1251" Date: Wed, 09 Jul 2014 08:08:23 -0700 Message-ID: <1404918503.7725.0.camel@bruno> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 15:08:33 -0000 > > > > > What do you folks want to do next with the Mk changes in these patches? > > Let’s try to get the changes reviewed to make sure we’re not stepping on each other… Why don’t you kick that off with phabric reviews? > > Warner Let the bikeshed commence. https://phabric.freebsd.org/D385 From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 16:10:46 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 147763A5 for ; Wed, 9 Jul 2014 16:10:46 +0000 (UTC) Received: from mail-ie0-f173.google.com (mail-ie0-f173.google.com [209.85.223.173]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CEE402E94 for ; Wed, 9 Jul 2014 16:10:45 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id x19so3665227ier.4 for ; Wed, 09 Jul 2014 09:10:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=A9d01WTLd6WKUQnpMuaTCUqErpHFe0PmqFdYknHQz7g=; b=bsoO9/WC5a8h+rIHAaTVQ9dKru8AqoCHsKalK8OBL2GjB+7bkdmwk1CtwdAgpjq/+/ 6vwDs86xe/6jWsPxbR1SS9UbDS1faPlSNxCuWCNsk6LY+lgDhgBmA4GA0Zm++e5DuiLP ri/lXPVYiv+ehVE1DeLSD5RnsRH4YuPvW1diBOaRptkO7Hd8N6vS40mMBESXerHqsS1a OB/DvZ4T8k9zitPM513TxVTMA15QVOKPWLBmlTeY7rTqpV7V80Hc2YoUcxUkA2wyUGxP I0HoRTf/JTDzfrZECmLRpbNmbXzKAdxRJ7ARFaTPIXA/rwWkMQb3lNPsSV1dd/7LgEPn aQOA== X-Gm-Message-State: ALoCoQkAhrbZO0JkdoHX3Q6oZ7hmZ2hoz9cH1KJ+jIHz6v0wzeVHB/Odt1ArYyvU+i40XMpHOzfx X-Received: by 10.42.186.2 with SMTP id cq2mr46969062icb.25.1404922239096; Wed, 09 Jul 2014 09:10:39 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id dz3sm16633594igb.3.2014.07.09.09.10.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 09 Jul 2014 09:10:38 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_114355A0-E7DD-464D-A856-EE61F48CEC28"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: [patch] axe RF_TIMESHARE? From: Warner Losh In-Reply-To: <20140709200848.Q1201@besplex.bde.org> Date: Wed, 9 Jul 2014 10:10:33 -0600 Message-Id: <2174B753-A1C9-4F8A-8E25-28612030AF78@bsdimp.com> References: <201407082254.s68MsaPS028312@gw.catspoiler.org> <20140709200848.Q1201@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.1878.6) Cc: arch@freebsd.org, Don Lewis , adrian@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 16:10:46 -0000 --Apple-Mail=_114355A0-E7DD-464D-A856-EE61F48CEC28 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Jul 9, 2014, at 4:08 AM, Bruce Evans wrote: > RF_TIMESHARE doesn't simplify this significantly. Ownership of the > interrupt should be claimed somewhere so that the interrupt resource > is available at first open time. That can probably be done using > a super-device or some hack to own the interrupt by the first device > that can share it. Then you don't need a flag for it. It seems too > hard (bloated) to generalize RF_TIMESHARE so that all of the OUT2 > complications can be handled at the new-bus level. I think that, while interesting, none of this has a bearing on = RF_TIMESHARE. For shared interrupts, we=92ve used RF_SHARED for a long time. I don=92t = think anybody ever actually implemented RF_TIMESHARE apart from an = aborted attempt by the ppcbus code which later wound up abandoning that = effort (I think before it even made it into the tree). It was an interesting concept, but we never used it and I think we can = light a bonfire under it. There are other ways to share resources that = have been used instead... Warner --Apple-Mail=_114355A0-E7DD-464D-A856-EE61F48CEC28 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTvWl5AAoJEGwc0Sh9sBEAHKgP/jHjvXmVcwX1QP56QNjf5kf1 95Bf8jMeI4KHkeZmrotKsl4YLfeCEpn74wEDnnXi5V22xFrZYsEh9xxOWIjNKjbq Og+hNhTjgFGzkE6U5oB6mLPMxzIgBWD28vxUVhu+1AU3PLXxmnXDeXf3r+dMrOgX jbu09r+40AKdTGIESzyPv2AAM56Hi4xIyUofIfacG4VbHHXioMsYssxdNqTqM+rf lHr511IypQcadjE3o1lFZvRtYg1T33AppRQolvv2loylm2XTbJsvHg/ibPi3k0sm tp/cn15Ye1r/uhOFf3gdJwVPBx2Bbr2RrbImmyxBRNspgJ31uXAy5iq1yzmJG1Au XKgBe70WIFNKGcYiSKQHC+GmwERChlsmTiA7MefaN0RyBFtsirrS2I5cNfUCJbQw MPe9XIaayqQ9gv7rjpWjeNpB3OBXAdWZD32vF1TAS18/DKFEm2h1g9ch1CjCWTm+ tniq2L75H9Xtzm8TbrdpNYu8aicOaQ/gQno54uU9Fx+eN9owDG3ZvdenBDcBos6b LY6kVx+El2SwpAxDfQV7m8kEAZPknugEydDqI4kEJPU7G+fMVCTUS8Q739+SJfuR PWma2EunIAbTfcUqfuF0RWUNBKGa1XEYTpAQFhQSlXyUkMt8c9BUvp8XN6PFdJ/L DcW5PymqsHdfDIk00YeA =L2ml -----END PGP SIGNATURE----- --Apple-Mail=_114355A0-E7DD-464D-A856-EE61F48CEC28-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 16:48:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5424E4D3 for ; Wed, 9 Jul 2014 16:48:35 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 271022221 for ; Wed, 9 Jul 2014 16:48:34 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-250-191.lns20.per2.internode.on.net [121.45.250.191]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s69GmOWM055050 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 9 Jul 2014 09:48:28 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <53BD7252.1010605@freebsd.org> Date: Thu, 10 Jul 2014 00:48:18 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: sys/proc.h inclusion of sys/time.h References: <53BC4F49.7000903@FreeBSD.org> <20140708210727.GA63071@stack.nl> In-Reply-To: <20140708210727.GA63071@stack.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 16:48:35 -0000 On 7/9/14, 5:07 AM, Jilles Tjoelker wrote: > On Tue, Jul 08, 2014 at 03:06:33PM -0500, Bryan Drewery wrote: >> In r34924 sys/proc.h was changed to only include sys/time.h if not >> building in kernel. >> [snip] >> (Why am I doing this? I need PID_MAX and NO_PID for a tcpdump change I >> am testing that is intended for upstreaming. Perhaps I can use >> kern.pid_max in __FreeBSD__ and other hacks on other platforms, I have >> not yet decided on this.) which I do every so often (*).. it's also needed for ibsc binaries (SCO?) which I have NEVER done. (*) about a year ago, compilinga freebsd-1.1 system on a fast modern system took just a few minutes. > The kern.pid_max sysctl is mostly intended for running FreeBSD 1.0 > binaries, which have a 16-bit pid_t. Therefore, it is run-time > adjustable and existing processes may have a pid higher than its value. > > Ideally, you do not need PID_MAX and NO_PID; try to use a variable of > type pid_t only for a process ID and store flags elsewhere. There may be > a problem if you need to read pid_t from an internal structure or > message, though. > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 9 20:40:56 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EEBA687D; Wed, 9 Jul 2014 20:40:56 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99BCE27BB; Wed, 9 Jul 2014 20:40:56 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2050AB953; Wed, 9 Jul 2014 16:40:55 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: callout(9) really this complicated? Date: Wed, 9 Jul 2014 12:11:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140704041521.GW45513@funkthat.com> <53B64694.9030100@selasky.org> <20140704155752.GB45513@funkthat.com> In-Reply-To: <20140704155752.GB45513@funkthat.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407091211.47642.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 09 Jul 2014 16:40:55 -0400 (EDT) Cc: Hans Petter Selasky , arch@freebsd.org, John-Mark Gurney X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2014 20:40:57 -0000 On Friday, July 04, 2014 11:57:52 am John-Mark Gurney wrote: > Hans Petter Selasky wrote this message on Fri, Jul 04, 2014 at 08:15 +0200: > > On 07/04/14 06:15, John-Mark Gurney wrote: > > >So, I was going to look at using callout(9) for some time delayed > > >actions... But upon reading the docs, a) the docs are inconsistent, > > >and b) the docs only talk about requirements in other section... > > > > > >Is there a better interface? If so, can we mark callout(9) deprecated? > > >If not, I have some questions... > > > > > >If you want callout_drain to work properly, you have to add extra code > > >to both your callout, and around the usage of it... > > > > > >callout_drain does not drain the callout: > > > However, the callout subsystem does guarantee that the callout will > > > be > > > fully stopped before callout_drain() returns. > > > > > >Yet other parse of the docs say that you can depend upon the callout > > >being fully stopped.. I've sent email to Ian (iedowse) about why he > > >added this statement... > > > > > >Second, the amount of work you have to do to make sure you drain > > >seems pretty crazy as documented in Avoiding Race Conditions... > > > > > >It seems like if I have created a callout w/ callout_init_mtx, > > >that I shouldn't have to do extra work to make it work correctly... > > > > > >When calling _callout_stop_safe as callout_drain, there is no assert > > >that the lock is held, though it is documented as requiring it by: > > > The function callout_drain() is identical to callout_stop() except > > > that > > > it will wait for the callout to be completed if it is already in > > > progress. > > > > > >Maybe we need to add an additional statement here? and assert that it > > >isn't locked? > > > > > >Also, I have tried to follow the code, but it's complicated, so I'm > > >hoping that I can get some help here. > > > > Probably the documentation needs an update. The implementation is fine. > > Probably... But w/ bad docs, it makes you wonder about the > implementation... If you use callout_init_mtx(), then you can use callout_reset() and callout_stop() while holding the mutex and everything will work as expected. You typically only need to use callout_drain() during a device detach routine to "destroy" the callout(). (A typical "detach" routine looks something like this: - detach external connections (ifnet, cdev, etc.) - stop the hardware itself (foo_stop in various NIC drivers) - this step typically does a callout_stop() with the relevant lock held - drain any pending async events - bus_teardown_intr() will block until the interrupt handler is idle - callout_drain() will block if the callout is pending because softclock had already dequeued it and it was waiting for the driver lock when you called callout_stop() earlier - taskqueue_drain() (note that tasks do not have implicit locking ala callout_init_mtx(), so you need to set some sort of flag that your task function checks and breaks out of any loops for in the "stop the hardware" step) - free resources and destroy locks, etc. > > drain is always called unlocked. > > Then why the whole long section about avoiding races? Point #3 is > the main one that I'm talking about... It seems like it'd be easier > for callout to properly maintain the active flag (maybe set a flag to > tell callout to do so), or not even maintain it since it really isn't > used for anything important if you can munge it all you want... There > aren't any statements about bad things happening if you call _deactivate > before the callout is called... The language is unclear, but you only need to use one of the 3 options to work around the races. Note that if you use callout_init_mtx() you fall into case #1 and don't need to worry about the others. If you want to use callout_init(.., CALLOUT_MPSAFE) and manage all the locking in your callout handler directly, then you need to handle the assorted races. However, you can generally get by with something far simpler than it suggests. You can just do what I described above for taskqueue_drain(), i.e. have your timer function do: foo(void *arg) { struct foo_softc *sc; sc = arg; FOO_LOCK(sc); if (sc->is_dead) { FOO_UNLOCK(sc); return; } .... callout_reset(...); FOO_UNLOCK(sc); } In this case, simply setting 'is_dead' in the "stop the hardware" step and using callout_drain() will work fine. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 05:34:31 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 318ACA0A; Thu, 10 Jul 2014 05:34:31 +0000 (UTC) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id B45D922FE; Thu, 10 Jul 2014 05:34:30 +0000 (UTC) Received: from c122-106-147-133.carlnfd1.nsw.optusnet.com.au (c122-106-147-133.carlnfd1.nsw.optusnet.com.au [122.106.147.133]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 398EC780B91; Thu, 10 Jul 2014 15:34:27 +1000 (EST) Date: Thu, 10 Jul 2014 15:34:22 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Warner Losh Subject: Re: [patch] axe RF_TIMESHARE? In-Reply-To: <2174B753-A1C9-4F8A-8E25-28612030AF78@bsdimp.com> Message-ID: <20140710151043.G985@besplex.bde.org> References: <201407082254.s68MsaPS028312@gw.catspoiler.org> <20140709200848.Q1201@besplex.bde.org> <2174B753-A1C9-4F8A-8E25-28612030AF78@bsdimp.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=dZS5gxne c=1 sm=1 tr=0 a=7NqvjVvQucbO2RlWB8PEog==:117 a=PO7r1zJSAAAA:8 a=IvhbSYBuX7IA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=hvOpnQn2JboAP5lmdFUA:9 a=CjuIK1q_8ugA:10 Cc: arch@freebsd.org, Don Lewis , adrian@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 05:34:31 -0000 On Wed, 9 Jul 2014, Warner Losh wrote: > > On Jul 9, 2014, at 4:08 AM, Bruce Evans wrote: >> RF_TIMESHARE doesn't simplify this significantly. Ownership of the >> interrupt should be claimed somewhere so that the interrupt resource >> is available at first open time. That can probably be done using >> a super-device or some hack to own the interrupt by the first device >> that can share it. Then you don't need a flag for it. It seems too >> hard (bloated) to generalize RF_TIMESHARE so that all of the OUT2 >> complications can be handled at the new-bus level. > > I think that, while interesting, none of this has a bearing on RF_TIMESHARE. It shows that RF_TIMESHARE was hard to use and was never used for one of the cases that it was designed for. > For shared interrupts, we\x92ve used RF_SHARED for a long time. I don\x92t think anybody ever actually implemented RF_TIMESHARE apart from an aborted attempt by the ppcbus code which later wound up abandoning that effort (I think before it even made it into the tree). Please use the ASCII apostrophe character and the ASCII line feed character in mail. pp[c?]bus is just another example for IRQ sharing of ISA IRQ7. It is even more interesting, since IRQ7 should be shareable across devices. It is not clear even what RF_TIMESHARE means for that. IRQ7 is special since "spurious" interrupts are directed to it. It should have a special handler for them. The correct handling seems to be to turn each spurious interrupt into a broadcast to all interrupt handlers. This was never implemented in FreeBSD. You don't want IRQ7 attached to a real device since the interrupt handling to distinguish between a real interrupt and a spurious one is more complicated and slower than normal interrupt handling. IRQ7 is normally attached to a printer. Printers used to be slow devices in 1982, so the extra slowness in the interrupt handler didn't matter. Now (but without ACPI), IRQ7 can be attached to the printer, a 1 Gbps ethernet, and even more devices, so slowness in it is not so good. OTOH, the printer driver is a good bit bucket for spurious interrupts since it mishandles them less badly than most. I rarely use[d] printers, and now have only a USB/wireless one not used under FreeBSD. I used to try to keep IRQ7 attached to lpt0 for use as a bit bucket. Now I disable the printer IRQ on all systems and use IRQ7 for ethernet on some systems. The BIOS configuration for this is painful. Just yesterday I had to type it in again since the CMOS battery is too dead to hold the state over a long mains power shutdown. Timedsharing of IRQ7 across devices would make the configuration work more automatically. It should be attached to lpt0 only while lpt0 is actually being used. Similarly for other devices. For simple timesharing, "use" could be delimited by open/close for some devices and up/down for ethernet devices. Sophisticated timesharing would attach it dynamically (move it automatically from ethernet to lpt if lpt needs it and ethernet has not been used recently). I don't want that complication. > It was an interesting concept, but we never used it and I think we can light a bonfire under it. There are other ways to share resources that have been used instead... Indeed. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 06:19:57 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B4CDE119; Thu, 10 Jul 2014 06:19:57 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E99425F9; Thu, 10 Jul 2014 06:19:57 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s6A6Jt5o022804 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jul 2014 23:19:56 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s6A6JtXR022803; Wed, 9 Jul 2014 23:19:55 -0700 (PDT) (envelope-from jmg) Date: Wed, 9 Jul 2014 23:19:55 -0700 From: John-Mark Gurney To: John Baldwin Subject: Re: callout(9) really this complicated? Message-ID: <20140710061955.GT45513@funkthat.com> Mail-Followup-To: John Baldwin , freebsd-arch@freebsd.org, Hans Petter Selasky , arch@freebsd.org References: <20140704041521.GW45513@funkthat.com> <53B64694.9030100@selasky.org> <20140704155752.GB45513@funkthat.com> <201407091211.47642.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201407091211.47642.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Wed, 09 Jul 2014 23:19:56 -0700 (PDT) Cc: Hans Petter Selasky , arch@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 06:19:57 -0000 John Baldwin wrote this message on Wed, Jul 09, 2014 at 12:11 -0400: > On Friday, July 04, 2014 11:57:52 am John-Mark Gurney wrote: > > Hans Petter Selasky wrote this message on Fri, Jul 04, 2014 at 08:15 +0200: > > > On 07/04/14 06:15, John-Mark Gurney wrote: > > > >So, I was going to look at using callout(9) for some time delayed > > > >actions... But upon reading the docs, a) the docs are inconsistent, > > > >and b) the docs only talk about requirements in other section... > > > > > > > >Is there a better interface? If so, can we mark callout(9) deprecated? > > > >If not, I have some questions... > > > > > > > >If you want callout_drain to work properly, you have to add extra code > > > >to both your callout, and around the usage of it... > > > > > > > >callout_drain does not drain the callout: > > > > However, the callout subsystem does guarantee that the callout will > > > > be > > > > fully stopped before callout_drain() returns. > > > > > > > >Yet other parse of the docs say that you can depend upon the callout > > > >being fully stopped.. I've sent email to Ian (iedowse) about why he > > > >added this statement... > > > > > > > >Second, the amount of work you have to do to make sure you drain > > > >seems pretty crazy as documented in Avoiding Race Conditions... > > > > > > > >It seems like if I have created a callout w/ callout_init_mtx, > > > >that I shouldn't have to do extra work to make it work correctly... > > > > > > > >When calling _callout_stop_safe as callout_drain, there is no assert > > > >that the lock is held, though it is documented as requiring it by: > > > > The function callout_drain() is identical to callout_stop() except > > > > that > > > > it will wait for the callout to be completed if it is already in > > > > progress. > > > > > > > >Maybe we need to add an additional statement here? and assert that it > > > >isn't locked? > > > > > > > >Also, I have tried to follow the code, but it's complicated, so I'm > > > >hoping that I can get some help here. > > > > > > > Probably the documentation needs an update. The implementation is fine. > > > > Probably... But w/ bad docs, it makes you wonder about the > > implementation... > > If you use callout_init_mtx(), then you can use callout_reset() and > callout_stop() while holding the mutex and everything will work as expected. > You typically only need to use callout_drain() during a device detach > routine to "destroy" the callout(). So, you're say that we should just remove this text: However, the callout subsystem does guarantee that the callout will be fully stopped before callout_drain() returns. > (A typical "detach" routine looks something like this: > > - detach external connections (ifnet, cdev, etc.) > - stop the hardware itself (foo_stop in various NIC drivers) > - this step typically does a callout_stop() with the relevant lock held > - drain any pending async events > - bus_teardown_intr() will block until the interrupt handler is > idle > - callout_drain() will block if the callout is pending because softclock > had already dequeued it and it was waiting for the driver lock when > you called callout_stop() earlier > - taskqueue_drain() (note that tasks do not have implicit locking ala > callout_init_mtx(), so you need to set some sort of flag that your > task function checks and breaks out of any loops for in the > "stop the hardware" step) > - free resources and destroy locks, etc. > > > > drain is always called unlocked. > > > > Then why the whole long section about avoiding races? Point #3 is > > the main one that I'm talking about... It seems like it'd be easier > > for callout to properly maintain the active flag (maybe set a flag to > > tell callout to do so), or not even maintain it since it really isn't > > used for anything important if you can munge it all you want... There > > aren't any statements about bad things happening if you call _deactivate > > before the callout is called... > > The language is unclear, but you only need to use one of the 3 options to > work around the races. Note that if you use callout_init_mtx() you fall > into case #1 and don't need to worry about the others. If you want to It isn't clear that point #1 only applies to callout_init_mtx, and that the other points don't apply... And I would say that the text: The callout subsystem provides a number of mechanisms to address these synchronization concerns: Is wrong, it only provides a mechanism for ONE case, the _init_mtx case, in all other cases you have to provide the mechanism to avoid the race.. > use callout_init(.., CALLOUT_MPSAFE) and manage all the locking in your > callout handler directly, then you need to handle the assorted races. > However, you can generally get by with something far simpler than it suggests. > You can just do what I described above for taskqueue_drain(), i.e. have > your timer function do: > > foo(void *arg) > { > struct foo_softc *sc; > > sc = arg; > FOO_LOCK(sc); > if (sc->is_dead) { > FOO_UNLOCK(sc); > return; > } > .... > callout_reset(...); > FOO_UNLOCK(sc); > } > > In this case, simply setting 'is_dead' in the "stop the hardware" step and > using callout_drain() will work fine. Ok, unless someone commits a patch soon, I'll create one.. Thanks for the clarification... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 19:37:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34C04BFB; Thu, 10 Jul 2014 19:37:43 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D2F1C2CA1; Thu, 10 Jul 2014 19:37:42 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C53BDB9C4; Thu, 10 Jul 2014 15:37:41 -0400 (EDT) From: John Baldwin To: "John-Mark Gurney" Subject: Re: callout(9) really this complicated? Date: Thu, 10 Jul 2014 15:30:32 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <20140704041521.GW45513@funkthat.com> <201407091211.47642.jhb@freebsd.org> <20140710061955.GT45513@funkthat.com> In-Reply-To: <20140710061955.GT45513@funkthat.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407101530.32533.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 10 Jul 2014 15:37:41 -0400 (EDT) Cc: Hans Petter Selasky , arch@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 19:37:43 -0000 On Thursday, July 10, 2014 2:19:55 am John-Mark Gurney wrote: > John Baldwin wrote this message on Wed, Jul 09, 2014 at 12:11 -0400: > > On Friday, July 04, 2014 11:57:52 am John-Mark Gurney wrote: > > > Hans Petter Selasky wrote this message on Fri, Jul 04, 2014 at 08:15 +0200: > > > > On 07/04/14 06:15, John-Mark Gurney wrote: > > > > >So, I was going to look at using callout(9) for some time delayed > > > > >actions... But upon reading the docs, a) the docs are inconsistent, > > > > >and b) the docs only talk about requirements in other section... > > > > > > > > > >Is there a better interface? If so, can we mark callout(9) deprecated? > > > > >If not, I have some questions... > > > > > > > > > >If you want callout_drain to work properly, you have to add extra code > > > > >to both your callout, and around the usage of it... > > > > > > > > > >callout_drain does not drain the callout: > > > > > However, the callout subsystem does guarantee that the callout will > > > > > be > > > > > fully stopped before callout_drain() returns. > > > > > > > > > >Yet other parse of the docs say that you can depend upon the callout > > > > >being fully stopped.. I've sent email to Ian (iedowse) about why he > > > > >added this statement... > > > > > > > > > >Second, the amount of work you have to do to make sure you drain > > > > >seems pretty crazy as documented in Avoiding Race Conditions... > > > > > > > > > >It seems like if I have created a callout w/ callout_init_mtx, > > > > >that I shouldn't have to do extra work to make it work correctly... > > > > > > > > > >When calling _callout_stop_safe as callout_drain, there is no assert > > > > >that the lock is held, though it is documented as requiring it by: > > > > > The function callout_drain() is identical to callout_stop() except > > > > > that > > > > > it will wait for the callout to be completed if it is already in > > > > > progress. > > > > > > > > > >Maybe we need to add an additional statement here? and assert that it > > > > >isn't locked? > > > > > > > > > >Also, I have tried to follow the code, but it's complicated, so I'm > > > > >hoping that I can get some help here. > > > > > > > > > > Probably the documentation needs an update. The implementation is fine. > > > > > > Probably... But w/ bad docs, it makes you wonder about the > > > implementation... > > > > If you use callout_init_mtx(), then you can use callout_reset() and > > callout_stop() while holding the mutex and everything will work as expected. > > You typically only need to use callout_drain() during a device detach > > routine to "destroy" the callout(). > > So, you're say that we should just remove this text: > However, the callout subsystem does guarantee that the callout will be > fully stopped before callout_drain() returns. No, the statement is correct (and I somewhat misspoke above). It is helpful to note that callout_drain() predated callout_init_mtx() IIRC (and this probably has made the language more confusing that it should be). Originally, callout_drain() was primarily concerned with solving one race. Namely, ensuring that the actual function registered with callout_reset() is not running so that 1) it is safe to remove the function (e.g. kldunload unmapping the text segment) and 2) the function isn't accessing data you care about. This same race is the one that both bus_teardown_intr() and taskqueue_drain() solve. callout_drain() does solve this in that it sabotages any callout_reset()'s that occur while it is waiting and it blocks until it is sure that the callout is either dequeued or that softclock() is finished with the callout. However, there are other issues with callout functions as you also have races with callout_stop() if you use a plain callout_init(). In general you would need to always have some sort of "dead" flag you set under the lock that you then check at the start of your callout routine (as I described below). callout_init_mtx() was added to provide a way to move all these checks into the callout implementation itself. It allows the "dead" flag to live in the callout structure and be protected by the lock you provide. softclock() is able to grab this lock to handle the callout_stop() race. The only downside is that it introduces a different kind of race. (And this is where I misspoke yesterday) Using callout_init_mtx() resolves the race between callout_stop() and the function registered via callout_reset(). However, it now stores a reference to your lock in the callout system. This reference is what the final callout_drain() is for. It blocks to ensure that softclock() will have a chance to grab your lock if it is currently waiting for it thus allowing you to safely destroy the lock after callout_drain() has returned. > > (A typical "detach" routine looks something like this: > > > > - detach external connections (ifnet, cdev, etc.) > > - stop the hardware itself (foo_stop in various NIC drivers) > > - this step typically does a callout_stop() with the relevant lock held > > - drain any pending async events > > - bus_teardown_intr() will block until the interrupt handler is > > idle > > - callout_drain() will block if the callout is pending because softclock > > had already dequeued it and it was waiting for the driver lock when > > you called callout_stop() earlier > > - taskqueue_drain() (note that tasks do not have implicit locking ala > > callout_init_mtx(), so you need to set some sort of flag that your > > task function checks and breaks out of any loops for in the > > "stop the hardware" step) > > - free resources and destroy locks, etc. > > > > > > drain is always called unlocked. > > > > > > Then why the whole long section about avoiding races? Point #3 is > > > the main one that I'm talking about... It seems like it'd be easier > > > for callout to properly maintain the active flag (maybe set a flag to > > > tell callout to do so), or not even maintain it since it really isn't > > > used for anything important if you can munge it all you want... There > > > aren't any statements about bad things happening if you call _deactivate > > > before the callout is called... > > > > The language is unclear, but you only need to use one of the 3 options to > > work around the races. Note that if you use callout_init_mtx() you fall > > into case #1 and don't need to worry about the others. If you want to > > It isn't clear that point #1 only applies to callout_init_mtx, and that > the other points don't apply... Hmm, I think the first part is quite clear as it explicitly mentions callout_init_mtx: 1. If the callout has an associated mutex that was specified using the callout_init_mtx() function ... That seems to imply that 1) only applies if you use callout_init_mtx(). It also says: FALSE), then this mutex is used to avoid the race conditions. And for further clarification it states almost exactly what I siad in my first mail: The associated mutex must be acquired by the caller before calling callout_stop() or callout_reset() and it is guaranteed that the callout will be correctly stopped or reset as expected. > And I would say that the text: > The callout subsystem provides a number of mechanisms to address these > synchronization concerns: > > Is wrong, it only provides a mechanism for ONE case, the _init_mtx case, > in all other cases you have to provide the mechanism to avoid the race.. Eh, I would also disagree. All three mechanisms require cooperation from the caller: 1) For this mode you have to agree to the contract that you will hold the associated lock when you invoke callout_stop() or callout_reset() and you have to use callout_drain() before you destroy the lock. 2) You have to hold the lock while you manage your "running" flag, but you do at least get notified by callout_stop() that you are in the midst of a race so you know when to invoke your special handling. 3) Again, you have to do your own locking, but you can use the flags to detect the different states your callout is in when your callout handler runs. That said, 1) is far simpler to use and is appropriate for almost all of the callouts I've worked with. I don't think I've ever used 2) or 3). > > use callout_init(.., CALLOUT_MPSAFE) and manage all the locking in your > > callout handler directly, then you need to handle the assorted races. > > However, you can generally get by with something far simpler than it suggests. > > You can just do what I described above for taskqueue_drain(), i.e. have > > your timer function do: > > > > foo(void *arg) > > { > > struct foo_softc *sc; > > > > sc = arg; > > FOO_LOCK(sc); > > if (sc->is_dead) { > > FOO_UNLOCK(sc); > > return; > > } > > .... > > callout_reset(...); > > FOO_UNLOCK(sc); > > } > > > > In this case, simply setting 'is_dead' in the "stop the hardware" step and > > using callout_drain() will work fine. > > Ok, unless someone commits a patch soon, I'll create one.. > > Thanks for the clarification... > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 22:59:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5AD06813 for ; Thu, 10 Jul 2014 22:59:23 +0000 (UTC) Received: from eastrmfepo102.cox.net (eastrmfepo102.cox.net [68.230.241.214]) by mx1.freebsd.org (Postfix) with ESMTP id 0294D2F07 for ; Thu, 10 Jul 2014 22:59:22 +0000 (UTC) Received: from eastrmimpo209 ([68.230.241.224]) by eastrmfepo102.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140710225916.PSBN18526.eastrmfepo102.cox.net@eastrmimpo209> for ; Thu, 10 Jul 2014 18:59:16 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo209 with cox id QmzF1o00941obj401mzGpJ; Thu, 10 Jul 2014 18:59:16 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020203.53BF1AC4.0089,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=H/cFNZki c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=IkcTkHD0fZMA:10 a=kviXuzpPAAAA:8 a=pGLkceISAAAA:8 a=6I5d2MoRAAAA:8 a=9yZdCLXBaxRpkGpTl2AA:9 a=QEXdDO2ut3YA:10 a=4vB-4DCPJfMA:10 a=MSl-tDqOz04A:10 a=SV7veod9ZcQA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53BF1AFF.8070707@cox.net> Date: Thu, 10 Jul 2014 19:00:15 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: Eitan Adler Subject: Re: Improve cron(8) References: <53A72666.8090101@cox.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org, =?UTF-8?B?VG9tZWsgV2HFgmFzemVr?= , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 22:59:23 -0000 Eitan Adler wrote: > On 22 June 2014 11:54, John D. Hendrickson and Sara Darnell > wrote: >> Eitan Adler wrote: >>> +arch since hackers@ seems to be silent. >>> >>> On 11 June 2014 23:56, Tomek Wałaszek wrote: >>>> Hello, >>>> I saw on the FreeBSD Ideas page topic about cron :). >>>> I've started updating the 'original' FreeBSD cron from sources to vixi >>>> cron >>>> 4.1. I think (well I hope :P) most of the features that were done in >>>> FreeBSD cron are now ported into vixi cron 4.1, there are unfortunately >>>> some missing features at the moment: >>>> - @every_second - this need to be done >>>> - -s and -o, in vixi cron 4.1 daylight time switches are enabled by >>>> default, at the moment there is no -s and -o options. So you need to >>>> remove >>>> '-s' from the cron rc script >>>> >>>> I've also added one feature from OpenBSD, crontab is poking cron using >>>> unix-domain socket so we don't need to have suid on crontab. >>>> >>>> Path is in the attachment. I'm testing it on my FreeBSD box and it looks >>>> good but anyway don't try it on production machines :). >>>> >>>> After the installation we have to do a few things: >>>> - Add crontab group >>>> - Change group to crontab on /var/cron/tabs >>>> - Add sticky bit on /var/cron/tabs >>>> - Add group write permissions on /var/cron/tabs >>>> >>>> This is still work in progress but if someone could have a look on this >>>> and >>>> give me some feedback it would be great. >>>> >>>> Regards, >>>> Tomasz Walaszek >>>> >>>> _______________________________________________ >>>> freebsd-hackers@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>>> To unsubscribe, send any mail to >>>> "freebsd-hackers-unsubscribe@freebsd.org" >>> >>> >>> >> you should up the version number or start your own renamed application > ... > > I don't know how to react to this message. You just told a potential > contributor not to contribute since he was not the founder of the > project. That goes against everything that open source is about. > > i think i read the origional too quickly. Tomek appears to say BAS cron was before Vixie Cron i didn't see that. however i'll go ahead and repeat it. contributing does not mean altering the way software other people are using works, like sed(1) or chron(1) i hope Not expect to all have to revisit years back, maybe many scripts and redo old scripts because one person wishes new features which WOULD cause problems: user permission and security features done in a different way and which require socket software i did not discourage a contribution. i asked the project retain a different name if the new features cause problems with critical software. anyone would be free to use the different name or rename it without conflicting with existing situations. i'm sure you know some software tries to use cron for system maintenance both in scripts and sometimes from C code. of course he should edit and recompile bsd things i'd like to as well i have not got the chance issue was naming and sharing work, ie, which branch or trunk is checked out when one installs bsd , so to speak From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 23:10:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1FA6A08 for ; Thu, 10 Jul 2014 23:10:37 +0000 (UTC) Received: from eastrmfepo201.cox.net (eastrmfepo201.cox.net [68.230.241.216]) by mx1.freebsd.org (Postfix) with ESMTP id 59E202097 for ; Thu, 10 Jul 2014 23:10:37 +0000 (UTC) Received: from eastrmimpo210 ([68.230.241.225]) by eastrmfepo201.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140710231030.FLGL31475.eastrmfepo201.cox.net@eastrmimpo210> for ; Thu, 10 Jul 2014 19:10:30 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo210 with cox id QnAV1o00E41obj401nAV3h; Thu, 10 Jul 2014 19:10:29 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020208.53BF1D65.01D2,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=aZC/a2Ut c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=IkcTkHD0fZMA:10 a=kviXuzpPAAAA:8 a=iPQX104nh9CmW5Tgo0oA:9 a=QEXdDO2ut3YA:10 a=2DpKuaG0z5UA:10 a=AhGx-PANj34A:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53BF1DA0.5040605@cox.net> Date: Thu, 10 Jul 2014 19:11:28 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: Adrian Chadd Subject: Re: Improve cron(8) References: <53A72666.8090101@cox.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , =?UTF-8?B?VG9tZWsgV2HFgmFzemVr?= , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 23:10:37 -0000 Adrian Chadd wrote: > Actually, I know how to react to that. > > Don't dissuade someone from contributing. Discuss the changes and > whether they're positive or negative. > > -a i did not intend to be negative about patches offered nor did i say not to offer a patch or discuss update you are certainly adding words in my mouth From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 23:36:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8A6A6F21 for ; Thu, 10 Jul 2014 23:36:13 +0000 (UTC) Received: from eastrmfepo101.cox.net (eastrmfepo101.cox.net [68.230.241.213]) by mx1.freebsd.org (Postfix) with ESMTP id 32B1B22DB for ; Thu, 10 Jul 2014 23:36:13 +0000 (UTC) Received: from eastrmimpo306 ([68.230.241.238]) by eastrmfepo101.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140710233606.VSXV30009.eastrmfepo101.cox.net@eastrmimpo306> for ; Thu, 10 Jul 2014 19:36:06 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo306 with cox id Qnc51o00C41obj401nc554; Thu, 10 Jul 2014 19:36:05 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A02020A.53BF2365.0198,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=c6J1t2Bl c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=XigzRtDixlUA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=8nJEP1OIZ-IA:10 a=kviXuzpPAAAA:8 a=zoPSNvUg5lPj358TR94A:9 a=wPNLvfGTeEIA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53BF23A0.1000603@cox.net> Date: Thu, 10 Jul 2014 19:37:04 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: John Baldwin Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <201406190919.04443.jhb@freebsd.org> <750618593.166408.1403191319583.JavaMail.zimbra@arkoon-netasq.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Emeric POUPON , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 23:36:13 -0000 John Baldwin wrote: > On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: >> Thanks for your answer! >> >> I was thinking about calling some parent device functions from the children > devices in order to perform IO accesses. >> But I imagine it would be "better" to expose a kind of bus interface from > the main driver? >> However, I'm not sure the extra work induced is worth it. What do you think? > > I think it's fine to have them call each other directly if they are going to > all live in the same module. > just wondering do you mean fine not to expose a feature ? or fine the Winbond chip circuit allows the procedure with no problems ? wow i thought winbond io would have been a done deal. wasn't that released in 1990s ? (rhetorical) From owner-freebsd-arch@FreeBSD.ORG Thu Jul 10 23:42:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9866C13C for ; Thu, 10 Jul 2014 23:42:11 +0000 (UTC) Received: from eastrmfepo203.cox.net (eastrmfepo203.cox.net [68.230.241.218]) by mx1.freebsd.org (Postfix) with ESMTP id 43249238A for ; Thu, 10 Jul 2014 23:42:10 +0000 (UTC) Received: from eastrmimpo110 ([68.230.241.223]) by eastrmfepo203.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140710234210.DYVE2658.eastrmfepo203.cox.net@eastrmimpo110> for ; Thu, 10 Jul 2014 19:42:10 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo110 with cox id Qni91o00L41obj401niAdS; Thu, 10 Jul 2014 19:42:10 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020208.53BF24D2.006D,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=GKHW5JxK c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=9cW_t1CCXrUA:10 a=f5xKl4ys9bwA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=8nJEP1OIZ-IA:10 a=kviXuzpPAAAA:8 a=YBTvvWOgv1z6kiV7-ccA:9 a=wPNLvfGTeEIA:10 a=tkvPNXmsOoIA:10 a=wgK2LWaXZbkA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53BF250D.6090201@cox.net> Date: Thu, 10 Jul 2014 19:43:09 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 Subject: Re: Improve cron(8) References: <53A72666.8090101@cox.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2014 23:42:11 -0000 i did have some concerns about "how easy is socket security", use of a /tmp as /etc, and effects on crontabs people have (diff perms) but i'll back out of the discussion you all discuss it thanks have a good day i'll head onto another topic :) have fun From owner-freebsd-arch@FreeBSD.ORG Fri Jul 11 02:37:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 65479BB7; Fri, 11 Jul 2014 02:37:58 +0000 (UTC) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "funkthat.com", Issuer "funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1782221AC; Fri, 11 Jul 2014 02:37:57 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id s6B2bo5B039393 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 10 Jul 2014 19:37:50 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id s6B2bomt039392; Thu, 10 Jul 2014 19:37:50 -0700 (PDT) (envelope-from jmg) Date: Thu, 10 Jul 2014 19:37:50 -0700 From: John-Mark Gurney To: John Baldwin Subject: Re: callout(9) really this complicated? Message-ID: <20140711023749.GA45513@funkthat.com> Mail-Followup-To: John Baldwin , freebsd-arch@freebsd.org, Hans Petter Selasky , arch@freebsd.org References: <20140704041521.GW45513@funkthat.com> <201407091211.47642.jhb@freebsd.org> <20140710061955.GT45513@funkthat.com> <201407101530.32533.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201407101530.32533.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Thu, 10 Jul 2014 19:37:51 -0700 (PDT) Cc: Hans Petter Selasky , arch@freebsd.org, freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 02:37:58 -0000 John Baldwin wrote this message on Thu, Jul 10, 2014 at 15:30 -0400: > On Thursday, July 10, 2014 2:19:55 am John-Mark Gurney wrote: > > John Baldwin wrote this message on Wed, Jul 09, 2014 at 12:11 -0400: > > > On Friday, July 04, 2014 11:57:52 am John-Mark Gurney wrote: > > > > Hans Petter Selasky wrote this message on Fri, Jul 04, 2014 at 08:15 +0200: > > > > > On 07/04/14 06:15, John-Mark Gurney wrote: > > > > > >So, I was going to look at using callout(9) for some time delayed > > > > > >actions... But upon reading the docs, a) the docs are inconsistent, > > > > > >and b) the docs only talk about requirements in other section... > > > > > > > > > > > >Is there a better interface? If so, can we mark callout(9) deprecated? > > > > > >If not, I have some questions... > > > > > > > > > > > >If you want callout_drain to work properly, you have to add extra code > > > > > >to both your callout, and around the usage of it... > > > > > > > > > > > >callout_drain does not drain the callout: > > > > > > However, the callout subsystem does guarantee that the callout will > > > > > > be > > > > > > fully stopped before callout_drain() returns. > > > > > > > > > > > >Yet other parse of the docs say that you can depend upon the callout > > > > > >being fully stopped.. I've sent email to Ian (iedowse) about why he > > > > > >added this statement... > > > > > > > > > > > >Second, the amount of work you have to do to make sure you drain > > > > > >seems pretty crazy as documented in Avoiding Race Conditions... > > > > > > > > > > > >It seems like if I have created a callout w/ callout_init_mtx, > > > > > >that I shouldn't have to do extra work to make it work correctly... > > > > > > > > > > > >When calling _callout_stop_safe as callout_drain, there is no assert > > > > > >that the lock is held, though it is documented as requiring it by: > > > > > > The function callout_drain() is identical to callout_stop() except > > > > > > that > > > > > > it will wait for the callout to be completed if it is already in > > > > > > progress. > > > > > > > > > > > >Maybe we need to add an additional statement here? and assert that it > > > > > >isn't locked? > > > > > > > > > > > >Also, I have tried to follow the code, but it's complicated, so I'm > > > > > >hoping that I can get some help here. > > > > > > > > > > > > > Probably the documentation needs an update. The implementation is fine. > > > > > > > > Probably... But w/ bad docs, it makes you wonder about the > > > > implementation... > > > > > > If you use callout_init_mtx(), then you can use callout_reset() and > > > callout_stop() while holding the mutex and everything will work as expected. > > > You typically only need to use callout_drain() during a device detach > > > routine to "destroy" the callout(). > > > > So, you're say that we should just remove this text: > > However, the callout subsystem does guarantee that the callout will be > > fully stopped before callout_drain() returns. > > No, the statement is correct (and I somewhat misspoke above). It is helpful > to note that callout_drain() predated callout_init_mtx() IIRC (and this probably > has made the language more confusing that it should be). Ok, I'm so confused now that I will NOT be creating a patch, and unless someone comes up w/ a patch to clarify it, I will add to the description that there are numerous misleading statements in this man page and link them to this thread... > Originally, callout_drain() was primarily concerned with solving one race. > Namely, ensuring that the actual function registered with callout_reset() is > not running so that 1) it is safe to remove the function (e.g. kldunload > unmapping the text segment) and 2) the function isn't accessing data you care > about. This same race is the one that both bus_teardown_intr() and > taskqueue_drain() solve. callout_drain() does solve this in that it sabotages > any callout_reset()'s that occur while it is waiting and it blocks until it > is sure that the callout is either dequeued or that softclock() is finished > with the callout. > > However, there are other issues with callout functions as you also have races > with callout_stop() if you use a plain callout_init(). In general you would > need to always have some sort of "dead" flag you set under the lock that you > then check at the start of your callout routine (as I described below). > callout_init_mtx() was added to provide a way to move all these checks into > the callout implementation itself. It allows the "dead" flag to live in the > callout structure and be protected by the lock you provide. softclock() is > able to grab this lock to handle the callout_stop() race. The only downside > is that it introduces a different kind of race. (And this is where I misspoke > yesterday) Using callout_init_mtx() resolves the race between callout_stop() > and the function registered via callout_reset(). However, it now stores a > reference to your lock in the callout system. This reference is what the > final callout_drain() is for. It blocks to ensure that softclock() will have > a chance to grab your lock if it is currently waiting for it thus allowing > you to safely destroy the lock after callout_drain() has returned. Yes, I figured that this is the case, but the problem is that when the _mtx version was added, they didn't make it clear what statements applied to which... > > > (A typical "detach" routine looks something like this: > > > > > > - detach external connections (ifnet, cdev, etc.) > > > - stop the hardware itself (foo_stop in various NIC drivers) > > > - this step typically does a callout_stop() with the relevant lock held > > > - drain any pending async events > > > - bus_teardown_intr() will block until the interrupt handler is > > > idle > > > - callout_drain() will block if the callout is pending because softclock > > > had already dequeued it and it was waiting for the driver lock when > > > you called callout_stop() earlier > > > - taskqueue_drain() (note that tasks do not have implicit locking ala > > > callout_init_mtx(), so you need to set some sort of flag that your > > > task function checks and breaks out of any loops for in the > > > "stop the hardware" step) > > > - free resources and destroy locks, etc. > > > > > > > > drain is always called unlocked. > > > > > > > > Then why the whole long section about avoiding races? Point #3 is > > > > the main one that I'm talking about... It seems like it'd be easier > > > > for callout to properly maintain the active flag (maybe set a flag to > > > > tell callout to do so), or not even maintain it since it really isn't > > > > used for anything important if you can munge it all you want... There > > > > aren't any statements about bad things happening if you call _deactivate > > > > before the callout is called... > > > > > > The language is unclear, but you only need to use one of the 3 options to > > > work around the races. Note that if you use callout_init_mtx() you fall > > > into case #1 and don't need to worry about the others. If you want to > > > > It isn't clear that point #1 only applies to callout_init_mtx, and that > > the other points don't apply... > > Hmm, I think the first part is quite clear as it explicitly mentions > callout_init_mtx: > > 1. If the callout has an associated mutex that was specified > using the callout_init_mtx() function ... > > That seems to imply that 1) only applies if you use callout_init_mtx(). > It also says: > > FALSE), then this mutex is used to avoid the race conditions. But it doesn't exclude the remaining points... It says there are a number of ways to address them, but doesn't say pick one, or clarifies which is needed for which version... Yes, 1 only applied to _mtx versions, but no text in 2 or 3 or the preceeding paragraph says that they don't apply to the _mtx version.. And it also doesn't further clarify that the _drain will properly terminate, it only says: expected. Note that it is still necessary to use callout_drain() before destroying the callout or its associ- ated mutex. Yet, the previous statement: However, the callout subsystem does guarantee that the callout will be fully stopped before callout_drain() returns. Has not been exluded from the _mtx version... > And for further clarification it states almost exactly what I siad in my first > mail: > > The associated mutex must be acquired by the caller before > calling callout_stop() or callout_reset() and it is guaranteed > that the callout will be correctly stopped or reset as > expected. This only applied to _stop and _reset, but does not apply to _drain... > > And I would say that the text: > > The callout subsystem provides a number of mechanisms to address these > > synchronization concerns: > > > > Is wrong, it only provides a mechanism for ONE case, the _init_mtx case, > > in all other cases you have to provide the mechanism to avoid the race.. > > Eh, I would also disagree. All three mechanisms require cooperation from > the caller: ok, then I would not say address, I would say helps instead.. because the mechanisms do not fully address the races w/o additional code... > 1) For this mode you have to agree to the contract that you will hold > the associated lock when you invoke callout_stop() or callout_reset() > and you have to use callout_drain() before you destroy the lock. > > 2) You have to hold the lock while you manage your "running" flag, but > you do at least get notified by callout_stop() that you are in the > midst of a race so you know when to invoke your special handling. > > 3) Again, you have to do your own locking, but you can use the flags to > detect the different states your callout is in when your callout > handler runs. > > That said, 1) is far simpler to use and is appropriate for almost all of > the callouts I've worked with. I don't think I've ever used 2) or 3). Here you seem to say that the three methods are of a pick on type, but the preceeding text does not say that... It says here are mechanisms that need to be used to avoid races... Not, Use one of the following methods to avoid races... > > > use callout_init(.., CALLOUT_MPSAFE) and manage all the locking in your > > > callout handler directly, then you need to handle the assorted races. > > > However, you can generally get by with something far simpler than it suggests. > > > You can just do what I described above for taskqueue_drain(), i.e. have > > > your timer function do: > > > > > > foo(void *arg) > > > { > > > struct foo_softc *sc; > > > > > > sc = arg; > > > FOO_LOCK(sc); > > > if (sc->is_dead) { > > > FOO_UNLOCK(sc); > > > return; > > > } > > > .... > > > callout_reset(...); > > > FOO_UNLOCK(sc); > > > } > > > > > > In this case, simply setting 'is_dead' in the "stop the hardware" step and > > > using callout_drain() will work fine. > > > > Ok, unless someone commits a patch soon, I'll create one.. > > > > Thanks for the clarification... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Fri Jul 11 15:34:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2095B3D; Fri, 11 Jul 2014 15:34:00 +0000 (UTC) Received: from dmz-mailsec-scanner-3.mit.edu (dmz-mailsec-scanner-3.mit.edu [18.9.25.14]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2A08A25D5; Fri, 11 Jul 2014 15:33:59 +0000 (UTC) X-AuditID: 1209190e-f79946d000007db1-d2-53c002b30016 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP id 12.6E.32177.3B200C35; Fri, 11 Jul 2014 11:28:51 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id s6BFSoOo025458; Fri, 11 Jul 2014 11:28:51 -0400 Received: from multics.mit.edu (system-low-sipb.mit.edu [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6BFSkaU006433 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 11 Jul 2014 11:28:50 -0400 Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id s6BFSk6P008332; Fri, 11 Jul 2014 11:28:46 -0400 (EDT) Date: Fri, 11 Jul 2014 11:28:45 -0400 (EDT) From: Benjamin Kaduk X-X-Sender: kaduk@multics.mit.edu To: John-Mark Gurney Subject: Re: callout(9) really this complicated? In-Reply-To: <20140711023749.GA45513@funkthat.com> Message-ID: References: <20140704041521.GW45513@funkthat.com> <201407091211.47642.jhb@freebsd.org> <20140710061955.GT45513@funkthat.com> <201407101530.32533.jhb@freebsd.org> <20140711023749.GA45513@funkthat.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrPIsWRmVeSWpSXmKPExsUixG6nrruZ6UCwwc4WQ4vZ06cxWew9cp3Z 4n7XEyYHZo8Zn+azeFw+tZk5gCmKyyYlNSezLLVI3y6BK+PT+9fsBQdZKs4/b2BsYNzD3MXI ySEhYCJx7sNaKFtM4sK99WxdjFwcQgKzmSQ+7elhhXA2MkrM6T/MCOEcYpJ4e+UQVFkDo8SG V81ADgcHi4C2xJc75iCj2ATUJB7vbWaFGKsosfnUJLAVIgLqEi/X9bCB2MwCNhLPrm5gAbGF BQwkTm0+xAhicwoYSex9eoAdxOYVcJRoXDMLatdxRontb0+CNYgK6Eis3j+FBaJIUOLkzCcs EEMtJc79uc42gVFoFpLULCSpBYxMqxhlU3KrdHMTM3OKU5N1i5MT8/JSi3SN9XIzS/RSU0o3 MYLCmVOSbwfj14NKhxgFOBiVeHgV1u8LFmJNLCuuzD3EKMnBpCTKG8V4IFiILyk/pTIjsTgj vqg0J7X4EKMEB7OSCO/VN/uDhXhTEiurUovyYVLSHCxK4rxvra2ChQTSE0tSs1NTC1KLYLIy HBxKErwvQYYKFqWmp1akZeaUIKSZODhBhvMADd8DUsNbXJCYW5yZDpE/xagoJc7rC5IQAElk lObB9cLSzStGcaBXhHmZgclHiAeYquC6XwENZgIavL0f5OrikkSElFQDo+ND2ZiEorP5qYLz XW6s38M0+2bt/qXbkt2Yv9U/X7nko9eR4E2MfIrzL89nKfpswJxWZF9Toq3ZJHn16srX+3+8 4Ft42Pm8zc3sKypLVCveWG6N8PjbX5vAlBVX+cXUZ+ej94rvJB54XbnUyiJTYj8/OKVb1lJ/ zpNZvrmKemsOhCZUucyNUmIpzkg01GIuKk4EAE3H8qISAwAA Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 15:34:00 -0000 [large pile of context trimmed] On Thu, 10 Jul 2014, John-Mark Gurney wrote: > Ok, I'm so confused now that I will NOT be creating a patch, and unless > someone comes up w/ a patch to clarify it, I will add to the description > that there are numerous misleading statements in this man page and link > them to this thread... I have saved jhb's messages in this thread and added to my TODO list to make a pass through this man page. There's quite a few things in front of this in my list, though -- I don't expect to have anything for maybe a month. -Ben From owner-freebsd-arch@FreeBSD.ORG Fri Jul 11 16:41:37 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3D350A78 for ; Fri, 11 Jul 2014 16:41:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14ECF2D2F for ; Fri, 11 Jul 2014 16:41:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id D0168B968; Fri, 11 Jul 2014 12:41:35 -0400 (EDT) From: John Baldwin To: johnandsara2@cox.net Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? Date: Fri, 11 Jul 2014 09:55:55 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <53BF23A0.1000603@cox.net> In-Reply-To: <53BF23A0.1000603@cox.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407110955.55568.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 11 Jul 2014 12:41:35 -0400 (EDT) Cc: Emeric POUPON , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 16:41:37 -0000 On Thursday, July 10, 2014 7:37:04 pm John D. Hendrickson and Sara Darnell wrote: > John Baldwin wrote: > > On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: > >> Thanks for your answer! > >> > >> I was thinking about calling some parent device functions from the children > > devices in order to perform IO accesses. > >> But I imagine it would be "better" to expose a kind of bus interface from > > the main driver? > >> However, I'm not sure the extra work induced is worth it. What do you think? > > > > I think it's fine to have them call each other directly if they are going to > > all live in the same module. > > > > just wondering > > do you mean fine not to expose a feature ? > > or fine the Winbond chip circuit allows the procedure with no problems ? > > wow i thought winbond io would have been a done deal. wasn't that > released in 1990s ? (rhetorical) No, the question is if you have two C files that are compiled into a single loading object (foo.ko), do they call each other's functions directly or do they use an indirection layer like kobj to call into each other. The indirection layers tend to be both slower and a bit more obfuscated code-wise, so if it is known for sure that these will always be in the same foo.ko, then I think direct calls is cleaner. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Jul 11 23:26:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C2C1A43E for ; Fri, 11 Jul 2014 23:26:28 +0000 (UTC) Received: from mail-qc0-x22a.google.com (mail-qc0-x22a.google.com [IPv6:2607:f8b0:400d:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E97C22C6 for ; Fri, 11 Jul 2014 23:26:28 +0000 (UTC) Received: by mail-qc0-f170.google.com with SMTP id c9so1636104qcz.29 for ; Fri, 11 Jul 2014 16:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=mO+5VR7GnbDKUPAGckghmMXQBn/DXBrmS8DxN45v2AY=; b=KTDz9vizpI04cHYnt4ISK5AL1Ls5dhw9I2zAQ3uR5x267mgR8iBQi41LcTLtqMnRQ6 rc/lMlqfhZ85ip65gvQZYGQSORofp6LJkrWFyfyN+W0rzPkqqRTKJo7/kSyR4drXG1iT VWNvSgICc0bl4CSkye7c4DfF4MG6CGbGcDb+kL2jhXfLO2SAXVXC4VFiVZSZJ8u9I9lh 1WOOjdLlT+cE5axgiF8s4XKSoZWZCIHMk2Mo8pE7BZ7YAlQBtZqzdNwRa/k/evheHhcU eOApnOqYR/cVc0/B2J6UawCV5eu9VEom83mKe0PZMh8xx5zzvJDcoYdqIdAHNYCOO7jN swTw== X-Received: by 10.140.28.161 with SMTP id 30mr3237975qgz.45.1405121187406; Fri, 11 Jul 2014 16:26:27 -0700 (PDT) Received: from pwnie.vrt.sourcefire.com (moist.vrt.sourcefire.com. [198.148.79.134]) by mx.google.com with ESMTPSA id k7sm6667073qas.24.2014.07.11.16.26.25 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Jul 2014 16:26:26 -0700 (PDT) Date: Fri, 11 Jul 2014 19:26:23 -0400 From: Shawn Webb To: freebsd-arch@freebsd.org Subject: [RFC] ASLR Whitepaper and Candidate Final Patch Message-ID: <20140711232623.GG41807@pwnie.vrt.sourcefire.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="++alDQ2ROsODg1x+" Content-Disposition: inline X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 23:26:28 -0000 --++alDQ2ROsODg1x+ Content-Type: multipart/mixed; boundary="vkEkAx9hr54EJ73W" Content-Disposition: inline --vkEkAx9hr54EJ73W Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hey All, Oliver Pinter and I have been working hard on our ASLR implementation. We're now in the final stages of development and would like to get feedback from the community. I've attached to this email a small whitepaper that details our implementation and the accompanying patch. There is one part of the patch that I wrote that is quite an ugly hack and would like to get some feedback on. I added a little hack to sys_mmap() to apply ASLR to calls to mmap(2) when MAP_32BIT is specified. I'd like to remove that ugly hack to something a bit more beautiful, so if anyone has any suggestions, I'm all ears. Other than that ugly hack, the code adheres to FreeBSD's style(9) standards. I believe we have an awesome implementation, one I've personally been using without issue for months. I'm looking forward to your comments and questions. I've CC'd the PaX team. Please keep them CC'd in your replies. Thank you very much, Shawn Webb CC: PaX Team CC: Oliver Pinter CC: des@freebsd.org CC: alc@rice.edu CC: bdrewery@freebsd.org --vkEkAx9hr54EJ73W Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="2014-07-10_aslr_whitepaper.txt" Content-Transfer-Encoding: quoted-printable Introducing ASLR For FreeBSD =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Shawn Webb Oliver Pinter 10 July 2014 http://www.hardenedbsd.org/ [ 1. Introduction ] Security in FreeBSD's is based primarily in policy-based technologies. Exis= ting tools such as jails, Capsicum, vnet/vimage, and the MAC framework, can make FreeBSD-based systems quite resilient against attacks. FreeBSD lacks basic low-level exploit mitigation, such as Address Space Layout Randomization (ASLR)[1]. ASLR randomizes the address space layout of an application, maki= ng exploitation difficult for an attacker. This paper and the associated implementation aim to provide a secure, robust, extensible, and easily-man= aged form of ASLR fit for production use within FreeBSD. [ 2. History ] On 14 May 2013, Oliver Pinter published to GitHub an initial patch[2]. His = work was inspired by Elad Efrat's work in NetBSD. The patch was submitted to Fre= eBSD as a bug report on 24 Aug 2013[3]. Independently of Oliver's work, on 30 Jun 2014, Shawn Webb posted on his tech blog that he was interested in implemen= ting ASLR for FreeBSD[4]. Oliver found the post and suggested that he and Shawn = work together. On 08 Jun 2014, preparatory work was committed to FreeBSD, adding Position-Independent Executable (PIE) support in base[5]. On 07 Apr 2014, SoldierX[6] agreed to sponsor the project and donated a sparc64 box and a beaglebone black to Shawn Webb. This hardware is used for testing and debug= ging ASLR on those platforms. [ 3. General Overview ] ASLR is enabled by default for all architectures and controlled by the PAX_= ASLR kernel option. This means ASLR will be applied to all supported application= s. If a user wishes to disable ASLR for a given application, the user must for= ce that application to opt-out (detailed later). Another kernel option, PAX_SYSCTLS, exposes additional tunables (via sysctl= ), allowing ASLR behavior control without requiring a reboot. By default, the sysctl security.pax.aslr.status can only be changed at boot time via /boot/loader.conf. Enabling the PAX_SYSCTLS kernel option allows a root user to modify security.pax.aslr.status. See Appendix B for a list of the tunabl= es. ASLR tunables are per-jail and each jail inherits its parent jail's setting= s. Having per-jail tunables allows more flexibility in shared-hosting environments. This structure also allows a user to selectively disable ASLR for applications that misbehave. ASLR-disabled applications will still have policy-based security applied to it by virtue of being jailed. The mac_bsdextended(4) MAC module and its corresponding ugidfw(8) applicati= on have been modified to allow a user to enable or disable ASLR for specific applications. The filesys object specification has been modified to pass the inode along with the filesystem id when the new paxflags option is specifie= d. The paxflags option is optionally placed at the end of the rule. An upper-c= ase "A" argument to the option signifies ASLR is enabled for the application an= d a lower-case "a" signifies ASLR is disabled for the application. Sample ugidf= w(8) rules are in Appendix C. [ 4. Implementation Details ] A new sysinit subroutine ID, SI_SUB_PAX, initializes all ASLR system variab= les.=20 Upon system boot, tunables from /boot/loader.conf are checked for validity.= Any invalid values, generate a warning message to the console and the tunable is set to a sensible default. For the sake of performance, the ASLR system relies on per-process deltas rather than calling arc4random(3) for each mapping. When a process calls execve(2), the ASLR system is initialized. Deltas are randomly generated for the execution base, mmap(2), and stack addresses. Only the execution base of applications compiled as PIEs are randomized. The execution base of non-PIE applications are not modified. The mappings of shared objects are randomized for both PIE and non-PIE applications. The deltas are used as a hint to the Virtual Memory (VM) system. The VM sys= tem may modify the hint to make a better fit for superpages and other alignment constraints. The delta applied to the PIE execbase is different than the delta applied to the base address of shared objects. In the Executable and Linkable File (EL= F) image handler, the execution base of PIE applications is randomized by addi= ng the delta controlled by security.pax.aslr.exec_len tunable to et_dyn_addr, which is initialized to be ET_DYN_LOAD_ADDR (an architecture-dependent macr= o). The base address of shared objects loaded by the runtime linker are randomi= zed by applying the delta controlled by the security.pax.aslr.mmap_len tunable = in sys_mmap(). Stack randomization is implemented using a stack gap[7]. On executable image activation, the stack delta is computed and then subtracted from the top of the stack. [ 5. Further Enhancements ] The existing gap-based stack randomization is not optimal. Mapping-base sta= ck randomization is more robust, but hard-coded kernel structures and addresse= s, especially PS_STRINGS, will need to be modified. The required changes to PS_STRINGS are major and will likely touch userland along with the kernel. The original PaX implementation, from which the FreeBSD implementation is inspired, uses a special ELF process header which requires modification of executable files. The authors of the FreeBSD implementation have deliberate= ly chosen to go a different route based on mac_bsdextended(4)/ugidfw(8). Suppo= rt for filesystem extended attributes will be added at a later time. FreeBSD's virtual Dynamic Shared Object (vDSO) implementation, an efficient technique for calling kernel code from userland, uses hardcoded, non-randomized addresses. The vDSO implementation should be reworked to be = at a randomized address, providing the address as an auxiliary vector passed to the image via the stack. [ 6. Known Issues ] ASLR does not function properly on 32bit ARM. When a process fork(2)s and c= alls execve(2) and the child process exits, the parent process crashes upon receiving the SIGCHLD signal. No matter which application crashed, the pc register ends up being 0xc0000000. The ktrace(1) facility proved that the application crashed upon receiving the SIGCHLD signal. [ Appendix A - References ] [1]: http://pax.grsecurity.net/docs/aslr.txt [2]: https://github.com/opntr/freebsd-patches-2013-tavasz/blob/master/r2499= 52%2BASLR/0001-PaX-ASLR-mmap-and-basic-stack-randomization.patch [3]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D181497 [4]: http://0xfeedface.org/blog/lattera/2013-06-29/long-term-plans [5]: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D267233 [6]: https://www.soldierx.com/ [7]: http://www.openbsd.org/papers/auug04/mgp00005.html [ Appendix B - ASLR Tunables ] NOTE: All tunables can only be changed during boot-time via /boot/loader.co= nf unless the kernel has been compiled with PAX_SYSCTLS. security.pax.aslr.status (integer): Description: Toggle system-wide ASLR protection. Values: 0 - ASLR disabled system-wide. Individual applications may NOT opt in. 1 - ASLR disabled by default. Individual applications may opt in. 2 - ASLR enabled by default. Individual applications may opt out. 3 - ASLR enabled system-wide. Individual applications may NOT opt out. Default: 2 security.pax.aslr.debug (integer): Description: Toggle debugging output. Values: 0 - Debug output disabled. 1 - Basic debug output enabled. 2 - Verbose debug output enabled. Default: 0 security.pax.aslr.mmap_len (integer): Description: Set the number of bits to be randomized for mmap(2) calls. Values: For 32bit systems, minimum of 8, maximum of 16. For 64bit systems, minimum of 16, maximum of 32. Default: For 32bit systems, 8. For 64bit systems, 21. security.pax.aslr.stack_len (integer): Description: Set the number of bits to be randomized for the stack. Values: For 32bit systems, minimum of 6, maximum of 12. For 64bit systems, minimum of 12, maximum of 21. Default: For 32bit systems, 6. For 64bit systems, 16. security.pax.aslr.exec_len (integer): Description: Set the number of bits to be randomized for the PIE exec base. Values: For 32bit systems, minimum of 6, maximum of 12. For 64bit systems, minimum of 12, maximum of 21. Default: For 32bit systems, 6. For 64bit systems, 21. [ Appendix C - Sample ugidfw(8) rules ] When security.pax.aslr.status is set to 2 (require applications to opt-out): ugidfw add subject uid shawn object filesys /bin/ls mode rx paxflags a - This adds a rule to disable ASLR for /bin/ls for the user shawn. ugidfw add subject uid 0:65535 object filesys /bin/ls mode rx paxflags a - This adds a rule to disable ASLR for /bin/ls for all users. When security.pax.aslr.status is set to 1 (require applications to opt-in): ugidfw add subject uid shawn object filesys /bin/ls mode rx paxflags A - This adds a rule to enable ASLR for /bin/ls for the user shawn. ugidfw add subject uid 0:65535 object filesys /bin/ls mode rx paxflags A - This adds a rule to enable ASLR for /bin/ls for all users. [ Appendix D - Files Modified/Created in 11-CURRENT ] lib/libugidfw/ugidfw.c lib/libugidfw/ugidfw.h release/Makefile sys/amd64/amd64/elf_machdep.c sys/amd64/include/vmparam.h sys/amd64/linux32/linux32_sysvec.c sys/arm/arm/elf_machdep.c sys/compat/freebsd32/freebsd32_misc.c sys/compat/ia32/ia32_sysvec.c sys/conf/NOTES sys/conf/files sys/conf/options sys/i386/i386/elf_machdep.c sys/i386/ibcs2/ibcs2_sysvec.c sys/i386/linux/linux_sysvec.c sys/kern/imgact_aout.c sys/kern/imgact_elf.c sys/kern/init_main.c sys/kern/kern_exec.c sys/kern/kern_fork.c sys/kern/kern_jail.c sys/kern/kern_pax.c sys/kern/kern_pax_aslr.c sys/kern/kern_pax_log.c sys/kern/kern_sig.c sys/mips/mips/elf_machdep.c sys/mips/mips/freebsd32_machdep.c sys/powerpc/powerpc/elf32_machdep.c sys/powerpc/powerpc/elf64_machdep.c sys/security/mac_bsdextended/mac_bsdextended.c sys/security/mac_bsdextended/mac_bsdextended.h sys/security/mac_bsdextended/ugidfw_internal.h sys/security/mac_bsdextended/ugidfw_system.c sys/security/mac_bsdextended/ugidfw_vnode.c sys/sparc64/sparc64/elf_machdep.c sys/sys/imgact.h sys/sys/jail.h sys/sys/kernel.h sys/sys/pax.h sys/sys/proc.h sys/sys/sysent.h sys/vm/vm_map.c sys/vm/vm_map.h sys/vm/vm_mmap.c usr.sbin/ugidfw/ugidfw.c --vkEkAx9hr54EJ73W Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="2014-07-10_aslr.patch" Content-Transfer-Encoding: quoted-printable diff --git a/lib/libugidfw/ugidfw.c b/lib/libugidfw/ugidfw.c index 0dc423d..070cd0e 100644 --- a/lib/libugidfw/ugidfw.c +++ b/lib/libugidfw/ugidfw.c @@ -36,6 +36,9 @@ #include #include #include +#include +#include +#include =20 #include =20 @@ -44,6 +47,8 @@ #include #include #include +#include +#include =20 #include "ugidfw.h" =20 @@ -195,7 +200,7 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule, = char *buf, size_t buflen) cur +=3D len; } if (rule->mbr_subject.mbs_flags & MBS_PRISON_DEFINED) { - len =3D snprintf(cur, left, "jailid %d ",=20 + len =3D snprintf(cur, left, "jailid %d ", rule->mbr_subject.mbs_prison); if (len < 0 || len > left) goto truncated; @@ -329,14 +334,19 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule= , char *buf, size_t buflen) cur +=3D len; } if (rule->mbr_object.mbo_flags & MBO_FSID_DEFINED) { - numfs =3D getmntinfo(&mntbuf, MNT_NOWAIT); - for (i =3D 0; i < numfs; i++) - if (memcmp(&(rule->mbr_object.mbo_fsid), - &(mntbuf[i].f_fsid), - sizeof(mntbuf[i].f_fsid)) =3D=3D 0) - break; - len =3D snprintf(cur, left, "filesys %s ",=20 - i =3D=3D numfs ? "???" : mntbuf[i].f_mntonname); + if (rule->mbr_object.mbo_inode =3D=3D 0) { + numfs =3D getmntinfo(&mntbuf, MNT_NOWAIT); + for (i =3D 0; i < numfs; i++) + if (memcmp(&(rule->mbr_object.mbo_fsid), + &(mntbuf[i].f_fsid), + sizeof(mntbuf[i].f_fsid)) =3D=3D 0) + break; + len =3D snprintf(cur, left, "filesys %s ", + i =3D=3D numfs ? "???" : mntbuf[i].f_mntonname); + } else { + len =3D snprintf(cur, left, "filesys %s ", + rule->mbr_object.mbo_paxpath); + } if (len < 0 || len > left) goto truncated; left -=3D len; @@ -500,6 +510,33 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule,= char *buf, size_t buflen) cur +=3D len; } =20 + if (rule->mbr_pax) { + len =3D snprintf(cur, left, " paxflags "); + if (len < 0 || len > left) + goto truncated; + left -=3D len; + cur +=3D len; + + if (rule->mbr_pax & MBI_FORCE_ASLR_ENABLED) { + len =3D snprintf(cur, left, "A"); + if (len < 0 || len > left) + goto truncated; + + left -=3D len; + cur +=3D len; + } + + if (rule->mbr_pax & MBI_FORCE_ASLR_DISABLED) { + len =3D snprintf(cur, left, "a"); + if (len < 0 || len > left) + goto truncated; + + left -=3D len; + cur +=3D len; + } + + } + return (0); =20 truncated: @@ -507,8 +544,8 @@ truncated: } =20 int -bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, - size_t buflen, char *errstr){ +bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, size_t buflen, cha= r *errstr) +{ struct passwd *pwd; uid_t uid1, uid2; char *spec1, *spec2, *endp; @@ -556,8 +593,8 @@ bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, } =20 int -bsde_parse_gidrange(char *spec, gid_t *min, gid_t *max, - size_t buflen, char *errstr){ +bsde_parse_gidrange(char *spec, gid_t *min, gid_t *max, size_t buflen, cha= r *errstr) +{ struct group *grp; gid_t gid1, gid2; char *spec1, *spec2, *endp; @@ -759,17 +796,22 @@ bsde_parse_type(char *spec, int *type, size_t buflen,= char *errstr) len =3D snprintf(errstr, buflen, "Unknown type code: %c", spec[i]); return (-1); - }=20 + } } =20 return (0); } =20 int -bsde_parse_fsid(char *spec, struct fsid *fsid, size_t buflen, char *errstr) +bsde_parse_fsid(char *spec, struct fsid *fsid, ino_t *inode, size_t buflen= , char *errstr) { size_t len; struct statfs buf; + struct stat sb; + int fd, paxstatus; + size_t bufsz; + + *inode =3D 0; =20 if (statfs(spec, &buf) < 0) { len =3D snprintf(errstr, buflen, "Unable to get id for %s: %s", @@ -779,6 +821,21 @@ bsde_parse_fsid(char *spec, struct fsid *fsid, size_t = buflen, char *errstr) =20 *fsid =3D buf.f_fsid; =20 + if (strcmp(buf.f_fstypename, "devfs") !=3D 0) { + bufsz =3D sizeof(int); + if (!sysctlbyname("kern.features.aslr", &paxstatus, &bufsz, + NULL, 0)) { + fd =3D open(spec, O_RDONLY); + if (fd !=3D -1) { + if (fstat(fd, &sb) =3D=3D 0) + if(S_ISDIR(sb.st_mode) =3D=3D 0) + *inode =3D sb.st_ino; + + close(fd); + } + } + } + return (0); } =20 @@ -852,13 +909,18 @@ bsde_parse_object(int argc, char *argv[], return (-1); } if (bsde_parse_fsid(argv[current+1], &fsid, - buflen, errstr) < 0) + &object->mbo_inode, buflen, errstr) < 0) return (-1); flags |=3D MBO_FSID_DEFINED; if (nextnot) { neg ^=3D MBO_FSID_DEFINED; nextnot =3D 0; } + if (object->mbo_inode) + snprintf(object->mbo_paxpath, MAXPATHLEN, "%s", + argv[current+1]); + else + memset(object->mbo_paxpath, 0x00, MAXPATHLEN); current +=3D 2; } else if (strcmp(argv[current], "suid") =3D=3D 0) { flags |=3D MBO_SUID; @@ -984,7 +1046,42 @@ bsde_parse_mode(int argc, char *argv[], mode_t *mode,= size_t buflen, len =3D snprintf(errstr, buflen, "Unknown mode letter: %c", argv[0][i]); return (-1); - }=20 + } + } + + return (0); +} + +int +bsde_parse_paxflags(int argc, char *argv[], uint32_t *pax, size_t buflen, = char *errstr) +{ + size_t len; + int i; + + if (argc =3D=3D 0) { + len =3D snprintf(errstr, buflen, "paxflags expects mode value"); + return (-1); + } + + if (argc !=3D 1) { + len =3D snprintf(errstr, buflen, "'%s' unexpected", argv[1]); + return (-1); + } + + *pax =3D 0; + for (i =3D 0; i < strlen(argv[0]); i++) { + switch (argv[0][i]) { + case 'A': + *pax |=3D MBI_FORCE_ASLR_ENABLED; + break; + case 'a': + *pax |=3D MBI_FORCE_ASLR_DISABLED; + break; + default: + len =3D snprintf(errstr, buflen, "Unknown mode letter: %c", + argv[0][i]); + return (-1); + } } =20 return (0); @@ -997,6 +1094,7 @@ bsde_parse_rule(int argc, char *argv[], struct mac_bsd= extended_rule *rule, int subject, subject_elements, subject_elements_length; int object, object_elements, object_elements_length; int mode, mode_elements, mode_elements_length; + int paxflags, paxflags_elements, paxflags_elements_length=3D0; int error, i; size_t len; =20 @@ -1037,11 +1135,23 @@ bsde_parse_rule(int argc, char *argv[], struct mac_= bsdextended_rule *rule, return (-1); } =20 + /* Search forward for paxflags */ + paxflags =3D -1; + for (i =3D 1; i < argc; i++) + if (strcmp(argv[i], "paxflags") =3D=3D 0) + paxflags =3D i; + + if (paxflags >=3D 0) { + paxflags_elements =3D paxflags + 1; + paxflags_elements_length =3D argc - paxflags_elements; + } + subject_elements_length =3D object - subject - 1; object_elements =3D object + 1; object_elements_length =3D mode - object_elements; mode_elements =3D mode + 1; - mode_elements_length =3D argc - mode_elements; + mode_elements_length =3D argc - mode_elements - + (paxflags_elements_length ? paxflags_elements_length+1 : 0); =20 error =3D bsde_parse_subject(subject_elements_length, argv + subject_elements, &rule->mbr_subject, buflen, errstr); @@ -1058,6 +1168,13 @@ bsde_parse_rule(int argc, char *argv[], struct mac_b= sdextended_rule *rule, if (error) return (-1); =20 + if (paxflags >=3D 0) { + error =3D bsde_parse_paxflags(paxflags_elements_length, argv + paxflags_= elements, + &rule->mbr_pax, buflen, errstr); + if (error) + return (-1); + } + return (0); } =20 diff --git a/lib/libugidfw/ugidfw.h b/lib/libugidfw/ugidfw.h index 5b7fcf2..cef469c 100644 --- a/lib/libugidfw/ugidfw.h +++ b/lib/libugidfw/ugidfw.h @@ -39,6 +39,8 @@ int bsde_rule_to_string(struct mac_bsdextended_rule *rule= , char *buf, size_t buflen); int bsde_parse_mode(int argc, char *argv[], mode_t *mode, size_t buflen, char *errstr); +int bsde_parse_paxflags(int argc, char *argv[], uint32_t *pax, size_t bufl= en, + char *errstr); int bsde_parse_rule(int argc, char *argv[], struct mac_bsdextended_rule *rule, size_t buflen, char *errstr); int bsde_parse_rule_string(const char *string, diff --git a/sys/amd64/amd64/elf_machdep.c b/sys/amd64/amd64/elf_machdep.c index fdc4d56..ffb5e31 100644 --- a/sys/amd64/amd64/elf_machdep.c +++ b/sys/amd64/amd64/elf_machdep.c @@ -26,12 +26,17 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include #include #include #include +#ifdef PAX_ASLR +#include +#endif #include #include #include @@ -81,6 +86,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf64_sysvec, &elf64_freebsd_sysvec); =20 diff --git a/sys/amd64/include/vmparam.h b/sys/amd64/include/vmparam.h index bda9722..5e83a8f 100644 --- a/sys/amd64/include/vmparam.h +++ b/sys/amd64/include/vmparam.h @@ -170,7 +170,7 @@ #define VM_MAXUSER_ADDRESS UVADDR(NUPML4E, 0, 0, 0) =20 #define SHAREDPAGE (VM_MAXUSER_ADDRESS - PAGE_SIZE) -#define USRSTACK SHAREDPAGE +#define USRSTACK (SHAREDPAGE - 4*PAGE_SIZE) =20 #define VM_MAX_ADDRESS UPT_MAX_ADDRESS #define VM_MIN_ADDRESS (0) diff --git a/sys/amd64/linux32/linux32_sysvec.c b/sys/amd64/linux32/linux32= _sysvec.c index c06ce11..f4f99f58 100644 --- a/sys/amd64/linux32/linux32_sysvec.c +++ b/sys/amd64/linux32/linux32_sysvec.c @@ -33,6 +33,7 @@ #include __FBSDID("$FreeBSD$"); #include "opt_compat.h" +#include "opt_pax.h" =20 #ifndef COMPAT_FREEBSD32 #error "Unable to compile Linux-emulator due to missing COMPAT_FREEBSD32 o= ption!" @@ -84,6 +85,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(linux, 1); =20 MALLOC_DEFINE(M_LINUX, "linux", "Linux mode structures"); @@ -1037,6 +1042,11 @@ struct sysentvec elf_linux_sysvec =3D { .sv_shared_page_base =3D LINUX32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_sysvec, &elf_linux_sysvec); =20 diff --git a/sys/arm/arm/elf_machdep.c b/sys/arm/arm/elf_machdep.c index 8ef9bd4..26e37e6 100644 --- a/sys/arm/arm/elf_machdep.c +++ b/sys/arm/arm/elf_machdep.c @@ -26,6 +26,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -46,6 +48,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf32_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -79,6 +85,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf32_Brandinfo freebsd_brand_info =3D { diff --git a/sys/compat/freebsd32/freebsd32_misc.c b/sys/compat/freebsd32/f= reebsd32_misc.c index 815a9b7..e92453f 100644 --- a/sys/compat/freebsd32/freebsd32_misc.c +++ b/sys/compat/freebsd32/freebsd32_misc.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include "opt_compat.h" #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -113,6 +114,10 @@ __FBSDID("$FreeBSD$"); =20 FEATURE(compat_freebsd_32bit, "Compatible with 32-bit FreeBSD"); =20 +#ifdef PAX_ASLR +#include +#endif + #ifndef __mips__ CTASSERT(sizeof(struct timeval32) =3D=3D 8); CTASSERT(sizeof(struct timespec32) =3D=3D 8); @@ -2766,6 +2771,10 @@ freebsd32_copyout_strings(struct image_params *imgp) szsigcode =3D 0; destp =3D (uintptr_t)arginfo; =20 +#ifdef PAX_ASLR + pax_aslr_stack(curthread, &destp); +#endif + /* * install sigcode */ diff --git a/sys/compat/ia32/ia32_sysvec.c b/sys/compat/ia32/ia32_sysvec.c index 770bbbf..67ad330 100644 --- a/sys/compat/ia32/ia32_sysvec.c +++ b/sys/compat/ia32/ia32_sysvec.c @@ -29,6 +29,7 @@ __FBSDID("$FreeBSD$"); =20 #include "opt_compat.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -74,6 +75,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + CTASSERT(sizeof(struct ia32_mcontext) =3D=3D 640); CTASSERT(sizeof(struct ia32_ucontext) =3D=3D 704); CTASSERT(sizeof(struct ia32_sigframe) =3D=3D 800); @@ -136,6 +141,11 @@ struct sysentvec ia32_freebsd_sysvec =3D { .sv_shared_page_base =3D FREEBSD32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_ia32_sysvec, &ia32_freebsd_sysvec); =20 diff --git a/sys/conf/NOTES b/sys/conf/NOTES index 6959425..bd6af74 100644 --- a/sys/conf/NOTES +++ b/sys/conf/NOTES @@ -2986,3 +2986,7 @@ options RANDOM_RWFILE # Read and write entropy cache =20 # Module to enable execution of application via emulators like QEMU options IMAGACT_BINMISC + +# Address Space Layout Randomization (ASLR) +options PAX_ASLR +options PAX_SYSCTLS diff --git a/sys/conf/files b/sys/conf/files index 64101d1..71f8c1a 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -2908,6 +2908,9 @@ kern/kern_mtxpool.c standard kern/kern_mutex.c standard kern/kern_ntptime.c standard kern/kern_osd.c standard +kern/kern_pax.c optional pax_aslr +kern/kern_pax_aslr.c optional pax_aslr +kern/kern_pax_log.c optional pax_aslr kern/kern_physio.c standard kern/kern_pmc.c standard kern/kern_poll.c optional device_polling diff --git a/sys/conf/options b/sys/conf/options index 8ec07e0..180f169 100644 --- a/sys/conf/options +++ b/sys/conf/options @@ -920,6 +920,12 @@ RACCT opt_global.h # Resource Limits RCTL opt_global.h =20 +# PaX - hardening options +PAX_ASLR opt_pax.h +PAX_ASLR_MAX_SEC opt_pax.h +PAX_MPROTECT opt_pax.h +PAX_SYSCTLS opt_pax.h + # Random number generator(s) RANDOM_YARROW opt_random.h RANDOM_FORTUNA opt_random.h diff --git a/sys/i386/i386/elf_machdep.c b/sys/i386/i386/elf_machdep.c index 034b4c4..9571252 100644 --- a/sys/i386/i386/elf_machdep.c +++ b/sys/i386/i386/elf_machdep.c @@ -26,6 +26,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -46,6 +48,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf32_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -81,6 +87,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/i386/ibcs2/ibcs2_sysvec.c b/sys/i386/ibcs2/ibcs2_sysvec.c index 5d007c7..1bb9d89 100644 --- a/sys/i386/ibcs2/ibcs2_sysvec.c +++ b/sys/i386/ibcs2/ibcs2_sysvec.c @@ -31,6 +31,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -50,6 +52,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(ibcs2, 1); =20 extern int bsd_to_ibcs2_errno[]; @@ -89,6 +95,11 @@ struct sysentvec ibcs2_svr3_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D NULL, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static int diff --git a/sys/i386/linux/linux_sysvec.c b/sys/i386/linux/linux_sysvec.c index 0ad6791..403070c 100644 --- a/sys/i386/linux/linux_sysvec.c +++ b/sys/i386/linux/linux_sysvec.c @@ -29,6 +29,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -72,6 +74,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(linux, 1); =20 MALLOC_DEFINE(M_LINUX, "linux", "Linux mode structures"); @@ -974,6 +980,11 @@ struct sysentvec linux_sysvec =3D { .sv_shared_page_base =3D LINUX_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(aout_sysvec, &linux_sysvec); =20 @@ -1012,6 +1023,11 @@ struct sysentvec elf_linux_sysvec =3D { .sv_shared_page_base =3D LINUX_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_sysvec, &elf_linux_sysvec); =20 diff --git a/sys/kern/imgact_aout.c b/sys/kern/imgact_aout.c index 3ae78de..aac03f1 100644 --- a/sys/kern/imgact_aout.c +++ b/sys/kern/imgact_aout.c @@ -27,6 +27,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -62,6 +64,10 @@ __FBSDID("$FreeBSD$"); #include #endif =20 +#ifdef PAX_ASLR +#include +#endif + static int exec_aout_imgact(struct image_params *imgp); static int aout_fixup(register_t **stack_base, struct image_params *imgp); =20 @@ -99,6 +105,11 @@ struct sysentvec aout_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 #elif defined(__amd64__) @@ -143,6 +154,11 @@ struct sysentvec aout_sysvec =3D { .sv_set_syscall_retval =3D ia32_set_syscall_retval, .sv_fetch_syscall_args =3D ia32_fetch_syscall_args, .sv_syscallnames =3D freebsd32_syscallnames, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; #else #error "Port me" diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c index 6342119..fe2976d 100644 --- a/sys/kern/imgact_elf.c +++ b/sys/kern/imgact_elf.c @@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_compat.h" #include "opt_core.h" +#include "opt_pax.h" =20 #include #include @@ -48,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -81,6 +83,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#if defined(PAX_ASLR) +#include +#endif + #define ELF_NOTE_ROUNDSIZE 4 #define OLD_EI_BRAND 8 =20 @@ -653,16 +659,16 @@ __elfN(load_file)(struct proc *p, const char *file, u= _long *addr, hdr =3D (const Elf_Ehdr *)imgp->image_header; if ((error =3D __elfN(check_header)(hdr)) !=3D 0) goto fail; - if (hdr->e_type =3D=3D ET_DYN) + if (hdr->e_type =3D=3D ET_DYN) { rbase =3D *addr; - else if (hdr->e_type =3D=3D ET_EXEC) + } else if (hdr->e_type =3D=3D ET_EXEC) { rbase =3D 0; - else { + } else { error =3D ENOEXEC; goto fail; } =20 - /* Only support headers that fit within first page for now */ + /* Only support headers that fit within first page for now */ if ((hdr->e_phoff > PAGE_SIZE) || (u_int)hdr->e_phentsize * hdr->e_phnum > PAGE_SIZE - hdr->e_phoff) { error =3D ENOEXEC; @@ -787,16 +793,7 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *i= mgp) if (hdr->e_type =3D=3D ET_DYN) { if ((brand_info->flags & BI_CAN_EXEC_DYN) =3D=3D 0) return (ENOEXEC); - /* - * Honour the base load address from the dso if it is - * non-zero for some reason. - */ - if (baddr =3D=3D 0) - et_dyn_addr =3D ET_DYN_LOAD_ADDR; - else - et_dyn_addr =3D 0; - } else - et_dyn_addr =3D 0; + } sv =3D brand_info->sysvec; if (interp !=3D NULL && brand_info->interp_newpath !=3D NULL) newinterp =3D brand_info->interp_newpath; @@ -817,6 +814,21 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *i= mgp) error =3D exec_new_vmspace(imgp, sv); imgp->proc->p_sysent =3D sv; =20 + et_dyn_addr =3D 0; + if (hdr->e_type =3D=3D ET_DYN) { + /* + * Honour the base load address from the dso if it is + * non-zero for some reason. + */ + if (baddr =3D=3D 0) { + et_dyn_addr =3D ET_DYN_LOAD_ADDR; +#ifdef PAX_ASLR + if (pax_aslr_active(NULL, imgp->proc)) + et_dyn_addr +=3D imgp->proc->p_vmspace->vm_aslr_delta_exec; +#endif + } + } + vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY); if (error) return (error); diff --git a/sys/kern/init_main.c b/sys/kern/init_main.c index 141d438..9301b57 100644 --- a/sys/kern/init_main.c +++ b/sys/kern/init_main.c @@ -410,6 +410,7 @@ struct sysentvec null_sysvec =3D { .sv_fetch_syscall_args =3D null_fetch_syscall_args, .sv_syscallnames =3D NULL, .sv_schedtail =3D NULL, + .sv_pax_aslr_init =3D NULL, }; =20 /* diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c index 489096b..81a451f 100644 --- a/sys/kern/kern_exec.c +++ b/sys/kern/kern_exec.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_hwpmc_hooks.h" #include "opt_ktrace.h" +#include "opt_pax.h" #include "opt_vm.h" =20 #include @@ -93,6 +94,10 @@ __FBSDID("$FreeBSD$"); dtrace_execexit_func_t dtrace_fasttrap_exec; #endif =20 +#if defined(PAX_ASLR) +#include +#endif + SDT_PROVIDER_DECLARE(proc); SDT_PROBE_DEFINE1(proc, kernel, , exec, "char *"); SDT_PROBE_DEFINE1(proc, kernel, , exec__failure, "int"); @@ -402,6 +407,11 @@ do_execve(td, args, mac_p) imgp->pagesizes =3D 0; imgp->pagesizeslen =3D 0; imgp->stack_prot =3D 0; + imgp->pax_flags =3D 0; + +#if defined(PAX_MPROTECT) || defined(PAX_ASLR) + pax_elf(imgp, 0); +#endif =20 #ifdef MAC error =3D mac_execve_enter(imgp, mac_p); @@ -1065,6 +1075,10 @@ exec_new_vmspace(imgp, sv) map =3D &vmspace->vm_map; } =20 +#ifdef PAX_ASLR + pax_aslr_init(curthread, imgp); +#endif + /* Map a shared page */ obj =3D sv->sv_shared_page_obj; if (obj !=3D NULL) { @@ -1099,6 +1113,9 @@ exec_new_vmspace(imgp, sv) */ vmspace->vm_ssize =3D sgrowsiz >> PAGE_SHIFT; vmspace->vm_maxsaddr =3D (char *)sv->sv_usrstack - ssiz; +#ifdef PAX_ASLR + vmspace->vm_maxsaddr -=3D vmspace->vm_aslr_delta_stack; +#endif =20 return (0); } @@ -1258,6 +1275,9 @@ exec_copyout_strings(imgp) szsigcode =3D *(p->p_sysent->sv_szsigcode); } destp =3D (uintptr_t)arginfo; +#ifdef PAX_ASLR + pax_aslr_stack(curthread, &destp); +#endif =20 /* * install sigcode diff --git a/sys/kern/kern_fork.c b/sys/kern/kern_fork.c index 8135afb..9577013 100644 --- a/sys/kern/kern_fork.c +++ b/sys/kern/kern_fork.c @@ -513,6 +513,11 @@ do_fork(struct thread *td, int flags, struct proc *p2,= struct thread *td2, } =20 /* + * XXXOP: this is the right place? + */ + p2->p_pax =3D p1->p_pax; + + /* * p_limit is copy-on-write. Bump its refcount. */ lim_fork(p1, p2); diff --git a/sys/kern/kern_jail.c b/sys/kern/kern_jail.c index 47cd568..d9036bd 100644 --- a/sys/kern/kern_jail.c +++ b/sys/kern/kern_jail.c @@ -33,6 +33,7 @@ __FBSDID("$FreeBSD$"); #include "opt_ddb.h" #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_pax.h" =20 #include #include @@ -74,6 +75,10 @@ __FBSDID("$FreeBSD$"); #endif /* INET6 */ #endif /* DDB */ =20 +#if defined(PAX_ASLR) +#include +#endif + #include =20 #define DEFAULT_HOSTUUID "00000000-0000-0000-0000-000000000000" @@ -117,6 +122,10 @@ struct prison prison0 =3D { }; MTX_SYSINIT(prison0, &prison0.pr_mtx, "jail mutex", MTX_DEF); =20 +#if defined(PAX_ASLR) +SYSINIT(pax, SI_SUB_PAX, SI_ORDER_MIDDLE, pax_init_prison, (void *) &priso= n0); +#endif + /* allprison, allprison_racct and lastprid are protected by allprison_lock= =2E */ struct sx allprison_lock; SX_SYSINIT(allprison_lock, &allprison_lock, "allprison"); @@ -1307,6 +1316,10 @@ kern_jail_set(struct thread *td, struct uio *optuio,= int flags) goto done_releroot; } =20 +#if defined(PAX_ASLR) + pax_init_prison(pr); +#endif + mtx_lock(&pr->pr_mtx); /* * New prisons do not yet have a reference, because we do not diff --git a/sys/kern/kern_pax.c b/sys/kern/kern_pax.c new file mode 100644 index 0000000..1bd5ad0 --- /dev/null +++ b/sys/kern/kern_pax.c @@ -0,0 +1,214 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "opt_compat.h" +#include "opt_pax.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include + +#include + +#include + +SYSCTL_NODE(_security, OID_AUTO, pax, CTLFLAG_RD, 0, + "PaX (exploit mitigation) features."); + +struct prison * +pax_get_prison(struct thread *td, struct proc *proc) +{ + + if (td !=3D NULL) { + if ((td->td_proc !=3D NULL) && (td->td_proc->p_ucred !=3D NULL)) + return (td->td_proc->p_ucred->cr_prison); + + return (NULL); + } + if ((proc =3D=3D NULL) || (proc->p_ucred =3D=3D NULL)) + return (NULL); + + return (proc->p_ucred->cr_prison); +} + +void +pax_elf(struct image_params *imgp, uint32_t mode) +{ + u_int flags =3D 0; + + if ((mode & MBI_ALLPAX) =3D=3D MBI_ALLPAX) + goto end; + + if (mode & MBI_FORCE_ASLR_ENABLED) + flags |=3D PAX_NOTE_ASLR; + else if (mode & MBI_FORCE_ASLR_DISABLED) + flags |=3D PAX_NOTE_NOASLR; + +end: + if (imgp !=3D NULL) { + imgp->pax_flags =3D flags; + if (imgp->proc !=3D NULL) { + PROC_LOCK(imgp->proc); + imgp->proc->p_pax =3D flags; + PROC_UNLOCK(imgp->proc); + } + } +} + + +/* + * print out PaX settings on boot time, and validate some of them + */ +void +pax_init(void) +{ +#if defined(PAX_ASLR) + const char *status_str[] =3D { + [0] =3D "disabled", + [1] =3D "opt-in", + [2] =3D "opt-out", + [3] =3D "force enabled", + [4] =3D "UNKNOWN -> changed to \"force enabled\"" + }; +#endif + +#ifdef PAX_ASLR + switch (pax_aslr_status) { + case 0: + case 1: + case 2: + case 3: + break; + default: + printf("[PAX ASLR] WARNING, invalid PAX settings in loader.conf!" + " (pax_aslr_status =3D %d)\n", pax_aslr_status); + pax_aslr_status =3D 3; + break; + } + printf("[PAX ASLR] status: %s\n", status_str[pax_aslr_status]); + printf("[PAX ASLR] mmap: %d bit\n", pax_aslr_mmap_len); + printf("[PAX ASLR] exec base: %d bit\n", pax_aslr_exec_len); + printf("[PAX ASLR] stack: %d bit\n", pax_aslr_stack_len); + +#ifdef COMPAT_FREEBSD32 + switch (pax_aslr_compat_status) { + case 0: + case 1: + case 2: + case 3: + break; + default: + printf("[PAX ASLR (compat)] WARNING, invalid PAX settings in loader.conf= ! " + "(pax_aslr_compat_status =3D %d)\n", pax_aslr_compat_status); + pax_aslr_compat_status =3D 3; + break; + } + printf("[PAX ASLR (compat)] status: %s\n", status_str[pax_aslr_compat_sta= tus]); + printf("[PAX ASLR (compat)] mmap: %d bit\n", pax_aslr_compat_mmap_len); + printf("[PAX ASLR (compat)] exec base: %d bit\n", pax_aslr_compat_exec_le= n); + printf("[PAX ASLR (compat)] stack: %d bit\n", pax_aslr_compat_stack_len); +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + + printf("[PAX LOG] logging to system: %d\n", pax_log_log); + printf("[PAX LOG] logging to user: %d\n", pax_log_ulog); +} +SYSINIT(pax, SI_SUB_PAX, SI_ORDER_FIRST, pax_init, NULL); + +void +pax_init_prison(struct prison *pr) +{ + + if (pr =3D=3D NULL) + return; + + if (pr->pr_pax_set) + return; + + mtx_lock(&(pr->pr_mtx)); + + if (pax_aslr_debug) + uprintf("[PaX ASLR] %s: Setting prison %s ASLR variables\n", + __func__, pr->pr_name); + +#ifdef PAX_ASLR + pr->pr_pax_aslr_status =3D pax_aslr_status; + pr->pr_pax_aslr_debug =3D pax_aslr_debug; + pr->pr_pax_aslr_mmap_len =3D pax_aslr_mmap_len; + pr->pr_pax_aslr_stack_len =3D pax_aslr_stack_len; + pr->pr_pax_aslr_exec_len =3D pax_aslr_exec_len; + +#ifdef COMPAT_FREEBSD32 + pr->pr_pax_aslr_compat_status =3D pax_aslr_compat_status; + pr->pr_pax_aslr_compat_mmap_len =3D pax_aslr_compat_mmap_len; + pr->pr_pax_aslr_compat_stack_len =3D pax_aslr_compat_stack_len; + pr->pr_pax_aslr_compat_exec_len =3D pax_aslr_compat_exec_len; +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + + pr->pr_pax_log_log =3D pax_log_log; + pr->pr_pax_log_ulog =3D pax_log_ulog; + + pr->pr_pax_set =3D 1; + + mtx_unlock(&(pr->pr_mtx)); +} diff --git a/sys/kern/kern_pax_aslr.c b/sys/kern/kern_pax_aslr.c new file mode 100644 index 0000000..37d7da1 --- /dev/null +++ b/sys/kern/kern_pax_aslr.c @@ -0,0 +1,680 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "opt_compat.h" +#include "opt_pax.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include + +#include + +FEATURE(aslr, "Address Space Layout Randomization."); + +int pax_aslr_status =3D PAX_ASLR_OPTOUT; +int pax_aslr_debug =3D 0; + +#ifdef PAX_ASLR_MAX_SEC +int pax_aslr_mmap_len =3D PAX_ASLR_DELTA_MMAP_MAX_LEN; +int pax_aslr_stack_len =3D PAX_ASLR_DELTA_STACK_MAX_LEN; +int pax_aslr_exec_len =3D PAX_ASLR_DELTA_EXEC_MAX_LEN; +#else +int pax_aslr_mmap_len =3D PAX_ASLR_DELTA_MMAP_DEF_LEN; +int pax_aslr_stack_len =3D PAX_ASLR_DELTA_STACK_DEF_LEN; +int pax_aslr_exec_len =3D PAX_ASLR_DELTA_EXEC_DEF_LEN; +#endif /* PAX_ASLR_MAX_SEC */ + +#ifdef COMPAT_FREEBSD32 +int pax_aslr_compat_status =3D PAX_ASLR_OPTOUT; +#ifdef PAX_ASLR_MAX_SEC +int pax_aslr_compat_mmap_len =3D PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN; +int pax_aslr_compat_stack_len =3D PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN; +int pax_aslr_compat_exec_len =3D PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN; +#else +int pax_aslr_compat_mmap_len =3D PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN; +int pax_aslr_compat_stack_len =3D PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN; +int pax_aslr_compat_exec_len =3D PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN; +#endif /* PAX_ASLR_MAX_SEC */ +#endif /* COMPAT_FREEBSD32 */ + +TUNABLE_INT("security.pax.aslr.status", &pax_aslr_status); +TUNABLE_INT("security.pax.aslr.mmap_len", &pax_aslr_mmap_len); +TUNABLE_INT("security.pax.aslr.debug", &pax_aslr_debug); +TUNABLE_INT("security.pax.aslr.stack_len", &pax_aslr_stack_len); +TUNABLE_INT("security.pax.aslr.exec_len", &pax_aslr_exec_len); +#ifdef COMPAT_FREEBSD32 +TUNABLE_INT("security.pax.aslr.compat.status", &pax_aslr_compat_status); +TUNABLE_INT("security.pax.aslr.compat.mmap", &pax_aslr_compat_mmap_len); +TUNABLE_INT("security.pax.aslr.compat.stack", &pax_aslr_compat_stack_len); +TUNABLE_INT("security.pax.aslr.compat.stack", &pax_aslr_compat_exec_len); +#endif + + +#ifdef PAX_SYSCTLS +/* + * sysctls and tunables + */ +static int sysctl_pax_aslr_debug(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_status(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_mmap(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_stack(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_exec(SYSCTL_HANDLER_ARGS); + +SYSCTL_DECL(_security_pax); + +SYSCTL_NODE(_security_pax, OID_AUTO, aslr, CTLFLAG_RD, 0, + "Address Space Layout Randomization."); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, status, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_status, "I", + "Restrictions status. " + "0 - disabled, " + "1 - opt-in, " + "2 - opt-out, " + "3 - force enabled"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, debug, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_debug, "I", + "ASLR debug mode"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, mmap_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_mmap, "I", + "Number of bits randomized for mmap(2) calls. " + "32 bit: [8,16] 64 bit: [16,32]"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, stack_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_stack, "I", + "Number of bits randomized for the stack. " + "32 bit: [6,12] 64 bit: [12,21]"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, exec_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_exec, "I", + "Number of bits randomized for the PIE exec base. " + "32 bit: [6,12] 64 bit: [12,21]"); + +static int +sysctl_pax_aslr_status(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_status : pax_aslr_status; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + switch (val) { + case PAX_ASLR_DISABLED: + case PAX_ASLR_OPTIN: + case PAX_ASLR_OPTOUT: + case PAX_ASLR_FORCE_ENABLED: + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_status =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_status =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + break; + default: + return (EINVAL); + } + + return (0); +} + +static int +sysctl_pax_aslr_debug(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_debug : pax_aslr_debug; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_debug =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_debug =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_mmap(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_mmap_len : pax_aslr_mmap_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_DELTA_MMAP_MIN_LEN || + val > PAX_ASLR_DELTA_MMAP_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_mmap_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_mmap_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_stack(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_stack_len : pax_aslr_stack_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_DELTA_STACK_MIN_LEN || + val > PAX_ASLR_DELTA_STACK_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_stack_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_stack_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_exec(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_exec_len : pax_aslr_exec_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + if (val < PAX_ASLR_DELTA_EXEC_MIN_LEN || + val > PAX_ASLR_DELTA_EXEC_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_exec_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_exec_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +/* + * COMPAT_FREEBSD32 and linuxulator.. + */ +#ifdef COMPAT_FREEBSD32 +static int sysctl_pax_aslr_compat_status(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_mmap(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_stack(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_exec(SYSCTL_HANDLER_ARGS); + +SYSCTL_NODE(_security_pax_aslr, OID_AUTO, compat, CTLFLAG_RD, 0, + "Setting for COMPAT_FREEBSD32 and linuxulator."); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, status, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_status, "I", + "Restrictions status. " + "0 - disabled, " + "1 - enabled, " + "2 - global enabled, " + "3 - force global enabled"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, mmap_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_mmap, "I", + "Number of bits randomized for mmap(2) calls. " + "32 bit: [8,16]"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, stack_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_stack, "I", + "Number of bits randomized for the stack. " + "32 bit: [6,12]"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, exec_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_exec, "I", + "Number of bits randomized for the PIE exec base. " + "32 bit: [6,12]"); + +static int +sysctl_pax_aslr_compat_status(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ?pr->pr_pax_aslr_compat_status : pax_aslr_compat_s= tatus; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + switch (val) { + case PAX_ASLR_DISABLED: + case PAX_ASLR_OPTIN: + case PAX_ASLR_OPTOUT: + case PAX_ASLR_FORCE_ENABLED: + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_status =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_status =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + break; + default: + return (EINVAL); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_mmap(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_compat_mmap_len : pax_aslr_compa= t_mmap_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_mmap_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_mmap_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_stack(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_compat_stack_len : pax_aslr_comp= at_stack_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_stack_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_stack_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_exec(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if (pr !=3D NULL) + val =3D pr->pr_pax_aslr_compat_exec_len; + else + val =3D pax_aslr_compat_exec_len; + + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_exec_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_exec_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_SYSCTLS */ + + +/* + * ASLR functions + */ +bool +pax_aslr_active(struct thread *td, struct proc *proc) +{ + int status; + struct prison *pr; + uint32_t flags; + + if ((td =3D=3D NULL) && (proc =3D=3D NULL)) + return (true); + + pr =3D pax_get_prison(td, proc); + + flags =3D (td !=3D NULL) ? td->td_proc->p_pax : proc->p_pax; + if (((flags & 0xaaaaaaaa) & ((flags & 0x55555555) << 1)) !=3D 0) { + pax_log_aslr(pr, __func__, "inconsistent paxflags: %x\n", flags); + pax_ulog_aslr(pr, NULL, "inconsistent paxflags: %x\n", flags); + return (true); + } + + if (pr !=3D NULL) + status =3D pr->pr_pax_aslr_status; + else + status =3D pax_aslr_status; + + switch (status) { + case PAX_ASLR_DISABLED: + return (false); + case PAX_ASLR_FORCE_ENABLED: + return (true); + case PAX_ASLR_OPTIN: + if ((flags & PAX_NOTE_ASLR) =3D=3D 0) { + pax_log_aslr(pr, __func__, + "ASLR is opt-in, and executable does not have ASLR enabled\n"); + pax_ulog_aslr(pr, NULL, + "ASLR is opt-in, and executable does not have ASLR enabled\n"); + return (false); + } + break; + case PAX_ASLR_OPTOUT: + if ((flags & PAX_NOTE_NOASLR) !=3D 0) { + pax_log_aslr(pr, __func__, + "ASLR is opt-out, and executable explicitly disabled ASLR\n"); + pax_ulog_aslr(pr, NULL, + "ASLR is opt-out, and executable explicitly disabled ASLR\n"); + return (false); + } + break; + default: + return (true); + } + + return (true); +} + +void +_pax_aslr_init(struct vmspace *vm, struct prison *pr) +{ + if (vm =3D=3D NULL) + panic("[PaX ASLR] %s: vm =3D=3D NULL", __func__); + + vm->vm_aslr_delta_mmap =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_MMAP_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_mmap_len : + pax_aslr_mmap_len); + vm->vm_aslr_delta_stack =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_STACK_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_stack_len : + pax_aslr_stack_len); + vm->vm_aslr_delta_stack =3D ALIGN(vm->vm_aslr_delta_stack); + vm->vm_aslr_delta_exec =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_EXEC_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_exec_len : + pax_aslr_exec_len); + + if ((pr !=3D NULL) && pr->pr_pax_aslr_debug) { + pax_log_aslr(pr, __func__, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_log_aslr(pr, __func__, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_log_aslr(pr, __func__, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + } +} + +#ifdef COMPAT_FREEBSD32 +void +_pax_aslr_init32(struct vmspace *vm, struct prison *pr) +{ + if (vm =3D=3D NULL) + panic("[PaX ASLR] %s: vm =3D=3D NULL", __func__); + + vm->vm_aslr_delta_mmap =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_COMPAT_DELTA_MMAP_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_mmap_len : + pax_aslr_compat_mmap_len); + vm->vm_aslr_delta_stack =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_COMPAT_DELTA_STACK_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_stack_len : + pax_aslr_compat_stack_len); + vm->vm_aslr_delta_stack =3D ALIGN(vm->vm_aslr_delta_stack); + vm->vm_aslr_delta_exec =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_EXEC_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_exec_len : + pax_aslr_compat_exec_len); + + if ((pr !=3D NULL) && pr->pr_pax_aslr_debug) { + pax_log_aslr(pr, __func__, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_log_aslr(pr, __func__, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_log_aslr(pr, __func__, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + } +} +#endif + +void +pax_aslr_init(struct thread *td, struct image_params *imgp) +{ + struct prison *pr; + struct vmspace *vm; + + pr =3D pax_get_prison(td, NULL); + + if (imgp =3D=3D NULL) + panic("[PaX ASLR] %s: imgp =3D=3D NULL", __func__); + + if (!pax_aslr_active(td, NULL)) + return; + + vm =3D imgp->proc->p_vmspace; + + if (imgp->sysent->sv_pax_aslr_init !=3D NULL) + imgp->sysent->sv_pax_aslr_init(vm, pr); +} + +void +pax_aslr_mmap(struct thread *td, vm_offset_t *addr, vm_offset_t orig_addr,= int flags) +{ + struct prison *pr; + + if (!pax_aslr_active(td, NULL)) + return; + + pr =3D pax_get_prison(td, NULL); + + if (!(flags & MAP_FIXED) && ((orig_addr =3D=3D 0) || !(flags & MAP_ANON))= ) { + pax_log_aslr(pr, __func__, "applying to %p orig_addr=3D%p flags=3D%x\n", + (void *)*addr, (void *)orig_addr, flags); + + *addr +=3D td->td_proc->p_vmspace->vm_aslr_delta_mmap; + pax_log_aslr(pr, __func__, "result %p\n", (void *)*addr); + } else { + pax_log_aslr(pr, __func__, "not applying to %p orig_addr=3D%p flags=3D%x= \n", + (void *)*addr, (void *)orig_addr, flags); + } +} + +void +pax_aslr_stack(struct thread *td, uintptr_t *addr) +{ + struct prison *pr; + uintptr_t orig_addr; + + if (!pax_aslr_active(td, NULL)) + return; + + pr =3D pax_get_prison(td, NULL); + + orig_addr =3D *addr; + *addr -=3D td->td_proc->p_vmspace->vm_aslr_delta_stack; + pax_log_aslr(pr, __func__, "orig_addr=3D%p, new_addr=3D%p\n", + (void *)orig_addr, (void *)*addr); + pax_ulog_aslr(pr, NULL, "orig_addr=3D%p, new_addr=3D%p\n", + (void *)orig_addr, (void *)*addr); +} diff --git a/sys/kern/kern_pax_log.c b/sys/kern/kern_pax_log.c new file mode 100644 index 0000000..943ac81 --- /dev/null +++ b/sys/kern/kern_pax_log.c @@ -0,0 +1,188 @@ +/*- + * Copyright (c) 2014, by Oliver Pinter + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#define __PAX_LOG_TEMPLATE(SUBJECT, name) \ +void \ +pax_log_##name(struct prison *pr, const char *caller_name, const char* fmt= , ...)\ +{ \ + struct sbuf *sb; \ + va_list args; \ + \ + if ((pr !=3D NULL) && (pr->pr_pax_log_log =3D=3D 0)) \ + return; \ + \ + sb =3D sbuf_new_auto(); \ + if (sb =3D=3D NULL) \ + panic("%s: Could not allocate memory", __func__); \ + sbuf_printf(sb, "[PAX "#SUBJECT"] "); \ + if (caller_name !=3D NULL) \ + sbuf_printf(sb, "%s: ", caller_name); \ + va_start(args, fmt); \ + sbuf_vprintf(sb, fmt, args); \ + va_end(args); \ + if (sbuf_finish(sb) !=3D 0) \ + panic("%s: Could not generate message", __func__); \ + \ + printf("%s", sbuf_data(sb)); \ + sbuf_delete(sb); \ +} \ + \ +void \ +pax_ulog_##name(struct prison *pr, const char *caller_name, const char* fm= t, ...)\ +{ \ + struct sbuf *sb; \ + va_list args; \ + \ + if ((pr !=3D NULL) && (pr->pr_pax_log_ulog =3D=3D 0)) \ + return; \ + \ + sb =3D sbuf_new_auto(); \ + if (sb =3D=3D NULL) \ + panic("%s: Could not allocate memory", __func__); \ + sbuf_printf(sb, "[PAX "#SUBJECT"] "); \ + if (caller_name !=3D NULL) \ + sbuf_printf(sb, "%s: ", caller_name); \ + va_start(args, fmt); \ + sbuf_vprintf(sb, fmt, args); \ + va_end(args); \ + if (sbuf_finish(sb) !=3D 0) \ + panic("%s: Could not generate message", __func__); \ + \ + uprintf("%s", sbuf_data(sb)); \ + sbuf_delete(sb); \ +} + + +static int sysctl_pax_log_log(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_log_ulog(SYSCTL_HANDLER_ARGS); + +int pax_log_log =3D PAX_LOG_LOG; +int pax_log_ulog =3D PAX_LOG_ULOG; + +SYSCTL_DECL(_security_pax); + +SYSCTL_NODE(_security_pax, OID_AUTO, log, CTLFLAG_RD, 0, + "PAX related logging facility."); + +SYSCTL_PROC(_security_pax_log, OID_AUTO, log, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_log_log, "I", + "log to syslog " + "0 - disabled, " + "1 - enabled "); +TUNABLE_INT("security.pax.log.log", &pax_log_log); + +SYSCTL_PROC(_security_pax_log, OID_AUTO, ulog, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_log_ulog, "I", + "log to user terminal" + "0 - disabled, " + "1 - enabled "); +TUNABLE_INT("security.pax.log.ulog", &pax_log_ulog); + +static int +sysctl_pax_log_log(SYSCTL_HANDLER_ARGS) +{ + int err; + int val; + struct prison *pr=3DNULL; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_log_log : pax_log_log; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_log_log =3D val; + if (pr !=3D NULL) + pr->pr_pax_log_log =3D val; + + return (0); +} + +static int +sysctl_pax_log_ulog(SYSCTL_HANDLER_ARGS) +{ + int err; + int val; + struct prison *pr=3DNULL; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_log_ulog : pax_log_ulog; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_log_ulog =3D val; + if (pr !=3D NULL) + pr->pr_pax_log_ulog =3D val; + + return (0); +} + + +__PAX_LOG_TEMPLATE(ASLR, aslr) diff --git a/sys/mips/mips/elf_machdep.c b/sys/mips/mips/elf_machdep.c index d374713..f95ba35 100644 --- a/sys/mips/mips/elf_machdep.c +++ b/sys/mips/mips/elf_machdep.c @@ -28,6 +28,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -49,6 +51,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + #ifdef __mips_n64 struct sysentvec elf64_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, @@ -83,6 +89,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf64_Brandinfo freebsd_brand_info =3D { @@ -139,6 +150,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf32_Brandinfo freebsd_brand_info =3D { diff --git a/sys/mips/mips/freebsd32_machdep.c b/sys/mips/mips/freebsd32_ma= chdep.c index dfdf70f..103ad84 100644 --- a/sys/mips/mips/freebsd32_machdep.c +++ b/sys/mips/mips/freebsd32_machdep.c @@ -31,6 +31,7 @@ */ =20 #include "opt_compat.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -66,6 +67,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + static void freebsd32_exec_setregs(struct thread *, struct image_params *,= u_long); static int get_mcontext32(struct thread *, mcontext32_t *, int); static int set_mcontext32(struct thread *, const mcontext32_t *); @@ -106,6 +111,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D freebsd32_syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/powerpc/powerpc/elf32_machdep.c b/sys/powerpc/powerpc/elf3= 2_machdep.c index dbe58df..229fe97 100644 --- a/sys/powerpc/powerpc/elf32_machdep.c +++ b/sys/powerpc/powerpc/elf32_machdep.c @@ -25,6 +25,8 @@ * $FreeBSD$ */ =20 +#include "opt_pax.h" + #include #include #include @@ -52,6 +54,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + #ifdef __powerpc64__ #include #include @@ -107,6 +113,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_shared_page_base =3D FREEBSD32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/powerpc/powerpc/elf64_machdep.c b/sys/powerpc/powerpc/elf6= 4_machdep.c index 0c41a8d..095f37b0 100644 --- a/sys/powerpc/powerpc/elf64_machdep.c +++ b/sys/powerpc/powerpc/elf64_machdep.c @@ -25,6 +25,8 @@ * $FreeBSD$ */ =20 +#include "opt_pax.h" + #include #include #include @@ -48,6 +50,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf64_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -83,6 +89,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf64_sysvec, &elf64_freebsd_sysvec); =20 diff --git a/sys/security/mac_bsdextended/mac_bsdextended.c b/sys/security/= mac_bsdextended/mac_bsdextended.c index 377fd25..f6b6173 100644 --- a/sys/security/mac_bsdextended/mac_bsdextended.c +++ b/sys/security/mac_bsdextended/mac_bsdextended.c @@ -47,6 +47,8 @@ * firewall-like rules regarding users and file system objects. */ =20 +#include "opt_pax.h" + #include #include #include @@ -56,14 +58,20 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include =20 +#ifdef PAX_ASLR +#include +#endif + #include #include #include @@ -116,7 +124,6 @@ SYSCTL_INT(_security_mac_bsdextended, OID_AUTO, firstma= tch_enabled, static int ugidfw_rule_valid(struct mac_bsdextended_rule *rule) { - if ((rule->mbr_subject.mbs_flags | MBS_ALL_FLAGS) !=3D MBS_ALL_FLAGS) return (EINVAL); if ((rule->mbr_subject.mbs_neg | MBS_ALL_FLAGS) !=3D MBS_ALL_FLAGS) @@ -128,8 +135,13 @@ ugidfw_rule_valid(struct mac_bsdextended_rule *rule) if ((rule->mbr_object.mbo_neg | MBO_TYPE_DEFINED) && (rule->mbr_object.mbo_type | MBO_ALL_TYPE) !=3D MBO_ALL_TYPE) return (EINVAL); +#ifdef PAX_ASLR + if ((rule->mbr_pax | MBI_ALLPAX) !=3D MBI_ALLPAX) + return (EINVAL); +#endif if ((rule->mbr_mode | MBI_ALLPERM) !=3D MBI_ALLPERM) return (EINVAL); + return (0); } =20 @@ -226,7 +238,7 @@ ugidfw_destroy(struct mac_policy_conf *mpc) =20 static int ugidfw_rulecheck(struct mac_bsdextended_rule *rule, - struct ucred *cred, struct vnode *vp, struct vattr *vap, int acc_mode) + struct ucred *cred, struct vnode *vp, struct vattr *vap, int acc_mode,= struct image_params *imgp) { int mac_granted, match, priv_granted; int i; @@ -304,6 +316,10 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, match =3D (bcmp(&(vp->v_mount->mnt_stat.f_fsid), &(rule->mbr_object.mbo_fsid), sizeof(rule->mbr_object.mbo_fsid)) =3D=3D 0); +#if defined(PAX_ASLR) + if (match && rule->mbr_object.mbo_inode) + match =3D (vap->va_fileid =3D=3D rule->mbr_object.mbo_inode); +#endif if (rule->mbr_object.mbo_neg & MBO_FSID_DEFINED) match =3D !match; if (!match) @@ -412,6 +428,11 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, return (EACCES); } =20 +#ifdef PAX_ASLR + if (imgp !=3D NULL) + pax_elf(imgp, rule->mbr_pax); +#endif + /* * If the rule matched, permits access, and first match is enabled, * return success. @@ -424,7 +445,7 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, =20 int ugidfw_check(struct ucred *cred, struct vnode *vp, struct vattr *vap, - int acc_mode) + int acc_mode, struct image_params *imgp) { int error, i; =20 @@ -440,7 +461,7 @@ ugidfw_check(struct ucred *cred, struct vnode *vp, stru= ct vattr *vap, if (rules[i] =3D=3D NULL) continue; error =3D ugidfw_rulecheck(rules[i], cred, - vp, vap, acc_mode); + vp, vap, acc_mode, imgp); if (error =3D=3D EJUSTRETURN) break; if (error) { @@ -453,7 +474,7 @@ ugidfw_check(struct ucred *cred, struct vnode *vp, stru= ct vattr *vap, } =20 int -ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode) +ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode, struct= image_params *imgp) { int error; struct vattr vap; @@ -463,7 +484,7 @@ ugidfw_check_vp(struct ucred *cred, struct vnode *vp, i= nt acc_mode) error =3D VOP_GETATTR(vp, &vap, cred); if (error) return (error); - return (ugidfw_check(cred, vp, &vap, acc_mode)); + return (ugidfw_check(cred, vp, &vap, acc_mode, imgp)); } =20 int diff --git a/sys/security/mac_bsdextended/mac_bsdextended.h b/sys/security/= mac_bsdextended/mac_bsdextended.h index c09abc0..c3cbf28 100644 --- a/sys/security/mac_bsdextended/mac_bsdextended.h +++ b/sys/security/mac_bsdextended/mac_bsdextended.h @@ -51,6 +51,9 @@ #define MBI_ADMIN 010000 #define MBI_STAT 020000 #define MBI_APPEND 040000 +#define MBI_FORCE_ASLR_ENABLED 0x01 +#define MBI_FORCE_ASLR_DISABLED 0x02 +#define MBI_ALLPAX (MBI_FORCE_ASLR_ENABLED | MBI_FORCE_ASLR_DISABLED) #define MBI_ALLPERM (MBI_EXEC | MBI_WRITE | MBI_READ | MBI_ADMIN | \ MBI_STAT | MBI_APPEND) =20 @@ -78,6 +81,7 @@ struct mac_bsdextended_subject { #define MBO_UID_SUBJECT 0x00000020 /* uid must match subject */ #define MBO_GID_SUBJECT 0x00000040 /* gid must match subject */ #define MBO_TYPE_DEFINED 0x00000080 /* object type should be matched */ +#define MBO_PAXPATH_DEFINED 0x00000100 /* TODO: paxpath should be matched = */ =20 #define MBO_ALL_FLAGS (MBO_UID_DEFINED | MBO_GID_DEFINED | MBO_FSID_DEFINE= D | \ MBO_SUID | MBO_SGID | MBO_UID_SUBJECT | MBO_GID_SUBJECT | \ @@ -103,12 +107,15 @@ struct mac_bsdextended_object { gid_t mbo_gid_max; struct fsid mbo_fsid; int mbo_type; + ino_t mbo_inode; + char mbo_paxpath[MAXPATHLEN]; }; =20 struct mac_bsdextended_rule { struct mac_bsdextended_subject mbr_subject; struct mac_bsdextended_object mbr_object; mode_t mbr_mode; /* maximum access */ + uint32_t mbr_pax; }; =20 #endif /* _SYS_SECURITY_MAC_BSDEXTENDED_H */ diff --git a/sys/security/mac_bsdextended/ugidfw_internal.h b/sys/security/= mac_bsdextended/ugidfw_internal.h index 5597fd1..18c74dc 100644 --- a/sys/security/mac_bsdextended/ugidfw_internal.h +++ b/sys/security/mac_bsdextended/ugidfw_internal.h @@ -36,8 +36,9 @@ */ int ugidfw_accmode2mbi(accmode_t accmode); int ugidfw_check(struct ucred *cred, struct vnode *vp, struct vattr *vap, - int acc_mode); -int ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode); + int acc_mode, struct image_params *imgp); +int ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode, + struct image_params *imgp); =20 /* * System access control checks. diff --git a/sys/security/mac_bsdextended/ugidfw_system.c b/sys/security/ma= c_bsdextended/ugidfw_system.c index 49e4f1d..2829a00 100644 --- a/sys/security/mac_bsdextended/ugidfw_system.c +++ b/sys/security/mac_bsdextended/ugidfw_system.c @@ -66,7 +66,7 @@ ugidfw_system_check_acct(struct ucred *cred, struct vnode= *vp, { =20 if (vp !=3D NULL) - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); else return (0); } @@ -77,7 +77,7 @@ ugidfw_system_check_auditctl(struct ucred *cred, struct v= node *vp, { =20 if (vp !=3D NULL) - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); else return (0); } @@ -87,5 +87,5 @@ ugidfw_system_check_swapon(struct ucred *cred, struct vno= de *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } diff --git a/sys/security/mac_bsdextended/ugidfw_vnode.c b/sys/security/mac= _bsdextended/ugidfw_vnode.c index 8ec2d48..2065e6e 100644 --- a/sys/security/mac_bsdextended/ugidfw_vnode.c +++ b/sys/security/mac_bsdextended/ugidfw_vnode.c @@ -65,7 +65,7 @@ ugidfw_vnode_check_access(struct ucred *cred, struct vnod= e *vp, struct label *vplabel, accmode_t accmode) { =20 - return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode))); + return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode), NULL)); } =20 int @@ -73,7 +73,7 @@ ugidfw_vnode_check_chdir(struct ucred *cred, struct vnode= *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -81,7 +81,7 @@ ugidfw_vnode_check_chroot(struct ucred *cred, struct vnod= e *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -89,7 +89,7 @@ ugidfw_check_create_vnode(struct ucred *cred, struct vnod= e *dvp, struct label *dvplabel, struct componentname *cnp, struct vattr *vap) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_WRITE)); + return (ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL)); } =20 int @@ -97,7 +97,7 @@ ugidfw_vnode_check_deleteacl(struct ucred *cred, struct v= node *vp, struct label *vplabel, acl_type_t type) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -105,7 +105,7 @@ ugidfw_vnode_check_deleteextattr(struct ucred *cred, st= ruct vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -114,7 +114,7 @@ ugidfw_vnode_check_exec(struct ucred *cred, struct vnod= e *vp, struct label *execlabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ|MBI_EXEC)); + return (ugidfw_check_vp(cred, vp, MBI_READ|MBI_EXEC, imgp)); } =20 int @@ -122,7 +122,7 @@ ugidfw_vnode_check_getacl(struct ucred *cred, struct vn= ode *vp, struct label *vplabel, acl_type_t type) { =20 - return (ugidfw_check_vp(cred, vp, MBI_STAT)); + return (ugidfw_check_vp(cred, vp, MBI_STAT, NULL)); } =20 int @@ -130,7 +130,7 @@ ugidfw_vnode_check_getextattr(struct ucred *cred, struc= t vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -140,10 +140,10 @@ ugidfw_vnode_check_link(struct ucred *cred, struct vn= ode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - error =3D ugidfw_check_vp(cred, vp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, vp, MBI_WRITE, NULL); if (error) return (error); return (0); @@ -154,7 +154,7 @@ ugidfw_vnode_check_listextattr(struct ucred *cred, stru= ct vnode *vp, struct label *vplabel, int attrnamespace) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -162,7 +162,7 @@ ugidfw_vnode_check_lookup(struct ucred *cred, struct vn= ode *dvp, struct label *dvplabel, struct componentname *cnp) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -170,7 +170,7 @@ ugidfw_vnode_check_open(struct ucred *cred, struct vnod= e *vp, struct label *vplabel, accmode_t accmode) { =20 - return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode))); + return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode), NULL)); } =20 int @@ -178,7 +178,7 @@ ugidfw_vnode_check_readdir(struct ucred *cred, struct v= node *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_READ)); + return (ugidfw_check_vp(cred, dvp, MBI_READ, NULL)); } =20 int @@ -186,7 +186,7 @@ ugidfw_vnode_check_readdlink(struct ucred *cred, struct= vnode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -196,10 +196,10 @@ ugidfw_vnode_check_rename_from(struct ucred *cred, st= ruct vnode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -209,11 +209,11 @@ ugidfw_vnode_check_rename_to(struct ucred *cred, stru= ct vnode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); if (vp !=3D NULL) - error =3D ugidfw_check_vp(cred, vp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, vp, MBI_WRITE, NULL); return (error); } =20 @@ -222,7 +222,7 @@ ugidfw_vnode_check_revoke(struct ucred *cred, struct vn= ode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -230,7 +230,7 @@ ugidfw_check_setacl_vnode(struct ucred *cred, struct vn= ode *vp, struct label *vplabel, acl_type_t type, struct acl *acl) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -238,7 +238,7 @@ ugidfw_vnode_check_setextattr(struct ucred *cred, struc= t vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -246,7 +246,7 @@ ugidfw_vnode_check_setflags(struct ucred *cred, struct = vnode *vp, struct label *vplabel, u_long flags) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -254,7 +254,7 @@ ugidfw_vnode_check_setmode(struct ucred *cred, struct v= node *vp, struct label *vplabel, mode_t mode) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -262,7 +262,7 @@ ugidfw_vnode_check_setowner(struct ucred *cred, struct = vnode *vp, struct label *vplabel, uid_t uid, gid_t gid) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -270,7 +270,7 @@ ugidfw_vnode_check_setutimes(struct ucred *cred, struct= vnode *vp, struct label *vplabel, struct timespec atime, struct timespec utime) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -278,7 +278,7 @@ ugidfw_vnode_check_stat(struct ucred *active_cred, struct ucred *file_cred, struct vnode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(active_cred, vp, MBI_STAT)); + return (ugidfw_check_vp(active_cred, vp, MBI_STAT, NULL)); } =20 int @@ -288,8 +288,8 @@ ugidfw_vnode_check_unlink(struct ucred *cred, struct vn= ode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } diff --git a/sys/sparc64/sparc64/elf_machdep.c b/sys/sparc64/sparc64/elf_ma= chdep.c index 4d55717..e0eba33 100644 --- a/sys/sparc64/sparc64/elf_machdep.c +++ b/sys/sparc64/sparc64/elf_machdep.c @@ -34,6 +34,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -52,6 +54,10 @@ __FBSDID("$FreeBSD$"); =20 #include =20 +#ifdef PAX_ASLR +#include +#endif + #include "linker_if.h" =20 static struct sysentvec elf64_freebsd_sysvec =3D { @@ -87,6 +93,11 @@ static struct sysentvec elf64_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf64_Brandinfo freebsd_brand_info =3D { diff --git a/sys/sys/imgact.h b/sys/sys/imgact.h index 17cfcc2..15c2c4f 100644 --- a/sys/sys/imgact.h +++ b/sys/sys/imgact.h @@ -78,6 +78,7 @@ struct image_params { unsigned long pagesizes; int pagesizeslen; vm_prot_t stack_prot; + int pax_flags; }; =20 #ifdef _KERNEL diff --git a/sys/sys/jail.h b/sys/sys/jail.h index 59d791c..699b21c 100644 --- a/sys/sys/jail.h +++ b/sys/sys/jail.h @@ -184,6 +184,19 @@ struct prison { char pr_hostname[MAXHOSTNAMELEN]; /* (p) jail hostname */ char pr_domainname[MAXHOSTNAMELEN]; /* (p) jail domainname */ char pr_hostuuid[HOSTUUIDLEN]; /* (p) jail hostuuid */ + /* Lock only needed for pax_* if pr_pax_set =3D=3D 0 */ + int pr_pax_set; /* (p) PaX settings initialized */ + int pr_pax_aslr_status; /* (p) PaX ASLR enabled */ + int pr_pax_aslr_debug; /* (p) PaX ASLR debug */ + int pr_pax_aslr_mmap_len; /* (p) Number of bits randomized with mmap */ + int pr_pax_aslr_stack_len; /* (p) Number of bits randomized with stack= */ + int pr_pax_aslr_exec_len; /* (p) Number of bits randomized with the ex= ecbase */ + int pr_pax_aslr_compat_status; /* (p) PaX ASLR enabled (compat32) */ + int pr_pax_aslr_compat_mmap_len; /* (p) Number of bits randomized with = mmap (compat32) */ + int pr_pax_aslr_compat_stack_len; /* (p) Number of bits randomized with= stack (compat32) */ + int pr_pax_aslr_compat_exec_len; /* (p) Number of bits randomized with = the execbase (compat32) */ + int pr_pax_log_log; /* (p) XXX */ + int pr_pax_log_ulog; /* (p) XXX */ }; =20 struct prison_racct { diff --git a/sys/sys/kernel.h b/sys/sys/kernel.h index 3c5258a..aedb52e 100644 --- a/sys/sys/kernel.h +++ b/sys/sys/kernel.h @@ -102,6 +102,7 @@ enum sysinit_sub_id { SI_SUB_WITNESS =3D 0x1A80000, /* witness initialization */ SI_SUB_MTX_POOL_DYNAMIC =3D 0x1AC0000, /* dynamic mutex pool */ SI_SUB_LOCK =3D 0x1B00000, /* various locks */ + SI_SUB_PAX =3D 0x1B50000, /* pax setup */ SI_SUB_EVENTHANDLER =3D 0x1C00000, /* eventhandler init */ SI_SUB_VNET_PRELINK =3D 0x1E00000, /* vnet init before modules */ SI_SUB_KLD =3D 0x2000000, /* KLD and module setup */ diff --git a/sys/sys/pax.h b/sys/sys/pax.h new file mode 100644 index 0000000..a0f2bf6 --- /dev/null +++ b/sys/sys/pax.h @@ -0,0 +1,226 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef __SYS_PAX_H +#define __SYS_PAX_H + +struct image_params; +struct prison; +struct thread; +struct vnode; +struct vmspace; +struct vm_offset_t; + +/* + * used in sysctl handler + */ +#define PAX_ASLR_DISABLED 0 +#define PAX_ASLR_OPTIN 1 +#define PAX_ASLR_OPTOUT 2 +#define PAX_ASLR_FORCE_ENABLED 3 + +#ifndef PAX_ASLR_DELTA +#define PAX_ASLR_DELTA(delta, lsb, len) \ + (((delta) & ((1UL << (len)) - 1)) << (lsb)) +#endif /* PAX_ASLR_DELTA */ + +#ifdef PAX_ASLR +/* + * generic ASLR values + * + * MMAP | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 8 bit | 16 bit | + * +-------+--------+--------+ + * | DEF | 8 bit | 21 bit | + * +-------+--------+--------+ + * | MAX | 16 bit | 32 bit | + * +-------+--------+--------+ + * + * STACK | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 6 bit | 12 bit | + * +-------+--------+--------+ + * | DEF | 6 bit | 16 bit | + * +-------+--------+--------+ + * | MAX | 10 bit | 21 bit | + * +-------+--------+--------+ + * + * EXEC | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 6 bit | 12 bit | + * +-------+--------+--------+ + * | DEF | 6 bit | 21 bit | + * +-------+--------+--------+ + * | MAX | 10 bit | 21 bit | + * +-------+--------+--------+ + * + */ +#ifndef PAX_ASLR_DELTA_MMAP_LSB +#define PAX_ASLR_DELTA_MMAP_LSB PAGE_SHIFT +#endif /* PAX_ASLR_DELTA_MMAP_LSB */ + +#ifndef PAX_ASLR_DELTA_MMAP_MIN_LEN +#define PAX_ASLR_DELTA_MMAP_MIN_LEN ((sizeof(void *) * NBBY) / 4) +#endif /* PAX_ASLR_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_MMAP_MAX_LEN +#define PAX_ASLR_DELTA_MMAP_MAX_LEN ((sizeof(void *) * NBBY) / 2) +#endif /* PAX_ASLR_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_STACK_LSB +#define PAX_ASLR_DELTA_STACK_LSB 3 +#endif /* PAX_ASLR_DELTA_STACK_LSB */ + +#ifndef PAX_ASLR_DELTA_STACK_MIN_LEN +#define PAX_ASLR_DELTA_STACK_MIN_LEN ((sizeof(void *) * NBBY) / 5) +#endif /* PAX_ASLR_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_STACK_MAX_LEN +#define PAX_ASLR_DELTA_STACK_MAX_LEN ((sizeof(void *) * NBBY) / 3) +#endif /* PAX_ASLR_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_EXEC_LSB +#define PAX_ASLR_DELTA_EXEC_LSB PAGE_SHIFT +#endif /* PAX_ASLR_DELTA_EXEC_LSB */ + +#ifndef PAX_ASLR_DELTA_EXEC_MIN_LEN +#define PAX_ASLR_DELTA_EXEC_MIN_LEN ((sizeof(void *) * NBBY) / 5) +#endif /* PAX_ASLR_DELTA_EXEC_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_EXEC_MAX_LEN +#define PAX_ASLR_DELTA_EXEC_MAX_LEN ((sizeof(void *) * NBBY) / 3) +#endif /* PAX_ASLR_DELTA_EXEC_MAX_LEN */ + +/* + * ASLR default values for native host + */ +#ifdef __amd64__ +#ifndef PAX_ASLR_DELTA_MMAP_DEF_LEN +#define PAX_ASLR_DELTA_MMAP_DEF_LEN 21 +#endif /* PAX_ASLR_DELTA_MMAP_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_STACK_DEF_LEN +#define PAX_ASLR_DELTA_STACK_DEF_LEN 16 +#endif /* PAX_ASLR_DELTA_STACK_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_EXEC_DEF_LEN +#define PAX_ASLR_DELTA_EXEC_DEF_LEN 21 +#endif /* PAX_ASLR_DELTA_EXEC_DEF_LEN */ +#else +#ifndef PAX_ASLR_DELTA_MMAP_DEF_LEN +#define PAX_ASLR_DELTA_MMAP_DEF_LEN PAX_ASLR_DELTA_MMAP_MIN_LEN +#endif /* PAX_ASLR_DELTA_MMAP_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_STACK_DEF_LEN +#define PAX_ASLR_DELTA_STACK_DEF_LEN PAX_ASLR_DELTA_STACK_MIN_LEN +#endif /* PAX_ASLR_DELTA_STACK_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_EXEC_DEF_LEN +#define PAX_ASLR_DELTA_EXEC_DEF_LEN PAX_ASLR_DELTA_EXEC_MIN_LEN +#endif /* PAX_ASLR_DELTA_EXEC_DEF_LEN */ +#endif /* __amd64__ */ + +/* + * ASLR values for COMPAT_FREEBSD32 and COMPAT_LINUX + */ +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_LSB +#define PAX_ASLR_COMPAT_DELTA_MMAP_LSB PAGE_SHIFT +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_LSB */ + +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN ((sizeof(int) * NBBY) / 4) +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN ((sizeof(int) * NBBY) / 2) +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_LSB +#define PAX_ASLR_COMPAT_DELTA_STACK_LSB 3 +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_LSB */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN ((sizeof(int) * NBBY) / 5) +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN ((sizeof(int) * NBBY) / 3) +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN ((sizeof(int) * NBBY) / 5) +#endif /* PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN ((sizeof(int) * NBBY) / 3) +#endif /* PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN */ + +extern int pax_aslr_status; +extern int pax_aslr_debug; + +extern int pax_aslr_mmap_len; +extern int pax_aslr_stack_len; +extern int pax_aslr_exec_len; +#ifdef COMPAT_FREEBSD32 +extern int pax_aslr_compat_status; +extern int pax_aslr_compat_mmap_len; +extern int pax_aslr_compat_stack_len; +extern int pax_aslr_compat_exec_len; +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + +extern int pax_log_log; +extern int pax_log_ulog; + +#define ELF_NOTE_TYPE_PAX_TAG 3 +#define PAX_NOTE_MPROTECT 0x01 +#define PAX_NOTE_NOMPROTECT 0x02 +#define PAX_NOTE_GUARD 0x04 +#define PAX_NOTE_NOGUARD 0x08 +#define PAX_NOTE_ASLR 0x10 +#define PAX_NOTE_NOASLR 0x20 + +#define PAX_LOG_LOG 0 +#define PAX_LOG_ULOG 0 + +void pax_init(void); +void pax_init_prison(struct prison *pr); +bool pax_aslr_active(struct thread *td, struct proc *proc); +void _pax_aslr_init(struct vmspace *vm, struct prison *pr); +void _pax_aslr_init32(struct vmspace *vm, struct prison *pr); +void pax_aslr_init(struct thread *td, struct image_params *imgp); +void pax_aslr_mmap(struct thread *td, vm_offset_t *addr, + vm_offset_t orig_addr, int flags); +void pax_aslr_stack(struct thread *td, uintptr_t *addr); +struct prison *pax_get_prison(struct thread *td, struct proc *proc); +void pax_elf(struct image_params *, uint32_t); + +void pax_log_aslr(struct prison *pr, const char *func, const char *fmt, ..= =2E); +void pax_ulog_aslr(struct prison *pr, const char *func, const char *fmt, .= =2E.); + +#endif /* __SYS_PAX_H */ diff --git a/sys/sys/proc.h b/sys/sys/proc.h index fbd064c..558d7bf 100644 --- a/sys/sys/proc.h +++ b/sys/sys/proc.h @@ -539,6 +539,7 @@ struct proc { u_int p_stops; /* (c) Stop event bitmask. */ u_int p_stype; /* (c) Stop event type. */ char p_step; /* (c) Process is stopped. */ + u_int p_pax; /* (b) PaX is enabled to this process */ u_char p_pfsflags; /* (c) Procfs flags. */ struct nlminfo *p_nlminfo; /* (?) Only used by/for lockd. */ struct kaioinfo *p_aioinfo; /* (y) ASYNC I/O info. */ diff --git a/sys/sys/sysent.h b/sys/sys/sysent.h index 0f1c256..83585b8 100644 --- a/sys/sys/sysent.h +++ b/sys/sys/sysent.h @@ -77,9 +77,11 @@ struct sysent { /* system call table */ #define SY_THR_INCR 0x8 =20 struct image_params; +struct prison; struct __sigset; struct syscall_args; struct trapframe; +struct vmspace; struct vnode; =20 struct sysentvec { @@ -130,6 +132,7 @@ struct sysentvec { uint32_t sv_timekeep_gen; void *sv_shared_page_obj; void (*sv_schedtail)(struct thread *); + void (*sv_pax_aslr_init)(struct vmspace *vm, struct prison *pr); }; =20 #define SV_ILP32 0x000100 diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index b5108e2..e39bbd0 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -65,6 +65,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -292,6 +294,12 @@ vmspace_alloc(vm_offset_t min, vm_offset_t max, pmap_p= init_t pinit) vm->vm_taddr =3D 0; vm->vm_daddr =3D 0; vm->vm_maxsaddr =3D 0; +#ifdef PAX_ASLR + vm->vm_aslr_delta_mmap =3D 0; + vm->vm_aslr_delta_stack =3D 0; + vm->vm_aslr_delta_exec =3D 0; +#endif + return (vm); } =20 diff --git a/sys/vm/vm_map.h b/sys/vm/vm_map.h index 8cced05..e8e9ffe 100644 --- a/sys/vm/vm_map.h +++ b/sys/vm/vm_map.h @@ -241,6 +241,9 @@ struct vmspace { caddr_t vm_taddr; /* (c) user virtual address of text */ caddr_t vm_daddr; /* (c) user virtual address of data */ caddr_t vm_maxsaddr; /* user VA at max stack growth */ + vm_size_t vm_aslr_delta_mmap; /* mmap() random delta for ASLR */ + vm_size_t vm_aslr_delta_stack; /* stack random delta for ASLR */ + vm_size_t vm_aslr_delta_exec; /* exec base random delta for ASLR */ volatile int vm_refcnt; /* number of references */ /* * Keep the PMAP last, so that CPU-specific variations of that diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c index 1ae7189..c2b6a4d 100644 --- a/sys/vm/vm_mmap.c +++ b/sys/vm/vm_mmap.c @@ -45,6 +45,7 @@ __FBSDID("$FreeBSD$"); =20 #include "opt_compat.h" #include "opt_hwpmc_hooks.h" +#include "opt_pax.h" =20 #include #include @@ -91,6 +92,10 @@ __FBSDID("$FreeBSD$"); #include #endif =20 +#ifdef PAX_ASLR +#include +#endif + int old_mlock =3D 0; SYSCTL_INT(_vm, OID_AUTO, old_mlock, CTLFLAG_RWTUN, &old_mlock, 0, "Do not apply RLIMIT_MEMLOCK on mlockall"); @@ -202,6 +207,9 @@ sys_mmap(td, uap) struct file *fp; struct vnode *vp; vm_offset_t addr; +#ifdef PAX_ASLR + vm_offset_t orig_addr; +#endif vm_size_t size, pageoff; vm_prot_t cap_maxprot, prot, maxprot; void *handle; @@ -212,6 +220,9 @@ sys_mmap(td, uap) cap_rights_t rights; =20 addr =3D (vm_offset_t) uap->addr; +#ifdef PAX_ASLR + orig_addr =3D addr; +#endif size =3D uap->len; prot =3D uap->prot & VM_PROT_ALL; flags =3D uap->flags; @@ -294,6 +305,12 @@ sys_mmap(td, uap) * do not bother moving the mapping past the heap (since * the heap is usually above 2GB). */ +#ifdef PAX_ASLR + /* Ugly hack for adding ASLR to 32bit mappings */ + pax_aslr_mmap(td, &addr, orig_addr, flags); + if (addr !=3D orig_addr) + addr =3D trunc_page(addr&0x0fffffff); +#endif if (addr + size > MAP_32BIT_MAX_ADDR) addr =3D 0; #endif @@ -307,6 +324,9 @@ sys_mmap(td, uap) * location. */ PROC_LOCK(td->td_proc); +#ifdef PAX_ASLR + pax_aslr_mmap(td, &addr, orig_addr, flags); +#endif if (addr =3D=3D 0 || (addr >=3D round_page((vm_offset_t)vms->vm_taddr) && addr < round_page((vm_offset_t)vms->vm_daddr + diff --git a/tools/build/options/WITHOUT_PIE b/tools/build/options/WITHOUT_= PIE new file mode 100644 index 0000000..82019ce --- /dev/null +++ b/tools/build/options/WITHOUT_PIE @@ -0,0 +1 @@ +Enable building of Position-Independent Executables (PIEs). diff --git a/usr.sbin/ugidfw/ugidfw.c b/usr.sbin/ugidfw/ugidfw.c index 977922a..515df16 100644 --- a/usr.sbin/ugidfw/ugidfw.c +++ b/usr.sbin/ugidfw/ugidfw.c @@ -46,6 +46,8 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#define UGIDFW_BUFSIZ (BUFSIZ*2) + void add_rule(int argc, char *argv[]); void list_rules(void); void remove_rule(int argc, char *argv[]); @@ -71,22 +73,22 @@ usage(void) void add_rule(int argc, char *argv[]) { - char errstr[BUFSIZ], charstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ], charstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; int error, rulenum; =20 - error =3D bsde_parse_rule(argc, argv, &rule, BUFSIZ, errstr); + error =3D bsde_parse_rule(argc, argv, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; } =20 - error =3D bsde_add_rule(&rulenum, &rule, BUFSIZ, errstr); + error =3D bsde_add_rule(&rulenum, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; } - if (bsde_rule_to_string(&rule, charstr, BUFSIZ) =3D=3D -1) + if (bsde_rule_to_string(&rule, charstr, UGIDFW_BUFSIZ) =3D=3D -1) warnx("Added rule, but unable to print string."); else printf("%d %s\n", rulenum, charstr); @@ -95,25 +97,25 @@ add_rule(int argc, char *argv[]) void list_rules(void) { - char errstr[BUFSIZ], charstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ], charstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; int error, i, rule_count, rule_slots; =20 - rule_slots =3D bsde_get_rule_slots(BUFSIZ, errstr); + rule_slots =3D bsde_get_rule_slots(UGIDFW_BUFSIZ, errstr); if (rule_slots =3D=3D -1) { warnx("unable to get rule slots; mac_bsdextended.ko " "may not be loaded"); errx(1, "bsde_get_rule_slots: %s", errstr); } =20 - rule_count =3D bsde_get_rule_count(BUFSIZ, errstr); + rule_count =3D bsde_get_rule_count(UGIDFW_BUFSIZ, errstr); if (rule_count =3D=3D -1) errx(1, "bsde_get_rule_count: %s", errstr); =20 printf("%d slots, %d rules\n", rule_slots, rule_count); =20 for (i =3D 0; i < rule_slots; i++) { - error =3D bsde_get_rule(i, &rule, BUFSIZ, errstr); + error =3D bsde_get_rule(i, &rule, UGIDFW_BUFSIZ, errstr); switch (error) { case -2: continue; @@ -124,7 +126,7 @@ list_rules(void) break; } =20 - if (bsde_rule_to_string(&rule, charstr, BUFSIZ) =3D=3D -1) + if (bsde_rule_to_string(&rule, charstr, UGIDFW_BUFSIZ) =3D=3D -1) warnx("unable to translate rule %d to string", i); else printf("%d %s\n", i, charstr); @@ -134,7 +136,7 @@ list_rules(void) void set_rule(int argc, char *argv[]) { - char errstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; long value; int error, rulenum; @@ -152,13 +154,13 @@ set_rule(int argc, char *argv[]) =20 rulenum =3D value; =20 - error =3D bsde_parse_rule(argc - 1, argv + 1, &rule, BUFSIZ, errstr); + error =3D bsde_parse_rule(argc - 1, argv + 1, &rule, UGIDFW_BUFSIZ, errst= r); if (error) { warnx("%s", errstr); return; } =20 - error =3D bsde_set_rule(rulenum, &rule, BUFSIZ, errstr); + error =3D bsde_set_rule(rulenum, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; @@ -168,7 +170,7 @@ set_rule(int argc, char *argv[]) void remove_rule(int argc, char *argv[]) { - char errstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ]; long value; int error, rulenum; char *endp; @@ -185,7 +187,7 @@ remove_rule(int argc, char *argv[]) =20 rulenum =3D value; =20 - error =3D bsde_delete_rule(rulenum, BUFSIZ, errstr); + error =3D bsde_delete_rule(rulenum, UGIDFW_BUFSIZ, errstr); if (error) warnx("%s", errstr); } --vkEkAx9hr54EJ73W-- --++alDQ2ROsODg1x+ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTwHKfAAoJEGqEZY9SRW7uHrIP/iA/Xsl2+S+/rFp9inDbijaZ g1Ow9SYqpxwLapWpgJzhnZdVPONsd9U4Y0Jm4088bsTB1LMAlodNmTq6a8jvmsNA uCkMjj6VmUbk5zT+BS1Xl3KvGvSbGGY/1ji2b+JTMP7siF7Xr+qGTaYPOIN0a0Po ppkHJMIU0XjSOfFFH8v71DOz65vW5q8oN4E+/n8cZOSGjYdyyyL1ccytQG1RTlUc frzGl9R+GXqqH8/MBkksZuNo3VjAizmRmAO2BbkAPzQjwVI9CilBRK69PQvrGw7R NL4ohkPr6fNdNXFg2J8v0opC/qRo9JSD4YTT8q2BEGER3g/9iqK7/lMzh2Hpo/0G jKSO+ZafZpglD3/n9dXGLABxTDzd/7rhk6S6N2/aAmRvh97KcIbJny6qRJAcb301 QGRqG8vogMlEfAVVBSkcaZxdFXeGBNdoq++Xo7CZb393I6rCCfw5HBZI5+QseMbA mR6wRosvLtUaLb/9kN3am9ee8DmCH9VTFdTM5IvqWkrpINiBOZxMRVeyKipbREaN StWmGcgbVbp8d3jMlf5ZAcH1jlYcPIF9OiUkHjEFnfgCQxsigJ6W6TFqdcNyEKAt Tyn5FyAeVEGj078CuGbn7pcIRr0Yh0ZCd39Jq0DFNzSvR8SVhAkcsFyoH5akQ6gN muoLiT5A6IcIvq+vIy7e =8Lwc -----END PGP SIGNATURE----- --++alDQ2ROsODg1x+-- From owner-freebsd-arch@FreeBSD.ORG Fri Jul 11 23:29:18 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5FDFE53F; Fri, 11 Jul 2014 23:29:18 +0000 (UTC) Received: from mail-qg0-x22c.google.com (mail-qg0-x22c.google.com [IPv6:2607:f8b0:400d:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE7A622E5; Fri, 11 Jul 2014 23:29:17 +0000 (UTC) Received: by mail-qg0-f44.google.com with SMTP id j107so1575421qga.17 for ; Fri, 11 Jul 2014 16:29:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=jzb3fi3LTAlxW5rt5+RpoKVNqAMQY/Meiv9t4BVwgq0=; b=ze7293XzbjIFmhi8mFKq8OmFjmW7wMoxggMTSY5TCkOepcSw/yfMiPhd8elAu8yD81 e49wsfidiejcVOhYmXHA6m90SCD/0csilkgjIfpimiPhA3r6+svoebl04WBRbI8nZyTE QVbRhcFaDbzyJIjgdUPH5z8LsXiqdMf9lMdRZL5gU7aR3PlL9w2ldIZNnE2HV1oOIXGn NYhQl7cAGkYF/nHVSOJ+awZTpwRWr04utZfYtXRFE150lH5Hhu/5DCkA6qM9pWXfILms 5P1TghM/IdSYdsQ03yPKcaLnXLCbfBiCqsNGZ0nioirDNCizq62kZrqyga0gmHfnNbGe X17g== X-Received: by 10.140.22.134 with SMTP id 6mr3444617qgn.4.1405121357044; Fri, 11 Jul 2014 16:29:17 -0700 (PDT) Received: from pwnie.vrt.sourcefire.com (moist.vrt.sourcefire.com. [198.148.79.134]) by mx.google.com with ESMTPSA id 7sm3701538qgg.25.2014.07.11.16.29.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Jul 2014 16:29:16 -0700 (PDT) Date: Fri, 11 Jul 2014 19:29:14 -0400 From: Shawn Webb To: freebsd-arch@freebsd.org Subject: [RFC] ASLR Whitepaper and Candidate Final Patch Message-ID: <20140711232914.GH41807@pwnie.vrt.sourcefire.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="asNXdz5DenlsLVVk" Content-Disposition: inline X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: Mutt/1.5.23 (2014-03-12) Cc: PaX Team , bdrewery@freebsd.org, alc@rice.edu, des@freebsd.org, Oliver Pinter X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jul 2014 23:29:18 -0000 --asNXdz5DenlsLVVk Content-Type: multipart/mixed; boundary="rfwNdt5cNUUjB/69" Content-Disposition: inline --rfwNdt5cNUUjB/69 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hey All, Oliver Pinter and I have been working hard on our ASLR implementation. We're now in the final stages of development and would like to get feedback from the community. I've attached to this email a small whitepaper that details our implementation and the accompanying patch. There is one part of the patch that I wrote that is quite an ugly hack and would like to get some feedback on. I added a little hack to sys_mmap() to apply ASLR to calls to mmap(2) when MAP_32BIT is specified. I'd like to remove that ugly hack to something a bit more beautiful, so if anyone has any suggestions, I'm all ears. Other than that ugly hack, the code adheres to FreeBSD's style(9) standards. I believe we have an awesome implementation, one I've personally been using without issue for months. I'm looking forward to your comments and questions. I've CC'd the PaX team. Please keep them CC'd in your replies. Thank you very much, Shawn Webb CC: PaX Team CC: Oliver Pinter CC: des@freebsd.org CC: alc@rice.edu CC: bdrewery@freebsd.org PS - Sorry for the duplicate emails. I hit the wrong key and didn't CC everyone. --rfwNdt5cNUUjB/69 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="2014-07-10_aslr_whitepaper.txt" Content-Transfer-Encoding: quoted-printable Introducing ASLR For FreeBSD =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Shawn Webb Oliver Pinter 10 July 2014 http://www.hardenedbsd.org/ [ 1. Introduction ] Security in FreeBSD's is based primarily in policy-based technologies. Exis= ting tools such as jails, Capsicum, vnet/vimage, and the MAC framework, can make FreeBSD-based systems quite resilient against attacks. FreeBSD lacks basic low-level exploit mitigation, such as Address Space Layout Randomization (ASLR)[1]. ASLR randomizes the address space layout of an application, maki= ng exploitation difficult for an attacker. This paper and the associated implementation aim to provide a secure, robust, extensible, and easily-man= aged form of ASLR fit for production use within FreeBSD. [ 2. History ] On 14 May 2013, Oliver Pinter published to GitHub an initial patch[2]. His = work was inspired by Elad Efrat's work in NetBSD. The patch was submitted to Fre= eBSD as a bug report on 24 Aug 2013[3]. Independently of Oliver's work, on 30 Jun 2014, Shawn Webb posted on his tech blog that he was interested in implemen= ting ASLR for FreeBSD[4]. Oliver found the post and suggested that he and Shawn = work together. On 08 Jun 2014, preparatory work was committed to FreeBSD, adding Position-Independent Executable (PIE) support in base[5]. On 07 Apr 2014, SoldierX[6] agreed to sponsor the project and donated a sparc64 box and a beaglebone black to Shawn Webb. This hardware is used for testing and debug= ging ASLR on those platforms. [ 3. General Overview ] ASLR is enabled by default for all architectures and controlled by the PAX_= ASLR kernel option. This means ASLR will be applied to all supported application= s. If a user wishes to disable ASLR for a given application, the user must for= ce that application to opt-out (detailed later). Another kernel option, PAX_SYSCTLS, exposes additional tunables (via sysctl= ), allowing ASLR behavior control without requiring a reboot. By default, the sysctl security.pax.aslr.status can only be changed at boot time via /boot/loader.conf. Enabling the PAX_SYSCTLS kernel option allows a root user to modify security.pax.aslr.status. See Appendix B for a list of the tunabl= es. ASLR tunables are per-jail and each jail inherits its parent jail's setting= s. Having per-jail tunables allows more flexibility in shared-hosting environments. This structure also allows a user to selectively disable ASLR for applications that misbehave. ASLR-disabled applications will still have policy-based security applied to it by virtue of being jailed. The mac_bsdextended(4) MAC module and its corresponding ugidfw(8) applicati= on have been modified to allow a user to enable or disable ASLR for specific applications. The filesys object specification has been modified to pass the inode along with the filesystem id when the new paxflags option is specifie= d. The paxflags option is optionally placed at the end of the rule. An upper-c= ase "A" argument to the option signifies ASLR is enabled for the application an= d a lower-case "a" signifies ASLR is disabled for the application. Sample ugidf= w(8) rules are in Appendix C. [ 4. Implementation Details ] A new sysinit subroutine ID, SI_SUB_PAX, initializes all ASLR system variab= les.=20 Upon system boot, tunables from /boot/loader.conf are checked for validity.= Any invalid values, generate a warning message to the console and the tunable is set to a sensible default. For the sake of performance, the ASLR system relies on per-process deltas rather than calling arc4random(3) for each mapping. When a process calls execve(2), the ASLR system is initialized. Deltas are randomly generated for the execution base, mmap(2), and stack addresses. Only the execution base of applications compiled as PIEs are randomized. The execution base of non-PIE applications are not modified. The mappings of shared objects are randomized for both PIE and non-PIE applications. The deltas are used as a hint to the Virtual Memory (VM) system. The VM sys= tem may modify the hint to make a better fit for superpages and other alignment constraints. The delta applied to the PIE execbase is different than the delta applied to the base address of shared objects. In the Executable and Linkable File (EL= F) image handler, the execution base of PIE applications is randomized by addi= ng the delta controlled by security.pax.aslr.exec_len tunable to et_dyn_addr, which is initialized to be ET_DYN_LOAD_ADDR (an architecture-dependent macr= o). The base address of shared objects loaded by the runtime linker are randomi= zed by applying the delta controlled by the security.pax.aslr.mmap_len tunable = in sys_mmap(). Stack randomization is implemented using a stack gap[7]. On executable image activation, the stack delta is computed and then subtracted from the top of the stack. [ 5. Further Enhancements ] The existing gap-based stack randomization is not optimal. Mapping-base sta= ck randomization is more robust, but hard-coded kernel structures and addresse= s, especially PS_STRINGS, will need to be modified. The required changes to PS_STRINGS are major and will likely touch userland along with the kernel. The original PaX implementation, from which the FreeBSD implementation is inspired, uses a special ELF process header which requires modification of executable files. The authors of the FreeBSD implementation have deliberate= ly chosen to go a different route based on mac_bsdextended(4)/ugidfw(8). Suppo= rt for filesystem extended attributes will be added at a later time. FreeBSD's virtual Dynamic Shared Object (vDSO) implementation, an efficient technique for calling kernel code from userland, uses hardcoded, non-randomized addresses. The vDSO implementation should be reworked to be = at a randomized address, providing the address as an auxiliary vector passed to the image via the stack. [ 6. Known Issues ] ASLR does not function properly on 32bit ARM. When a process fork(2)s and c= alls execve(2) and the child process exits, the parent process crashes upon receiving the SIGCHLD signal. No matter which application crashed, the pc register ends up being 0xc0000000. The ktrace(1) facility proved that the application crashed upon receiving the SIGCHLD signal. [ Appendix A - References ] [1]: http://pax.grsecurity.net/docs/aslr.txt [2]: https://github.com/opntr/freebsd-patches-2013-tavasz/blob/master/r2499= 52%2BASLR/0001-PaX-ASLR-mmap-and-basic-stack-randomization.patch [3]: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D181497 [4]: http://0xfeedface.org/blog/lattera/2013-06-29/long-term-plans [5]: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D267233 [6]: https://www.soldierx.com/ [7]: http://www.openbsd.org/papers/auug04/mgp00005.html [ Appendix B - ASLR Tunables ] NOTE: All tunables can only be changed during boot-time via /boot/loader.co= nf unless the kernel has been compiled with PAX_SYSCTLS. security.pax.aslr.status (integer): Description: Toggle system-wide ASLR protection. Values: 0 - ASLR disabled system-wide. Individual applications may NOT opt in. 1 - ASLR disabled by default. Individual applications may opt in. 2 - ASLR enabled by default. Individual applications may opt out. 3 - ASLR enabled system-wide. Individual applications may NOT opt out. Default: 2 security.pax.aslr.debug (integer): Description: Toggle debugging output. Values: 0 - Debug output disabled. 1 - Basic debug output enabled. 2 - Verbose debug output enabled. Default: 0 security.pax.aslr.mmap_len (integer): Description: Set the number of bits to be randomized for mmap(2) calls. Values: For 32bit systems, minimum of 8, maximum of 16. For 64bit systems, minimum of 16, maximum of 32. Default: For 32bit systems, 8. For 64bit systems, 21. security.pax.aslr.stack_len (integer): Description: Set the number of bits to be randomized for the stack. Values: For 32bit systems, minimum of 6, maximum of 12. For 64bit systems, minimum of 12, maximum of 21. Default: For 32bit systems, 6. For 64bit systems, 16. security.pax.aslr.exec_len (integer): Description: Set the number of bits to be randomized for the PIE exec base. Values: For 32bit systems, minimum of 6, maximum of 12. For 64bit systems, minimum of 12, maximum of 21. Default: For 32bit systems, 6. For 64bit systems, 21. [ Appendix C - Sample ugidfw(8) rules ] When security.pax.aslr.status is set to 2 (require applications to opt-out): ugidfw add subject uid shawn object filesys /bin/ls mode rx paxflags a - This adds a rule to disable ASLR for /bin/ls for the user shawn. ugidfw add subject uid 0:65535 object filesys /bin/ls mode rx paxflags a - This adds a rule to disable ASLR for /bin/ls for all users. When security.pax.aslr.status is set to 1 (require applications to opt-in): ugidfw add subject uid shawn object filesys /bin/ls mode rx paxflags A - This adds a rule to enable ASLR for /bin/ls for the user shawn. ugidfw add subject uid 0:65535 object filesys /bin/ls mode rx paxflags A - This adds a rule to enable ASLR for /bin/ls for all users. [ Appendix D - Files Modified/Created in 11-CURRENT ] lib/libugidfw/ugidfw.c lib/libugidfw/ugidfw.h release/Makefile sys/amd64/amd64/elf_machdep.c sys/amd64/include/vmparam.h sys/amd64/linux32/linux32_sysvec.c sys/arm/arm/elf_machdep.c sys/compat/freebsd32/freebsd32_misc.c sys/compat/ia32/ia32_sysvec.c sys/conf/NOTES sys/conf/files sys/conf/options sys/i386/i386/elf_machdep.c sys/i386/ibcs2/ibcs2_sysvec.c sys/i386/linux/linux_sysvec.c sys/kern/imgact_aout.c sys/kern/imgact_elf.c sys/kern/init_main.c sys/kern/kern_exec.c sys/kern/kern_fork.c sys/kern/kern_jail.c sys/kern/kern_pax.c sys/kern/kern_pax_aslr.c sys/kern/kern_pax_log.c sys/kern/kern_sig.c sys/mips/mips/elf_machdep.c sys/mips/mips/freebsd32_machdep.c sys/powerpc/powerpc/elf32_machdep.c sys/powerpc/powerpc/elf64_machdep.c sys/security/mac_bsdextended/mac_bsdextended.c sys/security/mac_bsdextended/mac_bsdextended.h sys/security/mac_bsdextended/ugidfw_internal.h sys/security/mac_bsdextended/ugidfw_system.c sys/security/mac_bsdextended/ugidfw_vnode.c sys/sparc64/sparc64/elf_machdep.c sys/sys/imgact.h sys/sys/jail.h sys/sys/kernel.h sys/sys/pax.h sys/sys/proc.h sys/sys/sysent.h sys/vm/vm_map.c sys/vm/vm_map.h sys/vm/vm_mmap.c usr.sbin/ugidfw/ugidfw.c --rfwNdt5cNUUjB/69 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="2014-07-10_aslr.patch" Content-Transfer-Encoding: quoted-printable diff --git a/lib/libugidfw/ugidfw.c b/lib/libugidfw/ugidfw.c index 0dc423d..070cd0e 100644 --- a/lib/libugidfw/ugidfw.c +++ b/lib/libugidfw/ugidfw.c @@ -36,6 +36,9 @@ #include #include #include +#include +#include +#include =20 #include =20 @@ -44,6 +47,8 @@ #include #include #include +#include +#include =20 #include "ugidfw.h" =20 @@ -195,7 +200,7 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule, = char *buf, size_t buflen) cur +=3D len; } if (rule->mbr_subject.mbs_flags & MBS_PRISON_DEFINED) { - len =3D snprintf(cur, left, "jailid %d ",=20 + len =3D snprintf(cur, left, "jailid %d ", rule->mbr_subject.mbs_prison); if (len < 0 || len > left) goto truncated; @@ -329,14 +334,19 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule= , char *buf, size_t buflen) cur +=3D len; } if (rule->mbr_object.mbo_flags & MBO_FSID_DEFINED) { - numfs =3D getmntinfo(&mntbuf, MNT_NOWAIT); - for (i =3D 0; i < numfs; i++) - if (memcmp(&(rule->mbr_object.mbo_fsid), - &(mntbuf[i].f_fsid), - sizeof(mntbuf[i].f_fsid)) =3D=3D 0) - break; - len =3D snprintf(cur, left, "filesys %s ",=20 - i =3D=3D numfs ? "???" : mntbuf[i].f_mntonname); + if (rule->mbr_object.mbo_inode =3D=3D 0) { + numfs =3D getmntinfo(&mntbuf, MNT_NOWAIT); + for (i =3D 0; i < numfs; i++) + if (memcmp(&(rule->mbr_object.mbo_fsid), + &(mntbuf[i].f_fsid), + sizeof(mntbuf[i].f_fsid)) =3D=3D 0) + break; + len =3D snprintf(cur, left, "filesys %s ", + i =3D=3D numfs ? "???" : mntbuf[i].f_mntonname); + } else { + len =3D snprintf(cur, left, "filesys %s ", + rule->mbr_object.mbo_paxpath); + } if (len < 0 || len > left) goto truncated; left -=3D len; @@ -500,6 +510,33 @@ bsde_rule_to_string(struct mac_bsdextended_rule *rule,= char *buf, size_t buflen) cur +=3D len; } =20 + if (rule->mbr_pax) { + len =3D snprintf(cur, left, " paxflags "); + if (len < 0 || len > left) + goto truncated; + left -=3D len; + cur +=3D len; + + if (rule->mbr_pax & MBI_FORCE_ASLR_ENABLED) { + len =3D snprintf(cur, left, "A"); + if (len < 0 || len > left) + goto truncated; + + left -=3D len; + cur +=3D len; + } + + if (rule->mbr_pax & MBI_FORCE_ASLR_DISABLED) { + len =3D snprintf(cur, left, "a"); + if (len < 0 || len > left) + goto truncated; + + left -=3D len; + cur +=3D len; + } + + } + return (0); =20 truncated: @@ -507,8 +544,8 @@ truncated: } =20 int -bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, - size_t buflen, char *errstr){ +bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, size_t buflen, cha= r *errstr) +{ struct passwd *pwd; uid_t uid1, uid2; char *spec1, *spec2, *endp; @@ -556,8 +593,8 @@ bsde_parse_uidrange(char *spec, uid_t *min, uid_t *max, } =20 int -bsde_parse_gidrange(char *spec, gid_t *min, gid_t *max, - size_t buflen, char *errstr){ +bsde_parse_gidrange(char *spec, gid_t *min, gid_t *max, size_t buflen, cha= r *errstr) +{ struct group *grp; gid_t gid1, gid2; char *spec1, *spec2, *endp; @@ -759,17 +796,22 @@ bsde_parse_type(char *spec, int *type, size_t buflen,= char *errstr) len =3D snprintf(errstr, buflen, "Unknown type code: %c", spec[i]); return (-1); - }=20 + } } =20 return (0); } =20 int -bsde_parse_fsid(char *spec, struct fsid *fsid, size_t buflen, char *errstr) +bsde_parse_fsid(char *spec, struct fsid *fsid, ino_t *inode, size_t buflen= , char *errstr) { size_t len; struct statfs buf; + struct stat sb; + int fd, paxstatus; + size_t bufsz; + + *inode =3D 0; =20 if (statfs(spec, &buf) < 0) { len =3D snprintf(errstr, buflen, "Unable to get id for %s: %s", @@ -779,6 +821,21 @@ bsde_parse_fsid(char *spec, struct fsid *fsid, size_t = buflen, char *errstr) =20 *fsid =3D buf.f_fsid; =20 + if (strcmp(buf.f_fstypename, "devfs") !=3D 0) { + bufsz =3D sizeof(int); + if (!sysctlbyname("kern.features.aslr", &paxstatus, &bufsz, + NULL, 0)) { + fd =3D open(spec, O_RDONLY); + if (fd !=3D -1) { + if (fstat(fd, &sb) =3D=3D 0) + if(S_ISDIR(sb.st_mode) =3D=3D 0) + *inode =3D sb.st_ino; + + close(fd); + } + } + } + return (0); } =20 @@ -852,13 +909,18 @@ bsde_parse_object(int argc, char *argv[], return (-1); } if (bsde_parse_fsid(argv[current+1], &fsid, - buflen, errstr) < 0) + &object->mbo_inode, buflen, errstr) < 0) return (-1); flags |=3D MBO_FSID_DEFINED; if (nextnot) { neg ^=3D MBO_FSID_DEFINED; nextnot =3D 0; } + if (object->mbo_inode) + snprintf(object->mbo_paxpath, MAXPATHLEN, "%s", + argv[current+1]); + else + memset(object->mbo_paxpath, 0x00, MAXPATHLEN); current +=3D 2; } else if (strcmp(argv[current], "suid") =3D=3D 0) { flags |=3D MBO_SUID; @@ -984,7 +1046,42 @@ bsde_parse_mode(int argc, char *argv[], mode_t *mode,= size_t buflen, len =3D snprintf(errstr, buflen, "Unknown mode letter: %c", argv[0][i]); return (-1); - }=20 + } + } + + return (0); +} + +int +bsde_parse_paxflags(int argc, char *argv[], uint32_t *pax, size_t buflen, = char *errstr) +{ + size_t len; + int i; + + if (argc =3D=3D 0) { + len =3D snprintf(errstr, buflen, "paxflags expects mode value"); + return (-1); + } + + if (argc !=3D 1) { + len =3D snprintf(errstr, buflen, "'%s' unexpected", argv[1]); + return (-1); + } + + *pax =3D 0; + for (i =3D 0; i < strlen(argv[0]); i++) { + switch (argv[0][i]) { + case 'A': + *pax |=3D MBI_FORCE_ASLR_ENABLED; + break; + case 'a': + *pax |=3D MBI_FORCE_ASLR_DISABLED; + break; + default: + len =3D snprintf(errstr, buflen, "Unknown mode letter: %c", + argv[0][i]); + return (-1); + } } =20 return (0); @@ -997,6 +1094,7 @@ bsde_parse_rule(int argc, char *argv[], struct mac_bsd= extended_rule *rule, int subject, subject_elements, subject_elements_length; int object, object_elements, object_elements_length; int mode, mode_elements, mode_elements_length; + int paxflags, paxflags_elements, paxflags_elements_length=3D0; int error, i; size_t len; =20 @@ -1037,11 +1135,23 @@ bsde_parse_rule(int argc, char *argv[], struct mac_= bsdextended_rule *rule, return (-1); } =20 + /* Search forward for paxflags */ + paxflags =3D -1; + for (i =3D 1; i < argc; i++) + if (strcmp(argv[i], "paxflags") =3D=3D 0) + paxflags =3D i; + + if (paxflags >=3D 0) { + paxflags_elements =3D paxflags + 1; + paxflags_elements_length =3D argc - paxflags_elements; + } + subject_elements_length =3D object - subject - 1; object_elements =3D object + 1; object_elements_length =3D mode - object_elements; mode_elements =3D mode + 1; - mode_elements_length =3D argc - mode_elements; + mode_elements_length =3D argc - mode_elements - + (paxflags_elements_length ? paxflags_elements_length+1 : 0); =20 error =3D bsde_parse_subject(subject_elements_length, argv + subject_elements, &rule->mbr_subject, buflen, errstr); @@ -1058,6 +1168,13 @@ bsde_parse_rule(int argc, char *argv[], struct mac_b= sdextended_rule *rule, if (error) return (-1); =20 + if (paxflags >=3D 0) { + error =3D bsde_parse_paxflags(paxflags_elements_length, argv + paxflags_= elements, + &rule->mbr_pax, buflen, errstr); + if (error) + return (-1); + } + return (0); } =20 diff --git a/lib/libugidfw/ugidfw.h b/lib/libugidfw/ugidfw.h index 5b7fcf2..cef469c 100644 --- a/lib/libugidfw/ugidfw.h +++ b/lib/libugidfw/ugidfw.h @@ -39,6 +39,8 @@ int bsde_rule_to_string(struct mac_bsdextended_rule *rule= , char *buf, size_t buflen); int bsde_parse_mode(int argc, char *argv[], mode_t *mode, size_t buflen, char *errstr); +int bsde_parse_paxflags(int argc, char *argv[], uint32_t *pax, size_t bufl= en, + char *errstr); int bsde_parse_rule(int argc, char *argv[], struct mac_bsdextended_rule *rule, size_t buflen, char *errstr); int bsde_parse_rule_string(const char *string, diff --git a/sys/amd64/amd64/elf_machdep.c b/sys/amd64/amd64/elf_machdep.c index fdc4d56..ffb5e31 100644 --- a/sys/amd64/amd64/elf_machdep.c +++ b/sys/amd64/amd64/elf_machdep.c @@ -26,12 +26,17 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include #include #include #include +#ifdef PAX_ASLR +#include +#endif #include #include #include @@ -81,6 +86,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf64_sysvec, &elf64_freebsd_sysvec); =20 diff --git a/sys/amd64/include/vmparam.h b/sys/amd64/include/vmparam.h index bda9722..5e83a8f 100644 --- a/sys/amd64/include/vmparam.h +++ b/sys/amd64/include/vmparam.h @@ -170,7 +170,7 @@ #define VM_MAXUSER_ADDRESS UVADDR(NUPML4E, 0, 0, 0) =20 #define SHAREDPAGE (VM_MAXUSER_ADDRESS - PAGE_SIZE) -#define USRSTACK SHAREDPAGE +#define USRSTACK (SHAREDPAGE - 4*PAGE_SIZE) =20 #define VM_MAX_ADDRESS UPT_MAX_ADDRESS #define VM_MIN_ADDRESS (0) diff --git a/sys/amd64/linux32/linux32_sysvec.c b/sys/amd64/linux32/linux32= _sysvec.c index c06ce11..f4f99f58 100644 --- a/sys/amd64/linux32/linux32_sysvec.c +++ b/sys/amd64/linux32/linux32_sysvec.c @@ -33,6 +33,7 @@ #include __FBSDID("$FreeBSD$"); #include "opt_compat.h" +#include "opt_pax.h" =20 #ifndef COMPAT_FREEBSD32 #error "Unable to compile Linux-emulator due to missing COMPAT_FREEBSD32 o= ption!" @@ -84,6 +85,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(linux, 1); =20 MALLOC_DEFINE(M_LINUX, "linux", "Linux mode structures"); @@ -1037,6 +1042,11 @@ struct sysentvec elf_linux_sysvec =3D { .sv_shared_page_base =3D LINUX32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_sysvec, &elf_linux_sysvec); =20 diff --git a/sys/arm/arm/elf_machdep.c b/sys/arm/arm/elf_machdep.c index 8ef9bd4..26e37e6 100644 --- a/sys/arm/arm/elf_machdep.c +++ b/sys/arm/arm/elf_machdep.c @@ -26,6 +26,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -46,6 +48,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf32_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -79,6 +85,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf32_Brandinfo freebsd_brand_info =3D { diff --git a/sys/compat/freebsd32/freebsd32_misc.c b/sys/compat/freebsd32/f= reebsd32_misc.c index 815a9b7..e92453f 100644 --- a/sys/compat/freebsd32/freebsd32_misc.c +++ b/sys/compat/freebsd32/freebsd32_misc.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include "opt_compat.h" #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -113,6 +114,10 @@ __FBSDID("$FreeBSD$"); =20 FEATURE(compat_freebsd_32bit, "Compatible with 32-bit FreeBSD"); =20 +#ifdef PAX_ASLR +#include +#endif + #ifndef __mips__ CTASSERT(sizeof(struct timeval32) =3D=3D 8); CTASSERT(sizeof(struct timespec32) =3D=3D 8); @@ -2766,6 +2771,10 @@ freebsd32_copyout_strings(struct image_params *imgp) szsigcode =3D 0; destp =3D (uintptr_t)arginfo; =20 +#ifdef PAX_ASLR + pax_aslr_stack(curthread, &destp); +#endif + /* * install sigcode */ diff --git a/sys/compat/ia32/ia32_sysvec.c b/sys/compat/ia32/ia32_sysvec.c index 770bbbf..67ad330 100644 --- a/sys/compat/ia32/ia32_sysvec.c +++ b/sys/compat/ia32/ia32_sysvec.c @@ -29,6 +29,7 @@ __FBSDID("$FreeBSD$"); =20 #include "opt_compat.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -74,6 +75,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + CTASSERT(sizeof(struct ia32_mcontext) =3D=3D 640); CTASSERT(sizeof(struct ia32_ucontext) =3D=3D 704); CTASSERT(sizeof(struct ia32_sigframe) =3D=3D 800); @@ -136,6 +141,11 @@ struct sysentvec ia32_freebsd_sysvec =3D { .sv_shared_page_base =3D FREEBSD32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_ia32_sysvec, &ia32_freebsd_sysvec); =20 diff --git a/sys/conf/NOTES b/sys/conf/NOTES index 6959425..bd6af74 100644 --- a/sys/conf/NOTES +++ b/sys/conf/NOTES @@ -2986,3 +2986,7 @@ options RANDOM_RWFILE # Read and write entropy cache =20 # Module to enable execution of application via emulators like QEMU options IMAGACT_BINMISC + +# Address Space Layout Randomization (ASLR) +options PAX_ASLR +options PAX_SYSCTLS diff --git a/sys/conf/files b/sys/conf/files index 64101d1..71f8c1a 100644 --- a/sys/conf/files +++ b/sys/conf/files @@ -2908,6 +2908,9 @@ kern/kern_mtxpool.c standard kern/kern_mutex.c standard kern/kern_ntptime.c standard kern/kern_osd.c standard +kern/kern_pax.c optional pax_aslr +kern/kern_pax_aslr.c optional pax_aslr +kern/kern_pax_log.c optional pax_aslr kern/kern_physio.c standard kern/kern_pmc.c standard kern/kern_poll.c optional device_polling diff --git a/sys/conf/options b/sys/conf/options index 8ec07e0..180f169 100644 --- a/sys/conf/options +++ b/sys/conf/options @@ -920,6 +920,12 @@ RACCT opt_global.h # Resource Limits RCTL opt_global.h =20 +# PaX - hardening options +PAX_ASLR opt_pax.h +PAX_ASLR_MAX_SEC opt_pax.h +PAX_MPROTECT opt_pax.h +PAX_SYSCTLS opt_pax.h + # Random number generator(s) RANDOM_YARROW opt_random.h RANDOM_FORTUNA opt_random.h diff --git a/sys/i386/i386/elf_machdep.c b/sys/i386/i386/elf_machdep.c index 034b4c4..9571252 100644 --- a/sys/i386/i386/elf_machdep.c +++ b/sys/i386/i386/elf_machdep.c @@ -26,6 +26,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -46,6 +48,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf32_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -81,6 +87,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/i386/ibcs2/ibcs2_sysvec.c b/sys/i386/ibcs2/ibcs2_sysvec.c index 5d007c7..1bb9d89 100644 --- a/sys/i386/ibcs2/ibcs2_sysvec.c +++ b/sys/i386/ibcs2/ibcs2_sysvec.c @@ -31,6 +31,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -50,6 +52,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(ibcs2, 1); =20 extern int bsd_to_ibcs2_errno[]; @@ -89,6 +95,11 @@ struct sysentvec ibcs2_svr3_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D NULL, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static int diff --git a/sys/i386/linux/linux_sysvec.c b/sys/i386/linux/linux_sysvec.c index 0ad6791..403070c 100644 --- a/sys/i386/linux/linux_sysvec.c +++ b/sys/i386/linux/linux_sysvec.c @@ -29,6 +29,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -72,6 +74,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + MODULE_VERSION(linux, 1); =20 MALLOC_DEFINE(M_LINUX, "linux", "Linux mode structures"); @@ -974,6 +980,11 @@ struct sysentvec linux_sysvec =3D { .sv_shared_page_base =3D LINUX_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(aout_sysvec, &linux_sysvec); =20 @@ -1012,6 +1023,11 @@ struct sysentvec elf_linux_sysvec =3D { .sv_shared_page_base =3D LINUX_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D linux_schedtail, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf_sysvec, &elf_linux_sysvec); =20 diff --git a/sys/kern/imgact_aout.c b/sys/kern/imgact_aout.c index 3ae78de..aac03f1 100644 --- a/sys/kern/imgact_aout.c +++ b/sys/kern/imgact_aout.c @@ -27,6 +27,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -62,6 +64,10 @@ __FBSDID("$FreeBSD$"); #include #endif =20 +#ifdef PAX_ASLR +#include +#endif + static int exec_aout_imgact(struct image_params *imgp); static int aout_fixup(register_t **stack_base, struct image_params *imgp); =20 @@ -99,6 +105,11 @@ struct sysentvec aout_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 #elif defined(__amd64__) @@ -143,6 +154,11 @@ struct sysentvec aout_sysvec =3D { .sv_set_syscall_retval =3D ia32_set_syscall_retval, .sv_fetch_syscall_args =3D ia32_fetch_syscall_args, .sv_syscallnames =3D freebsd32_syscallnames, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, /* XXXOP */ +#else + .sv_pax_aslr_init =3D NULL, +#endif }; #else #error "Port me" diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c index 6342119..fe2976d 100644 --- a/sys/kern/imgact_elf.c +++ b/sys/kern/imgact_elf.c @@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_compat.h" #include "opt_core.h" +#include "opt_pax.h" =20 #include #include @@ -48,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -81,6 +83,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#if defined(PAX_ASLR) +#include +#endif + #define ELF_NOTE_ROUNDSIZE 4 #define OLD_EI_BRAND 8 =20 @@ -653,16 +659,16 @@ __elfN(load_file)(struct proc *p, const char *file, u= _long *addr, hdr =3D (const Elf_Ehdr *)imgp->image_header; if ((error =3D __elfN(check_header)(hdr)) !=3D 0) goto fail; - if (hdr->e_type =3D=3D ET_DYN) + if (hdr->e_type =3D=3D ET_DYN) { rbase =3D *addr; - else if (hdr->e_type =3D=3D ET_EXEC) + } else if (hdr->e_type =3D=3D ET_EXEC) { rbase =3D 0; - else { + } else { error =3D ENOEXEC; goto fail; } =20 - /* Only support headers that fit within first page for now */ + /* Only support headers that fit within first page for now */ if ((hdr->e_phoff > PAGE_SIZE) || (u_int)hdr->e_phentsize * hdr->e_phnum > PAGE_SIZE - hdr->e_phoff) { error =3D ENOEXEC; @@ -787,16 +793,7 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *i= mgp) if (hdr->e_type =3D=3D ET_DYN) { if ((brand_info->flags & BI_CAN_EXEC_DYN) =3D=3D 0) return (ENOEXEC); - /* - * Honour the base load address from the dso if it is - * non-zero for some reason. - */ - if (baddr =3D=3D 0) - et_dyn_addr =3D ET_DYN_LOAD_ADDR; - else - et_dyn_addr =3D 0; - } else - et_dyn_addr =3D 0; + } sv =3D brand_info->sysvec; if (interp !=3D NULL && brand_info->interp_newpath !=3D NULL) newinterp =3D brand_info->interp_newpath; @@ -817,6 +814,21 @@ __CONCAT(exec_, __elfN(imgact))(struct image_params *i= mgp) error =3D exec_new_vmspace(imgp, sv); imgp->proc->p_sysent =3D sv; =20 + et_dyn_addr =3D 0; + if (hdr->e_type =3D=3D ET_DYN) { + /* + * Honour the base load address from the dso if it is + * non-zero for some reason. + */ + if (baddr =3D=3D 0) { + et_dyn_addr =3D ET_DYN_LOAD_ADDR; +#ifdef PAX_ASLR + if (pax_aslr_active(NULL, imgp->proc)) + et_dyn_addr +=3D imgp->proc->p_vmspace->vm_aslr_delta_exec; +#endif + } + } + vn_lock(imgp->vp, LK_EXCLUSIVE | LK_RETRY); if (error) return (error); diff --git a/sys/kern/init_main.c b/sys/kern/init_main.c index 141d438..9301b57 100644 --- a/sys/kern/init_main.c +++ b/sys/kern/init_main.c @@ -410,6 +410,7 @@ struct sysentvec null_sysvec =3D { .sv_fetch_syscall_args =3D null_fetch_syscall_args, .sv_syscallnames =3D NULL, .sv_schedtail =3D NULL, + .sv_pax_aslr_init =3D NULL, }; =20 /* diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c index 489096b..81a451f 100644 --- a/sys/kern/kern_exec.c +++ b/sys/kern/kern_exec.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); #include "opt_capsicum.h" #include "opt_hwpmc_hooks.h" #include "opt_ktrace.h" +#include "opt_pax.h" #include "opt_vm.h" =20 #include @@ -93,6 +94,10 @@ __FBSDID("$FreeBSD$"); dtrace_execexit_func_t dtrace_fasttrap_exec; #endif =20 +#if defined(PAX_ASLR) +#include +#endif + SDT_PROVIDER_DECLARE(proc); SDT_PROBE_DEFINE1(proc, kernel, , exec, "char *"); SDT_PROBE_DEFINE1(proc, kernel, , exec__failure, "int"); @@ -402,6 +407,11 @@ do_execve(td, args, mac_p) imgp->pagesizes =3D 0; imgp->pagesizeslen =3D 0; imgp->stack_prot =3D 0; + imgp->pax_flags =3D 0; + +#if defined(PAX_MPROTECT) || defined(PAX_ASLR) + pax_elf(imgp, 0); +#endif =20 #ifdef MAC error =3D mac_execve_enter(imgp, mac_p); @@ -1065,6 +1075,10 @@ exec_new_vmspace(imgp, sv) map =3D &vmspace->vm_map; } =20 +#ifdef PAX_ASLR + pax_aslr_init(curthread, imgp); +#endif + /* Map a shared page */ obj =3D sv->sv_shared_page_obj; if (obj !=3D NULL) { @@ -1099,6 +1113,9 @@ exec_new_vmspace(imgp, sv) */ vmspace->vm_ssize =3D sgrowsiz >> PAGE_SHIFT; vmspace->vm_maxsaddr =3D (char *)sv->sv_usrstack - ssiz; +#ifdef PAX_ASLR + vmspace->vm_maxsaddr -=3D vmspace->vm_aslr_delta_stack; +#endif =20 return (0); } @@ -1258,6 +1275,9 @@ exec_copyout_strings(imgp) szsigcode =3D *(p->p_sysent->sv_szsigcode); } destp =3D (uintptr_t)arginfo; +#ifdef PAX_ASLR + pax_aslr_stack(curthread, &destp); +#endif =20 /* * install sigcode diff --git a/sys/kern/kern_fork.c b/sys/kern/kern_fork.c index 8135afb..9577013 100644 --- a/sys/kern/kern_fork.c +++ b/sys/kern/kern_fork.c @@ -513,6 +513,11 @@ do_fork(struct thread *td, int flags, struct proc *p2,= struct thread *td2, } =20 /* + * XXXOP: this is the right place? + */ + p2->p_pax =3D p1->p_pax; + + /* * p_limit is copy-on-write. Bump its refcount. */ lim_fork(p1, p2); diff --git a/sys/kern/kern_jail.c b/sys/kern/kern_jail.c index 47cd568..d9036bd 100644 --- a/sys/kern/kern_jail.c +++ b/sys/kern/kern_jail.c @@ -33,6 +33,7 @@ __FBSDID("$FreeBSD$"); #include "opt_ddb.h" #include "opt_inet.h" #include "opt_inet6.h" +#include "opt_pax.h" =20 #include #include @@ -74,6 +75,10 @@ __FBSDID("$FreeBSD$"); #endif /* INET6 */ #endif /* DDB */ =20 +#if defined(PAX_ASLR) +#include +#endif + #include =20 #define DEFAULT_HOSTUUID "00000000-0000-0000-0000-000000000000" @@ -117,6 +122,10 @@ struct prison prison0 =3D { }; MTX_SYSINIT(prison0, &prison0.pr_mtx, "jail mutex", MTX_DEF); =20 +#if defined(PAX_ASLR) +SYSINIT(pax, SI_SUB_PAX, SI_ORDER_MIDDLE, pax_init_prison, (void *) &priso= n0); +#endif + /* allprison, allprison_racct and lastprid are protected by allprison_lock= =2E */ struct sx allprison_lock; SX_SYSINIT(allprison_lock, &allprison_lock, "allprison"); @@ -1307,6 +1316,10 @@ kern_jail_set(struct thread *td, struct uio *optuio,= int flags) goto done_releroot; } =20 +#if defined(PAX_ASLR) + pax_init_prison(pr); +#endif + mtx_lock(&pr->pr_mtx); /* * New prisons do not yet have a reference, because we do not diff --git a/sys/kern/kern_pax.c b/sys/kern/kern_pax.c new file mode 100644 index 0000000..1bd5ad0 --- /dev/null +++ b/sys/kern/kern_pax.c @@ -0,0 +1,214 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "opt_compat.h" +#include "opt_pax.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include + +#include + +#include + +SYSCTL_NODE(_security, OID_AUTO, pax, CTLFLAG_RD, 0, + "PaX (exploit mitigation) features."); + +struct prison * +pax_get_prison(struct thread *td, struct proc *proc) +{ + + if (td !=3D NULL) { + if ((td->td_proc !=3D NULL) && (td->td_proc->p_ucred !=3D NULL)) + return (td->td_proc->p_ucred->cr_prison); + + return (NULL); + } + if ((proc =3D=3D NULL) || (proc->p_ucred =3D=3D NULL)) + return (NULL); + + return (proc->p_ucred->cr_prison); +} + +void +pax_elf(struct image_params *imgp, uint32_t mode) +{ + u_int flags =3D 0; + + if ((mode & MBI_ALLPAX) =3D=3D MBI_ALLPAX) + goto end; + + if (mode & MBI_FORCE_ASLR_ENABLED) + flags |=3D PAX_NOTE_ASLR; + else if (mode & MBI_FORCE_ASLR_DISABLED) + flags |=3D PAX_NOTE_NOASLR; + +end: + if (imgp !=3D NULL) { + imgp->pax_flags =3D flags; + if (imgp->proc !=3D NULL) { + PROC_LOCK(imgp->proc); + imgp->proc->p_pax =3D flags; + PROC_UNLOCK(imgp->proc); + } + } +} + + +/* + * print out PaX settings on boot time, and validate some of them + */ +void +pax_init(void) +{ +#if defined(PAX_ASLR) + const char *status_str[] =3D { + [0] =3D "disabled", + [1] =3D "opt-in", + [2] =3D "opt-out", + [3] =3D "force enabled", + [4] =3D "UNKNOWN -> changed to \"force enabled\"" + }; +#endif + +#ifdef PAX_ASLR + switch (pax_aslr_status) { + case 0: + case 1: + case 2: + case 3: + break; + default: + printf("[PAX ASLR] WARNING, invalid PAX settings in loader.conf!" + " (pax_aslr_status =3D %d)\n", pax_aslr_status); + pax_aslr_status =3D 3; + break; + } + printf("[PAX ASLR] status: %s\n", status_str[pax_aslr_status]); + printf("[PAX ASLR] mmap: %d bit\n", pax_aslr_mmap_len); + printf("[PAX ASLR] exec base: %d bit\n", pax_aslr_exec_len); + printf("[PAX ASLR] stack: %d bit\n", pax_aslr_stack_len); + +#ifdef COMPAT_FREEBSD32 + switch (pax_aslr_compat_status) { + case 0: + case 1: + case 2: + case 3: + break; + default: + printf("[PAX ASLR (compat)] WARNING, invalid PAX settings in loader.conf= ! " + "(pax_aslr_compat_status =3D %d)\n", pax_aslr_compat_status); + pax_aslr_compat_status =3D 3; + break; + } + printf("[PAX ASLR (compat)] status: %s\n", status_str[pax_aslr_compat_sta= tus]); + printf("[PAX ASLR (compat)] mmap: %d bit\n", pax_aslr_compat_mmap_len); + printf("[PAX ASLR (compat)] exec base: %d bit\n", pax_aslr_compat_exec_le= n); + printf("[PAX ASLR (compat)] stack: %d bit\n", pax_aslr_compat_stack_len); +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + + printf("[PAX LOG] logging to system: %d\n", pax_log_log); + printf("[PAX LOG] logging to user: %d\n", pax_log_ulog); +} +SYSINIT(pax, SI_SUB_PAX, SI_ORDER_FIRST, pax_init, NULL); + +void +pax_init_prison(struct prison *pr) +{ + + if (pr =3D=3D NULL) + return; + + if (pr->pr_pax_set) + return; + + mtx_lock(&(pr->pr_mtx)); + + if (pax_aslr_debug) + uprintf("[PaX ASLR] %s: Setting prison %s ASLR variables\n", + __func__, pr->pr_name); + +#ifdef PAX_ASLR + pr->pr_pax_aslr_status =3D pax_aslr_status; + pr->pr_pax_aslr_debug =3D pax_aslr_debug; + pr->pr_pax_aslr_mmap_len =3D pax_aslr_mmap_len; + pr->pr_pax_aslr_stack_len =3D pax_aslr_stack_len; + pr->pr_pax_aslr_exec_len =3D pax_aslr_exec_len; + +#ifdef COMPAT_FREEBSD32 + pr->pr_pax_aslr_compat_status =3D pax_aslr_compat_status; + pr->pr_pax_aslr_compat_mmap_len =3D pax_aslr_compat_mmap_len; + pr->pr_pax_aslr_compat_stack_len =3D pax_aslr_compat_stack_len; + pr->pr_pax_aslr_compat_exec_len =3D pax_aslr_compat_exec_len; +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + + pr->pr_pax_log_log =3D pax_log_log; + pr->pr_pax_log_ulog =3D pax_log_ulog; + + pr->pr_pax_set =3D 1; + + mtx_unlock(&(pr->pr_mtx)); +} diff --git a/sys/kern/kern_pax_aslr.c b/sys/kern/kern_pax_aslr.c new file mode 100644 index 0000000..37d7da1 --- /dev/null +++ b/sys/kern/kern_pax_aslr.c @@ -0,0 +1,680 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "opt_compat.h" +#include "opt_pax.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include + +#include + +FEATURE(aslr, "Address Space Layout Randomization."); + +int pax_aslr_status =3D PAX_ASLR_OPTOUT; +int pax_aslr_debug =3D 0; + +#ifdef PAX_ASLR_MAX_SEC +int pax_aslr_mmap_len =3D PAX_ASLR_DELTA_MMAP_MAX_LEN; +int pax_aslr_stack_len =3D PAX_ASLR_DELTA_STACK_MAX_LEN; +int pax_aslr_exec_len =3D PAX_ASLR_DELTA_EXEC_MAX_LEN; +#else +int pax_aslr_mmap_len =3D PAX_ASLR_DELTA_MMAP_DEF_LEN; +int pax_aslr_stack_len =3D PAX_ASLR_DELTA_STACK_DEF_LEN; +int pax_aslr_exec_len =3D PAX_ASLR_DELTA_EXEC_DEF_LEN; +#endif /* PAX_ASLR_MAX_SEC */ + +#ifdef COMPAT_FREEBSD32 +int pax_aslr_compat_status =3D PAX_ASLR_OPTOUT; +#ifdef PAX_ASLR_MAX_SEC +int pax_aslr_compat_mmap_len =3D PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN; +int pax_aslr_compat_stack_len =3D PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN; +int pax_aslr_compat_exec_len =3D PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN; +#else +int pax_aslr_compat_mmap_len =3D PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN; +int pax_aslr_compat_stack_len =3D PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN; +int pax_aslr_compat_exec_len =3D PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN; +#endif /* PAX_ASLR_MAX_SEC */ +#endif /* COMPAT_FREEBSD32 */ + +TUNABLE_INT("security.pax.aslr.status", &pax_aslr_status); +TUNABLE_INT("security.pax.aslr.mmap_len", &pax_aslr_mmap_len); +TUNABLE_INT("security.pax.aslr.debug", &pax_aslr_debug); +TUNABLE_INT("security.pax.aslr.stack_len", &pax_aslr_stack_len); +TUNABLE_INT("security.pax.aslr.exec_len", &pax_aslr_exec_len); +#ifdef COMPAT_FREEBSD32 +TUNABLE_INT("security.pax.aslr.compat.status", &pax_aslr_compat_status); +TUNABLE_INT("security.pax.aslr.compat.mmap", &pax_aslr_compat_mmap_len); +TUNABLE_INT("security.pax.aslr.compat.stack", &pax_aslr_compat_stack_len); +TUNABLE_INT("security.pax.aslr.compat.stack", &pax_aslr_compat_exec_len); +#endif + + +#ifdef PAX_SYSCTLS +/* + * sysctls and tunables + */ +static int sysctl_pax_aslr_debug(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_status(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_mmap(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_stack(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_exec(SYSCTL_HANDLER_ARGS); + +SYSCTL_DECL(_security_pax); + +SYSCTL_NODE(_security_pax, OID_AUTO, aslr, CTLFLAG_RD, 0, + "Address Space Layout Randomization."); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, status, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_status, "I", + "Restrictions status. " + "0 - disabled, " + "1 - opt-in, " + "2 - opt-out, " + "3 - force enabled"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, debug, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_debug, "I", + "ASLR debug mode"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, mmap_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_mmap, "I", + "Number of bits randomized for mmap(2) calls. " + "32 bit: [8,16] 64 bit: [16,32]"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, stack_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_stack, "I", + "Number of bits randomized for the stack. " + "32 bit: [6,12] 64 bit: [12,21]"); + +SYSCTL_PROC(_security_pax_aslr, OID_AUTO, exec_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_aslr_exec, "I", + "Number of bits randomized for the PIE exec base. " + "32 bit: [6,12] 64 bit: [12,21]"); + +static int +sysctl_pax_aslr_status(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_status : pax_aslr_status; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + switch (val) { + case PAX_ASLR_DISABLED: + case PAX_ASLR_OPTIN: + case PAX_ASLR_OPTOUT: + case PAX_ASLR_FORCE_ENABLED: + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_status =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_status =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + break; + default: + return (EINVAL); + } + + return (0); +} + +static int +sysctl_pax_aslr_debug(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_debug : pax_aslr_debug; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_debug =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_debug =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_mmap(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_mmap_len : pax_aslr_mmap_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_DELTA_MMAP_MIN_LEN || + val > PAX_ASLR_DELTA_MMAP_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_mmap_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_mmap_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_stack(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_stack_len : pax_aslr_stack_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_DELTA_STACK_MIN_LEN || + val > PAX_ASLR_DELTA_STACK_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_stack_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_stack_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_exec(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr=3DNULL; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_exec_len : pax_aslr_exec_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + if (val < PAX_ASLR_DELTA_EXEC_MIN_LEN || + val > PAX_ASLR_DELTA_EXEC_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_exec_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_exec_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +/* + * COMPAT_FREEBSD32 and linuxulator.. + */ +#ifdef COMPAT_FREEBSD32 +static int sysctl_pax_aslr_compat_status(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_mmap(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_stack(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_aslr_compat_exec(SYSCTL_HANDLER_ARGS); + +SYSCTL_NODE(_security_pax_aslr, OID_AUTO, compat, CTLFLAG_RD, 0, + "Setting for COMPAT_FREEBSD32 and linuxulator."); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, status, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_status, "I", + "Restrictions status. " + "0 - disabled, " + "1 - enabled, " + "2 - global enabled, " + "3 - force global enabled"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, mmap_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_mmap, "I", + "Number of bits randomized for mmap(2) calls. " + "32 bit: [8,16]"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, stack_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_stack, "I", + "Number of bits randomized for the stack. " + "32 bit: [6,12]"); + +SYSCTL_PROC(_security_pax_aslr_compat, OID_AUTO, exec_len, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON, + NULL, 0, sysctl_pax_aslr_compat_exec, "I", + "Number of bits randomized for the PIE exec base. " + "32 bit: [6,12]"); + +static int +sysctl_pax_aslr_compat_status(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ?pr->pr_pax_aslr_compat_status : pax_aslr_compat_s= tatus; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || (req->newptr =3D=3D NULL)) + return (err); + + switch (val) { + case PAX_ASLR_DISABLED: + case PAX_ASLR_OPTIN: + case PAX_ASLR_OPTOUT: + case PAX_ASLR_FORCE_ENABLED: + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_status =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_status =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + break; + default: + return (EINVAL); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_mmap(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_compat_mmap_len : pax_aslr_compa= t_mmap_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_mmap_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_mmap_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_stack(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + val =3D (pr !=3D NULL) ? pr->pr_pax_aslr_compat_stack_len : pax_aslr_comp= at_stack_len; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_stack_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_stack_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +static int +sysctl_pax_aslr_compat_exec(SYSCTL_HANDLER_ARGS) +{ + struct prison *pr; + int err, val; + + pr =3D pax_get_prison(req->td, NULL); + + if (pr !=3D NULL) + val =3D pr->pr_pax_aslr_compat_exec_len; + else + val =3D pax_aslr_compat_exec_len; + + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + if (val < PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN || + val > PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN) + return (EINVAL); + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_aslr_compat_exec_len =3D val; + + if (pr !=3D NULL) { + mtx_lock(&(pr->pr_mtx)); + pr->pr_pax_aslr_compat_exec_len =3D val; + mtx_unlock(&(pr->pr_mtx)); + } + + return (0); +} + +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_SYSCTLS */ + + +/* + * ASLR functions + */ +bool +pax_aslr_active(struct thread *td, struct proc *proc) +{ + int status; + struct prison *pr; + uint32_t flags; + + if ((td =3D=3D NULL) && (proc =3D=3D NULL)) + return (true); + + pr =3D pax_get_prison(td, proc); + + flags =3D (td !=3D NULL) ? td->td_proc->p_pax : proc->p_pax; + if (((flags & 0xaaaaaaaa) & ((flags & 0x55555555) << 1)) !=3D 0) { + pax_log_aslr(pr, __func__, "inconsistent paxflags: %x\n", flags); + pax_ulog_aslr(pr, NULL, "inconsistent paxflags: %x\n", flags); + return (true); + } + + if (pr !=3D NULL) + status =3D pr->pr_pax_aslr_status; + else + status =3D pax_aslr_status; + + switch (status) { + case PAX_ASLR_DISABLED: + return (false); + case PAX_ASLR_FORCE_ENABLED: + return (true); + case PAX_ASLR_OPTIN: + if ((flags & PAX_NOTE_ASLR) =3D=3D 0) { + pax_log_aslr(pr, __func__, + "ASLR is opt-in, and executable does not have ASLR enabled\n"); + pax_ulog_aslr(pr, NULL, + "ASLR is opt-in, and executable does not have ASLR enabled\n"); + return (false); + } + break; + case PAX_ASLR_OPTOUT: + if ((flags & PAX_NOTE_NOASLR) !=3D 0) { + pax_log_aslr(pr, __func__, + "ASLR is opt-out, and executable explicitly disabled ASLR\n"); + pax_ulog_aslr(pr, NULL, + "ASLR is opt-out, and executable explicitly disabled ASLR\n"); + return (false); + } + break; + default: + return (true); + } + + return (true); +} + +void +_pax_aslr_init(struct vmspace *vm, struct prison *pr) +{ + if (vm =3D=3D NULL) + panic("[PaX ASLR] %s: vm =3D=3D NULL", __func__); + + vm->vm_aslr_delta_mmap =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_MMAP_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_mmap_len : + pax_aslr_mmap_len); + vm->vm_aslr_delta_stack =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_STACK_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_stack_len : + pax_aslr_stack_len); + vm->vm_aslr_delta_stack =3D ALIGN(vm->vm_aslr_delta_stack); + vm->vm_aslr_delta_exec =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_EXEC_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_exec_len : + pax_aslr_exec_len); + + if ((pr !=3D NULL) && pr->pr_pax_aslr_debug) { + pax_log_aslr(pr, __func__, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_log_aslr(pr, __func__, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_log_aslr(pr, __func__, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + } +} + +#ifdef COMPAT_FREEBSD32 +void +_pax_aslr_init32(struct vmspace *vm, struct prison *pr) +{ + if (vm =3D=3D NULL) + panic("[PaX ASLR] %s: vm =3D=3D NULL", __func__); + + vm->vm_aslr_delta_mmap =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_COMPAT_DELTA_MMAP_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_mmap_len : + pax_aslr_compat_mmap_len); + vm->vm_aslr_delta_stack =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_COMPAT_DELTA_STACK_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_stack_len : + pax_aslr_compat_stack_len); + vm->vm_aslr_delta_stack =3D ALIGN(vm->vm_aslr_delta_stack); + vm->vm_aslr_delta_exec =3D PAX_ASLR_DELTA(arc4random(), + PAX_ASLR_DELTA_EXEC_LSB, (pr !=3D NULL) ? + pr->pr_pax_aslr_compat_exec_len : + pax_aslr_compat_exec_len); + + if ((pr !=3D NULL) && pr->pr_pax_aslr_debug) { + pax_log_aslr(pr, __func__, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_log_aslr(pr, __func__, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_log_aslr(pr, __func__, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_mmap=3D%p\n", + (void *) vm->vm_aslr_delta_mmap); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_stack=3D%p\n", + (void *) vm->vm_aslr_delta_stack); + pax_ulog_aslr(pr, NULL, "vm_aslr_delta_exec=3D%p\n", + (void *) vm->vm_aslr_delta_exec); + } +} +#endif + +void +pax_aslr_init(struct thread *td, struct image_params *imgp) +{ + struct prison *pr; + struct vmspace *vm; + + pr =3D pax_get_prison(td, NULL); + + if (imgp =3D=3D NULL) + panic("[PaX ASLR] %s: imgp =3D=3D NULL", __func__); + + if (!pax_aslr_active(td, NULL)) + return; + + vm =3D imgp->proc->p_vmspace; + + if (imgp->sysent->sv_pax_aslr_init !=3D NULL) + imgp->sysent->sv_pax_aslr_init(vm, pr); +} + +void +pax_aslr_mmap(struct thread *td, vm_offset_t *addr, vm_offset_t orig_addr,= int flags) +{ + struct prison *pr; + + if (!pax_aslr_active(td, NULL)) + return; + + pr =3D pax_get_prison(td, NULL); + + if (!(flags & MAP_FIXED) && ((orig_addr =3D=3D 0) || !(flags & MAP_ANON))= ) { + pax_log_aslr(pr, __func__, "applying to %p orig_addr=3D%p flags=3D%x\n", + (void *)*addr, (void *)orig_addr, flags); + + *addr +=3D td->td_proc->p_vmspace->vm_aslr_delta_mmap; + pax_log_aslr(pr, __func__, "result %p\n", (void *)*addr); + } else { + pax_log_aslr(pr, __func__, "not applying to %p orig_addr=3D%p flags=3D%x= \n", + (void *)*addr, (void *)orig_addr, flags); + } +} + +void +pax_aslr_stack(struct thread *td, uintptr_t *addr) +{ + struct prison *pr; + uintptr_t orig_addr; + + if (!pax_aslr_active(td, NULL)) + return; + + pr =3D pax_get_prison(td, NULL); + + orig_addr =3D *addr; + *addr -=3D td->td_proc->p_vmspace->vm_aslr_delta_stack; + pax_log_aslr(pr, __func__, "orig_addr=3D%p, new_addr=3D%p\n", + (void *)orig_addr, (void *)*addr); + pax_ulog_aslr(pr, NULL, "orig_addr=3D%p, new_addr=3D%p\n", + (void *)orig_addr, (void *)*addr); +} diff --git a/sys/kern/kern_pax_log.c b/sys/kern/kern_pax_log.c new file mode 100644 index 0000000..943ac81 --- /dev/null +++ b/sys/kern/kern_pax_log.c @@ -0,0 +1,188 @@ +/*- + * Copyright (c) 2014, by Oliver Pinter + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#define __PAX_LOG_TEMPLATE(SUBJECT, name) \ +void \ +pax_log_##name(struct prison *pr, const char *caller_name, const char* fmt= , ...)\ +{ \ + struct sbuf *sb; \ + va_list args; \ + \ + if ((pr !=3D NULL) && (pr->pr_pax_log_log =3D=3D 0)) \ + return; \ + \ + sb =3D sbuf_new_auto(); \ + if (sb =3D=3D NULL) \ + panic("%s: Could not allocate memory", __func__); \ + sbuf_printf(sb, "[PAX "#SUBJECT"] "); \ + if (caller_name !=3D NULL) \ + sbuf_printf(sb, "%s: ", caller_name); \ + va_start(args, fmt); \ + sbuf_vprintf(sb, fmt, args); \ + va_end(args); \ + if (sbuf_finish(sb) !=3D 0) \ + panic("%s: Could not generate message", __func__); \ + \ + printf("%s", sbuf_data(sb)); \ + sbuf_delete(sb); \ +} \ + \ +void \ +pax_ulog_##name(struct prison *pr, const char *caller_name, const char* fm= t, ...)\ +{ \ + struct sbuf *sb; \ + va_list args; \ + \ + if ((pr !=3D NULL) && (pr->pr_pax_log_ulog =3D=3D 0)) \ + return; \ + \ + sb =3D sbuf_new_auto(); \ + if (sb =3D=3D NULL) \ + panic("%s: Could not allocate memory", __func__); \ + sbuf_printf(sb, "[PAX "#SUBJECT"] "); \ + if (caller_name !=3D NULL) \ + sbuf_printf(sb, "%s: ", caller_name); \ + va_start(args, fmt); \ + sbuf_vprintf(sb, fmt, args); \ + va_end(args); \ + if (sbuf_finish(sb) !=3D 0) \ + panic("%s: Could not generate message", __func__); \ + \ + uprintf("%s", sbuf_data(sb)); \ + sbuf_delete(sb); \ +} + + +static int sysctl_pax_log_log(SYSCTL_HANDLER_ARGS); +static int sysctl_pax_log_ulog(SYSCTL_HANDLER_ARGS); + +int pax_log_log =3D PAX_LOG_LOG; +int pax_log_ulog =3D PAX_LOG_ULOG; + +SYSCTL_DECL(_security_pax); + +SYSCTL_NODE(_security_pax, OID_AUTO, log, CTLFLAG_RD, 0, + "PAX related logging facility."); + +SYSCTL_PROC(_security_pax_log, OID_AUTO, log, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_log_log, "I", + "log to syslog " + "0 - disabled, " + "1 - enabled "); +TUNABLE_INT("security.pax.log.log", &pax_log_log); + +SYSCTL_PROC(_security_pax_log, OID_AUTO, ulog, + CTLTYPE_INT|CTLFLAG_RWTUN|CTLFLAG_PRISON|CTLFLAG_SECURE, + NULL, 0, sysctl_pax_log_ulog, "I", + "log to user terminal" + "0 - disabled, " + "1 - enabled "); +TUNABLE_INT("security.pax.log.ulog", &pax_log_ulog); + +static int +sysctl_pax_log_log(SYSCTL_HANDLER_ARGS) +{ + int err; + int val; + struct prison *pr=3DNULL; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_log_log : pax_log_log; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_log_log =3D val; + if (pr !=3D NULL) + pr->pr_pax_log_log =3D val; + + return (0); +} + +static int +sysctl_pax_log_ulog(SYSCTL_HANDLER_ARGS) +{ + int err; + int val; + struct prison *pr=3DNULL; + + pr =3D pax_get_prison(req->td, NULL); + + if ((pr !=3D NULL) && !(pr->pr_pax_set)) + pax_init_prison(pr); + + val =3D (pr !=3D NULL) ? pr->pr_pax_log_ulog : pax_log_ulog; + err =3D sysctl_handle_int(oidp, &val, sizeof(int), req); + if (err || !req->newptr) + return (err); + + switch (val) { + case 0: + case 1: + break; + default: + return (EINVAL); + + } + + if ((pr =3D=3D NULL) || (pr =3D=3D &prison0)) + pax_log_ulog =3D val; + if (pr !=3D NULL) + pr->pr_pax_log_ulog =3D val; + + return (0); +} + + +__PAX_LOG_TEMPLATE(ASLR, aslr) diff --git a/sys/mips/mips/elf_machdep.c b/sys/mips/mips/elf_machdep.c index d374713..f95ba35 100644 --- a/sys/mips/mips/elf_machdep.c +++ b/sys/mips/mips/elf_machdep.c @@ -28,6 +28,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -49,6 +51,10 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#ifdef PAX_ASLR +#include +#endif + #ifdef __mips_n64 struct sysentvec elf64_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, @@ -83,6 +89,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf64_Brandinfo freebsd_brand_info =3D { @@ -139,6 +150,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf32_Brandinfo freebsd_brand_info =3D { diff --git a/sys/mips/mips/freebsd32_machdep.c b/sys/mips/mips/freebsd32_ma= chdep.c index dfdf70f..103ad84 100644 --- a/sys/mips/mips/freebsd32_machdep.c +++ b/sys/mips/mips/freebsd32_machdep.c @@ -31,6 +31,7 @@ */ =20 #include "opt_compat.h" +#include "opt_pax.h" =20 #define __ELF_WORD_SIZE 32 =20 @@ -66,6 +67,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + static void freebsd32_exec_setregs(struct thread *, struct image_params *,= u_long); static int get_mcontext32(struct thread *, mcontext32_t *, int); static int set_mcontext32(struct thread *, const mcontext32_t *); @@ -106,6 +111,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D freebsd32_syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/powerpc/powerpc/elf32_machdep.c b/sys/powerpc/powerpc/elf3= 2_machdep.c index dbe58df..229fe97 100644 --- a/sys/powerpc/powerpc/elf32_machdep.c +++ b/sys/powerpc/powerpc/elf32_machdep.c @@ -25,6 +25,8 @@ * $FreeBSD$ */ =20 +#include "opt_pax.h" + #include #include #include @@ -52,6 +54,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + #ifdef __powerpc64__ #include #include @@ -107,6 +113,11 @@ struct sysentvec elf32_freebsd_sysvec =3D { .sv_shared_page_base =3D FREEBSD32_SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init32, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf32_sysvec, &elf32_freebsd_sysvec); =20 diff --git a/sys/powerpc/powerpc/elf64_machdep.c b/sys/powerpc/powerpc/elf6= 4_machdep.c index 0c41a8d..095f37b0 100644 --- a/sys/powerpc/powerpc/elf64_machdep.c +++ b/sys/powerpc/powerpc/elf64_machdep.c @@ -25,6 +25,8 @@ * $FreeBSD$ */ =20 +#include "opt_pax.h" + #include #include #include @@ -48,6 +50,10 @@ #include #include =20 +#ifdef PAX_ASLR +#include +#endif + struct sysentvec elf64_freebsd_sysvec =3D { .sv_size =3D SYS_MAXSYSCALL, .sv_table =3D sysent, @@ -83,6 +89,11 @@ struct sysentvec elf64_freebsd_sysvec =3D { .sv_shared_page_base =3D SHAREDPAGE, .sv_shared_page_len =3D PAGE_SIZE, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; INIT_SYSENTVEC(elf64_sysvec, &elf64_freebsd_sysvec); =20 diff --git a/sys/security/mac_bsdextended/mac_bsdextended.c b/sys/security/= mac_bsdextended/mac_bsdextended.c index 377fd25..f6b6173 100644 --- a/sys/security/mac_bsdextended/mac_bsdextended.c +++ b/sys/security/mac_bsdextended/mac_bsdextended.c @@ -47,6 +47,8 @@ * firewall-like rules regarding users and file system objects. */ =20 +#include "opt_pax.h" + #include #include #include @@ -56,14 +58,20 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include =20 +#ifdef PAX_ASLR +#include +#endif + #include #include #include @@ -116,7 +124,6 @@ SYSCTL_INT(_security_mac_bsdextended, OID_AUTO, firstma= tch_enabled, static int ugidfw_rule_valid(struct mac_bsdextended_rule *rule) { - if ((rule->mbr_subject.mbs_flags | MBS_ALL_FLAGS) !=3D MBS_ALL_FLAGS) return (EINVAL); if ((rule->mbr_subject.mbs_neg | MBS_ALL_FLAGS) !=3D MBS_ALL_FLAGS) @@ -128,8 +135,13 @@ ugidfw_rule_valid(struct mac_bsdextended_rule *rule) if ((rule->mbr_object.mbo_neg | MBO_TYPE_DEFINED) && (rule->mbr_object.mbo_type | MBO_ALL_TYPE) !=3D MBO_ALL_TYPE) return (EINVAL); +#ifdef PAX_ASLR + if ((rule->mbr_pax | MBI_ALLPAX) !=3D MBI_ALLPAX) + return (EINVAL); +#endif if ((rule->mbr_mode | MBI_ALLPERM) !=3D MBI_ALLPERM) return (EINVAL); + return (0); } =20 @@ -226,7 +238,7 @@ ugidfw_destroy(struct mac_policy_conf *mpc) =20 static int ugidfw_rulecheck(struct mac_bsdextended_rule *rule, - struct ucred *cred, struct vnode *vp, struct vattr *vap, int acc_mode) + struct ucred *cred, struct vnode *vp, struct vattr *vap, int acc_mode,= struct image_params *imgp) { int mac_granted, match, priv_granted; int i; @@ -304,6 +316,10 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, match =3D (bcmp(&(vp->v_mount->mnt_stat.f_fsid), &(rule->mbr_object.mbo_fsid), sizeof(rule->mbr_object.mbo_fsid)) =3D=3D 0); +#if defined(PAX_ASLR) + if (match && rule->mbr_object.mbo_inode) + match =3D (vap->va_fileid =3D=3D rule->mbr_object.mbo_inode); +#endif if (rule->mbr_object.mbo_neg & MBO_FSID_DEFINED) match =3D !match; if (!match) @@ -412,6 +428,11 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, return (EACCES); } =20 +#ifdef PAX_ASLR + if (imgp !=3D NULL) + pax_elf(imgp, rule->mbr_pax); +#endif + /* * If the rule matched, permits access, and first match is enabled, * return success. @@ -424,7 +445,7 @@ ugidfw_rulecheck(struct mac_bsdextended_rule *rule, =20 int ugidfw_check(struct ucred *cred, struct vnode *vp, struct vattr *vap, - int acc_mode) + int acc_mode, struct image_params *imgp) { int error, i; =20 @@ -440,7 +461,7 @@ ugidfw_check(struct ucred *cred, struct vnode *vp, stru= ct vattr *vap, if (rules[i] =3D=3D NULL) continue; error =3D ugidfw_rulecheck(rules[i], cred, - vp, vap, acc_mode); + vp, vap, acc_mode, imgp); if (error =3D=3D EJUSTRETURN) break; if (error) { @@ -453,7 +474,7 @@ ugidfw_check(struct ucred *cred, struct vnode *vp, stru= ct vattr *vap, } =20 int -ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode) +ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode, struct= image_params *imgp) { int error; struct vattr vap; @@ -463,7 +484,7 @@ ugidfw_check_vp(struct ucred *cred, struct vnode *vp, i= nt acc_mode) error =3D VOP_GETATTR(vp, &vap, cred); if (error) return (error); - return (ugidfw_check(cred, vp, &vap, acc_mode)); + return (ugidfw_check(cred, vp, &vap, acc_mode, imgp)); } =20 int diff --git a/sys/security/mac_bsdextended/mac_bsdextended.h b/sys/security/= mac_bsdextended/mac_bsdextended.h index c09abc0..c3cbf28 100644 --- a/sys/security/mac_bsdextended/mac_bsdextended.h +++ b/sys/security/mac_bsdextended/mac_bsdextended.h @@ -51,6 +51,9 @@ #define MBI_ADMIN 010000 #define MBI_STAT 020000 #define MBI_APPEND 040000 +#define MBI_FORCE_ASLR_ENABLED 0x01 +#define MBI_FORCE_ASLR_DISABLED 0x02 +#define MBI_ALLPAX (MBI_FORCE_ASLR_ENABLED | MBI_FORCE_ASLR_DISABLED) #define MBI_ALLPERM (MBI_EXEC | MBI_WRITE | MBI_READ | MBI_ADMIN | \ MBI_STAT | MBI_APPEND) =20 @@ -78,6 +81,7 @@ struct mac_bsdextended_subject { #define MBO_UID_SUBJECT 0x00000020 /* uid must match subject */ #define MBO_GID_SUBJECT 0x00000040 /* gid must match subject */ #define MBO_TYPE_DEFINED 0x00000080 /* object type should be matched */ +#define MBO_PAXPATH_DEFINED 0x00000100 /* TODO: paxpath should be matched = */ =20 #define MBO_ALL_FLAGS (MBO_UID_DEFINED | MBO_GID_DEFINED | MBO_FSID_DEFINE= D | \ MBO_SUID | MBO_SGID | MBO_UID_SUBJECT | MBO_GID_SUBJECT | \ @@ -103,12 +107,15 @@ struct mac_bsdextended_object { gid_t mbo_gid_max; struct fsid mbo_fsid; int mbo_type; + ino_t mbo_inode; + char mbo_paxpath[MAXPATHLEN]; }; =20 struct mac_bsdextended_rule { struct mac_bsdextended_subject mbr_subject; struct mac_bsdextended_object mbr_object; mode_t mbr_mode; /* maximum access */ + uint32_t mbr_pax; }; =20 #endif /* _SYS_SECURITY_MAC_BSDEXTENDED_H */ diff --git a/sys/security/mac_bsdextended/ugidfw_internal.h b/sys/security/= mac_bsdextended/ugidfw_internal.h index 5597fd1..18c74dc 100644 --- a/sys/security/mac_bsdextended/ugidfw_internal.h +++ b/sys/security/mac_bsdextended/ugidfw_internal.h @@ -36,8 +36,9 @@ */ int ugidfw_accmode2mbi(accmode_t accmode); int ugidfw_check(struct ucred *cred, struct vnode *vp, struct vattr *vap, - int acc_mode); -int ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode); + int acc_mode, struct image_params *imgp); +int ugidfw_check_vp(struct ucred *cred, struct vnode *vp, int acc_mode, + struct image_params *imgp); =20 /* * System access control checks. diff --git a/sys/security/mac_bsdextended/ugidfw_system.c b/sys/security/ma= c_bsdextended/ugidfw_system.c index 49e4f1d..2829a00 100644 --- a/sys/security/mac_bsdextended/ugidfw_system.c +++ b/sys/security/mac_bsdextended/ugidfw_system.c @@ -66,7 +66,7 @@ ugidfw_system_check_acct(struct ucred *cred, struct vnode= *vp, { =20 if (vp !=3D NULL) - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); else return (0); } @@ -77,7 +77,7 @@ ugidfw_system_check_auditctl(struct ucred *cred, struct v= node *vp, { =20 if (vp !=3D NULL) - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); else return (0); } @@ -87,5 +87,5 @@ ugidfw_system_check_swapon(struct ucred *cred, struct vno= de *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } diff --git a/sys/security/mac_bsdextended/ugidfw_vnode.c b/sys/security/mac= _bsdextended/ugidfw_vnode.c index 8ec2d48..2065e6e 100644 --- a/sys/security/mac_bsdextended/ugidfw_vnode.c +++ b/sys/security/mac_bsdextended/ugidfw_vnode.c @@ -65,7 +65,7 @@ ugidfw_vnode_check_access(struct ucred *cred, struct vnod= e *vp, struct label *vplabel, accmode_t accmode) { =20 - return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode))); + return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode), NULL)); } =20 int @@ -73,7 +73,7 @@ ugidfw_vnode_check_chdir(struct ucred *cred, struct vnode= *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -81,7 +81,7 @@ ugidfw_vnode_check_chroot(struct ucred *cred, struct vnod= e *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -89,7 +89,7 @@ ugidfw_check_create_vnode(struct ucred *cred, struct vnod= e *dvp, struct label *dvplabel, struct componentname *cnp, struct vattr *vap) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_WRITE)); + return (ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL)); } =20 int @@ -97,7 +97,7 @@ ugidfw_vnode_check_deleteacl(struct ucred *cred, struct v= node *vp, struct label *vplabel, acl_type_t type) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -105,7 +105,7 @@ ugidfw_vnode_check_deleteextattr(struct ucred *cred, st= ruct vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -114,7 +114,7 @@ ugidfw_vnode_check_exec(struct ucred *cred, struct vnod= e *vp, struct label *execlabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ|MBI_EXEC)); + return (ugidfw_check_vp(cred, vp, MBI_READ|MBI_EXEC, imgp)); } =20 int @@ -122,7 +122,7 @@ ugidfw_vnode_check_getacl(struct ucred *cred, struct vn= ode *vp, struct label *vplabel, acl_type_t type) { =20 - return (ugidfw_check_vp(cred, vp, MBI_STAT)); + return (ugidfw_check_vp(cred, vp, MBI_STAT, NULL)); } =20 int @@ -130,7 +130,7 @@ ugidfw_vnode_check_getextattr(struct ucred *cred, struc= t vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -140,10 +140,10 @@ ugidfw_vnode_check_link(struct ucred *cred, struct vn= ode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - error =3D ugidfw_check_vp(cred, vp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, vp, MBI_WRITE, NULL); if (error) return (error); return (0); @@ -154,7 +154,7 @@ ugidfw_vnode_check_listextattr(struct ucred *cred, stru= ct vnode *vp, struct label *vplabel, int attrnamespace) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -162,7 +162,7 @@ ugidfw_vnode_check_lookup(struct ucred *cred, struct vn= ode *dvp, struct label *dvplabel, struct componentname *cnp) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_EXEC)); + return (ugidfw_check_vp(cred, dvp, MBI_EXEC, NULL)); } =20 int @@ -170,7 +170,7 @@ ugidfw_vnode_check_open(struct ucred *cred, struct vnod= e *vp, struct label *vplabel, accmode_t accmode) { =20 - return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode))); + return (ugidfw_check_vp(cred, vp, ugidfw_accmode2mbi(accmode), NULL)); } =20 int @@ -178,7 +178,7 @@ ugidfw_vnode_check_readdir(struct ucred *cred, struct v= node *dvp, struct label *dvplabel) { =20 - return (ugidfw_check_vp(cred, dvp, MBI_READ)); + return (ugidfw_check_vp(cred, dvp, MBI_READ, NULL)); } =20 int @@ -186,7 +186,7 @@ ugidfw_vnode_check_readdlink(struct ucred *cred, struct= vnode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_READ)); + return (ugidfw_check_vp(cred, vp, MBI_READ, NULL)); } =20 int @@ -196,10 +196,10 @@ ugidfw_vnode_check_rename_from(struct ucred *cred, st= ruct vnode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -209,11 +209,11 @@ ugidfw_vnode_check_rename_to(struct ucred *cred, stru= ct vnode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); if (vp !=3D NULL) - error =3D ugidfw_check_vp(cred, vp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, vp, MBI_WRITE, NULL); return (error); } =20 @@ -222,7 +222,7 @@ ugidfw_vnode_check_revoke(struct ucred *cred, struct vn= ode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -230,7 +230,7 @@ ugidfw_check_setacl_vnode(struct ucred *cred, struct vn= ode *vp, struct label *vplabel, acl_type_t type, struct acl *acl) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -238,7 +238,7 @@ ugidfw_vnode_check_setextattr(struct ucred *cred, struc= t vnode *vp, struct label *vplabel, int attrnamespace, const char *name) { =20 - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } =20 int @@ -246,7 +246,7 @@ ugidfw_vnode_check_setflags(struct ucred *cred, struct = vnode *vp, struct label *vplabel, u_long flags) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -254,7 +254,7 @@ ugidfw_vnode_check_setmode(struct ucred *cred, struct v= node *vp, struct label *vplabel, mode_t mode) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -262,7 +262,7 @@ ugidfw_vnode_check_setowner(struct ucred *cred, struct = vnode *vp, struct label *vplabel, uid_t uid, gid_t gid) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -270,7 +270,7 @@ ugidfw_vnode_check_setutimes(struct ucred *cred, struct= vnode *vp, struct label *vplabel, struct timespec atime, struct timespec utime) { =20 - return (ugidfw_check_vp(cred, vp, MBI_ADMIN)); + return (ugidfw_check_vp(cred, vp, MBI_ADMIN, NULL)); } =20 int @@ -278,7 +278,7 @@ ugidfw_vnode_check_stat(struct ucred *active_cred, struct ucred *file_cred, struct vnode *vp, struct label *vplabel) { =20 - return (ugidfw_check_vp(active_cred, vp, MBI_STAT)); + return (ugidfw_check_vp(active_cred, vp, MBI_STAT, NULL)); } =20 int @@ -288,8 +288,8 @@ ugidfw_vnode_check_unlink(struct ucred *cred, struct vn= ode *dvp, { int error; =20 - error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE); + error =3D ugidfw_check_vp(cred, dvp, MBI_WRITE, NULL); if (error) return (error); - return (ugidfw_check_vp(cred, vp, MBI_WRITE)); + return (ugidfw_check_vp(cred, vp, MBI_WRITE, NULL)); } diff --git a/sys/sparc64/sparc64/elf_machdep.c b/sys/sparc64/sparc64/elf_ma= chdep.c index 4d55717..e0eba33 100644 --- a/sys/sparc64/sparc64/elf_machdep.c +++ b/sys/sparc64/sparc64/elf_machdep.c @@ -34,6 +34,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -52,6 +54,10 @@ __FBSDID("$FreeBSD$"); =20 #include =20 +#ifdef PAX_ASLR +#include +#endif + #include "linker_if.h" =20 static struct sysentvec elf64_freebsd_sysvec =3D { @@ -87,6 +93,11 @@ static struct sysentvec elf64_freebsd_sysvec =3D { .sv_fetch_syscall_args =3D cpu_fetch_syscall_args, .sv_syscallnames =3D syscallnames, .sv_schedtail =3D NULL, +#ifdef PAX_ASLR + .sv_pax_aslr_init =3D _pax_aslr_init, +#else + .sv_pax_aslr_init =3D NULL, +#endif }; =20 static Elf64_Brandinfo freebsd_brand_info =3D { diff --git a/sys/sys/imgact.h b/sys/sys/imgact.h index 17cfcc2..15c2c4f 100644 --- a/sys/sys/imgact.h +++ b/sys/sys/imgact.h @@ -78,6 +78,7 @@ struct image_params { unsigned long pagesizes; int pagesizeslen; vm_prot_t stack_prot; + int pax_flags; }; =20 #ifdef _KERNEL diff --git a/sys/sys/jail.h b/sys/sys/jail.h index 59d791c..699b21c 100644 --- a/sys/sys/jail.h +++ b/sys/sys/jail.h @@ -184,6 +184,19 @@ struct prison { char pr_hostname[MAXHOSTNAMELEN]; /* (p) jail hostname */ char pr_domainname[MAXHOSTNAMELEN]; /* (p) jail domainname */ char pr_hostuuid[HOSTUUIDLEN]; /* (p) jail hostuuid */ + /* Lock only needed for pax_* if pr_pax_set =3D=3D 0 */ + int pr_pax_set; /* (p) PaX settings initialized */ + int pr_pax_aslr_status; /* (p) PaX ASLR enabled */ + int pr_pax_aslr_debug; /* (p) PaX ASLR debug */ + int pr_pax_aslr_mmap_len; /* (p) Number of bits randomized with mmap */ + int pr_pax_aslr_stack_len; /* (p) Number of bits randomized with stack= */ + int pr_pax_aslr_exec_len; /* (p) Number of bits randomized with the ex= ecbase */ + int pr_pax_aslr_compat_status; /* (p) PaX ASLR enabled (compat32) */ + int pr_pax_aslr_compat_mmap_len; /* (p) Number of bits randomized with = mmap (compat32) */ + int pr_pax_aslr_compat_stack_len; /* (p) Number of bits randomized with= stack (compat32) */ + int pr_pax_aslr_compat_exec_len; /* (p) Number of bits randomized with = the execbase (compat32) */ + int pr_pax_log_log; /* (p) XXX */ + int pr_pax_log_ulog; /* (p) XXX */ }; =20 struct prison_racct { diff --git a/sys/sys/kernel.h b/sys/sys/kernel.h index 3c5258a..aedb52e 100644 --- a/sys/sys/kernel.h +++ b/sys/sys/kernel.h @@ -102,6 +102,7 @@ enum sysinit_sub_id { SI_SUB_WITNESS =3D 0x1A80000, /* witness initialization */ SI_SUB_MTX_POOL_DYNAMIC =3D 0x1AC0000, /* dynamic mutex pool */ SI_SUB_LOCK =3D 0x1B00000, /* various locks */ + SI_SUB_PAX =3D 0x1B50000, /* pax setup */ SI_SUB_EVENTHANDLER =3D 0x1C00000, /* eventhandler init */ SI_SUB_VNET_PRELINK =3D 0x1E00000, /* vnet init before modules */ SI_SUB_KLD =3D 0x2000000, /* KLD and module setup */ diff --git a/sys/sys/pax.h b/sys/sys/pax.h new file mode 100644 index 0000000..a0f2bf6 --- /dev/null +++ b/sys/sys/pax.h @@ -0,0 +1,226 @@ +/*- + * Copyright (c) 2006 Elad Efrat + * Copyright (c) 2013-2014, by Oliver Pinter + * Copyright (c) 2014, by Shawn Webb + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTI= ES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF US= E, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef __SYS_PAX_H +#define __SYS_PAX_H + +struct image_params; +struct prison; +struct thread; +struct vnode; +struct vmspace; +struct vm_offset_t; + +/* + * used in sysctl handler + */ +#define PAX_ASLR_DISABLED 0 +#define PAX_ASLR_OPTIN 1 +#define PAX_ASLR_OPTOUT 2 +#define PAX_ASLR_FORCE_ENABLED 3 + +#ifndef PAX_ASLR_DELTA +#define PAX_ASLR_DELTA(delta, lsb, len) \ + (((delta) & ((1UL << (len)) - 1)) << (lsb)) +#endif /* PAX_ASLR_DELTA */ + +#ifdef PAX_ASLR +/* + * generic ASLR values + * + * MMAP | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 8 bit | 16 bit | + * +-------+--------+--------+ + * | DEF | 8 bit | 21 bit | + * +-------+--------+--------+ + * | MAX | 16 bit | 32 bit | + * +-------+--------+--------+ + * + * STACK | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 6 bit | 12 bit | + * +-------+--------+--------+ + * | DEF | 6 bit | 16 bit | + * +-------+--------+--------+ + * | MAX | 10 bit | 21 bit | + * +-------+--------+--------+ + * + * EXEC | 32 bit | 64 bit | + * +-------+--------+--------+ + * | MIN | 6 bit | 12 bit | + * +-------+--------+--------+ + * | DEF | 6 bit | 21 bit | + * +-------+--------+--------+ + * | MAX | 10 bit | 21 bit | + * +-------+--------+--------+ + * + */ +#ifndef PAX_ASLR_DELTA_MMAP_LSB +#define PAX_ASLR_DELTA_MMAP_LSB PAGE_SHIFT +#endif /* PAX_ASLR_DELTA_MMAP_LSB */ + +#ifndef PAX_ASLR_DELTA_MMAP_MIN_LEN +#define PAX_ASLR_DELTA_MMAP_MIN_LEN ((sizeof(void *) * NBBY) / 4) +#endif /* PAX_ASLR_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_MMAP_MAX_LEN +#define PAX_ASLR_DELTA_MMAP_MAX_LEN ((sizeof(void *) * NBBY) / 2) +#endif /* PAX_ASLR_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_STACK_LSB +#define PAX_ASLR_DELTA_STACK_LSB 3 +#endif /* PAX_ASLR_DELTA_STACK_LSB */ + +#ifndef PAX_ASLR_DELTA_STACK_MIN_LEN +#define PAX_ASLR_DELTA_STACK_MIN_LEN ((sizeof(void *) * NBBY) / 5) +#endif /* PAX_ASLR_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_STACK_MAX_LEN +#define PAX_ASLR_DELTA_STACK_MAX_LEN ((sizeof(void *) * NBBY) / 3) +#endif /* PAX_ASLR_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_EXEC_LSB +#define PAX_ASLR_DELTA_EXEC_LSB PAGE_SHIFT +#endif /* PAX_ASLR_DELTA_EXEC_LSB */ + +#ifndef PAX_ASLR_DELTA_EXEC_MIN_LEN +#define PAX_ASLR_DELTA_EXEC_MIN_LEN ((sizeof(void *) * NBBY) / 5) +#endif /* PAX_ASLR_DELTA_EXEC_MAX_LEN */ + +#ifndef PAX_ASLR_DELTA_EXEC_MAX_LEN +#define PAX_ASLR_DELTA_EXEC_MAX_LEN ((sizeof(void *) * NBBY) / 3) +#endif /* PAX_ASLR_DELTA_EXEC_MAX_LEN */ + +/* + * ASLR default values for native host + */ +#ifdef __amd64__ +#ifndef PAX_ASLR_DELTA_MMAP_DEF_LEN +#define PAX_ASLR_DELTA_MMAP_DEF_LEN 21 +#endif /* PAX_ASLR_DELTA_MMAP_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_STACK_DEF_LEN +#define PAX_ASLR_DELTA_STACK_DEF_LEN 16 +#endif /* PAX_ASLR_DELTA_STACK_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_EXEC_DEF_LEN +#define PAX_ASLR_DELTA_EXEC_DEF_LEN 21 +#endif /* PAX_ASLR_DELTA_EXEC_DEF_LEN */ +#else +#ifndef PAX_ASLR_DELTA_MMAP_DEF_LEN +#define PAX_ASLR_DELTA_MMAP_DEF_LEN PAX_ASLR_DELTA_MMAP_MIN_LEN +#endif /* PAX_ASLR_DELTA_MMAP_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_STACK_DEF_LEN +#define PAX_ASLR_DELTA_STACK_DEF_LEN PAX_ASLR_DELTA_STACK_MIN_LEN +#endif /* PAX_ASLR_DELTA_STACK_DEF_LEN */ +#ifndef PAX_ASLR_DELTA_EXEC_DEF_LEN +#define PAX_ASLR_DELTA_EXEC_DEF_LEN PAX_ASLR_DELTA_EXEC_MIN_LEN +#endif /* PAX_ASLR_DELTA_EXEC_DEF_LEN */ +#endif /* __amd64__ */ + +/* + * ASLR values for COMPAT_FREEBSD32 and COMPAT_LINUX + */ +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_LSB +#define PAX_ASLR_COMPAT_DELTA_MMAP_LSB PAGE_SHIFT +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_LSB */ + +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_MMAP_MIN_LEN ((sizeof(int) * NBBY) / 4) +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN ((sizeof(int) * NBBY) / 2) +#endif /* PAX_ASLR_COMPAT_DELTA_MMAP_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_LSB +#define PAX_ASLR_COMPAT_DELTA_STACK_LSB 3 +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_LSB */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_STACK_MIN_LEN ((sizeof(int) * NBBY) / 5) +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN ((sizeof(int) * NBBY) / 3) +#endif /* PAX_ASLR_COMPAT_DELTA_STACK_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN +#define PAX_ASLR_COMPAT_DELTA_EXEC_MIN_LEN ((sizeof(int) * NBBY) / 5) +#endif /* PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN */ + +#ifndef PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN +#define PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN ((sizeof(int) * NBBY) / 3) +#endif /* PAX_ASLR_COMPAT_DELTA_EXEC_MAX_LEN */ + +extern int pax_aslr_status; +extern int pax_aslr_debug; + +extern int pax_aslr_mmap_len; +extern int pax_aslr_stack_len; +extern int pax_aslr_exec_len; +#ifdef COMPAT_FREEBSD32 +extern int pax_aslr_compat_status; +extern int pax_aslr_compat_mmap_len; +extern int pax_aslr_compat_stack_len; +extern int pax_aslr_compat_exec_len; +#endif /* COMPAT_FREEBSD32 */ +#endif /* PAX_ASLR */ + +extern int pax_log_log; +extern int pax_log_ulog; + +#define ELF_NOTE_TYPE_PAX_TAG 3 +#define PAX_NOTE_MPROTECT 0x01 +#define PAX_NOTE_NOMPROTECT 0x02 +#define PAX_NOTE_GUARD 0x04 +#define PAX_NOTE_NOGUARD 0x08 +#define PAX_NOTE_ASLR 0x10 +#define PAX_NOTE_NOASLR 0x20 + +#define PAX_LOG_LOG 0 +#define PAX_LOG_ULOG 0 + +void pax_init(void); +void pax_init_prison(struct prison *pr); +bool pax_aslr_active(struct thread *td, struct proc *proc); +void _pax_aslr_init(struct vmspace *vm, struct prison *pr); +void _pax_aslr_init32(struct vmspace *vm, struct prison *pr); +void pax_aslr_init(struct thread *td, struct image_params *imgp); +void pax_aslr_mmap(struct thread *td, vm_offset_t *addr, + vm_offset_t orig_addr, int flags); +void pax_aslr_stack(struct thread *td, uintptr_t *addr); +struct prison *pax_get_prison(struct thread *td, struct proc *proc); +void pax_elf(struct image_params *, uint32_t); + +void pax_log_aslr(struct prison *pr, const char *func, const char *fmt, ..= =2E); +void pax_ulog_aslr(struct prison *pr, const char *func, const char *fmt, .= =2E.); + +#endif /* __SYS_PAX_H */ diff --git a/sys/sys/proc.h b/sys/sys/proc.h index fbd064c..558d7bf 100644 --- a/sys/sys/proc.h +++ b/sys/sys/proc.h @@ -539,6 +539,7 @@ struct proc { u_int p_stops; /* (c) Stop event bitmask. */ u_int p_stype; /* (c) Stop event type. */ char p_step; /* (c) Process is stopped. */ + u_int p_pax; /* (b) PaX is enabled to this process */ u_char p_pfsflags; /* (c) Procfs flags. */ struct nlminfo *p_nlminfo; /* (?) Only used by/for lockd. */ struct kaioinfo *p_aioinfo; /* (y) ASYNC I/O info. */ diff --git a/sys/sys/sysent.h b/sys/sys/sysent.h index 0f1c256..83585b8 100644 --- a/sys/sys/sysent.h +++ b/sys/sys/sysent.h @@ -77,9 +77,11 @@ struct sysent { /* system call table */ #define SY_THR_INCR 0x8 =20 struct image_params; +struct prison; struct __sigset; struct syscall_args; struct trapframe; +struct vmspace; struct vnode; =20 struct sysentvec { @@ -130,6 +132,7 @@ struct sysentvec { uint32_t sv_timekeep_gen; void *sv_shared_page_obj; void (*sv_schedtail)(struct thread *); + void (*sv_pax_aslr_init)(struct vmspace *vm, struct prison *pr); }; =20 #define SV_ILP32 0x000100 diff --git a/sys/vm/vm_map.c b/sys/vm/vm_map.c index b5108e2..e39bbd0 100644 --- a/sys/vm/vm_map.c +++ b/sys/vm/vm_map.c @@ -65,6 +65,8 @@ #include __FBSDID("$FreeBSD$"); =20 +#include "opt_pax.h" + #include #include #include @@ -292,6 +294,12 @@ vmspace_alloc(vm_offset_t min, vm_offset_t max, pmap_p= init_t pinit) vm->vm_taddr =3D 0; vm->vm_daddr =3D 0; vm->vm_maxsaddr =3D 0; +#ifdef PAX_ASLR + vm->vm_aslr_delta_mmap =3D 0; + vm->vm_aslr_delta_stack =3D 0; + vm->vm_aslr_delta_exec =3D 0; +#endif + return (vm); } =20 diff --git a/sys/vm/vm_map.h b/sys/vm/vm_map.h index 8cced05..e8e9ffe 100644 --- a/sys/vm/vm_map.h +++ b/sys/vm/vm_map.h @@ -241,6 +241,9 @@ struct vmspace { caddr_t vm_taddr; /* (c) user virtual address of text */ caddr_t vm_daddr; /* (c) user virtual address of data */ caddr_t vm_maxsaddr; /* user VA at max stack growth */ + vm_size_t vm_aslr_delta_mmap; /* mmap() random delta for ASLR */ + vm_size_t vm_aslr_delta_stack; /* stack random delta for ASLR */ + vm_size_t vm_aslr_delta_exec; /* exec base random delta for ASLR */ volatile int vm_refcnt; /* number of references */ /* * Keep the PMAP last, so that CPU-specific variations of that diff --git a/sys/vm/vm_mmap.c b/sys/vm/vm_mmap.c index 1ae7189..c2b6a4d 100644 --- a/sys/vm/vm_mmap.c +++ b/sys/vm/vm_mmap.c @@ -45,6 +45,7 @@ __FBSDID("$FreeBSD$"); =20 #include "opt_compat.h" #include "opt_hwpmc_hooks.h" +#include "opt_pax.h" =20 #include #include @@ -91,6 +92,10 @@ __FBSDID("$FreeBSD$"); #include #endif =20 +#ifdef PAX_ASLR +#include +#endif + int old_mlock =3D 0; SYSCTL_INT(_vm, OID_AUTO, old_mlock, CTLFLAG_RWTUN, &old_mlock, 0, "Do not apply RLIMIT_MEMLOCK on mlockall"); @@ -202,6 +207,9 @@ sys_mmap(td, uap) struct file *fp; struct vnode *vp; vm_offset_t addr; +#ifdef PAX_ASLR + vm_offset_t orig_addr; +#endif vm_size_t size, pageoff; vm_prot_t cap_maxprot, prot, maxprot; void *handle; @@ -212,6 +220,9 @@ sys_mmap(td, uap) cap_rights_t rights; =20 addr =3D (vm_offset_t) uap->addr; +#ifdef PAX_ASLR + orig_addr =3D addr; +#endif size =3D uap->len; prot =3D uap->prot & VM_PROT_ALL; flags =3D uap->flags; @@ -294,6 +305,12 @@ sys_mmap(td, uap) * do not bother moving the mapping past the heap (since * the heap is usually above 2GB). */ +#ifdef PAX_ASLR + /* Ugly hack for adding ASLR to 32bit mappings */ + pax_aslr_mmap(td, &addr, orig_addr, flags); + if (addr !=3D orig_addr) + addr =3D trunc_page(addr&0x0fffffff); +#endif if (addr + size > MAP_32BIT_MAX_ADDR) addr =3D 0; #endif @@ -307,6 +324,9 @@ sys_mmap(td, uap) * location. */ PROC_LOCK(td->td_proc); +#ifdef PAX_ASLR + pax_aslr_mmap(td, &addr, orig_addr, flags); +#endif if (addr =3D=3D 0 || (addr >=3D round_page((vm_offset_t)vms->vm_taddr) && addr < round_page((vm_offset_t)vms->vm_daddr + diff --git a/tools/build/options/WITHOUT_PIE b/tools/build/options/WITHOUT_= PIE new file mode 100644 index 0000000..82019ce --- /dev/null +++ b/tools/build/options/WITHOUT_PIE @@ -0,0 +1 @@ +Enable building of Position-Independent Executables (PIEs). diff --git a/usr.sbin/ugidfw/ugidfw.c b/usr.sbin/ugidfw/ugidfw.c index 977922a..515df16 100644 --- a/usr.sbin/ugidfw/ugidfw.c +++ b/usr.sbin/ugidfw/ugidfw.c @@ -46,6 +46,8 @@ __FBSDID("$FreeBSD$"); #include #include =20 +#define UGIDFW_BUFSIZ (BUFSIZ*2) + void add_rule(int argc, char *argv[]); void list_rules(void); void remove_rule(int argc, char *argv[]); @@ -71,22 +73,22 @@ usage(void) void add_rule(int argc, char *argv[]) { - char errstr[BUFSIZ], charstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ], charstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; int error, rulenum; =20 - error =3D bsde_parse_rule(argc, argv, &rule, BUFSIZ, errstr); + error =3D bsde_parse_rule(argc, argv, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; } =20 - error =3D bsde_add_rule(&rulenum, &rule, BUFSIZ, errstr); + error =3D bsde_add_rule(&rulenum, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; } - if (bsde_rule_to_string(&rule, charstr, BUFSIZ) =3D=3D -1) + if (bsde_rule_to_string(&rule, charstr, UGIDFW_BUFSIZ) =3D=3D -1) warnx("Added rule, but unable to print string."); else printf("%d %s\n", rulenum, charstr); @@ -95,25 +97,25 @@ add_rule(int argc, char *argv[]) void list_rules(void) { - char errstr[BUFSIZ], charstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ], charstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; int error, i, rule_count, rule_slots; =20 - rule_slots =3D bsde_get_rule_slots(BUFSIZ, errstr); + rule_slots =3D bsde_get_rule_slots(UGIDFW_BUFSIZ, errstr); if (rule_slots =3D=3D -1) { warnx("unable to get rule slots; mac_bsdextended.ko " "may not be loaded"); errx(1, "bsde_get_rule_slots: %s", errstr); } =20 - rule_count =3D bsde_get_rule_count(BUFSIZ, errstr); + rule_count =3D bsde_get_rule_count(UGIDFW_BUFSIZ, errstr); if (rule_count =3D=3D -1) errx(1, "bsde_get_rule_count: %s", errstr); =20 printf("%d slots, %d rules\n", rule_slots, rule_count); =20 for (i =3D 0; i < rule_slots; i++) { - error =3D bsde_get_rule(i, &rule, BUFSIZ, errstr); + error =3D bsde_get_rule(i, &rule, UGIDFW_BUFSIZ, errstr); switch (error) { case -2: continue; @@ -124,7 +126,7 @@ list_rules(void) break; } =20 - if (bsde_rule_to_string(&rule, charstr, BUFSIZ) =3D=3D -1) + if (bsde_rule_to_string(&rule, charstr, UGIDFW_BUFSIZ) =3D=3D -1) warnx("unable to translate rule %d to string", i); else printf("%d %s\n", i, charstr); @@ -134,7 +136,7 @@ list_rules(void) void set_rule(int argc, char *argv[]) { - char errstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ]; struct mac_bsdextended_rule rule; long value; int error, rulenum; @@ -152,13 +154,13 @@ set_rule(int argc, char *argv[]) =20 rulenum =3D value; =20 - error =3D bsde_parse_rule(argc - 1, argv + 1, &rule, BUFSIZ, errstr); + error =3D bsde_parse_rule(argc - 1, argv + 1, &rule, UGIDFW_BUFSIZ, errst= r); if (error) { warnx("%s", errstr); return; } =20 - error =3D bsde_set_rule(rulenum, &rule, BUFSIZ, errstr); + error =3D bsde_set_rule(rulenum, &rule, UGIDFW_BUFSIZ, errstr); if (error) { warnx("%s", errstr); return; @@ -168,7 +170,7 @@ set_rule(int argc, char *argv[]) void remove_rule(int argc, char *argv[]) { - char errstr[BUFSIZ]; + char errstr[UGIDFW_BUFSIZ]; long value; int error, rulenum; char *endp; @@ -185,7 +187,7 @@ remove_rule(int argc, char *argv[]) =20 rulenum =3D value; =20 - error =3D bsde_delete_rule(rulenum, BUFSIZ, errstr); + error =3D bsde_delete_rule(rulenum, UGIDFW_BUFSIZ, errstr); if (error) warnx("%s", errstr); } --rfwNdt5cNUUjB/69-- --asNXdz5DenlsLVVk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTwHNHAAoJEGqEZY9SRW7u60IP/3NwO8IpCny/NrgqsWPXz7J0 AvqNJqIIBvBNo925Ysfyg62Q3Wsg7nWXMlIUlTr5maNntnX6qBNrPRwOoA703xg8 WLXg5gJVFj5WvXYpXFykUK0aQLS2qasOx8YsL1PUj4b4tY3Tn1wmmvY+rL3c1407 t7ERdr8/L6CfYnxivbCBCEtESrdndFBcPa2YY8thKvfJK8Ekj0DBD+8J2IHcCyoW PonlN374YS5URN92xeyQDR6hEBwyIJed/HJSptWHcttJhGVDWcqK0eJ9J/+VQ6Na O/RO8MRg5AQRn0COB5vhhCCAtR4KemB4dqCb0kXCfjmwCql3uHNICxHcQPHDlB2I LvMk2CZpUg22lO/NxCkVI8z7WqnIx3Qm1k66fXo1kY/L+wgkoRpnzlEzhOvspmi+ GXVg4bzE3SIA0OSEO34rViYaRv//WCVvyFK59ai6WIhDpdbCivgAN24soxJxE2ZS 7IleQND/DC4XvL3EMs2UF9bOsHi3dvvNhAV8PhkwymLhlLQJc39vW9omZZ2dzAWy ldwlg8+C8BQyKUSxJPHDdtlyehGkGMwodnKxHaQIB6PuAPMcgg8vnZeWmqr+DF9l 2pOun0niTz2/i2rwWMvVijO8HnHD/hpfHyBfy4BTB3k1fbBbF5FoYFoF4MFSZIVa 0MBKhuXYnZ7aboV6vJFx =HCCI -----END PGP SIGNATURE----- --asNXdz5DenlsLVVk-- From owner-freebsd-arch@FreeBSD.ORG Sat Jul 12 02:18:30 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A74EB82D for ; Sat, 12 Jul 2014 02:18:30 +0000 (UTC) Received: from eastrmfepo203.cox.net (eastrmfepo203.cox.net [68.230.241.218]) by mx1.freebsd.org (Postfix) with ESMTP id 49B8220B2 for ; Sat, 12 Jul 2014 02:18:29 +0000 (UTC) Received: from eastrmimpo210 ([68.230.241.225]) by eastrmfepo203.cox.net (InterMail vM.8.01.05.15 201-2260-151-145-20131218) with ESMTP id <20140712021822.QJO2658.eastrmfepo203.cox.net@eastrmimpo210> for ; Fri, 11 Jul 2014 22:18:22 -0400 Received: from [192.168.3.22] ([72.219.202.186]) by eastrmimpo210 with cox id REJN1o00341obj401EJNnG; Fri, 11 Jul 2014 22:18:22 -0400 X-CT-Class: Clean X-CT-Score: 0.00 X-CT-RefID: str=0001.0A020205.53C09AEE.007D,ss=1,re=0.000,fgs=0 X-CT-Spam: 0 X-Authority-Analysis: v=2.0 cv=aZC/a2Ut c=1 sm=1 a=k40gPPfQ5QH6qv5U/EJc3Q==:17 a=f5xKl4ys9bwA:10 a=XigzRtDixlUA:10 a=G8Uczd0VNMoA:10 a=Wajolswj7cQA:10 a=8nJEP1OIZ-IA:10 a=kviXuzpPAAAA:8 a=kIqromvITh2ow2PlRWgA:9 a=wPNLvfGTeEIA:10 a=k40gPPfQ5QH6qv5U/EJc3Q==:117 X-CM-Score: 0.00 Authentication-Results: cox.net; none Message-ID: <53C09B48.8000709@cox.net> Date: Fri, 11 Jul 2014 22:19:52 -0400 From: "John D. Hendrickson and Sara Darnell" Reply-To: johnandsara2@cox.net User-Agent: Thunderbird 2.0.0.24 (X11/20100228) MIME-Version: 1.0 To: John Baldwin Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <53BF23A0.1000603@cox.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Emeric POUPON , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 02:18:30 -0000 John Baldwin wrote: > On Thursday, July 10, 2014 7:37:04 pm John D. Hendrickson and Sara Darnell > wrote: >> John Baldwin wrote: >>> On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: >>>> Thanks for your answer! > > No, the question is if you have two C files that are compiled into a single > loading object (foo.ko), do they call each other's functions directly or do > they use an indirection layer like kobj to call into each other. thx. i shouldn't answer (i asked) i just read linux kernel at times. i just assume the "two files" are both for the same kernel module and it would be ok. in which case using two C files isn't necessary ... but might confuse the Makefiles macros if they guess one C per mod try put both in one C file and spin the wheel why not try ? two diff mods call each other, in one .o or not, diff story i think From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 03:55:05 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C359333D; Sun, 13 Jul 2014 03:55:05 +0000 (UTC) Received: from mail-we0-x236.google.com (mail-we0-x236.google.com [IPv6:2a00:1450:400c:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15B7A2F9E; Sun, 13 Jul 2014 03:55:04 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id q59so2704202wes.41 for ; Sat, 12 Jul 2014 20:55:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=GjcEfkGqiwGMwdsFdJLn3AXEFv0zBBhc6W03vsTupdQ=; b=UwXW+ojYeLhjPEMzAaCa65UhRtBNBbgQeHj016esOmyGHFMtIQN/UsP4e0XHfskZhp NuZTJlrnpnxA9Y0vuyUzhuDlyGipb6VDCWItAGdTrwFV6a8DQ/6c9AXazJtfTjd1fYj7 thI3cAQ7lPs83VwvrrbV6EHI2zSu+vCSDzZ7aFm9mf1NiH+WGf+KoVTcBpI07ENbZ9UL LB/RVLuB+wiBNEirVeQWM3a6C0MNRw4gzTvXJOlMWRSH+A/4otCtOHr7icf7LCl4RGrN IgA/3gwCn5iYkBmyULocqrJW+DFfrnA0iL7F4n2921chagywEdvm1HJ+7Y1PX5UUSJbt yh/A== X-Received: by 10.194.20.230 with SMTP id q6mr10054470wje.43.1405223703093; Sat, 12 Jul 2014 20:55:03 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id jb16sm14004315wic.10.2014.07.12.20.55.01 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Sat, 12 Jul 2014 20:55:02 -0700 (PDT) Date: Sun, 13 Jul 2014 05:55:00 +0200 From: Mateusz Guzik To: freebsd-arch@freebsd.org Subject: Getting rid of atomic_load_acq_int(&fdp->fd_nfiles)) from fget_unlocked Message-ID: <20140713035500.GC16884@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: kib@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 03:55:05 -0000 Currently: /* * Avoid reads reordering and then a first access to the * fdp->fd_ofiles table which could result in OOB operation. */ if (fd < 0 || fd >= atomic_load_acq_int(&fdp->fd_nfiles)) return (EBADF); However, if we put fd_nfiles and fd_otable into one atomically replaced structure the only need to: 1. make sure the pointer is read once 2. issue a data dependency barrier - this is a noop on all supported architectures and we don't even have approprate macro, so doing nothing seems fine The motivation is to boost performance to amortize for seqlock cost, in case it hits the tree. This has no impact on races with capability lookup. In a microbenchmark of 16 threads reading from the same pipe fd immediately returning EAGAIN the numbers are: x vanilla-readpipe-run-sum + noacq-readpipe-run-sum [..] N Min Max Median Avg Stddev x 20 13133671 14900364 13893331 13827075 471500.82 + 20 59479718 59527286 59496714 59499504 13752.968 Difference at 95.0% confidence 4.56724e+07 +/- 213483 330.312% +/- 1.54395% There are 3 steps: 1. tidy up capsicum to accept fde: http://people.freebsd.org/~mjg/patches/single-fdtable-read-capsicum.patch 2. add __READ_ONCE: http://people.freebsd.org/~mjg/patches/read-once.patch 3. put stuff into one structure: http://people.freebsd.org/~mjg/patches/filedescenttable.patch Comments? -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 13:25:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54C93458; Sun, 13 Jul 2014 13:25:28 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EBF1E2688; Sun, 13 Jul 2014 13:25:27 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6DDPLNw092428 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 13 Jul 2014 16:25:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s6DDPLNw092428 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6DDPLXo092426; Sun, 13 Jul 2014 16:25:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Jul 2014 16:25:21 +0300 From: Konstantin Belousov To: Mateusz Guzik Subject: Re: Getting rid of atomic_load_acq_int(&fdp->fd_nfiles)) from fget_unlocked Message-ID: <20140713132521.GY93733@kib.kiev.ua> References: <20140713035500.GC16884@dft-labs.eu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pDmaGFxwxkQ3NNKB" Content-Disposition: inline In-Reply-To: <20140713035500.GC16884@dft-labs.eu> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 13:25:28 -0000 --pDmaGFxwxkQ3NNKB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 13, 2014 at 05:55:00AM +0200, Mateusz Guzik wrote: > Currently: > /* > * Avoid reads reordering and then a first access to the > * fdp->fd_ofiles table which could result in OOB operation. > */ > if (fd < 0 || fd >=3D atomic_load_acq_int(&fdp->fd_nfiles)) > return (EBADF); >=20 > However, if we put fd_nfiles and fd_otable into one atomically replaced > structure the only need to: > 1. make sure the pointer is read once > 2. issue a data dependency barrier - this is a noop on all supported > architectures and we don't even have approprate macro, so doing nothing > seems fine >=20 > The motivation is to boost performance to amortize for seqlock cost, in > case it hits the tree. >=20 > This has no impact on races with capability lookup. >=20 > In a microbenchmark of 16 threads reading from the same pipe fd > immediately returning EAGAIN the numbers are: > x vanilla-readpipe-run-sum =20 > + noacq-readpipe-run-sum > [..] > N Min Max Median Avg Stdd= ev > x 20 13133671 14900364 13893331 13827075 471500.= 82 > + 20 59479718 59527286 59496714 59499504 13752.9= 68 > Difference at 95.0% confidence > 4.56724e+07 +/- 213483 > 330.312% +/- 1.54395% >=20 > There are 3 steps: > 1. tidy up capsicum to accept fde: > http://people.freebsd.org/~mjg/patches/single-fdtable-read-capsicum.patch > 2. add __READ_ONCE: > http://people.freebsd.org/~mjg/patches/read-once.patch > 3. put stuff into one structure: > http://people.freebsd.org/~mjg/patches/filedescenttable.patch >=20 > Comments? We use 4-space indent for the continuation lines. Look at the malloc(9) call in the patch 3. The filedescenttable is really long name. Could it be, for instance, fdescenttbl ? Other than that, I think that the patches 2 and 3 are fine. I did not looked at the patch 1. --pDmaGFxwxkQ3NNKB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTwojAAAoJEJDCuSvBvK1BtIsP/3zuyAmdtT1PqaKnkriAsFQX ZAfDbBn61QpeaGLH2Diz4Cpa+Hq2W8BACL5RlWWbiYBgZhjMzSwv4tc/boORQQUn ObBbmNbCvjvUsxKgD3TfZR6XX0zuPiYnKZymhL3v/25ZyR9iTgWitGywQPVPOvgr G8NiONimPIQT8Mn70f57YwLFdnqThmuLFAXGfKOVH9+1dtBsIqEfQ8r8dasd6rDK iozaCjXDeAtpqNCDJ3xuA4ZxErZHptqaL9oz0kdOOXHLKm++WKvcaGJD8JPfHWHq 2BsQ8HKAxKeipxMT1lTR1fpq6Jabjawed6mSrhZmFBYn37FcXSu5IFN86MiyR3sy cXCysNoa9jLnzmQirvzgTYs0l1H28nIE8YCHAWdOtWsOkGWr/0LhFXXAkMagrhum tC/+8IUMiK8STTPVSD34ejuylP7KpLrCsTWMbjgqisRiHBC+F7Mcw3XvCuv7cd8k vjR0JQ2kOFe1vctHd2PmtKG3Pzc8egARmIAfRcHSIYoB3d4yVLldJ+9OqWjyTIJA GhKJ4LV1PLQz/uI7ZxVQ7UBOlVfQUn/qIidoJ7+2QULAsbRSUmucoDEvJT1CsqsN 6zItQLo87f4S5VjREbT/86euGaoiThwDbdaXeDLkoWKksVQiLud8qmHhJPg4tNZj rN23tB0N9qoHWLrSFwY4 =nHRc -----END PGP SIGNATURE----- --pDmaGFxwxkQ3NNKB-- From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 13:34:26 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7ED9910; Sun, 13 Jul 2014 13:34:26 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 613A02746; Sun, 13 Jul 2014 13:34:26 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6DDYLhA094582 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 13 Jul 2014 16:34:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s6DDYLhA094582 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6DDYLpN094581; Sun, 13 Jul 2014 16:34:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 Jul 2014 16:34:21 +0300 From: Konstantin Belousov To: Mateusz Guzik Subject: Re: Getting rid of atomic_load_acq_int(&fdp->fd_nfiles)) from fget_unlocked Message-ID: <20140713133421.GA93733@kib.kiev.ua> References: <20140713035500.GC16884@dft-labs.eu> <20140713132521.GY93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zE+0D7H8clM/QbKf" Content-Disposition: inline In-Reply-To: <20140713132521.GY93733@kib.kiev.ua> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 13:34:27 -0000 --zE+0D7H8clM/QbKf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 13, 2014 at 04:25:21PM +0300, Konstantin Belousov wrote: > On Sun, Jul 13, 2014 at 05:55:00AM +0200, Mateusz Guzik wrote: > > Currently: > > /* > > * Avoid reads reordering and then a first access to the > > * fdp->fd_ofiles table which could result in OOB operation. > > */ > > if (fd < 0 || fd >=3D atomic_load_acq_int(&fdp->fd_nfiles)) > > return (EBADF); > >=20 > > However, if we put fd_nfiles and fd_otable into one atomically replaced > > structure the only need to: > > 1. make sure the pointer is read once > > 2. issue a data dependency barrier - this is a noop on all supported > > architectures and we don't even have approprate macro, so doing nothing > > seems fine > >=20 > > The motivation is to boost performance to amortize for seqlock cost, in > > case it hits the tree. > >=20 > > This has no impact on races with capability lookup. > >=20 > > In a microbenchmark of 16 threads reading from the same pipe fd > > immediately returning EAGAIN the numbers are: > > x vanilla-readpipe-run-sum =20 > > + noacq-readpipe-run-sum > > [..] > > N Min Max Median Avg St= ddev > > x 20 13133671 14900364 13893331 13827075 47150= 0.82 > > + 20 59479718 59527286 59496714 59499504 13752= =2E968 > > Difference at 95.0% confidence > > 4.56724e+07 +/- 213483 > > 330.312% +/- 1.54395% > >=20 > > There are 3 steps: > > 1. tidy up capsicum to accept fde: > > http://people.freebsd.org/~mjg/patches/single-fdtable-read-capsicum.pat= ch > > 2. add __READ_ONCE: > > http://people.freebsd.org/~mjg/patches/read-once.patch > > 3. put stuff into one structure: > > http://people.freebsd.org/~mjg/patches/filedescenttable.patch > >=20 > > Comments? >=20 > We use 4-space indent for the continuation lines. Look at the malloc(9) > call in the patch 3. >=20 > The filedescenttable is really long name. Could it be, for instance, > fdescenttbl ? >=20 > Other than that, I think that the patches 2 and 3 are fine. I did not > looked at the patch 1. As an afterthought, you do not need __READ_ONCE(), the __DEVOLATILE() alone would do what you need as well. --zE+0D7H8clM/QbKf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTwordAAoJEJDCuSvBvK1Bw4oP/3O1Zv2SYSRXPUfr666VceRr p6Xi7ljuuBizsWAhr0IOK6dL10cyTt+uxyEhUfJhPG3awUXwLV1iw2x42ACRKq1m vSqDFbTSzVf3Uj3zhQQ5CpHeKR7OdNhjCTu2KK0JL9gEOH46ZlNpPRQhrkhXhhmi xru8fBguBxcw7OGrMmPd9I5lHpqgzD6zX/xiob7w0DhGRRkd39eTpcuY37QHs3Ti Hhgb+hOfliTjvlPthixv4c9AUn00imQwYaudfLhBMCywBNzuxNgcUegRTaMM+lpI QlRhBZczyVnCiomK5POgRb0cOUKrWBKSPfSuU4YPzHIG84LnRpThDEHNTq6JKSDa b2LWGLHRFqhMTYlS8t1ecYP05mZl2edooQ1wSXqPujK/3vgNfH1nFs1C1eLcjpvL hlbtohRYuXox6Oj07LWTmgYcggP01wwPhxPzw/UiG/Y3IHGgx9TQC3RirEtXOWhL +p/i/QT1XyTQwR7XxLhPIp5SUd32zBgryAgwTL/lsyp/gnNJ6030T9EAkUNaoIEX LhxWn59jYYc9IV70jwI4Gw85yyBOviSjUmGfgm6bdNEGytvirTx+zhQCfIJ5yWey jc8LS6+1JDix+3AMFFpFqUdNqzvdsSKmChoNpdWoMmqH0HXw5qytN+BQUnG1vjAZ U1/qYhqgyS55biK0r5yC =M+4j -----END PGP SIGNATURE----- --zE+0D7H8clM/QbKf-- From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 14:11:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B7DA406; Sun, 13 Jul 2014 14:11:52 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 597DC2A18; Sun, 13 Jul 2014 14:11:51 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id CCCB21FE027; Sun, 13 Jul 2014 16:11:48 +0200 (CEST) Message-ID: <53C293AC.9020006@selasky.org> Date: Sun, 13 Jul 2014 16:11:56 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Benjamin Kaduk , John-Mark Gurney Subject: Re: callout(9) really this complicated? References: <20140704041521.GW45513@funkthat.com> <201407091211.47642.jhb@freebsd.org> <20140710061955.GT45513@funkthat.com> <201407101530.32533.jhb@freebsd.org> <20140711023749.GA45513@funkthat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 14:11:52 -0000 On 07/11/14 17:28, Benjamin Kaduk wrote: > [large pile of context trimmed] > > On Thu, 10 Jul 2014, John-Mark Gurney wrote: > >> Ok, I'm so confused now that I will NOT be creating a patch, and unless >> someone comes up w/ a patch to clarify it, I will add to the description >> that there are numerous misleading statements in this man page and link >> them to this thread... > > I have saved jhb's messages in this thread and added to my TODO list to > make a pass through this man page. There's quite a few things in front > of this in my list, though -- I don't expect to have anything for maybe > a month. > Hi, One additional comment: The "callout_drain()" function also ensures that not only the function callback is no longer called/executing, but also that the mutex which is passed as an argument to callout_init_mtx() is no longer being used. Maybe the "start/stop/drain" theorem could be explained in general somewhere, because the USB stack works similarly, and I believe it is the correct way to implement more advanced callback API's in a multithreaded system. --HPS From owner-freebsd-arch@FreeBSD.ORG Sun Jul 13 20:43:07 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 33632338; Sun, 13 Jul 2014 20:43:07 +0000 (UTC) Received: from mail-qa0-x234.google.com (mail-qa0-x234.google.com [IPv6:2607:f8b0:400d:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC787277C; Sun, 13 Jul 2014 20:43:06 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id j15so2570013qaq.39 for ; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=0KgQU9WCtoBM4fd6PjGffuyMmzI+1CCabI8fVQQ/7oE=; b=x6GQu3r5Jj9xq14djuGNjGb8KvB4cnOQl63OrTXv5Nr5a+uvtlPLn7p53P9qWP1HHs 8cCg7sRtNbLs0kRC1tD1QEjunAGrn4pTucn+X9w+jI9rozINo8ccB/1pKkevUgKyF9iA j+5agngwmYZVByPxY6XM+tGAY9aFVmskpqv5cP5cVuVCmcOBqv7+jbFtZU0aQ008AEIZ uBq/svn8FNzGzZQiSPlf0wnNWvI88BQGS6DgLSNPK3cr+2Bw27Wmju++ocBwzkAFhsjy NfCeoLf73SKDIhrjCvYcjzhyUIjXpuySV2RDwFanlbsf9gW0Hsxroy293yN4Efx9Bcri ZpPg== MIME-Version: 1.0 X-Received: by 10.140.88.243 with SMTP id t106mr3152478qgd.12.1405284185528; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Sun, 13 Jul 2014 13:43:05 -0700 (PDT) Date: Sun, 13 Jul 2014 13:43:05 -0700 X-Google-Sender-Auth: 5cDLK9T7hZnZRdDgGZpnptpU62M Message-ID: Subject: RSS update From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2014 20:43:07 -0000 Hi! for those who are interested, here's an update on my RSS hackery. I found a few hours to get through a few more things: * IPv4 TCP RSS is there; * IPv6 TCP RSS is now there; * Since it's IP options, there's a seperate set of IPv4 and IPv6 options; * I've verified that on igb and ixgbe that the incoming connections get redirected to the right RSS aware listen socket. Now, UDP is a pain - mostly because a lot of UDP traffic is going to be > 1 MTU and the ixgbe/igb/cxgbe NICs will hash IP fragments with the 2-tuple IP address. So things end up in the wrong spot. I however have UDPv4 RSS awareness working locally. I don't know when I'll get some more time to work on a way to handle IP fragments and requeuing the resulting frame to the correct netisr context. Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 00:44:16 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C48AF297; Tue, 15 Jul 2014 00:44:16 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78A1B256B; Tue, 15 Jul 2014 00:44:16 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so1848060qaj.7 for ; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=FvN3gfIZgZCEaegkqLkRZsZs+kB/LvfwkJn2UNHjVFI=; b=pCmSgKvUj/I0TZPmRsKZpoc0cKyOX/T0SUG960r++eYdKWaNdckyJ4nIGKt8AiaQqs O9J9mkK7XFXAiaoZeWUQMNfkkGYk2G3dghCf9Ws1Q1Htg4w0UNbSQxLI4NtkA0E4hn65 JkbvEAXXVDExhi5YFWFwVplp5Z0Zu13QkH2/JTy0x5fQKkKVELR0vWR/vdbCW2xEr3xO AOHXuoTETzhUeK0Zpb6S3kMGjbQunisd/qml9AwNbwrKw5eUuOIyLqGAHU7406v0VrM0 IQSpXvWvKAguja+Hw0KXVBNC3XrCwCIfiyj+cFofUQrw3UDFjUTSrLCMg6Oqwaeo7wVh QHpA== MIME-Version: 1.0 X-Received: by 10.224.171.197 with SMTP id i5mr28743881qaz.55.1405385055606; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Mon, 14 Jul 2014 17:44:15 -0700 (PDT) Date: Mon, 14 Jul 2014 17:44:15 -0700 X-Google-Sender-Auth: drKNq-70BpKBM9JhjBBupuyS5po Message-ID: Subject: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 00:44:16 -0000 Hi, Whilst digging into UDP receive side scaling on the intel ixgbe(4) NIC, I stumbled across how it hashes traffic between IP fragmented traffic and non IP-fragmented traffic. Here's how it surfaced: * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and TCP/UDP (4-tuple); * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple and comes into queue A; * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple and comes into queue B. So if there's a mix of small and large datagrams, we'll end up with some packets coming in via queue A and some by queue B. In normal operation that'll result in out of order packets. For the RSS stuff I'm working on it means that some packets will match the PCBGROUP setup and some won't. By default UDP configures a 2-tuple hash so it expects packets to come in hashed appropriately. But that only matches for large frames. For small frames it'll be hashed via the 4-tuple and it won't match. The ip reassembly code doesn't recalculate the flowid/flowtype once it's finished. It'd be nice to do that before further processing so it can be placed in the right netisr. So there's a couple of semi-overlapping issues: * Right now we could get TCP and UDP frames out of order. I'd like to at least have ixgbe(4) hash on the 2-tuple for UDP rather than the 4-tuple. That fixes that silly corner case. It's not likely going to show up except for things like forwarding workloads. Maybe people doing memcached work, I'm not sure. * Whether or not to calculate the flowid/flowtype in ip_reass() (or maybe in the netisr input path, in case there's no flowid assigned) so work is better distributed; * .. then if we do that, we could do 4-tuple UDP hashing again and we'd just recalculate for any large frames. Here's what happened with Linux and ixgbe in 2010 on this topic: http://comments.gmane.org/gmane.linux.network/166687 What do people think? -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 01:40:31 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 757E9B6D for ; Tue, 15 Jul 2014 01:40:31 +0000 (UTC) Received: from vps.hungerhost.com (vps.hungerhost.com [216.38.53.176]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E35929D9 for ; Tue, 15 Jul 2014 01:40:31 +0000 (UTC) Received: from c-76-19-231-75.hsd1.ma.comcast.net ([76.19.231.75]:59308 helo=[172.16.19.1]) by vps.hungerhost.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1X6riE-0003YJ-4i for arch@freebsd.org; Mon, 14 Jul 2014 21:39:23 -0400 From: "George Neville-Neil" To: arch@freebsd.org Subject: RFC: Addition of Murmur3 hash to FreeBSD for use by PF Date: Mon, 14 Jul 2014 21:39:13 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-Mailer: MailMate (1.7.2r3905) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - vps.hungerhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - neville-neil.com X-Get-Message-Sender-Via: vps.hungerhost.com: authenticated_id: gnn@neville-neil.com X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 01:40:31 -0000 Howdy, Turns out that the Jenkins hash we have has been surpassed by the Murmur3 hash. Switching to Murmur3 has increases PF's speed by 12% on a null forwarding test, i.e. a test in which PF looks at every packet but doesn't taken an action. Review here: https://phabric.freebsd.org/D406 Best, George From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 06:21:13 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D247DEBC; Tue, 15 Jul 2014 06:21:13 +0000 (UTC) Received: from mail-vc0-x22f.google.com (mail-vc0-x22f.google.com [IPv6:2607:f8b0:400c:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6CA222F90; Tue, 15 Jul 2014 06:21:13 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id hu12so4101117vcb.6 for ; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=DMDHrfojD3PK1R3qhR+G//uG+VuerRoZxYFEL11Q4JA=; b=w5iwvuAnCQVUqBQh6pu81hf1OAccd/kXErxe7XonRnbh6xJm+yO7AdFyJoSxLVD/R4 QRvir/fLw7gLQkNqzfmReZLUNTCvOsEUYxcimQM3Wrxt7tlg2aU/sq8Vc8MdLtaR/pDR he0eo8i3/xmpz3SQXqx46cNUIh7GLhI97gJcVWFjNRPjM/vyOZja4Ku09RyeEfdpsNWW 1EyymacyqagvJ1ffuHpPRI/qfeuQFi+aSY5XlJTeKqoEMgAHeo15LRpu/7cGd+K1MBxq CxatMme9d8VXsqwCHLltsnC87pHs1aL0bPG9iOiCTTq/KfnJHrVkULxlfkwcL2fLBik6 YIsA== MIME-Version: 1.0 X-Received: by 10.220.2.136 with SMTP id 8mr20340949vcj.17.1405405272376; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) Received: by 10.221.53.199 with HTTP; Mon, 14 Jul 2014 23:21:12 -0700 (PDT) In-Reply-To: References: Date: Mon, 14 Jul 2014 23:21:12 -0700 Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 06:21:13 -0000 I had missed the fact that Alex turned this off in the Linux driver, sounds to me like its the right thing to do for FreeBSD also. Jack On Mon, Jul 14, 2014 at 5:44 PM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 09:02:50 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E7293184; Tue, 15 Jul 2014 09:02:49 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 163962E50; Tue, 15 Jul 2014 09:02:48 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id e16so2191931lan.39 for ; Tue, 15 Jul 2014 02:02:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=n/4LTRxM71tw09Lb29UdD1Z/Zt09C9FQuAsVj1pSy8M=; b=rQKbDcAOrqMyjPb38vnOTiwSEzbGk97D9z+Og412gx+R7VnwWT9AvKStNL/7fAvfZ+ eDG3g9pC/tg2EtEXUHx0MawNpOqRu+jVrIzNjAUUruwIDtBqK+i18wJd5Y7CekqDUZH5 tYR1OxU/NpdtKVqZ1VTid4qWPtE9zrnLKouF9JIXPzMk6z5XYSlHSkWx/JBMFlHt9uUW AK0ntAamS0a0P2J14XTx+3/2TDVfOCdt0B2PHcUpebwEcgr1NCyhav7foHqyqiGtfc3y F3IxXxeHoFnDtb8HW4z41AXlTV8w37qzoBXUJjVvpPqJ9/touZfg+ztrQC/jsOwijyDP aD9g== X-Received: by 10.152.205.99 with SMTP id lf3mr1460665lac.63.1405414966866; Tue, 15 Jul 2014 02:02:46 -0700 (PDT) Received: from [192.168.2.30] ([2.176.190.226]) by mx.google.com with ESMTPSA id qx6sm19344336lbb.23.2014.07.15.02.01.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jul 2014 02:02:46 -0700 (PDT) Message-ID: <53C4EE00.5090705@gmail.com> Date: Tue, 15 Jul 2014 13:31:52 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:02:50 -0000 On 7/15/2014 5:14 AM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" Doesn't the problem applies to TCP too? TCP may be fragmented too but is less likely because of MSS. -- Best regards. Hooman Fazaeli From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 09:31:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 00B9CFCA for ; Tue, 15 Jul 2014 09:31:27 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B022820E7 for ; Tue, 15 Jul 2014 09:31:27 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1X6z53-000Nax-Bu for freebsd-arch@freebsd.org; Tue, 15 Jul 2014 13:31:25 +0400 Date: Tue, 15 Jul 2014 13:31:25 +0400 From: Slawa Olhovchenkov To: freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Message-ID: <20140715093125.GA89128@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53C4EE00.5090705@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 09:31:28 -0000 On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: > Doesn't the problem applies to TCP too? > TCP may be fragmented too but is less likely because of MSS. Don't forget GRE, IPIP, ESP and AH! From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 10:52:33 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4981BF5C for ; Tue, 15 Jul 2014 10:52:33 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3ED3288A for ; Tue, 15 Jul 2014 10:52:32 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so3866013lab.36 for ; Tue, 15 Jul 2014 03:52:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Z8+tSRFKpEXndKLxsfnliUOYcpPSwtVa7iKoK+wS/nk=; b=K74y8cQjkI0CwG/bnJ1nVK3431ZkT+m/yu0P2I7CZ9p2Oe5tXc67doxbbw+Cyt5qXF HeJN0jNtDmxu1CV+3YU5kZ3pQ8P2XlnEcdiB7pKmjdvHtVETEmbPnX1kSOi2nwB2LEiD 5BG5rCL85lSBNvSQiEcGqu1uhlm3JFqJc2vpoJa0rNptHIJnFRFlh5gHfT9CMjgogGYA aeJT3M6PtyTtInFIw9wbvqk/jbk4YtVSSmsRkAXEQXduf5MqoGgpBGvgS2rDkjWwrp2j bCcKoNNJBFUYtWn9n1VWvrN8J7FK08uJ1s3NLh2ca82LqlN6y4ytjDW46rP2Yu85j6WH aOOg== X-Received: by 10.112.235.72 with SMTP id uk8mr1529271lbc.80.1405421550429; Tue, 15 Jul 2014 03:52:30 -0700 (PDT) Received: from [192.168.2.30] ([2.176.190.226]) by mx.google.com with ESMTPSA id as6sm19679335lbc.31.2014.07.15.03.52.28 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jul 2014 03:52:29 -0700 (PDT) Message-ID: <53C507F2.2090604@gmail.com> Date: Tue, 15 Jul 2014 15:22:34 +0430 From: Hooman Fazaeli User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: Slawa Olhovchenkov Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> In-Reply-To: <20140715093125.GA89128@zxy.spb.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 10:52:33 -0000 On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: > >> Doesn't the problem applies to TCP too? >> TCP may be fragmented too but is less likely because of MSS. > Don't forget GRE, IPIP, ESP and AH! > These protocols don't use port numbers and the RSS hash is computed based on the (srcip,dstip) tuple, so the problem does not apply to them. -- Best regards. Hooman Fazaeli From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 11:33:23 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92B106C3 for ; Tue, 15 Jul 2014 11:33:23 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9A02BDA for ; Tue, 15 Jul 2014 11:33:22 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1X70z1-000P5C-7C for freebsd-arch@freebsd.org; Tue, 15 Jul 2014 15:33:19 +0400 Date: Tue, 15 Jul 2014 15:33:19 +0400 From: Slawa Olhovchenkov To: freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Message-ID: <20140715113319.GA96254@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53C507F2.2090604@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 11:33:23 -0000 On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: > On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: > > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: > > > >> Doesn't the problem applies to TCP too? > >> TCP may be fragmented too but is less likely because of MSS. > > Don't forget GRE, IPIP, ESP and AH! > > > These protocols don't use port numbers and the RSS hash is computed based on the > (srcip,dstip) tuple, so the problem does not apply to them. And all flows go to one queue? Bad. From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 14:33:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59EFC51D for ; Tue, 15 Jul 2014 14:33:58 +0000 (UTC) Received: from mail-qg0-x229.google.com (mail-qg0-x229.google.com [IPv6:2607:f8b0:400d:c04::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17CDB2C6C for ; Tue, 15 Jul 2014 14:33:58 +0000 (UTC) Received: by mail-qg0-f41.google.com with SMTP id q107so2735442qgd.0 for ; Tue, 15 Jul 2014 07:33:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=xChN/oTycv8YUAXPTwvPkTnW/nv39w0okzpstJW67Y4=; b=lkGzLT01XvNSN/AarNaAIkRrQVD8cn6g+mfDViZiYH6kJYgsUIeQWvARmZCOof2jf8 7AmqzjqNufJhUkWBhV3/CP5w6ZSlNuYCTnzUqqjFVvFtYm0+2RHh7XDxp3wtxjdNvWT1 NM0Zmq5qaIq6fhfJSSKMhIIpX2IP2MMlnfj0wwmEpZucDb3Qzm5IM2RYLp2yVNVhjHOx YDQRa5/9gFhuEJ44fm10R5G2WsRZt3SlLrmlsQFjaS2BTYAZzQ12QJgMU3ME4D7G6+K7 vSxwQV2+EvtXw6daKQPakDP91wVY2Wkj5ZvTqw2pTszqXOvahmxLwRaVNbgF4OqSFKqQ RbMA== MIME-Version: 1.0 X-Received: by 10.224.156.145 with SMTP id x17mr7292834qaw.49.1405434837249; Tue, 15 Jul 2014 07:33:57 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 07:33:57 -0700 (PDT) In-Reply-To: <20140715113319.GA96254@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> <20140715113319.GA96254@zxy.spb.ru> Date: Tue, 15 Jul 2014 07:33:57 -0700 X-Google-Sender-Auth: 0NY0qCPHCbjQa1o89rUe2SLgLY4 Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 14:33:58 -0000 On 15 July 2014 04:33, Slawa Olhovchenkov wrote: > On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: > >> On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: >> > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: >> > >> >> Doesn't the problem applies to TCP too? >> >> TCP may be fragmented too but is less likely because of MSS. >> > Don't forget GRE, IPIP, ESP and AH! >> > >> These protocols don't use port numbers and the RSS hash is computed based on the >> (srcip,dstip) tuple, so the problem does not apply to them. > > And all flows go to one queue? Bad. All the packets between two IP addresses for non-TCP, non-UDP get hashed to the same CPU core, yes. If you have 1000 IP tunnels to different end-hosts, they'll be on different CPUs. -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 14:36:39 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 38CE46E2 for ; Tue, 15 Jul 2014 14:36:39 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E57BC2C96 for ; Tue, 15 Jul 2014 14:36:38 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1X73qN-0001ND-5e for freebsd-arch@freebsd.org; Tue, 15 Jul 2014 18:36:35 +0400 Date: Tue, 15 Jul 2014 18:36:35 +0400 From: Slawa Olhovchenkov To: freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Message-ID: <20140715143635.GA5178@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> <20140715113319.GA96254@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 14:36:39 -0000 On Tue, Jul 15, 2014 at 07:33:57AM -0700, Adrian Chadd wrote: > On 15 July 2014 04:33, Slawa Olhovchenkov wrote: > > On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: > > > >> On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: > >> > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: > >> > > >> >> Doesn't the problem applies to TCP too? > >> >> TCP may be fragmented too but is less likely because of MSS. > >> > Don't forget GRE, IPIP, ESP and AH! > >> > > >> These protocols don't use port numbers and the RSS hash is computed based on the > >> (srcip,dstip) tuple, so the problem does not apply to them. > > > > And all flows go to one queue? Bad. > > All the packets between two IP addresses for non-TCP, non-UDP get > hashed to the same CPU core, yes. > > If you have 1000 IP tunnels to different end-hosts, they'll be on > different CPUs. What about two host, 10G link and 1000 TCP connections over IPsec? From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 14:37:31 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72341787 for ; Tue, 15 Jul 2014 14:37:31 +0000 (UTC) Received: from mail-qa0-x230.google.com (mail-qa0-x230.google.com [IPv6:2607:f8b0:400d:c00::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 309A22C9B for ; Tue, 15 Jul 2014 14:37:31 +0000 (UTC) Received: by mail-qa0-f48.google.com with SMTP id m5so2508849qaj.7 for ; Tue, 15 Jul 2014 07:37:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=nuBDe1p1pWC25TZcMErJOpyhMUGB+YuSZ8kKfz35GNM=; b=DfvfbRua++AAdpKG+1d++FOh9IFiVL4oQa7y+YxY96W7lpFMOTlO0JcfpRqATw4KIj jxtGjCjI706RjD/HMcU1smMdx0rGDRDBc7xb+FT8a4JDbULq3ufCKDCblTaV/HTQ5Ky3 qoDU/btA699TkyhsutrSlVUenl+1tBDiXoc4FKw0apww3n6FBzbmJ/NgHqW1M2atgYhq FVPARo9JuPVAbU74hbJoBopsg/Aiq86jmCwRsWUlCRMajtd9AymEOytTYzyB3XLe/ijq 1IJfOm/3mx7v0dAuCyhAIh78h3pFJA0xfibhxa6nCjFyNCPTp80CQWTYB0P9silFRIlx 2LRQ== MIME-Version: 1.0 X-Received: by 10.224.130.201 with SMTP id u9mr33161854qas.98.1405435050321; Tue, 15 Jul 2014 07:37:30 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 07:37:30 -0700 (PDT) In-Reply-To: <20140715143635.GA5178@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> <20140715113319.GA96254@zxy.spb.ru> <20140715143635.GA5178@zxy.spb.ru> Date: Tue, 15 Jul 2014 07:37:30 -0700 X-Google-Sender-Auth: DloIeYzagCoQcKQXcMbp9yPGWRE Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 14:37:31 -0000 On 15 July 2014 07:36, Slawa Olhovchenkov wrote: > On Tue, Jul 15, 2014 at 07:33:57AM -0700, Adrian Chadd wrote: > >> On 15 July 2014 04:33, Slawa Olhovchenkov wrote: >> > On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: >> > >> >> On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: >> >> > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: >> >> > >> >> >> Doesn't the problem applies to TCP too? >> >> >> TCP may be fragmented too but is less likely because of MSS. >> >> > Don't forget GRE, IPIP, ESP and AH! >> >> > >> >> These protocols don't use port numbers and the RSS hash is computed based on the >> >> (srcip,dstip) tuple, so the problem does not apply to them. >> > >> > And all flows go to one queue? Bad. >> >> All the packets between two IP addresses for non-TCP, non-UDP get >> hashed to the same CPU core, yes. >> >> If you have 1000 IP tunnels to different end-hosts, they'll be on >> different CPUs. > > What about two host, 10G link and 1000 TCP connections over IPsec? > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 14:38:55 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5D84684A for ; Tue, 15 Jul 2014 14:38:55 +0000 (UTC) Received: from mail-qa0-x22b.google.com (mail-qa0-x22b.google.com [IPv6:2607:f8b0:400d:c00::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C5292CAC for ; Tue, 15 Jul 2014 14:38:55 +0000 (UTC) Received: by mail-qa0-f43.google.com with SMTP id w8so4563868qac.2 for ; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=u16qXV0oIWEQzZjGLvJop6jlbyvwEHiOMxh/RMZSOzc=; b=pMTLtZtZfXsW9gpDMxVDiA95UNcscQ3e4ypxLyLt67WvYA1dhKCHYBKsGatPnV4W+E QnxuM13Woq/yEtz/lpCdnczEtqOsvqfBWDn2OBO3P85z3ICs8annN8/jQMuUJ9JOaqoO TFEA3nCT8vb4vCe364uYDZv90zZCXneNCPGefbQ22TVVx0u9Qe33T9rVhyW9OGZh3Mpp O3o/J4m2pK0IlNzm85LIb58rOUCglIOWznX4tnVM3jSXZzNfBkNaCdwnR2rAtyy4nHY0 VkPZrBnF/kPVNBECuiI/3cWFyyyy2hpJvZVEclAj1PHWKsIwD5ojMhFiAJpnkghLyeSa xCQw== MIME-Version: 1.0 X-Received: by 10.140.80.19 with SMTP id b19mr34103547qgd.102.1405435134351; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 07:38:54 -0700 (PDT) In-Reply-To: <20140715143635.GA5178@zxy.spb.ru> References: <53C4EE00.5090705@gmail.com> <20140715093125.GA89128@zxy.spb.ru> <53C507F2.2090604@gmail.com> <20140715113319.GA96254@zxy.spb.ru> <20140715143635.GA5178@zxy.spb.ru> Date: Tue, 15 Jul 2014 07:38:54 -0700 X-Google-Sender-Auth: QOx5XpbUkrD-V37QXDsWZXLbtA8 Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: Slawa Olhovchenkov Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 14:38:55 -0000 On 15 July 2014 07:36, Slawa Olhovchenkov wrote: > On Tue, Jul 15, 2014 at 07:33:57AM -0700, Adrian Chadd wrote: > >> On 15 July 2014 04:33, Slawa Olhovchenkov wrote: >> > On Tue, Jul 15, 2014 at 03:22:34PM +0430, Hooman Fazaeli wrote: >> > >> >> On 7/15/2014 2:01 PM, Slawa Olhovchenkov wrote: >> >> > On Tue, Jul 15, 2014 at 01:31:52PM +0430, Hooman Fazaeli wrote: >> >> > >> >> >> Doesn't the problem applies to TCP too? >> >> >> TCP may be fragmented too but is less likely because of MSS. >> >> > Don't forget GRE, IPIP, ESP and AH! >> >> > >> >> These protocols don't use port numbers and the RSS hash is computed based on the >> >> (srcip,dstip) tuple, so the problem does not apply to them. >> > >> > And all flows go to one queue? Bad. >> >> All the packets between two IP addresses for non-TCP, non-UDP get >> hashed to the same CPU core, yes. >> >> If you have 1000 IP tunnels to different end-hosts, they'll be on >> different CPUs. > > What about two host, 10G link and 1000 TCP connections over IPsec? If the IP data is encrypted, including the TCP header, then yes, receive occurs on one CPU for now. Transmit occurs from multiple CPUs but it'll likely be serialised through one transmit queue. If the TCP data is encrypted, but the TCP header is in clear-text and it's a normal TCP frame, then hardware will hash it across multiple CPUs. -a From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 15:38:12 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90D86D77 for ; Tue, 15 Jul 2014 15:38:12 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 09B2422EA for ; Tue, 15 Jul 2014 15:38:11 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E850EB91E; Tue, 15 Jul 2014 11:38:10 -0400 (EDT) From: John Baldwin To: johnandsara2@cox.net Subject: Re: How to properly handle several fonctions provided by the Winbond SuperIO chip? Date: Tue, 15 Jul 2014 10:36:15 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1118241087.138096.1403180509132.JavaMail.zimbra@arkoon-netasq.com> <53C09B48.8000709@cox.net> In-Reply-To: <53C09B48.8000709@cox.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201407151036.15266.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 15 Jul 2014 11:38:11 -0400 (EDT) Cc: Emeric POUPON , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 15:38:12 -0000 On Friday, July 11, 2014 10:19:52 pm John D. Hendrickson and Sara Darnell wrote: > John Baldwin wrote: > > On Thursday, July 10, 2014 7:37:04 pm John D. Hendrickson and Sara Darnell > > wrote: > >> John Baldwin wrote: > >>> On Thursday, June 19, 2014 11:21:59 am Emeric POUPON wrote: > >>>> Thanks for your answer! > > > > No, the question is if you have two C files that are compiled into a single > > loading object (foo.ko), do they call each other's functions directly or do > > they use an indirection layer like kobj to call into each other. > > thx. i shouldn't answer (i asked) i just read linux kernel > at times. > > i just assume the "two files" are both for the same kernel module and > it would be ok. in which case using two C files isn't necessary Often times code is split into multiple C files so it is easier for people to understand even if the computer doesn't really care. > ... but might confuse the Makefiles macros if they guess one C per mod > > try put both in one C file and spin the wheel why not try ? > > two diff mods call each other, in one .o or not, diff story i think Correct. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 19:32:30 2014 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 24EACDEA; Tue, 15 Jul 2014 19:32:30 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 874FF2BA6; Tue, 15 Jul 2014 19:32:29 +0000 (UTC) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s6FJWOQ4026052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 15 Jul 2014 22:32:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua s6FJWOQ4026052 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s6FJWOPa026051; Tue, 15 Jul 2014 22:32:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 15 Jul 2014 22:32:24 +0300 From: Konstantin Belousov To: arch@freebsd.org Subject: p_oppid and real parent calculation for traced processes Message-ID: <20140715193224.GT93733@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8Ne50Y5IYlza1jvg" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: mjg@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 19:32:30 -0000 --8Ne50Y5IYlza1jvg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I created a phabricator issue https://phabric.freebsd.org/D417 , which contains the patch, aimed to solve the remaining issues with reparenting of the traced process away from the real process parent. The link above contains more detailed description and patch itself. Reviews are welcome. --8Ne50Y5IYlza1jvg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJTxYHHAAoJEJDCuSvBvK1BgrYP/RafigUlhvtz2U/cevcnqhNk bZNHFP6JRg9McUOP3Cn5EL63Oefl0TRw+5VFdpctbqcdv38+mkUlaliNjgcx+LQH XKROCtuWfLx4+3kP3OWxP3UXe5b/C5q0nUec7I7PFTguhm4EHPKkUvkpkFbsFJJ2 qZJGDaKbept96mt7MkVPY4B9ToBG9zn7+xDVUX8KILBYbVjw4Fo6nfT9KcGCZCdI E3PHBhWrMsfDOTtVlZd865Ou0HB6rV/WtqovDxp+9TbUKZpK63fWsuWEunVMYGJW rVTpZTOs33zosTUDuAOs6dXBL4dhziYuPa19JT/uupfxiNWB5kWX+9KEIJECwJ5p nlUHhmSloVIsosxMzUqDgt2O977GzRutiwp3BNKtaSCzYkTjgoQP9eS3F5yjpa+S bMDKVSdDv8yn9+4u9IbM5rw+4sFQ9y/SdDiR2vbVvnOD37IQBgHftH0XXOxT+DY3 D2OGdm4OfHIqnRKwxKJa1uzFYe41Tu9UKY0MQdhu+l4blWIAxgKoMoWGkO2IZV8M JQbrn7ukiTZphGeW6QOC7ZD1EgCWs/1jRvbvAGS3b0miwnZZ5bbR9EdmhrS2dfgi O8F3gtq68AcaUYSJfw7G1ja0Z0N+GfY7yStKe1OxYQJLABr9SDT5V37dW0K9biRG VNwFvgDVSN9+uaiNecd7 =ijzB -----END PGP SIGNATURE----- --8Ne50Y5IYlza1jvg-- From owner-freebsd-arch@FreeBSD.ORG Tue Jul 15 22:52:43 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A29FB2E6; Tue, 15 Jul 2014 22:52:43 +0000 (UTC) Received: from mail-qg0-x22e.google.com (mail-qg0-x22e.google.com [IPv6:2607:f8b0:400d:c04::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5715D2E6C; Tue, 15 Jul 2014 22:52:43 +0000 (UTC) Received: by mail-qg0-f46.google.com with SMTP id z60so67409qgd.19 for ; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=aSOMGrTJZXB82yk4BJ8BorMlH2iLLEVWb0th88MjbZc=; b=uo7fuXEt4CTnvJ2c+4bDMPz/z5DjpiP87AtUE9dveQETy/Vz+zqKNTpx9DA/awuMvE x1iuznOwqB9YEwzkmsx0brYmjAJy+i59OJGVqYtvJkSRt/B6BaZcT8cOmXsGL8Ow6gdn KrMpuKe/rA/eYl4wgK53HOM/gj60TNGvMfK0yRRTdHU6U9pxCuYTKUwNP7Kjk94r+naV tWxlXFx7rvpTK0iDLAmL5Cjics6F0l70Y1HJ3AVY7tqciYJE84YXB4H3VjtVazxxDAXE yn7azr7nWfACj1RJtGvLK819c0XYOfduRloH2lJhGfmtEeMS3EOhfSJKxLSaT5iP8fiq 3SiQ== MIME-Version: 1.0 X-Received: by 10.140.89.18 with SMTP id u18mr38478104qgd.87.1405464762428; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 15:52:42 -0700 (PDT) Date: Tue, 15 Jul 2014 15:52:42 -0700 X-Google-Sender-Auth: SIstEZ94FVCCWHZOkZJ-40V12_A Message-ID: Subject: ixgbe and igb - how many queues? From: Adrian Chadd To: "freebsd-arch@freebsd.org" , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jul 2014 22:52:43 -0000 hiya, How many queues does the ixgbe and igb hardware support? The documentation reflects many more transmit and receive queues than the 8 currently auto-configured. For RSS configuration I'd like to expand it out to the 16 that I _think_ the hardware supports, but I'd like some confirmation that the documentation mostly reflects the reality out there and I wouldn't be breaking things on other peoples obscure/onboard/laptop/etc intel platforms. Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 00:04:06 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 296FA925; Wed, 16 Jul 2014 00:04:06 +0000 (UTC) Received: from mail-oa0-x22b.google.com (mail-oa0-x22b.google.com [IPv6:2607:f8b0:4003:c02::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D108B24A7; Wed, 16 Jul 2014 00:04:05 +0000 (UTC) Received: by mail-oa0-f43.google.com with SMTP id i7so172844oag.2 for ; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UMZo+wW62WvzdTs7XM8Vrr4J0JMp/xwQ9aCZcj6B86M=; b=CcTY6RM1Xh0HZ7m30B44JlL1lamRGlcfmqnhgec8poHWoRfBiT3cXTV6E7lbFdUQBa L8yi0iFvroPKyGs5CbtFv22rt00lOxjxg4P/xr5Gj/8DnukQLKQ1JjezJ955OeEQpP4b zRlabGDrIQX7oS2H1vkSErWtfAUlF+evbN6/t5QylrVz7zJDzWTyNnasNDXqIqnmoD7P Uy0ht/qP3/jitv3q0w7+x/I65BTgsA01Z5JBcg0bPRFxMQmvgtX7kxaBKsqRpIh8iWgg ocJEW0LfwjK7Yy2VIp27s1hOMdS58lq8JT9dwL7t+fzYT4LTB8K0ctxEaQ2L1ILtnknb kHDw== MIME-Version: 1.0 X-Received: by 10.182.241.130 with SMTP id wi2mr30381342obc.27.1405469045214; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) Received: by 10.76.213.7 with HTTP; Tue, 15 Jul 2014 17:04:05 -0700 (PDT) In-Reply-To: References: Date: Tue, 15 Jul 2014 20:04:05 -0400 Message-ID: Subject: Re: ixgbe and igb - how many queues? From: Ryan Stone To: Adrian Chadd Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:04:06 -0000 The oldest hardware supported by the ixgbe driver is the 82598, which supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). I believe that the 82599 and X520 are more capable. I have no idea what the situation is on igb, as there seems to be a lot more hardware supported by that driver and I don't work with the 1GB devices very much. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 00:49:00 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C7A0559C; Wed, 16 Jul 2014 00:49:00 +0000 (UTC) Received: from mga03.intel.com (mga03.intel.com [143.182.124.21]) by mx1.freebsd.org (Postfix) with ESMTP id 7D3F82818; Wed, 16 Jul 2014 00:49:00 +0000 (UTC) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga101.ch.intel.com with ESMTP; 15 Jul 2014 17:47:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,668,1400050800"; d="scan'208";a="457453656" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by azsmga001.ch.intel.com with ESMTP; 15 Jul 2014 17:47:51 -0700 Received: from orsmsx152.amr.corp.intel.com (10.22.226.39) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 15 Jul 2014 17:47:50 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.5]) by ORSMSX152.amr.corp.intel.com ([169.254.8.128]) with mapi id 14.03.0123.003; Tue, 15 Jul 2014 17:47:50 -0700 From: "Pieper, Jeffrey E" To: Ryan Stone , Adrian Chadd Subject: RE: ixgbe and igb - how many queues? Thread-Topic: ixgbe and igb - how many queues? Thread-Index: AQHPoImBUZxqX6tP/0aByQuTV1j2BJuh2KXA Date: Wed, 16 Jul 2014 00:47:50 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:49:00 -0000 82598, 82599, X520, and X540 can have up to 64 Tx/Rx queues (1 per CPU), bu= t are limited to 16 RSS queues.=20 For igb it varies, depending on HW. I can put together a list. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Ryan Stone Sent: Tuesday, July 15, 2014 5:04 PM To: Adrian Chadd Cc: FreeBSD Net; freebsd-arch@freebsd.org Subject: Re: ixgbe and igb - how many queues? The oldest hardware supported by the ixgbe driver is the 82598, which supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). I believe that the 82599 and X520 are more capable. I have no idea what the situation is on igb, as there seems to be a lot more hardware supported by that driver and I don't work with the 1GB devices very much. _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 00:49:53 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E81286CF; Wed, 16 Jul 2014 00:49:53 +0000 (UTC) Received: from mail-qc0-x236.google.com (mail-qc0-x236.google.com [IPv6:2607:f8b0:400d:c01::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 973E6282D; Wed, 16 Jul 2014 00:49:53 +0000 (UTC) Received: by mail-qc0-f182.google.com with SMTP id r5so151123qcx.13 for ; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=RJl4Z7zxfRGoKJEp77aVtTR8FOpfhxfNaJjL8cHrifY=; b=asztDofMTuA8FYTRMNjZEoUwuGrrUg3wMnQLMHhWbJYZdGEwy3D+q3aA0b3AjCBG29 e0rWUBkA4jT9l7eJxxV99yyMszEp+VjF8lv/AzTIGCnVN5tygBd+cI9osVH976Qh0lmF J+moL4aipCmpIxSfHw04cFNeV/tqXZaSDzhdGTWdCILlvmAIcQRAwEYULstlc/j1Ei4W lfyT0A9nNEwRyeyINRIP3gCOlsl+i7ELxn3wk16UpdQUnhVo+9WCtYnSWXTb1j7pNEpO got7WguhLWpkjibj4lQEej1UxjuGVjaZMcyobR54iMQMeyXDbpUUBHAJ/2klMcyuz5jj yG3Q== MIME-Version: 1.0 X-Received: by 10.224.130.201 with SMTP id u9mr37977348qas.98.1405471792714; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 17:49:52 -0700 (PDT) In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> References: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> Date: Tue, 15 Jul 2014 17:49:52 -0700 X-Google-Sender-Auth: -MmkgzV3fMZQnmcQeSoRMjvwO-Q Message-ID: Subject: Re: ixgbe and igb - how many queues? From: Adrian Chadd To: "Pieper, Jeffrey E" Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 00:49:54 -0000 On 15 July 2014 17:47, Pieper, Jeffrey E wrote: > 82598, 82599, X520, and X540 can have up to 64 Tx/Rx queues (1 per CPU), but are limited to 16 RSS queues. > For igb it varies, depending on HW. I can put together a list. That'd be great, thanks! -a From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 01:01:42 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 300359A3; Wed, 16 Jul 2014 01:01:42 +0000 (UTC) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx1.freebsd.org (Postfix) with ESMTP id 0AEFD2988; Wed, 16 Jul 2014 01:01:41 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP; 15 Jul 2014 18:01:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,668,1400050800"; d="scan'208";a="562368945" Received: from orsmsx109.amr.corp.intel.com ([10.22.240.7]) by fmsmga001.fm.intel.com with ESMTP; 15 Jul 2014 18:01:34 -0700 Received: from orsmsx153.amr.corp.intel.com (10.22.226.247) by ORSMSX109.amr.corp.intel.com (10.22.240.7) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 15 Jul 2014 18:01:34 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.5]) by ORSMSX153.amr.corp.intel.com ([169.254.12.126]) with mapi id 14.03.0123.003; Tue, 15 Jul 2014 18:01:34 -0700 From: "Pieper, Jeffrey E" To: Hooman Fazaeli , Adrian Chadd Subject: RE: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Topic: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing Thread-Index: AQHPn8XtuhYKXvJgqkGKs4ElIQPwu5uhTKMAgACVKiA= Date: Wed, 16 Jul 2014 01:01:33 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> References: <53C4EE00.5090705@gmail.com> In-Reply-To: <53C4EE00.5090705@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: FreeBSD Net , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:01:42 -0000 I believe the Linux driver DOES however support UDP RSS port hash. It is us= ed for environments where you can ensure there is no fragmentation. In such= use cases, this allows for a significant performance benefit. Jeff -----Original Message----- From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-net@freebsd.org] = On Behalf Of Hooman Fazaeli Sent: Tuesday, July 15, 2014 2:02 AM To: Adrian Chadd Cc: FreeBSD Net; freebsd-arch@freebsd.org Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with ha= rdware hashing On 7/15/2014 5:14 AM, Adrian Chadd wrote: > Hi, > > Whilst digging into UDP receive side scaling on the intel ixgbe(4) > NIC, I stumbled across how it hashes traffic between IP fragmented > traffic and non IP-fragmented traffic. > > Here's how it surfaced: > > * the ixgbe(4) NIC is configured to hash on both IP (2-tuple) and > TCP/UDP (4-tuple); > * when a non-fragmented UDP frame comes in, it's hashed on the 4-tuple > and comes into queue A; > * when a fragmented UDP frame comes in, it's hashed on the IP 2-tuple > and comes into queue B. > > So if there's a mix of small and large datagrams, we'll end up with > some packets coming in via queue A and some by queue B. In normal > operation that'll result in out of order packets. > > For the RSS stuff I'm working on it means that some packets will match > the PCBGROUP setup and some won't. By default UDP configures a 2-tuple > hash so it expects packets to come in hashed appropriately. But that > only matches for large frames. For small frames it'll be hashed via > the 4-tuple and it won't match. > > The ip reassembly code doesn't recalculate the flowid/flowtype once > it's finished. It'd be nice to do that before further processing so it > can be placed in the right netisr. > > So there's a couple of semi-overlapping issues: > > * Right now we could get TCP and UDP frames out of order. I'd like to > at least have ixgbe(4) hash on the 2-tuple for UDP rather than the > 4-tuple. That fixes that silly corner case. It's not likely going to > show up except for things like forwarding workloads. Maybe people > doing memcached work, I'm not sure. > > * Whether or not to calculate the flowid/flowtype in ip_reass() (or > maybe in the netisr input path, in case there's no flowid assigned) so > work is better distributed; > > * .. then if we do that, we could do 4-tuple UDP hashing again and > we'd just recalculate for any large frames. > > Here's what happened with Linux and ixgbe in 2010 on this topic: > > http://comments.gmane.org/gmane.linux.network/166687 > > What do people think? > > > -a > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >Doesn't the problem applies to TCP too? >TCP may be fragmented too but is less likely because of MSS. > >--=20 > >Best regards. >Hooman Fazaeli _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 01:24:19 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8EB3AF05 for ; Wed, 16 Jul 2014 01:24:19 +0000 (UTC) Received: from mail-oa0-f49.google.com (mail-oa0-f49.google.com [209.85.219.49]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 530CE2B40 for ; Wed, 16 Jul 2014 01:24:19 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id eb12so237641oac.8 for ; Tue, 15 Jul 2014 18:24:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=Zd6oBsPkKN0eAXNPakUamXFBkz4ZPZ10fNGtT2UFKLM=; b=eFvqaBg932PFK6ixXYzSfbmQDR5T0rCHer+dr679nwOQQYKMnK1yM8tZlG52048TSC lMdH5aVGqE8yFj2C352E5N1JViB4HDInWwh4rbWGZ08viw1KMJnr3shtlUNiDfYUd1CW MIe8LDyj12WMLBcvohU+ZhkTtffp76i6F0cmT0dkqKdZplmbA9Uj5Dm1xaWko2i3X1sb crN/IFmEcnM9baSqvxvRrhNneWGk3vgao8IpEf3yzdgCWunWiUOOwk5GWReLjirzDhry H0HKCSlrqTiLbR0owhkP/bUJIFbrARFs+SkkhnKj41nNLDJhsvvmj4yen5kDQDIOAmFd xFmQ== X-Gm-Message-State: ALoCoQk0wnT+auyH7Azc/XWe20uqri8TQEJ4pIQlh+NUbfm/jnyOTgp2obJD0sLmxUfDySTR7iNF X-Received: by 10.182.142.42 with SMTP id rt10mr15504669obb.80.1405473852309; Tue, 15 Jul 2014 18:24:12 -0700 (PDT) Received: from [172.21.0.13] (65-36-83-120.static.grandenetworks.net. [65.36.83.120]) by mx.google.com with ESMTPSA id v8sm32900930obo.7.2014.07.15.18.24.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jul 2014 18:24:11 -0700 (PDT) References: Mime-Version: 1.0 (1.0) In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <09C05822-716B-4D78-A47E-7ADE4226546D@netgate.com> X-Mailer: iPhone Mail (11D257) From: Jim Thompson Subject: Re: ixgbe and igb - how many queues? Date: Tue, 15 Jul 2014 20:24:11 -0500 To: Ryan Stone Cc: FreeBSD Net , Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:24:19 -0000 But only 8 per VF. -- Jim > On Jul 15, 2014, at 19:04, Ryan Stone wrote: > > The oldest hardware supported by the ixgbe driver is the 82598, which > supports up to 16 RSS queues (see Table 3-48 in the 82598 datasheet). > I believe that the 82599 and X520 are more capable. > > I have no idea what the situation is on igb, as there seems to be a > lot more hardware supported by that driver and I don't work with the > 1GB devices very much. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 01:41:35 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD6961AB; Wed, 16 Jul 2014 01:41:35 +0000 (UTC) Received: from mail-qc0-x22c.google.com (mail-qc0-x22c.google.com [IPv6:2607:f8b0:400d:c01::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE4F2C28; Wed, 16 Jul 2014 01:41:35 +0000 (UTC) Received: by mail-qc0-f172.google.com with SMTP id l6so184643qcy.31 for ; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=qVdL5AYYcrH6sI1Xlk8UYZfFh/iXccnGKUmLW64pG+M=; b=Aw0zb2pE8IN+wuCSGYcCZZFztuIUW1XAz+qPs2KybgCVMkRoYuDHepUwprmENRaE1j DXDHaqW4TlmlDi7yCGwknpFJ36gPfVLcVwXvp36y4x22q8zTFH0DCPavgQMTkXWazEZn SaToyq3eh9zk1GcmqnY10O1MpmKjlAqswVFdzhxgsX4DHZ0Mw4eIt0xElzb1Xb+sPV5W bkAnMlMIEhU/JaWFuMnLoHYKX/swnwt/5Wj8U2mgL/yW95CRZ1pTW54ZcLSWW26NwQWE ialSTViF299gRp0A7qquTJi5x2yE2ApIeLcJH3KjHTr6k4Sv1nQd3K4X/fuGmlMryoJ7 vgpw== MIME-Version: 1.0 X-Received: by 10.140.88.243 with SMTP id t106mr24164894qgd.12.1405474894742; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.224.202.193 with HTTP; Tue, 15 Jul 2014 18:41:34 -0700 (PDT) In-Reply-To: <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> References: <53C4EE00.5090705@gmail.com> <2A35EA60C3C77D438915767F458D65687E8FC789@ORSMSX111.amr.corp.intel.com> Date: Tue, 15 Jul 2014 18:41:34 -0700 X-Google-Sender-Auth: BA0wfbTNtPsFAkkl6s4E4QBbpkk Message-ID: Subject: Re: UDP/TCP versus IP frames - subtle out of order packets with hardware hashing From: Adrian Chadd To: "Pieper, Jeffrey E" Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Net , Hooman Fazaeli , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 01:41:36 -0000 On 15 July 2014 18:01, Pieper, Jeffrey E wrote: > I believe the Linux driver DOES however support UDP RSS port hash. It is used for environments where you can ensure there is no fragmentation. In such use cases, this allows for a significant performance benefit. > Right. For my RSS hacking, I've added a new in_rss.c function that returns the currently supported hashes, so it'll be possible for an administrator to define whether to support UDP hashing on 4-tuple or 2-tuple. I plan on adding a software hash calculation function to optionally calculate the 2-tuple or 4-tuple RSS hash as appropriately required so it can be run after IP reassembly if it's required. So, if the admin wants a 4-tuple UDP hash and it receives fragmented UDP, then it will re-assemble, calculate 4-tuple and kick it up. If they want 2-tuple, then it'll only calculate a software hash if it wasn't given one by the driver. Argh, so many weird corner cases to handle and I haven't even started hacking on the memcached/varnish stuff :( Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 12:14:11 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 668B4BB for ; Wed, 16 Jul 2014 12:14:11 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 21F8F20FE for ; Wed, 16 Jul 2014 12:14:10 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.82 (FreeBSD)) (envelope-from ) id 1X7O63-000Hfn-JL for freebsd-arch@freebsd.org; Wed, 16 Jul 2014 16:14:07 +0400 Date: Wed, 16 Jul 2014 16:14:07 +0400 From: Slawa Olhovchenkov To: freebsd-arch@freebsd.org Subject: Re: ixgbe and igb - how many queues? Message-ID: <20140716121407.GA67137@zxy.spb.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 12:14:11 -0000 On Tue, Jul 15, 2014 at 03:52:42PM -0700, Adrian Chadd wrote: > How many queues does the ixgbe and igb hardware support? > > The documentation reflects many more transmit and receive queues than > the 8 currently auto-configured. > > For RSS configuration I'd like to expand it out to the 16 that I > _think_ the hardware supports, but I'd like some confirmation that the > documentation mostly reflects the reality out there and I wouldn't be > breaking things on other peoples obscure/onboard/laptop/etc intel > platforms. I am configure ixgbe queues count to half of cores (all cores of one chip). Other chip fully occupied by nginx. Now I am read http://sigops.org/sosp/sosp13/papers/p33-david.pdf and wish try more optimize solution with fully locale memory reservation. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 16 21:01:52 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 03D58C43; Wed, 16 Jul 2014 21:01:52 +0000 (UTC) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx1.freebsd.org (Postfix) with ESMTP id C5953233F; Wed, 16 Jul 2014 21:01:51 +0000 (UTC) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 16 Jul 2014 13:55:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,673,1400050800"; d="scan'208";a="544419551" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga001.jf.intel.com with ESMTP; 16 Jul 2014 14:00:40 -0700 Received: from orsmsx111.amr.corp.intel.com ([169.254.11.231]) by ORSMSX106.amr.corp.intel.com ([169.254.5.72]) with mapi id 14.03.0123.003; Wed, 16 Jul 2014 14:00:39 -0700 From: "Pieper, Jeffrey E" To: Adrian Chadd Subject: RE: ixgbe and igb - how many queues? Thread-Topic: ixgbe and igb - how many queues? Thread-Index: AQHPoImBUZxqX6tP/0aByQuTV1j2BJuh2KXAgAB7VQCAANquwA== Date: Wed, 16 Jul 2014 21:00:39 +0000 Message-ID: <2A35EA60C3C77D438915767F458D65687E909341@ORSMSX111.amr.corp.intel.com> References: <2A35EA60C3C77D438915767F458D65687E8FC755@ORSMSX111.amr.corp.intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 Cc: FreeBSD Net , Ryan Stone , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2014 21:01:52 -0000 DQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBhZHJpYW4uY2hhZGRAZ21haWwu Y29tIFttYWlsdG86YWRyaWFuLmNoYWRkQGdtYWlsLmNvbV0gT24gQmVoYWxmIE9mIEFkcmlhbiBD aGFkZA0KU2VudDogVHVlc2RheSwgSnVseSAxNSwgMjAxNCA1OjUwIFBNDQpUbzogUGllcGVyLCBK ZWZmcmV5IEUNCkNjOiBSeWFuIFN0b25lOyBGcmVlQlNEIE5ldDsgZnJlZWJzZC1hcmNoQGZyZWVi c2Qub3JnDQpTdWJqZWN0OiBSZTogaXhnYmUgYW5kIGlnYiAtIGhvdyBtYW55IHF1ZXVlcz8NCg0K T24gMTUgSnVseSAyMDE0IDE3OjQ3LCBQaWVwZXIsIEplZmZyZXkgRSA8amVmZnJleS5lLnBpZXBl ckBpbnRlbC5jb20+IHdyb3RlOg0KPiA4MjU5OCwgODI1OTksIFg1MjAsIGFuZCBYNTQwIGNhbiBo YXZlIHVwIHRvIDY0IFR4L1J4IHF1ZXVlcyAoMSBwZXIgQ1BVKSwgYnV0IGFyZSBsaW1pdGVkIHRv IDE2IFJTUyBxdWV1ZXMuDQo+IEZvciBpZ2IgaXQgdmFyaWVzLCBkZXBlbmRpbmcgb24gSFcuIEkg Y2FuIHB1dCB0b2dldGhlciBhIGxpc3QuDQo+DQo+VGhhdCdkIGJlIGdyZWF0LCB0aGFua3MhDQo+ DQo+DQo+LWENCg0KSSB3YXMgaW5jb3JyZWN0IGFib3V0IHRoZSBUeC9SeCBxdWV1ZXMgZm9yIGl4 Z2JlOg0KDQotaWdiLQ0KODI1NzUgLSAgIDQgVHgvUngsICAgNCBSU1MNCjgyNTc2IC0gMTYgVHgv UngsIDE2IFJTUw0KODI1ODAgLSAgIDggVHgvUngsICA4IFJTUw0KaTM1MCAgLSAgICAgIDggVHgv UngsICA4IFJTUw0KaTIxMCAgLSAgICAgIDQgVHgvUngsICA0IFJTUw0KaTIxMSAgLSAgICAgIDIg VHgvUngsICAyIFJTUw0KDQotaXhnYmUtDQo4MjU5OCAtICAgNjQgUngvMzIgVHgsIDE2IFJTUw0K ODI1OTkgLSAxMjggUngvVHgsICAgICAgMTYgUlNTDQpYNTQwIC0gICAxMjggUngvVHgsICAgICAg MTYgUlNTDQoNCkhvcGUgdGhpcyBoZWxwcywNCg0KSmVmZg0KDQoNCg==