From owner-freebsd-arch@FreeBSD.ORG Sun Apr 4 02:13:21 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79311106566C; Sun, 4 Apr 2010 02:13:21 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (unknown [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 52BB88FC08; Sun, 4 Apr 2010 02:13:21 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id o342D9cb074495 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 3 Apr 2010 19:13:09 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id o342D9Ol074494; Sat, 3 Apr 2010 19:13:09 -0700 (PDT) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA03194; Sat, 3 Apr 10 18:02:24 PST Date: Sat, 03 Apr 2010 19:01:52 -0700 From: perryh@pluto.rain.com To: mail25@bzerk.org Message-Id: <4bb7f310.gIt41bTtJElBEKHS%perryh@pluto.rain.com> References: <20100402165002.71A8B1CC09@ptavv.es.net> <77FCD9C7615944DBBD3369EC4E5A5C94@rivendell> <2972DD89-7D7D-4869-9280-305BACBFEC5A@bway.net> <20100403135703.GA63262@ei.bzerk.org> In-Reply-To: <20100403135703.GA63262@ei.bzerk.org> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Results of BIND RFC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Apr 2010 02:13:21 -0000 Ruben de Groot wrote: > defer all questions about moving out of the base system ... Last I knew, X was not _in_ the base system :) From owner-freebsd-arch@FreeBSD.ORG Sun Apr 4 17:51:41 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC219106564A for ; Sun, 4 Apr 2010 17:51:41 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id B1E738FC0C for ; Sun, 4 Apr 2010 17:51:41 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id 1987D35A847 for ; Sun, 4 Apr 2010 19:51:41 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 058A217531; Sun, 4 Apr 2010 19:51:41 +0200 (CEST) Date: Sun, 4 Apr 2010 19:51:40 +0200 From: Jilles Tjoelker To: freebsd-arch@freebsd.org Message-ID: <20100404175140.GB40499@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Subject: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Apr 2010 17:51:42 -0000 The SX locks so_snd_sx and so_rcv_sx can be legitimately held for arbitrary amounts of time, while waiting until a send or recv is possible. Any other threads wanting to send/recv on the socket will sleep interruptibly on the corresponding SX. If deadlkres is activated, it may detect a deadlock even though there is no problem. If a SIGSTOP or similar comes in while waiting until send/recv is possible, the SX is held across the suspension and noone will be able to send/recv on the socket until the process is resumed. On the other hand a thread waiting for the SX can be suspended without harm. Adding PBDRY to various sleeps may help but may also introduce other problems (SIGSTOP disturbing the functioning of the process more, possibly with stuff like SO_RCVTIMEO). Example (using the fact that fifos are implemented using sockets): term1% mkfifo testfifo term1% cat >testfifo term2% cat testfifo term3% cat testfifo Letting this sit for half an hour will trigger deadlkres. If the reader in term2 (started first) is suspended, the reader in term3 will not get any input written to the fifo. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sun Apr 4 18:30:15 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40201106566C for ; Sun, 4 Apr 2010 18:30:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id B4C9A8FC13 for ; Sun, 4 Apr 2010 18:30:14 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o34ICXQH091413 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 4 Apr 2010 21:12:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o34ICXut087210; Sun, 4 Apr 2010 21:12:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o34ICXTV087209; Sun, 4 Apr 2010 21:12:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 4 Apr 2010 21:12:33 +0300 From: Kostik Belousov To: Jilles Tjoelker Message-ID: <20100404181233.GH2415@deviant.kiev.zoral.com.ua> References: <20100404175140.GB40499@stack.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ie9RhXlkjyREuElA" Content-Disposition: inline In-Reply-To: <20100404175140.GB40499@stack.nl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Apr 2010 18:30:15 -0000 --ie9RhXlkjyREuElA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Apr 04, 2010 at 07:51:40PM +0200, Jilles Tjoelker wrote: > The SX locks so_snd_sx and so_rcv_sx can be legitimately held for > arbitrary amounts of time, while waiting until a send or recv is > possible. Any other threads wanting to send/recv on the socket will > sleep interruptibly on the corresponding SX. If deadlkres is activated, > it may detect a deadlock even though there is no problem. >=20 > If a SIGSTOP or similar comes in while waiting until send/recv is > possible, the SX is held across the suspension and noone will be able to > send/recv on the socket until the process is resumed. On the other hand > a thread waiting for the SX can be suspended without harm. Adding PBDRY > to various sleeps may help but may also introduce other problems > (SIGSTOP disturbing the functioning of the process more, possibly with > stuff like SO_RCVTIMEO). Could you, please, elaborate on this (interaction between PBDRY and some socket timeout stuff) ? >=20 > Example (using the fact that fifos are implemented using sockets): >=20 > term1% mkfifo testfifo > term1% cat >testfifo >=20 > term2% cat testfifo >=20 > term3% cat testfifo >=20 > Letting this sit for half an hour will trigger deadlkres. >=20 > If the reader in term2 (started first) is suspended, the reader in term3 > will not get any input written to the fifo. >=20 > --=20 > Jilles Tjoelker > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" --ie9RhXlkjyREuElA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAku41pEACgkQC3+MBN1Mb4he3wCg5ANH7iJrD7n3ayYPwB7hVRbY sWwAoMyZhD+fmq1vTWpYK4bzK0kcaD56 =aQm1 -----END PGP SIGNATURE----- --ie9RhXlkjyREuElA-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 4 20:40:58 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C5141065678 for ; Sun, 4 Apr 2010 20:40:58 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id 33E828FC45 for ; Sun, 4 Apr 2010 20:40:58 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id 9E88735A844; Sun, 4 Apr 2010 22:40:57 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 8A0B717531; Sun, 4 Apr 2010 22:40:57 +0200 (CEST) Date: Sun, 4 Apr 2010 22:40:57 +0200 From: Jilles Tjoelker To: Kostik Belousov Message-ID: <20100404204057.GA43042@stack.nl> References: <20100404175140.GB40499@stack.nl> <20100404181233.GH2415@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100404181233.GH2415@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Apr 2010 20:40:58 -0000 On Sun, Apr 04, 2010 at 09:12:33PM +0300, Kostik Belousov wrote: > On Sun, Apr 04, 2010 at 07:51:40PM +0200, Jilles Tjoelker wrote: > > The SX locks so_snd_sx and so_rcv_sx can be legitimately held for > > arbitrary amounts of time, while waiting until a send or recv is > > possible. Any other threads wanting to send/recv on the socket will > > sleep interruptibly on the corresponding SX. If deadlkres is activated, > > it may detect a deadlock even though there is no problem. > > If a SIGSTOP or similar comes in while waiting until send/recv is > > possible, the SX is held across the suspension and noone will be able to > > send/recv on the socket until the process is resumed. On the other hand > > a thread waiting for the SX can be suspended without harm. Adding PBDRY > > to various sleeps may help but may also introduce other problems > > (SIGSTOP disturbing the functioning of the process more, possibly with > > stuff like SO_RCVTIMEO). > Could you, please, elaborate on this (interaction between PBDRY and some > socket timeout stuff) ? A SIGSTOP will currently not affect a SO_RCVTIMEO timeout, just like it does not affect a nanosleep(2). Demonstration is analogous to my earlier example with /bin/sleep, where ./socktimeout is the program below: % /usr/bin/time ./socktimeout ^Z % fg # such that the program can run again 10 seconds after starting socktimeout: read: Resource temporarily unavailable 10.00 real 0.00 user 0.00 sys % PBDRY would cause an immediate Interrupted system call return. I don't know how much SO_RCVTIMEO/SO_SNDTIMEO are used (a POSIX application cannot rely on their support for any kind of socket), and it is likely applications using it can already get EINTR from signal handlers. #include #include #include #include #include #include int main(int argc, char *argv[]) { int sock; sock = socket(AF_LOCAL, SOCK_DGRAM, 0); if (sock == -1) err(1, "socket"); if (setsockopt(sock, SOL_SOCKET, SO_RCVTIMEO, &(const struct timeval){ 10, 0 }, sizeof(struct timeval)) == -1) err(1, "setsockopt(SO_RCVTIMEO)"); errno = 0; if (read(sock, (char[]){ 0 }, 1) != 1) err(1, "read"); return 0; } -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sun Apr 4 21:15:40 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11424106566B; Sun, 4 Apr 2010 21:15:40 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail17.syd.optusnet.com.au (mail17.syd.optusnet.com.au [211.29.132.198]) by mx1.freebsd.org (Postfix) with ESMTP id 8D32A8FC0A; Sun, 4 Apr 2010 21:15:39 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c122-106-253-149.belrs3.nsw.optusnet.com.au [122.106.253.149]) by mail17.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o34LFOqb028268 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 5 Apr 2010 07:15:25 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id o34LFO3f020187; Mon, 5 Apr 2010 07:15:24 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id o34LFMaU020186; Mon, 5 Apr 2010 07:15:22 +1000 (EST) (envelope-from peter) Date: Mon, 5 Apr 2010 07:15:22 +1000 From: Peter Jeremy To: perryh@pluto.rain.com Message-ID: <20100404211522.GI86236@server.vk2pj.dyndns.org> References: <20100402165002.71A8B1CC09@ptavv.es.net> <77FCD9C7615944DBBD3369EC4E5A5C94@rivendell> <2972DD89-7D7D-4869-9280-305BACBFEC5A@bway.net> <20100403135703.GA63262@ei.bzerk.org> <4bb7f310.gIt41bTtJElBEKHS%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IbA9xpzOQlG26JSn" Content-Disposition: inline In-Reply-To: <4bb7f310.gIt41bTtJElBEKHS%perryh@pluto.rain.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) X-CMAE-Score: 0 Cc: mail25@bzerk.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Results of BIND RFC X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Apr 2010 21:15:40 -0000 --IbA9xpzOQlG26JSn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Apr-03 19:01:52 -0700, perryh@pluto.rain.com wrote: >Ruben de Groot wrote: >> defer all questions about moving out of the base system ... > >Last I knew, X was not _in_ the base system :) Well, that's an excellent topic for another bikeshed - Should X be made part of the base system? I know it is on OpenBSD. :-) :-) --=20 Peter Jeremy --IbA9xpzOQlG26JSn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAku5AWoACgkQ/opHv/APuIcBAwCdE7yXAEp94LGfpC0aPc+E1GB1 8l4An3yWcDE96SvR7xlk9cnRCgz8sGPB =/jLJ -----END PGP SIGNATURE----- --IbA9xpzOQlG26JSn-- From owner-freebsd-arch@FreeBSD.ORG Mon Apr 5 11:06:56 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1500A1065672 for ; Mon, 5 Apr 2010 11:06:56 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id DDACA8FC15 for ; Mon, 5 Apr 2010 11:06:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o35B6tg3027735 for ; Mon, 5 Apr 2010 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o35B6tfD027733 for freebsd-arch@FreeBSD.org; Mon, 5 Apr 2010 11:06:55 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 5 Apr 2010 11:06:55 GMT Message-Id: <201004051106.o35B6tfD027733@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Apr 2010 11:06:56 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Apr 5 22:13:32 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 57D711065678 for ; Mon, 5 Apr 2010 22:13:32 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3471D8FC26 for ; Mon, 5 Apr 2010 22:13:32 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id B2D5C46B29; Mon, 5 Apr 2010 18:13:31 -0400 (EDT) Date: Mon, 5 Apr 2010 23:13:31 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Jilles Tjoelker In-Reply-To: <20100404175140.GB40499@stack.nl> Message-ID: References: <20100404175140.GB40499@stack.nl> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Apr 2010 22:13:32 -0000 On Sun, 4 Apr 2010, Jilles Tjoelker wrote: > The SX locks so_snd_sx and so_rcv_sx can be legitimately held for arbitrary > amounts of time, while waiting until a send or recv is possible. Any other > threads wanting to send/recv on the socket will sleep interruptibly on the > corresponding SX. If deadlkres is activated, it may detect a deadlock even > though there is no problem. Actually, on the whole, socket buffer sx lock acquisition is interruptible. There are a few exceptions, such as during socket shutdown, and in an RPC layer or two, but in the general case we want Ctrl-C to interrupt recv() and send(), so use interruptible sleeps. Robert From owner-freebsd-arch@FreeBSD.ORG Tue Apr 6 05:18:26 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F0411065670 for ; Tue, 6 Apr 2010 05:18:26 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 0E87F8FC2B for ; Tue, 6 Apr 2010 05:18:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o365DxWJ004184 for ; Mon, 5 Apr 2010 23:13:59 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 05 Apr 2010 23:09:35 -0600 (MDT) Message-Id: <20100405.230935.131509470607402730.imp@bsdimp.com> To: arch@freebsd.org From: "M. Warner Losh" X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: Minor make cleanup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2010 05:18:26 -0000 Greetings, I'd like to simplify how make gets its MACHINE and MACHINE_ARCH settings a little. Right now we have some special case code for pc98 that's not been needed for a while (it is to cope on running on FreeBSD/pc98 6.x kernels). Also, the MACHINE and MACHINE_ARCH is a little to complex, and can be simplified. It is now trivial to determine machine_arch at run-time, so we do that rather than relying on a hard-coded default value. I've also removed the MACHINE_CPU setting. I don't think it is needed since we always set it via sys.mk's includsion of bsd.cpu.mk and have for a very long time. It doesn't default to anything sensible anyway. Comments? Warner Index: main.c =================================================================== --- main.c (revision 206241) +++ main.c (working copy) @@ -872,7 +872,6 @@ { const char *machine; const char *machine_arch; - const char *machine_cpu; Boolean outOfDate = TRUE; /* FALSE if all targets up to date */ const char *p; const char *pathp; @@ -881,6 +880,8 @@ char obpath[MAXPATHLEN]; char cdpath[MAXPATHLEN]; char found_dir[MAXPATHLEN + 1]; /* for searching for sys.mk */ + char machine_[40]; + char machine_arch_[40]; char *cp = NULL, *start; save_argv = argv; @@ -933,61 +934,30 @@ #endif /* - * Prior to 7.0, FreeBSD/pc98 kernel used to set the - * utsname.machine to "i386", and MACHINE was defined as - * "i386", so it could not be distinguished from FreeBSD/i386. - * Therefore, we had to check machine.ispc98 and adjust the - * MACHINE variable. NOTE: The code is still here to be able - * to compile new make binary on old FreeBSD/pc98 systems, and - * have the MACHINE variable set properly. + * MACHINE is the specific type of machine that is build from + * MACHINE_ARCH processors. Often they are the same, but can + * be different in a number of circumstances. The kernel for + * this MACHINE is buils from sys/MACHINE/conf/BLAH. The cpu + * support for libc, comes from lib/libc/MACHINE_ARCH/blah. + * + * Some ports support muliple disparate types of hardware with + * a single MACHINE type (eg mips and powerpc support all + * their differnet kernels under the mips or powerpc umbrella + * respectively). */ if ((machine = getenv("MACHINE")) == NULL) { - int ispc98; - size_t len; - - len = sizeof(ispc98); - if (!sysctlbyname("machdep.ispc98", &ispc98, &len, NULL, 0)) { - if (ispc98) - machine = "pc98"; - } + size_t len = sizeof(machine_); + if (sysctlbyname("hw.machine", machine_, &len, NULL, 0) == 0) + machine = machine_; } - - /* - * Get the name of this type of MACHINE from utsname - * so we can share an executable for similar machines. - * (i.e. m68k: amiga hp300, mac68k, sun3, ...) - * - * Note that both MACHINE and MACHINE_ARCH are decided at - * run-time. - */ - if (machine == NULL) { - static struct utsname utsname; - - if (uname(&utsname) == -1) - err(2, "uname"); - machine = utsname.machine; - } - if ((machine_arch = getenv("MACHINE_ARCH")) == NULL) { -#ifdef MACHINE_ARCH - machine_arch = MACHINE_ARCH; -#else - machine_arch = "unknown"; -#endif + size_t len = sizeof(machine_arch_); + if (sysctlbyname("hw.machine_arch", machine_arch_, &len, + NULL, 0) == 0) + machine_arch = machine_arch_; } /* - * Set machine_cpu to the minumum supported CPU revision based - * on the target architecture, if not already set. - */ - if ((machine_cpu = getenv("MACHINE_CPU")) == NULL) { - if (!strcmp(machine_arch, "i386")) - machine_cpu = "i386"; - else - machine_cpu = "unknown"; - } - - /* * Initialize the parsing, directory and variable modules to prepare * for the reading of inclusion paths and variable settings on the * command line @@ -1016,7 +986,6 @@ Var_SetGlobal("MFLAGS", ""); Var_SetGlobal("MACHINE", machine); Var_SetGlobal("MACHINE_ARCH", machine_arch); - Var_SetGlobal("MACHINE_CPU", machine_cpu); #ifdef MAKE_VERSION Var_SetGlobal("MAKE_VERSION", MAKE_VERSION); #endif From owner-freebsd-arch@FreeBSD.ORG Tue Apr 6 07:10:44 2010 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 990D81065672; Tue, 6 Apr 2010 07:10:44 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mail.0x20.net (mail.0x20.net [217.69.67.217]) by mx1.freebsd.org (Postfix) with ESMTP id 548BD8FC1A; Tue, 6 Apr 2010 07:10:43 +0000 (UTC) Received: by mail.0x20.net (Postfix, from userid 1002) id 70CE0398B3; Tue, 6 Apr 2010 08:51:59 +0200 (CEST) Date: Tue, 6 Apr 2010 08:51:59 +0200 From: Lars Engels To: Andriy Gapon Message-ID: <20100406065159.GC34881@e.0x20.net> References: <20100326211706.GI18894@FreeBSD.org> <20100327102729.3bb8fba4@ernst.jennejohn.org> <1269687444.35918.9.camel@balrog.2hip.net> <4BB763BB.6090304@icyb.net.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Guc8tSoCmQxpftue" Content-Disposition: inline In-Reply-To: <4BB763BB.6090304@icyb.net.ua> X-Editor: VIM - Vi IMproved 7.2 X-Operation-System: FreeBSD 5.5-RELEASE-p19 User-Agent: Mutt/1.5.20 (2009-06-14) Cc: arch@FreeBSD.org, Gleb Smirnoff , gary.jennejohn@freenet.de, Robert Noland Subject: Re: touch panel support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2010 07:10:44 -0000 --Guc8tSoCmQxpftue Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 03, 2010 at 06:50:19PM +0300, Andriy Gapon wrote: > on 27/03/2010 12:57 Robert Noland said the following: > > I'm not certain what the best way to achieve this is, but our input > > layer is currently pretty frustrating to deal with, especially when you > > throw HAL or DeviceKit into the mix. We have to go to a lot of effort > > to figure out if mice are using moused or not, and if they should be > > advertised to X. >=20 > Just my 2 bits: horizontal mouse scroll is also not possible at the momen= t. Are you aware of this? http://wiki.freebsd.org/SynapticsTouchpad Would be nice if someone could enhance it and get it committed to base. --Guc8tSoCmQxpftue Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAku62g8ACgkQKc512sD3afiHNgCgt+IoDlJ14P2HvkR8ZPIqBbbl ScMAoI3v+gvpusGGq5cBeegWSNL1Td7f =aEZa -----END PGP SIGNATURE----- --Guc8tSoCmQxpftue-- From owner-freebsd-arch@FreeBSD.ORG Tue Apr 6 09:33:04 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE44F1065678 for ; Tue, 6 Apr 2010 09:33:04 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout2.freenet.de (mout2.freenet.de [IPv6:2001:748:100:40::2:4]) by mx1.freebsd.org (Postfix) with ESMTP id 8505A8FC25 for ; Tue, 6 Apr 2010 09:33:04 +0000 (UTC) Received: from [195.4.92.23] (helo=13.mx.freenet.de) by mout2.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #3) id 1Nz59a-0001AR-S4; Tue, 06 Apr 2010 11:33:02 +0200 Received: from p57ae1de7.dip0.t-ipconnect.de ([87.174.29.231]:20361 helo=ernst.jennejohn.org) by 13.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #3) id 1Nz59a-0003Aq-DF; Tue, 06 Apr 2010 11:33:02 +0200 Date: Tue, 6 Apr 2010 11:33:01 +0200 From: Gary Jennejohn To: "M. Warner Losh" Message-ID: <20100406113301.5a535c5c@ernst.jennejohn.org> In-Reply-To: <20100405.230935.131509470607402730.imp@bsdimp.com> References: <20100405.230935.131509470607402730.imp@bsdimp.com> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: Minor make cleanup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2010 09:33:05 -0000 On Mon, 05 Apr 2010 23:09:35 -0600 (MDT) "M. Warner Losh" wrote: > Greetings, > > I'd like to simplify how make gets its MACHINE and MACHINE_ARCH > settings a little. Right now we have some special case code for pc98 > that's not been needed for a while (it is to cope on running on > FreeBSD/pc98 6.x kernels). Also, the MACHINE and MACHINE_ARCH is a > little to complex, and can be simplified. It is now trivial to > determine machine_arch at run-time, so we do that rather than relying > on a hard-coded default value. > > I've also removed the MACHINE_CPU setting. I don't think it is needed > since we always set it via sys.mk's includsion of bsd.cpu.mk and have > for a very long time. It doesn't default to anything sensible anyway. > > Comments? > > Warner > > Index: main.c > =================================================================== > --- main.c (revision 206241) > +++ main.c (working copy) > @@ -872,7 +872,6 @@ > { > const char *machine; > const char *machine_arch; > - const char *machine_cpu; > Boolean outOfDate = TRUE; /* FALSE if all targets up to date */ > const char *p; > const char *pathp; > @@ -881,6 +880,8 @@ > char obpath[MAXPATHLEN]; > char cdpath[MAXPATHLEN]; > char found_dir[MAXPATHLEN + 1]; /* for searching for sys.mk */ > + char machine_[40]; > + char machine_arch_[40]; > char *cp = NULL, *start; > > save_argv = argv; > @@ -933,61 +934,30 @@ > #endif > > /* > - * Prior to 7.0, FreeBSD/pc98 kernel used to set the > - * utsname.machine to "i386", and MACHINE was defined as > - * "i386", so it could not be distinguished from FreeBSD/i386. > - * Therefore, we had to check machine.ispc98 and adjust the > - * MACHINE variable. NOTE: The code is still here to be able > - * to compile new make binary on old FreeBSD/pc98 systems, and > - * have the MACHINE variable set properly. > + * MACHINE is the specific type of machine that is build from > + * MACHINE_ARCH processors. Often they are the same, but can > + * be different in a number of circumstances. The kernel for > + * this MACHINE is buils from sys/MACHINE/conf/BLAH. The cpu > + * support for libc, comes from lib/libc/MACHINE_ARCH/blah. > + * > + * Some ports support muliple disparate types of hardware with > + * a single MACHINE type (eg mips and powerpc support all > + * their differnet kernels under the mips or powerpc umbrella > + * respectively). > */ > if ((machine = getenv("MACHINE")) == NULL) { > - int ispc98; > - size_t len; > - > - len = sizeof(ispc98); > - if (!sysctlbyname("machdep.ispc98", &ispc98, &len, NULL, 0)) { > - if (ispc98) > - machine = "pc98"; > - } > + size_t len = sizeof(machine_); > + if (sysctlbyname("hw.machine", machine_, &len, NULL, 0) == 0) > + machine = machine_; > } > - > - /* > - * Get the name of this type of MACHINE from utsname > - * so we can share an executable for similar machines. > - * (i.e. m68k: amiga hp300, mac68k, sun3, ...) > - * > - * Note that both MACHINE and MACHINE_ARCH are decided at > - * run-time. > - */ > - if (machine == NULL) { > - static struct utsname utsname; > - > - if (uname(&utsname) == -1) > - err(2, "uname"); > - machine = utsname.machine; > - } > - > if ((machine_arch = getenv("MACHINE_ARCH")) == NULL) { > -#ifdef MACHINE_ARCH > - machine_arch = MACHINE_ARCH; > -#else > - machine_arch = "unknown"; > -#endif > + size_t len = sizeof(machine_arch_); > + if (sysctlbyname("hw.machine_arch", machine_arch_, &len, > + NULL, 0) == 0) > + machine_arch = machine_arch_; > } > > /* > - * Set machine_cpu to the minumum supported CPU revision based > - * on the target architecture, if not already set. > - */ > - if ((machine_cpu = getenv("MACHINE_CPU")) == NULL) { > - if (!strcmp(machine_arch, "i386")) > - machine_cpu = "i386"; > - else > - machine_cpu = "unknown"; > - } > - > - /* > * Initialize the parsing, directory and variable modules to prepare > * for the reading of inclusion paths and variable settings on the > * command line > @@ -1016,7 +986,6 @@ > Var_SetGlobal("MFLAGS", ""); > Var_SetGlobal("MACHINE", machine); > Var_SetGlobal("MACHINE_ARCH", machine_arch); > - Var_SetGlobal("MACHINE_CPU", machine_cpu); > #ifdef MAKE_VERSION > Var_SetGlobal("MAKE_VERSION", MAKE_VERSION); > #endif > I know it's unlikely, but since you're checking whether sysctlbyname() succeeds or not it seems like you should do what the old code does and set the variables to "unknown" in case it does fail for completeness. Lots of typos in the comments :) -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Tue Apr 6 13:51:33 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09EAB1065676 for ; Tue, 6 Apr 2010 13:51:33 +0000 (UTC) (envelope-from alexanderchuranov@gmail.com) Received: from mail-gx0-f211.google.com (mail-gx0-f211.google.com [209.85.217.211]) by mx1.freebsd.org (Postfix) with ESMTP id B98638FC1E for ; Tue, 6 Apr 2010 13:51:32 +0000 (UTC) Received: by gxk3 with SMTP id 3so3730369gxk.13 for ; Tue, 06 Apr 2010 06:51:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:received:message-id :subject:from:to:content-type; bh=z7cZJRL/FAoxT8Ia+XwypOPlVY45oxZFSyPV90fLR88=; b=EFYmxrnjGRvy8pZzd3LlxAPM6vYxhq74QrCD/vAxjry179s1alD715epGuLqXhX90q sbYMZ9ry7AOLyc7Nu5Q4ANB1m8JFXYIKjCh6Hu5UOXiHDcd64/wKwRHJ7e3LHYftjUWk bqyw9pCqGmhpv13W4E95xJqCrIyyHxQ9czNx0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=KUF1Ez4B2iNHTgJuz90gnzHBGqv2kFvpqfR1oDuZGXXHkT+IBRgc3wyh0yl9PKntMv 1j3p3hcswjFx3pD/wMZKMpMf68dAX0YzWmcmwzWO3XtmpHkUAwN1Ggz+4aEMcC5O7s/1 rnLP0Z8KtxHrcDnX71mz4n0+T6GhYGZVyjzpA= MIME-Version: 1.0 Received: by 10.90.119.15 with HTTP; Tue, 6 Apr 2010 06:27:29 -0700 (PDT) Date: Tue, 6 Apr 2010 17:27:29 +0400 Received: by 10.91.65.19 with SMTP id s19mr2185929agk.34.1270560450165; Tue, 06 Apr 2010 06:27:30 -0700 (PDT) Message-ID: From: Alexander Churanov To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: New "scallhook" feature. Is is OK to create a proposal? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2010 13:51:33 -0000 Folks, My friend, Vladislav Soldatov, and I are going to propose and implement a new "scallhook" feature: the generic modular solution to monitoring, filtering and translating system calls. The feature differs from OpenBSD systrace: it is much more general, going to be modular and have strong foundation for security application. The project includes implementing the kernel-side code, the userland configuration utility, some of most required filtering/translating modules as well as a new handbook (otherbooks) section on configuration and extending, plus articles on the web. The future additions to the project may be a system for sandboxing application every time it is started and an extension to ports system which would automatically sandbox application when it is being installed. About me: I am software engineer, currently working in Cisco Systems, specializing in C/C++/UNIX. My additional interests are software quality and security. I am a port maintainer for devel/boost-* and was participating in extending syscons driver, until the project was superseded by syscons rewrite by Ed Schouten. About Vladislav: Vladislav is a PhD of computer science, has experience with developing in C and C++ for FreeBSD. Before writing the full proposal on the wiki, I'd like to receive the first approval. What do you think of this? Will be the feature accepted? Alexander Churanov From owner-freebsd-arch@FreeBSD.ORG Tue Apr 6 15:48:42 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 810521065672 for ; Tue, 6 Apr 2010 15:48:42 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5BE428FC15 for ; Tue, 6 Apr 2010 15:48:42 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 1A03A46B0D; Tue, 6 Apr 2010 11:48:42 -0400 (EDT) Date: Tue, 6 Apr 2010 16:48:41 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Alexander Churanov In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@freebsd.org Subject: Re: New "scallhook" feature. Is is OK to create a proposal? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Apr 2010 15:48:42 -0000 On Tue, 6 Apr 2010, Alexander Churanov wrote: > My friend, Vladislav Soldatov, and I are going to propose and implement a > new "scallhook" feature: the generic modular solution to monitoring, > filtering and translating system calls. > > The feature differs from OpenBSD systrace: it is much more general, going to > be modular and have strong foundation for security application. Hi Alexander-- This sounds like an interesting project. Could you say a bit more about how you envision such a system working, and in particular, how it might address concerns about the safety of the approach for some of the purposes you describe -- specifically, sandboxing. On the whole, OS vendors have eshewed the syscall wrapper approach due to its vulnerability to race conditions, and that concern would be critical concern in considering adopting any such system for FreeBSD. In fact, the MAC Framework, FreeBSD's extensible kernel security framework (now also used in Mac OS X), is specifically designed to avoid these sorts of races by offering integrated access to kernel data structures while holding appropriate locks. One of the ideas we have on the proposal ideas list is a path-based policy similar to Apple's Seatbelt policy module, offering lightweight sandboxing using path-based policy module. One caveat: our notion of "path" is a bit weaker than the abstraction of a path in Mac OS X, so some careful thinking woudl need to be done. Robert > > The project includes implementing the kernel-side code, the userland > configuration utility, some of most required filtering/translating modules > as well as a new handbook (otherbooks) section on configuration and > extending, plus articles on the web. The future additions to the project may > be a system for sandboxing application every time it is started and an > extension to ports system which would automatically sandbox application when > it is being installed. > > About me: > > I am software engineer, currently working in Cisco Systems, specializing in > C/C++/UNIX. My additional interests are software quality and security. I am > a port maintainer for devel/boost-* and was participating in extending > syscons driver, until the project was superseded by syscons rewrite by Ed > Schouten. > > About Vladislav: > Vladislav is a PhD of computer science, has experience with developing in C > and C++ for FreeBSD. > > Before writing the full proposal on the wiki, I'd like to receive the first > approval. > > What do you think of this? > Will be the feature accepted? > > Alexander Churanov > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Apr 7 09:41:29 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAD8A106564A for ; Wed, 7 Apr 2010 09:41:29 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id A8EDF8FC15 for ; Wed, 7 Apr 2010 09:41:29 +0000 (UTC) Received: by qyk11 with SMTP id 11so167665qyk.13 for ; Wed, 07 Apr 2010 02:41:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:received:message-id :subject:from:to:cc:content-type; bh=zfI4IvL3IjUS0VouZhOI203LUmECDw10o7FYuClWsGc=; b=BBaJ9YMhL0ZqIkOC3hQfmNYFK3ACZ1P2WAsz7chAIfRqNS/mdX3FRJIyeuPSgXvGfD nhlUvxcS99vPgKU2+qPoo+mO7jZtfVf9UqpX9Uph6qx26O9f87iNtxZ/8Cb/P1dLxYbY 7BqduGHSbZAbnFUI8/DThdDg0jdnFHmXhM5D4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=ZeC3Kpd6Yod2vdApq5ACZP7eoya/qHGzkxEHaTzpcrEi2fil0n5FdsOS7Ey9lDYg7S gle+z3c42uDvF3nwxC18i9kn2TgWSUKl9TsmuKnprqlKrMX8zKL4l4yQf9Conay7N21F 9NEZjgE7PwfIja7TncpVitXH3OPI9UKUcwI18= MIME-Version: 1.0 Received: by 10.229.33.72 with HTTP; Wed, 7 Apr 2010 02:16:37 -0700 (PDT) Date: Wed, 7 Apr 2010 02:16:37 -0700 Received: by 10.229.192.10 with SMTP id do10mr3356983qcb.48.1270631797779; Wed, 07 Apr 2010 02:16:37 -0700 (PDT) Message-ID: From: Garrett Cooper To: arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: portmgr@freebsd.org Subject: [RFC] Remove pkg_add -C (chroot functionality) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2010 09:41:30 -0000 Hi Arch folks, I've talked with flz really briefly about this, and while he said it was ok, I want to make sure that removing the functionality makes logical sense. There's an outstanding bug to fix chroot(2)'ing functionality with pkg_add(1) [1]. Anyone that has tried this functionality knows that it's currently crippled. Whilst fixing this seems trivial (apply a chroot to only the top-level caller), it would require API changes, as well as converting some of the vsystem (variadic system(3) lib-) calls to fork(3) -> exec(3) to get things working properly with chroot(2), and could in fact further add to the spaghetti mess that is pkg_install(1) today. The goal that several folks are undertaking to improve pkg_install is to simplify it by pruning away unnecessary features (wherever possible), and this functionality isn't actively used in any of the mainstream tools (portmaster, portupgrade, sysinstall, etc). Does anyone pragmatically oppose this change? If so, why? Also, I'll be sure to run it through ports@ too so people get a little bit of say in this (unless I receive an astounding no here), but given that there was only one situation where this functioned for me, I'd much rather have a complete functional solution in < 20 lines of script that I can adjust to my needs than a broken (and pretty useless) feature in pkg_add(1). Thanks, -Garrett [1] bin/109334 Reasons for removing the feature: 1. Broken. 2. There's a large degree of environment setup required in order to have a functional chroot / jail, that can't be resolved by pkg_add -C . 3. The equivalent command is simple: chroot $DESTDIR pkg_add <...> 4. Certain pkg_install(8) operations can only be performed on a live system (adding a user for instance, checking a font cache, etc). From owner-freebsd-arch@FreeBSD.ORG Wed Apr 7 13:55:31 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 812171065672; Wed, 7 Apr 2010 13:55:31 +0000 (UTC) (envelope-from v.velox@vvelox.net) Received: from vulpes.vvelox.net (vulpes.vvelox.net [99.69.115.42]) by mx1.freebsd.org (Postfix) with ESMTP id 42BF68FC22; Wed, 7 Apr 2010 13:55:30 +0000 (UTC) Received: from vixen42.vulpes.vvelox.net (unknown [192.168.14.1]) (Authenticated sender: v.velox) by vulpes.vvelox.net (Postfix) with ESMTP id 33D80B822; Wed, 7 Apr 2010 08:35:36 -0500 (CDT) Date: Wed, 7 Apr 2010 08:38:57 -0500 From: Vulpes Velox To: freebsd-arch@freebsd.org, Gleb Smirnoff Message-ID: <20100407083857.053b669c@vixen42.vulpes.vvelox.net> In-Reply-To: <20100326211706.GI18894@FreeBSD.org> References: <20100326211706.GI18894@FreeBSD.org> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; i386-portbld-freebsd7.2) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/OBvc.589WrjlldZUWz=79xV"; protocol="application/pgp-signature" Cc: Subject: Re: touch panel support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Apr 2010 13:55:31 -0000 --Sig_/OBvc.589WrjlldZUWz=79xV Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sat, 27 Mar 2010 00:17:06 +0300 Gleb Smirnoff wrote: > Hello, >=20 > I've got a display with touch panel, and I'd like to get in > working in FreeBSD. The touch panel is supported by NetBSD's > uep(4). So far, I have written uep(4) for FreeBSD, that > successfully reads and parses data from the USB touch panel device. >=20 > And then I've got a problem. Our mouse subsystem is not ready for > touch panels. Our mouse(4) protocol does not support mouse driver > passing _absolute_ coordinates to the mouse(4) subsystem. It only > expects a relative movement of the mouse. But _absolute_ > coordinates are principal idea of any touch panel. >=20 > The lesser problem is lack of generic support for touch panel > calibration. >=20 > Both of these problems are solved in NetBSD. They've got a > wsmux(4) device, just like our kbdmux(4), but for mice. This mouse > multiplexer can also understand absolute coordinates from > underlying mice drivers. NetBSD also has a generic support for > calibration of touch panels. >=20 > What is the FreeBSD future way to go: port things for NetBSD? > Write something different? I am curious, what would it take to get to get it working in a relative manner? I personally find this useful myself and generally how I prefer to use my Wacom. --Sig_/OBvc.589WrjlldZUWz=79xV Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAku8ivgACgkQqrJJy0yxYQDyigCfZARltSuanQ6Ghf938PIA+RVC TRYAn2iB2S28FQw1UuUASgbvtjh7P1Kq =5uUb -----END PGP SIGNATURE----- --Sig_/OBvc.589WrjlldZUWz=79xV-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 02:36:12 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F33A51065674; Thu, 8 Apr 2010 02:36:11 +0000 (UTC) (envelope-from kientzle@freebsd.org) Received: from monday.kientzle.com (kientzle.com [66.166.149.50]) by mx1.freebsd.org (Postfix) with ESMTP id A72518FC16; Thu, 8 Apr 2010 02:36:11 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.3/8.14.3) id o382MHYo072983; Thu, 8 Apr 2010 02:22:17 GMT (envelope-from kientzle@freebsd.org) Received: from horton.x.kientzle.com (fw2.kientzle.com [10.123.1.2]) by kientzle.com with SMTP id 8rackpa725pptxeb3j83xyc6xi; Thu, 08 Apr 2010 02:22:17 +0000 (UTC) (envelope-from kientzle@freebsd.org) Message-ID: <4BBD3DCB.4030902@freebsd.org> Date: Wed, 07 Apr 2010 19:22:03 -0700 From: Tim Kientzle User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.1.23) Gecko/20100314 SeaMonkey/1.1.18 MIME-Version: 1.0 To: Garrett Cooper References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, portmgr@freebsd.org Subject: Re: [RFC] Remove pkg_add -C (chroot functionality) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2010 02:36:12 -0000 Garrett Cooper wrote: > There's an outstanding bug to fix chroot(2)'ing functionality with > pkg_add(1) [1]. Anyone that has tried this functionality knows that > it's currently crippled If it's currently broken, it's better to simply remove it. I'm not sure I understand why anyone wants pkg_add to call chroot(2) at the top, though. As you pointed out, "chroot pkg_add" exists already. The feature we need should: * update $ROOTDIR/var/db/pkg * Add $ROOTDIR as a prefix to all installed file locations This would allow pkg_add when running outside of a chroot to install packages into a chrooted system, which is what you can't easily do with "chroot pkg_add". Of course, this isn't particularly easy to do when forking tar(1), so the libarchive integration might be a necessary prerequisite. (Command-line tar doesn't support re-rooting absolute paths. There is a --chroot option to tar, but it's currently broken in some rather complex ways. As I'm composing this message, I'm starting to wonder if it also should not use chroot(2). Hmmm....) The hard part is @exec/@unexec. On the one hand, you don't necessarily want to require the install dependencies of a package to be installed in the chroot, which argues against running those under chroot(2). On the other hand, a lot of the commands used within @exec/@unexec manipulate common system databases (e.g., ldconfig), which argues that those commands should be run under chroot(2). Cheers, Tim From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 03:49:26 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F2C71065670; Thu, 8 Apr 2010 03:49:26 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id 1F47A8FC0A; Thu, 8 Apr 2010 03:49:25 +0000 (UTC) Received: by qyk11 with SMTP id 11so1111442qyk.13 for ; Wed, 07 Apr 2010 20:49:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=9YpFvk2Wm07U/BVzIlyCkV1XSuUBz7YGADb6v6PIeYA=; b=pH/+wBF6bIeex7eEf5+cIY6RsjzVlArNSymdonron28spo7/GM4mkugOZs6cvqecRM bGC3dIrMUACwch7IyDHiyEAbk5zLe16Vv+cpPKonZX5UZ9p6sJkx/mGt2Q6Eqdmk+zIh JJkBFm6bcw1yF1XRX0YAkY5ipsW/CS2WitUj8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=AO1r7036Hz2zVtGszIZ/y4ZcRtBqYw18mlSBJLyUynOlHTqelQlzFmSQ/q4x5XeSdb m8+/lpPMdkzLcyEKQyLOESjopWivxTjIzfMuJiNZfS18nFd1YC4vd+Xh9Z8SaB13azJb g6CBRy3pSkqQBGL+Bbm5VGdXk4lSG54naPUkM= MIME-Version: 1.0 Received: by 10.229.33.72 with HTTP; Wed, 7 Apr 2010 20:49:25 -0700 (PDT) In-Reply-To: <4BBD3DCB.4030902@freebsd.org> References: <4BBD3DCB.4030902@freebsd.org> Date: Wed, 7 Apr 2010 20:49:25 -0700 Received: by 10.229.35.80 with SMTP id o16mr14714862qcd.93.1270698565280; Wed, 07 Apr 2010 20:49:25 -0700 (PDT) Message-ID: From: Garrett Cooper To: Tim Kientzle Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, portmgr@freebsd.org Subject: Re: [RFC] Remove pkg_add -C (chroot functionality) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2010 03:49:26 -0000 On Wed, Apr 7, 2010 at 7:22 PM, Tim Kientzle wrote: > Garrett Cooper wrote: >> >> =A0 =A0There's an outstanding bug to fix chroot(2)'ing functionality wit= h >> pkg_add(1) [1]. Anyone that has tried this functionality knows that >> it's currently crippled > > If it's currently broken, it's better to > simply remove it. > > I'm not sure I understand why anyone wants > pkg_add to call chroot(2) at the top, though. > As you pointed out, "chroot pkg_add" exists > already. > > The feature we need should: > =A0* update $ROOTDIR/var/db/pkg > =A0* Add $ROOTDIR as a prefix to all installed file locations > This would allow pkg_add when running outside of > a chroot to install packages into a chrooted > system, which is what you can't easily do with > "chroot pkg_add". As discussed in #bsdports with flz, it's much more complicated because in the future we'll most likely have mainstream support for cross-building where this isn't possible, i.e. build arm on i386 -- the point is that there are some steps that must be run on the target system or chroot, and this quite frankly isn't possible without being able to install directly on the target. Regardless, cross-building right now requires that one define the proper environment, and again that can't be delivered with pkg_add without introducing unneeded complexity. Point being, if someone wants to chroot, and they understand the complete exercise, or if we have documentation provided which demonstrates how to do so, people will use it, and potentially grow upon it in a positive way. Right now this is just a vestige of brokenness in pkg_install that should go IMO. > Of course, this isn't particularly easy to do when > forking tar(1), so the libarchive integration > might be a necessary prerequisite. =A0(Command-line > tar doesn't support re-rooting absolute paths. > There is a --chroot option to tar, but it's > currently broken in some rather complex ways. > As I'm composing this message, I'm starting to > wonder if it also should not use chroot(2). > Hmmm....) > > The hard part is @exec/@unexec. =A0On the one > hand, you don't necessarily want to require > the install dependencies of a package to be > installed in the chroot, which argues against > running those under chroot(2). =A0On the other hand, > a lot of the commands used within @exec/@unexec > manipulate common system databases (e.g., ldconfig), > which argues that those commands should be run > under chroot(2). Bingo -- that is the cruxt of the problem with doing chroot(2) with pkg_add. From a support perspective it's a nightmare because when something does go south, you're up a crik without a paddle trying to figure out what the heck is going on... it's better for us that someone is running with a stable, pre-defined system than try and force them to use a home grown / unknown method built around a shaky base. Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 09:57:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B7C8106566B for ; Thu, 8 Apr 2010 09:57:13 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 683E18FC1A for ; Thu, 8 Apr 2010 09:57:11 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so767135fga.13 for ; Thu, 08 Apr 2010 02:57:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:received:message-id:subject :from:to:cc:content-type; bh=NCi9vXF88mcnsjKF1ZAgfMuv0nrbpfBnNBOV8i1pSug=; b=P81VGZluEGRXAPh7kPWWbrXPLYkxBB61X5GJJWFjD8IMvyKHIwb7E3wOweX0Jbuv8S XUBaGSa5vtawtipJCvs+nexUkEIz/7yVx0xzoDEZPGX9uPz3THmzupdTULIvP1oxmLZx mA65VEb/zOKoVY/7F8TL0Flq9h+mKz1uaNYoY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=jdBLGioc+e69EtPhA4PQn+gDidzyz9/hishG+cT1S0SKqT86wQ4J6ASFRzco0F3UZD BOPmOpuos2CCpA4iFPQKTQGdj14dtpBs0dJmyYSPONO4lahWIyU6hETNn9bsXHW4O4GS LH5W4ZiytT9CSvDy3UrE6o1OQeMq9Zdi9KzlE= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.239.137.131 with HTTP; Thu, 8 Apr 2010 02:57:10 -0700 (PDT) In-Reply-To: <20100404175140.GB40499@stack.nl> References: <20100404175140.GB40499@stack.nl> Date: Thu, 8 Apr 2010 11:57:10 +0200 X-Google-Sender-Auth: 4b922fec160f06e2 Received: by 10.239.184.72 with SMTP id x8mr935082hbg.44.1270720630898; Thu, 08 Apr 2010 02:57:10 -0700 (PDT) Message-ID: From: Attilio Rao To: Jilles Tjoelker Content-Type: text/plain; charset=UTF-8 Cc: Ed Maste , freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2010 09:57:13 -0000 2010/4/4 Jilles Tjoelker : > The SX locks so_snd_sx and so_rcv_sx can be legitimately held for > arbitrary amounts of time, while waiting until a send or recv is > possible. Any other threads wanting to send/recv on the socket will > sleep interruptibly on the corresponding SX. If deadlkres is activated, > it may detect a deadlock even though there is no problem. > > If a SIGSTOP or similar comes in while waiting until send/recv is > possible, the SX is held across the suspension and noone will be able to > send/recv on the socket until the process is resumed. On the other hand > a thread waiting for the SX can be suspended without harm. Adding PBDRY > to various sleeps may help but may also introduce other problems > (SIGSTOP disturbing the functioning of the process more, possibly with > stuff like SO_RCVTIMEO). > > Example (using the fact that fifos are implemented using sockets): > > term1% mkfifo testfifo > term1% cat >testfifo > > term2% cat testfifo > > term3% cat testfifo > > Letting this sit for half an hour will trigger deadlkres. Jilles, may you please try this patch?: http://www.freebsd.org/~attilio/Sandvine/deadlkres/deadlkres-blessed.diff It does introduce a blessed list where the sx_lock can be whitelisted. It also fixes, for long sleeps, the wrap-up of ticks object. The obvious fallout is that deadlkres can't help much with deadlocks for whitelisted sxlock or lockmgr but that is a fair price I think. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 09:58:11 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69E89106566B for ; Thu, 8 Apr 2010 09:58:11 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id ED2568FC1F for ; Thu, 8 Apr 2010 09:58:10 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id l26so1733145fgb.13 for ; Thu, 08 Apr 2010 02:58:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:received:message-id:subject :from:to:cc:content-type; bh=9WUuwuzl6AedpshxT6zAAIfLfObFrchCGILsya6JeWo=; b=hNjcdBpOrNAuT3+gwp8df1sFVowv+ZclfgsBEVXdpy3AKdsy5FWsabT/zg1JZZsiSL F24Ec5b/41cMphlsqR7ey6+W4jooqSwKaIUTiXErusMjGfyqcBFIj3OWkMYK5o7D6+1m YXflnku9ZRsWw/+GssLpP5LF4Pzsn9YWqrHUI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=bMbeyv8f40x8/6tE/CWvEAxjpFwpU5bVZtZeR2H578+uG9rzguFe0BU4IbGxKxI1zX aglUIQDLGFZR8erSH2UEmRpEk3OQ+/Lf4XkymkF26NjUQiAURjgZBpPAN1dNB1nm4ufp 95ixv5gtqu7Ue4GYxEkbVCVISo8sN/JMG4aMY= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.239.137.131 with HTTP; Thu, 8 Apr 2010 02:58:08 -0700 (PDT) In-Reply-To: References: <20100404175140.GB40499@stack.nl> Date: Thu, 8 Apr 2010 11:58:08 +0200 X-Google-Sender-Auth: 9e1991996186e240 Received: by 10.239.187.129 with SMTP id l1mr957269hbh.86.1270720689902; Thu, 08 Apr 2010 02:58:09 -0700 (PDT) Message-ID: From: Attilio Rao To: Jilles Tjoelker Content-Type: text/plain; charset=UTF-8 Cc: Ed Maste , freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2010 09:58:11 -0000 2010/4/8 Attilio Rao : > 2010/4/4 Jilles Tjoelker : >> The SX locks so_snd_sx and so_rcv_sx can be legitimately held for >> arbitrary amounts of time, while waiting until a send or recv is >> possible. Any other threads wanting to send/recv on the socket will >> sleep interruptibly on the corresponding SX. If deadlkres is activated, >> it may detect a deadlock even though there is no problem. >> >> If a SIGSTOP or similar comes in while waiting until send/recv is >> possible, the SX is held across the suspension and noone will be able to >> send/recv on the socket until the process is resumed. On the other hand >> a thread waiting for the SX can be suspended without harm. Adding PBDRY >> to various sleeps may help but may also introduce other problems >> (SIGSTOP disturbing the functioning of the process more, possibly with >> stuff like SO_RCVTIMEO). >> >> Example (using the fact that fifos are implemented using sockets): >> >> term1% mkfifo testfifo >> term1% cat >testfifo >> >> term2% cat testfifo >> >> term3% cat testfifo >> >> Letting this sit for half an hour will trigger deadlkres. > > Jilles, > may you please try this patch?: > http://www.freebsd.org/~attilio/Sandvine/deadlkres/deadlkres-blessed.diff > > It does introduce a blessed list where the sx_lock can be whitelisted. > It also fixes, for long sleeps, the wrap-up of ticks object. > The obvious fallout is that deadlkres can't help much with deadlocks > for whitelisted sxlock or lockmgr but that is a fair price I think. Forgot to mention: mutex are left out from this design by purpose because a similar problem might never happen for them by design. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-arch@FreeBSD.ORG Thu Apr 8 15:42:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D4803106564A; Thu, 8 Apr 2010 15:42:29 +0000 (UTC) (envelope-from alexanderchuranov@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 798F28FC0A; Thu, 8 Apr 2010 15:42:29 +0000 (UTC) Received: by gyh20 with SMTP id 20so1339983gyh.13 for ; Thu, 08 Apr 2010 08:42:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type; bh=U9okjS4SlLwn4H6f2VZBvUfM+w0m+NfMVxJbOnPQDp0=; b=fH/YBJZx2Em7t5/A3OuSIenH7P0kz6MaLn42okMeEnQLrAi7osnvdgvT9P5iKKh9Bx QowtykGuXSVuJJ57Bzu5gu1+90NnCw4J0QILMl699wWejfVXi4NyD5rQfUtVfbCV3zUY GEbiISvaz8LCv59Pj1mBozWA2R8qA5sxR58lg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Rf+cVrNBGxl5LzsyKZnv3WRDiz8ID771yEtqO0k+KMeyJcjvLa7VXJT76EvNiBsj1E FUMq5H1zUOrmClSqm5b5E2b+AlVfKBxU5g7Qv2DBTmiFIT7j+hHy4J0uYFAhX7Fa9XRT faBcYUfEZ2JviON3Rhr2SXP467VL2obcIfvjc= MIME-Version: 1.0 Received: by 10.90.119.15 with HTTP; Thu, 8 Apr 2010 08:42:24 -0700 (PDT) In-Reply-To: References: Date: Thu, 8 Apr 2010 19:42:24 +0400 Received: by 10.91.152.12 with SMTP id e12mr101186ago.73.1270741348378; Thu, 08 Apr 2010 08:42:28 -0700 (PDT) Message-ID: From: Alexander Churanov To: Robert Watson , "M. Warner Losh" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-arch@freebsd.org Subject: Re: New "scallhook" feature. Is is OK to create a proposal? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Apr 2010 15:42:29 -0000 Robert, Warner, and the mailing list, Some things need to be clarified: 1) This is NOT a Google Summer of Code project and we are not students. 2) There are NO plans to create a wrapper for system calls. The feature should be an integral part of the kernel. 3) There are plans to avoid races by design. The feature is to be implemented into several steps: * Deal with direct arguments only. This is not hard and easy to get right. * Deal with indirect arguments of some calls by copying values. When passing control to the actual syscall, substitute original indirect arguments with copies. * Optimize indirect arguments handling by eliminating extra copies and processing. This is actually hard. It's necessary to mention that this is not an all-or-nothing project. The goal is to provide actually useful and safe features. It's expected that some calls may be left unhookable. 4) The quality is to be maintained by using automated unit-tests, integration tests, stress and capacity testing as well as utilizing working exploits for system call wrappers. 5) The feature will never provide privilege elevation (unlike systrace). It's intended to be more like securelevel and ulimit: there will be a single operation for adding a module to the list of hooks for the process. The lists, of course are inherited. The application will be unable to examine the list. We are going to create a page on FreeBSD wiki with details of the proposal: motivation, comparison to other security features, feature details, test plan and implementation plan. When the page is finished and before the implementation, we'd like to gather reviews of the plan. Then the information about progress will be posted periodically to the lists. If everything is right with the project, based on the information provided here, then we are going to proceed with the wiki. Should we go ahead? Alexander Churanov From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 08:08:15 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 02CAC106566B for ; Fri, 9 Apr 2010 08:08:15 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4BB1A8FC1D for ; Fri, 9 Apr 2010 08:08:14 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA13482; Fri, 09 Apr 2010 10:54:42 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1O0934-0004di-2z; Fri, 09 Apr 2010 10:54:42 +0300 Message-ID: <4BBEDD41.4070101@freebsd.org> Date: Fri, 09 Apr 2010 10:54:41 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: arch@freebsd.org X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: panic: getblk: size(%d) > MAXBSIZE(%d) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 08:08:15 -0000 Some context. Recently we've investigated a problem with FAT filesystems that have more than 64K bytes per cluster (on media with sector size > 512). One of the symptoms was panic in the subject line. I've got curious why we have that check and what would go wrong if we allowed that condition to be violated. The check was added quite a long time ago, in 1996. I wonder if anyone remembers details of a problem that it fixed. Also, if MAXBSIZE limit was chosen appropriately. And, if the protected code changed enough since then to warrant a re-evaluation of this check. I tried to inspect the code and find any hard limits that would be exceeded when a large enough size parameter is passed. The only limit that I've found so far is a size of b_pages array in struct buf: struct vm_page *b_pages[btoc(MAXPHYS)]; Of course, current default value of MAXPHYS is only 2 * MAXBSIZE. But many people already use higher non-default values and there was a talk of bumping the default value too. (Also there is a question of appropriate uses for MAXBSIZE, but that's a different topic). If the b_pages is the only limit indeed, then the check could be changed to something like: if (size > MAXPHYS) ... Also, there is a question if larger size would lead to a suboptimal behavior in buffer cache / VM code even if it doesn't break anything. Thank you in advance for any information and analysis! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 08:18:41 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43256106566B for ; Fri, 9 Apr 2010 08:18:41 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 889E38FC15 for ; Fri, 9 Apr 2010 08:18:40 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA13817 for ; Fri, 09 Apr 2010 11:18:39 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1O09QE-0004fI-NE for arch@freebsd.org; Fri, 09 Apr 2010 11:18:38 +0300 Message-ID: <4BBEE2DD.3090409@freebsd.org> Date: Fri, 09 Apr 2010 11:18:37 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: arch@freebsd.org X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 08:18:41 -0000 As you know MAXBSIZE is currently defined to 64K (seems to a popular value for constants). There is the following comment for this parameter and its value: MAXBSIZE - Filesystems are made out of blocks of at most MAXBSIZE bytes per block. MAXBSIZE may be made larger without effecting any existing filesystems as long as it does not exceed MAXPHYS, and may be made smaller at the risk of not being able to use filesystems which require a block size exceeding MAXBSIZE. I haven't done any historical research but I think that history behind MAXBSIZE could be something like this. There was only one filesystem and it was UFS. People liked to use statically allocated/sized arrays for various stuff, so they came with a limit for a UFS block size. A sufficiently high and reasonable value was chosen for it. As time passed MAXBSIZE proliferated beyond UFS through VFS and other infrastructure code, partly because for correct reasons, partly because of lazy programming, partly because it just worked. Then, new filesystems had to obey this limit too. MAXBSIZE proliferated even further. And I'd argued that it became a more fundamental constant than what it is documented to be. Nowadays several questions could be asked about MAXBSIZE. - Will we have to consider increasing MAXBSIZE? Provided ever increasing media sizes, typical filesystem sizes, typical file sizes (all that multimedia) and even media sector sizes. - Do we need a universal limit on block size for all filesystems? E.g. ZFS already has maximum block size greater than MAXBSIZE, but it lives (has to live?) in a somewhat parallel world to other filesystems. - What are appropriate and inappropriate uses for MAXBSIZE given the questions above? In other words, what would immediately break were we to simplemindedly bump MAXBSIZE value. I have no answers but have some observations. I strongly believe that all uses of MAXBSIZE in sys/dev/ are inappropriate. For me it's inconceivable that a hardware driver would need to know maximum size of a filesystem block. There are 39 occurrences of MAXBSIZE in sys/dev/. Perhaps DFLTPHYS (currently has the same 64K value) was supposed to be used there? Or even MAXPHYS? Probably each driver needs to be evaluated individually. Curiously enough there are only 14 occurrences of DFLTPHYS in sys/dev/. Feedback welcome. Thanks! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 09:27:38 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D07C106566B; Fri, 9 Apr 2010 09:27:38 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe05.swip.net [212.247.154.129]) by mx1.freebsd.org (Postfix) with ESMTP id C73238FC0A; Fri, 9 Apr 2010 09:27:37 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=IkcTkHD0fZMA:10 a=M8b_wTzEtboA:10 a=MnI1ikcADjEx7bvsp0jZvQ==:17 a=CiGHVtXQaeqWVOkG9DMA:9 a=gh83i6PurHjv94mFO57wEyh1VwkA:4 a=QEXdDO2ut3YA:10 Received: from [188.126.201.140] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe05.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 1287024327; Fri, 09 Apr 2010 11:27:35 +0200 From: Hans Petter Selasky To: freebsd-arch@freebsd.org Date: Fri, 9 Apr 2010 11:25:31 +0200 User-Agent: KMail/1.12.4 (FreeBSD/8.0-STABLE; KDE/4.3.4; amd64; ; ) References: <4BBEE2DD.3090409@freebsd.org> In-Reply-To: <4BBEE2DD.3090409@freebsd.org> X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'(; _IjlA: hGE..Ew, XAQ*o#\/M~SC=S1-f9{EzRfT'|Hhll5Q]ha5Bt-s|oTlKMusi:1e[wJl}kd}GR Z0adGx-x_0zGbZj'e(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201004091125.31395.hselasky@c2i.net> Cc: Andriy Gapon Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 09:27:38 -0000 On Friday 09 April 2010 10:18:37 Andriy Gapon wrote: > As you know MAXBSIZE is currently defined to 64K (seems to a popular value > for constants). There is the following comment for this parameter and its > value: > > > Feedback welcome. > Thanks! > Hi, Some hardware will break if you do more than 64K at a time. Especially USB memory sticks. --HPS From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 09:35:05 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2266A106574A for ; Fri, 9 Apr 2010 09:35:05 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 543338FC3C for ; Fri, 9 Apr 2010 09:35:04 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA15081; Fri, 09 Apr 2010 12:35:00 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1O0Ac8-0004lq-IN; Fri, 09 Apr 2010 12:35:00 +0300 Message-ID: <4BBEF4C3.3090902@freebsd.org> Date: Fri, 09 Apr 2010 12:34:59 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: Hans Petter Selasky References: <4BBEE2DD.3090409@freebsd.org> <201004091125.31395.hselasky@c2i.net> In-Reply-To: <201004091125.31395.hselasky@c2i.net> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 09:35:05 -0000 on 09/04/2010 12:25 Hans Petter Selasky said the following: > On Friday 09 April 2010 10:18:37 Andriy Gapon wrote: >> As you know MAXBSIZE is currently defined to 64K (seems to a popular value >> for constants). There is the following comment for this parameter and its >> value: >> > >> Feedback welcome. >> Thanks! >> > > Hi, > > Some hardware will break if you do more than 64K at a time. Especially USB > memory sticks. I know, but I am not sure what your point is. The drivers should use appropriate limits for their hardware. MAXBSIZE is not related to hardware. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 13:06:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 458BC1065673 for ; Fri, 9 Apr 2010 13:06:13 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 018EE8FC12 for ; Fri, 9 Apr 2010 13:06:13 +0000 (UTC) Received: from lemongrass.sec.cl.cam.ac.uk (lemongrass.sec.cl.cam.ac.uk [128.232.18.47]) by cyrus.watson.org (Postfix) with ESMTPSA id 223B146B65; Fri, 9 Apr 2010 09:06:12 -0400 (EDT) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: Date: Fri, 9 Apr 2010 14:06:10 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Alexander Churanov X-Mailer: Apple Mail (2.1078) Cc: freebsd-arch@freebsd.org Subject: Re: New "scallhook" feature. Is is OK to create a proposal? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 13:06:13 -0000 On 8 Apr 2010, at 16:42, Alexander Churanov wrote: > 2) There are NO plans to create a wrapper for system calls. The = feature should be an integral part of the kernel. To my mind, "wrapper" describes any piece of code that wraps the system = call, regardless of whether it's compiled into the kernel, a loadable = module, or a process reached via upcalls. If it happens before/after the = remainder of the syscall runs, it's a wrapper. Wrappers are not, = fundamentally, an evil technology: we use them in countless ways = (although only for wrapping syscalls in the cases of things like = LD_PRELOADS on libc, Linux ABI emulation, etc) -- what concerns me is = using wrappers to enforce security properties that they are unable to = enforce correctly. > 3) There are plans to avoid races by design. The feature is to be = implemented into several steps: >=20 > * Deal with direct arguments only. This is not hard and easy to get = right. > * Deal with indirect arguments of some calls by copying values. When = passing control to the actual syscall, substitute original indirect = arguments with copies. > * Optimize indirect arguments handling by eliminating extra copies = and processing. This is actually hard. It's necessary to mention that = this is not an all-or-nothing project. The goal is to provide actually = useful and safe features. It's expected that some calls may be left = unhookable. It's worth observing that, although I didn't illustrate exploits for = them in the paper, direct arguments, even if they can't be rewritten, = are subject to race conditions in wrapper environments. Likewise even if = the kernel moves effectively to message-passing, semantic = vulnerabilities (mentioned) in the paper may occur. A few examples: - If an indirect argument includes a path and the wrapper interprets the = path with respect to the file system (i.e., looks it up to query whether = the target is a symlink), then the interpretation of the same path = (mapping to file system objects) can change between wrapper execution = and lookup by the kernel. - If the direct argument is for a file descriptor number, likewise any = attempt to look up the file descriptor in the wrapper may be non-atomic = with respect to "the kernel's" lookup of the same file descriptor = number: the kernel might use a different file than the wrapper. Hence the design of our pluggable access control framework, the MAC = Framework, which provides atomic access to the objects being mediated by = a policy. For example, the MLS policy can hook into the file open = implementation at a point where it's guaranteed to be performing tests = on the same vnode that I/O will be performed on. Likewise, the ability = to catch a socket write in the socket code, rather than before the = system call runs, so that it's working with socket structures rather = than file descriptor numbers. This sort of safety in the presence of = concurrency was one of the motivating factors in choosing to go with a = system like the MAC Framework rather than the more popular system call = wrapping techniques of the late 1990's. However, there are certain types = of security operations that the MAC Framework doesn't support = implementing -- for example, privilege escalation of certain types, and = substituting object references. Given the above, could you say a bit more about how these sorts of = vulnerabilities are avoided in your system? Argument transformation has = value, but only when the underlying policies being implemented are safe. = For example, substituting a uid argument to setuid() is "safe", but = substituting paths may not be due to features like symlinks, concurrency = attacks on the file system layout from another thread, etc. Robert= From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 13:39:24 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C12C106566B for ; Fri, 9 Apr 2010 13:39:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id AEFB88FC14 for ; Fri, 9 Apr 2010 13:39:23 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEANLKvkuDaFvI/2dsb2JhbACbO3G7G4UJBA X-IronPort-AV: E=Sophos;i="4.52,177,1270440000"; d="scan'208";a="71677827" Received: from darling.cs.uoguelph.ca ([131.104.91.200]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 09 Apr 2010 09:39:23 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id BBA059400CF; Fri, 9 Apr 2010 09:39:22 -0400 (EDT) X-Virus-Scanned: amavisd-new at darling.cs.uoguelph.ca Received: from darling.cs.uoguelph.ca ([127.0.0.1]) by localhost (darling.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hYz-FXDLICKU; Fri, 9 Apr 2010 09:39:21 -0400 (EDT) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by darling.cs.uoguelph.ca (Postfix) with ESMTP id 7A8A894014F; Fri, 9 Apr 2010 09:39:21 -0400 (EDT) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o39Dr2S17359; Fri, 9 Apr 2010 09:53:02 -0400 (EDT) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Fri, 9 Apr 2010 09:53:02 -0400 (EDT) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Andriy Gapon In-Reply-To: <4BBEE2DD.3090409@freebsd.org> Message-ID: References: <4BBEE2DD.3090409@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 13:39:24 -0000 On Fri, 9 Apr 2010, Andriy Gapon wrote: > > Nowadays several questions could be asked about MAXBSIZE. > - Will we have to consider increasing MAXBSIZE? Provided ever increasing media > sizes, typical filesystem sizes, typical file sizes (all that multimedia) and > even media sector sizes. I would certainly like to see a larger MAXBSIZE for NFS. Solaris10 currently uses 128K as a default I/O size and allows up to 1Mb. Using larger I/O sizes for NFS is a simpler way to increase bulk data transfer rate than more buffers and more agressive read-ahead/write-behind. I had assumed that MAXBSIZE is the largest sized block handled by the buffer cache? I have no idea what effect just increasing it has, but had planned on experimenting with it someday. > - Do we need a universal limit on block size for all filesystems? E.g. ZFS > already has maximum block size greater than MAXBSIZE, but it lives (has to > live?) in a somewhat parallel world to other filesystems. It's always going to be a tradeoff between using a generic buffer cache mechanism shared between multiple file systems vs one specifically designed for a given file system. (There are times when the generic buffer cache is far from ideal for the NFS client, but I haven't felt that the issues were serious enough to warrant building one specifically for NFS. > - What are appropriate and inappropriate uses for MAXBSIZE given the questions > above? In other words, what would immediately break were we to simplemindedly > bump MAXBSIZE value. > > I have no answers but have some observations. > I strongly believe that all uses of MAXBSIZE in sys/dev/ are inappropriate. For > me it's inconceivable that a hardware driver would need to know maximum size of > a filesystem block. I think you are on the right track here. It seems to me that MAXBSIZE (or some new constant instead of it) should define the maximum block size handled by the buffer cache. I have no idea what the implications of increasing it are, but it sounds like some drivers will need to be fixed, as you've noted. Just my $0.00 worth, rick From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 13:52:32 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75FEC1065673; Fri, 9 Apr 2010 13:52:32 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 336C78FC34; Fri, 9 Apr 2010 13:52:31 +0000 (UTC) Received: from [127.0.0.1] (pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o39DqTqY085704; Fri, 9 Apr 2010 07:52:29 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: Date: Fri, 9 Apr 2010 07:52:29 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> References: <4BBEE2DD.3090409@freebsd.org> To: Rick Macklem X-Mailer: Apple Mail (2.1078) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: arch@freebsd.org, Andriy Gapon Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 13:52:32 -0000 On Apr 9, 2010, at 7:53 AM, Rick Macklem wrote: >=20 >> - What are appropriate and inappropriate uses for MAXBSIZE given the = questions >> above? In other words, what would immediately break were we to = simplemindedly >> bump MAXBSIZE value. >>=20 >> I have no answers but have some observations. >> I strongly believe that all uses of MAXBSIZE in sys/dev/ are = inappropriate. For >> me it's inconceivable that a hardware driver would need to know = maximum size of >> a filesystem block. >=20 > I think you are on the right track here. It seems to me that MAXBSIZE > (or some new constant instead of it) should define the maximum block > size handled by the buffer cache. I have no idea what the implications > of increasing it are, but it sounds like some drivers will need to be > fixed, as you've noted. Storage drivers are insulated from the details of MAXBSIZE by GEOM = honoring the driver's advertised max-i/o-size attribute. What I see when I grep = through the sources are mostly uses in busdma attributes, which themselves probably = came via cut-n-paste from prior drivers. I can't come up with any = explanation for that which makes good design sense, so I'll agree that storage drivers = shouldn't reference MAXBSIZE. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 14:29:44 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8427A1065670 for ; Fri, 9 Apr 2010 14:29:44 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C26D58FC23 for ; Fri, 9 Apr 2010 14:29:43 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA20645; Fri, 09 Apr 2010 17:29:28 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4BBF39C7.4050308@freebsd.org> Date: Fri, 09 Apr 2010 17:29:27 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100319) MIME-Version: 1.0 To: Scott Long References: <4BBEE2DD.3090409@freebsd.org> <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> In-Reply-To: <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Rick Macklem Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 14:29:44 -0000 on 09/04/2010 16:52 Scott Long said the following: > > Storage drivers are insulated from the details of MAXBSIZE by GEOM honoring > the driver's advertised max-i/o-size attribute. What I see when I grep through the > sources are mostly uses in busdma attributes, which themselves probably came > via cut-n-paste from prior drivers. I can't come up with any explanation for that > which makes good design sense, so I'll agree that storage drivers shouldn't > reference MAXBSIZE. Should DFLTPHYS be used there? Or is there a better DMA-specific constant? Or, perhaps, each driver should just use its won private constant based on its hardware capabilities? -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 14:35:22 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F45E106566B; Fri, 9 Apr 2010 14:35:22 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id E67BC8FC2A; Fri, 9 Apr 2010 14:35:21 +0000 (UTC) Received: from [127.0.0.1] (pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o39EZIvl086621; Fri, 9 Apr 2010 08:35:18 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BBF39C7.4050308@freebsd.org> Date: Fri, 9 Apr 2010 08:35:18 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <682A6F1E-31E3-4920-A66E-452221866945@samsco.org> References: <4BBEE2DD.3090409@freebsd.org> <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> <4BBF39C7.4050308@freebsd.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1078) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: arch@freebsd.org, Rick Macklem Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 14:35:22 -0000 On Apr 9, 2010, at 8:29 AM, Andriy Gapon wrote: > on 09/04/2010 16:52 Scott Long said the following: >>=20 >> Storage drivers are insulated from the details of MAXBSIZE by GEOM = honoring >> the driver's advertised max-i/o-size attribute. What I see when I = grep through the >> sources are mostly uses in busdma attributes, which themselves = probably came >> via cut-n-paste from prior drivers. I can't come up with any = explanation for that >> which makes good design sense, so I'll agree that storage drivers = shouldn't >> reference MAXBSIZE. >=20 > Should DFLTPHYS be used there? > Or is there a better DMA-specific constant? > Or, perhaps, each driver should just use its won private constant = based on its > hardware capabilities? Each driver should be advertising its own maxio attribute, with the = exception of CAM drivers. Advertising is optional in CAM, and is defaulted to = 64k. But yes, each driver should define and use its own constants here. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 14:40:38 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A42781065675 for ; Fri, 9 Apr 2010 14:40:38 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E87808FC21 for ; Fri, 9 Apr 2010 14:40:37 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA20904; Fri, 09 Apr 2010 17:40:26 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4BBF3C5A.7040009@freebsd.org> Date: Fri, 09 Apr 2010 17:40:26 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100319) MIME-Version: 1.0 To: Rick Macklem References: <4BBEE2DD.3090409@freebsd.org> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 14:40:38 -0000 on 09/04/2010 16:53 Rick Macklem said the following: > > > On Fri, 9 Apr 2010, Andriy Gapon wrote: > >> >> Nowadays several questions could be asked about MAXBSIZE. >> - Will we have to consider increasing MAXBSIZE? Provided ever >> increasing media >> sizes, typical filesystem sizes, typical file sizes (all that >> multimedia) and >> even media sector sizes. > > I would certainly like to see a larger MAXBSIZE for NFS. Solaris10 > currently uses 128K as a default I/O size and allows up to 1Mb. Using > larger I/O sizes for NFS is a simpler way to increase bulk data transfer > rate than more buffers and more agressive read-ahead/write-behind. > > I had assumed that MAXBSIZE is the largest sized block handled by the > buffer cache? I have no idea what effect just increasing it has, but > had planned on experimenting with it someday. I have lightly tested this under qemu. I used my avgfs:) modified to issue 4*MAXBSIZE bread-s. I removed size > MAXBSIZE check in getblk (see a parallel thread "panic: getblk: size(%d) > MAXBSIZE(%d)"). And I bumped MAXPHYS to 1MB. Some results. I got no panics, data was read correctly and system remained stable, which is good. But I observed reading process (dd bs=1m on avgfs) spending a lot of time sleeping on needsbuffer in getnewbuf. needsbuffer value was VFS_BIO_NEED_ANY. Apparently there was some shortage of free buffers. Perhaps some limits/counts were incorrectly auto-tuned. Also, I later tried to double MAXBSIZE value and got exactly the same results. No panics, no corruptions. But, of course, this doesn't mean that there are no problems. Especially in the hardware drivers. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 14:41:57 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE28D106566B for ; Fri, 9 Apr 2010 14:41:57 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0FC318FC08 for ; Fri, 9 Apr 2010 14:41:53 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA20921; Fri, 09 Apr 2010 17:41:38 +0300 (EEST) (envelope-from avg@freebsd.org) Message-ID: <4BBF3CA1.1040001@freebsd.org> Date: Fri, 09 Apr 2010 17:41:37 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100319) MIME-Version: 1.0 To: Scott Long References: <4BBEE2DD.3090409@freebsd.org> <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> <4BBF39C7.4050308@freebsd.org> <682A6F1E-31E3-4920-A66E-452221866945@samsco.org> In-Reply-To: <682A6F1E-31E3-4920-A66E-452221866945@samsco.org> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org, Rick Macklem Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 14:41:57 -0000 on 09/04/2010 17:35 Scott Long said the following: > On Apr 9, 2010, at 8:29 AM, Andriy Gapon wrote: > >> on 09/04/2010 16:52 Scott Long said the following: >>> Storage drivers are insulated from the details of MAXBSIZE by GEOM honoring >>> the driver's advertised max-i/o-size attribute. What I see when I grep through the >>> sources are mostly uses in busdma attributes, which themselves probably came >>> via cut-n-paste from prior drivers. I can't come up with any explanation for that >>> which makes good design sense, so I'll agree that storage drivers shouldn't >>> reference MAXBSIZE. >> Should DFLTPHYS be used there? >> Or is there a better DMA-specific constant? >> Or, perhaps, each driver should just use its won private constant based on its >> hardware capabilities? > > Each driver should be advertising its own maxio attribute, with the exception > of CAM drivers. Advertising is optional in CAM, and is defaulted to 64k. But > yes, each driver should define and use its own constants here. I actually meant not what drivers advertise but what they use in busdma. Or are those directly related? -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 14:57:07 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFF05106566B; Fri, 9 Apr 2010 14:57:07 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 6FF0B8FC12; Fri, 9 Apr 2010 14:57:07 +0000 (UTC) Received: from [127.0.0.1] (pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o39Ev4q1086706; Fri, 9 Apr 2010 08:57:04 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BBF3CA1.1040001@freebsd.org> Date: Fri, 9 Apr 2010 08:57:04 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <998F1648-E06E-49C3-A5B8-802B8B6D64F9@samsco.org> References: <4BBEE2DD.3090409@freebsd.org> <07A7155D-0836-4D8C-BCF4-70FC16C77B69@samsco.org> <4BBF39C7.4050308@freebsd.org> <682A6F1E-31E3-4920-A66E-452221866945@samsco.org> <4BBF3CA1.1040001@freebsd.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1078) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: arch@freebsd.org, Rick Macklem Subject: Re: (in)appropriate uses for MAXBSIZE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 14:57:07 -0000 On Apr 9, 2010, at 8:41 AM, Andriy Gapon wrote: > on 09/04/2010 17:35 Scott Long said the following: >> On Apr 9, 2010, at 8:29 AM, Andriy Gapon wrote: >>=20 >>> on 09/04/2010 16:52 Scott Long said the following: >>>> Storage drivers are insulated from the details of MAXBSIZE by GEOM = honoring >>>> the driver's advertised max-i/o-size attribute. What I see when I = grep through the >>>> sources are mostly uses in busdma attributes, which themselves = probably came >>>> via cut-n-paste from prior drivers. I can't come up with any = explanation for that >>>> which makes good design sense, so I'll agree that storage drivers = shouldn't >>>> reference MAXBSIZE. >>> Should DFLTPHYS be used there? >>> Or is there a better DMA-specific constant? >>> Or, perhaps, each driver should just use its won private constant = based on its >>> hardware capabilities? >>=20 >> Each driver should be advertising its own maxio attribute, with the = exception >> of CAM drivers. Advertising is optional in CAM, and is defaulted to = 64k. But >> yes, each driver should define and use its own constants here. >=20 > I actually meant not what drivers advertise but what they use in = busdma. > Or are those directly related? >=20 Yes, the should normally be directly related except in maybe a few very = rare cases. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 23:07:38 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC57410657D3; Fri, 9 Apr 2010 23:07:38 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) by mx1.freebsd.org (Postfix) with ESMTP id 834CF8FC17; Fri, 9 Apr 2010 23:07:38 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id C2FD21DD640; Sat, 10 Apr 2010 01:07:36 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id AAA3317533; Sat, 10 Apr 2010 01:07:36 +0200 (CEST) Date: Sat, 10 Apr 2010 01:07:36 +0200 From: Jilles Tjoelker To: Attilio Rao Message-ID: <20100409230736.GA7317@stack.nl> References: <20100404175140.GB40499@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Ed Maste , freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 23:07:38 -0000 On Thu, Apr 08, 2010 at 11:57:10AM +0200, Attilio Rao wrote: > Jilles, > may you please try this patch?: > http://www.freebsd.org/~attilio/Sandvine/deadlkres/deadlkres-blessed.diff > It does introduce a blessed list where the sx_lock can be whitelisted. > It also fixes, for long sleeps, the wrap-up of ticks object. > The obvious fallout is that deadlkres can't help much with deadlocks > for whitelisted sxlock or lockmgr but that is a fair price I think. This fixes the inappropriate deadlkres triggering for me. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Fri Apr 9 23:09:46 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 362CD1065673; Fri, 9 Apr 2010 23:09:46 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id F035C8FC0A; Fri, 9 Apr 2010 23:09:45 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id B2BF3358C2C; Sat, 10 Apr 2010 01:09:44 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 9E73A17533; Sat, 10 Apr 2010 01:09:44 +0200 (CEST) Date: Sat, 10 Apr 2010 01:09:44 +0200 From: Jilles Tjoelker To: Robert Watson Message-ID: <20100409230944.GB7317@stack.nl> References: <20100404175140.GB40499@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-arch@freebsd.org Subject: Re: so_rcv_sx, deadlkres, SIGSTOP X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2010 23:09:46 -0000 On Mon, Apr 05, 2010 at 11:13:31PM +0100, Robert Watson wrote: > On Sun, 4 Apr 2010, Jilles Tjoelker wrote: > > The SX locks so_snd_sx and so_rcv_sx can be legitimately held for arbitrary > > amounts of time, while waiting until a send or recv is possible. Any other > > threads wanting to send/recv on the socket will sleep interruptibly on the > > corresponding SX. If deadlkres is activated, it may detect a deadlock even > > though there is no problem. > Actually, on the whole, socket buffer sx lock acquisition is interruptible. > There are a few exceptions, such as during socket shutdown, and in an RPC > layer or two, but in the general case we want Ctrl-C to interrupt recv() and > send(), so use interruptible sleeps. The exceptions are: * portalfs uses SB_NOINTR to make both sblock() and sbwait() uninterruptible, for connections to a daemon on the local machine * sorflush() (part of shutdown) uses SBL_NOINTR to wait for socket buffer activity to terminate after calling socantrcvmore(); this seems OK because it will not wait for the network, and must be because shutdown(2) should not return EINTR * kern_sendfile() uses SBL_NOINTR with sblock() but then sbwait()s interruptibly; this doesn't make sense to me because sblock() waits for the network just like sbwait() except that another thread happens to be waiting already -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Sat Apr 10 07:34:56 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5DE9106564A; Sat, 10 Apr 2010 07:34:55 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.26]) by mx1.freebsd.org (Postfix) with ESMTP id 533A88FC08; Sat, 10 Apr 2010 07:34:55 +0000 (UTC) Received: by qw-out-2122.google.com with SMTP id 5so1361230qwi.7 for ; Sat, 10 Apr 2010 00:34:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=MT88zgAhKXSdm95kwgOXkERDt3X362GB6qjJIgCk5Vw=; b=q847DIP64w/jCwJn43FQlsyrbGRmW2Wv5pImS3VJszOAwbnCdP6v/WHeLuS3OgDPq2 7tR+yjbtsh+eAis+mjkRgSfy/kYcRuBYZomzScPgpRbhdWSBvuud3g0ryhnFqSXApa47 eks+BbCaUmI0oy6BAF8BtrTy2v0rwJc3FdNE4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=W5Ozry1toQYQVPKnYq5wVajaiTYmgwMYZVxBpLGypPp7GjQSOCOH+OgMJeXGabGqmX b6myWaEITTtBBqcIzjo+ThdP1DSQkrbLaI4BLssfv2gNzAeSp2/DlSnUqikdvRAH3OVE d5BtdeULcsq+6lQ/4xFGKNKgo8eCxQDmQ1lUA= MIME-Version: 1.0 Received: by 10.229.28.85 with HTTP; Sat, 10 Apr 2010 00:34:53 -0700 (PDT) In-Reply-To: References: <4BBD3DCB.4030902@freebsd.org> Date: Sat, 10 Apr 2010 00:34:53 -0700 Received: by 10.229.239.149 with SMTP id kw21mr1609712qcb.99.1270884893614; Sat, 10 Apr 2010 00:34:53 -0700 (PDT) Message-ID: From: Garrett Cooper To: Tim Kientzle Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: arch@freebsd.org, freebsd-ports , portmgr@freebsd.org Subject: Re: [RFC] Remove pkg_add -C (chroot functionality) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Apr 2010 07:34:56 -0000 On Wed, Apr 7, 2010 at 8:49 PM, Garrett Cooper wrote: > On Wed, Apr 7, 2010 at 7:22 PM, Tim Kientzle wrote= : >> Garrett Cooper wrote: >>> >>> =A0 =A0There's an outstanding bug to fix chroot(2)'ing functionality wi= th >>> pkg_add(1) [1]. Anyone that has tried this functionality knows that >>> it's currently crippled >> >> If it's currently broken, it's better to >> simply remove it. >> >> I'm not sure I understand why anyone wants >> pkg_add to call chroot(2) at the top, though. >> As you pointed out, "chroot pkg_add" exists >> already. >> >> The feature we need should: >> =A0* update $ROOTDIR/var/db/pkg >> =A0* Add $ROOTDIR as a prefix to all installed file locations >> This would allow pkg_add when running outside of >> a chroot to install packages into a chrooted >> system, which is what you can't easily do with >> "chroot pkg_add". > > As discussed in #bsdports with flz, it's much more complicated because > in the future we'll most likely have mainstream support for > cross-building where this isn't possible, i.e. build arm on i386 -- > the point is that there are some steps that must be run on the target > system or chroot, and this quite frankly isn't possible without being > able to install directly on the target. Regardless, cross-building > right now requires that one define the proper environment, and again > that can't be delivered with pkg_add without introducing unneeded > complexity. Point being, if someone wants to chroot, and they > understand the complete exercise, or if we have documentation provided > which demonstrates how to do so, people will use it, and potentially > grow upon it in a positive way. Right now this is just a vestige of > brokenness in pkg_install that should go IMO. > >> Of course, this isn't particularly easy to do when >> forking tar(1), so the libarchive integration >> might be a necessary prerequisite. =A0(Command-line >> tar doesn't support re-rooting absolute paths. >> There is a --chroot option to tar, but it's >> currently broken in some rather complex ways. >> As I'm composing this message, I'm starting to >> wonder if it also should not use chroot(2). >> Hmmm....) >> >> The hard part is @exec/@unexec. =A0On the one >> hand, you don't necessarily want to require >> the install dependencies of a package to be >> installed in the chroot, which argues against >> running those under chroot(2). =A0On the other hand, >> a lot of the commands used within @exec/@unexec >> manipulate common system databases (e.g., ldconfig), >> which argues that those commands should be run >> under chroot(2). > > Bingo -- that is the cruxt of the problem with doing chroot(2) with > pkg_add. From a support perspective it's a nightmare because when > something does go south, you're up a crik without a paddle trying to > figure out what the heck is going on... it's better for us that > someone is running with a stable, pre-defined system than try and > force them to use a home grown / unknown method built around a shaky > base. It's been two days without a lot of commentary. Expanding to a larger audience... Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Apr 10 22:52:50 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 489EC1065672; Sat, 10 Apr 2010 22:52:50 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id DEF948FC14; Sat, 10 Apr 2010 22:52:49 +0000 (UTC) Received: by qyk11 with SMTP id 11so3770715qyk.13 for ; Sat, 10 Apr 2010 15:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:received:message-id :subject:from:to:cc:content-type; bh=eKD7F2usQ7dJ6srnHt7HjXQyI+C3B99ctXSEYmFv4sI=; b=HUJmh99vCkMdo7vbguhvKGzyMZQ3Ba+llHij2DFdtCN+APIA2Cmyd0g0CeKj1qHG8h JtJ8cwoAOKGVSjx+grFtH2vLlKUfGB6pWZMp+rneD6j7ZI/t1FYLBy645SwfQ7HH56ng ZIzR406OBGGOOYIMStnXmTU1xlTp/rNAf6p+s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=r2Q+w36lKhpAg1uWGimephqi7eneIFcKEQh1X43h2QU3p0ZaSDCMAM68XlTMGKOFQv ErKBYQCnWjvgteE8XwmYFUK9cgoNnydalRDCA2kskiaMZSMS9f3I7adEod4bj3b4OCWn kVssLWeycdGO1TqeOnZrO0BL5XRVZrSLbWqhI= MIME-Version: 1.0 Received: by 10.229.28.85 with HTTP; Sat, 10 Apr 2010 15:52:48 -0700 (PDT) Date: Sat, 10 Apr 2010 15:52:48 -0700 Received: by 10.229.128.82 with SMTP id j18mr3105767qcs.61.1270939969012; Sat, 10 Apr 2010 15:52:49 -0700 (PDT) Message-ID: From: Garrett Cooper To: arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: portmgr@freebsd.org Subject: [RFC] Remove @owner and @user from package list X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Apr 2010 22:52:50 -0000 Hi again arch, When doing some research, it appears that while functionality in theory exists for @owner and @user in the package list, it isn't actually used in the pkg_install code at all, adding unnecessary bloat to package lists; FWIW this functionality (just like @exec and @unexec) can be implemented via pkg-install or more reliably via an mtree file. Thoughts? Thanks, -Garrett From owner-freebsd-arch@FreeBSD.ORG Sat Apr 10 22:57:43 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21B38106566B; Sat, 10 Apr 2010 22:57:43 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.221.181]) by mx1.freebsd.org (Postfix) with ESMTP id BC5228FC17; Sat, 10 Apr 2010 22:57:42 +0000 (UTC) Received: by qyk11 with SMTP id 11so3772595qyk.13 for ; Sat, 10 Apr 2010 15:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:received:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=1wBfDSsjCTFpWVxez8xV/mVxadway3n7l8vHxu0Dyzs=; b=dWP5rZ5iyYwU7D5iI8+XhT9rjGGRBq7kjCitKa71bHivr8SJN6ulQ7zwzN3k9qwbpO ZXA9g6oig1Iwsr5kSw4IFe1N1HCDkVoZYIIDQRLq8TWRNocizL7tqxqhRljhBFwx/8sF Nv5mV40yXlYPd7ubFbtrEceUDEYk6rs/1KOV8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Qk1gIYQ4ANBykq0nUaFL3rqU8e6zFyhiF7Wbtb9sK23TXvkS7nHuvsLIW1u7oMR6IN vkn5WQ9Mz78cR4OZWRHh8j9StKockE6H/UnqAfA0cys9U0YyiMev/XYcIh1PNbUdU8Jl B8pHyJ7CqWin14ceovnYHz9kcf6lJYnZh4ZSk= MIME-Version: 1.0 Received: by 10.229.28.85 with HTTP; Sat, 10 Apr 2010 15:57:41 -0700 (PDT) In-Reply-To: References: Date: Sat, 10 Apr 2010 15:57:41 -0700 Received: by 10.229.239.148 with SMTP id kw20mr3040431qcb.10.1270940261693; Sat, 10 Apr 2010 15:57:41 -0700 (PDT) Message-ID: From: Garrett Cooper To: arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: portmgr@freebsd.org Subject: Re: [RFC] Remove @owner and @user from package list X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Apr 2010 22:57:43 -0000 On Sat, Apr 10, 2010 at 3:52 PM, Garrett Cooper wrote: > Hi again arch, > =A0 =A0When doing some research, it appears that while functionality in > theory exists for @owner and @user in the package list, it isn't > actually used in the pkg_install code at all, adding unnecessary bloat > to package lists; > =A0 =A0FWIW this functionality (just like @exec and @unexec) can be > implemented via pkg-install or more reliably via an mtree file. > =A0 =A0Thoughts? Nevermind; I was misreading the code. Thanks, -Garrett