From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 03:49:26 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7306C106566C for ; Sun, 23 Jan 2011 03:49:26 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3E17A8FC0A for ; Sun, 23 Jan 2011 03:49:25 +0000 (UTC) Received: by iyb26 with SMTP id 26so3028515iyb.13 for ; Sat, 22 Jan 2011 19:49:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=PPSfQ3quxlIrnCXDhAh8nPrPZARcbVo6mTkhT0uHob4=; b=CfEG++OWaV3NK5RdDXTPYR8A2wbg/3uHhKdVsSTAeUDlaLyZ2JxSHQBXhT1E6yQ0Gx DWJMdQLSnbgR/hFCxjEqnwlDoREDUCl+5iQ8lXRaKTXMnc8tbwk9hMXfFIf/i64COHGH 8ayuZgW1AZp2D332c0hS/YWeJWO8TMQhQkHJc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; b=jjnKYMpkEB60xEi5r94S5Bd4cheJvcmjbJAnrPGJMubr0Fdh3kZIE6BeK4vbIib3we o4CsMYxRLfyShfiIKbBT43fV5f1VtAON4IkiIE1YLRO7FlBxQkJW6i8xNwf951+Dj2Qp 79OVL6VeHACfo0YpXHbk6kADInXX7y3KxHVaY= MIME-Version: 1.0 Received: by 10.231.191.16 with SMTP id dk16mr2907561ibb.23.1295752820256; Sat, 22 Jan 2011 19:20:20 -0800 (PST) Received: by 10.231.200.210 with HTTP; Sat, 22 Jan 2011 19:20:20 -0800 (PST) Date: Sun, 23 Jan 2011 03:20:20 +0000 Message-ID: From: Ali Mashtizadeh To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Exporting kernel symbols X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 03:49:26 -0000 Hi Folks, I tried to build a geom kernel module that uses the alq(9) facility to log some data. The module builds fine but it seems that the kernel isn't exporting the alq(9) symbols. Could someone point me how I can export particular symbols? --=20 Ali Mashtizadeh =D8=B9=D9=84=DB=8C =D9=85=D8=B4=D8=AA=DB=8C =D8=B2=D8=A7=D8=AF=D9=87 From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 10:16:33 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E4721065675; Sun, 23 Jan 2011 10:16:33 +0000 (UTC) (envelope-from aurelien@aurel32.net) Received: from hall.aurel32.net (hall.aurel32.net [IPv6:2001:470:1f15:c4f::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0E50F8FC17; Sun, 23 Jan 2011 10:16:33 +0000 (UTC) Received: from static-qvn-qvu-163095.business.bouyguestelecom.com ([89.81.163.95] helo=ohm.aurel32.net) by hall.aurel32.net with esmtpsa (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1Pgwzn-0006Hx-3t; Sun, 23 Jan 2011 11:16:32 +0100 Received: from aurel32 by ohm.aurel32.net with local (Exim 4.72) (envelope-from ) id 1PgwVe-0008NJ-QM; Sun, 23 Jan 2011 10:45:22 +0100 Date: Sun, 23 Jan 2011 10:45:22 +0100 From: Aurelien Jarno To: Jaakko Heinonen Message-ID: <20110123094522.GA32156@ohm.aurel32.net> References: <20110114122454.GA4805@jh> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20110114122454.GA4805@jh> X-Mailer: Mutt 1.5.20 (2009-06-14) User-Agent: Mutt/1.5.20 (2009-06-14) X-Mailman-Approved-At: Sun, 23 Jan 2011 12:25:51 +0000 Cc: rodrigc@FreeBSD.org, freebsd-hackers@FreeBSD.org Subject: Re: [patch] nmount ro, rw and negated option handling X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 10:16:33 -0000 On Fri, Jan 14, 2011 at 02:24:54PM +0200, Jaakko Heinonen wrote: > > Hi, Hi, > Currently nmount(2) allows a mount point to have "ro", "rw", and "noro" > string options concurrently active. This can cause erratic behavior > demonstrated by this example: > > 1. Have mountd(8) running. > 2. # mdconfig -a -t vnode -f ufsimg > 3. # mount -o ro,rw /dev/md0 /mnt > > After these steps the mount point has string options "ro", "rw" and > "noro" active but the MNT_RDONLY flag is not set. Eventually this will > lead to "ffs_sync: rofs mod" (or similar) panic because the ffs code > marks the file system read-only due to the "ro" string option. > (MNT_RDONLY flag is used in most places for read-only check.) > > I wrote a patch to do following changes: > > - vfs_equalopts() now recognizes "ro" and "rw" as equal options > - vfs_mergeopts() uses vfs_sanitizeopts() to merge options. This ensures > that if the same option shows up several times (negated or not), only > the last one is taken in account. There is still a problem when for > example option "foo" and "nofoo" are merged: the "nofoo" option will > become an active option. This is not a regression however and > currently I don't know an easy way to solve this because the list of > valid options is not available in vfs_mergeopts(). > - vfs_donmount() always converts "norw"/"rdonly" to "ro" and "noro" to > "rw". Thus the mount point will always have either "rw" or "ro" > option. I haven't seen any in-tree file system to test for "noro" but > at least ZFS tests for "rw". That's why I chose "rw" instead or > "noro". > > The patch is available here: > > http://people.freebsd.org/~jh/patches/nmount-ro-rw.diff > > Reviews and testing would be appreciated. > Thanks for the patch, I confirm it fixes the issue I reported in kern/150206. Aurelien -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurelien@aurel32.net http://www.aurel32.net From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 14:42:12 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 5F4CF1065673; Sun, 23 Jan 2011 14:42:12 +0000 (UTC) Date: Sun, 23 Jan 2011 14:42:12 +0000 From: Alexander Best To: Ryan Stone Message-ID: <20110123144212.GA32293@freebsd.org> References: <20110113190625.GA76352@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110113190625.GA76352@freebsd.org> Cc: FreeBSD Hackers Subject: Re: What does the FreeBSD/i386 ABI say about stack alignment? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 14:42:12 -0000 On Thu Jan 13 11, Alexander Best wrote: > On Thu Jan 13 11, Ryan Stone wrote: > > I've been trying to get an application compiled with gcc 4.5.1 running > > on FreeBSD 8.1, but it's been crashing during startup with a SIGBUS. > > It turns out that the problem is that gcc is issuing SSE > > instructions(in my case, a movdqa) that assume that the stack will be > > aligned to a 16-byte boundary. It seems that Linux/i386 guarantees > > this, and I worry that gcc has extended this assumption to all i386 > > architectures. I'm assuming that FreeBSD doesn't make any such > > promises based on the fact that I'm getting crashes. > > > > There does seem to be a flag (-mstackrealign) that you can set to > > force gcc to align the stack to what it wants, but that pessimizes the > > generated code a bit. Some googling would seem to indicate that > > -mpreferred-stack-boundary won't always handle this problem correctly. > > > > Any ideas? My inclination, at least for our local source tree here at > > $WORK, would be to accommodate gcc and guarantee the stack alignment > > that it wants rather than pessimize our application. It seems we have > > an old local patch/hack in our FreeBSD 6.1 tree(apparently based on > > this: http://www.freebsd.org/cgi/getmsg.cgi?fetch=438552+0+/usr/local/www/db/text/2000/freebsd-current/20000507.freebsd-current). > > I believe that this patch is the reason why we haven't seen the > > problem when running on 6.1, but the patch doesn't seem to work > > anymore on 8.1. > > i'm experiencing a similar issue on amd64 with mplayer (svn snapshot) and > gcc46: > > otaku% ./mplayer ~/filme/wiedhow.mkv > MPlayer SVN-r32787-4.6.0 (C) 2000-2011 MPlayer Team > 161 audio & 350 video codecs > > Playing /home/arundel/filme/wiedhow.mkv. > zsh: illegal hardware instruction (core dumped) ./mplayer ~/filme/wiedhow.mkv > otaku% echo $? > 132 just wanted to report that the issue i was having with mplayer vanished. so it seems this was in fact a bug in mplayer itself. cheers. alex > > i'm not sure however, if both matters are related. gdb and 'bt' report: > > #0 0x0000000805327f03 in sscanf (str=0xa
, fmt=0x7fffffbffe60
) at /usr/subversion-src/lib/libc/stdio/sscanf.c:46 > 46 { > [New Thread 80a407400 (LWP 105253/initial thread)] > (gdb) bt > #0 0x0000000805327f03 in sscanf (str=0xa
, fmt=0x7fffffbffe60
) at /usr/subversion-src/lib/libc/stdio/sscanf.c:46 > #1 0x000000000062fa25 in vsscanf () > #2 0x0000000805327f84 in sscanf (str=Variable "str" is not available. > ) at /usr/subversion-src/lib/libc/stdio/sscanf.c:51 > #3 0x000000000062fa25 in vsscanf () > > cheers. > alex > > -- > a13x -- a13x From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 15:37:02 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 33ADA1065672; Sun, 23 Jan 2011 15:37:02 +0000 (UTC) Date: Sun, 23 Jan 2011 15:37:02 +0000 From: Alexander Best To: Anonymous Message-ID: <20110123153702.GA39977@freebsd.org> References: <20110113190625.GA76352@freebsd.org> <20110123144212.GA32293@freebsd.org> <86lj2bwshm.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86lj2bwshm.fsf@gmail.com> Cc: FreeBSD Hackers , Ryan Stone Subject: Re: What does the FreeBSD/i386 ABI say about stack alignment? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 15:37:02 -0000 On Sun Jan 23 11, Anonymous wrote: > Alexander Best writes: > > >> otaku% ./mplayer ~/filme/wiedhow.mkv > >> MPlayer SVN-r32787-4.6.0 (C) 2000-2011 MPlayer Team > >> 161 audio & 350 video codecs > >> > >> Playing /home/arundel/filme/wiedhow.mkv. > >> zsh: illegal hardware instruction (core dumped) ./mplayer ~/filme/wiedhow.mkv > >> otaku% echo $? > >> 132 > > > > just wanted to report that the issue i was having with mplayer vanished. so it > > seems this was in fact a bug in mplayer itself. > > Did you update lang/gcc46 recently? I've seen many similar bugs on *the* > devel branch of GCC, i.e. gcc46 and before 4.5.0 release on gcc45. i did actually. so you think this was a gcc46 bug that was resolved? recent mplayer versions don't seem to build with the base gcc. cheers. alex -- a13x From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 15:57:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 708F7106566B for ; Sun, 23 Jan 2011 15:57:57 +0000 (UTC) (envelope-from swell.k@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 275F08FC20 for ; Sun, 23 Jan 2011 15:57:56 +0000 (UTC) Received: by ywp6 with SMTP id 6so1078047ywp.13 for ; Sun, 23 Jan 2011 07:57:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-type; bh=ax53XIgjnWwRVIMykVq0UkDeZeEi55z2O7iRCerNQtE=; b=W5ip0YpnyfhuJrBkwHuTQdd8h5fbPgkLPXsXZrvL97J3S0TNKcdgHjugXVsGoVeV3d JmdZ7HSfh+TqJcWBVe4TBhyfJy1VjY4QUdzclc/uAsry3UJdZepaj7IkX2a/LFXhfcfQ 5V+6b3/oaSCJMW1AEAEGdL58L3tGNlQ1cMYi8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; b=ifk00uQ8x8jc2WZlIdzDo+QGfwhRlSfbAqtwULt04QLMYq7orV+s4ckHjYS1LG/vkX WFRaDvXXdjI28AMERMEnsPBL2M1+FQWirgkiJweOAxOZO/cOSSrbbKa59HiKjFfaSHHs feHAuJXxCFo4ENlPdrqUqxrYnDDlCN+EUGCdE= Received: by 10.90.54.17 with SMTP id c17mr3667211aga.43.1295796437780; Sun, 23 Jan 2011 07:27:17 -0800 (PST) Received: from localhost ([199.48.147.41]) by mx.google.com with ESMTPS id e24sm14721791ana.22.2011.01.23.07.27.14 (version=SSLv3 cipher=RC4-MD5); Sun, 23 Jan 2011 07:27:17 -0800 (PST) From: Anonymous To: Alexander Best References: <20110113190625.GA76352@freebsd.org> <20110123144212.GA32293@freebsd.org> Date: Sun, 23 Jan 2011 18:27:01 +0300 In-Reply-To: <20110123144212.GA32293@freebsd.org> (Alexander Best's message of "Sun, 23 Jan 2011 14:42:12 +0000") Message-ID: <86lj2bwshm.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain Cc: FreeBSD Hackers , Ryan Stone Subject: Re: What does the FreeBSD/i386 ABI say about stack alignment? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 15:57:57 -0000 Alexander Best writes: >> otaku% ./mplayer ~/filme/wiedhow.mkv >> MPlayer SVN-r32787-4.6.0 (C) 2000-2011 MPlayer Team >> 161 audio & 350 video codecs >> >> Playing /home/arundel/filme/wiedhow.mkv. >> zsh: illegal hardware instruction (core dumped) ./mplayer ~/filme/wiedhow.mkv >> otaku% echo $? >> 132 > > just wanted to report that the issue i was having with mplayer vanished. so it > seems this was in fact a bug in mplayer itself. Did you update lang/gcc46 recently? I've seen many similar bugs on *the* devel branch of GCC, i.e. gcc46 and before 4.5.0 release on gcc45. From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 15:35:44 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A97ED106566C for ; Sun, 23 Jan 2011 15:35:44 +0000 (UTC) (envelope-from egrosbein@rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221]) by mx1.freebsd.org (Postfix) with ESMTP id EF37D8FC14 for ; Sun, 23 Jan 2011 15:35:43 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.4/8.14.4) with ESMTP id p0NF6RQa069316; Sun, 23 Jan 2011 21:06:27 +0600 (NOVT) (envelope-from egrosbein@rdtc.ru) Message-ID: <4D3C43EE.8060307@rdtc.ru> Date: Sun, 23 Jan 2011 21:06:22 +0600 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110112 Thunderbird/3.1.7 MIME-Version: 1.0 To: "net@freebsd.org" Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sun, 23 Jan 2011 16:02:46 +0000 Cc: hackers@freebsd.org Subject: Querying bsnmpd through /var/run/snmpd.sock X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 15:35:44 -0000 Hi! bsnmpd running with mibII module opens local socket /var/run/snmpd.sock mentioned in snmp_mibII(3) manual page: The mibII module opens a socket that is used to execute all network related ioctl(2) functions. This socket is globally available under the name mib_netsock. How do I use the socket? I hope to be able to call mib_find_if_sys() function from another process using the socket. Is there a documentation for this? Eugene Grosbein From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 16:06:13 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EDE601065675 for ; Sun, 23 Jan 2011 16:06:13 +0000 (UTC) (envelope-from timon@timon.net.nz) Received: from frost.plasmahost.ru (plasmahost.ru [178.63.60.242]) by mx1.freebsd.org (Postfix) with ESMTP id 71D5C8FC19 for ; Sun, 23 Jan 2011 16:06:13 +0000 (UTC) Received: from [192.168.0.246] ([213.141.141.158]) (AUTH: PLAIN timon@timon.net.nz, SSL: TLSv1/SSLv3, 256bits, CAMELLIA256-SHA) by frost.plasmahost.ru with esmtp; Sun, 23 Jan 2011 15:55:30 +0000 id 00017725.000000004D3C4F74.00010F72 Message-ID: <4D3C4F94.7020307@timon.net.nz> Date: Sun, 23 Jan 2011 18:56:04 +0300 From: Alexandr Matveev User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110101 Thunderbird/3.1.7 MIME-Version: 1.0 To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: How to read non-physical memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 16:06:14 -0000 Hello. When FreeBSD boots with 'boot -v' it show SMAP: SMAP type=01 base=0000000000000000 len=000000000009d800 SMAP type=02 base=000000000009d800 len=0000000000002800 SMAP type=02 base=00000000000e0000 len=0000000000020000 SMAP type=01 base=0000000000100000 len=00000000bfdb0000 <...> SMAP type=02 base=00000000fed1c000 len=0000000000004000 <...> Memory range 0xfed1c000 - 0xfed2000 belongs to ICH8 mapped to memory configuration registers. If I correctly understand mem man page, /dev/mem allow to read only real physical memory. How can I read data from this range? -- Alexandr Matveev From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 19:56:57 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB0281065670 for ; Sun, 23 Jan 2011 19:56:57 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 61A198FC1A for ; Sun, 23 Jan 2011 19:56:57 +0000 (UTC) Received: by wwf26 with SMTP id 26so3311203wwf.31 for ; Sun, 23 Jan 2011 11:56:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=4dkOCwwc3C3ybAd7JttE79kDMf4R+51gSrCMG9kE2ZY=; b=F6QC+XniVzKx+pq/ie2xPHz5ZcbY95mDhc8tex1UqInFnpx4OVvltcdnbegybc4aXe f79pv8LX1+a5Ubqa2kQqPwvLCTHMwmlPwc4Blxj06xsNWNNhX9+rf1DiyWBUDYtZrS3N v6QITD1ZKTqaJFcfaqS4yTAwuoN7rZ2xaO3rg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=ZAgjFk3qGuzRpz1TCeK6FdWyAUkMKfhENkDSOCQQqGQSJEYqkCtgpMSOVZrzSTwxtz YYXpT46hXdP77MG6HIq2VYEBHc7SmFEAQVIv8C3HGtaq+goHtEI065IkgDyVq1ixI/LX boBplIoA5nSV0fvNkMkhsrZdinkt4d8gvXagA= MIME-Version: 1.0 Received: by 10.216.63.132 with SMTP id a4mr2867845wed.3.1295810783005; Sun, 23 Jan 2011 11:26:23 -0800 (PST) Received: by 10.216.168.134 with HTTP; Sun, 23 Jan 2011 11:26:22 -0800 (PST) In-Reply-To: <4D3C4F94.7020307@timon.net.nz> References: <4D3C4F94.7020307@timon.net.nz> Date: Sun, 23 Jan 2011 11:26:22 -0800 Message-ID: From: Neel Natu To: Alexandr Matveev Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org Subject: Re: How to read non-physical memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 19:56:57 -0000 Hi Alexandr, On Sun, Jan 23, 2011 at 7:56 AM, Alexandr Matveev wrot= e: > Hello. > > =A0When FreeBSD boots with 'boot -v' it show SMAP: > SMAP type=3D01 base=3D0000000000000000 len=3D000000000009d800 > SMAP type=3D02 base=3D000000000009d800 len=3D0000000000002800 > SMAP type=3D02 base=3D00000000000e0000 len=3D0000000000020000 > SMAP type=3D01 base=3D0000000000100000 len=3D00000000bfdb0000 > <...> > SMAP type=3D02 base=3D00000000fed1c000 len=3D0000000000004000 > <...> > > =A0Memory range 0xfed1c000 - 0xfed2000 belongs to ICH8 mapped to memory > configuration registers. > =A0If I correctly understand mem man page, /dev/mem allow to read only re= al > physical memory. How can I read data from this range? I am not sure what the man page says but you can certainly use /dev/mem to access mmio space. I have used 'dd' with 'if=3D/dev/mem' to dump out mmio registers on amd64. YMMV with other architectures. best Neel > > -- > Alexandr Matveev > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 20:03:11 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0360B106564A for ; Sun, 23 Jan 2011 20:03:11 +0000 (UTC) (envelope-from timon@timon.net.nz) Received: from frost.plasmahost.ru (plasmahost.ru [178.63.60.242]) by mx1.freebsd.org (Postfix) with ESMTP id A45DD8FC0A for ; Sun, 23 Jan 2011 20:03:10 +0000 (UTC) Received: from [192.168.1.50] ([85.21.143.38]) (AUTH: PLAIN timon@timon.net.nz, SSL: TLSv1/SSLv3, 256bits, CAMELLIA256-SHA) by frost.plasmahost.ru with esmtp; Sun, 23 Jan 2011 20:02:31 +0000 id 00017C63.000000004D3C8959.000012EE Message-ID: <4D3C8976.5060109@timon.net.nz> Date: Sun, 23 Jan 2011 23:03:02 +0300 From: Alexandr Matveev User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110101 Thunderbird/3.1.7 MIME-Version: 1.0 To: Neel Natu References: <4D3C4F94.7020307@timon.net.nz> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org Subject: Re: How to read non-physical memory? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 20:03:11 -0000 On 23.01.2011 22:26, Neel Natu wrote: > Hi Alexandr, > > On Sun, Jan 23, 2011 at 7:56 AM, Alexandr Matveev wrote: >> Hello. >> >> When FreeBSD boots with 'boot -v' it show SMAP: >> SMAP type=01 base=0000000000000000 len=000000000009d800 >> SMAP type=02 base=000000000009d800 len=0000000000002800 >> SMAP type=02 base=00000000000e0000 len=0000000000020000 >> SMAP type=01 base=0000000000100000 len=00000000bfdb0000 >> <...> >> SMAP type=02 base=00000000fed1c000 len=0000000000004000 >> <...> >> >> Memory range 0xfed1c000 - 0xfed2000 belongs to ICH8 mapped to memory >> configuration registers. >> If I correctly understand mem man page, /dev/mem allow to read only real >> physical memory. How can I read data from this range? > I am not sure what the man page says but you can certainly use > /dev/mem to access mmio space. > > I have used 'dd' with 'if=/dev/mem' to dump out mmio registers on > amd64. YMMV with other architectures. Thank you very much. dd if=/dev/mem of=mem.dump bs=4 count=4095 skip=1068789760 works perfectly! -- Alexandr Matveev From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 23:00:02 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7AF65106564A for ; Sun, 23 Jan 2011 23:00:02 +0000 (UTC) (envelope-from cvs-src@yandex.ru) Received: from forward16.mail.yandex.net (forward16.mail.yandex.net [95.108.253.141]) by mx1.freebsd.org (Postfix) with ESMTP id 073328FC1B for ; Sun, 23 Jan 2011 23:00:01 +0000 (UTC) Received: from smtp16.mail.yandex.net (smtp16.mail.yandex.net [95.108.252.16]) by forward16.mail.yandex.net (Yandex) with ESMTP id 6FEB523A80AF for ; Mon, 24 Jan 2011 02:00:00 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1295823600; bh=vfT4OqvtaF80akpvCKEfFD2jshsiUPH45TI1lpCDhPQ=; h=Message-ID:Date:From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=VQyihUeEyU0iihNMa7gxlv3Miayh+oEz2HPCEFr5vQQCj/eh4a9/j4cmma2Bej/Te IK67eR85Y1cXHYcJROBQoHFjef3JvlOQ0l0oN3/ySLQeb9aqwpjjlCSEhNW560BbgF Y7EV7pTaIqFRMOqERmZMXjoUZC0Ww39wwyBRh1tI= Received: from smeshariki2.local (unknown [213.138.87.92]) by smtp16.mail.yandex.net (Yandex) with ESMTPSA id 09B51160004E for ; Mon, 24 Jan 2011 02:00:00 +0300 (MSK) Message-ID: <4D3CB2AF.9050003@yandex.ru> Date: Mon, 24 Jan 2011 01:58:55 +0300 From: Ruslan Mahmatkhanov User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110106 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Tracking down a problem with php on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 23:00:02 -0000 Good day! We are using custom php application on FreeBSD 8.1R amd64. It is started with php-fpm 5.3.3 from ports as backend and nginx 0.8.54 as frontend. Several times per day this app is making self unavailable. Simple php-fpm restart solves the problem, but i need to track it down to the cause of this situation and ask for your assistance and instructions on how to debug it. Some facts about this: - I don't know how to manually reproduce this, but it happens several times every day - It happens on FreeBSD 7.x too - It happens with apache+mod_php instead php-fpm - It happens with lighthttpd instead nginx - It happens with both SHED_4BSD and SHED_ULE - It doesn't happen on php =< 5.2.12 - It happens with and w/o eaccelerator - `top -mio` shows very high (80000-90000 for VCSW) VCSW/IVCSW values for php-fpm processes and LA is more than 120 - user seeing http 502 error code in browser - php-fpm log has many of this lines in time of crash: Jan 23 17:56:58.176425 [WARNING] [pool world] server reached max_children setting (100), consider raising it Jan 23 17:56:59.528873 [WARNING] [pool world] child 686, script '/usr/local/www/world/public_html/script.php' execution timed out (10.074251 sec), terminating Jan 23 17:57:03.221677 [WARNING] [pool world] child 686 exited on signal 15 (SIGTERM) after 59.424804 seconds from start Jan 23 17:57:03.222580 [NOTICE] [pool world] child 4407 started Jan 23 17:57:03.222709 [NOTICE] Finishing ... Jan 23 17:57:03.260991 [WARNING] [pool site] child 3962, script '/usr/local/www/site/public_html/script.php' execution timed out (12.644470 sec), terminating - nginx log has many of this lines in time of crash: 2011/01/23 17:57:00 [error] 38018#0: *26006023 writev() failed (54: Connection reset by peer) while sending request to upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat& a=chatList&__path=chat_list&h=8093b9e1cf448762d5677e21bded97ae& h1=38ca8b747a46098c3b1a4f39e6658170 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:00 [error] 38016#0: *26029878 kevent() reported about an closed connection (54: Connection reset by peer) while reading response header from upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Location&a=refresh& __path=refresh&h=276f591df26a65d9a1736f6e1006f4ab& h1=3c0916c16b1fc2e7015b71e90bbc3d23 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:02 [crit] 38020#0: *26034390 open() "/tmp/nginx /client_temp/1/74/0000000741" failed (13: Permission denied) while sending request to upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& h=4a27d8d382ba9b1059412323a451ef84& h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:02 [alert] 38020#0: *26034390 http request count is zero while sending to client, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& h=4a27d8d382ba9b1059412323a451ef84& h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:03 [error] 38014#0: *25997903 upstream prematurely closed connection while reading response header from upstream, client: 109.229.69.186, server: some.server.org, request: "POST /?ctrl=Chat&a=chatList&__path=chat_list& h=c8723de73c4f8ebb98f9bf746d75e965& h1=3ab289760a009b07b73c6d96cc94a509 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" `top -mio` output in time of crash: http://pastebin.com/yrCwxnbr kernel config: http://pastebin.com/nGA0518A php port options: # cat /var/db/ports/php5/options WITH_CLI=true WITH_CGI=true WITH_FPM=true WITH_APACHE=true WITHOUT_AP2FILTER=true WITHOUT_DEBUG=true WITH_SUHOSIN=true WITH_MULTIBYTE=true WITHOUT_IPV6=true WITH_MAILHEAD=true WITHOUT_LINKTHR=true php-fpm pool configuration: [world] listen = 127.0.0.1:9002 listen.allowed_clients = 127.0.0.1 listen.owner = www listen.group = www listen.mode = 0666 user = www group = www pm = dynamic pm.max_children = 100 pm.start_servers = 20 pm.min_spare_servers = 5 pm.max_spare_servers = 35 pm.max_requests = 5000 pm.status_path = /JWorldStatus request_terminate_timeout = 10s Any help is highly appreciated. Thank you all in advance. -- Regards, Ruslan From owner-freebsd-hackers@FreeBSD.ORG Sun Jan 23 21:11:21 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A78B1065672 for ; Sun, 23 Jan 2011 21:11:21 +0000 (UTC) (envelope-from cvs-src@yandex.ru) Received: from forward18.mail.yandex.net (forward18.mail.yandex.net [95.108.253.143]) by mx1.freebsd.org (Postfix) with ESMTP id F31E78FC16 for ; Sun, 23 Jan 2011 21:11:20 +0000 (UTC) Received: from smtp19.mail.yandex.net (smtp19.mail.yandex.net [95.108.252.19]) by forward18.mail.yandex.net (Yandex) with ESMTP id 29C2D29A05FE for ; Sun, 23 Jan 2011 23:55:29 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1295816129; bh=OVLh646veNO4m30Ogjk7AQJ/j7EwZf/UHvWOLF1s8AY=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:Content-Type: Content-Transfer-Encoding; b=YsoRwbQgpozXfTZeidlQIItIjnKXggo8l8KLeRF+3SKoHSiqrD9P87ouf4J4lk14q eLFqKXv6mSX8lyyp7Kg+lQGSfO42Oy2Or5jzwgK2EY+EuXqvSltBBidbdEeHc0RHtK oIkIqaiFaj1VVyh5LN/9aV5JN3Y4Ojfd5woHgCdk= Received: from smeshariki2.local (unknown [213.138.87.92]) by smtp19.mail.yandex.net (Yandex) with ESMTPSA id D89C6287005C; Sun, 23 Jan 2011 23:55:28 +0300 (MSK) Message-ID: <4D3C957F.6080209@yandex.ru> Date: Sun, 23 Jan 2011 23:54:23 +0300 From: Ruslan Mahmatkhanov User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.2.13) Gecko/20110106 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Mon, 24 Jan 2011 01:48:40 +0000 Cc: cvs-src@yandex.ru Subject: Tracking down a problem with php on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2011 21:11:21 -0000 Good day! We are using custom php application on FreeBSD 8.1R amd64. It is started with php-fpm 5.3.3 from ports as backend and nginx 0.8.54 as frontend. Several times per day this app is making self unavailable. Simple php-fpm restart solves the problem, but i need to track it down to the cause of this situation and ask for your assistance and instructions on how to debug it. Some facts about this: - I don't know how to manually reproduce this, but it happens several times every day - It happens on FreeBSD 7.x too - It happens with apache+mod_php instead php-fpm - It happens with lighthttpd instead nginx - It happens with both SHED_4BSD and SHED_ULE - It doesn't happen on php =< 5.2.12 - It happens with and w/o eaccelerator - `top -mio` shows very high (80000-90000 for VCSW) VCSW/IVCSW values for php-fpm processes and LA is more than 120 - user seeing http 502 error code in browser - php-fpm log has many of this lines in time of crash: Jan 23 17:56:58.176425 [WARNING] [pool world] server reached max_children setting (100), consider raising it Jan 23 17:56:59.528873 [WARNING] [pool world] child 686, script '/usr/local/www/world/public_html/script.php' execution timed out (10.074251 sec), terminating Jan 23 17:57:03.221677 [WARNING] [pool world] child 686 exited on signal 15 (SIGTERM) after 59.424804 seconds from start Jan 23 17:57:03.222580 [NOTICE] [pool world] child 4407 started Jan 23 17:57:03.222709 [NOTICE] Finishing ... Jan 23 17:57:03.260991 [WARNING] [pool site] child 3962, script '/usr/local/www/site/public_html/script.php' execution timed out (12.644470 sec), terminating - nginx log has many of this lines in time of crash: 2011/01/23 17:57:00 [error] 38018#0: *26006023 writev() failed (54: Connection reset by peer) while sending request to upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat& a=chatList&__path=chat_list&h=8093b9e1cf448762d5677e21bded97ae& h1=38ca8b747a46098c3b1a4f39e6658170 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:00 [error] 38016#0: *26029878 kevent() reported about an closed connection (54: Connection reset by peer) while reading response header from upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Location&a=refresh& __path=refresh&h=276f591df26a65d9a1736f6e1006f4ab& h1=3c0916c16b1fc2e7015b71e90bbc3d23 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://world.mist-game.ru/" 2011/01/23 17:57:02 [crit] 38020#0: *26034390 open() "/tmp/nginx /client_temp/1/74/0000000741" failed (13: Permission denied) while sending request to upstream, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& h=4a27d8d382ba9b1059412323a451ef84& h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://world.mist-game.ru/" 2011/01/23 17:57:02 [alert] 38020#0: *26034390 http request count is zero while sending to client, client: xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& h=4a27d8d382ba9b1059412323a451ef84& h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" 2011/01/23 17:57:03 [error] 38014#0: *25997903 upstream prematurely closed connection while reading response header from upstream, client: 109.229.69.186, server: some.server.org, request: "POST /?ctrl=Chat&a=chatList&__path=chat_list& h=c8723de73c4f8ebb98f9bf746d75e965& h1=3ab289760a009b07b73c6d96cc94a509 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: "http://some.server.org/" `top -mio` output in time of crash: http://pastebin.com/yrCwxnbr kernel config: http://pastebin.com/nGA0518A php port options: # cat /var/db/ports/php5/options WITH_CLI=true WITH_CGI=true WITH_FPM=true WITH_APACHE=true WITHOUT_AP2FILTER=true WITHOUT_DEBUG=true WITH_SUHOSIN=true WITH_MULTIBYTE=true WITHOUT_IPV6=true WITH_MAILHEAD=true WITHOUT_LINKTHR=true php-fpm pool configuration: [world] listen = 127.0.0.1:9002 listen.allowed_clients = 127.0.0.1 listen.owner = www listen.group = www listen.mode = 0666 user = www group = www pm = dynamic pm.max_children = 100 pm.start_servers = 20 pm.min_spare_servers = 5 pm.max_spare_servers = 35 pm.max_requests = 5000 pm.status_path = /JWorldStatus request_terminate_timeout = 10s Any help is highly appreciated. Thank you all in advance. PS. Please keep me in cc:. I'm not subscribed to freebsd-hackers@. -- Regards, Ruslan From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 04:53:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BB65106567A for ; Mon, 24 Jan 2011 04:53:50 +0000 (UTC) (envelope-from neelnatu@gmail.com) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id F3E828FC25 for ; Mon, 24 Jan 2011 04:53:49 +0000 (UTC) Received: by wwi17 with SMTP id 17so2801691wwi.1 for ; Sun, 23 Jan 2011 20:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Ky05XY1FGu2OABNWQQMaiejhLuqhAlozeSK0FwY++go=; b=v4Rdzti6F+MXAK+nwPP4SOkBrtXvCz+ITVm9LkrjsmbbeTM+j0wxg1VS/UXsA4ZxQ0 wZxWZOdhtpV6OjTF7oySAvkmqKABntnQdJSqvu4IumLFPcQZIfJ0HbGVr+eGjwk0kQ+W /hfhqvTOjh4bXHev8OB3UcwD6d3U/lbPlG3Nw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=N/KDz5MKr0Yukg4Hy91/ic5fu0TWopyjp7tAbv4qZOUEt1qUAD+3j4JAGvujVVD19c XMZcB+4dYdwZH1TzdB0IYW8bGbk01MaiqaLBq97wuMQvA5x4IO/NlllUxKTQ9rO7aV1W 7IBznSE9OYf32/EQ6M9FHxshetRjj+yJ1UviM= MIME-Version: 1.0 Received: by 10.216.164.14 with SMTP id b14mr3553170wel.33.1295843137324; Sun, 23 Jan 2011 20:25:37 -0800 (PST) Received: by 10.216.168.134 with HTTP; Sun, 23 Jan 2011 20:25:37 -0800 (PST) In-Reply-To: References: Date: Sun, 23 Jan 2011 20:25:37 -0800 Message-ID: From: Neel Natu To: Ali Mashtizadeh Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers Subject: Re: Exporting kernel symbols X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 04:53:50 -0000 Hi Ali, On Sat, Jan 22, 2011 at 7:20 PM, Ali Mashtizadeh wr= ote: > Hi Folks, > > I tried to build a geom kernel module that uses the alq(9) facility to > log some data. The module builds fine but it seems that the kernel > isn't exporting the alq(9) symbols. Could someone point me how I can > export particular symbols? > Do you have the kernel compiled with the 'alq' option defined? FreeBSD 8.1 GENERIC does not define this by default. best Neel > -- > Ali Mashtizadeh > =D8=B9=D9=84=DB=8C =D9=85=D8=B4=D8=AA=DB=8C =D8=B2=D8=A7=D8=AF=D9=87 > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 08:13:48 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2607106564A for ; Mon, 24 Jan 2011 08:13:47 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 896548FC13 for ; Mon, 24 Jan 2011 08:13:47 +0000 (UTC) Received: by wyf19 with SMTP id 19so3814045wyf.13 for ; Mon, 24 Jan 2011 00:13:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=/zXGiMcZuLWK0orIYOpxUaS5AkZ8ih/xDz/yAmmf/pg=; b=H0hljmhC19BVbIC18XfE28LM3pcBZSKK6aCFLEL37+4RNSikNc/pBnFZcRxIsVy41S 4AfVNsZtgvsFlEn7MJ3bvaIN+bpRjfQwGQWck+cAhRx7rXMGVyFLZGjrOQr/EWEAa8ex 3A1StqlWqT+egeSGxnOfV9fmCUu6fthcOECSA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=RDlKRRG85pAiBobj5H716JIBF2VMjBRHwv+zDby++uWa3muzyHsLUFRm8yrOi4w0ay oxvCphqpcXiGSEaCZZJmsru82S6JXE1Bwc0ZO5AEGYhAjlftzPC27I5HsSWP+5/kWzcS JfRe/pj88Aig3jV1HuZixv4Au5BRRcvyQ1vWg= MIME-Version: 1.0 Received: by 10.216.141.75 with SMTP id f53mr3301207wej.16.1295856826294; Mon, 24 Jan 2011 00:13:46 -0800 (PST) Received: by 10.216.254.226 with HTTP; Mon, 24 Jan 2011 00:13:46 -0800 (PST) Date: Mon, 24 Jan 2011 00:13:46 -0800 Message-ID: From: Garrett Cooper To: Peter Jeremy Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "Simon L. B. Nielsen" , Matthew Fleming , freebsd-hackers@freebsd.org, Mike Tancsa Subject: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 08:13:48 -0000 On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: > On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wro= te: >>Perhaps we should just set the tinderbox up to sync directly of cvsup-mas= ter instead if that makes it more useful? > > Can cvsup-master still lose atomicity of commits? =A0I suspect it can, > in which case syncing directly off the SVN master would seem a better > approach. I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I wonder if it's time to look at another solution to this problem as these annoying stability issues don't appear to be going away. What about git? Just some things I'm able to rattle off that come to mind with git.. Some arguments `for git'... 1. One tool to rule them all: - cvsup/csup can be replaced with git archive [1]. - cvs git scales a bit better. - less support cost for p4 and lower likelihood of downtime in the event of critical failure (perforce's proprietary DB is a pain to recover I've recently discovered from other dealings). - svn <-> cvs exporter is no longer required as it's all one SCM. - As a side-effect, the bits present in CVS and SVN would now be 100% in sync, unlike cvs which can lead svn in terms of commits (at least that was the case when I last talked to someone about version numbering in pkg_install done by re@). 2. More evolved tool: - branches are cheap and can be local or remote. - distributed SCM seem to work well with large groups of developers. - works better with branching and merging from what I've seen. Some arguments against git... - The one caveat to cvsup/csup that's awesome is its componentization capability, i.e. being able to selectively download components in src / ports; I'm not 100% sure but there doesn't appear to be a clear analog in git. It might be achievable through gits remote. in git-config, git-remote, etc, but I would need to prototype whether or not this is true. - Higher learning curve. - Some slightly annoying nits with stashing local changes when working on separate branches (need to talk to git maintainers). - Some more git experienced folks could comment here, but it would be nice to unify all of the systems under `one flag' for the sake of simplicity and hopefully the sanity of the tool maintainers (Simon, et all). Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 11:33:06 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id D20DE106567A; Mon, 24 Jan 2011 11:33:06 +0000 (UTC) Date: Mon, 24 Jan 2011 11:33:06 +0000 From: Alexander Best To: Garrett Cooper Message-ID: <20110124113306.GA79890@freebsd.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Cc: "Simon L. B. Nielsen" , Mike Tancsa , Matthew Fleming , freebsd-hackers@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 11:33:06 -0000 On Mon Jan 24 11, Garrett Cooper wrote: > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: > > On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wrote: > >>Perhaps we should just set the tinderbox up to sync directly of cvsup-master instead if that makes it more useful? > > > > Can cvsup-master still lose atomicity of commits?  I suspect it can, > > in which case syncing directly off the SVN master would seem a better > > approach. > > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I > wonder if it's time to look at another solution to this problem as > these annoying stability issues don't appear to be going away. What > about git? > > Just some things I'm able to rattle off that come to mind with git.. it would also be nice to have github running on freebsd.org. that way it would be much easier to discuss src changes without having to point people at a file, a function or even a specific line. also it would finally kill the mailinglists, which have lots of issues: spam, broken mailman installation, people going berserker when they see lines > 80 etc. there have been a few attempts to introduce a code review system, but since that was all hosted on foreign websites the idea never cought on and afaik those websites weren't being supported/promoted by freebsd.org. but personally i don't expect a change like this to happen in the near future. basically most of the freebsd administrative people are quite conservative. i wouldn't be surprised if some them are still trying to run freebsd on their typewriters. ;) cheers. alex > > Some arguments `for git'... > > 1. One tool to rule them all: > - cvsup/csup can be replaced with git archive [1]. > - cvs git scales a bit better. > - less support cost for p4 and lower likelihood of downtime in the > event of critical failure (perforce's proprietary DB is a pain to > recover I've recently discovered from other dealings). > - svn <-> cvs exporter is no longer required as it's all one SCM. > - As a side-effect, the bits present in CVS and SVN would now be > 100% in sync, unlike cvs which can lead svn in terms of commits (at > least that was the case when I last talked to someone about version > numbering in pkg_install done by re@). > 2. More evolved tool: > - branches are cheap and can be local or remote. > - distributed SCM seem to work well with large groups of developers. > - works better with branching and merging from what I've seen. > > Some arguments against git... > - The one caveat to cvsup/csup that's awesome is its componentization > capability, i.e. being able to selectively download components in src > / ports; I'm not 100% sure but there doesn't appear to be a clear > analog in git. It might be achievable through gits remote. in > git-config, git-remote, etc, but I would need to prototype whether or > not this is true. > - Higher learning curve. > - Some slightly annoying nits with stashing local changes when working > on separate branches (need to talk to git maintainers). > - > > Some more git experienced folks could comment here, but it would > be nice to unify all of the systems under `one flag' for the sake of > simplicity and hopefully the sanity of the tool maintainers (Simon, et > all). > Thanks! > -Garrett -- a13x From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 13:33:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BB36106564A for ; Mon, 24 Jan 2011 13:33:33 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id DC08C8FC16 for ; Mon, 24 Jan 2011 13:33:32 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PhMXz-0002Lx-J7 for freebsd-hackers@freebsd.org; Mon, 24 Jan 2011 14:33:31 +0100 Received: from 49-168.dsl.iskon.hr ([89.164.49.168]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 24 Jan 2011 14:33:31 +0100 Received: from ivoras by 49-168.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 24 Jan 2011 14:33:31 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Mon, 24 Jan 2011 14:33:25 +0100 Lines: 42 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 49-168.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 In-Reply-To: Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 13:33:33 -0000 On 24.1.2011 9:13, Garrett Cooper wrote: > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: >> On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wrote: >>> Perhaps we should just set the tinderbox up to sync directly of cvsup-master instead if that makes it more useful? >> >> Can cvsup-master still lose atomicity of commits? I suspect it can, >> in which case syncing directly off the SVN master would seem a better >> approach. I think des is working on "svnup" to work directly on the SVN tree. > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I > wonder if it's time to look at another solution to this problem as > these annoying stability issues don't appear to be going away. What > about git? As long as we're choosing bikeshed colour, I would like to drop "mercurial" here :) Mainly because of this: > - Higher learning curve. I found Mercurial to have an easier learning curve and to be something like a "DSCM for the users of CVS/SVN". > - Some slightly annoying nits with stashing local changes when working > on separate branches (need to talk to git maintainers). I don't know if we're talking about the same thing, but I've also noticed git tends to do things the long way around which should be simple. Git's also much "lower level". They both support pretty much the same feature set; here's a cute but dated comparison: http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/ Hg is/was AFAIK used by Sun. Anyway, personally, svn is good enough :) From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 13:54:26 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D0D81065670 for ; Mon, 24 Jan 2011 13:54:26 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from smtp2.dlr.de (smtp1.dlr.de [129.247.252.32]) by mx1.freebsd.org (Postfix) with ESMTP id 13EEE8FC13 for ; Mon, 24 Jan 2011 13:54:25 +0000 (UTC) Received: from DLREXHUB01.intra.dlr.de ([172.21.152.130]) by smtp2.dlr.de with Microsoft SMTPSVC(6.0.3790.4675); Mon, 24 Jan 2011 14:33:16 +0100 Received: from beagle.kn.op.dlr.de (129.247.178.136) by smtp.dlr.de (172.21.152.151) with Microsoft SMTP Server (TLS) id 14.1.270.1; Mon, 24 Jan 2011 14:33:15 +0100 Date: Mon, 24 Jan 2011 14:33:16 +0100 From: Harti Brandt X-X-Sender: brandt_h@beagle.kn.op.dlr.de To: Eugene Grosbein In-Reply-To: <4D3C43EE.8060307@rdtc.ru> Message-ID: <20110124143206.V47002@beagle.kn.op.dlr.de> References: <4D3C43EE.8060307@rdtc.ru> X-OpenPGP-Key: harti@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" X-Originating-IP: [129.247.178.136] X-OriginalArrivalTime: 24 Jan 2011 13:33:16.0427 (UTC) FILETIME=[41A4DDB0:01CBBBCB] Cc: hackers@freebsd.org, "net@freebsd.org" Subject: Re: Querying bsnmpd through /var/run/snmpd.sock X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Harti Brandt List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 13:54:26 -0000 On Sun, 23 Jan 2011, Eugene Grosbein wrote: EG>bsnmpd running with mibII module opens local socket /var/run/snmpd.sock EG>mentioned in snmp_mibII(3) manual page: EG> EG> The mibII module opens a socket that is used to execute all network EG> related ioctl(2) functions. This socket is globally available under the EG> name mib_netsock. EG> EG>How do I use the socket? I hope to be able to call mib_find_if_sys() function EG>from another process using the socket. Is there a documentation for this? The socket works just like a UDP socket with the additional plus that the daemon knows whether you're root or not. As I said in my previous mail, easiest would be to implement an additional table with the sysindex as index and ifIndex as a row. harti From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 13:54:30 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61E2C1065694 for ; Mon, 24 Jan 2011 13:54:30 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id CDA8B8FC12 for ; Mon, 24 Jan 2011 13:54:29 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PhMsG-00043X-G0 for freebsd-hackers@freebsd.org; Mon, 24 Jan 2011 14:54:28 +0100 Received: from 49-168.dsl.iskon.hr ([89.164.49.168]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 24 Jan 2011 14:54:28 +0100 Received: from ivoras by 49-168.dsl.iskon.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 24 Jan 2011 14:54:28 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Mon, 24 Jan 2011 14:54:22 +0100 Lines: 95 Message-ID: References: <4D3CB2AF.9050003@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 49-168.dsl.iskon.hr User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 In-Reply-To: <4D3CB2AF.9050003@yandex.ru> Subject: Re: Tracking down a problem with php on FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 13:54:30 -0000 On 23.1.2011 23:58, Ruslan Mahmatkhanov wrote: > > Good day! > > We are using custom php application on FreeBSD 8.1R amd64. It is started > with php-fpm 5.3.3 from ports as backend and nginx 0.8.54 as frontend. > Several times per day this app is making self unavailable. I think it would be more appropriate to ask this on the stable@ list. > Simple php-fpm restart solves the problem, but i need to track it down > to the cause of this situation and ask for your assistance and > instructions on how to debug it. Some facts about this: On one hand, FPM is said to be very experimental... Personally, I've been using apache22-worker or apache22-event + mod_fcgid for years without trouble. > - I don't know how to manually reproduce this, but it happens several > times every day > - It happens on FreeBSD 7.x too > - It happens with apache+mod_php instead php-fpm > - It happens with lighthttpd instead nginx > - It happens with both SHED_4BSD and SHED_ULE > - It doesn't happen on php =< 5.2.12 > - It happens with and w/o eaccelerator It looks very application-specific, possibly not really an OS problem (or maybe a problem of different expectations from the OS when porting from Linux). > - `top -mio` shows very high (80000-90000 for VCSW) VCSW/IVCSW values > for php-fpm processes and LA is more than 120 How many "real" user request are in these 120? Do any users at the time of problem (this doesn't look like a "crash") receive valid responses? > - user seeing http 502 error code in browser > - php-fpm log has many of this lines in time of crash: > Jan 23 17:56:58.176425 [WARNING] [pool world] server reached > max_children setting (100), consider raising it Did you try raising it? Does the error happen ONLY when this limit is reached? > 2011/01/23 17:57:00 [error] 38018#0: *26006023 writev() failed (54: > Connection reset by peer) while sending request to upstream, client: > xx.xx.xx.xx, server: some.server.org, request: "POST /?ctrl=Chat& > a=chatList&__path=chat_list&h=8093b9e1cf448762d5677e21bded97ae& > h1=38ca8b747a46098c3b1a4f39e6658170 HTTP/1.1", upstream: > "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: > "http://some.server.org/" > 2011/01/23 17:57:00 [error] 38016#0: *26029878 kevent() reported > about an closed connection (54: Connection reset by peer) while > reading response header from upstream, client: xx.xx.xx.xx, server: > some.server.org, request: "POST /?ctrl=Location&a=refresh& > __path=refresh&h=276f591df26a65d9a1736f6e1006f4ab& > h1=3c0916c16b1fc2e7015b71e90bbc3d23 HTTP/1.1", upstream: > "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: > "http://some.server.org/" > 2011/01/23 17:57:02 [crit] 38020#0: *26034390 open() "/tmp/nginx > /client_temp/1/74/0000000741" failed (13: Permission denied) while > sending request to upstream, client: xx.xx.xx.xx, server: > some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& > h=4a27d8d382ba9b1059412323a451ef84& > h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: > "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: > "http://some.server.org/" > 2011/01/23 17:57:02 [alert] 38020#0: *26034390 http request count is > zero while sending to client, client: xx.xx.xx.xx, server: > some.server.org, request: "POST /?ctrl=Chat&a=send&__path=chat_send& > h=4a27d8d382ba9b1059412323a451ef84& > h1=b0a53c86e3c744a01356a5030559ed1a HTTP/1.1", upstream: > "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: > "http://some.server.org/" > 2011/01/23 17:57:03 [error] 38014#0: *25997903 upstream prematurely > closed connection while reading response header from upstream, > client: 109.229.69.186, server: some.server.org, request: "POST > /?ctrl=Chat&a=chatList&__path=chat_list& > h=c8723de73c4f8ebb98f9bf746d75e965& > h1=3ab289760a009b07b73c6d96cc94a509 HTTP/1.1", upstream: > "fastcgi://127.0.0.1:9002", host: "some.server.org", referrer: > "http://some.server.org/" These are some very varied errors, not especially consistent with each other. Did you try some generic socket & TCP tuning like described in http://serverfault.com/questions/64356/freebsd-performance-tuning-sysctls-loader-conf-kernel ? Other than that, you will probably have to debug the php-fpm processes. Start by observing in which state they are (top without "-mio"). If the processes are blocking, try "procstat -k " on them. From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 14:06:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 610271065670; Mon, 24 Jan 2011 14:06:47 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id C16338FC0C; Mon, 24 Jan 2011 14:06:46 +0000 (UTC) Received: by wwf26 with SMTP id 26so3927651wwf.31 for ; Mon, 24 Jan 2011 06:06:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=4k/KykZxGsvD/3ybM5GAjOT7ROGBBOc5bR8gVyw3Y5A=; b=qGLcKtmXr0epKJur01ZLceJqDExCYn5FkBlO5xMAFOURCRx6bz7LTujKJmRqxKuAID aJTzuNFrG6i0l6wTrEwEXRS9YGY5IO9lHH8lwKnMA8c/mH51UdMxu/m6pDJwGbtp/7BZ 1l/lmo/fTrxHMaB4ld8avK6zXMh0PCdutQEx0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=eLsOgyDursv+TaPGNXblgcSEQBRed0dSly/RfuMeLs7hIzvoktfgdeog7dGrlbrhEJ UPGXiGCalkGRisL+RoGkGTXw/AwKaMyLn/0FWQqmkO5HvBLhlhbQx3W8vG+l9jywAMOU gVYh8dW9enVfYfoYwhu/HYSR+XLCz/ii3kK4s= MIME-Version: 1.0 Received: by 10.216.49.15 with SMTP id w15mr2412380web.1.1295878004865; Mon, 24 Jan 2011 06:06:44 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Mon, 24 Jan 2011 06:06:44 -0800 (PST) In-Reply-To: References: Date: Mon, 24 Jan 2011 06:06:44 -0800 X-Google-Sender-Auth: 7WWw1lQqYvhqEAFxKsCsIwWYH8s Message-ID: From: Garrett Cooper To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 14:06:47 -0000 On Mon, Jan 24, 2011 at 5:33 AM, Ivan Voras wrote: > On 24.1.2011 9:13, Garrett Cooper wrote: >> >> On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy =A0wr= ote: >>> >>> On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" >>> =A0wrote: >>>> >>>> Perhaps we should just set the tinderbox up to sync directly of >>>> cvsup-master instead if that makes it more useful? >>> >>> Can cvsup-master still lose atomicity of commits? =A0I suspect it can, >>> in which case syncing directly off the SVN master would seem a better >>> approach. > > I think des is working on "svnup" to work directly on the SVN tree. > >> I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I >> wonder if it's time to look at another solution to this problem as >> these annoying stability issues don't appear to be going away. What >> about git? > > As long as we're choosing bikeshed colour, I would like to drop "mercuria= l" > here :) > > Mainly because of this: > >> - Higher learning curve. > > I found Mercurial to have an easier learning curve and to be something li= ke > a "DSCM for the users of CVS/SVN". > >> - Some slightly annoying nits with stashing local changes when working >> on separate branches (need to talk to git maintainers). > > I don't know if we're talking about the same thing, but I've also noticed > git tends to do things the long way around which should be simple. Git's > also much "lower level". > > They both support pretty much the same feature set; here's a cute but dat= ed > comparison: > > http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/ > > Hg is/was AFAIK used by Sun. > > Anyway, personally, svn is good enough :) Ok. Obviously this was just a fleeting thought so let's close the git topic. I do hope that whatever des has cooking up though can replace cvsup/csup reliably though, and if CVS would die at least that would be nice... Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 18:50:58 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2817106564A; Mon, 24 Jan 2011 18:50:58 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id CAE1E8FC08; Mon, 24 Jan 2011 18:50:58 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 100DD2282A; Mon, 24 Jan 2011 11:30:09 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id EE8345CB5; Mon, 24 Jan 2011 13:31:02 -0500 (EST) Date: Mon, 24 Jan 2011 13:31:02 -0500 From: Diane Bruce To: Ivan Voras Message-ID: <20110124183102.GB68940@night.db.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 18:50:59 -0000 On Mon, Jan 24, 2011 at 02:33:25PM +0100, Ivan Voras wrote: > On 24.1.2011 9:13, Garrett Cooper wrote: > >On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: > >>On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" > >>wrote: > >>>Perhaps we should just set the tinderbox up to sync directly of ... > > As long as we're choosing bikeshed colour, I would like to drop > "mercurial" here :) As long as it is not GPL. - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 19:08:58 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD87D10656A6 for ; Mon, 24 Jan 2011 19:08:58 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6624A8FC17 for ; Mon, 24 Jan 2011 19:08:58 +0000 (UTC) Received: by vws9 with SMTP id 9so1934394vws.13 for ; Mon, 24 Jan 2011 11:08:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=jxY3j2VrcXMZNe8QuZRNbz4mfUz5lOARFK2rTuM2394=; b=MIF3q/59T/bywcNC+Uy+tHkfGc6kH0yZl3kbEqaE0b+V17Mu/QzxEBS+FEnTcEQBLl altcWobU1AeqZBsJrrU8czeR3nFAPZpJ+r9YJmRXKkU96pOGcT5NeRlpWNIgFxgjSioO xPIpcT4AIFSExKOTmx8R9Qnv5NQ+rAU1C8bG0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=D/tSyEj9Gs6nHXtnnv3yzeiZZwHoki482H8zC5ryIb5cUWZfzVBSYOimXG648SKacS Dy/i94YzUWcSxUFszMfql87IoSCnJ7FYczmqPnd1nyFzW9ko+tlxOaBO7AfQWrrzUbRZ vLcce98I/uU3IESbiz6mICXexdHx/A93sRuc4= Received: by 10.229.228.2 with SMTP id jc2mr4097849qcb.177.1295896136771; Mon, 24 Jan 2011 11:08:56 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Mon, 24 Jan 2011 11:02:37 -0800 (PST) In-Reply-To: <20110124183102.GB68940@night.db.net> References: <20110124183102.GB68940@night.db.net> From: Ivan Voras Date: Mon, 24 Jan 2011 20:02:37 +0100 X-Google-Sender-Auth: txs2aYLJ1RdS0leCPs2xdnzAOuw Message-ID: To: Diane Bruce Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 19:08:58 -0000 On 24 January 2011 19:31, Diane Bruce wrote: > As long as it is not GPL. Unless there's a missing smiley in that sentence there, it is a tough requirement. Of the major SCMs, only Subversion is non-GPL-ed (even CVS is...). From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 19:48:32 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FD90106564A; Mon, 24 Jan 2011 19:48:32 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id E9C198FC15; Mon, 24 Jan 2011 19:48:31 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id D4C802282A; Mon, 24 Jan 2011 12:47:35 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id 376D75CA3; Mon, 24 Jan 2011 14:48:30 -0500 (EST) Date: Mon, 24 Jan 2011 14:48:30 -0500 From: Diane Bruce To: Ivan Voras Message-ID: <20110124194830.GA70207@night.db.net> References: <20110124183102.GB68940@night.db.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, Diane Bruce Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 19:48:32 -0000 On Mon, Jan 24, 2011 at 08:02:37PM +0100, Ivan Voras wrote: > On 24 January 2011 19:31, Diane Bruce wrote: > > > As long as it is not GPL. > > Unless there's a missing smiley in that sentence there, it is a tough IRL I'm known to be very dry humoured, I am deadly in e-mail or IRC. > requirement. Of the major SCMs, only Subversion is non-GPL-ed (even QED > CVS is...). CVS is/was dual licenced. There is also the work openbsd started with CVS sometime ago. Given the work that is being done on clang/llvm to get a non GPL compiler into the tree, perhaps efforts would be better spent on finding SCMs that were also non GPL. There certainly would not be a chance of putting mercurial or git into base for example. Perhaps a point to consider. - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 20:12:22 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05C311065672; Mon, 24 Jan 2011 20:12:22 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 64EAF8FC15; Mon, 24 Jan 2011 20:12:20 +0000 (UTC) Received: by wwf26 with SMTP id 26so4293635wwf.31 for ; Mon, 24 Jan 2011 12:12:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=8pEQk+U6EAV+d/xcNgioUbCZLa/vXBLNvizwMg2z6qU=; b=bMSWf1p3jZIFG2k1lVzcn1lm2cGhKLdkrEbHEZZ1KM/n/5f12+luRQciQ4gA/Q+8ln ZfELWxwu3VppujvIJgSh73FzW1ECY9tya0O6AGTgWbvXlaG1I5bwieEew+Lh0rmbvp6J v444cF+jYaoiITg9a+16hvGuqZNy7UzljukcQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=iWT7Ge5IstxEzeXrFDj7MhT2zFTasj9g9ABkTTdCew4wOQ8inp3wDGBs5WR0ypC3UW by/Y3gCiC+1mhGXHun7hgyB4n7E2JG8Ro9ks1hJdwdTg829J5SCoftFTmu6ptw7BBHsS ejcKT+AyEB0MykQLKRk8IhqlEbKATHK6Qps3c= MIME-Version: 1.0 Received: by 10.216.191.215 with SMTP id g65mr2870960wen.16.1295899939814; Mon, 24 Jan 2011 12:12:19 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Mon, 24 Jan 2011 12:12:19 -0800 (PST) In-Reply-To: <20110124194830.GA70207@night.db.net> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> Date: Mon, 24 Jan 2011 12:12:19 -0800 X-Google-Sender-Auth: L8S4j6933ZQqhmDmEW-bv-SlX0M Message-ID: From: Garrett Cooper To: Diane Bruce Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 20:12:22 -0000 On Mon, Jan 24, 2011 at 11:48 AM, Diane Bruce wrote: > On Mon, Jan 24, 2011 at 08:02:37PM +0100, Ivan Voras wrote: >> On 24 January 2011 19:31, Diane Bruce wrote: >> >> > As long as it is not GPL. >> >> Unless there's a missing smiley in that sentence there, it is a tough > > IRL I'm known to be very dry humoured, I am deadly in e-mail or IRC. > >> requirement. Of the major SCMs, only Subversion is non-GPL-ed (even > > QED > >> CVS is...). > > CVS is/was dual licenced. There is also the work openbsd started with CVS > sometime ago. > > Given the work that is being done on clang/llvm to get a non GPL compiler > into the tree, perhaps efforts would be better spent on finding SCMs > that were also non GPL. There certainly would not be a chance of putting > mercurial or git into base for example. But we don't compile CVS, SVN, etc into our sources. I thought that was the whole point of doing the gcc -> clang (and friends) conversion, not that the GPL is an undesirable license. Maybe I was missing something about the whole textproc stuff being replaced though (groff, etc) with NetBSD equivalents *shrugs*. Given that this is getting more philosophical than technical, maybe we should move the discussion elsewhere (i.e. not hackers@)? > Perhaps a point to consider. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 21:21:13 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B595106564A; Mon, 24 Jan 2011 21:21:13 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id 0B71F8FC08; Mon, 24 Jan 2011 21:21:12 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id CD3832282A; Mon, 24 Jan 2011 14:20:16 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id 661B05CA3; Mon, 24 Jan 2011 16:21:11 -0500 (EST) Date: Mon, 24 Jan 2011 16:21:11 -0500 From: Diane Bruce To: Garrett Cooper Message-ID: <20110124212111.GA71398@night.db.net> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, Diane Bruce , Ivan Voras Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 21:21:13 -0000 On Mon, Jan 24, 2011 at 12:12:19PM -0800, Garrett Cooper wrote: > On Mon, Jan 24, 2011 at 11:48 AM, Diane Bruce wrote: > > On Mon, Jan 24, 2011 at 08:02:37PM +0100, Ivan Voras wrote: > >> On 24 January 2011 19:31, Diane Bruce wrote: > >> ... > But we don't compile CVS, SVN, etc into our sources. I thought which rcs. If you check, the file format on the SVN server is rcs compatible, in fact local checkins using svn will work just fine. > Given that this is getting more philosophical than technical, > maybe we should move the discussion elsewhere (i.e. not hackers@)? I'd suggest we support fossil (devel/fossil) http://fossil-scm.org No. But in future, keep in mind us old timers are not _that_ conservative. > > Perhaps a point to consider. > > Thanks! You are welcome. > -Garrett - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Mon Jan 24 22:17:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 921E0106564A for ; Mon, 24 Jan 2011 22:17:57 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 1E56E8FC0C for ; Mon, 24 Jan 2011 22:17:56 +0000 (UTC) Received: from park.js.berklix.net (p5B22CF75.dip.t-dialin.net [91.34.207.117]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p0OMHsuR049407 for ; Mon, 24 Jan 2011 22:17:55 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p0OMHjV7069931 for ; Mon, 24 Jan 2011 23:17:46 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p0OMHe1H085270 for ; Mon, 24 Jan 2011 23:17:45 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201101242217.p0OMHe1H085270@fire.js.berklix.net> To: freebsd-hackers@freebsd.org From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Mon, 24 Jan 2011 14:33:25 +0100." Date: Mon, 24 Jan 2011 23:17:40 +0100 Sender: jhs@berklix.com Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2011 22:17:57 -0000 > They both support pretty much the same feature set; here's a cute but > dated comparison: > > http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/ http://wiki.freebsd.org/VersionControl Table of features comparing SVN HG GIT MTN http://wiki.freebsd.org/ section Development resources (near base of page) Clickable to docs on Mercurial & HG & Git etc. Anyone having info not documented there already, could help by contributing there ? Maybe add another tool column or add another attribute row, whatever ? Disclaimer: I'm not an author, just a reader, no opinion on one over the other, but that table seems a central place to work toward informed choice ? Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 00:53:31 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 263BB106566B for ; Tue, 25 Jan 2011 00:53:31 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id CCB608FC14 for ; Tue, 25 Jan 2011 00:53:30 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id p0P0p1Yi030039 for ; Mon, 24 Jan 2011 17:51:01 -0700 (MST) (envelope-from imp@bsdimp.com) Message-ID: <4D3E1E75.20107@bsdimp.com> Date: Mon, 24 Jan 2011 17:51:01 -0700 From: Warner Losh User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101211 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 00:53:31 -0000 Regardless of the benefits, unless there's someone to setup the infrastructure to run things, we're not going to change. We should at least have a master seed for git that people can pull from before we talk about doing anything further. git has the ability to pull from svn, so this should be relatively easy to get going. Warner On 01/24/2011 01:13, Garrett Cooper wrote: > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: >> On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wrote: >>> Perhaps we should just set the tinderbox up to sync directly of cvsup-master instead if that makes it more useful? >> Can cvsup-master still lose atomicity of commits? I suspect it can, >> in which case syncing directly off the SVN master would seem a better >> approach. > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I > wonder if it's time to look at another solution to this problem as > these annoying stability issues don't appear to be going away. What > about git? > > Just some things I'm able to rattle off that come to mind with git.. > > Some arguments `for git'... > > 1. One tool to rule them all: > - cvsup/csup can be replaced with git archive [1]. > - cvs git scales a bit better. > - less support cost for p4 and lower likelihood of downtime in the > event of critical failure (perforce's proprietary DB is a pain to > recover I've recently discovered from other dealings). > - svn<-> cvs exporter is no longer required as it's all one SCM. > - As a side-effect, the bits present in CVS and SVN would now be > 100% in sync, unlike cvs which can lead svn in terms of commits (at > least that was the case when I last talked to someone about version > numbering in pkg_install done by re@). > 2. More evolved tool: > - branches are cheap and can be local or remote. > - distributed SCM seem to work well with large groups of developers. > - works better with branching and merging from what I've seen. > > Some arguments against git... > - The one caveat to cvsup/csup that's awesome is its componentization > capability, i.e. being able to selectively download components in src > / ports; I'm not 100% sure but there doesn't appear to be a clear > analog in git. It might be achievable through gits remote. in > git-config, git-remote, etc, but I would need to prototype whether or > not this is true. > - Higher learning curve. > - Some slightly annoying nits with stashing local changes when working > on separate branches (need to talk to git maintainers). > - > > Some more git experienced folks could comment here, but it would > be nice to unify all of the systems under `one flag' for the sake of > simplicity and hopefully the sanity of the tool maintainers (Simon, et > all). > Thanks! > -Garrett > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 10:32:16 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C5E0106564A; Tue, 25 Jan 2011 10:32:16 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id DE1C18FC08; Tue, 25 Jan 2011 10:32:15 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p0PAWCkw035959 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 25 Jan 2011 02:32:12 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p0PAWCt4035958; Tue, 25 Jan 2011 02:32:12 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA26732; Tue, 25 Jan 11 02:23:37 PST Date: Tue, 25 Jan 2011 02:22:34 -0800 From: perryh@pluto.rain.com To: db@db.net Message-Id: <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> In-Reply-To: <20110124194830.GA70207@night.db.net> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 10:32:16 -0000 Diane Bruce wrote: > There certainly would not be a chance of putting > mercurial or git into base for example. Completely apart from licensing, another strike against mercurial is that it is written in Python, so it couldn't go into base unless Python also went into base. BTW this topic came up on ports@ about 3 months ago: http://lists.freebsd.org/pipermail/freebsd-ports/2010-September/063675.html From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 12:10:59 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8498D1065670 for ; Tue, 25 Jan 2011 12:10:59 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 375938FC13 for ; Tue, 25 Jan 2011 12:10:58 +0000 (UTC) Received: by qyk8 with SMTP id 8so3768253qyk.13 for ; Tue, 25 Jan 2011 04:10:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=KsYPa63rdqmQ/KnNaHtVD6Rp4+iL9XloIQYIAzwWUk4=; b=pB4pNczKgSt6ZjBaEeQ7qZpOEnyXAP2fzUdGT3bODpxB2Nipevj2ajPE+5WbHEf8Fi xoMVFBOa50TKb5YIAWlKnIRLa4THNSO4PgnElLNRZCfvekA2tHBNXX5Oil+3Rw+5dXiX 3UOeJPInwxh7C6ApH8cgKv8ItKD7Lq64BKihg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=vF8ZyM9vQRAcOzlvOzPG5xsSH71cVbD3Z21gl3LXWG/7O3OcVaofMSr/ht439MkcHS jWfUOXzkr4BYhzI9V/xNuMD0Q2iSapjNQ5NoVgyyk9XFAD/TNXO7CRf+YslNig6l1p7s anHSh1rkzv83XMeYrG3YQC0KN7VePiDk34xus= Received: by 10.229.213.130 with SMTP id gw2mr4720898qcb.253.1295957458076; Tue, 25 Jan 2011 04:10:58 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Tue, 25 Jan 2011 04:10:17 -0800 (PST) In-Reply-To: <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> From: Ivan Voras Date: Tue, 25 Jan 2011 13:10:17 +0100 X-Google-Sender-Auth: OU448259ZX0dzeiY1JCN47rxVQE Message-ID: To: perryh@pluto.rain.com Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, db@db.net Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 12:10:59 -0000 On 25 January 2011 11:22, wrote: > Diane Bruce wrote: > >> There certainly would not be a chance of putting >> mercurial or git into base for example. > > Completely apart from licensing, another strike against > mercurial is that it is written in Python, so it couldn't > go into base unless Python also went into base. Of course. OTOH, this topic will only become relevant if anyone notices that Subversion gets committed into base ;) From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 12:35:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D080106564A for ; Tue, 25 Jan 2011 12:35:47 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 271068FC16 for ; Tue, 25 Jan 2011 12:35:46 +0000 (UTC) Received: by pzk32 with SMTP id 32so1010872pzk.13 for ; Tue, 25 Jan 2011 04:35:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=fMow1608eOV3uDe/ufjUFUou9mSnvE+Sa8EK505smEM=; b=RfBuVjYCl54lptCeAzE5NB8/O8YBh908wgZYJBiI5fCGICJk0eNEtEnvVw5zdKZtSq ytncSZ0u6Evnle4HShQEoL7J+sGoj9REVteMDnZuscafwV0F6iAPZXYonn9UBrEaQA7D ccklQ0rM9bIJQDWSj7uEFgadvC/n/9EQLc5QI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=djGKaGJBbKDcvM12x+6YU14EdEMfcff8D5U3C4SdB8FglujiJn2yX3yH6iZAJGLOUc RBd7x11V0dO4wk0KtbM7mCeP5+5/PfCA4nXNg5Hxd6PjL5+4zTBOa1S5AWe98TpteQuQ lyEJcl5PCtzTvD1dFVdEaZU7PzYK5FB7Bh4f4= MIME-Version: 1.0 Received: by 10.142.164.10 with SMTP id m10mr3253751wfe.2.1295957342453; Tue, 25 Jan 2011 04:09:02 -0800 (PST) Received: by 10.142.229.3 with HTTP; Tue, 25 Jan 2011 04:09:02 -0800 (PST) Date: Tue, 25 Jan 2011 07:09:02 -0500 Message-ID: From: grarpamp To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Tue, 25 Jan 2011 12:39:22 +0000 Subject: Why not give git a try? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 12:35:47 -0000 Hi. This is good topic. I am no body. But I want to mention things. I've use RCS, CVS, SVN, Hg and Git. To me, first three are really much one in same. Of later two still learning, Hg can be slightly easier, but Git has simple analogs too, not much hard to get. We all learn new thing. But overall, Git seems to be on curve as the one become defacto for all good reasons/purpose in new world. Just as was CVS. Coders will know it. Gitweb, etc. So it momentum to maybe break ties, lame but can be true. I am ok with it. Distributed also maybe, maybe, save project bandwidth, as ppl tracking multiple branches should mirror and checkout locally. Not checkout multiple branch over wire. And can work fully local. Yes, maybe distrib not as much enforce commit from private coders often. But people already have those involved/not, regular/not, central/not habits, repo make no diff. Yet better, distrib enables some new good models not possible before too, while still letting some old style happen too. Also, very important this one. Git provide hash authenticated chained repo possibility by default and native with every commit from init of new repo. CVS/SVN do not do that. FreeBSD is big project (with big users). With big project infrastruct. With big group with commit people login from bfe from some fubar systems sometime unknown. All good ppl and systems yes, but deteching errors and provide proofs and chains is important now days in IT worlds. Too, FreeBSD only sign release cd's and only on maj.min release, not security branche, releng branche etc. There is no strong trace back to repo, so link is cut there. It is repo hash that should be sign, first before cd, and from there all things roll down as extra bonus. Does pictures on this important one be clear? And if move beyond CVS/SVN legacy wanted, yes?, start tests, eval, pick and move on :) My local stuffs went CVS to Git, but is no matter that choice to this project topic. I try agnostic. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 12:41:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A984106564A for ; Tue, 25 Jan 2011 12:41:29 +0000 (UTC) (envelope-from keramida@ceid.upatras.gr) Received: from igloo.linux.gr (igloo.linux.gr [62.1.205.36]) by mx1.freebsd.org (Postfix) with ESMTP id EC5C38FC19 for ; Tue, 25 Jan 2011 12:41:28 +0000 (UTC) X-Spam-Status: No X-Hellug-MailScanner-From: keramida@ceid.upatras.gr X-Hellug-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-2.9, required 5, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90) X-Hellug-MailScanner: Found to be clean X-Hellug-MailScanner-ID: p0PCeos2018628 Received: from gkeramidas-glaptop.linux.gr ([74.125.57.36]) (authenticated bits=0) by igloo.linux.gr (8.14.3/8.14.3/Debian-9.4) with ESMTP id p0PCeos2018628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 25 Jan 2011 14:40:58 +0200 From: Giorgos Keramidas To: perryh@pluto.rain.com References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> Date: Tue, 25 Jan 2011 13:40:50 +0100 In-Reply-To: <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> (perryh@pluto.rain.com's message of "Tue, 25 Jan 2011 02:22:34 -0800") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Cc: freebsd-hackers@freebsd.org, db@db.net, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 12:41:29 -0000 On Tue, 25 Jan 2011 02:22:34 -0800, perryh@pluto.rain.com wrote: >Diane Bruce wrote: >> There certainly would not be a chance of putting mercurial or git >> into base for example. > > Completely apart from licensing, another strike against mercurial is > that it is written in Python, so it couldn't go into base unless > Python also went into base. This argument is actually a bit weak for most of the VCS'es out there (including svn by the way). We don't really *need* to import the full VCS itself into FreeBSD. For instance, Subversion is also not part of the base system. It works fine as a port that people can install. There's really _nothing_ wrong with a VCS that is a port/package. We used to have CVS into the base system as "the official VCS", but this is no longer the case for the subversion repo of src/. IMO this hasn't really caused any major problem with the people who want to check out and patch the source tree. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 13:20:16 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 436DD1065670; Tue, 25 Jan 2011 13:20:16 +0000 (UTC) (envelope-from danger@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 162A58FC08; Tue, 25 Jan 2011 13:20:16 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p0PDKFFi002927; Tue, 25 Jan 2011 13:20:15 GMT (envelope-from danger@freefall.freebsd.org) Received: (from danger@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p0PDKFdI002918; Tue, 25 Jan 2011 13:20:15 GMT (envelope-from danger) Date: Tue, 25 Jan 2011 13:20:15 +0000 From: Daniel Gerzo To: hackers@FreeBSD.org Message-ID: <20110125132015.GA1789@freefall.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.4.2.3i Cc: stable@FreeBSD.org, current@FreeBSD.org Subject: FreeBSD Status Report - 4Q/2010 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 13:20:16 -0000 FreeBSD Quarterly Status Report Introduction This report covers FreeBSD-related projects between October and December 2010. It is the last of the four reports planned for 2010. The work on the new minor versions of FreeBSD, 7.4 and 8.2, has been progressing well and they should be released around the end of this month. Thanks to all the reporters for the excellent work! This report contains 37 entries and we hope you enjoy reading it. Please note that the deadline for submissions covering the period between January and March 2011 is April 15th, 2011. __________________________________________________________________ Projects * BSDInstall * Non-executable Stacks * Webcamd * xz Compression for Packages and Log Files * ZFS pool version 28 FreeBSD Team Reports * FreeBSD Bugbusting Team Status Report * Release Engineering Team Status Report * The FreeBSD Foundation Status Report Network Infrastructure * DIstributed Firewall and Flow-shaper Using Statistical Evidence (DIFFUSE) * Ethernet Switch Framework * Five New TCP Congestion Control Algorithms for FreeBSD * FreeBSD 802.11n * FreeBSD VirtIO Network Driver * Generic IEEE 802.3 annex 31B full duplex flow control support for Ethernet in mii(4) * IPv6 and VIMAGE * TCP SMP scalability project Kernel * Resource Containers * SYSCTL Type Safety * TRIM support for UFS Documentation * mdocml Replacing groff For manpage Rendering * The FreeBSD German Documentation Project Status Report * The FreeBSD Japanese Documentation Project Userland Programs * FreeBSD Services Control (fsc) * GEOM-based ataraid(4) Replacement -- geom_raid * gpart Improvements Architectures * Bringing up OMAP3 * FreeBSD on the Playstation 3 * FreeBSD/EC2 * FreeBSD/sparc64 Ports * Chromium * FreeBSD as Home Theater PC * Port-Sandbox * Portmaster * Ports Additions * Ports Collection * Robot Operating System Miscellaneous * FOSDEM 2011 __________________________________________________________________ Bringing up OMAP3 URL: http://people.FreeBSD.org/~raj/patches/arm/dove_v6.diff Contact: Warner Losh Contact: Mohammed Farrag The attached file is an old patch for ARM. We are developing new patch and then we are going toward Porting OMAP3. __________________________________________________________________ BSDInstall URL: http://wiki.FreeBSD.org/BSDInstall Contact: Nathan Whitehorn BSDInstall is a replacement for the venerable sysinstall installer. It is designed to be modular and easily extensible, while being fully scriptable and streamlining the installation process. It is mostly complete, and installs working systems on i386, amd64, sparc64, powerpc, and powerpc64, with untested PC98 support. New Features: * Allows installation onto GPT disks on x86 systems * Can do installations spanning multiple disks * Allows installation into jails * Eases PXE installation * Virtualization friendly: can install from a live system onto disk images * Works on PowerPC * Streamlined system installation * More flexible scripting * Easily tweakable * All install CDs are live CDs Open tasks: 1. Wireless networking configuration wizard. 2. ZFS installation support. 3. Itanium disk setup. __________________________________________________________________ Chromium URL: http://www.chromium.org/Home URL: http://people.FreeBSD.org/~bapt/chrome9-fbsd.png Contact: René Ladan We are working on updating the Chromium web browser in our ports to stay up to date with the latest supported release. We currently have the Chromium 9 beta running, but not all features are fully implemented and the port still needs some polish before it can be committed to the Ports Collection. We have also been making arrangements with Google to merge our work with their upstream, which should ease the number of features and fixes we have to maintain for ourselves in the future. Our first release should be in a few weeks and coincide with the official release of Chromium 9. __________________________________________________________________ DIstributed Firewall and Flow-shaper Using Statistical Evidence (DIFFUSE) URL: http://caia.swin.edu.au/urp/diffuse/ URL: http://caia.swin.edu.au/urp/diffuse/downloads.html Contact: Sebastian Zander Contact: Grenville Armitage DIFFUSE is a system enabling FreeBSD's IPFW firewall subsystem to classify IP traffic based on statistical traffic properties. With DIFFUSE, IPFW computes statistics (such as packet lengths or inter-packet time intervals) for observed flows, and uses ML (machine learning) techniques to assign flows into classes. In addition to traditional packet inspection rules, IPFW rules may now also be expressed in terms of traffic statistics or classes identified by ML classification. This can be helpful when direct packet inspection is problematic (perhaps for administrative reasons, or because port numbers do not reliably identify applications). DIFFUSE also enables one instance of IPFW to send flow information and classes to other IPFW instances, which then can act on such traffic (e.g. prioritise, accept, deny, etc) according to its class. This allows for distributed architectures, where classification at one location in your network is used to control fire-walling or rate-shaping actions at other locations. In December 2010 we released DIFFUSE v0.1, a set of patches for FreeBSD-CURRENT. It can be downloaded from the project's web site. The web site also contains a more comprehensive introduction, including application examples, links to related work and documentation describing the software design. We hope to release DIFFUSE v0.2 soon. Keep an eye on the freebsd-ipfw and freebsd-net mailing lists for project-related announcements. __________________________________________________________________ Ethernet Switch Framework URL: http://loos.no-ip.org/rspro/switch-1.diff Contact: Luiz Otavio O. Souza Implementation of a framework for ethernet switch control (directly connected to the ethernet MAC controller) usually found on embedded systems. Currently based on ifconfig keywords, adds the vlan control (filter/pass) on each switch port and adds the possibility for the management of media state on interfaces with multiple PHYs. Currently, the code supports the IP175D (from some mikrotik routerboards) and AR8316 (from Ubiquiti RSPRO) switches. Open tasks: 1. Finish the IP175C driver (and maybe IP178x). 2. Better integration with miibus (rewrite of switchbus). 3. Fix (some) ifconfig keywords (better keywords, better usage compatibility). 4. Export the ports statistics through SNMP (if available on switch chip). 5. Add a swctl tool (?) for global settings management. 6. Write usage examples and the man page information about the new ifconfig(8) keywords. __________________________________________________________________ Five New TCP Congestion Control Algorithms for FreeBSD URL: http://caia.swin.edu.au/freebsd/5cc/ URL: http://caia.swin.edu.au/urp/newtcp/ URL: http://www.FreeBSDFoundation.org/projects.shtml URL: http://people.FreeBSD.org/~lstewart/patches/5cc/ Contact: David Hayes Contact: Lawrence Stewart Contact: Grenville Armitage Contact: Rui Paulo Contact: Bjoern Zeeb The project is nearing completion, with the following code already available in the svn head branch: * Modular congestion control framework. * Modularised implementations of NewReno, CUBIC and HTCP congestion control algorithms. * Khelp (Kernel Helper) and Hhook (Helper Hook) frameworks. * Basic Khelp/Hhook integration with the TCP stack. The ERTT (Enhanced Round Trip Time) Khelp module is days away from being imported, which will then pave the way for the delay based congestion control algorithms to follow. Finally, a large documentation dump will be committed in the form of new and updated man pages. We anticipate the project will conclude around the end of January 2011. Open tasks: 1. Import the ERTT Khelp module. 2. Import the VEGAS, HD and CHD delay based congestion control algorithm modules. 3. Import the documentation dump for all the code contributed/developed as part of the project. __________________________________________________________________ FOSDEM 2011 URL: http://www.FOSDEM.org Contact: Marius Nuennerich Contact: Daniel Seuffert FOSDEM 2011 will be held from Saturday, February 5th to Sunday February 6th in Brussels, Belgium. We will have a FreeBSD booth and a developers room. At the booth there will be friendly supporters and a FreeBSD Foundation member answering questions. The devroom will have 6 1-hour long talks about different topics, technical and social. FOSDEM is one of the biggest open-source events in Europe. It is completly free and no registration is required. Open tasks: 1. Get more people involved as helpers for the booth and the devroom are still needed. Please contact Daniel or Marius if you want to help out. __________________________________________________________________ FreeBSD 802.11n URL: http://wiki.FreeBSD.org/AdrianChadd/AtherosStuff Contact: Adrian Chadd * Net80211 station mode works in 2.4ghz HT/20 mode. HT/40 and 5ghz do not currently work. * Basic 802.11 TX and RX on the AR9160 works, from MCS0 to MCS15 * TX A-MPDU and A-MSDU do not currently implemented - so no aggregate TX will happen * RX A-MPDU and A-MSDU is implemented and is supposed to work but does not -- this needs to be debugged * 802.11n RTS/CTS protection for legacy packets does not currently work. There is some magic required to fix the TX packet length. This is in progress. * WPA2 now works - a commit which enabled the hardware multicast broke AES-CCMP encryption on at least the AR9160. Further investigation is needed to fix this (and any other hardware encryption bugs that are lurking. __________________________________________________________________ FreeBSD as Home Theater PC URL: http://wiki.FreeBSD.org/HTPC Contact: Bernhard Froehlich Contact: Juergen Lock FreeBSD could be a much better platform for a Home Theater PC than it currently is. We are focusing on improving support for media center applications. Extending the major ports (MythTV, VDR, XBMC) and create some documentation to guide interested people. Open tasks: 1. Improve remote control support in webcamd and with lirc. 2. Port more Media Center applications (Enna, me-tv, ...). 3. Create a small guide on how to build a great FreeBSD Home Theater PC. __________________________________________________________________ FreeBSD Bugbusting Team Status Report URL: http://www.FreeBSD.org/support.html#gnats URL: http://wiki.FreeBSD.org/BugBusting URL: http://people.FreeBSD.org/~linimon/studies/prs/ Contact: Gavin Atkinson Contact: Mark Linimon Contact: Remko Lodder Contact: Volker Werth The number of non-ports PRs has held relatively steady over the last three months, with a slightly improved resolution rate being offset by a slightly increased rate of new arrivals. Ports PRs have increased slightly in numbers, due in part to the ports freeze in the lead up to the release of FreeBSD 7.4 and FreeBSD 8.2. The numbers traditionally drop quickly again once the freeze is lifted. In October, Gavin Atkinson and Mark Linimon held a session at the FreeBSD Developers' Summit at EuroBSDCon, which led to some productive discussions, and a number of people expressing interest in becoming more involved with PR triaging and resolution. The bugbusting team continue work on trying to make the contents of the GNATS PR database cleaner, more accessible and easier for committers to find and resolve PRs, by tagging PRs to indicate the areas involved, and by ensuring that there is sufficient info within each PR to resolve each issue. Reports continue to be produced from the PR database, all of which can be found from the links above. Committers interested in custom reports are encouraged to discuss requirements with bugmeister@ - we are happy to create new reports where needs are identified. As always, anybody interested in helping out with the PR queue is encouraged to do so, the easiest way being to join us on IRC in #freebsd-bugbusters on EFnet. We are always looking for additional help, whether your interests lie in triaging incoming PRs, generating patches to resolve existing problems, or simply helping with the database housekeeping (identifying duplicate PRs, ones that have already been resolved, etc). This is a great way of getting more involved with FreeBSD! Open tasks: 1. Try to find ways to get more committers helping us with closing PRs that the team has already analyzed. 2. Try to get more non-committers involved with the triaging of PRs as they come in, and generating patches to fix reported problems. __________________________________________________________________ FreeBSD on the Playstation 3 Contact: Nathan Whitehorn On January 5, support for the Playstation 3 was imported into FreeBSD 9.0-CURRENT. This port is still somewhat raw (only netbooting is supported, no access to the SPUs, etc.), but hardware support should be more fleshed out by the time FreeBSD 9.0 is released. The port uses the OtherOS mechanism, and so requires a "fat" console with firmware earlier than 3.21. Open tasks: 1. SATA driver. 2. Sound support. 3. SPU driver. __________________________________________________________________ FreeBSD Services Control (fsc) Contact: Tom Rhodes FreeBSD Services Control is a mix of binaries which integrate into the rc.d system and provide for service (daemon) monitoring. It knows about signals, pidfiles, and uses very little resources. The fscd utilities will be set up as a port and, hopefully, dropped into the ports collection in the coming weeks. This will allow easier testing by everyone and it should make migration into -CURRENT much easier. __________________________________________________________________ FreeBSD VirtIO Network Driver URL: http://www.linux-kvm.org/page/Virtio URL: http://lists.FreeBSD.org/pipermail/freebsd-current/2011-January/022036. html Contact: Bryan V. VirtIO is a device framework offered by KVM/Qemu and Virtualbox to allow guests to achieve better I/O performance. A beta network driver was made available earlier this month, and work continues on completing the block device and refinements the existing network driver. __________________________________________________________________ FreeBSD/EC2 URL: http://www.daemonology.net/freebsd-on-ec2/ Contact: Colin Percival FreeBSD is now able to run on t1.micro instances in the Amazon EC2 cloud. FreeBSD 9.0 is not very stable, but it seems likely that FreeBSD 8.2-RELEASE will approach the stability normally expected of FreeBSD. A list of available FreeBSD AMIs (EC2 machine images) appears on the FreeBSD/EC2 status page. Open tasks: 1. Bring FreeBSD to a wider range of EC2 instance types. 2. Completely rework the locking in head/sys/i386/xen/pmap.c to eliminate races and make 9.0-CURRENT stable under paravirtualization. 3. Track down several possibly-related problems with scheduling and timekeeping. 4. Fix other issues shown on the FreeBSD/EC2 status page. __________________________________________________________________ FreeBSD/sparc64 Contact: Marius Strobl CPUTYPE support for sparc64 has been added to CURRENT in r216820. The three flavors currently supported are "ultrasparc", "ultrasparc3" and "v9". So it is now possible to let the compiler produce code optimize for the family of UltraSPARC-III CPUs by setting CPUTYPE to "ultrasparc3". Setting it to "ultrasparc" as well as omitting it completely optimizes for UltraSPARC-I/II family CPUs as before. Support for generating generic 64-bit V9 code was mainly added for reference purposes. As it turned out, at least for SPARC64-V CPUs running code optimized for UltraSPARC-III CPUs does not perform measurably better than UltraSPARC-I/II one though so the default is just fine for these. This change was merged into 7-STABLE in r217005 and into 8-STABLE in r217004 respectively, neither 7.4-RELEASE nor 8.2-RELEASE will include it though. Support for a certain feature available with UltraSPARC-III+ and greater, i.e. with all sun4u CPUs following the original UltraSPARC-III, has been added to CURRENT in r216803. The net effect of this change is that we now can use a kernel TSB and thus a kernel address space of virtually any size up to the full 64-bit address space on machines equipped with these CPUs, apart from the fact that 1GB of address space still takes up 4MB worth of data structures. Before, the theoretical limit was 16GB due to the fact that the MMUs of these UltraSPARC CPUs only have 16 lockable TLB slots (UltraSPARC-I/II have 64 and SPARC64 CPUs again have at least 32), with the actual limit being several GB below that because we need some of these slots also for mapping the PROM, the kernel itself and in MP-systems the per-CPU page. Currently, the kernel TSB and thus the kernel virtual address space is now always sized one time the physical memory present in these machines with the plan being to actually allow to it extend beyond the size of the RAM as this helps especially ZFS. Most of this is implemented by patching the instructions used to access the kernel TSB based on the CPU present, so the run-time overhead of this change is rather low. Once it is also enabled and successfully tested with SPARC64 CPUs this change will be merged back into the supported stable branch(es). Theoretically it should be also possible to use the same approach for the user TSB, which already is not locked into the TLB but can cause nested traps. However, for reasons I do not understand yet, OpenSolaris only does this with SPARC64 CPUs. On the other hand I think that also using it for the user TSB and thus avoiding nested traps would get us closer to running the FreeBSD/sparc64 code on machines equipped with sun4v CPUs, which only supports trap level 0 and 1, too, so eventually we could have a single kernel which runs on both sun4u and sun4v machines (as does Linux and OpenBSD). Work on adding support for Sun Fire 3800 and similar models has begun but still is in its early stages. __________________________________________________________________ Generic IEEE 802.3 annex 31B full duplex flow control support for Ethernet in mii(4) URL: http://en.wikipedia.org/wiki/Ethernet_flow_control Contact: Marius Strobl In r213878 a NetBSD-compatible mii_attach() was added to mii(4) as an replacement for mii_phy_probe() and subsequently all Ethernet device drivers in the tree which use this framework were converted to take advantage of the former. This allowed to considerably clean up mii(4) as well as the converted MAC and PHY drivers and get rid of quite a few hacks, amongst others the infamous "EVIL HACK". However, the main motivation of this change was to allow the addition of generic IEEE 802.3 annex 31B full duplex flow control support to mii(4), which was ported from NetBSD but also enhanced and fixed quite a bit and committed in r215297. Along with this bge(4), bce(4), msk(4), nfe(4) and stge(4) as well as brgphy(4), e1000phy(4) and ip1000phy(4), which previously all implemented their own flow control support based on mostly undocumented special media flags separately, were converted to take advantage of the generic support. At least for CURRENT this means that these drivers now no longer unconditionally advertise support for flow control but only do so if flow control was selected as media option. The reason for implementing the generic flow control support that way was to allow it to be switched on and off via ifconfig(8) with the PHY specific default to typically being off in order to protect from unwanted effects. Subsequently support for flow control based on the generic support was added to alc(4), fxp(4), cas(4), gem(4), jme(4), re(4) and xl(4) as well as atphy(4), bmtphy(4), gentbi(4), inphy(4), jmphy(4), nsgphy(4), nsphyter(4) and rgephy(4). For several of the remaining Ethernet drivers it also would only require minor changes to enable flow control support if supported by the respective MAC. Due to the fact that each implementation should be thoroughly tested and tuned this was only done for drivers were hardware was available though. An example for identifying support for flow control based on the generic implementation in the dmesg-output for a certain MAC-PHY-combination would be: bge0: mem 0 xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 25 at device 2.0 on pci2 bge0: CHIP ID 0x00002003; ASIC REV 0x02; CHIP REV 0x20; PCI-X miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow or in the output of ifconfig -m for a given device: supported media: media autoselect mediaopt flowcontrol The latter also is what one would use to enable flow control for such a device, i.e.: ifconfig bge0 media autoselect mediaopt flowcontrol or in order to turn it off again: ifconfig bge0 media autoselect -mediaopt flowcontrol Note that some PHY drivers, currently only rgephy(4) though, also support enabling flow control support when using manual media configuration like in the following example: ifconfig re0 media autoselect mediaopt full-fuplex,flowcontrol In CURRENT this can also be further abbreviated (support for this will eventually be merged back into the supported stable branch(es) but not be present in 7.4-RELEASE or 8.2-RELEASE) as: ifconfig re0 media auto mediaopt fdx,flow For a device which has successfully negotiated flow control support with its link partner will report it in the output of ifconfig along with the available directions like in the following example: media: Ethernet autoselect (100baseTX ) Another thing that was introduced with r215297 was generic support for setting 1000baseT master mode via a media option when using manual media configuration. Consequently, brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) have been converted to take advantage of this generic support. At least for CURRENT this means that these drivers now no longer take the link0 parameter for selecting master mode but the master media option has to be used instead like in the following example: ifconfig bge0 media 1000baseT mediaopt full-duplex,master Selection of master mode now is also available with all other PHY drivers supporting 1000baseT. With the exception of the media option abbreviations all of the above mentioned changes were merged into 7-STABLE in r215879 and into 8-STABLE in r215881 respectively. This means that they will be part of 7.4-RELEASE and 8.2-RELEASE. In order to no break POLA, unlike as in CURRENT bge(4), bce(4), msk(4), nfe(4) and stge(4) were changed to continue to always advertise support of flow control to their link partners in these stable branches with no way to turn that off as they also did before with their custom implementations. Additionally, brgphy(4), ciphy(4), e1000phy(4) as well as ip1000phy(4) were changed to still also accept the link0 parameter in addition to the master media option for setting master mode. Open tasks: 1. We actually miserably fail to properly document the available media types and options in manual pages. For example several of the media lists in manual pages of MAC drivers like bge(4) already were outdated and with the addition of generic flow control and 1000baseT master mode support these are now even more outdated. Yet worse is the fact that for MAC drivers which use the mii(4) framework it is technically just plain wrong to include these lists in their manual page as the PHY drivers actually are responsible for handling the media types and options. However, given that the PHY drivers determine the available media types and options mostly dynamically at run-time it generally makes no sense to have static documentation of these in their manual pages (apart from the fact that we currently have no manual pages for PHY drivers). One good way out of this should be to replace the media lists in MAC drivers using mii(4) with just a note to check the output of ifconfig -m to get a list of the media types and options actually supported by a given device and to add a generic ifmedia(4) manual page which provides some general background information about media types and options similar to what NetBSD and OpenBSD also have. __________________________________________________________________ GEOM-based ataraid(4) Replacement -- geom_raid URL: http://people.FreeBSD.org/~mav/graid_design.h URL: http://svn.FreeBSD.org/viewvc/base/projects/graid/ Contact: Alexander Motin Contact: M. Warner Losh New project started to create GEOM-based replacement for ataraid(4) -- software RAID, that will be obsoleted by migration to the new CAM-based ATA implementation. This implementation planned with accent to modular design, that includes common core and two sets of modules, handling data transformations (RAID levels) and on-disk metadata formats specifics. Such design should make further extension easier. At this moment work focused around RAID0/RAID1 transformations and Intel metadata format. Module is now able to read, write and create Intel volumes. Error recovery and rebuild work is now in progress. Support for other RAID levels and metadata formats, supported by ataraid(4), planned later. This project is sponsored by Cisco Systems, Inc. Open tasks: 1. Complete error recovery/rebuild work and stabilize modules API. 2. Implement metadata modules for other formats. 3. Implement transformation modules for other RAID levels. __________________________________________________________________ gpart Improvements Contact: Andrey V. Elsukov GEOM class PART is the default disk partitioning class since FreeBSD 8.0. Compared to 8.1 now it does have several new features: Partition resizing. New "gpart resize" subcommand was implemented for all partitioning schemes but EBR. GPT recovering. Guid Partition Table does have redundant metadata and it can be recovered when some of them is damaged. New "gpart recover" subcommand was implemented for that purpose. Ability to backup/restore of partition table. New "gpart backup" and "gpart restore" subcommands were implemented. __________________________________________________________________ IPv6 and VIMAGE URL: http://ecdysis.viagenie.ca/ Contact: Bjoern A. Zeeb During the last quarter a lot of work was spent on quality time hunting down and fixing open bugs and races in the network stack, mostly IPv6, as well as testing and getting virtualized network stack parts more stable. Tests for the pf(4) firewall update were started with VIMAGE. In addition Viagenie's NAT64 patch was ported over. __________________________________________________________________ mdocml Replacing groff For manpage Rendering URL: http://mdocml.bsd.lv/ URL: https://www.spoerlein.net/cgit/cgit.cgi/freebsd.work/log/?h=mdocml Contact: Ulrich Spörlein Kristaps' groff-replacement (only for rendering manual pages) is already available in NetBSD and OpenBSD, and used to render the base system manpages for the latter. This project aims to do similar things for FreeBSD. Since the last status report, mdocml has grown rudimentary tbl(1) support and a whole lot of bugfixes have gone in. A groff port has been created and needs some more testing before it can be committed to the tree. Also the WITHOUT_GROFF support in base has been fleshed out and is awaiting review before commit. Open tasks: 1. Get ru@ to review WITHOUT_GROFF changes. 2. Get textproc/groff tested and committed. 3. Push more mdoc fixes into the tree. 4. Import mandoc(1), switch to catpages for base. Discuss future of groff in base wrt. share/doc. 5. Supply necessary ports infrastructure to opt-in to mandoc(1). __________________________________________________________________ Non-executable Stacks Contact: Konstantin Belousov The support for non-executable stacks, using the approach identical to one used by GNU toolchain and Linux'es, is implemented for amd64 and PowerPC. The support is already committed to HEAD. For now, non-executable stacks are turned off by default. I plan to provide a detailed information to ports@ and switch the knob after port tree is unfrozed for 7.4/8.2 releases. __________________________________________________________________ Port-Sandbox URL: http://www.arjmobile.com/Marcelo_Araujo/Blog/Entries/2010/11/22_Port-sa ndbox.html URL: http://gitorious.org/port-sandbox/port-sandbox/trees/master URL: http://www.arjmobile.com/Marcelo_Araujo/My_Albums/Pages/Port-Sandbox.ht ml Contact: Marcelo Araujo Port-Sandbox now works properly and it is able to run by itself through an embedded web server and bring a lot of information about the port build process and all dependencies related. Currently Port-Sandbox is in the final stage and needs only only a few code changes, more tests and should also be included in the ports tree. Open tasks: 1. Change the way how it connects to database, fix it to maintain a persistent connection. 2. Remove any kind of internal configuration from source code to an external file configuration. 3. Create a Port-Sandbox port with all dependencies related to it and test it in a clean system. 4. Create some documentation to let other people to keep helping Port-Sandbox to grow up. 5. Finally, release it. __________________________________________________________________ Portmaster URL: http://dougbarton.us/portmaster-proposal.html Contact: Doug Barton Portmaster version 3.6.1 is now in the ports tree, and the emphasis in the last year has been on improving the stability and performance of existing features, with a few new features sprinkled in. A lot of work has gone into error handling, both for unexpected states in the ports system and for user input. For example, all prompts are now wrapped in code to verify that what was entered was one of the valid options. Perhaps the most interesting new element is that for the features -e, -s, --clean-distfiles, --clean-packages, --check-depends and --check-port-dbdir you can now specify either -y or -n to automatically provide the corresponding answer to the yes/no questions. The -o, -r, and --index-only options have received major overhauls, and now either work better or at least as advertised. There has also been a lot of work put into reducing the memory footprint, especially in the environment variables that are shared between the parent and child processes. And for those operating without a local ports tree (--index-only/--packages-only) all of the features that can work without the ports tree now do. Significant support for the upgrading of operating without a ports tree was provided by GridFury, LLC. Their support, as well as the support received from other members of the community continues to be greatly appreciated. Open tasks: 1. There are still interesting features that have been suggested by users listed on the page above that I have not been able to work on, but would like to be able to. __________________________________________________________________ Ports Additions URL: http://bigbluebutton.org URL: http://smb4k.berlios.de/ URL: http://www.freeswitch.org/ Contact: Josh Paetzel Bigbluebutton has joined the list of ready to run applications in the ports tree. Dru Lavigne has been instrumental on getting it to run, as well as offering suggestions for improvements to the port. smb4k was updated to the latest release version, which requires kde4. This was enough of a change that a new port was created, net/smb4k-kde4. the initial port went through a number of quick changes, including a patch to the source code to fix a FreeBSD source code submitted by PC-BSD's Kris Moore. This application greatly eases the task of working with samba shares in a FreeBSD environment. Freeswitch is the result of 3 Asterisk developers working on a VoIP package that fulfills their goals. They have switched away from a release model to a "just run latest SVN checkout" model. With the help of Richard Neese and Eric Crist, static snapshots of their SVN repo have been taken, the port has been modified to use the newer version, and extensive build and run testing has been done. __________________________________________________________________ Ports Collection URL: http://www.FreeBSD.org/ports/ URL: http://www.FreeBSD.org/doc/en_US.ISO8859-1/articles/contributing-ports/ URL: http://portsmon.FreeBSD.org/index.html URL: http://www.FreeBSD.org/portmgr/index.html URL: http://blogs.FreeBSDish.org/portmgr/ URL: http://www.twitter.com/freebsd_portmgr/ URL: http://www.facebook.com/group.php?gid=135441496471197 URL: http://tinderbox.marcuscom.com/ Contact: Thomas Abthorpe Contact: Port Management Team The ports tree slowly moves up closer to 23,000. The PR count still remains at about 1000. In Q4 we added 2 new committers, took in 2 commit bit for safe keeping, and welcomed back 4 returning committers. The Ports Management team bid farewell to Kris Kennaway in November 2010. Kris was the root of krismail, the mail we all got from time to time when ports broke on pointyhat. Kris did a lot of work benchmarking and testing FreeBSD for stability, scalability and usability. Mark Linimon has put a lot of effort into refactoring and refining the code that runs the 'pointyhat' package build dispatch system. In 2010, the FreeBSD Foundation purchased for portmgr a pair of new machines, pointyhat-west and pointyhat-east, to take over from the existing machine. (The new machines have much greater RAM, CPU, and disk capacity.) However, to properly utilize them, the existing code needed to be generalized. Persistent bugs, and some hardware troubles, have delayed the rollout far beyond what was originally planned, but there appears to be light at the end of the tunnel. (And, this time, it does not appear to be an oncoming train.) A document entitled "Mentoring Guidelines" as been circulated among ports developers, and has been greeted with a lot of positive feedback, and updates have been included. In the short term, updated copies will be maintained at http://people.FreeBSD.org/~portmgr/mentor_guidelines.txt.asc. The Ports Management team have been running -exp runs on an ongoing basis, verifying how base system updates may affect the ports tree, as well as providing QA runs for major ports updates. Of note, -exp runs were done for: * ade: multiple runs for autotools refactoring * ed: test to replace libgcc.a with libcompiler_rt.a * jiles: test sh(1) against r212508 * kde: Qt 4.7.0 update * kde: KDE 4.5.4 updte * kwm: Gnome 2.32 update * ports/144164: ensure package-noinstall target include rc.d scripts * ports/145598: include etc/devd in mtree * ports/145955: silence make fetch-required-list * ports/147701: perform DESKTOP_ENTRIES sanity check * ports/149657: removal of MD5 checksums * ports/149670: remove checks in _OPTIONSFILE * ports/150303: for INSTALL_LIBS * ports/150337: for PLIST_DIRSTRY * ports/151047: pass CPP to CONFIGURE/MAKE_ENV * ports/151799: fix PLIST_DIRSTRY * ports/151806: remove 2004 legacy hack * ports/152055 and ports/152059: for pear infrastructure * ports/152558: boost update * ports/152626: fix pkg-message display if installed from package * ports/152964: embed LICENSE name for STDOUT * ports/153018: implement variables in Mozilla dependencies * ports/153033: fix un-escaped shell metacharacters * ports/153041: clean up ruby plists * ports/153132: autotools cleanup * ports/153318: set PGSQL default to 8.4 Open tasks: 1. Looking for help fixing ports broken on CURRENT. 2. Looking for help with Tier-2 architectures. 3. Most ports PRs are assigned, we now need to focus on testing, committing and closing. __________________________________________________________________ Release Engineering Team Status Report URL: http://www.FreeBSD.org/releng/ Contact: Release Engineering Team The Release Engineering Team reports the joint release of FreeBSD 7.4 and 8.2 has been delayed slightly but should be completed within a week or two of the original schedule: http://www.FreeBSD.org/releases/7.4R/schedule.html http://www.FreeBSD.org/releases/8.2R/schedule.html __________________________________________________________________ Resource Containers Contact: Edward Tomasz Napierala The goal of this project is to implement resource containers and a per-jail resource limits mechanism, so that system administrators can partition resources like memory or CPU between jails and prevent users from DoS-ing the whole system. Project is close to completion. One big item that needs to be fixed before releasing a patch for people to test is %CPU accounting; initial idea of just using %CPU calculated by the scheduler turned out to be useless. Implementing it cleanly will also make it easier to support other similar resources (e.g. writes-per-second) in the future. __________________________________________________________________ Robot Operating System URL: http://www.ros.org/wiki/ URL: ftp://rene-ladan.nl/pub/ros-freebsd.pdf Contact: René Ladan Porting ROS to FreeBSD started in March 2010. In May 2010, it was possible to build devel/ros without needing to apply patches, but some more changes were necessary to be able to write a port for it. Currently this and several other ports related to ROS are available, most notably devel/ros-tutorials to get up and running with ROS and devel/ros-nxt to use LEGO Mindstorms NXT robots with ROS and FreeBSD. Open tasks: 1. Port the software required for nxt-rviz-plugin, which is part of devel/ros-nxt but currently excluded from the build. __________________________________________________________________ SYSCTL Type Safety Contact: Matthew Fleming I started upstreaming a patch from Isilon that adds type-checking to the various SYSCTL_FOO and SYSCTL_ADD_FOO macros for various scalar types, which has turned into quite the discussion on the src mailing list. The type-checking macros are committed to sys/sysctl.h but under #if 0. Open tasks: 1. As of right now, it looks like I will be rolling a new sysctl macro for the kernel that detects they type at compile time and does the Right Thing. Existing uses of the legacy SYSCTL_FOO and SYSCTL_ADD_FOO for scalar types can be replaced, and will probably turn into invocations of the new interface via preprocessor macro. __________________________________________________________________ TCP SMP scalability project Contact: Robert Watson A long-running TCP SMP scalability project is beginning to wrap up, with the goal of committing a large outstanding patch to the FreeBSD 9.x tree in the next month. This work implements a derivative of Willman, Rixner, and Cox's TCP connection group model, blended with support for hardware load distribution features in contemporary NICs (including RSS). Additional software distribution support can do work redistribution based on new notions of CPU affinity for individual TCP connections. On-going work is refining performance on non-RSS supporting configurations, and adding APIs to allow socket affinity to be queried (and where supported) set by applications. These changes significantly improve network scalability by reducing global lock contention, encouraging CPU affinity for connections, and avoiding cache line contention. The goal is to allow steady-state TCP connections to use only CPU-local cache lines, with work distributed to all CPUs. Current performance results are extremely promising. This project has been sponsored by Juniper Networks. Open tasks: 1. Allow the hash model to be selected at boot-time or run-time rather than compile-time; currently "options RSS" enables RSS support unconditionally -- for systems without RSS NICs, this leads to a small one-time performance penalty at the creation of each call to bind() or connect(). 2. Add missing socket options to query (and override) default CPU affinity for connections, which is derived from the active software or hardware hash model. 3. Teach the network stack and appropriate NIC drivers to propagate software-overridden connection affinity to hardware using new device driver ioctls for managing TCAMs and hardware hash tables. 4. Refine software redistribution of work in the event that there are fewer hardware queues than available CPU threads in which to process packets; the current prototype is able to do this with significant performance benefits, but the model requires refining. 5. Experiment with (and measure) software work redistribution at run-time based on RSS bucket rearrangement. This will require a new event notification to device drivers so that they can update hardware caches of the network stack's authoritative table. 6. Commit. __________________________________________________________________ The FreeBSD Foundation Status Report URL: http://www.FreeBSDFoundation.org Contact: Deb Goodkin We raised $325,000 towards our goal of $350,000 for 2010! This will allow us to increase our project development and equipment spending for 2011. We were proud to be a sponsor for EuroBSDCon 2010, BSDDay Argentina 2010, MeetBSD California 2010, and NYBSDCon 2010. Completed the Foundation funded projects: DAHDI Project by Max Khon and BSNMP Improvements by Shteryana Sotirova. We kicked off a new project by the University of Melbourne called Feed-Forward Clock Synchronization Algorithms Project. The Five New TCP Congestion Control Algorithms for FreeBSD Project by Swinburne University also officially started. We continued our work on infrastructure projects to beef up hardware for package-building, network-testing, etc. This includes purchasing equipment as well as managing equipment donations. Stop by and visit with us at FOSDEM (Feb 5-6), SCALE (Feb 26), AsiaBSDCon (March 17-20), and Indiana Linuxfest (March 26). Read more about how we supported the project and community by reading our end-of-year newsletter at: http://www.FreeBSDFoundation.org/press/2010Dec-newsletter.shtml. We are fund-raising for 2011 now! Find out more at http://www.FreeBSDFoundation.org/donate/. __________________________________________________________________ The FreeBSD German Documentation Project Status Report URL: http://doc.bsdgroup.de Contact: Johann Kois Contact: Benedict Reuschling The committers to the German Documentation Project managed to update the German documentation just in time to get the changes included into the next FreeBSD releases. The website translations were also kept in sync with the ones on FreeBSD.org. We tried to re-activate committers who did not contribute for some time but most of them are currently unable to free up enough time. We hope to gain fresh contributor blood as we are getting occasional reports about bugs and grammar in the german translation. Open tasks: 1. Submit grammar, spelling or other errors you find in the german documents and the website. 2. Translate more articles and other open handbook sections. __________________________________________________________________ The FreeBSD Japanese Documentation Project URL: http://www.FreeBSD.org/ja/ URL: http://www.jp.FreeBSD.org/doc-jp/ Contact: Hiroki Sato Contact: Ryusuke Suzuki Although there is no radical change in this effort since the last report, the www/ja and doc/ja_JP.eucJP/books/handbook have constantly been updated. During this period, generating translated RSS feed for newsflash was started and links to the manual pages were fixed in the Books and Articles documentation. Some more progress has been made in the Porter's Handbook and Contributing to FreeBSD as well. Open tasks: 1. Further translation of the FreeBSD Handbook and contents of the www.FreeBSD.org website to the Japanese language. 2. Pre-/post-commit review of the translation. __________________________________________________________________ TRIM support for UFS Contact: Kirk McKusick Contact: Konstantin Belousov TRIM support for UFS is implemented in HEAD. Potentially, this may increase the steady speed and longevity of SSDs. Due to concerns with the speed of TRIM operations on many SSDs, and not a lot of experience with the real-world behaviour, the support is off by default, and should be enabled on the per-filesystem basis. __________________________________________________________________ Webcamd URL: http://www.selasky.org/hans_petter/video4bsd URL: http://www.freshports.org/multimedia/webcamd/ Contact: Hans Petter Selasky Webcamd is a small daemon that enables about 1500 different USB based webcam, DVB and remote control USB devices under the FreeBSD-8.0 and later operating system. The webcam daemon is basically an application which is a port of Video4Linux USB drivers into userspace on FreeBSD. The daemon currently depends on libc, pthreads, libusb and libcuse4bsd. During Q3 2010 webcamd got manpages thanks to Dru Lavigne. Open tasks: 1. I hope to get a Google summer of code project this year building the default Linux Kernel 2.6.37+ and allowing use of relevant Linux USB device drivers under FreeBSD. Webcamd is not a replacement for native FreeBSD kernel drivers and will only be used when no existing FreeBSD drivers exist for a given device staying clear of any GPLv2 issues. If you are a student and/or is interested in participating in such a project feel free to send an e-mail to hselasky@FreeBSD.org. __________________________________________________________________ xz Compression for Packages and Log Files Contact: Martin Matuska Creating and processing xz-compressed packages is now supported by pkg_create(1), pkg_add(1) and bsdtar(1) in both 9-CURRENT and 8-STABLE. Users can test working with .txz packages by adding "PKG_SUFX=.txz" into /etc/make.conf. The ports-mgmt/portupgrade utility supports .txz packages from version 2.4.8 and a patch for ports-mgmt/portmaster has been submitted but not yet accepted by the author. A patch for newsyslog(8) with a rewrite of the use of compression tools supporting xz compression is under maintainer review. Open tasks: 1. Import xz(1) compression support into newsyslog(8). 2. Add .txz package support to ports-mgmt/portmaster. 3. Add .txz package support to the FreeBSD port building cluster (pointyhat). 4. Test building all packages in .txz format and compare results with .tbz. __________________________________________________________________ ZFS pool version 28 URL: http://lists.freebsd.org/pipermail/freebsd-fs/2010-December/010292.html URL: http://lists.freebsd.org/pipermail/freebsd-fs/2010-December/010321.html Contact: Pawel Jakub Dawidek Contact: Martin Matuska A new version of the ZFS pool v28 patch was released for testing, this time for 9-CURRENT and 8-STABLE. Compared to the previous patch it does include updated boot support, improved sendfile(2) handling, a compatibility layer with older ZFS and several other bugfixes. If there are no major issues we can expect ZFS v28 imported into the FreeBSD-CURRENT after 8.2 is released. Open tasks: 1. Import of ZFS v28 into FreeBSD-CURRENT. __________________________________________________________________ 2011 The FreeBSD Project. All rights reserved. From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 14:05:17 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id C15F0106566B; Tue, 25 Jan 2011 14:05:17 +0000 (UTC) Date: Tue, 25 Jan 2011 14:05:17 +0000 From: Alexander Best To: Giorgos Keramidas Message-ID: <20110125140517.GA74156@freebsd.org> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org, db@db.net, perryh@pluto.rain.com, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 14:05:17 -0000 On Tue Jan 25 11, Giorgos Keramidas wrote: > On Tue, 25 Jan 2011 02:22:34 -0800, perryh@pluto.rain.com wrote: > >Diane Bruce wrote: > >> There certainly would not be a chance of putting mercurial or git > >> into base for example. > > > > Completely apart from licensing, another strike against mercurial is > > that it is written in Python, so it couldn't go into base unless > > Python also went into base. > > This argument is actually a bit weak for most of the VCS'es out there > (including svn by the way). > > We don't really *need* to import the full VCS itself into FreeBSD. For > instance, Subversion is also not part of the base system. It works fine > as a port that people can install. > > There's really _nothing_ wrong with a VCS that is a port/package. We > used to have CVS into the base system as "the official VCS", but this is > no longer the case for the subversion repo of src/. IMO this hasn't > really caused any major problem with the people who want to check out > and patch the source tree. imo having a VCS in base is a bad idea. it makes it much harder to keep it in sync with the vendor version. to update a port involves very little administrative work. on the other hand updating vendor software in base involves quite a lot of importing etc. also please don't forget that having a VCS in base means there's only *one* version of the VCS available. having it in the ports tree makes it possible to have a legacy release, a stable release, an experimental release and also a development snapshot. of course there could be a stable release in base and if users want to they can install a more recent version from ports, but that doesn't really make sense imo. also a VCS is not really an essential part of an OS. with clang/llvm e.g. a version must exist in base, since it's not possible to invoke /usr/local in order to build the system. the VCS however is not part of the build toolchain however (except 'make update' maybe). so -1 for moving a new VCS to base. also i think freebsd should stick to a major VCS, like git and not some lesser known obscure VCS. with git you have all the linux people behind it, which means it will be updated regularly, bugs will be fixed quite fast etc. there's no point in switching to some rather unknown VCS where the developers decide after 2 years that they don't have any more spare time and have to abandon the project. to be quite honest: the freebsd community doesn't have the manpower to maintain a VCS by itself. you *need* other developers in order to keep it up to date. just look at GNATS e.g. gnu abandoned it quite a while ago and the bugbusting community is just to small to either update to a more recent version of GNATS or switch to something like bugzilla. again: you *need* support from other developers in order to maintain a VCS properly. so git is the best choice imo. just my 0.02$ cheers. alex > -- a13x From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 15:15:26 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F315C1065673 for ; Tue, 25 Jan 2011 15:15:26 +0000 (UTC) (envelope-from philip-freebsd1@soeberg.net) Received: from pasmtpA.tele.dk (pasmtpa.tele.dk [80.160.77.114]) by mx1.freebsd.org (Postfix) with ESMTP id 9EC4E8FC14 for ; Tue, 25 Jan 2011 15:15:26 +0000 (UTC) Received: from mail.soeberg.net (0x573f534a.cpe.ge-1-1-0-1109.bynqu1.customer.tele.dk [87.63.83.74]) by pasmtpA.tele.dk (Postfix) with ESMTP id 35C0780074A for ; Tue, 25 Jan 2011 15:46:57 +0100 (CET) Received: from [10.240.10.87] ([188.120.77.114]) (authenticated user philip@soeberg.net) by mail.soeberg.net (using TLSv1/SSLv3 with cipher AES256-SHA (256 bits)) for freebsd-hackers@freebsd.org; Tue, 25 Jan 2011 15:47:02 +0100 Message-ID: <4D3EE287.3010203@soeberg.net> Date: Tue, 25 Jan 2011 15:47:35 +0100 From: Philip Soeberg User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: pci_suspend/pci_resume of custom pcie board X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: philip-freebsd1@soeberg.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 15:15:27 -0000 Hi, I'm in a particular problem where I need to set my custom pcie adapter into d3hot power-mode and a couple of seconds later reset it back to d0. The board has an FPGA directly attached to the pcie interface, and as I need to re-configure the FPGA on the fly, I have to ensure the datalink layer between the upstream bridge and my device is idle to prevent any hickups. On linux I simply do a pci_save_state(device) followed by pci_set_power_state(device, d3hot), then after my magic on my board, I do the reverse: pci_set_power_state(device, d0) followed by pci_restore_state(device). On FreeBSD, say 8, I've found the pci_set_powerstate function, which is documented in PCI(9), but that function does not save nor restore the config space. I've tried, just for the fun of it, to go via pci_cfg_save(device, dinfo, 0) with dinfo being device_get_ivars(device) and then subsequently restoring the config space back via pci_cfg_restore(), but since both those functions are declared in I'm not sure if I'm supposed to use those directly or not.. Besides, I'm not really having any luck with that approach. Reading high and low on the net suggest that not all too many driver devs are concerned with suspend/resume operation of their device, and if they are, leave it to user-space to decide when to suspend/resume a device.. I would like to be able to save off my device' config space, put it to sleep (d3hot), wake it back up (d0) and restore the device' config space directly from the device' own driver.. Anyone who can help me with this? Thanks, Phil From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 15:45:05 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73E6A106567A; Tue, 25 Jan 2011 15:45:05 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id 4EED18FC1D; Tue, 25 Jan 2011 15:45:04 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 2D3262283B; Tue, 25 Jan 2011 08:44:07 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id 910795CB4; Tue, 25 Jan 2011 10:45:03 -0500 (EST) Date: Tue, 25 Jan 2011 10:45:03 -0500 From: Diane Bruce To: Giorgos Keramidas Message-ID: <20110125154503.GA99045@night.db.net> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, db@db.net, perryh@pluto.rain.com, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 15:45:05 -0000 On Tue, Jan 25, 2011 at 01:40:50PM +0100, Giorgos Keramidas wrote: > On Tue, 25 Jan 2011 02:22:34 -0800, perryh@pluto.rain.com wrote: > >Diane Bruce wrote: > >> There certainly would not be a chance of putting mercurial or git > >> into base for example. > > > > Completely apart from licensing, another strike against mercurial is > > that it is written in Python, so it couldn't go into base unless > > Python also went into base. > > This argument is actually a bit weak for most of the VCS'es out there > (including svn by the way). Notice I never ever suggested we might want to do such a thing. All I said was there would be not a chance of GPL code being added. The additional argument of mercurial not being added because of its dependancy on python is immaterial. > > We don't really *need* to import the full VCS itself into FreeBSD. For > instance, Subversion is also not part of the base system. It works fine > as a port that people can install. I agree. Indeed, I would argue that there is other code in base that should come out. But I don't want to get that flamefest going again. ;-) The argument is not putting a VCS into base, the argument is what VCS to look at in future. 'fossil' is certainly a viable candidate. I am agreeing the arguments on which VCS to use should be based on the merits of the VCS itself, not on its licence. However, if in the long term we chose 'fossil' and then decided that 'fossil' was absolutely necessary for base, then it would have far less resistance than a GPL VCS. That's a small plus, I never said it was a strong plus. - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 15:51:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09B3F1065670; Tue, 25 Jan 2011 15:51:41 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id DD7328FC18; Tue, 25 Jan 2011 15:51:40 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id C2D4622841; Tue, 25 Jan 2011 08:50:42 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id 3A15A5CA3; Tue, 25 Jan 2011 10:51:39 -0500 (EST) Date: Tue, 25 Jan 2011 10:51:39 -0500 From: Diane Bruce To: Alexander Best Message-ID: <20110125155139.GB99045@night.db.net> References: <20110124183102.GB68940@night.db.net> <20110124194830.GA70207@night.db.net> <4d3ea46a.NnLdm55dZduBwzWN%perryh@pluto.rain.com> <20110125140517.GA74156@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110125140517.GA74156@freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: Giorgos Keramidas , freebsd-hackers@freebsd.org, db@db.net, perryh@pluto.rain.com, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 15:51:41 -0000 On Tue, Jan 25, 2011 at 02:05:17PM +0000, Alexander Best wrote: > On Tue Jan 25 11, Giorgos Keramidas wrote: > > On Tue, 25 Jan 2011 02:22:34 -0800, perryh@pluto.rain.com wrote: > > >Diane Bruce wrote: > > >> There certainly would not be a chance of putting mercurial or git > > >> into base for example. ... > > no longer the case for the subversion repo of src/. IMO this hasn't > > really caused any major problem with the people who want to check out > > and patch the source tree. Note I never said it should be importable into base. > also i think freebsd should stick to a major VCS, like git and not some lesser Argumentum ad populum. If the VCS is 1) easy to use 2) easy to maintain and has a body of people working on it already 3) can import other VCS formats, then why not? Also note that using an appeal to popularity suggests we should go with the flow and stop working on llvm/clang. Afer all, everyone else uses gcc. As it happens, fossil is quite tiny, fairly feature rich, BSDL'd and in active development. QED All that being said, I am not interested in bikeshedding right now. Those who will look at the alternatives intelligently should do so. > cheers. > alex - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 17:23:45 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7D751065672 for ; Tue, 25 Jan 2011 17:23:45 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from gw02.mail.saunalahti.fi (gw02.mail.saunalahti.fi [195.197.172.116]) by mx1.freebsd.org (Postfix) with ESMTP id 4E8358FC08 for ; Tue, 25 Jan 2011 17:23:45 +0000 (UTC) Received: from a91-153-123-205.elisa-laajakaista.fi (a91-153-123-205.elisa-laajakaista.fi [91.153.123.205]) by gw02.mail.saunalahti.fi (Postfix) with SMTP id 6863D13995B; Tue, 25 Jan 2011 19:23:40 +0200 (EET) Date: Tue, 25 Jan 2011 19:23:40 +0200 From: Jaakko Heinonen To: Craig Rodrigues Message-ID: <20110125172340.GA1535@a91-153-123-205.elisa-laajakaista.fi> References: <20110114122454.GA4805@jh> <20110119083407.GA51372@crodrigues.org> <20110119151210.GA2182@a91-153-123-205.elisa-laajakaista.fi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110119151210.GA2182@a91-153-123-205.elisa-laajakaista.fi> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org Subject: Re: [patch] nmount ro, rw and negated option handling X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 17:23:45 -0000 On 2011-01-19, Jaakko Heinonen wrote: > On 2011-01-19, Craig Rodrigues wrote: > > I disagree with your patch and do not approve it. > > > > 1. Have mountd(8) running. > > > 2. # mdconfig -a -t vnode -f ufsimg > > > 3. # mount -o ro,rw /dev/md0 /mnt > > With your patch[1] after the third step the mount point has both "ro" > and and "noro" active but the MNT_RDONLY flag is not set. Again, you > will eventually get the "ffs_sync: rofs mod" (or similar) panic because > the "ro" option is active during remount. Could you please elaborate on why my patch isn't acceptable and/or suggest an alternative approach to fix the bugs. As I said earlier the patch you posted doesn't solve the issues. I really want to get these bugs fixed. One option would be to revert the file system code to use the MNT_RDONLY flag instead of checking for the "ro" string option until nmount gets fixed but I don't think it's feasible because too many file systems are already testing for "ro". A quick fix for the particular problem above would be to commit the vfs_equalopts() part of my patch. I believe that the change is correct as the code stands now. It is not enough to fix PR kern/150206. Another related bug is in vfs_domount_update(): if VFS_MOUNT() succeeds but vfs_export() fails, old mount flags and options are restored. I think this shouldn't happen when VFS_MOUNT() has been already successfully completed and this is the final reason why FFS fs_ronly and the MNT_RDONLY flag get out of sync. Does the patch below look sane? %%% Index: sys/kern/vfs_mount.c =================================================================== --- sys/kern/vfs_mount.c (revision 217792) +++ sys/kern/vfs_mount.c (working copy) @@ -683,8 +683,6 @@ bail: } } - if (error != 0) - vfs_freeopts(optlist); return (error); } @@ -800,6 +798,7 @@ vfs_domount_first( } if (error != 0) { vput(vp); + vfs_freeopts(fsdata); return (error); } VOP_UNLOCK(vp, 0); @@ -824,6 +823,7 @@ vfs_domount_first( vp->v_iflag &= ~VI_MOUNT; VI_UNLOCK(vp); vrele(vp); + vfs_freeopts(fsdata); return (error); } @@ -881,15 +881,15 @@ vfs_domount_update( struct oexport_args oexport; struct export_args export; struct mount *mp; - int error, flag; + int error, export_error, flag; mtx_assert(&Giant, MA_OWNED); ASSERT_VOP_ELOCKED(vp, __func__); KASSERT((fsflags & MNT_UPDATE) != 0, ("MNT_UPDATE should be here")); if ((vp->v_vflag & VV_ROOT) == 0) { - vput(vp); - return (EINVAL); + error = EINVAL; + goto fail; } mp = vp->v_mount; /* @@ -898,28 +898,26 @@ vfs_domount_update( */ flag = mp->mnt_flag; if ((fsflags & MNT_RELOAD) != 0 && (flag & MNT_RDONLY) == 0) { - vput(vp); - return (EOPNOTSUPP); /* Needs translation */ + error = EOPNOTSUPP; /* Needs translation */ + goto fail; } /* * Only privileged root, or (if MNT_USER is set) the user that * did the original mount is permitted to update it. */ error = vfs_suser(mp, td); - if (error != 0) { - vput(vp); - return (error); - } + if (error != 0) + goto fail; if (vfs_busy(mp, MBF_NOWAIT)) { - vput(vp); - return (EBUSY); + error = EBUSY; + goto fail; } VI_LOCK(vp); if ((vp->v_iflag & VI_MOUNT) != 0 || vp->v_mountedhere != NULL) { VI_UNLOCK(vp); vfs_unbusy(mp); - vput(vp); - return (EBUSY); + error = EBUSY; + goto fail; } vp->v_iflag |= VI_MOUNT; VI_UNLOCK(vp); @@ -942,11 +940,12 @@ vfs_domount_update( */ error = VFS_MOUNT(mp); + export_error = 0; if (error == 0) { /* Process the export option. */ if (vfs_copyopt(mp->mnt_optnew, "export", &export, sizeof(export)) == 0) { - error = vfs_export(mp, &export); + export_error = vfs_export(mp, &export); } else if (vfs_copyopt(mp->mnt_optnew, "export", &oexport, sizeof(oexport)) == 0) { export.ex_flags = oexport.ex_flags; @@ -958,7 +957,7 @@ vfs_domount_update( export.ex_masklen = oexport.ex_masklen; export.ex_indexfile = oexport.ex_indexfile; export.ex_numsecflavors = 0; - error = vfs_export(mp, &export); + export_error = vfs_export(mp, &export); } } @@ -1005,6 +1004,11 @@ end: vp->v_iflag &= ~VI_MOUNT; VI_UNLOCK(vp); vrele(vp); + return (error != 0 ? error : export_error); + +fail: + vput(vp); + vfs_freeopts(fsdata); return (error); } %%% -- Jaakko From owner-freebsd-hackers@FreeBSD.ORG Tue Jan 25 18:55:56 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E9BC1065670; Tue, 25 Jan 2011 18:55:56 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 00FCD8FC0C; Tue, 25 Jan 2011 18:55:56 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AB7EC46B0C; Tue, 25 Jan 2011 13:55:55 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6FD4F8A009; Tue, 25 Jan 2011 13:55:54 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org, philip-freebsd1@soeberg.net Date: Tue, 25 Jan 2011 13:55:53 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <4D3EE287.3010203@soeberg.net> In-Reply-To: <4D3EE287.3010203@soeberg.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101251355.53944.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 25 Jan 2011 13:55:54 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Warner Losh Subject: Re: pci_suspend/pci_resume of custom pcie board X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2011 18:55:56 -0000 On Tuesday, January 25, 2011 9:47:35 am Philip Soeberg wrote: > Hi, > > I'm in a particular problem where I need to set my custom pcie adapter > into d3hot power-mode and a couple of seconds later reset it back to d0. > The board has an FPGA directly attached to the pcie interface, and as I > need to re-configure the FPGA on the fly, I have to ensure the datalink > layer between the upstream bridge and my device is idle to prevent any > hickups. > > On linux I simply do a pci_save_state(device) followed by > pci_set_power_state(device, d3hot), then after my magic on my board, I > do the reverse: pci_set_power_state(device, d0) followed by > pci_restore_state(device). > > On FreeBSD, say 8, I've found the pci_set_powerstate function, which is > documented in PCI(9), but that function does not save nor restore the > config space. > > I've tried, just for the fun of it, to go via pci_cfg_save(device, > dinfo, 0) with dinfo being device_get_ivars(device) and then > subsequently restoring the config space back via pci_cfg_restore(), but > since both those functions are declared in I'm > not sure if I'm supposed to use those directly or not.. Besides, I'm not > really having any luck with that approach. > > Reading high and low on the net suggest that not all too many driver > devs are concerned with suspend/resume operation of their device, and if > they are, leave it to user-space to decide when to suspend/resume a > device.. I would like to be able to save off my device' config space, > put it to sleep (d3hot), wake it back up (d0) and restore the device' > config space directly from the device' own driver.. > > Anyone who can help me with this? Use this: pci_cfg_save(dev, dinfo, 0); pci_set_powerstate(dev, PCI_POWERSTATE_D3); /* do stuff */ /* Will set state to D0. */ pci_cfg_restore(dev, dinfo); We probably should create some wrapper routines (pci_save_state() and pci_restore_state() would be fine) that hide the 'dinfo' detail as that isn't something device drivers should have to know. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 02:32:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27909106564A for ; Wed, 26 Jan 2011 02:32:57 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id F37B68FC14 for ; Wed, 26 Jan 2011 02:32:56 +0000 (UTC) Received: by iwn39 with SMTP id 39so470976iwn.13 for ; Tue, 25 Jan 2011 18:32:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.169.133 with SMTP id b5mr7636978icz.241.1296009174604; Tue, 25 Jan 2011 18:32:54 -0800 (PST) Received: by 10.42.8.147 with HTTP; Tue, 25 Jan 2011 18:32:54 -0800 (PST) X-Originating-IP: [68.239.249.252] Date: Tue, 25 Jan 2011 21:32:54 -0500 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Is svn.freebsd.org broken X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 02:32:57 -0000 All I noticed this if you try to check out stable/6 or stable/7 from svn.freebsd.org you get stuck whit a missing file Here is the last few lines of my co A freebsd/tools/regression/usr.bin/jot/regress.wx.out A freebsd/tools/regression/usr.bin/jot/regress.dddd.out svn: In directory 'freebsd/tools/regression/usr.bin/jot' svn: Can't open file 'freebsd/tools/regression/usr.bin/jot/.svn/tmp/text-base/regress.wx.out.svn-base': No such file or directory -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 02:40:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04D79106564A for ; Wed, 26 Jan 2011 02:40:43 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id D16BD8FC13 for ; Wed, 26 Jan 2011 02:40:42 +0000 (UTC) Received: by iwn39 with SMTP id 39so475556iwn.13 for ; Tue, 25 Jan 2011 18:40:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.174.5 with SMTP id t5mr6341584icz.509.1296009642135; Tue, 25 Jan 2011 18:40:42 -0800 (PST) Received: by 10.42.8.147 with HTTP; Tue, 25 Jan 2011 18:40:42 -0800 (PST) X-Originating-IP: [68.239.249.252] Date: Tue, 25 Jan 2011 21:40:42 -0500 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 02:40:43 -0000 Hello Hackers The NetBSD folks have a nice improvement with the rtld-elf subsystem, known as "Negative Symbol Cache" . http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative Roy Marples roy@ has a simple write up of the change. I took the basic idea from FreeBSD, but improved the performance drastically. Basically, the huge win is by caching both breadth and depth of the needed/weak symbol lookup. Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row where we cache both rows and columns. Has anyone looked into porting the changes back to FreeBSD ? The improvement on load time for things like firefox, openoffice, and java is huge on NetBSD. It looks like this change could improve load times on FreeBSD in the same ways. -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 03:12:00 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4829B106564A for ; Wed, 26 Jan 2011 03:12:00 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id D0C2E8FC0A for ; Wed, 26 Jan 2011 03:11:59 +0000 (UTC) Received: by wwf26 with SMTP id 26so488492wwf.31 for ; Tue, 25 Jan 2011 19:11:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=qvbOY0Lmaj3OFxTjvFxRxxGj3zVTXrNyT0E3oPbC2n8=; b=p6g2HbuiruxHP4XcD9AuY64DCnrUnCAe2bWh/gBYqUUutHvRH/daFBNcZbXPLMIGhc qkzuXNvAZTiXW6rtjPdIAmu371rD1Xrl3bRJbLdYuuLGUNLldJJWIyIdPSzmxR1HD4ye cwljf4P/olCdsvhUQBWPDteEtz++UmpPs3UiQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=I6q/0lypbA202Gqu2qRLutjbul0bGG/vbbHo0nXvuhCT5/v1kkfX2NcysMsQjXGw6N mC5+jnW19j9fBXipyZYBlCCqjphM8MqKbEi1iySZiMLpCuDmBdBFXuv2OIKLFYS/EbCc gq7kWFVOCNNO0ssjL/oPZhD6HVYdp4H+ZiyMU= MIME-Version: 1.0 Received: by 10.216.141.75 with SMTP id f53mr642564wej.16.1296011517162; Tue, 25 Jan 2011 19:11:57 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.254.226 with HTTP; Tue, 25 Jan 2011 19:11:57 -0800 (PST) In-Reply-To: References: Date: Tue, 25 Jan 2011 19:11:57 -0800 X-Google-Sender-Auth: _yrAs4Yqv0vjn8TG0PAHrwSRzeI Message-ID: From: Garrett Cooper To: Mark Saad Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Is svn.freebsd.org broken X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 03:12:00 -0000 On Tue, Jan 25, 2011 at 6:32 PM, Mark Saad wrote: > All > =A0I noticed this if you try to check out stable/6 or stable/7 from > svn.freebsd.org you get stuck whit a missing file > > Here is the last few lines of my co > > A =A0 =A0freebsd/tools/regression/usr.bin/jot/regress.wx.out > A =A0 =A0freebsd/tools/regression/usr.bin/jot/regress.dddd.out > svn: In directory 'freebsd/tools/regression/usr.bin/jot' > svn: Can't open file > 'freebsd/tools/regression/usr.bin/jot/.svn/tmp/text-base/regress.wx.out.s= vn-base': > No such file or directory Worked for me :/. -Garrett From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 05:15:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC0DB106564A for ; Wed, 26 Jan 2011 05:15:30 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 9966D8FC0C for ; Wed, 26 Jan 2011 05:15:30 +0000 (UTC) Received: by vws9 with SMTP id 9so272350vws.13 for ; Tue, 25 Jan 2011 21:15:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:x-mailer:mime-version:content-type; bh=+KIp9PFDPoud5qTW0o8fVeLIMNLgII8f1YBEwtR/CIM=; b=Ph5DYpHvWfhdgzPNNvzo5/o7j7mMDnfSjSneeBGUEHsekpuYj/JM4mRK76f++2Xt5q HougKL42GR5mYVjL4b7OMkDjuszWz+1WfR97al6qYWVj6eGuycpe8cX/MN/7pdRcQ+6Y m0bht3qKvm8Q9o5e2/dEURNIJ0pWlwrFSedKU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=rjNnSUTdp0l3dlYb6cD6/vNDSl84mvQC23xmtYxjC3CEO2gbG+z6H4rgNW2yRPuho9 65fg5GAXzBoZDiS8oRNzaTskFEyRQr3vBuCX69bZAwXt5wQ5gWFrJvrikBqQUzEjRxZQ 6mwbWDUrOWm7OwYb7qkZgdCAKXieY5TFoSuco= Received: by 10.220.190.193 with SMTP id dj1mr1306420vcb.221.1296017365001; Tue, 25 Jan 2011 20:49:25 -0800 (PST) Received: from kan.dnsalias.net (c-24-63-226-98.hsd1.ma.comcast.net [24.63.226.98]) by mx.google.com with ESMTPS id h18sm6532782vbr.14.2011.01.25.20.49.22 (version=SSLv3 cipher=RC4-MD5); Tue, 25 Jan 2011 20:49:23 -0800 (PST) Date: Tue, 25 Jan 2011 23:49:11 -0500 From: Alexander Kabaev To: Mark Saad Message-ID: <20110125234911.223d8f75@kan.dnsalias.net> In-Reply-To: References: X-Mailer: Claws Mail 3.7.8 (GTK+ 2.22.1; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/qv9ymm95AA3elad7fvJWoMH"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 05:15:31 -0000 --Sig_/qv9ymm95AA3elad7fvJWoMH Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Tue, 25 Jan 2011 21:40:42 -0500 Mark Saad wrote: > Hello Hackers >=20 > The NetBSD folks have a nice improvement with the rtld-elf subsystem, > known as "Negative Symbol Cache" . >=20 > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative >=20 > Roy Marples roy@ has a simple write up of the change. >=20 > I took the basic idea from FreeBSD, but improved the performance > drastically. Basically, the huge win is by caching both breadth and > depth of the needed/weak symbol lookup. > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row > where we cache both rows and columns. >=20 > Has anyone looked into porting the changes back to FreeBSD ? The > improvement on load time for things like firefox, openoffice, and java > is huge on NetBSD. It looks like this change could improve load times > on FreeBSD in the same ways. >=20 This is a second time someone posts this to public mailing list and curiously enough is a second time it suggested that someone else is to do the investigation. From the quick look, the commit in question is more or less a direct rip-off of Donelists we had for ages and as such is completely over-hyped. The only extra quirk that said commit does is an optimization of a dlsym() call, which is hardly ever in critical performance path. Said optimization is trivial and easy to try. Here you have it: http://people.freebsd.org/~kan/rtld-symlook-depth.diff Since it only applies to dlsym, it only affects programs that are heavy plugin users, which I suppose is the category OpenOffice and firefox both fall into. Care to do some benchmarks with and without the patch and report the results? I frankly doubt that you'll see any noticeable difference compared to our stock rtld's performance. --=20 Alexander Kabaev --Sig_/qv9ymm95AA3elad7fvJWoMH Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iD8DBQFNP6fMQ6z1jMm+XZYRApDcAJ9Qo4tdIl4xVxTwK+k61vAWxX7ZFwCeJNbs 9Bha2Y0q9hEBtn/L7WpwA2E= =0nHC -----END PGP SIGNATURE----- --Sig_/qv9ymm95AA3elad7fvJWoMH-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 07:09:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0E1B106566B; Wed, 26 Jan 2011 07:09:44 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 14E158FC12; Wed, 26 Jan 2011 07:09:43 +0000 (UTC) Received: by wwf26 with SMTP id 26so602542wwf.31 for ; Tue, 25 Jan 2011 23:09:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=3Nm/5GyYuuD7d1Qh16Z5zYBULC3iOS1XgJ+K5Mct1jE=; b=m9+2zbtyTz66VHlJjIar5DIRxep3qLuvRe8EZFmxSSGYYF5c+776svFBj+uZ1cTm0d N0D4N/A7BHE+nDKxb1IgR1xEi8p56l9bn9En3kOZ0F31eIAj0YEtszimL5VLk4kZQZUB 3Rrw9w+GeVIA4OLUUNTX8o7N5fDaki3lUP8aM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; b=F6VcF729EGqjwzm0ZrGAiDMKZVj9U5mOSOJSlcP1P8gCq8gZzQE2xM/eJIjQ5nMBub 3QGoZtsAFR8RZR9Y9PeM9hZJRWILObYF2nVUwR13kBBqRa2lpzPvw7G2/lwcnuuqKNtm Sa0OGimOehlN+1sms9+jtLQYEJ6gYUDV5e+h4= Received: by 10.227.154.129 with SMTP id o1mr7119974wbw.109.1296025781940; Tue, 25 Jan 2011 23:09:41 -0800 (PST) Received: from localhost (lan-78-157-92-5.vln.skynet.lt [78.157.92.5]) by mx.google.com with ESMTPS id u9sm624196wbg.6.2011.01.25.23.09.39 (version=SSLv3 cipher=RC4-MD5); Tue, 25 Jan 2011 23:09:40 -0800 (PST) Date: Wed, 26 Jan 2011 09:09:32 +0200 From: Gleb Kurtsou To: Alexander Best Message-ID: <20110126070932.GA3086@tops.skynet.lt> References: <20110124113306.GA79890@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110124113306.GA79890@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Garrett Cooper , "Simon L. B. Nielsen" , Matthew Fleming , freebsd-hackers@freebsd.org, Mike Tancsa Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 07:09:44 -0000 On (24/01/2011 11:33), Alexander Best wrote: > On Mon Jan 24 11, Garrett Cooper wrote: > > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wrote: > > > On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wrote: > > >>Perhaps we should just set the tinderbox up to sync directly of cvsup-master instead if that makes it more useful? > > > > > > Can cvsup-master still lose atomicity of commits?  I suspect it can, > > > in which case syncing directly off the SVN master would seem a better > > > approach. > > > > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I > > wonder if it's time to look at another solution to this problem as > > these annoying stability issues don't appear to be going away. What > > about git? > > > > Just some things I'm able to rattle off that come to mind with git.. > > it would also be nice to have github running on freebsd.org. that way it would > be much easier to discuss src changes without having to point people at a file, > a function or even a specific line. also it would finally kill the > mailinglists, which have lots of issues: spam, broken mailman installation, > people going berserker when they see lines > 80 etc. there have been a few > attempts to introduce a code review system, but since that was all hosted on > foreign websites the idea never cought on and afaik those websites weren't > being supported/promoted by freebsd.org. Having github would be nice, but it's not open source. Another option could be gitorious, there are merge requests with review option[1], patch review, already hosted freebsd repository[2]. All we need as a first step is developers starting accepting merge requests from each other, people use it already[3]. 1. http://blog.gitorious.org/2009/11/06/awesome-code-review/ 2. http://gitorious.org/freebsd 3. http://gitorious.org/freebsd/repositories > but personally i don't expect a change like this to happen in the near future. > basically most of the freebsd administrative people are quite conservative. i > wouldn't be surprised if some them are still trying to run freebsd on their > typewriters. ;) > > cheers. > alex > > > > > Some arguments `for git'... > > > > 1. One tool to rule them all: > > - cvsup/csup can be replaced with git archive [1]. > > - cvs git scales a bit better. > > - less support cost for p4 and lower likelihood of downtime in the > > event of critical failure (perforce's proprietary DB is a pain to > > recover I've recently discovered from other dealings). > > - svn <-> cvs exporter is no longer required as it's all one SCM. > > - As a side-effect, the bits present in CVS and SVN would now be > > 100% in sync, unlike cvs which can lead svn in terms of commits (at > > least that was the case when I last talked to someone about version > > numbering in pkg_install done by re@). > > 2. More evolved tool: > > - branches are cheap and can be local or remote. > > - distributed SCM seem to work well with large groups of developers. > > - works better with branching and merging from what I've seen. > > > > Some arguments against git... > > - The one caveat to cvsup/csup that's awesome is its componentization > > capability, i.e. being able to selectively download components in src > > / ports; I'm not 100% sure but there doesn't appear to be a clear > > analog in git. It might be achievable through gits remote. in > > git-config, git-remote, etc, but I would need to prototype whether or > > not this is true. > > - Higher learning curve. > > - Some slightly annoying nits with stashing local changes when working > > on separate branches (need to talk to git maintainers). > > - > > > > Some more git experienced folks could comment here, but it would > > be nice to unify all of the systems under `one flag' for the sake of > > simplicity and hopefully the sanity of the tool maintainers (Simon, et > > all). > > Thanks! > > -Garrett > > -- > a13x > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 07:15:18 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E64AC106566B; Wed, 26 Jan 2011 07:15:17 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 530B78FC12; Wed, 26 Jan 2011 07:15:16 +0000 (UTC) Received: by wyf19 with SMTP id 19so636822wyf.13 for ; Tue, 25 Jan 2011 23:15:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=KXoqMVqjkqCOTNuTCbE6F+kqqWiI6ak6MsNMqSicNno=; b=P8BlNsMt9vxzuY8R6keUPF+jMR3iRzyr5ykKXmj+9w82FlTubR9wtwNpIjkCwI7ecR OwXeZecUJ+4PfuASAvs+wxk2t8Syq5ShPgidjZLa3eroa9SNtC3Ibhpvra7mMYITEObL b7lCLZs1oyjGcUkP+dmvzbg75RL3I/0Mct2BY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=YwmgeeXmem98GDDef6IC9uMdfJ9Y7x3ZRnANqn5/nueHpo/1GECxF3QvbGvVjKQRzH qKdAceZJHGthTASkyEZea8/Fu5AiWqaxt8omWSpVDhecwYvfGJMPExWD5fiyd/A2UKdy tHOpSJD3AHNhMg0Dra/0KTR9OSYfBANnGZuhU= MIME-Version: 1.0 Received: by 10.216.185.142 with SMTP id u14mr4682386wem.31.1296026115994; Tue, 25 Jan 2011 23:15:15 -0800 (PST) Received: by 10.216.254.226 with HTTP; Tue, 25 Jan 2011 23:15:15 -0800 (PST) In-Reply-To: <20110126070932.GA3086@tops.skynet.lt> References: <20110124113306.GA79890@freebsd.org> <20110126070932.GA3086@tops.skynet.lt> Date: Tue, 25 Jan 2011 23:15:15 -0800 Message-ID: From: Garrett Cooper To: Gleb Kurtsou Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Alexander Best , Mike Tancsa , Matthew Fleming , freebsd-hackers@freebsd.org, "Simon L. B. Nielsen" Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 07:15:18 -0000 On Tue, Jan 25, 2011 at 11:09 PM, Gleb Kurtsou wro= te: > On (24/01/2011 11:33), Alexander Best wrote: >> On Mon Jan 24 11, Garrett Cooper wrote: >> > On Sun, Jan 23, 2011 at 9:16 PM, Peter Jeremy wr= ote: >> > > On 2011-Jan-21 20:01:32 +0100, "Simon L. B. Nielsen" wrote: >> > >>Perhaps we should just set the tinderbox up to sync directly of cvsu= p-master instead if that makes it more useful? >> > > >> > > Can cvsup-master still lose atomicity of commits? =A0I suspect it ca= n, >> > > in which case syncing directly off the SVN master would seem a bette= r >> > > approach. >> > >> > I've seen a lot of `self-healing' failures lately w.r.t. cvsup, so I >> > wonder if it's time to look at another solution to this problem as >> > these annoying stability issues don't appear to be going away. What >> > about git? >> > >> > Just some things I'm able to rattle off that come to mind with git.. >> >> it would also be nice to have github running on freebsd.org. that way it= would >> be much easier to discuss src changes without having to point people at = a file, >> a function or even a specific line. also it would finally kill the >> mailinglists, which have lots of issues: spam, broken mailman installati= on, >> people going berserker when they see lines > 80 etc. there have been a f= ew >> attempts to introduce a code review system, but since that was all hoste= d on >> foreign websites the idea never cought on and afaik those websites weren= 't >> being supported/promoted by freebsd.org. > > Having github would be nice, but it's not open source. Another option > could be gitorious, there are merge requests with review option[1], patch > review, already hosted freebsd repository[2]. > > All we need as a first step is developers starting accepting merge > requests from each other, people use it already[3]. > > 1. http://blog.gitorious.org/2009/11/06/awesome-code-review/ > 2. http://gitorious.org/freebsd > 3. http://gitorious.org/freebsd/repositories This is only for src though. I was going to start up some mirroring of ports as well on gitorious, but it would have been nice if it could have been covered like so: freebsd/docs.git freebsd/ports.git freebsd/src.git So that way people could just clone one of the above and work merrily in the component as they felt fit, or check out all three and work away as necessary. Plus it would be more 1:1 than it currently is. Thanks! -Garrett From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 07:53:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3F6C106564A for ; Wed, 26 Jan 2011 07:53:54 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net [IPv6:2001:7b8:2ff:146::2]) by mx1.freebsd.org (Postfix) with ESMTP id 85F6C8FC14 for ; Wed, 26 Jan 2011 07:53:54 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:2927:9020:dfab:77ae] (unknown [IPv6:2001:7b8:3a7:0:2927:9020:dfab:77ae]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EEBBA5C59; Wed, 26 Jan 2011 08:53:52 +0100 (CET) Message-ID: <4D3FD311.6040201@FreeBSD.org> Date: Wed, 26 Jan 2011 08:53:53 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.2.14pre) Gecko/20110122 Lanikai/3.1.8pre MIME-Version: 1.0 To: Mark Saad References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Is svn.freebsd.org broken X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 07:53:54 -0000 On 2011-01-26 03:32, Mark Saad wrote: > svn: In directory 'freebsd/tools/regression/usr.bin/jot' > svn: Can't open file > 'freebsd/tools/regression/usr.bin/jot/.svn/tmp/text-base/regress.wx.out.svn-base': > No such file or directory Your local checkout is most likely broken. Subversion's working copy code is extremely fragile, unfortunately. :( Try running "svn cleanup" in your working copy, in most cases that fixes the failure. If not, just delete the whole working copy, and checkout from scratch again. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 15:52:32 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68DE51065670 for ; Wed, 26 Jan 2011 15:52:32 +0000 (UTC) (envelope-from feld@feld.me) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3A6078FC14 for ; Wed, 26 Jan 2011 15:52:31 +0000 (UTC) Received: by iwn39 with SMTP id 39so1036341iwn.13 for ; Wed, 26 Jan 2011 07:52:31 -0800 (PST) Received: by 10.42.229.1 with SMTP id jg1mr596599icb.374.1296055530170; Wed, 26 Jan 2011 07:25:30 -0800 (PST) Received: from tech304 (supranet-tech.secure-on.net [66.170.8.18]) by mx.google.com with ESMTPS id d21sm13007654ibg.21.2011.01.26.07.25.28 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 26 Jan 2011 07:25:28 -0800 (PST) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-hackers@freebsd.org References: <20110125234911.223d8f75@kan.dnsalias.net> Date: Wed, 26 Jan 2011 09:25:27 -0600 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Mark Felder" Message-ID: In-Reply-To: <20110125234911.223d8f75@kan.dnsalias.net> User-Agent: Opera Mail/11.01 (FreeBSD) Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 15:52:32 -0000 On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev wrote: > The only extra quirk that said commit > does is an optimization of a dlsym() call, which is hardly ever in > critical performance path. It's really not my place to say, but it seems strange that if an optimization is available people would ignore it because they don't think it's important enough. I don't understand this mentality; if it's not going to break anything and it obviously can improve performance in certain use cases, why not merge it and make FreeBSD even better? Regards, Mark From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 16:02:14 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CA60106566C for ; Wed, 26 Jan 2011 16:02:14 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id BC3878FC0C for ; Wed, 26 Jan 2011 16:02:13 +0000 (UTC) Received: by qwj9 with SMTP id 9so1088631qwj.13 for ; Wed, 26 Jan 2011 08:02:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:x-mailer:mime-version:content-type; bh=leFoZGTkqLxXtVxpYzl7Hj0UwSXmuvDRyVbtBWsp1Fo=; b=SD+Y0qK2aTl7IUHkmKuXh43NVPfsg0PoTAUwNo/EK5/VfZMhT/eDfwMPblVdW/rWcX hlsFZneB4dEBWMsXp/N1HN3dtCXGELHnt+QzwSSYY1WfAin+KokCRDnwlthZJ3sBdUsq yIdrKUkBpiX+1XLBFJ6efGrcgDz978BzeUi+o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=HauoXBNNOX5Rw2yafGyFAiVD/GaK3ui5V9BrZtQyMfkYINNeCauu6i5cZra0/IszT8 LtxJCQDcaISg9iQTO2NdDcE1BjAsLjYqJ6JxZE3WUSzZin91+QYDWEqt1jadkd9EBeYY q0eqlDhHLMHi4V8+YGah69lidfY6Cvvor5gvM= Received: by 10.229.181.9 with SMTP id bw9mr525789qcb.143.1296057732767; Wed, 26 Jan 2011 08:02:12 -0800 (PST) Received: from kan.dnsalias.net (c-24-63-226-98.hsd1.ma.comcast.net [24.63.226.98]) by mx.google.com with ESMTPS id t7sm11091591qcs.4.2011.01.26.08.02.11 (version=SSLv3 cipher=RC4-MD5); Wed, 26 Jan 2011 08:02:11 -0800 (PST) Date: Wed, 26 Jan 2011 11:02:04 -0500 From: Alexander Kabaev To: "Mark Felder" Message-ID: <20110126110204.074b5844@kan.dnsalias.net> In-Reply-To: References: <20110125234911.223d8f75@kan.dnsalias.net> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.22.1; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/azEq504onH=hb/AypFpCqA0"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:02:14 -0000 --Sig_/azEq504onH=hb/AypFpCqA0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 26 Jan 2011 09:25:27 -0600 "Mark Felder" wrote: > On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev > wrote: >=20 > > The only extra quirk that said commit > > does is an optimization of a dlsym() call, which is hardly ever in > > critical performance path. >=20 > It's really not my place to say, but it seems strange that if an =20 > optimization is available people would ignore it because they don't > think it's important enough. I don't understand this mentality; if > it's not going to break anything and it obviously can improve > performance in certain use cases, why not merge it and make FreeBSD > even better? >=20 >=20 The link to patch with said optimization is provided and one would hope the next email on this thread provides something more alike to hard numbers proving that it actually helps, instead of mentality discussions. --=20 Alexander Kabaev --Sig_/azEq504onH=hb/AypFpCqA0 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iD8DBQFNQEWBQ6z1jMm+XZYRAjZ+AKDVLMZJd/Bss0XHO6Utmo53LExoSACdE/7L L4t0kO5xil3UgPhl4X1kP38= =WfM2 -----END PGP SIGNATURE----- --Sig_/azEq504onH=hb/AypFpCqA0-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 16:14:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11247106566B for ; Wed, 26 Jan 2011 16:14:25 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id ED95D8FC1A for ; Wed, 26 Jan 2011 16:14:24 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 9810F2282A; Wed, 26 Jan 2011 09:13:24 -0700 (MST) Received: by night.db.net (Postfix, from userid 100) id 390D25CA3; Wed, 26 Jan 2011 11:14:23 -0500 (EST) Date: Wed, 26 Jan 2011 11:14:23 -0500 From: Diane Bruce To: Alexander Kabaev Message-ID: <20110126161423.GA49438@night.db.net> References: <20110125234911.223d8f75@kan.dnsalias.net> <20110126110204.074b5844@kan.dnsalias.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110126110204.074b5844@kan.dnsalias.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-hackers@freebsd.org, Mark Felder Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:14:25 -0000 On Wed, Jan 26, 2011 at 11:02:04AM -0500, Alexander Kabaev wrote: > On Wed, 26 Jan 2011 09:25:27 -0600 > "Mark Felder" wrote: > > > On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev > > wrote: ... > numbers proving that it actually helps, instead of mentality Or even slowing things down. > Alexander Kabaev - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 16:40:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C3949106566B for ; Wed, 26 Jan 2011 16:40:43 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 93ECD8FC14 for ; Wed, 26 Jan 2011 16:40:43 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id DD43846B0D; Wed, 26 Jan 2011 11:40:42 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 18D408A009; Wed, 26 Jan 2011 11:40:42 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Wed, 26 Jan 2011 11:40:13 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <20110125234911.223d8f75@kan.dnsalias.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201101261140.14024.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 26 Jan 2011 11:40:42 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Mark Felder Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 16:40:43 -0000 On Wednesday, January 26, 2011 10:25:27 am Mark Felder wrote: > On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev > wrote: > > > The only extra quirk that said commit > > does is an optimization of a dlsym() call, which is hardly ever in > > critical performance path. > > It's really not my place to say, but it seems strange that if an > optimization is available people would ignore it because they don't think > it's important enough. I don't understand this mentality; if it's not > going to break anything and it obviously can improve performance in > certain use cases, why not merge it and make FreeBSD even better? Many things that seem obvious aren't actually true, hence the need for actual testing and benchmarks. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 17:37:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14D241065701 for ; Wed, 26 Jan 2011 17:37:49 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 972B58FC16 for ; Wed, 26 Jan 2011 17:37:48 +0000 (UTC) Received: by wwf26 with SMTP id 26so1121554wwf.31 for ; Wed, 26 Jan 2011 09:37:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:x-mailer:mime-version:content-type :content-transfer-encoding; bh=eu8GNs63L0zk61aW6N6b7SLemyPhrOOd407SCahZwoQ=; b=MtWTvcLYtcz8HTFGLb9dkN2YfHUaYxd963peKuEtoLsmE/h1+p2PlYUFpm+gMJ+0DZ 5TQUBLXHDlmKm8t7JVpfSMO3t7+nEzGx9xlGgdqangGlMWhFt/Mdmxn4DFWaJs6DPK/h 11gFi4DLshVElWzqmFQY4rwdsWo/hJXr1hULI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=dV71A0ARuAcJ0MU/uSk8rj6iT26GC22JDNPArQSuJo0X7GKX4xWiu8NeRr0EvOoWu3 2YmnfSELxYtKro5fQc3Y8pBS8BEagv8xJ2U1VUy0xrhxrj2+XM1nli7uoO2jw5SsXatg fDTayosW5ZWiZvQEofSCoBg9GQa4G7T/9MLoI= Received: by 10.227.135.75 with SMTP id m11mr7824128wbt.122.1296063466396; Wed, 26 Jan 2011 09:37:46 -0800 (PST) Received: from ernst.jennejohn.org (p578E15B7.dip.t-dialin.net [87.142.21.183]) by mx.google.com with ESMTPS id y29sm5978679wbd.4.2011.01.26.09.37.44 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 26 Jan 2011 09:37:45 -0800 (PST) Date: Wed, 26 Jan 2011 18:37:43 +0100 From: Gary Jennejohn To: John Baldwin Message-ID: <20110126183743.56f6406c@ernst.jennejohn.org> In-Reply-To: <201101261140.14024.jhb@freebsd.org> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101261140.14024.jhb@freebsd.org> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Mark Felder Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 17:37:49 -0000 On Wed, 26 Jan 2011 11:40:13 -0500 John Baldwin wrote: > On Wednesday, January 26, 2011 10:25:27 am Mark Felder wrote: > > On Tue, 25 Jan 2011 22:49:11 -0600, Alexander Kabaev > > wrote: > > > > > The only extra quirk that said commit > > > does is an optimization of a dlsym() call, which is hardly ever in > > > critical performance path. > > > > It's really not my place to say, but it seems strange that if an > > optimization is available people would ignore it because they don't think > > it's important enough. I don't understand this mentality; if it's not > > going to break anything and it obviously can improve performance in > > certain use cases, why not merge it and make FreeBSD even better? > > Many things that seem obvious aren't actually true, hence the need for > actual testing and benchmarks. > I can't claim to have rigorously benchmarked this, but I am running with a patched ld-elf.so.1 right now and can state that *subjectively* there is absolutely no difference in the perceived performance. firefox, opera and OpenOffice still seem to be dogs the first time they start up. Since this is all about perception I see no benefit in applying the patch, although it doesn't seem to have broken anything either. -- Gary Jennejohn (gj@) From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 20:01:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED39A1065672 for ; Wed, 26 Jan 2011 20:01:44 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 788998FC19 for ; Wed, 26 Jan 2011 20:01:44 +0000 (UTC) Received: from park.js.berklix.net (p5B22F5C0.dip.t-dialin.net [91.34.245.192]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p0QK1Tmi029932; Wed, 26 Jan 2011 20:01:31 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p0QK1O5d003186; Wed, 26 Jan 2011 21:01:25 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p0QK0fGT005251; Wed, 26 Jan 2011 21:00:46 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201101262000.p0QK0fGT005251@fire.js.berklix.net> To: Alexander Best From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Tue, 25 Jan 2011 14:05:17 GMT." <20110125140517.GA74156@freebsd.org> Date: Wed, 26 Jan 2011 21:00:41 +0100 Sender: jhs@berklix.com Cc: Giorgos Keramidas , freebsd-hackers@freebsd.org, db@db.net, perryh@pluto.rain.com, ivoras@freebsd.org Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 20:01:45 -0000 Hi, Alex > order to build the system. the VCS however is not part of the build toolchain > however (except 'make update' maybe). Some good points, but also remember make release also uses cvs grep CVS /usr/src/release/Makefile Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, Not HTML, Not base 64. Reply below text sections not at top, to avoid breaking cumulative context. From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 26 22:13:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91E2A106564A; Wed, 26 Jan 2011 22:13:49 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id CE3198FC16; Wed, 26 Jan 2011 22:13:48 +0000 (UTC) Received: by wyf19 with SMTP id 19so1441746wyf.13 for ; Wed, 26 Jan 2011 14:13:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=OqQWjwhieeOxG0tgkjHSgMgB1WX2LeTIiH7l6+rOqjM=; b=Y/pr9eMuW1m93OnfkXfN2n346DuA9tOBN3MZ4jIzDfF8vDlxes9QnhR5ddsmDAvkgw X7Ee/A8nS2PruEBYQvd29qHq7K+FQeCp2naMB5Qg5WFyzso56eiA6GiaQbAVTurAFCEc 4qkgPRHmjZu0i6gFTiaPWxnA+jqspPnNaMOX8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=kfwYjc5a55pUbev2fKzA9voe3cSv7WmbE+S3mb5J0kDF7KhO60o6acE2KpmQGFanwl pVUceErpKO7B/5azEnkcizYG4LYm/8E/g9wmbCz3bjJ3QejXTK1I/9mixhAvkpmHAz0y 5l5sJM2VicjruDRx4iyMqyPsZk1lw7oxi4IDk= MIME-Version: 1.0 Received: by 10.216.29.71 with SMTP id h49mr1033968wea.46.1296080027605; Wed, 26 Jan 2011 14:13:47 -0800 (PST) Received: by 10.216.254.226 with HTTP; Wed, 26 Jan 2011 14:13:47 -0800 (PST) In-Reply-To: <201101262000.p0QK0fGT005251@fire.js.berklix.net> References: <20110125140517.GA74156@freebsd.org> <201101262000.p0QK0fGT005251@fire.js.berklix.net> Date: Wed, 26 Jan 2011 14:13:47 -0800 Message-ID: From: Garrett Cooper To: "Julian H. Stacey" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: db@db.net, perryh@pluto.rain.com, freebsd-hackers@freebsd.org, ivoras@freebsd.org, Giorgos Keramidas , Alexander Best Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2011 22:13:49 -0000 On Wed, Jan 26, 2011 at 12:00 PM, Julian H. Stacey wrote: > Hi, > Alex >> order to build the system. the VCS however is not part of the build tool= chain >> however (except 'make update' maybe). > > Some good points, =A0but also remember make release also uses cvs > =A0 =A0 =A0 =A0 grep CVS /usr/src/release/Makefile For convenience reasons, just like cdrtools exists purely for utility in release as well. As long as the license doesn't say "[if you use our tool,] ur srcs are ours", then I don't see why license matters here. As and long as we package the source with the OS and all of our changes, or provide the ability to install it via a port, we should be ok. clang, llvm, compiler_rt, etc are a different can of worms as the GPLv2 // LGPLv2 is viral and says "[if you use our tool with your srcs,] ur srcs have to be open" (paraphrased of course), which doesn't work for companies who have proprietary IP. Thanks, -Garrett PS IANAL. From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 06:03:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B99FF106564A for ; Thu, 27 Jan 2011 06:03:31 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id 7D4478FC08 for ; Thu, 27 Jan 2011 06:03:31 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id 9B2A27E8D7; Thu, 27 Jan 2011 16:43:37 +1100 (EST) Message-ID: <4D410609.3000104@freebsd.org> Date: Thu, 27 Jan 2011 16:43:37 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101215 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Andriy Gapon References: <4AD6067E.2010503@icyb.net.ua> In-Reply-To: <4AD6067E.2010503@icyb.net.ua> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on lauren.room52.net Cc: freebsd-hackers@freebsd.org Subject: Re: heci: a new driver for review and testing X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 06:03:31 -0000 Hi Andriy, On 10/15/09 04:12, Andriy Gapon wrote: > > Some time ago I posted some ideas about HECI/MEI driver for FreeBSD: > http://docs.freebsd.org/cgi/mid.cgi?4968E9A1.3080006 > > I actually got around to implementing it (in initial/basic form): > http://people.freebsd.org/~avg/heci.tgz An old thread I know, but I just noticed my desktop has AMT support and was investigating if I could get access to the Serial Over Lan feature. I stumbled across your HECI driver and thought I'd give it a spin. It loads and attaches fine and unloads without issue as well. I tested with device "HECI_DEV_ID_ICH9_82Q35" on a HP DC7800 minitower machine. I get no output when I run your "heci-qst" program though. Few more details: heci0@pci0:0:3:0: class=0x078000 card=0x2819103c chip=0x29b48086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = 'Intel(R) Management Engine Interface (HECI) (Q35-Chipset)' class = simple comms lstewart@lstewart> uname -a FreeBSD lstewart 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #0 r217387M: Fri Jan 14 15:52:16 EST 2011 lstewart@lstewart:/usr/obj/usr/src/sys/DESKTOP amd64 Cheers, Lawrence From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 09:20:59 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73E481065674 for ; Thu, 27 Jan 2011 09:20:59 +0000 (UTC) (envelope-from philip-freebsd1@soeberg.net) Received: from pasmtpA.tele.dk (pasmtpa.tele.dk [80.160.77.114]) by mx1.freebsd.org (Postfix) with ESMTP id 357138FC0C for ; Thu, 27 Jan 2011 09:20:58 +0000 (UTC) Received: from mail.soeberg.net (0x573f534a.cpe.ge-1-1-0-1109.bynqu1.customer.tele.dk [87.63.83.74]) by pasmtpA.tele.dk (Postfix) with ESMTP id DC6E58006A5 for ; Thu, 27 Jan 2011 10:20:57 +0100 (CET) Received: from [10.240.10.87] ([188.120.77.114]) (authenticated user philip@soeberg.net) by mail.soeberg.net (using TLSv1/SSLv3 with cipher AES256-SHA (256 bits)) for freebsd-hackers@freebsd.org; Thu, 27 Jan 2011 10:21:08 +0100 Message-ID: <4D413936.8070301@soeberg.net> Date: Thu, 27 Jan 2011 10:21:58 +0100 From: Philip Soeberg User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4D3EE287.3010203@soeberg.net> <201101251355.53944.jhb@freebsd.org> In-Reply-To: <201101251355.53944.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: pci_suspend/pci_resume of custom pcie board X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: philip-freebsd1@soeberg.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 09:20:59 -0000 On 25-01-2011 19:55, John Baldwin wrote: > Use this: > > pci_cfg_save(dev, dinfo, 0); > pci_set_powerstate(dev, PCI_POWERSTATE_D3); > > /* do stuff */ > > /* Will set state to D0. */ > pci_cfg_restore(dev, dinfo); > > We probably should create some wrapper routines (pci_save_state() and > pci_restore_state() would be fine) that hide the 'dinfo' detail as that isn't > something device drivers should have to know. Excellent, Thank you.. I must have been half asleep as couldn't for the love of god get that config space restore to work when I tried it yesterday, but now all is peachy. Must have forgotten a crucial step in that.. 3 step list you provided.. sighh.. :) No need to wrap dinfo.. It's actually quite nice to have the cache at hand -if- the device should "choose" to change a parameter, say its device id. I'd rather we'd focus on extending the PCI_POWERSTATE_D3 to include a specific Hot version. D3 can mean both Cold and Hot, and there's a distinct difference between the two (whether to remove power from the device or not).. Not that I know of any PM implementation who actually differentiates as of yet, but I wouldn't be surprised if all this green energy wave thing we see in the server segment push towards powering off a slot once its device has entered d3cold. Anyway, Many thanks for your help. /Phil From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 11:33:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E0E5106564A for ; Thu, 27 Jan 2011 11:33:41 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 305788FC12 for ; Thu, 27 Jan 2011 11:33:40 +0000 (UTC) Received: by gxk8 with SMTP id 8so592623gxk.13 for ; Thu, 27 Jan 2011 03:33:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:from:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id; bh=mU2RAZbA+mM21PIN7uIW/TLpfz1/kh5vghcldEj5ZCA=; b=UPuOlVog/8nFJr36jJ9R+USMWi1KSSAtEAzqnuANYgY8RURBHr9zt/T4nIoortk8Ce ZI5NP/8pgesgtSU6UXokG10N/8XfeEGYrRwz7E+Nk1FpXoSSSDUGH7mipNAweMdBvqzc BHljy58fItt7j93LPSuj2YMgTS8UAiNuSgwZU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding:message-id; b=MKtyfHbj+qWyKAs65pyU57ykFNfKlq4ExSy/q+lGU3aTTFUUM7vr08i2WtTAc9EDR4 9Ov2oajlddyodOgAE6Yo5p4nq/UfG+cl0o9jnu+2a4vl8CDA+pEGKqHm7X/jxYfs9+Ud V4L87HV4raS4Nt/1D4HRNHtSy+qmCROFMvjuw= Received: by 10.90.232.6 with SMTP id e6mr2869219agh.52.1296126334655; Thu, 27 Jan 2011 03:05:34 -0800 (PST) Received: from dragon.dg (41-132-211-236.dsl.mweb.co.za [41.132.211.236]) by mx.google.com with ESMTPS id b19sm20313583ana.27.2011.01.27.03.05.31 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 27 Jan 2011 03:05:33 -0800 (PST) From: David Naylor To: Alexander Kabaev Date: Thu, 27 Jan 2011 13:05:17 +0200 User-Agent: KMail/1.13.5 (FreeBSD/9.0-CURRENT; KDE/4.5.4; amd64; ; ) References: <20110125234911.223d8f75@kan.dnsalias.net> In-Reply-To: <20110125234911.223d8f75@kan.dnsalias.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1565754.HW3NjhCSA8"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201101271305.21510.naylor.b.david@gmail.com> Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 11:33:41 -0000 --nextPart1565754.HW3NjhCSA8 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: > On Tue, 25 Jan 2011 21:40:42 -0500 >=20 > Mark Saad wrote: > > Hello Hackers > >=20 > > The NetBSD folks have a nice improvement with the rtld-elf subsystem, > > known as "Negative Symbol Cache" . > >=20 > > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative > >=20 > > Roy Marples roy@ has a simple write up of the change. > >=20 > > I took the basic idea from FreeBSD, but improved the performance > > drastically. Basically, the huge win is by caching both breadth and > > depth of the needed/weak symbol lookup. > > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row > > where we cache both rows and columns. > >=20 > > Has anyone looked into porting the changes back to FreeBSD ? The > > improvement on load time for things like firefox, openoffice, and java > > is huge on NetBSD. It looks like this change could improve load times > > on FreeBSD in the same ways. >=20 > This is a second time someone posts this to public mailing list and > curiously enough is a second time it suggested that someone else is to > do the investigation. From the quick look, the commit in question is > more or less a direct rip-off of Donelists we had for ages and as > such is completely over-hyped. The only extra quirk that said commit > does is an optimization of a dlsym() call, which is hardly ever in > critical performance path. Said optimization is trivial and easy to > try. Here you have it: > http://people.freebsd.org/~kan/rtld-symlook-depth.diff >=20 > Since it only applies to dlsym, it only affects programs that are heavy > plugin users, which I suppose is the category OpenOffice and firefox > both fall into. Care to do some benchmarks with and without the > patch and report the results? I frankly doubt that you'll see any > noticeable difference compared to our stock rtld's performance. I benchmarked the impact said patch has on the boot-time of my system. I=20 timed the boot-time to when KDE launches autostart programs and once all=20 programs have loaded (I run a few extra programs, such as amarok). The lat= ter=20 measure requires human action thus it has extra, human, variance in its=20 measure. =20 I tried an older version of rtld (about 2 months old), current version of r= tld=20 and the new (patched) rtld. I ran each test three times. There was little= =20 variance in the tests and I am confident that there is no difference betwee= n=20 the different rtld versions and my boot-time. =20 Here is a summary of my boot times (in seconds). First measure is when KDE= =20 autostarts programs, the latter is when I determined when all programs had= =20 launched. =20 rtld-old: 69 96 rtld: 69 94 rtld-new: 69 94 Please note that kernel boot time is approximately 10 seconds and kdm is=20 delayed by about 10 seconds thus 20 seconds can be removed from above numbe= rs=20 to determine non-kernel boot wall-time. =20 I would like to add that the blog entry claims a substantial improvement fo= r=20 some use cases. Is it not worth to optimise these fringe cases as one mans= =20 fringe case is another mans normal case (or woman as one prefers)? =20 --nextPart1565754.HW3NjhCSA8 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iEYEABECAAYFAk1BUXEACgkQUaaFgP9pFrJbSgCggF7p2zHoGI7vTAYqTrfeIb6r 37oAnR4OPPQZ+uLWBMZx1tpzoDi9Vfkn =NgxZ -----END PGP SIGNATURE----- --nextPart1565754.HW3NjhCSA8-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 12:04:55 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B9F610656A8; Thu, 27 Jan 2011 12:04:55 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id E3C668FC1D; Thu, 27 Jan 2011 12:04:54 +0000 (UTC) Received: from mx1.psconsult.nl (psc11.adsl.iaf.nl [80.89.238.138]) by mx1.psconsult.nl (8.14.4/8.14.4) with ESMTP id p0RC4l75050188 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Jan 2011 13:04:53 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.4/8.14.4/Submit) id p0RC4lp0050187; Thu, 27 Jan 2011 13:04:47 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Thu, 27 Jan 2011 13:04:47 +0100 From: Paul Schenkeveld To: freebsd-jail@freebsd.org, freebsd-hackers@freebsd.org Message-ID: <20110127120447.GA40060@psconsult.nl> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="SLDf9lqlvOQaIe6s" Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) Cc: Subject: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 12:04:55 -0000 --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, The order in which jails are started by rc.d/jail is the order in which jails are listed in $jail_list which is fine. On shutdown, jails are stopped in the same order they were started which in some cases is not fine. If jail B depends on functionality provided by jail A, one would like to start A before B but shutdown B before A. Would it make sense to reverse the order in which jails are stopped during shutdown by reversing the nales in $jail_list? The attached patch reverses $jail_list during shutdown. Regards, Paul Schenkeveld --- etc/rc.d/jail.orig 2009-08-15 14:00:54.000000000 +0200 +++ etc/rc.d/jail 2011-01-27 13:03:17.000000000 +0100 @@ -678,7 +678,7 @@ jail_stop() { echo -n 'Stopping jails:' - for _jail in ${jail_list} + for _jail in `reverse_list ${jail_list}` do if [ -f "/var/run/jail_${_jail}.id" ]; then _jail_id=$(cat /var/run/jail_${_jail}.id) --SLDf9lqlvOQaIe6s Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="rc_d_jail.patch" --- etc/rc.d/jail.orig 2009-08-15 14:00:54.000000000 +0200 +++ etc/rc.d/jail 2011-01-27 13:03:17.000000000 +0100 @@ -678,7 +678,7 @@ jail_stop() { echo -n 'Stopping jails:' - for _jail in ${jail_list} + for _jail in `reverse_list ${jail_list}` do if [ -f "/var/run/jail_${_jail}.id" ]; then _jail_id=$(cat /var/run/jail_${_jail}.id) --SLDf9lqlvOQaIe6s-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 16:08:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D659106566B for ; Thu, 27 Jan 2011 16:08:49 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [217.13.206.130]) by mx1.freebsd.org (Postfix) with ESMTP id 63F358FC1B for ; Thu, 27 Jan 2011 16:08:47 +0000 (UTC) Received: (qmail 25010 invoked from network); 27 Jan 2011 15:42:06 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with AES256-SHA encrypted SMTP; 27 Jan 2011 15:42:06 -0000 Date: Thu, 27 Jan 2011 16:42:05 +0100 (CET) From: Dirk Engling To: Paul Schenkeveld In-Reply-To: <20110127120447.GA40060@psconsult.nl> Message-ID: References: <20110127120447.GA40060@psconsult.nl> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org, freebsd-jail@freebsd.org Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 16:08:49 -0000 On Thu, 27 Jan 2011, Paul Schenkeveld wrote: > like to start A before B but shutdown B before A. Would it make sense > to reverse the order in which jails are stopped during shutdown by > reversing the nales in $jail_list? Yikes, it does indeed make sense, that's why I already do it in the ezjail utility, besides having jail dependencies worked out by rcorder... Now, if /etc/rc.d/jail starts to reverse the list ezjail reverses before, I'm in trouble. Maybe it's time to think about moving some more jail abstraction - including jail dependencies - to the base system. erdgeist From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 17:18:26 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 860A11065672 for ; Thu, 27 Jan 2011 17:18:26 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [217.13.206.130]) by mx1.freebsd.org (Postfix) with ESMTP id C14278FC21 for ; Thu, 27 Jan 2011 17:18:25 +0000 (UTC) Received: (qmail 48817 invoked from network); 27 Jan 2011 17:18:24 -0000 Received: from elektropost.org (HELO elektropost.org) (erdgeist@erdgeist.org) by elektropost.org with AES256-SHA encrypted SMTP; 27 Jan 2011 17:18:24 -0000 Date: Thu, 27 Jan 2011 18:18:21 +0100 (CET) From: Dirk Engling To: Jamie Gritton In-Reply-To: <4D41A65C.70204@FreeBSD.org> Message-ID: References: <20110127120447.GA40060@psconsult.nl> <4D41A65C.70204@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@FreeBSD.org, freebsd-jail@FreeBSD.org, Paul Schenkeveld Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 17:18:26 -0000 On Thu, 27 Jan 2011, Jamie Gritton wrote: > That's where it's headed. I've been slow on progress lately, but I'm > working on a jail(8) that takes a config file instead of rc shell > variables, and takes care of dependency issues among other things. Looks like I completely missed the discussion that went on in June this year. If I can be of any assistance working on a new jail(8), I'd be too happy to know how I can help. Is the progress visible somewhere? erdgeist From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 17:37:30 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A5891065675; Thu, 27 Jan 2011 17:37:30 +0000 (UTC) (envelope-from jamie@FreeBSD.org) Received: from gritton.org (gritton.org [208.92.232.93]) by mx1.freebsd.org (Postfix) with ESMTP id 252728FC34; Thu, 27 Jan 2011 17:37:29 +0000 (UTC) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.14.3/8.14.3) with ESMTP id p0RHbRhJ013772; Thu, 27 Jan 2011 10:37:27 -0700 (MST) (envelope-from jamie@FreeBSD.org) Message-ID: <4D41AD52.6030007@FreeBSD.org> Date: Thu, 27 Jan 2011 10:37:22 -0700 From: Jamie Gritton User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20110107 Thunderbird/3.1.6 MIME-Version: 1.0 To: Dirk Engling References: <20110127120447.GA40060@psconsult.nl> <4D41A65C.70204@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, freebsd-jail@FreeBSD.org Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 17:37:30 -0000 It's in the the subversion tree, under projects/jailconf. I've got the dependency stuff there (actually turns out to be a large chunk of the code). I've got it doing almost everything that rc.d/jail does now, though it doesn't yet handle errors like it should. After I get that fixed up, I plan on putting it in HEAD. After that, I still have a todo list mostly of suggestions from others. Feel free to give me any "todo" suggestions, or any other feedback :-). - Jamie On 01/27/11 10:18, Dirk Engling wrote: > On Thu, 27 Jan 2011, Jamie Gritton wrote: > >> That's where it's headed. I've been slow on progress lately, but I'm >> working on a jail(8) that takes a config file instead of rc shell >> variables, and takes care of dependency issues among other things. > > Looks like I completely missed the discussion that went on in June this > year. If I can be of any assistance working on a new jail(8), I'd be too > happy to know how I can help. Is the progress visible somewhere? From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 17:38:14 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB86B1065675 for ; Thu, 27 Jan 2011 17:38:14 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-yi0-f54.google.com (mail-yi0-f54.google.com [209.85.218.54]) by mx1.freebsd.org (Postfix) with ESMTP id 83F628FC14 for ; Thu, 27 Jan 2011 17:38:14 +0000 (UTC) Received: by yie19 with SMTP id 19so736302yie.13 for ; Thu, 27 Jan 2011 09:38:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.223.73 with SMTP id ij9mr2458468icb.442.1296149874438; Thu, 27 Jan 2011 09:37:54 -0800 (PST) Received: by 10.42.8.147 with HTTP; Thu, 27 Jan 2011 09:37:54 -0800 (PST) X-Originating-IP: [216.223.13.183] In-Reply-To: <201101271305.21510.naylor.b.david@gmail.com> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> Date: Thu, 27 Jan 2011 12:37:54 -0500 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 17:38:14 -0000 On Thu, Jan 27, 2011 at 6:05 AM, David Naylor wr= ote: > On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: >> On Tue, 25 Jan 2011 21:40:42 -0500 >> >> Mark Saad wrote: >> > Hello Hackers >> > >> > The NetBSD folks have a nice improvement with the rtld-elf subsystem, >> > known as "Negative Symbol Cache" . >> > >> > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative >> > >> > =C2=A0Roy Marples roy@ has a simple write up of the change. >> > >> > I took the basic idea from FreeBSD, but improved the performance >> > drastically. Basically, the huge win is by caching both breadth and >> > depth of the needed/weak symbol lookup. >> > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row >> > where we cache both rows and columns. >> > >> > Has anyone looked into porting the changes back to FreeBSD ? =C2=A0The >> > improvement on load time for things like firefox, openoffice, and java >> > is huge on NetBSD. It looks like this change could improve load times >> > on FreeBSD in the same ways. >> >> This is a second time someone posts this to public mailing list and >> curiously enough is a second time it suggested that someone else is to >> do the investigation. From the quick look, the commit in question is >> more or less a direct rip-off of Donelists we had for ages and as >> such is completely over-hyped. The only extra quirk that said commit >> does is an optimization of a dlsym() call, which is hardly ever in >> critical performance path. Said optimization is trivial and easy to >> try. Here you have it: >> http://people.freebsd.org/~kan/rtld-symlook-depth.diff >> >> Since it only applies to dlsym, it only affects programs that are heavy >> plugin users, which I suppose is the category OpenOffice and firefox >> both fall into. Care to do some benchmarks with and without the >> patch and report the results? I frankly doubt that you'll see any >> noticeable difference compared to our stock rtld's performance. > > I benchmarked the impact said patch has on the boot-time of my system. = =C2=A0I > timed the boot-time to when KDE launches autostart programs and once all > programs have loaded (I run a few extra programs, such as amarok). =C2=A0= The latter > measure requires human action thus it has extra, human, variance in its > measure. > > I tried an older version of rtld (about 2 months old), current version of= rtld > and the new (patched) rtld. =C2=A0I ran each test three times. =C2=A0Ther= e was little > variance in the tests and I am confident that there is no difference betw= een > the different rtld versions and my boot-time. > > Here is a summary of my boot times (in seconds). =C2=A0First measure is w= hen KDE > autostarts programs, the latter is when I determined when all programs ha= d > launched. > rtld-old: 69 96 > rtld: =C2=A0 =C2=A0 69 94 > rtld-new: 69 94 > > Please note that kernel boot time is approximately 10 seconds and kdm is > delayed by about 10 seconds thus 20 seconds can be removed from above num= bers > to determine non-kernel boot wall-time. > > I would like to add that the blog entry claims a substantial improvement = for > some use cases. =C2=A0Is it not worth to optimism these fringe cases as o= ne mans > fringe case is another mans normal case (or woman as one prefers)? > So I figured out how to properly fit my foot in my mouth and set out to retesting this on netbsd. Turns out that in most cases the speed up is not as dramatic. Firefox 3.6.16 on amd64 old ld.elf_so: 4.07 seconds new ld.elf_so: 3.89 seconds Openoffice 3.1 on amd64 old ld.elf_so: 2.67 seconds new ld.elf_so: 2.60 seconds I am slightly perturbed that I can start openoffice faster then I can start firefox, oh well. --=20 mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 17:43:28 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BE341065679; Thu, 27 Jan 2011 17:43:28 +0000 (UTC) (envelope-from jamie@FreeBSD.org) Received: from gritton.org (gritton.org [208.92.232.93]) by mx1.freebsd.org (Postfix) with ESMTP id 42F7D8FC1C; Thu, 27 Jan 2011 17:43:27 +0000 (UTC) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.14.3/8.14.3) with ESMTP id p0RH7jZn013600; Thu, 27 Jan 2011 10:07:46 -0700 (MST) (envelope-from jamie@FreeBSD.org) Message-ID: <4D41A65C.70204@FreeBSD.org> Date: Thu, 27 Jan 2011 10:07:40 -0700 From: Jamie Gritton User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20110107 Thunderbird/3.1.6 MIME-Version: 1.0 To: Dirk Engling References: <20110127120447.GA40060@psconsult.nl> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, freebsd-jail@FreeBSD.org, Paul Schenkeveld Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 17:43:28 -0000 That's where it's headed. I've been slow on progress lately, but I'm working on a jail(8) that takes a config file instead of rc shell variables, and takes care of dependency issues among other things. - Jamie On 01/27/11 08:42, Dirk Engling wrote: > On Thu, 27 Jan 2011, Paul Schenkeveld wrote: > >> like to start A before B but shutdown B before A. Would it make sense >> to reverse the order in which jails are stopped during shutdown by >> reversing the nales in $jail_list? > > Yikes, it does indeed make sense, that's why I already do it in the > ezjail utility, besides having jail dependencies worked out by rcorder... > > Now, if /etc/rc.d/jail starts to reverse the list ezjail reverses > before, I'm in trouble. Maybe it's time to think about moving some more > jail abstraction - including jail dependencies - to the base system. From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 20:30:21 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E336F1065694 for ; Thu, 27 Jan 2011 20:30:21 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 45E0A8FC20 for ; Thu, 27 Jan 2011 20:30:20 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p0RKUFZX095568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Jan 2011 22:30:15 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p0RKUF3m053472; Thu, 27 Jan 2011 22:30:15 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p0RKUFOD053471; Thu, 27 Jan 2011 22:30:15 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 27 Jan 2011 22:30:15 +0200 From: Kostik Belousov To: Jamie Gritton Message-ID: <20110127203015.GM2518@deviant.kiev.zoral.com.ua> References: <20110127120447.GA40060@psconsult.nl> <4D41A65C.70204@FreeBSD.org> <4D41AD52.6030007@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wYxcee34aMWaJ3Dn" Content-Disposition: inline In-Reply-To: <4D41AD52.6030007@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Dirk Engling , freebsd-jail@freebsd.org Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 20:30:22 -0000 --wYxcee34aMWaJ3Dn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 27, 2011 at 10:37:22AM -0700, Jamie Gritton wrote: > It's in the the subversion tree, under projects/jailconf. >=20 > I've got the dependency stuff there (actually turns out to be a large > chunk of the code). I've got it doing almost everything that rc.d/jail > does now, though it doesn't yet handle errors like it should. After I > get that fixed up, I plan on putting it in HEAD. After that, I still > have a todo list mostly of suggestions from others. >=20 > Feel free to give me any "todo" suggestions, or any other feedback :-). Are per-jail init and console in the list ? --wYxcee34aMWaJ3Dn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1B1dcACgkQC3+MBN1Mb4igdwCffCk7kdUOY/mMVSRVO7BCoCEf zecAnRk69arJ45oTNBm13w67m+ChCaml =MFrp -----END PGP SIGNATURE----- --wYxcee34aMWaJ3Dn-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 20:31:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D46A106566C for ; Thu, 27 Jan 2011 20:31:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1C5C18FC13 for ; Thu, 27 Jan 2011 20:31:30 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p0RKVQml095612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Jan 2011 22:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p0RKVQQ5053485; Thu, 27 Jan 2011 22:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p0RKVQSP053484; Thu, 27 Jan 2011 22:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 27 Jan 2011 22:31:26 +0200 From: Kostik Belousov To: Mark Saad Message-ID: <20110127203126.GN2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lMeYSPNN/8081OSC" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 20:31:31 -0000 --lMeYSPNN/8081OSC Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 27, 2011 at 12:37:54PM -0500, Mark Saad wrote: > On Thu, Jan 27, 2011 at 6:05 AM, David Naylor = wrote: > > On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: > >> On Tue, 25 Jan 2011 21:40:42 -0500 > >> > >> Mark Saad wrote: > >> > Hello Hackers > >> > > >> > The NetBSD folks have a nice improvement with the rtld-elf subsystem, > >> > known as "Negative Symbol Cache" . > >> > > >> > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative > >> > > >> > =9ARoy Marples roy@ has a simple write up of the change. > >> > > >> > I took the basic idea from FreeBSD, but improved the performance > >> > drastically. Basically, the huge win is by caching both breadth and > >> > depth of the needed/weak symbol lookup. > >> > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row > >> > where we cache both rows and columns. > >> > > >> > Has anyone looked into porting the changes back to FreeBSD ? =9AThe > >> > improvement on load time for things like firefox, openoffice, and ja= va > >> > is huge on NetBSD. It looks like this change could improve load times > >> > on FreeBSD in the same ways. > >> > >> This is a second time someone posts this to public mailing list and > >> curiously enough is a second time it suggested that someone else is to > >> do the investigation. From the quick look, the commit in question is > >> more or less a direct rip-off of Donelists we had for ages and as > >> such is completely over-hyped. The only extra quirk that said commit > >> does is an optimization of a dlsym() call, which is hardly ever in > >> critical performance path. Said optimization is trivial and easy to > >> try. Here you have it: > >> http://people.freebsd.org/~kan/rtld-symlook-depth.diff > >> > >> Since it only applies to dlsym, it only affects programs that are heavy > >> plugin users, which I suppose is the category OpenOffice and firefox > >> both fall into. Care to do some benchmarks with and without the > >> patch and report the results? I frankly doubt that you'll see any > >> noticeable difference compared to our stock rtld's performance. > > > > I benchmarked the impact said patch has on the boot-time of my system. = =9AI > > timed the boot-time to when KDE launches autostart programs and once all > > programs have loaded (I run a few extra programs, such as amarok). =9AT= he latter > > measure requires human action thus it has extra, human, variance in its > > measure. > > > > I tried an older version of rtld (about 2 months old), current version = of rtld > > and the new (patched) rtld. =9AI ran each test three times. =9AThere wa= s little > > variance in the tests and I am confident that there is no difference be= tween > > the different rtld versions and my boot-time. > > > > Here is a summary of my boot times (in seconds). =9AFirst measure is wh= en KDE > > autostarts programs, the latter is when I determined when all programs = had > > launched. > > rtld-old: 69 96 > > rtld: =9A =9A 69 94 > > rtld-new: 69 94 > > > > Please note that kernel boot time is approximately 10 seconds and kdm is > > delayed by about 10 seconds thus 20 seconds can be removed from above n= umbers > > to determine non-kernel boot wall-time. > > > > I would like to add that the blog entry claims a substantial improvemen= t for > > some use cases. =9AIs it not worth to optimism these fringe cases as on= e mans > > fringe case is another mans normal case (or woman as one prefers)? > > >=20 >=20 > So I figured out how to properly fit my foot in my mouth and set out > to retesting this on netbsd. > Turns out that in most cases the speed up is not as dramatic. >=20 > Firefox 3.6.16 on amd64 >=20 > old ld.elf_so: 4.07 seconds > new ld.elf_so: 3.89 seconds >=20 > Openoffice 3.1 on amd64 >=20 > old ld.elf_so: 2.67 seconds > new ld.elf_so: 2.60 seconds >=20 > I am slightly perturbed that I can start openoffice faster then I can > start firefox, oh well. Can you, please, satisfy my curiousity ? How did you fixated the moment of finishing the startup of interactive applications like ff or oo ? --lMeYSPNN/8081OSC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1B1h4ACgkQC3+MBN1Mb4hnkACgzH58bv40XmpiNaeSNNNPLKt0 cqMAoK64gtTp8y/p/rP74lIIWhl51I7l =0ctD -----END PGP SIGNATURE----- --lMeYSPNN/8081OSC-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 20:59:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1494D10656A3 for ; Thu, 27 Jan 2011 20:59:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5C7968FC1A for ; Thu, 27 Jan 2011 20:59:11 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p0RKx7We097176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Jan 2011 22:59:07 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p0RKx7Cv053668; Thu, 27 Jan 2011 22:59:07 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p0RKx70M053667; Thu, 27 Jan 2011 22:59:07 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 27 Jan 2011 22:59:07 +0200 From: Kostik Belousov To: Devin Teske Message-ID: <20110127205907.GP2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lvRZZFMRNRLg22E3" Content-Disposition: inline In-Reply-To: <1296161448.20060.40.camel@dt.vicor.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 20:59:12 -0000 --lvRZZFMRNRLg22E3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: > On Thu, 2011-01-27 at 22:31 +0200, Kostik Belousov wrote: >=20 > > On Thu, Jan 27, 2011 at 12:37:54PM -0500, Mark Saad wrote: > > > On Thu, Jan 27, 2011 at 6:05 AM, David Naylor wrote: > > > > On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: > > > >> On Tue, 25 Jan 2011 21:40:42 -0500 > > > >> > > > >> Mark Saad wrote: > > > >> > Hello Hackers > > > >> > > > > >> > The NetBSD folks have a nice improvement with the rtld-elf subsy= stem, > > > >> > known as "Negative Symbol Cache" . > > > >> > > > > >> > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_neg= ative > > > >> > > > > >> > Roy Marples roy@ has a simple write up of the change. > > > >> > > > > >> > I took the basic idea from FreeBSD, but improved the performance > > > >> > drastically. Basically, the huge win is by caching both breadth = and > > > >> > depth of the needed/weak symbol lookup. > > > >> > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a= row > > > >> > where we cache both rows and columns. > > > >> > > > > >> > Has anyone looked into porting the changes back to FreeBSD ? The > > > >> > improvement on load time for things like firefox, openoffice, an= d java > > > >> > is huge on NetBSD. It looks like this change could improve load = times > > > >> > on FreeBSD in the same ways. > > > >> > > > >> This is a second time someone posts this to public mailing list and > > > >> curiously enough is a second time it suggested that someone else i= s to > > > >> do the investigation. From the quick look, the commit in question = is > > > >> more or less a direct rip-off of Donelists we had for ages and as > > > >> such is completely over-hyped. The only extra quirk that said comm= it > > > >> does is an optimization of a dlsym() call, which is hardly ever in > > > >> critical performance path. Said optimization is trivial and easy to > > > >> try. Here you have it: > > > >> http://people.freebsd.org/~kan/rtld-symlook-depth.diff > > > >> > > > >> Since it only applies to dlsym, it only affects programs that are = heavy > > > >> plugin users, which I suppose is the category OpenOffice and firef= ox > > > >> both fall into. Care to do some benchmarks with and without the > > > >> patch and report the results? I frankly doubt that you'll see any > > > >> noticeable difference compared to our stock rtld's performance. > > > > > > > > I benchmarked the impact said patch has on the boot-time of my syst= em. I > > > > timed the boot-time to when KDE launches autostart programs and onc= e all > > > > programs have loaded (I run a few extra programs, such as amarok). = The latter > > > > measure requires human action thus it has extra, human, variance in= its > > > > measure. > > > > > > > > I tried an older version of rtld (about 2 months old), current vers= ion of rtld > > > > and the new (patched) rtld. I ran each test three times. There wa= s little > > > > variance in the tests and I am confident that there is no differenc= e between > > > > the different rtld versions and my boot-time. > > > > > > > > Here is a summary of my boot times (in seconds). First measure is = when KDE > > > > autostarts programs, the latter is when I determined when all progr= ams had > > > > launched. > > > > rtld-old: 69 96 > > > > rtld: 69 94 > > > > rtld-new: 69 94 > > > > > > > > Please note that kernel boot time is approximately 10 seconds and k= dm is > > > > delayed by about 10 seconds thus 20 seconds can be removed from abo= ve numbers > > > > to determine non-kernel boot wall-time. > > > > > > > > I would like to add that the blog entry claims a substantial improv= ement for > > > > some use cases. Is it not worth to optimism these fringe cases as = one mans > > > > fringe case is another mans normal case (or woman as one prefers)? > > > > > > >=20 > > >=20 > > > So I figured out how to properly fit my foot in my mouth and set out > > > to retesting this on netbsd. > > > Turns out that in most cases the speed up is not as dramatic. > > >=20 > > > Firefox 3.6.16 on amd64 > > >=20 > > > old ld.elf_so: 4.07 seconds > > > new ld.elf_so: 3.89 seconds > > >=20 > > > Openoffice 3.1 on amd64 > > >=20 > > > old ld.elf_so: 2.67 seconds > > > new ld.elf_so: 2.60 seconds > > >=20 > > > I am slightly perturbed that I can start openoffice faster then I can > > > start firefox, oh well. > >=20 > > Can you, please, satisfy my curiousity ? How did you fixated the moment > > of finishing the startup of interactive applications like ff or oo ? >=20 >=20 > Probably did something like this: >=20 > time sh -c '( firefox & ); sleep 10000000' >=20 > and then pressed Ctrl-C when he felt that firefox was finished loading. > The moment Ctrl-C is pressed, time(1) shows how long it ran up until you > pressed Ctrl-C. > NOTE: Pressing Ctrl-C will not terminate the firefox instance. You cannot have 1/100 of seconds precision with this method. This is why I am asking, seeing < 0.1 seconds difference. Not to mention some methodical questions, like whether the caches were warmed before the measurement by several runs before the actual test. --lvRZZFMRNRLg22E3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1B3JoACgkQC3+MBN1Mb4hgKgCfd3W6bVufyXRpyxcBhCrfjLu9 B2oAoOsQrBPZ2G5Q/DhoqXyisgSkvRL5 =NGuJ -----END PGP SIGNATURE----- --lvRZZFMRNRLg22E3-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:12:28 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 228DE1065670 for ; Thu, 27 Jan 2011 21:12:28 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 059BC8FC1D for ; Thu, 27 Jan 2011 21:12:27 +0000 (UTC) Received: from [192.82.228.132] (port=51146) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1PiZ8i-0008Ku-W1; Thu, 27 Jan 2011 13:12:27 -0800 From: Devin Teske To: Kostik Belousov In-Reply-To: <20110127205907.GP2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> Organization: VICOR, Inc. Date: Thu, 27 Jan 2011 21:12:34 +0000 Message-ID: <1296162754.20060.42.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 8b29faa9f9de30d9292f37256ba111e5 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:12:28 -0000 On Thu, 2011-01-27 at 22:59 +0200, Kostik Belousov wrote: > On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: > > On Thu, 2011-01-27 at 22:31 +0200, Kostik Belousov wrote: > > > > > On Thu, Jan 27, 2011 at 12:37:54PM -0500, Mark Saad wrote: > > > > On Thu, Jan 27, 2011 at 6:05 AM, David Naylor wrote: > > > > > On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: > > > > >> On Tue, 25 Jan 2011 21:40:42 -0500 > > > > >> > > > > >> Mark Saad wrote: > > > > >> > Hello Hackers > > > > >> > > > > > >> > The NetBSD folks have a nice improvement with the rtld-elf subsystem, > > > > >> > known as "Negative Symbol Cache" . > > > > >> > > > > > >> > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative > > > > >> > > > > > >> > Roy Marples roy@ has a simple write up of the change. > > > > >> > > > > > >> > I took the basic idea from FreeBSD, but improved the performance > > > > >> > drastically. Basically, the huge win is by caching both breadth and > > > > >> > depth of the needed/weak symbol lookup. > > > > >> > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row > > > > >> > where we cache both rows and columns. > > > > >> > > > > > >> > Has anyone looked into porting the changes back to FreeBSD ? The > > > > >> > improvement on load time for things like firefox, openoffice, and java > > > > >> > is huge on NetBSD. It looks like this change could improve load times > > > > >> > on FreeBSD in the same ways. > > > > >> > > > > >> This is a second time someone posts this to public mailing list and > > > > >> curiously enough is a second time it suggested that someone else is to > > > > >> do the investigation. From the quick look, the commit in question is > > > > >> more or less a direct rip-off of Donelists we had for ages and as > > > > >> such is completely over-hyped. The only extra quirk that said commit > > > > >> does is an optimization of a dlsym() call, which is hardly ever in > > > > >> critical performance path. Said optimization is trivial and easy to > > > > >> try. Here you have it: > > > > >> http://people.freebsd.org/~kan/rtld-symlook-depth.diff > > > > >> > > > > >> Since it only applies to dlsym, it only affects programs that are heavy > > > > >> plugin users, which I suppose is the category OpenOffice and firefox > > > > >> both fall into. Care to do some benchmarks with and without the > > > > >> patch and report the results? I frankly doubt that you'll see any > > > > >> noticeable difference compared to our stock rtld's performance. > > > > > > > > > > I benchmarked the impact said patch has on the boot-time of my system. I > > > > > timed the boot-time to when KDE launches autostart programs and once all > > > > > programs have loaded (I run a few extra programs, such as amarok). The latter > > > > > measure requires human action thus it has extra, human, variance in its > > > > > measure. > > > > > > > > > > I tried an older version of rtld (about 2 months old), current version of rtld > > > > > and the new (patched) rtld. I ran each test three times. There was little > > > > > variance in the tests and I am confident that there is no difference between > > > > > the different rtld versions and my boot-time. > > > > > > > > > > Here is a summary of my boot times (in seconds). First measure is when KDE > > > > > autostarts programs, the latter is when I determined when all programs had > > > > > launched. > > > > > rtld-old: 69 96 > > > > > rtld: 69 94 > > > > > rtld-new: 69 94 > > > > > > > > > > Please note that kernel boot time is approximately 10 seconds and kdm is > > > > > delayed by about 10 seconds thus 20 seconds can be removed from above numbers > > > > > to determine non-kernel boot wall-time. > > > > > > > > > > I would like to add that the blog entry claims a substantial improvement for > > > > > some use cases. Is it not worth to optimism these fringe cases as one mans > > > > > fringe case is another mans normal case (or woman as one prefers)? > > > > > > > > > > > > > > > > > So I figured out how to properly fit my foot in my mouth and set out > > > > to retesting this on netbsd. > > > > Turns out that in most cases the speed up is not as dramatic. > > > > > > > > Firefox 3.6.16 on amd64 > > > > > > > > old ld.elf_so: 4.07 seconds > > > > new ld.elf_so: 3.89 seconds > > > > > > > > Openoffice 3.1 on amd64 > > > > > > > > old ld.elf_so: 2.67 seconds > > > > new ld.elf_so: 2.60 seconds > > > > > > > > I am slightly perturbed that I can start openoffice faster then I can > > > > start firefox, oh well. > > > > > > Can you, please, satisfy my curiousity ? How did you fixated the moment > > > of finishing the startup of interactive applications like ff or oo ? > > > > > > Probably did something like this: > > > > time sh -c '( firefox & ); sleep 10000000' > > > > and then pressed Ctrl-C when he felt that firefox was finished loading. > > The moment Ctrl-C is pressed, time(1) shows how long it ran up until you > > pressed Ctrl-C. > > NOTE: Pressing Ctrl-C will not terminate the firefox instance. > > You cannot have 1/100 of seconds precision with this method. > This is why I am asking, seeing < 0.1 seconds difference. > Not to mention some methodical questions, like whether the caches were > warmed before the measurement by several runs before the actual > test. Really? $ time sh -c '( firefox & ); sleep 10000000' ^C real 0m5.270s user 0m0.000s sys 0m0.005s I'd call that 1/100th of a second precision, wouldn't you? HINT: Try using bash instead of csh. -- Devin From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:14:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67D35106566C for ; Thu, 27 Jan 2011 21:14:01 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 452E38FC0A for ; Thu, 27 Jan 2011 21:14:01 +0000 (UTC) Received: from [192.82.228.132] (port=51054) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1PiYnf-00059u-7m; Thu, 27 Jan 2011 12:50:41 -0800 From: Devin Teske To: Kostik Belousov In-Reply-To: <20110127203126.GN2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> Organization: VICOR, Inc. Date: Thu, 27 Jan 2011 20:50:48 +0000 Message-ID: <1296161448.20060.40.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 6872980f4047590eea166e185c613f25 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, Mark Saad , Teske , Devin Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:14:01 -0000 On Thu, 2011-01-27 at 22:31 +0200, Kostik Belousov wrote: > On Thu, Jan 27, 2011 at 12:37:54PM -0500, Mark Saad wrote: > > On Thu, Jan 27, 2011 at 6:05 AM, David Naylor wrote: > > > On Wednesday 26 January 2011 06:49:11 Alexander Kabaev wrote: > > >> On Tue, 25 Jan 2011 21:40:42 -0500 > > >> > > >> Mark Saad wrote: > > >> > Hello Hackers > > >> > > > >> > The NetBSD folks have a nice improvement with the rtld-elf subsystem, > > >> > known as "Negative Symbol Cache" . > > >> > > > >> > http://blog.netbsd.org/tnf/entry/netbsd_runtime_linker_gains_negative > > >> > > > >> > Roy Marples roy@ has a simple write up of the change. > > >> > > > >> > I took the basic idea from FreeBSD, but improved the performance > > >> > drastically. Basically, the huge win is by caching both breadth and > > >> > depth of the needed/weak symbol lookup. > > >> > Easiest to think of a,b,c,d as a matrix and FreeBSD just cache a row > > >> > where we cache both rows and columns. > > >> > > > >> > Has anyone looked into porting the changes back to FreeBSD ? The > > >> > improvement on load time for things like firefox, openoffice, and java > > >> > is huge on NetBSD. It looks like this change could improve load times > > >> > on FreeBSD in the same ways. > > >> > > >> This is a second time someone posts this to public mailing list and > > >> curiously enough is a second time it suggested that someone else is to > > >> do the investigation. From the quick look, the commit in question is > > >> more or less a direct rip-off of Donelists we had for ages and as > > >> such is completely over-hyped. The only extra quirk that said commit > > >> does is an optimization of a dlsym() call, which is hardly ever in > > >> critical performance path. Said optimization is trivial and easy to > > >> try. Here you have it: > > >> http://people.freebsd.org/~kan/rtld-symlook-depth.diff > > >> > > >> Since it only applies to dlsym, it only affects programs that are heavy > > >> plugin users, which I suppose is the category OpenOffice and firefox > > >> both fall into. Care to do some benchmarks with and without the > > >> patch and report the results? I frankly doubt that you'll see any > > >> noticeable difference compared to our stock rtld's performance. > > > > > > I benchmarked the impact said patch has on the boot-time of my system. I > > > timed the boot-time to when KDE launches autostart programs and once all > > > programs have loaded (I run a few extra programs, such as amarok). The latter > > > measure requires human action thus it has extra, human, variance in its > > > measure. > > > > > > I tried an older version of rtld (about 2 months old), current version of rtld > > > and the new (patched) rtld. I ran each test three times. There was little > > > variance in the tests and I am confident that there is no difference between > > > the different rtld versions and my boot-time. > > > > > > Here is a summary of my boot times (in seconds). First measure is when KDE > > > autostarts programs, the latter is when I determined when all programs had > > > launched. > > > rtld-old: 69 96 > > > rtld: 69 94 > > > rtld-new: 69 94 > > > > > > Please note that kernel boot time is approximately 10 seconds and kdm is > > > delayed by about 10 seconds thus 20 seconds can be removed from above numbers > > > to determine non-kernel boot wall-time. > > > > > > I would like to add that the blog entry claims a substantial improvement for > > > some use cases. Is it not worth to optimism these fringe cases as one mans > > > fringe case is another mans normal case (or woman as one prefers)? > > > > > > > > > So I figured out how to properly fit my foot in my mouth and set out > > to retesting this on netbsd. > > Turns out that in most cases the speed up is not as dramatic. > > > > Firefox 3.6.16 on amd64 > > > > old ld.elf_so: 4.07 seconds > > new ld.elf_so: 3.89 seconds > > > > Openoffice 3.1 on amd64 > > > > old ld.elf_so: 2.67 seconds > > new ld.elf_so: 2.60 seconds > > > > I am slightly perturbed that I can start openoffice faster then I can > > start firefox, oh well. > > Can you, please, satisfy my curiousity ? How did you fixated the moment > of finishing the startup of interactive applications like ff or oo ? Probably did something like this: time sh -c '( firefox & ); sleep 10000000' and then pressed Ctrl-C when he felt that firefox was finished loading. The moment Ctrl-C is pressed, time(1) shows how long it ran up until you pressed Ctrl-C. NOTE: Pressing Ctrl-C will not terminate the firefox instance. -- Devin From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:34:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BB9D106566B for ; Thu, 27 Jan 2011 21:34:51 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id B0D5D8FC08 for ; Thu, 27 Jan 2011 21:34:50 +0000 (UTC) Received: by ewy24 with SMTP id 24so1245390ewy.13 for ; Thu, 27 Jan 2011 13:34:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=BzZw8JTwhJ6fGrp8rK1DOd7ihctW9kjRr5ChI59hN5E=; b=sPY6fZ2qgpsqDKkQu9lJT1QU7aJccDJSqx69dbVbuEOkdEADibHe8FO3AVH1wKvrWw 4BLiMBEz6dpa1tuTVOHRoCl9jhNLcc0bZpb/twbn9Vj7zcg7F9inbXLImzBytPzA/k56 hyuhrUXs6pUGZsSdAuO8bntMFnmbxkVBjMc88= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=UUUNmaeo2cbH6X33P2kVsB/dbmCRslYxk4Mwwg0IUR9WvkdbGOOLUc9lI3Hr4Jadvb dofCcZE05GZyQ7e0+tuGBfDaAn2sBK/yhEOO12mlI9d9QxuJDFnORR5bspQj5R5dD7Vf xklis8uiXF7v8BLzGI40IOHgraRjvYnlLh4yE= MIME-Version: 1.0 Received: by 10.213.30.3 with SMTP id s3mr3798877ebc.22.1296164089450; Thu, 27 Jan 2011 13:34:49 -0800 (PST) Received: by 10.213.22.14 with HTTP; Thu, 27 Jan 2011 13:34:49 -0800 (PST) In-Reply-To: <1296162754.20060.42.camel@dt.vicor.com> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> Date: Thu, 27 Jan 2011 16:34:49 -0500 Message-ID: From: Ryan Stone To: Devin Teske Content-Type: text/plain; charset=ISO-8859-1 Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:34:51 -0000 > I'd call that 1/100th of a second precision, wouldn't you? You have much better reflexes than I do. :) From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:35:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E725106566C for ; Thu, 27 Jan 2011 21:35:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 11A118FC21 for ; Thu, 27 Jan 2011 21:35:40 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p0RLZaLO099304 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 27 Jan 2011 23:35:36 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p0RLZaPH053875; Thu, 27 Jan 2011 23:35:36 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p0RLZasv053874; Thu, 27 Jan 2011 23:35:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 27 Jan 2011 23:35:36 +0200 From: Kostik Belousov To: Devin Teske Message-ID: <20110127213536.GR2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="PI2T0JrbAUafY2qi" Content-Disposition: inline In-Reply-To: <1296162754.20060.42.camel@dt.vicor.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:35:41 -0000 --PI2T0JrbAUafY2qi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 27, 2011 at 09:12:34PM +0000, Devin Teske wrote: > On Thu, 2011-01-27 at 22:59 +0200, Kostik Belousov wrote: >=20 > > On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: > > > Probably did something like this: > > >=20 > > > time sh -c '( firefox & ); sleep 10000000' > > >=20 > > > and then pressed Ctrl-C when he felt that firefox was finished loadin= g. > > > The moment Ctrl-C is pressed, time(1) shows how long it ran up until = you > > > pressed Ctrl-C. > > > NOTE: Pressing Ctrl-C will not terminate the firefox instance. > >=20 > > You cannot have 1/100 of seconds precision with this method. > > This is why I am asking, seeing < 0.1 seconds difference. > > Not to mention some methodical questions, like whether the caches were > > warmed before the measurement by several runs before the actual > > test. >=20 >=20 > Really? >=20 > $ time sh -c '( firefox & ); sleep 10000000' > ^C >=20 > real 0m5.270s > user 0m0.000s > sys 0m0.005s >=20 >=20 > I'd call that 1/100th of a second precision, wouldn't you? >=20 > HINT: Try using bash instead of csh. (I supposed that) obvious point of my mail is that you cannot reliably measure 1/100 second intervals when human interaction is involved. To make it completely obvious: human has to press CTRL-C, I did not mean reading the numbers from display. --PI2T0JrbAUafY2qi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1B5SgACgkQC3+MBN1Mb4iyEACcDA+e2npfczgWpJsikfv1yXNA hW4AmgI+O15s0tDj/nW8abxKYCsdbGOD =EDM/ -----END PGP SIGNATURE----- --PI2T0JrbAUafY2qi-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:49:16 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C792A1065693 for ; Thu, 27 Jan 2011 21:49:16 +0000 (UTC) (envelope-from geo.liaskos@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7EA198FC0C for ; Thu, 27 Jan 2011 21:49:16 +0000 (UTC) Received: by gxk8 with SMTP id 8so870483gxk.13 for ; Thu, 27 Jan 2011 13:49:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=GYGbRbjLtbItcAPv9aU3PckinQ0XvV0JTv8DbXppVbo=; b=kUOI4lQyIsSvYgVQ8/LFW1ywy9BpQ2jePBE2QvTok363A6f3asUGGeEWknotGc8o/w x2lVO7HE1q28BfJZANJihBgVbWvSpyikSXIqXgs1KqcjNvFbTAGsfWG521qBJiEM0l/r X1yrMEiMr6ikBSgcSiTYExq8EMYBsM08rMrl0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=FzIAxfTEpc1RxY+IhFuxczWrNb6ncXjZZRrxrlrLajJlI20Ph3nDGQ+ZHC3OU5qv3c 761lzmBOomzeplMDZiHIYbfpnAKUaH4vf4bCAYbBt58bVRW7SC6C3tJEUe/YVKLLxaE6 HyFCJJyyFCqbQmlhGFmPwxPNQa/Kn+VNvxBRs= MIME-Version: 1.0 Received: by 10.100.106.18 with SMTP id e18mr950695anc.80.1296163574840; Thu, 27 Jan 2011 13:26:14 -0800 (PST) Received: by 10.101.133.9 with HTTP; Thu, 27 Jan 2011 13:26:14 -0800 (PST) In-Reply-To: <20110127205907.GP2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> Date: Thu, 27 Jan 2011 23:26:14 +0200 Message-ID: From: George Liaskos To: Kostik Belousov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Mark Saad , Devin Teske Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:49:16 -0000 FYI, there is an API in Firefox 4 for start up time measurement. http://blog.mozilla.com/tglek/2011/01/14/builtin-startup-measurement/ From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 21:58:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42903106566C for ; Thu, 27 Jan 2011 21:58:57 +0000 (UTC) (envelope-from dteske@vicor.com) Received: from postoffice.vicor.com (postoffice.vicor.com [69.26.56.53]) by mx1.freebsd.org (Postfix) with ESMTP id 24EFA8FC12 for ; Thu, 27 Jan 2011 21:58:56 +0000 (UTC) Received: from [192.82.228.132] (port=51236) by postoffice.vicor.com with esmtpsa (SSLv3:AES256-SHA:256) (Exim 4.71) (envelope-from ) id 1PiZrd-00012D-Bg; Thu, 27 Jan 2011 13:58:53 -0800 From: Devin Teske To: Kostik Belousov In-Reply-To: <20110127213536.GR2518@deviant.kiev.zoral.com.ua> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> <20110127213536.GR2518@deviant.kiev.zoral.com.ua> Organization: VICOR, Inc. Date: Thu, 27 Jan 2011 21:58:58 +0000 Message-ID: <1296165538.20060.43.camel@dt.vicor.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 X-Scan-Signature: 20d467bac746d2f6d3c5e71211b3ea13 X-Scan-Host: postoffice.vicor.com Content-Type: text/plain; charset="cp1252" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, Mark Saad Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 21:58:57 -0000 On Thu, 2011-01-27 at 23:35 +0200, Kostik Belousov wrote: > On Thu, Jan 27, 2011 at 09:12:34PM +0000, Devin Teske wrote: > > On Thu, 2011-01-27 at 22:59 +0200, Kostik Belousov wrote: > > > > > On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: > > > > Probably did something like this: > > > > > > > > time sh -c '( firefox & ); sleep 10000000' > > > > > > > > and then pressed Ctrl-C when he felt that firefox was finished loading. > > > > The moment Ctrl-C is pressed, time(1) shows how long it ran up until you > > > > pressed Ctrl-C. > > > > NOTE: Pressing Ctrl-C will not terminate the firefox instance. > > > > > > You cannot have 1/100 of seconds precision with this method. > > > This is why I am asking, seeing < 0.1 seconds difference. > > > Not to mention some methodical questions, like whether the caches were > > > warmed before the measurement by several runs before the actual > > > test. > > > > > > Really? > > > > $ time sh -c '( firefox & ); sleep 10000000' > > ^C > > > > real 0m5.270s > > user 0m0.000s > > sys 0m0.005s > > > > > > I'd call that 1/100th of a second precision, wouldn't you? > > > > HINT: Try using bash instead of csh. > (I supposed that) obvious point of my mail is that you cannot reliably > measure 1/100 second intervals when human interaction is involved. > To make it completely obvious: human has to press CTRL-C, I did not > mean reading the numbers from display. I hypothesized that the results would always be subjected because we couldn't tell whether one person would consider "complete" to be "web page loaded" versus another person might consider "complete" to be when the window frame/decorations are drawn by the window manager. However, if Firefox provides a way of timing it in an objective manner... that would surely be superior to the method I've suggested which can lag by a few hundreds of even a few tenths of a second (depends on the subject's ability to quickly invoke focus-follows-mouse and press Ctrl-C). So yeah... my suggestion isn't perfect... but is generally "good enough" if the application you're testing doesn't provide a hook for determining when it is officially done loading. Now... here's an idea that might work (since we're talking about a web browser -- this obviously won't work for OpenOffice, though variations on the idea can be had): 1. Launch Firefox from the command-line with arguments to point it at a web page which uses JavaScript to print the current date and time in the web page 2. Execute: ps axo pid,state,command,lstart | grep firefox 3. Calculate the time difference between the "lstart" and the time printed in the browser. I do believe that should be sufficient to get accurate/objective results. NOTE: For OpenOffice, you could perhaps have the spreadsheet program open a spreadsheet with a macro to display the current date/time then use the lstart data from ps(1) to calculate the results. Similarly, you could have OpenOffice.org Writer open a document with a header-macro to display the current date/time. -- Devin From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 22:53:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95AFA106566C for ; Thu, 27 Jan 2011 22:53:50 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 652998FC1B for ; Thu, 27 Jan 2011 22:53:50 +0000 (UTC) Received: by iwn39 with SMTP id 39so2563173iwn.13 for ; Thu, 27 Jan 2011 14:53:49 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.177.137 with SMTP id bi9mr2971113icb.107.1296168829766; Thu, 27 Jan 2011 14:53:49 -0800 (PST) Received: by 10.42.8.147 with HTTP; Thu, 27 Jan 2011 14:53:49 -0800 (PST) X-Originating-IP: [209.66.78.50] In-Reply-To: <1296165538.20060.43.camel@dt.vicor.com> References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> <20110127213536.GR2518@deviant.kiev.zoral.com.ua> <1296165538.20060.43.camel@dt.vicor.com> Date: Thu, 27 Jan 2011 17:53:49 -0500 Message-ID: From: Mark Saad To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 22:53:50 -0000 On Thu, Jan 27, 2011 at 4:58 PM, Devin Teske wrote: > On Thu, 2011-01-27 at 23:35 +0200, Kostik Belousov wrote: > > On Thu, Jan 27, 2011 at 09:12:34PM +0000, Devin Teske wrote: >> On Thu, 2011-01-27 at 22:59 +0200, Kostik Belousov wrote: >> >> > On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: >> > > Probably did something like this: >> > > >> > > time sh -c '( firefox & ); sleep 10000000' >> > > >> > > and then pressed Ctrl-C when he felt that firefox was finished >> > > loading. >> > > The moment Ctrl-C is pressed, time(1) shows how long it ran up until >> > > you >> > > pressed Ctrl-C. >> > > NOTE: Pressing Ctrl-C will not terminate the firefox instance. >> > >> > You cannot have 1/100 of seconds precision with this method. >> > This is why I am asking, seeing < 0.1 seconds difference. >> > Not to mention some methodical questions, like whether the caches were >> > warmed before the measurement by several runs before the actual >> > test. >> >> >> Really? >> >> $ time sh -c '( firefox & ); sleep 10000000' >> ^C >> >> real 0m5.270s >> user 0m0.000s >> sys 0m0.005s >> >> >> I'd call that 1/100th of a second precision, wouldn't you? >> >> HINT: Try using bash instead of csh. > (I supposed that) obvious point of my mail is that you cannot reliably > measure 1/100 second intervals when human interaction is involved. > To make it completely obvious: human has to press CTRL-C, I did not > mean reading the numbers from display. > > I hypothesized that the results would always be subjected because we > couldn't tell whether one person would consider "complete" to be "web page > loaded" versus another person might consider "complete" to be when the > window frame/decorations are drawn by the window manager. > > However, if Firefox provides a way of timing it in an objective manner... > that would surely be superior to the method I've suggested which can lag by > a few hundreds of even a few tenths of a second (depends on the subject's > ability to quickly invoke focus-follows-mouse and press Ctrl-C). > > So yeah... my suggestion isn't perfect... but is generally "good enough" if > the application you're testing doesn't provide a hook for determining when > it is officially done loading. > > Now... here's an idea that might work (since we're talking about a web > browser -- this obviously won't work for OpenOffice, though variations on > the idea can be had): > > 1. Launch Firefox from the command-line with arguments to point it at a web > page which uses JavaScript to print the current date and time in the web > page > > 2. Execute: ps axo pid,state,command,lstart | grep firefox > > 3. Calculate the time difference between the "lstart" and the time printed > in the browser. > > I do believe that should be sufficient to get accurate/objective results. > > NOTE: For OpenOffice, you could perhaps have the spreadsheet program open a > spreadsheet with a macro to display the current date/time then use the > lstart data from ps(1) to calculate the results. Similarly, you could have > OpenOffice.org Writer open a document with a header-macro to display the > current date/time. > -- > Devin So for both I took the simplest approach . I spent to much time on looking for more exotic solutions. This seamed to be the easiest to run and somewhat os / platform agnostic. 1. time application " time firefox" 2. kill it when its fully loaded. * 3. record time 4. rerun test 10 times make an average to normalize my response time in closing the application. * I have the window manager open the window in the same place so I could be ready to hit the close widget. * For firefox I had it set to load a local page on startup * For openoffice I waited till it was at the "what do you want to do" window I know there is a lot of room here to debate how fast I can close and application, what's cached , how busy is the disk etc. I tried using mplayer but I did not get good results. My logic in using mplayer was that its linked to almost every audio / video lib you can name. Making it do some task, dump every frame into a jpg or remove the audio track from some video etc, should be simple to test without using a gui or requiring human input to stop it. I am stumped, at first I just was curious at first if there was any truth in the post, and wanted to know if anyone looked into it. Now I want to know how you test this correctly. -- mark saad | nonesuch@longcount.org From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 23:24:29 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40FC3106564A for ; Thu, 27 Jan 2011 23:24:29 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id D871E8FC0C for ; Thu, 27 Jan 2011 23:24:28 +0000 (UTC) Received: by qwj9 with SMTP id 9so2710341qwj.13 for ; Thu, 27 Jan 2011 15:24:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:x-mailer:mime-version:content-type; bh=C7V58JHL1cThqUC10vc8/jBPk7Vr7EbaEsVkeew04Ms=; b=xANVdZjnBEMG55vjhu238IY2ULFGrDttQpGnAcnG9Y1igu/OPywKJErXnUTkpy4ip9 Yp/0cwqYBARWy5qjBdXI9yS2DWDUdyECIGYBIKzLpiC/om1n9gb6/6YXJMe9JwkYJjmc 3n5e8Sij2ozjqS505GSmysTCb4pUpOoTwKhHc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=BxnjRZdGjz4ptP1ArmUA7HyHByNEPZXWHvY/1nlngAO+qEjjaj73Dv0XOJ12iljFzE QC9NP7nlJ1tW19WGNeQzQ4HE+umta2yNs0TdARvIEqsMifEFxCjbGrgQE7tGXW/AzIXg M5wmakpc1QKfpsb/XzPb2wfj6rssqaBJCmjO4= Received: by 10.229.233.196 with SMTP id jz4mr2116747qcb.135.1296170666735; Thu, 27 Jan 2011 15:24:26 -0800 (PST) Received: from kan.dnsalias.net (c-24-63-226-98.hsd1.ma.comcast.net [24.63.226.98]) by mx.google.com with ESMTPS id e29sm12182376qck.27.2011.01.27.15.24.24 (version=SSLv3 cipher=RC4-MD5); Thu, 27 Jan 2011 15:24:25 -0800 (PST) Date: Thu, 27 Jan 2011 18:24:18 -0500 From: Alexander Kabaev To: Mark Saad Message-ID: <20110127182418.77c53c60@kan.dnsalias.net> In-Reply-To: References: <20110125234911.223d8f75@kan.dnsalias.net> <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> <20110127213536.GR2518@deviant.kiev.zoral.com.ua> <1296165538.20060.43.camel@dt.vicor.com> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.22.1; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_//rdBBBf+zz=.dpEJjCVbXkU"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 23:24:29 -0000 --Sig_//rdBBBf+zz=.dpEJjCVbXkU Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Thu, 27 Jan 2011 17:53:49 -0500 Mark Saad wrote: > On Thu, Jan 27, 2011 at 4:58 PM, Devin Teske wrote: > > On Thu, 2011-01-27 at 23:35 +0200, Kostik Belousov wrote: > > > > On Thu, Jan 27, 2011 at 09:12:34PM +0000, Devin Teske wrote: > >> On Thu, 2011-01-27 at 22:59 +0200, Kostik Belousov wrote: > >> > >> > On Thu, Jan 27, 2011 at 08:50:48PM +0000, Devin Teske wrote: > >> > > Probably did something like this: > >> > > > >> > > time sh -c '( firefox & ); sleep 10000000' > >> > > > >> > > and then pressed Ctrl-C when he felt that firefox was finished > >> > > loading. > >> > > The moment Ctrl-C is pressed, time(1) shows how long it ran up > >> > > until you > >> > > pressed Ctrl-C. > >> > > NOTE: Pressing Ctrl-C will not terminate the firefox instance. > >> > > >> > You cannot have 1/100 of seconds precision with this method. > >> > This is why I am asking, seeing < 0.1 seconds difference. > >> > Not to mention some methodical questions, like whether the > >> > caches were warmed before the measurement by several runs before > >> > the actual test. > >> > >> > >> Really? > >> > >> $ time sh -c '( firefox & ); sleep 10000000' > >> ^C > >> > >> real 0m5.270s > >> user 0m0.000s > >> sys 0m0.005s > >> > >> > >> I'd call that 1/100th of a second precision, wouldn't you? > >> > >> HINT: Try using bash instead of csh. > > (I supposed that) obvious point of my mail is that you cannot > > reliably measure 1/100 second intervals when human interaction is > > involved. To make it completely obvious: human has to press CTRL-C, > > I did not mean reading the numbers from display. > > > > I hypothesized that the results would always be subjected because we > > couldn't tell whether one person would consider "complete" to be > > "web page loaded" versus another person might consider "complete" > > to be when the window frame/decorations are drawn by the window > > manager. > > > > However, if Firefox provides a way of timing it in an objective > > manner... that would surely be superior to the method I've > > suggested which can lag by a few hundreds of even a few tenths of a > > second (depends on the subject's ability to quickly invoke > > focus-follows-mouse and press Ctrl-C). > > > > So yeah... my suggestion isn't perfect... but is generally "good > > enough" if the application you're testing doesn't provide a hook > > for determining when it is officially done loading. > > > > Now... here's an idea that might work (since we're talking about a > > web browser -- this obviously won't work for OpenOffice, though > > variations on the idea can be had): > > > > 1. Launch Firefox from the command-line with arguments to point it > > at a web page which uses JavaScript to print the current date and > > time in the web page > > > > 2. Execute: ps axo pid,state,command,lstart | grep firefox > > > > 3. Calculate the time difference between the "lstart" and the time > > printed in the browser. > > > > I do believe that should be sufficient to get accurate/objective > > results. > > > > NOTE: For OpenOffice, you could perhaps have the spreadsheet > > program open a spreadsheet with a macro to display the current > > date/time then use the lstart data from ps(1) to calculate the > > results. Similarly, you could have OpenOffice.org Writer open a > > document with a header-macro to display the current date/time. > > -- > > Devin >=20 > So for both I took the simplest approach . I spent to much time on > looking for more exotic solutions. > This seamed to be the easiest to run and somewhat os / platform > agnostic. >=20 > 1. time application " time firefox" > 2. kill it when its fully loaded. * > 3. record time > 4. rerun test 10 times make an average to normalize my response time > in closing the application. >=20 > * I have the window manager open the window in the same place so I > could be ready to hit the close widget. > * For firefox I had it set to load a local page on startup > * For openoffice I waited till it was at the "what do you want to do" > window >=20 > I know there is a lot of room here to debate how fast I can close and > application, what's cached , how busy is the disk etc. I tried using > mplayer but I did not get good results. My logic in using mplayer was > that its linked to almost every audio / video lib you can name. Making > it do some task, dump every frame into a jpg or remove the audio track > from some video etc, should be simple to test without using a gui or > requiring human input to stop it. >=20 > I am stumped, at first I just was curious at first if there was any > truth in the post, and wanted to know if anyone looked into it. Now I > want to know how you test this correctly. >=20 For starters, the number of libraries given binary is linked too is completely and utterly irrelevant :) The change NetBSD guys claims to revolutionize his application startup times only applies to programs that dlopen (read - load dynamically) libraries with long largely identical dependency chains and calls dlsym on them many, many thousand times. I do not think you will find any real app out there that fits this description close enough to actually demonstrate the effect of the change that is distinguishable from the statistical noise. Pressing ^C certainly not precise enough for that. --=20 Alexander Kabaev --Sig_//rdBBBf+zz=.dpEJjCVbXkU Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iD8DBQFNQf6nQ6z1jMm+XZYRAsA6AJ94tgMPX0U+29kv654bmvp78Hvk8ACeO9+i waVZDwAmMgsYDws0V2ORYu0= =Ojtt -----END PGP SIGNATURE----- --Sig_//rdBBBf+zz=.dpEJjCVbXkU-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jan 27 23:47:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D945106566B for ; Thu, 27 Jan 2011 23:47:45 +0000 (UTC) (envelope-from joerg@britannica.bec.de) Received: from mo-p00-ob.rzone.de (mo-p00-ob.rzone.de [81.169.146.160]) by mx1.freebsd.org (Postfix) with ESMTP id 9562D8FC0A for ; Thu, 27 Jan 2011 23:47:44 +0000 (UTC) X-RZG-AUTH: :JiIXek6mfvEEUpFQdo7Fj1/zg48CFjWjQv0cW+St/nW/afgnrylsiWuobih/yw== X-RZG-CLASS-ID: mo00 Received: from britannica.bec.de (ip-109-45-5-133.web.vodafone.de [109.45.5.133]) by post.strato.de (jimi mo9) (RZmta 25.1) with (DHE-RSA-AES128-SHA encrypted) ESMTPA id m03859n0RKBcIc for ; Fri, 28 Jan 2011 00:47:41 +0100 (MET) Received: by britannica.bec.de (sSMTP sendmail emulation); Fri, 28 Jan 2011 00:47:37 +0100 Date: Fri, 28 Jan 2011 00:47:37 +0100 From: Joerg Sonnenberger To: freebsd-hackers@freebsd.org Message-ID: <20110127234737.GA31603@britannica.bec.de> Mail-Followup-To: freebsd-hackers@freebsd.org References: <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> <20110127213536.GR2518@deviant.kiev.zoral.com.ua> <1296165538.20060.43.camel@dt.vicor.com> <20110127182418.77c53c60@kan.dnsalias.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110127182418.77c53c60@kan.dnsalias.net> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2011 23:47:45 -0000 On Thu, Jan 27, 2011 at 06:24:18PM -0500, Alexander Kabaev wrote: > For starters, the number of libraries given binary is linked too is > completely and utterly irrelevant :) The change NetBSD guys claims to > revolutionize his application startup times only applies to programs > that dlopen (read - load dynamically) libraries with long largely > identical dependency chains and calls dlsym on them many, many > thousand times. I do not think you will find any real app out there > that fits this description close enough to actually demonstrate the > effect of the change that is distinguishable from the statistical noise. > Pressing ^C certainly not precise enough for that. (1) The program itself needs to have a large enough number of linked libraries. (2) The program needs to dlopen() modules. (3) The program needs to search for missing symbols often enough. >From memory, the real world example that fulfilled all three cases was Evolution. (1) and (2) are pretty typical though. Joerg From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 00:04:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F6381065670 for ; Fri, 28 Jan 2011 00:04:31 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 23D308FC15 for ; Fri, 28 Jan 2011 00:04:30 +0000 (UTC) Received: by qwj9 with SMTP id 9so2739757qwj.13 for ; Thu, 27 Jan 2011 16:04:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:in-reply-to :references:x-mailer:mime-version:content-type; bh=oo77vy/ilRDZUS+1sCsAW1nLgDgDP7mn2Mj5ync1ApA=; b=CNVlsN4h1eSdzNITVK0LPH+g4K866rmyoRidAy+GCwELvqFDES2UgaLLD0ESNafMHd t/LhFcPmIT00caaxYX4okLv9heupeM+EyU2LBXFeXLbMb/r5gErDxjGT31/qCqFYps/b rFuKIkDqisHRu29CmBGoIG0ULJnMvK+gixFDk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; b=Fp1KbL/T5j+ESgzvcUgF3XBm56XhPcaiqLLdYQQ2GR3jcYaMunEciWmx/2/yObRbNf ekIb3tsH+suY4wojpIigl89aA82jyt76ZnycfrscwdYibyGeV8OL4gP48Hft7dMXKhCG L9mGlMM7/4rRXC9MK8mkByeSZXh154knpRNuE= Received: by 10.224.19.203 with SMTP id c11mr2274316qab.170.1296173070384; Thu, 27 Jan 2011 16:04:30 -0800 (PST) Received: from kan.dnsalias.net (c-24-63-226-98.hsd1.ma.comcast.net [24.63.226.98]) by mx.google.com with ESMTPS id nb15sm12214094qcb.38.2011.01.27.16.04.28 (version=SSLv3 cipher=RC4-MD5); Thu, 27 Jan 2011 16:04:29 -0800 (PST) Date: Thu, 27 Jan 2011 19:04:23 -0500 From: Alexander Kabaev To: Joerg Sonnenberger Message-ID: <20110127190423.785cb2d3@kan.dnsalias.net> In-Reply-To: <20110127234737.GA31603@britannica.bec.de> References: <201101271305.21510.naylor.b.david@gmail.com> <20110127203126.GN2518@deviant.kiev.zoral.com.ua> <1296161448.20060.40.camel@dt.vicor.com> <20110127205907.GP2518@deviant.kiev.zoral.com.ua> <1296162754.20060.42.camel@dt.vicor.com> <20110127213536.GR2518@deviant.kiev.zoral.com.ua> <1296165538.20060.43.camel@dt.vicor.com> <20110127182418.77c53c60@kan.dnsalias.net> <20110127234737.GA31603@britannica.bec.de> X-Mailer: Claws Mail 3.7.8 (GTK+ 2.22.1; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/rkOlWEZaVxTAWFpFqlO2mmx"; protocol="application/pgp-signature" Cc: freebsd-hackers@freebsd.org Subject: Re: rtld optimizations X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 00:04:31 -0000 --Sig_/rkOlWEZaVxTAWFpFqlO2mmx Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Fri, 28 Jan 2011 00:47:37 +0100 Joerg Sonnenberger wrote: > On Thu, Jan 27, 2011 at 06:24:18PM -0500, Alexander Kabaev wrote: > > For starters, the number of libraries given binary is linked too is > > completely and utterly irrelevant :) The change NetBSD guys claims > > to revolutionize his application startup times only applies to > > programs that dlopen (read - load dynamically) libraries with long > > largely identical dependency chains and calls dlsym on them many, > > many thousand times. I do not think you will find any real app out > > there that fits this description close enough to actually > > demonstrate the effect of the change that is distinguishable from > > the statistical noise. Pressing ^C certainly not precise enough for > > that. >=20 > (1) The program itself needs to have a large enough number of linked > libraries. That is wrong, of course. symlook_needed is only called when dlsym(, ) is called and the number of objects on main and global DAGS is very much an irrelevant detail hereirrelevant here. > (2) The program needs to dlopen() modules. Yep. > (3) The program needs to search for missing symbols often enough. > ... by means of explicit dlsym() call. =20 > From memory, the real world example that fulfilled all three cases was > Evolution. (1) and (2) are pretty typical though. >=20 Never claimed otherwise. It is just ones with nested trees deep enough to matter is what in very short supply. I would be very interested in seeing the Evolution benchmarked, but I dount very much than even Evolution will manage to break through statistical noise. --=20 Alexander Kabaev --Sig_/rkOlWEZaVxTAWFpFqlO2mmx Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iD8DBQFNQggMQ6z1jMm+XZYRAoyYAKCeGHhlVjj4thRtrzfL6Pq68zz96wCg2suV /0BQGbO16Ms3/q/kXqq/NL4= =0/Zp -----END PGP SIGNATURE----- --Sig_/rkOlWEZaVxTAWFpFqlO2mmx-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 00:20:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32EB01065679 for ; Fri, 28 Jan 2011 00:20:43 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 8F6228FC15 for ; Fri, 28 Jan 2011 00:20:42 +0000 (UTC) Received: (qmail 16277 invoked by uid 399); 28 Jan 2011 00:20:41 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 28 Jan 2011 00:20:41 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D420BD7.1080702@FreeBSD.org> Date: Thu, 27 Jan 2011 16:20:39 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20101212 Thunderbird/3.1.7 MIME-Version: 1.0 To: "Julian H. Stacey" References: <201101262000.p0QK0fGT005251@fire.js.berklix.net> In-Reply-To: <201101262000.p0QK0fGT005251@fire.js.berklix.net> X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: db@db.net, perryh@pluto.rain.com, freebsd-hackers@freebsd.org, ivoras@freebsd.org, Giorgos Keramidas , Alexander Best Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 00:20:43 -0000 On 01/26/2011 12:00, Julian H. Stacey wrote: > Hi, > Alex >> order to build the system. the VCS however is not part of the build toolchain >> however (except 'make update' maybe). > > Some good points, but also remember make release also uses cvs > grep CVS /usr/src/release/Makefile A) The fact that 'make release' uses cvs is not a feature B) It is quite trivially worked around (I speak from experience) hth, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 13:46:25 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 22A991065694 for ; Fri, 28 Jan 2011 13:46:25 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A2F648FC12 for ; Fri, 28 Jan 2011 13:46:24 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Pioec-0000Wz-Ip for freebsd-hackers@freebsd.org; Fri, 28 Jan 2011 14:46:22 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Jan 2011 14:46:22 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 28 Jan 2011 14:46:22 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Date: Fri, 28 Jan 2011 14:46:07 +0100 Lines: 32 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 X-Enigmail-Version: 1.1.2 Subject: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 13:46:25 -0000 I have this situation on a PHP server: 36623 www 1 76 0 237M 30600K *Name 6 0:14 47.27% php-cgi 36638 www 1 76 0 237M 30600K *Name 3 0:14 46.97% php-cgi 36628 www 1 105 0 237M 30600K *Name 2 0:14 46.88% php-cgi 36627 www 1 105 0 237M 30600K *Name 0 0:14 46.78% php-cgi 36639 www 1 105 0 237M 30600K *Name 5 0:14 46.58% php-cgi 36643 www 1 105 0 237M 30600K *Name 7 0:14 46.39% php-cgi 36629 www 1 76 0 237M 30600K *Name 1 0:14 46.39% php-cgi 36642 www 1 105 0 237M 30600K *Name 2 0:14 46.39% php-cgi 36626 www 1 105 0 237M 30600K *Name 5 0:14 46.19% php-cgi 36654 www 1 105 0 237M 30600K *Name 7 0:13 46.19% php-cgi 36645 www 1 105 0 237M 30600K *Name 1 0:14 45.75% php-cgi 36625 www 1 105 0 237M 30600K *Name 0 0:14 45.56% php-cgi 36624 www 1 105 0 237M 30600K *Name 6 0:14 45.56% php-cgi 36630 www 1 76 0 237M 30600K *Name 7 0:14 45.17% php-cgi 36631 www 1 105 0 237M 30600K RUN 4 0:14 45.17% php-cgi 36636 www 1 105 0 237M 30600K *Name 3 0:14 44.87% php-cgi It looks like periodically most or all of the php-cgi processes are blocked in "*Name" for long enough that "top" notices, then continue, probably in a "thundering herd" way. From grepping inside /sys the most likely suspect seems to be something in the namecache, but I can't find exactly a symbol named "Name" or string beginning with "Name" that would be connected to a lock. Has anyone investigated this before? Any ideas where to look? The PHP script used above should not do any filesystem writing but it stats and reads a lot of small files and libraries. This is 8-stable, UFS. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 15:34:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 728361065673; Fri, 28 Jan 2011 15:34:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 489028FC15; Fri, 28 Jan 2011 15:34:33 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 02A8E46B2E; Fri, 28 Jan 2011 10:34:33 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 223878A027; Fri, 28 Jan 2011 10:34:32 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 28 Jan 2011 10:15:36 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201101281015.36218.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 28 Jan 2011 10:34:32 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Ivan Voras Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 15:34:33 -0000 On Friday, January 28, 2011 8:46:07 am Ivan Voras wrote: > I have this situation on a PHP server: > > 36623 www 1 76 0 237M 30600K *Name 6 0:14 47.27% php-cgi > 36638 www 1 76 0 237M 30600K *Name 3 0:14 46.97% php-cgi > 36628 www 1 105 0 237M 30600K *Name 2 0:14 46.88% php-cgi > 36627 www 1 105 0 237M 30600K *Name 0 0:14 46.78% php-cgi > 36639 www 1 105 0 237M 30600K *Name 5 0:14 46.58% php-cgi > 36643 www 1 105 0 237M 30600K *Name 7 0:14 46.39% php-cgi > 36629 www 1 76 0 237M 30600K *Name 1 0:14 46.39% php-cgi > 36642 www 1 105 0 237M 30600K *Name 2 0:14 46.39% php-cgi > 36626 www 1 105 0 237M 30600K *Name 5 0:14 46.19% php-cgi > 36654 www 1 105 0 237M 30600K *Name 7 0:13 46.19% php-cgi > 36645 www 1 105 0 237M 30600K *Name 1 0:14 45.75% php-cgi > 36625 www 1 105 0 237M 30600K *Name 0 0:14 45.56% php-cgi > 36624 www 1 105 0 237M 30600K *Name 6 0:14 45.56% php-cgi > 36630 www 1 76 0 237M 30600K *Name 7 0:14 45.17% php-cgi > 36631 www 1 105 0 237M 30600K RUN 4 0:14 45.17% php-cgi > 36636 www 1 105 0 237M 30600K *Name 3 0:14 44.87% php-cgi > > It looks like periodically most or all of the php-cgi processes are > blocked in "*Name" for long enough that "top" notices, then continue, > probably in a "thundering herd" way. From grepping inside /sys the most > likely suspect seems to be something in the namecache, but I can't find > exactly a symbol named "Name" or string beginning with "Name" that would > be connected to a lock. In vfs_cache.c: static struct rwlock cache_lock; RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); What are the php scripts doing? Do they all try to create and delete files at the same time (or do renames)? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 16:04:34 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C605C1065672; Fri, 28 Jan 2011 16:04:34 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 6D0C38FC14; Fri, 28 Jan 2011 16:04:34 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id p0SFP6wv094447 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Jan 2011 09:25:07 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.4/8.14.4) with ESMTP id p0SFP6Di093128 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 28 Jan 2011 09:25:06 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.4/8.14.4/Submit) id p0SFP6R9093127; Fri, 28 Jan 2011 09:25:06 -0600 (CST) (envelope-from dan) Date: Fri, 28 Jan 2011 09:25:06 -0600 From: Dan Nelson To: Ivan Voras Message-ID: <20110128152505.GP75125@dan.emsphone.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 8.2-PRERELEASE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.96.4 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Fri, 28 Jan 2011 09:25:07 -0600 (CST) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: freebsd-hackers@freebsd.org Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 16:04:34 -0000 In the last episode (Jan 28), Ivan Voras said: > I have this situation on a PHP server: > > 36623 www 1 76 0 237M 30600K *Name 6 0:14 47.27% php-cgi > 36638 www 1 76 0 237M 30600K *Name 3 0:14 46.97% php-cgi > 36628 www 1 105 0 237M 30600K *Name 2 0:14 46.88% php-cgi > 36627 www 1 105 0 237M 30600K *Name 0 0:14 46.78% php-cgi > 36639 www 1 105 0 237M 30600K *Name 5 0:14 46.58% php-cgi > 36643 www 1 105 0 237M 30600K *Name 7 0:14 46.39% php-cgi > 36629 www 1 76 0 237M 30600K *Name 1 0:14 46.39% php-cgi > 36642 www 1 105 0 237M 30600K *Name 2 0:14 46.39% php-cgi > 36626 www 1 105 0 237M 30600K *Name 5 0:14 46.19% php-cgi > 36654 www 1 105 0 237M 30600K *Name 7 0:13 46.19% php-cgi > 36645 www 1 105 0 237M 30600K *Name 1 0:14 45.75% php-cgi > 36625 www 1 105 0 237M 30600K *Name 0 0:14 45.56% php-cgi > 36624 www 1 105 0 237M 30600K *Name 6 0:14 45.56% php-cgi > 36630 www 1 76 0 237M 30600K *Name 7 0:14 45.17% php-cgi > 36631 www 1 105 0 237M 30600K RUN 4 0:14 45.17% php-cgi > 36636 www 1 105 0 237M 30600K *Name 3 0:14 44.87% php-cgi > > It looks like periodically most or all of the php-cgi processes are > blocked in "*Name" for long enough that "top" notices, then continue, > probably in a "thundering herd" way. From grepping inside /sys the most > likely suspect seems to be something in the namecache, but I can't find > exactly a symbol named "Name" or string beginning with "Name" that would > be connected to a lock. My guess would be: kern/vfs_cache.c:151 static struct rwlock cache_lock; kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lots of possible contention points. procstat -ka and/or dtrace might help you determine exactly where. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 16:11:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA7D510656A3 for ; Fri, 28 Jan 2011 16:11:12 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4308D8FC2C for ; Fri, 28 Jan 2011 16:11:11 +0000 (UTC) Received: by qyk36 with SMTP id 36so3384281qyk.13 for ; Fri, 28 Jan 2011 08:11:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=tCx4sSYbBoV0OtrQKZOh8iTLgzkFO+H+8LcgQTRbtNQ=; b=wZzcjEn7b5LgO2NwYpRSPtJAjGssP47kImLSWwe7GujyUTtWFhHjpep8EjFG/174CR 5TlOQvKK1aJXuih70Dzp5FiA8mskJGLx8Ir9CVkXYdLAwBBMvmYE+XESTjk2YK4IzttE pUIGhrvtM9azkU6iZ/WMEU6QGRy4GOG0HDKuk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=s4vvwysBJE6vUmNxV5/m2ryV4UKn1E7tKwa7isbW45caqHGcXPgpP+JFjDqeY4NjD1 NcLArr2FwCBNM9PMsu6AkWplitLSPgTOgHEL9c8TQusa5n9PEcHcwTlhEi4wV2lkedXy 8yyj3tkbhBFJ+t+AIav/3yY0V6Skac4NiBWu0= Received: by 10.229.43.195 with SMTP id x3mr2824332qce.291.1296231070847; Fri, 28 Jan 2011 08:11:10 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Fri, 28 Jan 2011 08:10:30 -0800 (PST) In-Reply-To: <201101281015.36218.jhb@freebsd.org> References: <201101281015.36218.jhb@freebsd.org> From: Ivan Voras Date: Fri, 28 Jan 2011 17:10:30 +0100 X-Google-Sender-Auth: kBkPrn7-lgMZcF7GQf9qxHICa18 Message-ID: To: John Baldwin Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 16:11:12 -0000 On 28 January 2011 16:15, John Baldwin wrote: > On Friday, January 28, 2011 8:46:07 am Ivan Voras wrote: >> I have this situation on a PHP server: >> >> 36623 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A076 =C2=A0 =C2=A00 =C2=A0 2= 37M 30600K *Name =C2=A0 6 =C2=A0 0:14 47.27% php-cgi >> 36638 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A076 =C2=A0 =C2=A00 =C2=A0 2= 37M 30600K *Name =C2=A0 3 =C2=A0 0:14 46.97% php-cgi >> 36628 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 2 =C2=A0 0:14 46.88% php-cgi >> 36627 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 0 =C2=A0 0:14 46.78% php-cgi >> 36639 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 5 =C2=A0 0:14 46.58% php-cgi >> 36643 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 7 =C2=A0 0:14 46.39% php-cgi >> 36629 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A076 =C2=A0 =C2=A00 =C2=A0 2= 37M 30600K *Name =C2=A0 1 =C2=A0 0:14 46.39% php-cgi >> 36642 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 2 =C2=A0 0:14 46.39% php-cgi >> 36626 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 5 =C2=A0 0:14 46.19% php-cgi >> 36654 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 7 =C2=A0 0:13 46.19% php-cgi >> 36645 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 1 =C2=A0 0:14 45.75% php-cgi >> 36625 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 0 =C2=A0 0:14 45.56% php-cgi >> 36624 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 6 =C2=A0 0:14 45.56% php-cgi >> 36630 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 =C2=A076 =C2=A0 =C2=A00 =C2=A0 2= 37M 30600K *Name =C2=A0 7 =C2=A0 0:14 45.17% php-cgi >> 36631 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K RUN =C2=A0 =C2=A0 4 =C2=A0 0:14 45.17% php-cgi >> 36636 www =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 105 =C2=A0 =C2=A00 =C2=A0 237M 3= 0600K *Name =C2=A0 3 =C2=A0 0:14 44.87% php-cgi >> >> It looks like periodically most or all of the php-cgi processes are >> blocked in "*Name" for long enough that "top" notices, then continue, >> probably in a "thundering herd" way. From grepping inside /sys the most >> likely suspect seems to be something in the namecache, but I can't find >> exactly a symbol named "Name" or string beginning with "Name" that would >> be connected to a lock. > > In vfs_cache.c: > > static struct rwlock cache_lock; > RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); You're right, I misread it as SYSCTL at a glance. > What are the php scripts doing? =C2=A0Do they all try to create and delet= e files at > the same time (or do renames)? Right again - they do simultaneously create session files and in rare occasions (1%) delete them. These are "sharded" into a two-level directory structure by single letter (/storage/a/b/file, i.e. 32^2 directories); dirhash is large enough. During all this, the web server did around 60 PHP pages per second so it doesn't look to me like there should be such noticable contention (i.e. at most, there are 60 files/s created and on average 60/100 deletes). The file system is on softupdates, there's only light IO. Typical vmstat is: procs memory page disks faults cp= u r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id 17 0 0 8730M 1240M 3 0 0 0 206 0 1 0 1948 266928 15079 65 34 1 19 0 0 8730M 1240M 0 0 0 0 290 0 1 24 1835 260618 15132 63 35 2 7 0 0 8730M 1239M 0 0 0 0 200 0 0 0 1822 260783 14851 63 35 2 16 0 0 8730M 1239M 0 0 0 0 199 0 788 0 2744 259902 20465 61 37 2 16 0 0 8730M 1239M 0 0 0 0 210 0 0 0 1755 265081 17564 61 37 2 (8 cores; around 35% sys load across them - I'm trying to find out why). From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 17:20:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E87EE10656A4 for ; Fri, 28 Jan 2011 17:20:57 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 9FE928FC17 for ; Fri, 28 Jan 2011 17:20:57 +0000 (UTC) Received: by vxa40 with SMTP id 40so148634vxa.13 for ; Fri, 28 Jan 2011 09:20:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=lqVbz7ocdOx/mRvRtWZ6PvuYsi1Pdlu5zo58LQ09lGM=; b=wDYeiHBXpxZbQuiqtr+sk7Mzgb2gwhhBhflpPrM9a0zoECXR3gbzG9z5TnfdehJZ83 kOx7KrWVxu/EttoEAukE3hV2s7sgHRh2PC/xLjf2weTuIZLexjbcMaZZBAEWLUDSVBgz 2itDSeG5VrO+wwRF441ml2tE5sL7f4FKOxKhA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=Fc2yx5y7gzugZmKnsF5psDyOUajfV0/q4Whgb6/c5kEA++L8JkXZc/MGXis/arV1Vt 0I+CAUfuRKBQ/4rqgX/x+1+RnzXeLd6csFRM/Fjk3yuSq6rN4Om84jmuhq6HWSz2WOdM rkIiwuUO2zXWgW9flXC/vnabjy1Eu/M++Fw2I= Received: by 10.229.181.74 with SMTP id bx10mr2868447qcb.215.1296235255812; Fri, 28 Jan 2011 09:20:55 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Fri, 28 Jan 2011 09:20:15 -0800 (PST) In-Reply-To: <20110128152505.GP75125@dan.emsphone.com> References: <20110128152505.GP75125@dan.emsphone.com> From: Ivan Voras Date: Fri, 28 Jan 2011 18:20:15 +0100 X-Google-Sender-Auth: ZFH-TGykQUhuI0BKDIT95Y_FMOM Message-ID: To: Dan Nelson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 17:20:58 -0000 On 28 January 2011 16:25, Dan Nelson wrote: > My guess would be: > > kern/vfs_cache.c:151 static struct rwlock cache_lock; > kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); > > The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lot= s > of possible contention points. =C2=A0procstat -ka and/or dtrace might hel= p you > determine exactly where. I'm new with dtrace so I tried this: lockstat:::rw-block { @traces[stack()] =3D count(); } with these results: http://ivoras.net/stuff/rw-block.txt It's informative because most of the traces are namecache-related. As suspected, the most blocking occurs in stat(). As this is a rwlock I'd interpret it as waiting for a write lock to be lifted so the readers can acquire it, but I need to confirm this as there's a lot of things that can in theory be stat()ed here. I'm going to continue investigating with dtrace but I'd appreciate pointers on how to make the output more useful (like including filenames from stat()). From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 17:41:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B8621065675 for ; Fri, 28 Jan 2011 17:41:10 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 11CD58FC0C for ; Fri, 28 Jan 2011 17:41:09 +0000 (UTC) Received: by wyf19 with SMTP id 19so3518214wyf.13 for ; Fri, 28 Jan 2011 09:41:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=iHa6ESFhL44Qvvr5AMoQSx8dCrU6eJtaA8JTFU+Aiso=; b=idXgDL4I4jBE/w0BITPjvz22BAf7WCwf2fIdqlO/fAFsnSKDrtdR3mCR8Ng8JGjffl auvHvn1dF16R5eIrj2Yol+n7xWfcG2HJrNtNDox8zComEilGRzpsUgURHpPAMRX6qrpk 2ZZb95LrLM3U3Bpno4iZVDRUd6jjbzJ2hUlW4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=uNSVH5A5y6plpvMbo+2Ch6mOGuQmsmGjKUApgVXvc9gy0YcPno8FtC/6F88q8ScZn8 P8dIm8D2Ed3UF7omAQYhfx9h3wMAjBNIs/NmbKwxC1f07qb2cuR9JcRXXhrmcHJha3iN OPIIeVYIqgH++oytTFF8m7UEGqoPyL9tlT51A= MIME-Version: 1.0 Received: by 10.216.156.6 with SMTP id l6mr8091412wek.55.1296236468370; Fri, 28 Jan 2011 09:41:08 -0800 (PST) Received: by 10.216.62.203 with HTTP; Fri, 28 Jan 2011 09:41:08 -0800 (PST) Date: Fri, 28 Jan 2011 09:41:08 -0800 Message-ID: From: Matthew Fleming To: freebsd-hackers Content-Type: text/plain; charset=ISO-8859-1 Subject: Divide-by-zero in loader X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 17:41:10 -0000 I spent a few days chasing down a bug and I'm wondering if a loader change would be appropriate. So we have these new front-panel LCDs, and like everything these days it's a SoC. Normally it presents to FreeBSD as a USB communications device (ucom), but when the SoC is sitting in its own boot loader, it presents as storage (umass). If the box is rebooted in this state, the reboot gets into /boot/loader and then reboots itself. (It took a few days just to figure out I was getting into /boot/loader, since the only prompt I could definitively stop at was boot2). Anyways, I eventually debugged it to the device somehow presenting itself to /boot/loader with a geometry of 1024/256/0, and since od_sec is 0 that causes a divide-by-zero error in bd_io() while the loader is trying to figure out if this is GPT or MBR formatted. We're still trying to figure out why the loader sees this incorrect geometry. But meanwhile, this patch fixes the issue, and I wonder if it would be a useful safety-belt for other devices where an incorrect geometry can be seen? Thanks, matthew Index: i386/libi386/biosdisk.c =================================================================== --- i386/libi386/biosdisk.c (.../head/src/sys/boot) (revision 172978) +++ i386/libi386/biosdisk.c (.../branches/BR_BUG_73454/src/sys/boot) (revision 172978) @@ -2064,30 +2064,38 @@ bd_getgeom(struct open_disk *od) v86.addr = 0x13; v86.eax = 0x800; v86.edx = od->od_unit; v86int(); if ((v86.efl & 0x1) || /* carry set */ ((v86.edx & 0xff) <= (unsigned)(od->od_unit & 0x7f))) /* unit # bad */ return(1); /* convert max cyl # -> # of cylinders */ od->od_cyl = ((v86.ecx & 0xc0) << 2) + ((v86.ecx & 0xff00) >> 8) + 1; /* convert max head # -> # of heads */ od->od_hds = ((v86.edx & 0xff00) >> 8) + 1; od->od_sec = v86.ecx & 0x3f; + if (od->od_sec == 0) { + printf("Bad disk geometry on unit %d, bios unit %d, chs %d/%d/%d\n", + od->od_dkunit, od->od_unit, od->od_cyl, od->od_hds, od->od_sec); + return (1); + } + DEBUG("unit 0x%x geometry %d/%d/%d", od->od_unit, od->od_cyl, od->od_hds, od->od_sec); return(0); } /* * Return the BIOS geometry of a given "fixed drive" in a format * suitable for the legacy bootinfo structure. Since the kernel is * expecting raw int 0x13/0x8 values for N_BIOS_GEOM drives, we * prefer to get the information directly, rather than rely on being * able to put it together from information already maintained for * different purposes and for a probably different number of drives. * * For valid drives, the geometry is expected in the format (31..0) * "000000cc cccccccc hhhhhhhh 00ssssss"; and invalid drives are * indicated by returning the geometry of a "1.2M" PC-format floppy From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 19:09:15 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A29A71065670 for ; Fri, 28 Jan 2011 19:09:15 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 75CC68FC1A for ; Fri, 28 Jan 2011 19:09:15 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 2E63F46B37; Fri, 28 Jan 2011 14:09:15 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 43CEE8A01D; Fri, 28 Jan 2011 14:09:14 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 28 Jan 2011 14:00:01 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101281400.01840.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 28 Jan 2011 14:09:14 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: Matthew Fleming Subject: Re: Divide-by-zero in loader X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 19:09:15 -0000 On Friday, January 28, 2011 12:41:08 pm Matthew Fleming wrote: > I spent a few days chasing down a bug and I'm wondering if a loader > change would be appropriate. > > So we have these new front-panel LCDs, and like everything these days > it's a SoC. Normally it presents to FreeBSD as a USB communications > device (ucom), but when the SoC is sitting in its own boot loader, it > presents as storage (umass). If the box is rebooted in this state, > the reboot gets into /boot/loader and then reboots itself. (It took a > few days just to figure out I was getting into /boot/loader, since the > only prompt I could definitively stop at was boot2). > > Anyways, I eventually debugged it to the device somehow presenting > itself to /boot/loader with a geometry of 1024/256/0, and since od_sec > is 0 that causes a divide-by-zero error in bd_io() while the loader is > trying to figure out if this is GPT or MBR formatted. We're still > trying to figure out why the loader sees this incorrect geometry. > > But meanwhile, this patch fixes the issue, and I wonder if it would be > a useful safety-belt for other devices where an incorrect geometry can > be seen? That's probably fine. A sector count of zero is invalid for CHS. However, probably we should not even be using C/H/S at all if the device claims to support EDD. We already use raw LBAs if it supports EDD, and we should probably just ignore C/H/S altogether if it supports EDD. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 19:14:46 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF1B41065673; Fri, 28 Jan 2011 19:14:46 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1FA078FC08; Fri, 28 Jan 2011 19:14:45 +0000 (UTC) Received: by wyf19 with SMTP id 19so3621135wyf.13 for ; Fri, 28 Jan 2011 11:14:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=/Vo9PlTDqO3Jkz77D3ajECRZ7TNjSBUPf46R+P5ZtV0=; b=oxhn4E7O7RqF+hF93m0wCOz7kAsGUekZ9H7MVut9/Xc9zsFWksHmISFpjghNT2Lsk9 A5Y2g13vK3aqfDCdTFRk5gRtrqTWrxUp0oykHaQnyCq8dlzBhbcyvGUs9bD0r1tCSPBh vq7cHFq4Y7PUrxquPsKIDEbCOJbyV0OZmwoaM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=dOyjLogE06M0Hmm8012PvqRzIIxgGJaJcNphrnjSsGhSAaSM9Ojaf50E35uQ+BKdIj PXcPwqSson5612E1KJKKY3q2G9EG0XljBfAysYJfZXnE44qbjdIO63gAtHYPFOAHiUDh bc9sO8KpSf63wwTjC4VwYAdq/JWQczNk9qyOI= MIME-Version: 1.0 Received: by 10.216.153.147 with SMTP id f19mr8238796wek.40.1296242085080; Fri, 28 Jan 2011 11:14:45 -0800 (PST) Received: by 10.216.62.203 with HTTP; Fri, 28 Jan 2011 11:14:45 -0800 (PST) In-Reply-To: <201101281400.01840.jhb@freebsd.org> References: <201101281400.01840.jhb@freebsd.org> Date: Fri, 28 Jan 2011 11:14:45 -0800 Message-ID: From: Matthew Fleming To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Divide-by-zero in loader X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 19:14:46 -0000 On Fri, Jan 28, 2011 at 11:00 AM, John Baldwin wrote: > On Friday, January 28, 2011 12:41:08 pm Matthew Fleming wrote: >> I spent a few days chasing down a bug and I'm wondering if a loader >> change would be appropriate. >> >> So we have these new front-panel LCDs, and like everything these days >> it's a SoC. =A0Normally it presents to FreeBSD as a USB communications >> device (ucom), but when the SoC is sitting in its own boot loader, it >> presents as storage (umass). =A0If the box is rebooted in this state, >> the reboot gets into /boot/loader and then reboots itself. =A0(It took a >> few days just to figure out I was getting into /boot/loader, since the >> only prompt I could definitively stop at was boot2). >> >> Anyways, I eventually debugged it to the device somehow presenting >> itself to /boot/loader with a geometry of 1024/256/0, and since od_sec >> is 0 that causes a divide-by-zero error in bd_io() while the loader is >> trying to figure out if this is GPT or MBR formatted. =A0We're still >> trying to figure out why the loader sees this incorrect geometry. >> >> But meanwhile, this patch fixes the issue, and I wonder if it would be >> a useful safety-belt for other devices where an incorrect geometry can >> be seen? > > That's probably fine. =A0A sector count of zero is invalid for CHS. =A0Ho= wever, > probably we should not even be using C/H/S at all if the device claims to > support EDD. =A0We already use raw LBAs if it supports EDD, and we should > probably just ignore C/H/S altogether if it supports EDD. This is all almost entirely outside my knowledge, but at the moment bd_eddprobe() requres a geometry of 1023/255/63 before it attempts to check if EDD can be used. Is that check incorrect? In my specific case I know there's no bootable stuff on this disk; the earlier layers bypassed it correctly without a problem. Thanks, matthew From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 19:27:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 270D7106566B for ; Fri, 28 Jan 2011 19:27:31 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id ED7DF8FC1C for ; Fri, 28 Jan 2011 19:27:30 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A2D2646B06; Fri, 28 Jan 2011 14:27:30 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A114E8A009; Fri, 28 Jan 2011 14:27:29 -0500 (EST) From: John Baldwin To: Matthew Fleming Date: Fri, 28 Jan 2011 14:23:02 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.4-CBSD-20110107; KDE/4.4.5; amd64; ; ) References: <201101281400.01840.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101281423.02701.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 28 Jan 2011 14:27:29 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: freebsd-hackers@freebsd.org Subject: Re: Divide-by-zero in loader X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 19:27:31 -0000 On Friday, January 28, 2011 2:14:45 pm Matthew Fleming wrote: > On Fri, Jan 28, 2011 at 11:00 AM, John Baldwin wrote: > > On Friday, January 28, 2011 12:41:08 pm Matthew Fleming wrote: > >> I spent a few days chasing down a bug and I'm wondering if a loader > >> change would be appropriate. > >> > >> So we have these new front-panel LCDs, and like everything these days > >> it's a SoC. Normally it presents to FreeBSD as a USB communications > >> device (ucom), but when the SoC is sitting in its own boot loader, it > >> presents as storage (umass). If the box is rebooted in this state, > >> the reboot gets into /boot/loader and then reboots itself. (It took a > >> few days just to figure out I was getting into /boot/loader, since the > >> only prompt I could definitively stop at was boot2). > >> > >> Anyways, I eventually debugged it to the device somehow presenting > >> itself to /boot/loader with a geometry of 1024/256/0, and since od_sec > >> is 0 that causes a divide-by-zero error in bd_io() while the loader is > >> trying to figure out if this is GPT or MBR formatted. We're still > >> trying to figure out why the loader sees this incorrect geometry. > >> > >> But meanwhile, this patch fixes the issue, and I wonder if it would be > >> a useful safety-belt for other devices where an incorrect geometry can > >> be seen? > > > > That's probably fine. A sector count of zero is invalid for CHS. However, > > probably we should not even be using C/H/S at all if the device claims to > > support EDD. We already use raw LBAs if it supports EDD, and we should > > probably just ignore C/H/S altogether if it supports EDD. > > This is all almost entirely outside my knowledge, but at the moment > bd_eddprobe() requres a geometry of 1023/255/63 before it attempts to > check if EDD can be used. Is that check incorrect? Well, it is very conservative in that it only uses EDD if it thinks it can't use C/H/S. It would be interesting to see if simply checking for a sector count of 0 there would avoid the divide-by-zero and let your device "work". However, it might actually be useful to always use EDD if possible, esp. EDD3 since that lets you not use bounce buffers down in 1MB. > In my specific case I know there's no bootable stuff on this disk; the > earlier layers bypassed it correctly without a problem. > > Thanks, > matthew > -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 19:35:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3E1D1065704; Fri, 28 Jan 2011 19:35:57 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 431938FC1F; Fri, 28 Jan 2011 19:35:56 +0000 (UTC) Received: by wyf19 with SMTP id 19so3646671wyf.13 for ; Fri, 28 Jan 2011 11:35:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=JSJfQWT0GX0wX8tCDEpDCclWJ6EEkJyLnCVlGPXjkMM=; b=cJfVeCW7j8Nha8YSmIYYcXlX3dC1U3YWHcs7Jaehx0X4jMEErZpl+b4IAQ2pISeJUY wvSrgLgHeadzjVyviWjOCbQq9to6dwha9foELmFj4I4eqHF/NwEZzCt4F9F6bA/dMLdb RFrjdFPfK0wuxNCf4f8P27e0MAlJr/EalTOoU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=nSXfud4sDizru//KS3smD3xG7fcXTgWg8/Af8ZSC5fAUrTz1oygea9MPBz17nKalSQ d2lAoS5HRrUZyvPU9XBvi41tzxULHFo+TBGo9mJTW10+I5zPRy8Ou2jSgGEx8UfIlfxh Imn2PpbsmOqp1nXne4JAIEoFS9ltFB9MqoU+Q= MIME-Version: 1.0 Received: by 10.216.153.147 with SMTP id f19mr8255709wek.40.1296243356047; Fri, 28 Jan 2011 11:35:56 -0800 (PST) Received: by 10.216.62.203 with HTTP; Fri, 28 Jan 2011 11:35:55 -0800 (PST) In-Reply-To: <201101281423.02701.jhb@freebsd.org> References: <201101281400.01840.jhb@freebsd.org> <201101281423.02701.jhb@freebsd.org> Date: Fri, 28 Jan 2011 11:35:55 -0800 Message-ID: From: Matthew Fleming To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: Divide-by-zero in loader X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 19:35:57 -0000 On Fri, Jan 28, 2011 at 11:23 AM, John Baldwin wrote: > On Friday, January 28, 2011 2:14:45 pm Matthew Fleming wrote: >> On Fri, Jan 28, 2011 at 11:00 AM, John Baldwin wrote: >> > On Friday, January 28, 2011 12:41:08 pm Matthew Fleming wrote: >> >> I spent a few days chasing down a bug and I'm wondering if a loader >> >> change would be appropriate. >> >> >> >> So we have these new front-panel LCDs, and like everything these days >> >> it's a SoC. =A0Normally it presents to FreeBSD as a USB communication= s >> >> device (ucom), but when the SoC is sitting in its own boot loader, it >> >> presents as storage (umass). =A0If the box is rebooted in this state, >> >> the reboot gets into /boot/loader and then reboots itself. =A0(It too= k a >> >> few days just to figure out I was getting into /boot/loader, since th= e >> >> only prompt I could definitively stop at was boot2). >> >> >> >> Anyways, I eventually debugged it to the device somehow presenting >> >> itself to /boot/loader with a geometry of 1024/256/0, and since od_se= c >> >> is 0 that causes a divide-by-zero error in bd_io() while the loader i= s >> >> trying to figure out if this is GPT or MBR formatted. =A0We're still >> >> trying to figure out why the loader sees this incorrect geometry. >> >> >> >> But meanwhile, this patch fixes the issue, and I wonder if it would b= e >> >> a useful safety-belt for other devices where an incorrect geometry ca= n >> >> be seen? >> > >> > That's probably fine. =A0A sector count of zero is invalid for CHS. = =A0However, >> > probably we should not even be using C/H/S at all if the device claims= to >> > support EDD. =A0We already use raw LBAs if it supports EDD, and we sho= uld >> > probably just ignore C/H/S altogether if it supports EDD. >> >> This is all almost entirely outside my knowledge, but at the moment >> bd_eddprobe() requres a geometry of 1023/255/63 before it attempts to >> check if EDD can be used. =A0Is that check incorrect? > > Well, it is very conservative in that it only uses EDD if it thinks it ca= n't > use C/H/S. =A0It would be interesting to see if simply checking for a sec= tor > count of 0 there would avoid the divide-by-zero and let your device "work= ". I'll need a few more pointers to try this. As structured today, bd_io is what does the divide-by-zero, and it always looks at od_sec to determine the maximum transfer size. It's only later in bd_io that it checks the EDD mode to decide how to read. /* * Play it safe and don't cross track boundaries. * (XXX this is probably unnecessary) */ sec =3D dblk % od->od_sec; /* offset into track */ x =3D min(od->od_sec - sec, resid); if (maxfer > 0) x =3D min(x, maxfer); /* fit bounce buffer */ However, I suppose this may be safe-ish to do as just: x =3D resid; if (maxfer > 0 && maxfer < resid) x =3D maxfer; /* fit bounce buffer */ In which case the only uses of od_sec is now in bd_chs_io, and some printf'= s. Though note that currently also the bd_eddprobe is only done in the bd_open_mbr() step which is only if the GPT reads showed it isn't a GPT device. I suppose that the EDD probe can be part of bd_getgeom, though. I'll try this and see what happens on my device. Thanks, matthew > However, it might actually be useful to always use EDD if possible, esp. > EDD3 since that lets you not use bounce buffers down in 1MB. > >> In my specific case I know there's no bootable stuff on this disk; the >> earlier layers bypassed it correctly without a problem. >> >> Thanks, >> matthew >> > > -- > John Baldwin > From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 20:13:01 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B76611065670; Fri, 28 Jan 2011 20:13:01 +0000 (UTC) (envelope-from jamie@FreeBSD.org) Received: from gritton.org (gritton.org [208.92.232.93]) by mx1.freebsd.org (Postfix) with ESMTP id 7B7A48FC16; Fri, 28 Jan 2011 20:13:01 +0000 (UTC) Received: from guppy.corp.verio.net (fw.oremut02.us.wh.verio.net [198.65.168.24]) (authenticated bits=0) by gritton.org (8.14.3/8.14.3) with ESMTP id p0SKCveo027090; Fri, 28 Jan 2011 13:12:57 -0700 (MST) (envelope-from jamie@FreeBSD.org) Message-ID: <4D432344.5050001@FreeBSD.org> Date: Fri, 28 Jan 2011 13:12:52 -0700 From: Jamie Gritton User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20110107 Thunderbird/3.1.6 MIME-Version: 1.0 To: Kostik Belousov References: <20110127120447.GA40060@psconsult.nl> <4D41A65C.70204@FreeBSD.org> <4D41AD52.6030007@FreeBSD.org> <20110127203015.GM2518@deviant.kiev.zoral.com.ua> In-Reply-To: <20110127203015.GM2518@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, Dirk Engling , freebsd-jail@FreeBSD.org Subject: Re: rc.d/jail issues X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 20:13:01 -0000 No - this is entirely a user-space project. Those are both things I'd like to add to jails after I get through the mess on the ofhter side of the kernel divide. - Jamie On 01/27/11 13:30, Kostik Belousov wrote: > On Thu, Jan 27, 2011 at 10:37:22AM -0700, Jamie Gritton wrote: >> It's in the the subversion tree, under projects/jailconf. >> >> I've got the dependency stuff there (actually turns out to be a large >> chunk of the code). I've got it doing almost everything that rc.d/jail >> does now, though it doesn't yet handle errors like it should. After I >> get that fixed up, I plan on putting it in HEAD. After that, I still >> have a todo list mostly of suggestions from others. >> >> Feel free to give me any "todo" suggestions, or any other feedback :-). > Are per-jail init and console in the list ? From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 21:18:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30E6A10656C1; Fri, 28 Jan 2011 21:18:51 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8B9B28FC13; Fri, 28 Jan 2011 21:18:50 +0000 (UTC) Received: by eyf6 with SMTP id 6so1844610eyf.13 for ; Fri, 28 Jan 2011 13:18:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=8vEXFk2jvvz/FZZg8IL1CAW4JrFZxHO0nXsBjWaCRaQ=; b=X12WRfXVS6n5MgvdC4SS6ZCg9NwxI15zxccm7Is39y6nW6YywcBEpSLbrgz2ghlAK8 yCKJYQ3kpLMndTe5nxOaEfEa2yyVuKRSNzao6bhkTY/3JxuZrHs52FlF7uDGVeXEsc05 CN5zkg65bPfI0cMkvh9V0y35HWyexIQkfToeE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; b=rnfApNG2JIM8MOSBNjnWcz8SEIWEDIY4/JGM1FLnuRFvhCf52oDwjA8J1CnnhIyuB6 nbYMhX8VtOaJfYGCe9Bj351hNDMlMuB1l6df1sQH4gu5VUyVRqaD7kotkF9IC+kn18j/ u3dgtJOQJIFEb68p5y9Yz4jWHzfVk/5ovaNAg= Received: by 10.213.104.140 with SMTP id p12mr5515372ebo.76.1296249528504; Fri, 28 Jan 2011 13:18:48 -0800 (PST) Received: from localhost (lan-78-157-92-5.vln.skynet.lt [78.157.92.5]) by mx.google.com with ESMTPS id x54sm14132705eeh.17.2011.01.28.13.18.47 (version=SSLv3 cipher=RC4-MD5); Fri, 28 Jan 2011 13:18:47 -0800 (PST) Date: Fri, 28 Jan 2011 23:18:34 +0200 From: Gleb Kurtsou To: Ivan Voras Message-ID: <20110128211834.GA84881@tops.skynet.lt> References: <20110128152505.GP75125@dan.emsphone.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, Dan Nelson Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 21:18:51 -0000 On (28/01/2011 18:20), Ivan Voras wrote: > On 28 January 2011 16:25, Dan Nelson wrote: > > > My guess would be: > > > > kern/vfs_cache.c:151 static struct rwlock cache_lock; > > kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache"); > > > > The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lots > > of possible contention points.  procstat -ka and/or dtrace might help you > > determine exactly where. > > I'm new with dtrace so I tried this: > > lockstat:::rw-block > { > @traces[stack()] = count(); > } > > with these results: > > http://ivoras.net/stuff/rw-block.txt > > It's informative because most of the traces are namecache-related. As > suspected, the most blocking occurs in stat(). > > As this is a rwlock I'd interpret it as waiting for a write lock to be > lifted so the readers can acquire it, but I need to confirm this as > there's a lot of things that can in theory be stat()ed here. > > I'm going to continue investigating with dtrace but I'd appreciate > pointers on how to make the output more useful (like including > filenames from stat()). You could try replacing rwlock with plain mutex to check if there are priority propagation issues among readers/writers. SX locks should also work but would likely to be a considerable performance regression. Finding out home much activity is there outside of storage/a/b/file (sessions storage) could also be helpful. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 21:23:14 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE4C71065670 for ; Fri, 28 Jan 2011 21:23:13 +0000 (UTC) (envelope-from dhesser@accima.com) Received: from mail.odessaoffice.com (mail.odessaoffice.com [64.146.146.8]) by mx1.freebsd.org (Postfix) with ESMTP id D0DCF8FC13 for ; Fri, 28 Jan 2011 21:23:13 +0000 (UTC) Received: from belinda.androcles.org ([::ffff:199.204.207.9]) (AUTH: CRAM-MD5 dhesser@accima.com, TLS: TLSv1/SSLv3, 256bits, AES256-SHA) by mail.odessaoffice.com with esmtp; Fri, 28 Jan 2011 12:43:13 -0800 id 00364028.4D432A61.00002C86 Date: Fri, 28 Jan 2011 12:43:12 -0800 From: "Duane H. Hesser" To: hackers@freebsd.org Message-Id: <20110128124312.a837f288.dhesser@accima.com> X-Mailer: Sylpheed 3.0.3 (GTK+ 2.22.1; i386-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Subject: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dhesser@accima.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 21:23:14 -0000 I am attempting to replace the 'nv' X11 driver with the "official" nvidia driver from ithe x11/nvidia-driver port, in order to handle the AVCHD video files from my Canon HF S20. I have been trying for several days now, having read the nvidia README file in /usr/local/share and everything Google has to offer. Unfortunately devilfs is smarter and meaner than I. The 'xorg.conf' file is created by nividia-xconfig. The console output when calling 'startx' to begin the frustration is =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= X.Org X Server 1.7.5 Release Date: 2010-02-16 X Protocol Version 11, Revision 0 Build Operating System: FreeBSD 8.1-RELEASE i386 Current Operating System: FreeBSD belinda.androcles.org 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #3: Thu Jan 27 13:45:06 PST 2011 root@belinda.androcles.org:/usr/obj/usr/src/sys/BELINDA i386 Build Date: 08 January 2011 05:52:50PM Current version of pixman: 0.18.4 Before reporting problems, check http://wiki.x.org to make sure that you have the latest version. Markers: (--) probed, (**) from config file, (==) default setting, (++) from command line, (!!) notice, (II) informational, (WW) warning, (EE) error, (NI) not implemented, (??) unknown. (==) Log file: "/var/log/Xorg.0.log", Time: Fri Jan 28 11:32:46 2011 (==) Using config file: "/etc/X11/xorg.conf" NVIDIA: could not open the device file /dev/nvidiactl (No such file or directory). (EE) Jan 28 11:32:46 NVIDIA(0): Failed to initialize the NVIDIA kernel module. Please see the (EE) Jan 28 11:32:46 NVIDIA(0): system's kernel log for additional error messages and (EE) Jan 28 11:32:46 NVIDIA(0): consult the NVIDIA README for details. (EE) NVIDIA(0): *** Aborting *** (EE) Screen(s) found, but none have a usable configuration. Fatal server error: no screens found =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= There must be others out there who have been using this driver under 8.x, which suggests that I must be uttering the wrong incantation (or that devilfs is smarter and meaner than I). I have uploaded all relevant files to my ISP's website http://accima.com/members/dhesser/fbsd8.2P-nvidia-driver/ so that anyone willing to make suggestions may access the information. My primary question at this point is "Is this driver known to work under 8.2 PRERELEASE?" The files on the website represent the most recent attempt, after a "make update" at noon (PST8PDT) yesterday (Jan 28, 2011) followed by the usual upgrade process, and a deinstall/reinstall of the nvidias driver port. There is one further point of confusion. I've compared the list of files which the README claims to install, and found that 3 or 4 files related to "libvdpau.so" are not actually installed by tne driver port. These files are installed by the "multimedia/libvdpau" port. I have installed those files from that port, with no change in the behavior of the driver port. Versions of these files are actually included in x11/nvidia-driver/work/NVIDIA-FreeBSD-x86-256.53/obj, but are not installed by the Makefile. I've tried those too, with no change. I have currently re-installed the multimedia/libvdpau files. Anyone know which is correct, or why the nvidia-driver port does not explain? Note: the "hastd" problem has been fixed in stable, but now the "jme" module fails to compile. -- Duane H. Hesser From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 21:44:54 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 455D0106566B for ; Fri, 28 Jan 2011 21:44:54 +0000 (UTC) (envelope-from rdivacky@vlk.vlakno.cz) Received: from vlakno.cz (lev.vlakno.cz [77.93.215.190]) by mx1.freebsd.org (Postfix) with ESMTP id 9CE428FC17 for ; Fri, 28 Jan 2011 21:44:53 +0000 (UTC) Received: from lev.vlakno.cz (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 3A12D9CB879; Fri, 28 Jan 2011 22:27:38 +0100 (CET) X-Virus-Scanned: amavisd-new at vlakno.cz Received: from vlakno.cz ([127.0.0.1]) by lev.vlakno.cz (lev.vlakno.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QwtHk-+HJDLI; Fri, 28 Jan 2011 22:27:37 +0100 (CET) Received: from vlk.vlakno.cz (localhost [127.0.0.1]) by vlakno.cz (Postfix) with ESMTP id 3C9969CB87F; Fri, 28 Jan 2011 22:27:37 +0100 (CET) Received: (from rdivacky@localhost) by vlk.vlakno.cz (8.14.4/8.14.4/Submit) id p0SLRaLo044934; Fri, 28 Jan 2011 22:27:36 +0100 (CET) (envelope-from rdivacky) Date: Fri, 28 Jan 2011 22:27:36 +0100 From: Roman Divacky To: "Duane H. Hesser" Message-ID: <20110128212736.GA44464@freebsd.org> References: <20110128124312.a837f288.dhesser@accima.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110128124312.a837f288.dhesser@accima.com> User-Agent: Mutt/1.4.2.3i Cc: hackers@freebsd.org Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 21:44:54 -0000 does "kldstat | grep nvidia" show anything? if not kldload nvidia or echo nvidia_load=\"YES\" >> /boot/loader.conf reboot does this change anything? you seem to be missing /dev/nvidiactl which probably means that the nvidia module is not loaded On Fri, Jan 28, 2011 at 12:43:12PM -0800, Duane H. Hesser wrote: > I am attempting to replace the 'nv' X11 driver with the "official" > nvidia driver from ithe x11/nvidia-driver port, in order to handle > the AVCHD video files from my Canon HF S20. > > I have been trying for several days now, having read the nvidia > README file in /usr/local/share and everything Google has to offer. > > Unfortunately devilfs is smarter and meaner than I. > > The 'xorg.conf' file is created by nividia-xconfig. The console > output when calling 'startx' to begin the frustration is > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > X.Org X Server 1.7.5 > Release Date: 2010-02-16 > X Protocol Version 11, Revision 0 > Build Operating System: FreeBSD 8.1-RELEASE i386 > Current Operating System: FreeBSD belinda.androcles.org 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #3: Thu Jan 27 13:45:06 PST 2011 root@belinda.androcles.org:/usr/obj/usr/src/sys/BELINDA i386 > Build Date: 08 January 2011 05:52:50PM > > Current version of pixman: 0.18.4 > Before reporting problems, check http://wiki.x.org > to make sure that you have the latest version. > Markers: (--) probed, (**) from config file, (==) default setting, > (++) from command line, (!!) notice, (II) informational, > (WW) warning, (EE) error, (NI) not implemented, (??) unknown. > (==) Log file: "/var/log/Xorg.0.log", Time: Fri Jan 28 11:32:46 2011 > (==) Using config file: "/etc/X11/xorg.conf" > NVIDIA: could not open the device file /dev/nvidiactl (No such file or directory). > (EE) Jan 28 11:32:46 NVIDIA(0): Failed to initialize the NVIDIA kernel module. Please see the > (EE) Jan 28 11:32:46 NVIDIA(0): system's kernel log for additional error messages and > (EE) Jan 28 11:32:46 NVIDIA(0): consult the NVIDIA README for details. > (EE) NVIDIA(0): *** Aborting *** > (EE) Screen(s) found, but none have a usable configuration. > > Fatal server error: > no screens found > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > There must be others out there who have been using this driver under 8.x, > which suggests that I must be uttering the wrong incantation (or that > devilfs is smarter and meaner than I). > > I have uploaded all relevant files to my ISP's website > > http://accima.com/members/dhesser/fbsd8.2P-nvidia-driver/ > > so that anyone willing to make suggestions may access the information. > > My primary question at this point is "Is this driver known to work under > 8.2 PRERELEASE?" > > The files on the website represent the most recent attempt, after a > "make update" at noon (PST8PDT) yesterday (Jan 28, 2011) followed > by the usual upgrade process, and a deinstall/reinstall of the nvidias > driver port. > > There is one further point of confusion. I've compared the list of > files which the README claims to install, and found that 3 or 4 files > related to "libvdpau.so" are not actually installed by tne driver > port. These files are installed by the "multimedia/libvdpau" port. > I have installed those files from that port, with no change in the > behavior of the driver port. Versions of these files are actually > included in x11/nvidia-driver/work/NVIDIA-FreeBSD-x86-256.53/obj, > but are not installed by the Makefile. I've tried those too, with > no change. I have currently re-installed the multimedia/libvdpau > files. > > Anyone know which is correct, or why the nvidia-driver port does > not explain? > > Note: the "hastd" problem has been fixed in stable, but now the > "jme" module fails to compile. > -- > Duane H. Hesser > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 22:00:02 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92E1D106564A for ; Fri, 28 Jan 2011 22:00:02 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-qy0-f182.google.com (mail-qy0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3F27E8FC17 for ; Fri, 28 Jan 2011 22:00:02 +0000 (UTC) Received: by qyk36 with SMTP id 36so3706761qyk.13 for ; Fri, 28 Jan 2011 14:00:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type; bh=09vf0QW4kYLLC0t4kXBXsBzsb8lwp+OcyWRBItXceXo=; b=QVudTDLO1v5A+d/JkW9FLE3nCzJjmoG8QDrO5KrvPdmwfEYjK8+fp3wvhH1RUnsmLF TKndePrgZH+v8lgEBLGpyI2GGGLrcoCN6m6YObCARkZ0UBLKJNk5zcntbyaq4056wVXU zh6rMfGxwm668ohGgv/0KY4ILL0bAPudDuzjE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=ECMEE/r9Jzg0wfQuBNWWHRKXdTbrdmnDGXlQvOt07R07cshFdk5xn0Si0pJMx7RNne hHFoGIaJGZD2dfD6yVagkrf3U/nz7aT5PAhTUPp9N2Q7GAVDrxRsSq2GRlFpCbLfqbGx XNwZtn2XUfMw/XedtpOwOvXTRbnEmuWZUThMw= Received: by 10.229.181.74 with SMTP id bx10mr3090427qcb.215.1296252001515; Fri, 28 Jan 2011 14:00:01 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Fri, 28 Jan 2011 13:59:21 -0800 (PST) In-Reply-To: <20110128211834.GA84881@tops.skynet.lt> References: <20110128152505.GP75125@dan.emsphone.com> <20110128211834.GA84881@tops.skynet.lt> From: Ivan Voras Date: Fri, 28 Jan 2011 22:59:21 +0100 X-Google-Sender-Auth: Cyk60N9fr4-oYyxA5k1S6QcMJqA Message-ID: To: Gleb Kurtsou Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Dan Nelson Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 22:00:02 -0000 On 28 January 2011 22:18, Gleb Kurtsou wrote: > You could try replacing rwlock with plain mutex to check if there are > priority propagation issues among readers/writers. How would that manifest? (i.e. how would it be detectable) > SX locks should also > work but would likely to be a considerable performance regression. With mutexes I'd lose all shared (read) acquisitions so I doubt sx locks would do much more harm :) > Finding out home much activity is there outside of storage/a/b/file > (sessions storage) could also be helpful. Here's more information: * The session storage (i.e. mostly file creates / writes in this particular workload) is on a separate file system than the core of the application (which only does reads) * The dtrace output I've send is from around thirty seconds of operation, so around 2000 PHP runs. (PHP in this case is FastCGI, so the processes are persistent instead of constantly respawning). In these 2000 runs there have been around 20,000 rw-block events in cache_lookup - which is strange. * Here's another dtrace output without rwlock mutex inlining, showing a different picture than what I've earlier thought: most rw-blocks events are in wlock! http://ivoras.net/stuff/rw-block-noinline.txt -- there are also some blocks without a rwlock function in the trace; I don't understand how rwlock inlining is implemented, maybe the readers are always inlined? Next step - find out how to make dtrace print files for which this happens. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 22:37:20 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A74C5106564A; Fri, 28 Jan 2011 22:37:20 +0000 (UTC) (envelope-from gleb.kurtsou@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0EC098FC13; Fri, 28 Jan 2011 22:37:19 +0000 (UTC) Received: by ewy24 with SMTP id 24so1840957ewy.13 for ; Fri, 28 Jan 2011 14:37:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:date:from:to:cc:subject:message-id:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=sCzvsduS9hHCAaP/zcFkCUp23gL43Z7xx+NFf+c89iY=; b=AfEWCiYBoMnFx/3LQFVJHd/ixeTkWQhI1Yagi9sZF+J5IFD+ZiMbGucXP1xEhaLDrv nb6AqnzJqNNLNaGXPtYcPIkFCCw2+Z6I4XTA8kA30dh1ovttG0Cl5AOCOl2s5PDBf/cP pcFnHMJOKAiZapfrPBR3ywwGvZQBhX3UkAbl4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=Prz5IFemvnUXES3bH654t/U7JvolFosBAW5BbHj/Ai+e2TjwT4ytuqTL4HaVzdiNvo F+q+ew1ZKn0pOXffewG7gmJ1QpFW0PAFJrZUPECmLEbBRrHHYVhGX/Ip4RCu0vQKaEUO QI1X24Tkoh6o3dEiMYixlcaduytQMI45wZsGU= Received: by 10.213.21.211 with SMTP id k19mr5536413ebb.66.1296254237254; Fri, 28 Jan 2011 14:37:17 -0800 (PST) Received: from localhost (lan-78-157-92-5.vln.skynet.lt [78.157.92.5]) by mx.google.com with ESMTPS id t5sm14195326eeh.20.2011.01.28.14.37.16 (version=SSLv3 cipher=RC4-MD5); Fri, 28 Jan 2011 14:37:16 -0800 (PST) Date: Sat, 29 Jan 2011 00:37:03 +0200 From: Gleb Kurtsou To: Ivan Voras Message-ID: <20110128223703.GA91590@tops.skynet.lt> References: <20110128152505.GP75125@dan.emsphone.com> <20110128211834.GA84881@tops.skynet.lt> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, Dan Nelson Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 22:37:20 -0000 On (28/01/2011 22:59), Ivan Voras wrote: > On 28 January 2011 22:18, Gleb Kurtsou wrote: > > > You could try replacing rwlock with plain mutex to check if there are > > priority propagation issues among readers/writers. > > How would that manifest? (i.e. how would it be detectable) Benchmark. If there are prio propagation issues mutex version should be faster than original locking. Let me know if you need help with patch. > > SX locks should also > > work but would likely to be a considerable performance regression. > > With mutexes I'd lose all shared (read) acquisitions so I doubt sx > locks would do much more harm :) Yeah, but sxlocks are more heavy weight. The question is what actual readers/writers ratio in your test is, e.g. all namecache entries for a dir may be purged on rename. > > Finding out home much activity is there outside of storage/a/b/file > > (sessions storage) could also be helpful. > > Here's more information: > > * The session storage (i.e. mostly file creates / writes in this > particular workload) is on a separate file system than the core of the > application (which only does reads) Changing namecache lock to become per-file system would hardly improve situation in general. > * The dtrace output I've send is from around thirty seconds of > operation, so around 2000 PHP runs. (PHP in this case is FastCGI, so > the processes are persistent instead of constantly respawning). In > these 2000 runs there have been around 20,000 rw-block events in > cache_lookup - which is strange. Are there rename, rmdir calls? - these purge namecache. If cache is empty, VOP_LOOKUP acquires write lock to populate the cache. > * Here's another dtrace output without rwlock mutex inlining, showing > a different picture than what I've earlier thought: most rw-blocks > events are in wlock! http://ivoras.net/stuff/rw-block-noinline.txt -- > there are also some blocks without a rwlock function in the trace; I > don't understand how rwlock inlining is implemented, maybe the readers > are always inlined? Add options RWLOCK_NOINLINE, recompiling with -O0 might also be good idea. > > Next step - find out how to make dtrace print files for which this happens. From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 28 22:46:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0DE8610656A6 for ; Fri, 28 Jan 2011 22:46:41 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id B15A38FC23 for ; Fri, 28 Jan 2011 22:46:40 +0000 (UTC) Received: by qwj9 with SMTP id 9so3791837qwj.13 for ; Fri, 28 Jan 2011 14:46:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:from :date:x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=XroTaRrO0JKhNBQeexwtf3O6PVo0t0+1l+pmCU7d9qg=; b=c8nUuRS9KMQ9L5wLBl4DNAqinzVLWDOzWIGvO5OYefq0u14P3OuCPi2ZQVBKeFJ0Ly Geid4pPdq/4J6F/RVQ1Rw/SCXWQkeDyji6lwrs+BW+43jszpGVv1EiXxkJYibGmJYcFN Og2JpjOWjD2t2I3VeSWrmGGGX4v29MsU7AGyE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=toI6Qc4nefF+CeCuVD4X0zMWxIbEuIBPTXmpeMjQR70CjqlgW7JB75S0v2Fu3uj+6S Hiw4zQ3LsirqEHF4lYaC+iWk9YlhatPj9igRszq1dr3w2OUtRA7EwvF9EoBly7tRxBJk h2UKqLO8h1iv+E9bfKgHxuFjD0EaMA3gOaPWw= Received: by 10.229.235.4 with SMTP id ke4mr3281972qcb.63.1296254799808; Fri, 28 Jan 2011 14:46:39 -0800 (PST) MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.229.67.2 with HTTP; Fri, 28 Jan 2011 14:45:59 -0800 (PST) In-Reply-To: <20110128223703.GA91590@tops.skynet.lt> References: <20110128152505.GP75125@dan.emsphone.com> <20110128211834.GA84881@tops.skynet.lt> <20110128223703.GA91590@tops.skynet.lt> From: Ivan Voras Date: Fri, 28 Jan 2011 23:45:59 +0100 X-Google-Sender-Auth: mgGpk7GmGWTBibmhA-vSLB79jP4 Message-ID: To: Gleb Kurtsou Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Dan Nelson Subject: Re: Namecache lock contention? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2011 22:46:41 -0000 On 28 January 2011 23:37, Gleb Kurtsou wrote: >> * The dtrace output I've send is from around thirty seconds of >> operation, so around 2000 PHP runs. (PHP in this case is FastCGI, so >> the processes are persistent instead of constantly respawning). In >> these 2000 runs there have been around 20,000 rw-block events in >> cache_lookup - which is strange. > Are there rename, rmdir calls? - these purge namecache. > If cache is empty, VOP_LOOKUP acquires write lock to populate the cache. No, only creates and deletes on files, no directory operations at all. >> * Here's another dtrace output without rwlock mutex inlining, showing >> a different picture than what I've earlier thought: most rw-blocks >> events are in wlock! http://ivoras.net/stuff/rw-block-noinline.txt =C2= =A0-- >> there are also some blocks without a rwlock function in the trace; I >> don't understand how rwlock inlining is implemented, maybe the readers >> are always inlined? > Add options RWLOCK_NOINLINE, recompiling with -O0 might also be good > idea. That's what I meant by "without rwlock mutex inlining". The default -O2 is enough - aggressive inlining only begins at -O3. From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 07:17:02 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E883106566B for ; Sat, 29 Jan 2011 07:17:02 +0000 (UTC) (envelope-from rank1seeker@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id D98B88FC08 for ; Sat, 29 Jan 2011 07:17:01 +0000 (UTC) Received: by fxm16 with SMTP id 16so4258903fxm.13 for ; Fri, 28 Jan 2011 23:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:message-id:from:to:subject:date:in-reply-to :references:x-mailer; bh=4/7M5D32RsZT+rP3TY6bONEYldaRnSOEmqYVcBGYOHw=; b=jBD6+1h1AsBB7ab51L3ZAQSh9jRh1rSPfPd7DbcM7EdlretICRmeuahcmWHkTrGTh8 EJ3/CqdE2icFRfeYxFPr6IuaCrPGU8dMyf9T8D67kPAme8S0mZEjO0tYbBzGX+zdg8sr GiHDOX7Gj56JD04Nzn01uOvuiUL7uGl0MQfno= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:subject:date:in-reply-to:references:x-mailer; b=TfovC3ssq7omLuywky6NELThkjnb0LzT+qLHdf9mU0CrDsfYQp0J1I/Vn8uZ/V3I9i DKVO74m8WmCnO37daRIvDGmSW7R8F5qmC66m4Zca7sUrzIujg19AS7ASGOWj2qUxxn/p KGQ/thHk+2nlnL2RWieIuZJYAOFHVGtUvBJl0= Received: by 10.223.83.144 with SMTP id f16mr3398010fal.4.1296283775318; Fri, 28 Jan 2011 22:49:35 -0800 (PST) Received: from DEV ([82.193.208.173]) by mx.google.com with ESMTPS id o17sm6701744fal.25.2011.01.28.22.49.32 (version=SSLv3 cipher=RC4-MD5); Fri, 28 Jan 2011 22:49:33 -0800 (PST) Message-ID: <20110129.064924.078.1@DEV> From: rank1seeker@gmail.com To: dhesser@accima.com, freebsd-hackers@freebsd.org Date: Sat, 29 Jan 2011 07:49:24 +0100 In-Reply-To: <20110128124312.a837f288.dhesser@accima.com> References: <20110128124312.a837f288.dhesser@accima.com> X-Mailer: POP Peeper (3.7.0.0) Cc: Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 07:17:02 -0000 ----- Original Message ----- From: "Duane H. Hesser" To: hackers@freebsd.org Date: Fri, 28 Jan 2011 12:43:12 -0800 Subject: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease > I am attempting ... BLA, BLA, ... > NVIDIA: could not open the device file /dev/nvidiactl (No such file or directory). > (EE) Jan 28 11:32:46 NVIDIA(0): Failed to initialize the NVIDIA kernel module. Please see the > (EE) Jan 28 11:32:46 NVIDIA(0): system's kernel log for additional error messages and > (EE) Jan 28 11:32:46 NVIDIA(0): consult the NVIDIA README for details. > (EE) NVIDIA(0): *** Aborting *** > (EE) Screen(s) found, but none have a usable configuration. > -- > Duane H. Hesser > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > echo 'debug.acpi.disabled="sysres"' >> /boot/loader.conf # reboot From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 17:42:16 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AADF0106566B for ; Sat, 29 Jan 2011 17:42:16 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5126C8FC08 for ; Sat, 29 Jan 2011 17:42:15 +0000 (UTC) Received: by fxm16 with SMTP id 16so4551900fxm.13 for ; Sat, 29 Jan 2011 09:42:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.111.137 with SMTP id s9mr209870fap.98.1296321615184; Sat, 29 Jan 2011 09:20:15 -0800 (PST) Received: by 10.223.118.84 with HTTP; Sat, 29 Jan 2011 09:20:15 -0800 (PST) Date: Sat, 29 Jan 2011 18:20:15 +0100 Message-ID: From: Damien Fleuriot To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 29 Jan 2011 17:55:46 +0000 Cc: freebsd-hackers@freebsd.org Subject: KERN - mfi driver for Dell raid h200 on r210 servers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 17:42:16 -0000 Hello lists, I'm trying to get FreeBSD 8.0 or 8.1 to run on a Dell poweredge r210 server. It ships with a SATA/SAS h200 RAID controller. Sadly, the MFI driver doesn't seem to register for this card... Find below the pciconf -lcvb -- none6@pci0:1:0:0: class=0x010700 card=0x1f1d1028 chip=0x00721000 rev=0x02 hdr=0x00 vendor = 'LSI Logic (Was: Symbios Logic, NCR)' class = mass storage subclass = SAS bar [10] = type I/O Port, range 32, base 0xfc00, size 256, enabled bar [14] = type Memory, range 64, base 0xdf2b0000, size 65536, enabled bar [1c] = type Memory, range 64, base 0xdf2c0000, size 262144, enabled cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0 cap 10[68] = PCI-Express 2 endpoint max data 256(4096) link x8(x8) cap 03[d0] = VPD cap 05[a8] = MSI supports 1 message, 64 bit cap 11[c0] = MSI-X supports 15 messages in map 0x14 -- I have added the following to /usr/src/sys/dev/mfi/mfi_pci.c at line 119: -- {0x1000, 0x0072, 0x1028, 0x1f1e, MFI_FLAGS_GEN2, "Dell PERC H200 Integrated"}, -- Rebuilt the kernel, tried it on the server, still no luck recognizing the RAID card. As a last resort I'll be setting the drives to passthrough and using a software RAID, but I'd much rather this worked out. Note that mfi actually recognizes Dell's h700 and h800 cards. Anyone managed to get the h200 card working yet ? @hackers: please keep me cc'd, I'm not subscribed to this list. Regards, -- dfl From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 18:15:21 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 093EC1065672; Sat, 29 Jan 2011 18:15:21 +0000 (UTC) (envelope-from jhelfman@e-e.com) Received: from mail.experts-exchange.com (mail.experts-exchange.com [72.29.183.251]) by mx1.freebsd.org (Postfix) with ESMTP id D98458FC0A; Sat, 29 Jan 2011 18:15:20 +0000 (UTC) Received: from mail.experts-exchange.com (localhost [127.0.0.1]) by mail.experts-exchange.com (Postfix) with ESMTP id DDD6A1DE8D21; Sat, 29 Jan 2011 10:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=e-e.com; h= content-transfer-encoding:content-type:content-type:mime-version :user-agent:from:from:subject:subject:date:date:references :in-reply-to:message-id:received:received:received; s=ee; t= 1296324012; x=1298138412; bh=ugNVhmCnGcvtt5LzkVhmJBgkRsro2ajAw7e yO9nkvxA=; b=JiZc1Nq1V2zrtV5eNd2DAAy3IDGC/Nbzmt1jCSByJ6HpafTnL5T s7z3TUY9E7zIR6YmUQb43mOKMVAaJtD+v6fhap1gHhsEapdJm7EXQm8DKdjHgUuJ opvp1lTUVgTgyTpB7usb/E/ahfTOwTwQ1SB+D6vREXuKOfIH4EAZhUxQ= X-Virus-Scanned: amavisd-new at experts-exchange.com Received: from mail.experts-exchange.com ([127.0.0.1]) by mail.experts-exchange.com (mail.experts-exchange.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i1ycnmm63HZF; Sat, 29 Jan 2011 10:00:12 -0800 (PST) Received: from mail.experts-exchange.com (localhost [127.0.0.1]) by mail.experts-exchange.com (Postfix) with ESMTP id 9D9A11DE8D16; Sat, 29 Jan 2011 10:00:12 -0800 (PST) Received: from 76.209.222.14 (SquirrelMail authenticated user jhelfman) by mail.experts-exchange.com with HTTP; Sat, 29 Jan 2011 10:00:12 -0800 Message-ID: <74e2d52b6ce67708a1bb42955b6370fb.squirrel@mail.experts-exchange.com> In-Reply-To: References: Date: Sat, 29 Jan 2011 10:00:12 -0800 From: jhelfman@e-e.com To: "Damien Fleuriot" User-Agent: SquirrelMail/1.4.20 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-hackers@freebsd.org, freebsd-stable@freebsd.org Subject: Re: KERN - mfi driver for Dell raid h200 on r210 servers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 18:15:21 -0000 > Hello lists, > > > > I'm trying to get FreeBSD 8.0 or 8.1 to run on a Dell poweredge r210 > server. > > It ships with a SATA/SAS h200 RAID controller. > > > Sadly, the MFI driver doesn't seem to register for this card... > > > Find below the pciconf -lcvb > > -- > none6@pci0:1:0:0: class=0x010700 card=0x1f1d1028 chip=0x00721000 > rev=0x02 hdr=0x00 > vendor = 'LSI Logic (Was: Symbios Logic, NCR)' > class = mass storage > subclass = SAS > bar [10] = type I/O Port, range 32, base 0xfc00, size 256, enabled > bar [14] = type Memory, range 64, base 0xdf2b0000, size 65536, > enabled > bar [1c] = type Memory, range 64, base 0xdf2c0000, size 262144, > enabled > cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0 > cap 10[68] = PCI-Express 2 endpoint max data 256(4096) link x8(x8) > cap 03[d0] = VPD > cap 05[a8] = MSI supports 1 message, 64 bit > cap 11[c0] = MSI-X supports 15 messages in map 0x14 > -- > > > I have added the following to /usr/src/sys/dev/mfi/mfi_pci.c at line 119: > -- > {0x1000, 0x0072, 0x1028, 0x1f1e, MFI_FLAGS_GEN2, "Dell PERC H200 > Integrated"}, > -- > > > Rebuilt the kernel, tried it on the server, still no luck recognizing > the RAID card. > > As a last resort I'll be setting the drives to passthrough and using a > software RAID, but I'd much rather this worked out. > > Note that mfi actually recognizes Dell's h700 and h800 cards. > > > > > Anyone managed to get the h200 card working yet ? > > > > > @hackers: please keep me cc'd, I'm not subscribed to this list. > > > > Regards, > > -- > dfl > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > Hi Damien, I would suspect that you would need to compile the 'mpt' driver to get this to work. I found the same issue in the r310, however I already had mpt compiled in my kernel for another hardware platform. Good Luck! Jason From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 19:20:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6585106566C for ; Sat, 29 Jan 2011 19:20:50 +0000 (UTC) (envelope-from dhesser@accima.com) Received: from mail.odessaoffice.com (mail.odessaoffice.com [64.146.146.8]) by mx1.freebsd.org (Postfix) with ESMTP id BF0FE8FC19 for ; Sat, 29 Jan 2011 19:20:50 +0000 (UTC) Received: from belinda.androcles.org ([::ffff:199.204.207.9]) (AUTH: CRAM-MD5 dhesser@accima.com, TLS: TLSv1/SSLv3, 256bits, AES256-SHA) by mail.odessaoffice.com with esmtp; Sat, 29 Jan 2011 11:10:49 -0800 id 0036801F.4D446639.00005BC4 Date: Sat, 29 Jan 2011 11:10:48 -0800 From: "Duane H. Hesser" To: freebsd-hackers@freebsd.org Message-Id: <20110129111048.653a8d6f.dhesser@accima.com> In-Reply-To: <20110129.064924.078.1@DEV> References: <20110128124312.a837f288.dhesser@accima.com> <20110129.064924.078.1@DEV> X-Mailer: Sylpheed 3.0.3 (GTK+ 2.22.1; i386-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dhesser@accima.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 19:20:50 -0000 On Sat, 29 Jan 2011 07:49:24 +0100 rank1seeker@gmail.com wrote: > > echo 'debug.acpi.disabled="sysres"' >> /boot/loader.conf > > # reboot I will try that ASAP, thanks. (I have compiled the port with and without acpi support, with no change). -- Duane H. Hesser From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 19:55:24 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC17F106566B for ; Sat, 29 Jan 2011 19:55:24 +0000 (UTC) (envelope-from andy@fud.org.nz) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 7FBA28FC15 for ; Sat, 29 Jan 2011 19:55:23 +0000 (UTC) Received: by wwf26 with SMTP id 26so4310526wwf.31 for ; Sat, 29 Jan 2011 11:55:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.227.128.82 with SMTP id j18mr4332696wbs.138.1296329178050; Sat, 29 Jan 2011 11:26:18 -0800 (PST) Sender: andy@fud.org.nz Received: by 10.227.156.149 with HTTP; Sat, 29 Jan 2011 11:26:18 -0800 (PST) In-Reply-To: References: Date: Sun, 30 Jan 2011 08:26:18 +1300 X-Google-Sender-Auth: 9nrRBfE_NtE4SscBE4kn7VsfNbo Message-ID: From: Andrew Thompson To: Damien Fleuriot Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org, freebsd-stable@freebsd.org Subject: Re: KERN - mfi driver for Dell raid h200 on r210 servers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 19:55:24 -0000 On 30 January 2011 06:20, Damien Fleuriot wrote: > Hello lists, > > > > I'm trying to get FreeBSD 8.0 or 8.1 to run on a Dell poweredge r210 server. > > It ships with a SATA/SAS h200 RAID controller. > This card may need the mps(4) driver which is only in HEAD at the moment. Andrew From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 20:35:31 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8D381065673 for ; Sat, 29 Jan 2011 20:35:31 +0000 (UTC) (envelope-from dhesser@accima.com) Received: from mail.odessaoffice.com (mail.odessaoffice.com [64.146.146.8]) by mx1.freebsd.org (Postfix) with ESMTP id D0F328FC18 for ; Sat, 29 Jan 2011 20:35:31 +0000 (UTC) Received: from belinda.androcles.org ([::ffff:199.204.207.9]) (AUTH: CRAM-MD5 dhesser@accima.com, TLS: TLSv1/SSLv3, 256bits, AES256-SHA) by mail.odessaoffice.com with esmtp; Sat, 29 Jan 2011 12:35:31 -0800 id 0036C012.4D447A13.00002553 Date: Sat, 29 Jan 2011 12:35:30 -0800 From: "Duane H. Hesser" To: rank1seeker@gmail.com Message-Id: <20110129123530.0f4586d8.dhesser@accima.com> In-Reply-To: <20110129.064924.078.1@DEV> References: <20110128124312.a837f288.dhesser@accima.com> <20110129.064924.078.1@DEV> X-Mailer: Sylpheed 3.0.3 (GTK+ 2.22.1; i386-portbld-freebsd8.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: dhesser@accima.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 20:35:32 -0000 On Sat, 29 Jan 2011 07:49:24 +0100 rank1seeker@gmail.com wrote: > > > echo 'debug.acpi.disabled="sysres"' >> /boot/loader.conf > > # reboot I added the above sysctl to loader.conf and rebooted. No change, but that is likely because no such sysctl exists in the stable kernel. Are you running current? # sysctl debug.acpi debug.acpi.suspend_bounce: 0 debug.acpi.reset_clock: 1 debug.acpi.do_powerstate: 1 debug.acpi.interpreter_slack: 1 debug.acpi.enable_debug_objects: 0 debug.acpi.acpi_ca_version: 20101013 debug.acpi.ec.timeout: 750 debug.acpi.ec.polled: 0 debug.acpi.ec.burst: 0 debug.acpi.batt.batt_sleep_ms: 0 debug.acpi.resume_beep: 0 Please let me take this opportunity to explain something (too everyone on this list who is willing to read it). My original post was not a plea for help. I appreciate the posts which have offered possible "tuning" items to resolve the problem, but my position is this: There is a significant regression in current stable sources which prevent the nvidia-driver port from initializing due to the absence of the /dev/nvidactl device. This is *probably* due to another bug in 'devfs'. I was willing to consider that I might have missed something in installing the port, but that is very unlikely...I've been doing this for a long time. I offered the opportunity for someone running a recent version of stable to report that they were running the nvidia-driver just fine, but that hasn't happened yet. Apparently it is working on current. It is NOT working on stable. I have examined the nvidia-driver port sources, and consider it unlikely that the problem is there (but again, ..could be wrong). Thanks again to those who have offered suggestions, but I submit that if a simple sysctl or the selection of port options determines whether device nodes are available, then 'things' have become far too complex. -- Duane H. Hesser From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 20:40:08 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DEEC4106564A for ; Sat, 29 Jan 2011 20:40:08 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 693F18FC12 for ; Sat, 29 Jan 2011 20:40:08 +0000 (UTC) Received: (qmail 19308 invoked by uid 399); 29 Jan 2011 20:40:07 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 29 Jan 2011 20:40:07 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D447B26.3030100@FreeBSD.org> Date: Sat, 29 Jan 2011 12:40:06 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20110129 Thunderbird/3.1.7 MIME-Version: 1.0 To: dhesser@accima.com References: <20110128124312.a837f288.dhesser@accima.com> <20110129.064924.078.1@DEV> <20110129123530.0f4586d8.dhesser@accima.com> In-Reply-To: <20110129123530.0f4586d8.dhesser@accima.com> X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 20:40:08 -0000 Sorry if I missed it, but I haven't seen an answer to the question of whether you've kldload'ed the nvidia module. When you type 'kldstat' does the nvidia module show up in the list? If not, type 'kldload nvidia' and try again. hth, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 20:51:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65E72106566C; Sat, 29 Jan 2011 20:51:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id C67828FC08; Sat, 29 Jan 2011 20:51:09 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p0TKp5iG009847 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 29 Jan 2011 22:51:05 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p0TKp5CP052051; Sat, 29 Jan 2011 22:51:05 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p0TKp5SF052050; Sat, 29 Jan 2011 22:51:05 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 29 Jan 2011 22:51:05 +0200 From: Kostik Belousov To: Juergen Lock Message-ID: <20110129205105.GI2518@deviant.kiev.zoral.com.ua> References: <20110129201000.GA10774@triton8.kn-bremen.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DaUEczEJLvNJIL0g" Content-Disposition: inline In-Reply-To: <20110129201000.GA10774@triton8.kn-bremen.de> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, freebsd-emulation@freebsd.org Subject: Re: Can vm_mmap()/vm_map_remove() be called with giant held? (linuxolator dvb patches) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 20:51:10 -0000 --DaUEczEJLvNJIL0g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 29, 2011 at 09:10:00PM +0100, Juergen Lock wrote: > Hi! >=20 > I was kinda hoping to be able to post a correct patch in public but > getting an answer to ${Subject} seems to be more difficult than I > thought... :) So, does anyone here know? copyout_map() and You do not need Giant locked for vm_map* functions. > copyout_unmap() are copied from ksyms_map() from sys/dev/ksyms/ksyms.c > - should there maybe be global versions instead of two static copies > each, and what would be good names? And giant is taken by linux_ioctl() Would you make a patch for this ? > in the same source file before calling the parts I added. So here > comes the patch, it is to add support for dvb ioctls to the linuxolator > as discussed on -emulation earlier in this thread: >=20 > http://lists.freebsd.org/pipermail/freebsd-multimedia/2011-January/01157= 5.html >=20 > (patch also at: >=20 > http://people.freebsd.org/~nox/dvb/linux-dvb.patch >=20 > and a version for 8, which is what I tested with w_scan on dvb-s2 > and dvb-t, and Andrew Gallatin also tested it with SageTV: >=20 > http://people.freebsd.org/~nox/dvb/linux-dvb-8.patch >=20 > ) >=20 > Thanx! > Juergen >=20 > Index: src/sys/compat/linux/linux_ioctl.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > RCS file: /home/scvs/src/sys/compat/linux/linux_ioctl.c,v > retrieving revision 1.167 > diff -u -p -r1.167 linux_ioctl.c > --- src/sys/compat/linux/linux_ioctl.c 30 Dec 2010 02:18:04 -0000 1.167 > +++ src/sys/compat/linux/linux_ioctl.c 18 Jan 2011 17:10:21 -0000 > @@ -59,6 +59,14 @@ __FBSDID("$FreeBSD: src/sys/compat/linux > #include > #include > #include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > =20 > #include > #include > @@ -83,6 +91,9 @@ __FBSDID("$FreeBSD: src/sys/compat/linux > #include > #include > =20 > +#include > +#include > + > CTASSERT(LINUX_IFNAMSIZ =3D=3D IFNAMSIZ); > =20 > static linux_ioctl_function_t linux_ioctl_cdrom; > @@ -97,6 +108,7 @@ static linux_ioctl_function_t linux_ioct > static linux_ioctl_function_t linux_ioctl_drm; > static linux_ioctl_function_t linux_ioctl_sg; > static linux_ioctl_function_t linux_ioctl_v4l; > +static linux_ioctl_function_t linux_ioctl_dvb; > static linux_ioctl_function_t linux_ioctl_special; > static linux_ioctl_function_t linux_ioctl_fbsd_usb; > =20 > @@ -124,6 +136,8 @@ static struct linux_ioctl_handler sg_han > { linux_ioctl_sg, LINUX_IOCTL_SG_MIN, LINUX_IOCTL_SG_MAX }; > static struct linux_ioctl_handler video_handler =3D > { linux_ioctl_v4l, LINUX_IOCTL_VIDEO_MIN, LINUX_IOCTL_VIDEO_MAX }; > +static struct linux_ioctl_handler dvb_handler =3D > +{ linux_ioctl_dvb, LINUX_IOCTL_DVB_MIN, LINUX_IOCTL_DVB_MAX }; > static struct linux_ioctl_handler fbsd_usb =3D > { linux_ioctl_fbsd_usb, FBSD_LUSB_MIN, FBSD_LUSB_MAX }; > =20 > @@ -139,6 +153,7 @@ DATA_SET(linux_ioctl_handler_set, privat > DATA_SET(linux_ioctl_handler_set, drm_handler); > DATA_SET(linux_ioctl_handler_set, sg_handler); > DATA_SET(linux_ioctl_handler_set, video_handler); > +DATA_SET(linux_ioctl_handler_set, dvb_handler); > DATA_SET(linux_ioctl_handler_set, fbsd_usb); > =20 > struct handler_element > @@ -2989,6 +3004,251 @@ linux_ioctl_special(struct thread *td, s > } > =20 > /* > + * Map some anonymous memory in user space of size sz, rounded up to the= page > + * boundary. > + */ > +static int > +copyout_map(struct thread *td, vm_offset_t *addr, size_t sz) > +{ > + struct vmspace *vms =3D td->td_proc->p_vmspace; Style recommends not to initialize in the declaration section. [I do not repeat systematic style violations notes below]. > + int error; > + vm_size_t size; > + > + One empty line is enough. > + /* > + * Map somewhere after heap in process memory. > + */ > + PROC_LOCK(td->td_proc); > + *addr =3D round_page((vm_offset_t)vms->vm_daddr + > + lim_max(td->td_proc, RLIMIT_DATA)); > + PROC_UNLOCK(td->td_proc); Are you sure that this is needed ? Why not leave the address selection to the VM ? > + > + /* round size up to page boundry */ > + size =3D (vm_size_t) round_page(sz); > + > + error =3D vm_mmap(&vms->vm_map, addr, size, PROT_READ | PROT_WRITE, > + VM_PROT_ALL, MAP_PRIVATE | MAP_ANON, OBJT_DEFAULT, NULL, 0); > + > + return (error); > +} > + > +/* > + * Unmap memory in user space. > + */ > +static int > +copyout_unmap(struct thread *td, vm_offset_t addr, size_t sz) > +{ > + vm_map_t map; > + vm_size_t size; > + > + map =3D &td->td_proc->p_vmspace->vm_map; > + size =3D (vm_size_t) round_page(sz); > + > + if (!vm_map_remove(map, addr, addr + size)) Better do if (vm_map_remove(...) !=3D KERN_SUCCESS) > + return (EINVAL); > + > + return (0); > +} > + > +static int > +linux_to_bsd_dtv_properties(struct l_dtv_properties *lvps, struct dtv_pr= operties *vps) > +{ Empty line between (empty) declaration section and executable statements. > + vps->num =3D lvps->num; > + vps->props =3D PTRIN(lvps->props); /* possible pointer size conversion = */ > + return (0); > +} > + > +static int > +linux_to_bsd_dtv_property(struct l_dtv_property *lvp, struct dtv_propert= y *vp) > +{ > + /* > + * everything until u.buffer.reserved2 is fixed size so > + * just memcpy it. > + */ > + memcpy(vp, lvp, offsetof(struct l_dtv_property, u.buffer.reserved2)); > + /* > + * the pointer may be garbage since it's part of a union, > + * currently no Linux code uses it so just set it to NULL. > + */ > + vp->u.buffer.reserved2 =3D NULL; > + vp->result =3D lvp->result; > + return (0); > +} > + > +static int > +bsd_to_linux_dtv_property(struct dtv_property *vp, struct l_dtv_property= *lvp) > +{ > + /* > + * everything until u.buffer.reserved2 is fixed size so > + * just memcpy it. > + */ > + memcpy(lvp, vp, offsetof(struct l_dtv_property, u.buffer.reserved2)); > + /* > + * the pointer may be garbage since it's part of a union, > + * currently no Linux code uses it so just set it to NULL. > + */ > + lvp->u.buffer.reserved2 =3D PTROUT(NULL); > + lvp->result =3D vp->result; > + return (0); > +} > + > +static int > +linux_ioctl_dvb(struct thread *td, struct linux_ioctl_args *args) > +{ > + struct file *fp; > + int error, i; > + struct l_dtv_properties l_vps; > + struct dtv_properties vps; > + struct l_dtv_property *l_vp =3D NULL, *l_p; > + struct dtv_property *vp =3D NULL, *p; > + size_t l_propsiz, propsiz; > + vm_offset_t uvp; > + > + switch (args->cmd & 0xffff) { > + case LINUX_AUDIO_STOP: > + case LINUX_AUDIO_PLAY: > + case LINUX_AUDIO_PAUSE: > + case LINUX_AUDIO_CONTINUE: > + case LINUX_AUDIO_SELECT_SOURCE: > + case LINUX_AUDIO_SET_MUTE: > + case LINUX_AUDIO_SET_AV_SYNC: > + case LINUX_AUDIO_SET_BYPASS_MODE: > + case LINUX_AUDIO_CHANNEL_SELECT: > + case LINUX_AUDIO_CLEAR_BUFFER: > + case LINUX_AUDIO_SET_ID: > + case LINUX_AUDIO_SET_STREAMTYPE: > + case LINUX_AUDIO_SET_EXT_ID: > + case LINUX_AUDIO_BILINGUAL_CHANNEL_SELECT: > + case LINUX_DMX_START: > + case LINUX_DMX_STOP: > + case LINUX_DMX_SET_BUFFER_SIZE: > + case LINUX_NET_REMOVE_IF: > + case LINUX_FE_DISEQC_RESET_OVERLOAD: > + case LINUX_FE_DISEQC_SEND_BURST: > + case LINUX_FE_SET_TONE: > + case LINUX_FE_SET_VOLTAGE: > + case LINUX_FE_ENABLE_HIGH_LNB_VOLTAGE: > + case LINUX_FE_DISHNETWORK_SEND_LEGACY_CMD: > + case LINUX_FE_SET_FRONTEND_TUNE_MODE: > + case LINUX_CA_RESET: > + if ((args->cmd & IOC_DIRMASK) !=3D LINUX_IOC_VOID) > + return ENOIOCTL; > + args->cmd =3D (args->cmd & 0xffff) | IOC_VOID; > + break; > + > + case LINUX_DMX_REMOVE_PID: > + /* overlaps with LINUX_NET_ADD_IF */ > + if ((args->cmd & IOC_DIRMASK) =3D=3D LINUX_IOC_INOUT) > + goto net_add_if; > + /* FALLTHRU */ > + case LINUX_AUDIO_SET_MIXER: > + case LINUX_AUDIO_SET_ATTRIBUTES: > + case LINUX_AUDIO_SET_KARAOKE: > + case LINUX_DMX_SET_FILTER: > + case LINUX_DMX_SET_PES_FILTER: > + case LINUX_DMX_SET_SOURCE: > + case LINUX_DMX_ADD_PID: > + case LINUX_FE_DISEQC_SEND_MASTER_CMD: > + case LINUX_FE_SET_FRONTEND: > + case LINUX_CA_SEND_MSG: > + case LINUX_CA_SET_DESCR: > + case LINUX_CA_SET_PID: > + args->cmd =3D (args->cmd & ~IOC_DIRMASK) | IOC_IN; > + break; > + > + case LINUX_AUDIO_GET_STATUS: > + case LINUX_AUDIO_GET_CAPABILITIES: > + case LINUX_AUDIO_GET_PTS: > + case LINUX_DMX_GET_PES_PIDS: > + case LINUX_DMX_GET_CAPS: > + case LINUX_FE_GET_INFO: > + case LINUX_FE_DISEQC_RECV_SLAVE_REPLY: > + case LINUX_FE_READ_STATUS: > + case LINUX_FE_READ_BER: > + case LINUX_FE_READ_SIGNAL_STRENGTH: > + case LINUX_FE_READ_SNR: > + case LINUX_FE_READ_UNCORRECTED_BLOCKS: > + case LINUX_FE_GET_FRONTEND: > + case LINUX_FE_GET_EVENT: > + case LINUX_CA_GET_CAP: > + case LINUX_CA_GET_SLOT_INFO: > + case LINUX_CA_GET_DESCR_INFO: > + case LINUX_CA_GET_MSG: > + args->cmd =3D (args->cmd & ~IOC_DIRMASK) | IOC_OUT; > + break; > + > + case LINUX_DMX_GET_STC: > + case LINUX_NET_GET_IF: > + net_add_if: > + args->cmd =3D (args->cmd & ~IOC_DIRMASK) | IOC_INOUT; > + break; > + > + case LINUX_FE_SET_PROPERTY: > + case LINUX_FE_GET_PROPERTY: > + error =3D copyin((void *) args->arg, &l_vps, sizeof(l_vps)); > + if (error) > + return (error); > + linux_to_bsd_dtv_properties(&l_vps, &vps); > + if ((vps.num =3D=3D 0) || vps.num > DTV_IOCTL_MAX_MSGS) > + return EINVAL; > + > + l_propsiz =3D vps.num * sizeof(*l_vp); > + propsiz =3D vps.num * sizeof(*vp); > + if (((l_vp =3D malloc(l_propsiz, M_LINUX, M_WAITOK)) =3D=3D NULL) || > + (vp =3D malloc(propsiz, M_LINUX, M_WAITOK)) =3D=3D NULL) { Malloc(M_WAITOK) is guaranteed to never return NULL, you do not need the checks. > + error =3D ENOMEM; > + goto out2; > + } > + error =3D copyin((void *) vps.props, l_vp, l_propsiz); > + if (error) > + goto out2; > + for (i =3D vps.num, l_p =3D l_vp, p =3D vp; i--; ++l_p, ++p) > + linux_to_bsd_dtv_property(l_p, p); > + > + error =3D copyout_map(td, &uvp, propsiz); > + if (error) > + goto out2; > + copyout(vp, (void *) uvp, propsiz); > + > + if ((error =3D fget(td, args->fd, &fp)) !=3D 0) { > + (void) copyout_unmap(td, uvp, propsiz); > + goto out2; > + } > + vps.props =3D (void *) uvp; > + if ((args->cmd & 0xffff) =3D=3D LINUX_FE_SET_PROPERTY) > + error =3D fo_ioctl(fp, FE_SET_PROPERTY, &vps, td->td_ucred, td); > + else > + error =3D fo_ioctl(fp, FE_GET_PROPERTY, &vps, td->td_ucred, td); > + if (error) { > + (void) copyout_unmap(td, uvp, propsiz); > + goto out; > + } > + error =3D copyin((void *) uvp, vp, propsiz); > + (void) copyout_unmap(td, uvp, propsiz); No need in space between cast and expression. Bigger issue is that I do not understand this fragment. You do copyout_map(), and then immediately call copyout_unmap() for the address range returned by copyout_map(), or am I missing something ? [snip] --DaUEczEJLvNJIL0g Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1EfbcACgkQC3+MBN1Mb4hvVgCgsrpJXvp28a9tqjPRfbXwysl3 CmkAn2hYTdD44azUhI6WfRHWhaOnrfKS =0ED8 -----END PGP SIGNATURE----- --DaUEczEJLvNJIL0g-- From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 21:18:40 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89C181065694 for ; Sat, 29 Jan 2011 21:18:40 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4CD298FC0A for ; Sat, 29 Jan 2011 21:18:39 +0000 (UTC) Received: by gxk8 with SMTP id 8so1629940gxk.13 for ; Sat, 29 Jan 2011 13:18:39 -0800 (PST) Received: by 10.150.225.7 with SMTP id x7mr6231064ybg.11.1296334340849; Sat, 29 Jan 2011 12:52:20 -0800 (PST) Received: from papi.localnet ([187.78.127.43]) by mx.google.com with ESMTPS id k1sm12459832ybj.0.2011.01.29.12.52.18 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 29 Jan 2011 12:52:20 -0800 (PST) From: Mario Lobo To: freebsd-hackers@freebsd.org Date: Sat, 29 Jan 2011 17:52:11 -0300 User-Agent: KMail/1.13.5 (FreeBSD/8.2-PRERELEASE; KDE/4.5.2; amd64; ; ) References: <20110128124312.a837f288.dhesser@accima.com> <20110129123530.0f4586d8.dhesser@accima.com> <4D447B26.3030100@FreeBSD.org> In-Reply-To: <4D447B26.3030100@FreeBSD.org> X-KMail-Markup: true MIME-Version: 1.0 Message-Id: <201101291752.12285.lobo@bsd.com.br> Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: NVIDIA (port) driver fails to create /dev/nvidactl; 8.2Prerelease X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 21:18:40 -0000 On Saturday 29 January 2011 17:40:06 Doug Barton wrote: > Sorry if I missed it, but I haven't seen an answer to the question of > whether you've kldload'ed the nvidia module. When you type 'kldstat' > does the nvidia module show up in the list? If not, type 'kldload > nvidia' and try again. > > > hth, > > Doug Why don't you try the driver from the nvidia site? It's always the latest version, compiles and install perfectly on i386 or amd64. -- Mario Lobo http://www.mallavoodoo.com.br FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] (99% winblows FREE) From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 21:58:04 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FFAA1065675 for ; Sat, 29 Jan 2011 21:58:04 +0000 (UTC) (envelope-from mwm-keyword-freebsdhackers2.e313df@mired.org) Received: from mired.org (three.mired.org [74.143.213.44]) by mx1.freebsd.org (Postfix) with ESMTP id B926F8FC20 for ; Sat, 29 Jan 2011 21:58:03 +0000 (UTC) Received: (qmail 79171 invoked by uid 501); 29 Jan 2011 16:31:20 -0500 Received: from bhuda.mired.org (192.168.195.96) by goat.mired.org (tmda-ofmipd) with ESMTP; Sat, 29 Jan 2011 16:31:20 -0500 Date: Sat, 29 Jan 2011 16:31:19 -0500 To: freebsd-hackers@freebsd.org Message-ID: <20110129163119.410349f1@bhuda.mired.org> In-Reply-To: References: Organization: Meyer Consulting X-Mailer: Claws Mail 3.7.7 (GTK+ 2.22.1; amd64-portbld-freebsd8.1) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAG1BMVEXguIzRkGnhyaz069mXhW0WHRnbrnR9WCQ6LB0CchNMAAACSUlEQVQ4jV2TQW7jMAxFGaPQOgQEdZaGMsgBrAvUA03dCxj1Uu4U2gfwQD7AGNax51NK07RcxXz6/CSl0Ij450vkPG1jzpIZM1UwDCl/xB14TWnNX8A00Qj5a0mnVFVbVUz4MeErea2HikSRqZzY894zwg9p2+/AtO8LzxFED+tNAUFeU29iFOLRxlZAcdo9A8wi8ZBMV4BKPde82Oxrvs6BTkulQIClte0DLFzzsKk9j1MBex8iUaP00Bd78S/muyFScrTXz6zLkEUxJp+SabQfNOs4f4Jpx5qSZ/304PWwlEWP1cOn/mJQR7EOD+uKhjcBLziuL7xoY5Xm+VFAUSw/LwwwsHEHxihpwV4EJH0xXRkbw1PkRw+X4pEuSJwBggqk+HEYKkiL5/74/nQkogigzQsAFrakxZyfw3wMIEEZPv4AWMfxwqE5GNxGaERjmH+PG8AE0L4/w9g0lsp1raLYAN5azQa+AOoO9NwcpFkTrG2VKNMNEL5UKUUAw34tha0z7onUG0oBoNtczE04GwFE3wCHc0ChezAJ6A1WMV81AtY7wDAJSlXwV+4cwBvsOsrQMRawfQEBz0deEZ7WNpV2szckIKo5VpDHDSDvF1GItwqqAlG01Hh50BGtVhuUkjkasg/14bYFGCgWg1fSWHvmOoJck2xdp9ZvZBHzDVTzX23TkrOn7qe5U2COEw5D4Vx3qEQpFY2Z/3QFnJxzp7YCmSMG19nOUoe869zZfOQb5ywQuWu0yCn5+8gxZz+BE7vG3j4/wbf4D/sXN9Wug1s7AAAAAElFTkSuQmCC Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Mike Meyer Subject: Re: Why not give git a try? (was "Re: [head tinderbox] failure on amd64/amd64") X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 21:58:04 -0000 Catching up on mail after a couple of weeks with the flue..... On Mon, 24 Jan 2011 00:13:46 -0800 Garrett Cooper wrote: > - The one caveat to cvsup/csup that's awesome is its componentization > capability, i.e. being able to selectively download components in src > / ports; I'm not 100% sure but there doesn't appear to be a clear > analog in git. It might be achievable through gits remote. in > git-config, git-remote, etc, but I would need to prototype whether or > not this is true. Since no one else mentioned it - mercurial handles this with subrepos. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 22:25:08 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80DBB106566C; Sat, 29 Jan 2011 22:25:08 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id 137DB8FC0C; Sat, 29 Jan 2011 22:25:08 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 16A0241C65E; Sat, 29 Jan 2011 23:25:07 +0100 (CET) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id nZwaXOgphtGP; Sat, 29 Jan 2011 23:25:06 +0100 (CET) Received: by mail.cksoft.de (Postfix, from userid 66) id 0833041C677; Sat, 29 Jan 2011 23:25:06 +0100 (CET) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id 45E4B4448F3; Sat, 29 Jan 2011 22:24:56 +0000 (UTC) Date: Sat, 29 Jan 2011 22:24:56 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Damien Fleuriot In-Reply-To: Message-ID: <20110129222400.R39951@maildrop.int.zabbadoz.net> References: X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-hackers@freebsd.org, freebsd-stable@freebsd.org Subject: Re: KERN - mfi driver for Dell raid h200 on r210 servers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 22:25:08 -0000 On Sat, 29 Jan 2011, Damien Fleuriot wrote: > Hello lists, > > > > I'm trying to get FreeBSD 8.0 or 8.1 to run on a Dell poweredge r210 server. > > It ships with a SATA/SAS h200 RAID controller. > > > Sadly, the MFI driver doesn't seem to register for this card... > > > Find below the pciconf -lcvb > > -- > none6@pci0:1:0:0: class=0x010700 card=0x1f1d1028 chip=0x00721000 > rev=0x02 hdr=0x00 > vendor = 'LSI Logic (Was: Symbios Logic, NCR)' > class = mass storage > subclass = SAS > bar [10] = type I/O Port, range 32, base 0xfc00, size 256, enabled > bar [14] = type Memory, range 64, base 0xdf2b0000, size 65536, enabled > bar [1c] = type Memory, range 64, base 0xdf2c0000, size 262144, enabled > cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0 > cap 10[68] = PCI-Express 2 endpoint max data 256(4096) link x8(x8) > cap 03[d0] = VPD > cap 05[a8] = MSI supports 1 message, 64 bit > cap 11[c0] = MSI-X supports 15 messages in map 0x14 > -- > > > I have added the following to /usr/src/sys/dev/mfi/mfi_pci.c at line 119: > -- > {0x1000, 0x0072, 0x1028, 0x1f1e, MFI_FLAGS_GEN2, "Dell PERC H200 Integrated"}, > -- It's 1f1d not 1f1e. I hope it was only a paste problem into email? > > Rebuilt the kernel, tried it on the server, still no luck recognizing > the RAID card. > > As a last resort I'll be setting the drives to passthrough and using a > software RAID, but I'd much rather this worked out. > > Note that mfi actually recognizes Dell's h700 and h800 cards. > > > > > Anyone managed to get the h200 card working yet ? > > > > > @hackers: please keep me cc'd, I'm not subscribed to this list. > > > > Regards, > > -- > dfl > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > -- Bjoern A. Zeeb You have to have visions! Going to jail sucks -- All my daemons like it! http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 23:19:28 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D127106564A for ; Sat, 29 Jan 2011 23:19:28 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id BBC258FC14 for ; Sat, 29 Jan 2011 23:19:27 +0000 (UTC) Received: by wwf26 with SMTP id 26so4399701wwf.31 for ; Sat, 29 Jan 2011 15:19:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=JaO5Kz+/HHiSb1sbUsZmtzx1WLGg8aXrp54NgDjNczE=; b=FJz2ihJtHhnkBx/qRiRyldehHbe9ysa12NpWKjfiNw3B/RpnUeXP9VmA04JzdqgxOc KL3s7ewcDi86BwgB7sD6uArdUZfRbHy7SHswFJ+86qBWitem2WPEelXorrWELbg73hYI G3XH6db5ZN302rqlyzzWMyEZ5YuASB1EixxpI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=SyWD/5BWrB2BTucAgWrvtXC7iyBgjowUXKzhltyEfUgUteYBZ9vJvcgWit9S5vtpJ1 FGVTblWZsfUxuA8mdR9mmBIxECts1x8RSBae/AXZSw+D5UGmp4B3FPhvRezOVlw39yal BH+Uy/7SXFY7RCs2VhYMwwQJuR2M6XSBOuiNg= MIME-Version: 1.0 Received: by 10.216.220.219 with SMTP id o69mr4649469wep.57.1296343166565; Sat, 29 Jan 2011 15:19:26 -0800 (PST) Sender: yanegomi@gmail.com Received: by 10.216.71.200 with HTTP; Sat, 29 Jan 2011 15:19:26 -0800 (PST) In-Reply-To: <20110129222400.R39951@maildrop.int.zabbadoz.net> References: <20110129222400.R39951@maildrop.int.zabbadoz.net> Date: Sat, 29 Jan 2011 15:19:26 -0800 X-Google-Sender-Auth: so6EqY1CeKrE4HqV9BbnNajFT0Q Message-ID: From: Garrett Cooper To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Damien Fleuriot , freebsd-stable@freebsd.org Subject: Re: KERN - mfi driver for Dell raid h200 on r210 servers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 23:19:28 -0000 On Sat, Jan 29, 2011 at 2:24 PM, Bjoern A. Zeeb wrote: > On Sat, 29 Jan 2011, Damien Fleuriot wrote: > >> Hello lists, >> >> >> >> I'm trying to get FreeBSD 8.0 or 8.1 to run on a Dell poweredge r210 >> server. >> >> It ships with a SATA/SAS h200 RAID controller. >> >> >> Sadly, the MFI driver doesn't seem to register for this card... >> >> >> Find below the pciconf -lcvb >> >> -- >> none6@pci0:1:0:0: =A0 =A0 =A0 class=3D0x010700 card=3D0x1f1d1028 chip=3D= 0x00721000 >> rev=3D0x02 hdr=3D0x00 >> =A0 vendor =A0 =A0 =3D 'LSI Logic (Was: Symbios Logic, NCR)' >> =A0 class =A0 =A0 =A0=3D mass storage >> =A0 subclass =A0 =3D SAS >> =A0 bar =A0 [10] =3D type I/O Port, range 32, base 0xfc00, size 256, ena= bled >> =A0 bar =A0 [14] =3D type Memory, range 64, base 0xdf2b0000, size 65536,= enabled >> =A0 bar =A0 [1c] =3D type Memory, range 64, base 0xdf2c0000, size 262144= , >> enabled >> =A0 cap 01[50] =3D powerspec 3 =A0supports D0 D1 D2 D3 =A0current D0 >> =A0 cap 10[68] =3D PCI-Express 2 endpoint max data 256(4096) link x8(x8) >> =A0 cap 03[d0] =3D VPD >> =A0 cap 05[a8] =3D MSI supports 1 message, 64 bit >> =A0 cap 11[c0] =3D MSI-X supports 15 messages in map 0x14 >> -- >> >> >> I have added the following to /usr/src/sys/dev/mfi/mfi_pci.c at line 119= : >> -- >> {0x1000, 0x0072, 0x1028, 0x1f1e, MFI_FLAGS_GEN2, =A0"Dell PERC H200 >> Integrated"}, >> -- > > It's 1f1d not 1f1e. =A0I hope it was only a paste problem into email? +1. PERC7 should be supported by mfi(4) (required bits may need to be added to mfi(4) though as you've pointed out, but additional details may need to be obtained from Dell). Thanks, -Garrett From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 29 20:46:04 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B1F4106564A for ; Sat, 29 Jan 2011 20:46:04 +0000 (UTC) (envelope-from nox@jelal.kn-bremen.de) Received: from smtp.kn-bremen.de (gelbbaer.kn-bremen.de [78.46.108.116]) by mx1.freebsd.org (Postfix) with ESMTP id A75658FC16 for ; Sat, 29 Jan 2011 20:46:03 +0000 (UTC) Received: by smtp.kn-bremen.de (Postfix, from userid 10) id 247401E000CB; Sat, 29 Jan 2011 21:28:45 +0100 (CET) Received: from triton8.kn-bremen.de (noident@localhost [127.0.0.1]) by triton8.kn-bremen.de (8.14.4/8.14.3) with ESMTP id p0TKA1Lb011702; Sat, 29 Jan 2011 21:10:01 +0100 (CET) (envelope-from nox@triton8.kn-bremen.de) Received: (from nox@localhost) by triton8.kn-bremen.de (8.14.4/8.14.3/Submit) id p0TKA1uR011701; Sat, 29 Jan 2011 21:10:01 +0100 (CET) (envelope-from nox) From: Juergen Lock Date: Sat, 29 Jan 2011 21:10:00 +0100 To: freebsd-hackers@FreeBSD.org Message-ID: <20110129201000.GA10774@triton8.kn-bremen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Mailman-Approved-At: Sat, 29 Jan 2011 23:42:06 +0000 Cc: freebsd-emulation@FreeBSD.org Subject: Can vm_mmap()/vm_map_remove() be called with giant held? (linuxolator dvb patches) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2011 20:46:04 -0000 Hi! I was kinda hoping to be able to post a correct patch in public but getting an answer to ${Subject} seems to be more difficult than I thought... :) So, does anyone here know? copyout_map() and copyout_unmap() are copied from ksyms_map() from sys/dev/ksyms/ksyms.c - should there maybe be global versions instead of two static copies each, and what would be good names? And giant is taken by linux_ioctl() in the same source file before calling the parts I added. So here comes the patch, it is to add support for dvb ioctls to the linuxolator as discussed on -emulation earlier in this thread: http://lists.freebsd.org/pipermail/freebsd-multimedia/2011-January/011575.html (patch also at: http://people.freebsd.org/~nox/dvb/linux-dvb.patch and a version for 8, which is what I tested with w_scan on dvb-s2 and dvb-t, and Andrew Gallatin also tested it with SageTV: http://people.freebsd.org/~nox/dvb/linux-dvb-8.patch ) Thanx! Juergen Index: src/sys/compat/linux/linux_ioctl.c =================================================================== RCS file: /home/scvs/src/sys/compat/linux/linux_ioctl.c,v retrieving revision 1.167 diff -u -p -r1.167 linux_ioctl.c --- src/sys/compat/linux/linux_ioctl.c 30 Dec 2010 02:18:04 -0000 1.167 +++ src/sys/compat/linux/linux_ioctl.c 18 Jan 2011 17:10:21 -0000 @@ -59,6 +59,14 @@ __FBSDID("$FreeBSD: src/sys/compat/linux #include #include #include +#include +#include +#include + +#include +#include +#include +#include #include #include @@ -83,6 +91,9 @@ __FBSDID("$FreeBSD: src/sys/compat/linux #include #include +#include +#include + CTASSERT(LINUX_IFNAMSIZ == IFNAMSIZ); static linux_ioctl_function_t linux_ioctl_cdrom; @@ -97,6 +108,7 @@ static linux_ioctl_function_t linux_ioct static linux_ioctl_function_t linux_ioctl_drm; static linux_ioctl_function_t linux_ioctl_sg; static linux_ioctl_function_t linux_ioctl_v4l; +static linux_ioctl_function_t linux_ioctl_dvb; static linux_ioctl_function_t linux_ioctl_special; static linux_ioctl_function_t linux_ioctl_fbsd_usb; @@ -124,6 +136,8 @@ static struct linux_ioctl_handler sg_han { linux_ioctl_sg, LINUX_IOCTL_SG_MIN, LINUX_IOCTL_SG_MAX }; static struct linux_ioctl_handler video_handler = { linux_ioctl_v4l, LINUX_IOCTL_VIDEO_MIN, LINUX_IOCTL_VIDEO_MAX }; +static struct linux_ioctl_handler dvb_handler = +{ linux_ioctl_dvb, LINUX_IOCTL_DVB_MIN, LINUX_IOCTL_DVB_MAX }; static struct linux_ioctl_handler fbsd_usb = { linux_ioctl_fbsd_usb, FBSD_LUSB_MIN, FBSD_LUSB_MAX }; @@ -139,6 +153,7 @@ DATA_SET(linux_ioctl_handler_set, privat DATA_SET(linux_ioctl_handler_set, drm_handler); DATA_SET(linux_ioctl_handler_set, sg_handler); DATA_SET(linux_ioctl_handler_set, video_handler); +DATA_SET(linux_ioctl_handler_set, dvb_handler); DATA_SET(linux_ioctl_handler_set, fbsd_usb); struct handler_element @@ -2989,6 +3004,251 @@ linux_ioctl_special(struct thread *td, s } /* + * Map some anonymous memory in user space of size sz, rounded up to the page + * boundary. + */ +static int +copyout_map(struct thread *td, vm_offset_t *addr, size_t sz) +{ + struct vmspace *vms = td->td_proc->p_vmspace; + int error; + vm_size_t size; + + + /* + * Map somewhere after heap in process memory. + */ + PROC_LOCK(td->td_proc); + *addr = round_page((vm_offset_t)vms->vm_daddr + + lim_max(td->td_proc, RLIMIT_DATA)); + PROC_UNLOCK(td->td_proc); + + /* round size up to page boundry */ + size = (vm_size_t) round_page(sz); + + error = vm_mmap(&vms->vm_map, addr, size, PROT_READ | PROT_WRITE, + VM_PROT_ALL, MAP_PRIVATE | MAP_ANON, OBJT_DEFAULT, NULL, 0); + + return (error); +} + +/* + * Unmap memory in user space. + */ +static int +copyout_unmap(struct thread *td, vm_offset_t addr, size_t sz) +{ + vm_map_t map; + vm_size_t size; + + map = &td->td_proc->p_vmspace->vm_map; + size = (vm_size_t) round_page(sz); + + if (!vm_map_remove(map, addr, addr + size)) + return (EINVAL); + + return (0); +} + +static int +linux_to_bsd_dtv_properties(struct l_dtv_properties *lvps, struct dtv_properties *vps) +{ + vps->num = lvps->num; + vps->props = PTRIN(lvps->props); /* possible pointer size conversion */ + return (0); +} + +static int +linux_to_bsd_dtv_property(struct l_dtv_property *lvp, struct dtv_property *vp) +{ + /* + * everything until u.buffer.reserved2 is fixed size so + * just memcpy it. + */ + memcpy(vp, lvp, offsetof(struct l_dtv_property, u.buffer.reserved2)); + /* + * the pointer may be garbage since it's part of a union, + * currently no Linux code uses it so just set it to NULL. + */ + vp->u.buffer.reserved2 = NULL; + vp->result = lvp->result; + return (0); +} + +static int +bsd_to_linux_dtv_property(struct dtv_property *vp, struct l_dtv_property *lvp) +{ + /* + * everything until u.buffer.reserved2 is fixed size so + * just memcpy it. + */ + memcpy(lvp, vp, offsetof(struct l_dtv_property, u.buffer.reserved2)); + /* + * the pointer may be garbage since it's part of a union, + * currently no Linux code uses it so just set it to NULL. + */ + lvp->u.buffer.reserved2 = PTROUT(NULL); + lvp->result = vp->result; + return (0); +} + +static int +linux_ioctl_dvb(struct thread *td, struct linux_ioctl_args *args) +{ + struct file *fp; + int error, i; + struct l_dtv_properties l_vps; + struct dtv_properties vps; + struct l_dtv_property *l_vp = NULL, *l_p; + struct dtv_property *vp = NULL, *p; + size_t l_propsiz, propsiz; + vm_offset_t uvp; + + switch (args->cmd & 0xffff) { + case LINUX_AUDIO_STOP: + case LINUX_AUDIO_PLAY: + case LINUX_AUDIO_PAUSE: + case LINUX_AUDIO_CONTINUE: + case LINUX_AUDIO_SELECT_SOURCE: + case LINUX_AUDIO_SET_MUTE: + case LINUX_AUDIO_SET_AV_SYNC: + case LINUX_AUDIO_SET_BYPASS_MODE: + case LINUX_AUDIO_CHANNEL_SELECT: + case LINUX_AUDIO_CLEAR_BUFFER: + case LINUX_AUDIO_SET_ID: + case LINUX_AUDIO_SET_STREAMTYPE: + case LINUX_AUDIO_SET_EXT_ID: + case LINUX_AUDIO_BILINGUAL_CHANNEL_SELECT: + case LINUX_DMX_START: + case LINUX_DMX_STOP: + case LINUX_DMX_SET_BUFFER_SIZE: + case LINUX_NET_REMOVE_IF: + case LINUX_FE_DISEQC_RESET_OVERLOAD: + case LINUX_FE_DISEQC_SEND_BURST: + case LINUX_FE_SET_TONE: + case LINUX_FE_SET_VOLTAGE: + case LINUX_FE_ENABLE_HIGH_LNB_VOLTAGE: + case LINUX_FE_DISHNETWORK_SEND_LEGACY_CMD: + case LINUX_FE_SET_FRONTEND_TUNE_MODE: + case LINUX_CA_RESET: + if ((args->cmd & IOC_DIRMASK) != LINUX_IOC_VOID) + return ENOIOCTL; + args->cmd = (args->cmd & 0xffff) | IOC_VOID; + break; + + case LINUX_DMX_REMOVE_PID: + /* overlaps with LINUX_NET_ADD_IF */ + if ((args->cmd & IOC_DIRMASK) == LINUX_IOC_INOUT) + goto net_add_if; + /* FALLTHRU */ + case LINUX_AUDIO_SET_MIXER: + case LINUX_AUDIO_SET_ATTRIBUTES: + case LINUX_AUDIO_SET_KARAOKE: + case LINUX_DMX_SET_FILTER: + case LINUX_DMX_SET_PES_FILTER: + case LINUX_DMX_SET_SOURCE: + case LINUX_DMX_ADD_PID: + case LINUX_FE_DISEQC_SEND_MASTER_CMD: + case LINUX_FE_SET_FRONTEND: + case LINUX_CA_SEND_MSG: + case LINUX_CA_SET_DESCR: + case LINUX_CA_SET_PID: + args->cmd = (args->cmd & ~IOC_DIRMASK) | IOC_IN; + break; + + case LINUX_AUDIO_GET_STATUS: + case LINUX_AUDIO_GET_CAPABILITIES: + case LINUX_AUDIO_GET_PTS: + case LINUX_DMX_GET_PES_PIDS: + case LINUX_DMX_GET_CAPS: + case LINUX_FE_GET_INFO: + case LINUX_FE_DISEQC_RECV_SLAVE_REPLY: + case LINUX_FE_READ_STATUS: + case LINUX_FE_READ_BER: + case LINUX_FE_READ_SIGNAL_STRENGTH: + case LINUX_FE_READ_SNR: + case LINUX_FE_READ_UNCORRECTED_BLOCKS: + case LINUX_FE_GET_FRONTEND: + case LINUX_FE_GET_EVENT: + case LINUX_CA_GET_CAP: + case LINUX_CA_GET_SLOT_INFO: + case LINUX_CA_GET_DESCR_INFO: + case LINUX_CA_GET_MSG: + args->cmd = (args->cmd & ~IOC_DIRMASK) | IOC_OUT; + break; + + case LINUX_DMX_GET_STC: + case LINUX_NET_GET_IF: + net_add_if: + args->cmd = (args->cmd & ~IOC_DIRMASK) | IOC_INOUT; + break; + + case LINUX_FE_SET_PROPERTY: + case LINUX_FE_GET_PROPERTY: + error = copyin((void *) args->arg, &l_vps, sizeof(l_vps)); + if (error) + return (error); + linux_to_bsd_dtv_properties(&l_vps, &vps); + if ((vps.num == 0) || vps.num > DTV_IOCTL_MAX_MSGS) + return EINVAL; + + l_propsiz = vps.num * sizeof(*l_vp); + propsiz = vps.num * sizeof(*vp); + if (((l_vp = malloc(l_propsiz, M_LINUX, M_WAITOK)) == NULL) || + (vp = malloc(propsiz, M_LINUX, M_WAITOK)) == NULL) { + error = ENOMEM; + goto out2; + } + error = copyin((void *) vps.props, l_vp, l_propsiz); + if (error) + goto out2; + for (i = vps.num, l_p = l_vp, p = vp; i--; ++l_p, ++p) + linux_to_bsd_dtv_property(l_p, p); + + error = copyout_map(td, &uvp, propsiz); + if (error) + goto out2; + copyout(vp, (void *) uvp, propsiz); + + if ((error = fget(td, args->fd, &fp)) != 0) { + (void) copyout_unmap(td, uvp, propsiz); + goto out2; + } + vps.props = (void *) uvp; + if ((args->cmd & 0xffff) == LINUX_FE_SET_PROPERTY) + error = fo_ioctl(fp, FE_SET_PROPERTY, &vps, td->td_ucred, td); + else + error = fo_ioctl(fp, FE_GET_PROPERTY, &vps, td->td_ucred, td); + if (error) { + (void) copyout_unmap(td, uvp, propsiz); + goto out; + } + error = copyin((void *) uvp, vp, propsiz); + (void) copyout_unmap(td, uvp, propsiz); + if (error) + goto out; + for (i = vps.num, l_p = l_vp, p = vp; i--; ++l_p, ++p) + bsd_to_linux_dtv_property(p, l_p); + linux_to_bsd_dtv_properties(&l_vps, &vps); + copyout(l_vp, (void *) vps.props, l_propsiz); + + out: + fdrop(fp, td); + out2: + if (l_vp) + free(l_vp, M_LINUX); + if (vp) + free(vp, M_LINUX); + return (error); + + default: return (ENOIOCTL); + } + + error = ioctl(td, (struct ioctl_args *)args); + return (error); +} + +/* * Support for emulators/linux-libusb. This port uses FBSD_LUSB* macros * instead of USB* ones. This lets us to provide correct values for cmd. * 0xffffffe0 -- 0xffffffff range seemed to be the least collision-prone. Index: src/sys/compat/linux/linux_ioctl.h =================================================================== RCS file: /home/scvs/src/sys/compat/linux/linux_ioctl.h,v retrieving revision 1.32 diff -u -p -r1.32 linux_ioctl.h --- src/sys/compat/linux/linux_ioctl.h 30 Dec 2010 02:18:04 -0000 1.32 +++ src/sys/compat/linux/linux_ioctl.h 18 Jan 2011 17:48:27 -0000 @@ -32,6 +32,17 @@ #define _LINUX_IOCTL_H_ /* + * ioctl + * + * XXX comments in Linux' indicate these + * could be arch-dependant... + */ +#define LINUX_IOC_VOID 0 +#define LINUX_IOC_IN 0x40000000 +#define LINUX_IOC_OUT 0x80000000 +#define LINUX_IOC_INOUT (LINUX_IOC_IN|LINUX_IOC_OUT) + +/* * disk */ #define LINUX_BLKROSET 0x125d @@ -613,6 +624,83 @@ int linux_ifname(struct ifnet *, char #define LINUX_IOCTL_VIDEO_MAX LINUX_VIDIOCSVBIFMT /* + * DVB (osd.h and video.h not handled) + */ +#define LINUX_AUDIO_STOP 0x6f01 /* 0x00006f01 */ +#define LINUX_AUDIO_PLAY 0x6f02 /* 0x00006f02 */ +#define LINUX_AUDIO_PAUSE 0x6f03 /* 0x00006f03 */ +#define LINUX_AUDIO_CONTINUE 0x6f04 /* 0x00006f04 */ +#define LINUX_AUDIO_SELECT_SOURCE 0x6f05 /* 0x00006f05 */ +#define LINUX_AUDIO_SET_MUTE 0x6f06 /* 0x00006f06 */ +#define LINUX_AUDIO_SET_AV_SYNC 0x6f07 /* 0x00006f07 */ +#define LINUX_AUDIO_SET_BYPASS_MODE 0x6f08 /* 0x00006f08 */ +#define LINUX_AUDIO_CHANNEL_SELECT 0x6f09 /* 0x00006f09 */ +#define LINUX_AUDIO_GET_STATUS 0x6f0a /* 0x80206f0a */ +#define LINUX_AUDIO_GET_CAPABILITIES 0x6f0b /* 0x80046f0b */ +#define LINUX_AUDIO_CLEAR_BUFFER 0x6f0c /* 0x00006f0c */ +#define LINUX_AUDIO_SET_ID 0x6f0d /* 0x00006f0d */ +#define LINUX_AUDIO_SET_MIXER 0x6f0e /* 0x40086f0e */ +#define LINUX_AUDIO_SET_STREAMTYPE 0x6f0f /* 0x00006f0f */ +#define LINUX_AUDIO_SET_EXT_ID 0x6f10 /* 0x00006f10 */ +#define LINUX_AUDIO_SET_ATTRIBUTES 0x6f11 /* 0x40026f11 */ +#define LINUX_AUDIO_SET_KARAOKE 0x6f12 /* 0x400c6f12 */ +#define LINUX_AUDIO_GET_PTS 0x6f13 /* 0x80086f13 */ +#define LINUX_AUDIO_BILINGUAL_CHANNEL_SELECT 0x6f14 /* 0x00006f14 */ +#define LINUX_DMX_START 0x6f29 /* 0x00006f29 */ +#define LINUX_DMX_STOP 0x6f2a /* 0x00006f2a */ +#define LINUX_DMX_SET_FILTER 0x6f2b /* 0x403c6f2b */ +#define LINUX_DMX_SET_PES_FILTER 0x6f2c /* 0x40146f2c */ +#define LINUX_DMX_SET_BUFFER_SIZE 0x6f2d /* 0x00006f2d */ +#define LINUX_DMX_GET_PES_PIDS 0x6f2f /* 0x800a6f2f */ +#define LINUX_DMX_GET_CAPS 0x6f30 /* 0x80086f30 */ +#define LINUX_DMX_SET_SOURCE 0x6f31 /* 0x40046f31 */ +#define LINUX_DMX_GET_STC 0x6f32 /* 0xc0106f32 */ +#define LINUX_DMX_ADD_PID 0x6f33 /* 0x40026f33 */ +#define LINUX_DMX_REMOVE_PID 0x6f34 /* 0x40026f34 */ +#define LINUX_FE_GET_INFO 0x6f3d /* 0x80a86f3d */ +#define LINUX_FE_DISEQC_RESET_OVERLOAD 0x6f3e /* 0x00006f3e */ +#define LINUX_FE_DISEQC_SEND_MASTER_CMD 0x6f3f /* 0x40076f3f */ +#define LINUX_FE_DISEQC_RECV_SLAVE_REPLY 0x6f40 /* 0x800c6f40 */ +#define LINUX_FE_DISEQC_SEND_BURST 0x6f41 /* 0x00006f41 */ +#define LINUX_FE_SET_TONE 0x6f42 /* 0x00006f42 */ +#define LINUX_FE_SET_VOLTAGE 0x6f43 /* 0x00006f43 */ +#define LINUX_FE_ENABLE_HIGH_LNB_VOLTAGE 0x6f44 /* 0x00006f44 */ +#define LINUX_FE_READ_STATUS 0x6f45 /* 0x80046f45 */ +#define LINUX_FE_READ_BER 0x6f46 /* 0x80046f46 */ +#define LINUX_FE_READ_SIGNAL_STRENGTH 0x6f47 /* 0x80026f47 */ +#define LINUX_FE_READ_SNR 0x6f48 /* 0x80026f48 */ +#define LINUX_FE_READ_UNCORRECTED_BLOCKS 0x6f49 /* 0x80046f49 */ +#define LINUX_FE_SET_FRONTEND 0x6f4c /* 0x40246f4c */ +#define LINUX_FE_GET_FRONTEND 0x6f4d /* 0x80246f4d */ +#define LINUX_FE_GET_EVENT 0x6f4e /* 0x80286f4e */ +#define LINUX_FE_DISHNETWORK_SEND_LEGACY_CMD 0x6f50 /* 0x00006f50 */ +#define LINUX_FE_SET_FRONTEND_TUNE_MODE 0x6f51 /* 0x00006f51 */ +#define LINUX_FE_SET_PROPERTY 0x6f52 /* 0x40086f52 */ +#define LINUX_FE_GET_PROPERTY 0x6f53 /* 0x80086f53 */ +#define LINUX_CA_RESET 0x6f80 /* 0x00006f80 */ +#define LINUX_CA_GET_CAP 0x6f81 /* 0x80106f81 */ +#define LINUX_CA_GET_SLOT_INFO 0x6f82 /* 0x800c6f82 */ +#define LINUX_CA_GET_DESCR_INFO 0x6f83 /* 0x80086f83 */ +#define LINUX_CA_GET_MSG 0x6f84 /* 0x810c6f84 */ +#define LINUX_CA_SEND_MSG 0x6f85 /* 0x410c6f85 */ +#define LINUX_CA_SET_DESCR 0x6f86 /* 0x40106f86 */ +#define LINUX_CA_SET_PID 0x6f87 /* 0x40086f87 */ + +/* + * DVB net.h + * (LINUX_NET_ADD_IF and LINUX___NET_ADD_IF_OLD overlap with + * LINUX_DMX_REMOVE_PID) + */ +#define LINUX_NET_ADD_IF 0x6f34 /* 0xc0066f34 */ +#define LINUX_NET_REMOVE_IF 0x6f35 /* 0x00006f35 */ +#define LINUX_NET_GET_IF 0x6f36 /* 0xc0066f36 */ +#define LINUX___NET_ADD_IF_OLD 0x6f34 /* 0xc0046f34 */ +#define LINUX___NET_GET_IF_OLD 0x6f36 /* 0xc0046f36 */ + +#define LINUX_IOCTL_DVB_MIN LINUX_AUDIO_STOP +#define LINUX_IOCTL_DVB_MAX LINUX_CA_SET_PID + +/* * Our libusb(8) calls emulated within linux(4). */ #define FBSD_LUSB_DEVICEENUMERATE 0xffff Index: src/sys/compat/linux/linux_dvb.h @@ -0,0 +1,63 @@ +/* + * Extracted from , which is: + * + * Copyright (C) 2000 Marcus Metzler + * Ralph Metzler + * Holger Waechtler + * Andre Draszik + * for convergence integrated media GmbH + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + */ + +#ifndef __LINUX_DVB_H +#define __LINUX_DVB_H + +#include + +struct dtv_property { + uint32_t cmd; + uint32_t reserved[3]; + union { + uint32_t data; + struct { + uint8_t data[32]; + uint32_t len; + uint32_t reserved1[3]; + void *reserved2; + } buffer; + } u; + int result; +} __attribute__ ((packed)); + +/* num of properties cannot exceed DTV_IOCTL_MAX_MSGS per ioctl */ +#define DTV_IOCTL_MAX_MSGS 64 + +struct dtv_properties { + uint32_t num; + struct dtv_property *props; +}; + +#define FE_SET_PROPERTY _IOW('o', 82, struct dtv_properties) +/* + * This is broken on linux as well but they workaround it in the driver. + * Since this is impossible to do on FreeBSD fix the header instead. + * Detailed and discussion : + * http://lists.freebsd.org/pipermail/freebsd-multimedia/2010-April/010958.html + */ +#define FE_GET_PROPERTY _IOW('o', 83, struct dtv_properties) + +#endif /*__LINUX_DVB_H*/ Index: src/sys/compat/linux/linux_dvb_compat.h @@ -0,0 +1,26 @@ +#ifndef __LINUX_DVB_COMPAT_H +#define __LINUX_DVB_COMPAT_H + +#include + +struct l_dtv_property { + uint32_t cmd; + uint32_t reserved[3]; + union { + uint32_t data; + struct { + uint8_t data[32]; + uint32_t len; + uint32_t reserved1[3]; + l_uintptr_t reserved2; + } buffer; + } u; + l_int result; +} __attribute__ ((packed)); + +struct l_dtv_properties { + uint32_t num; + l_uintptr_t props; +}; + +#endif /*__LINUX_DVB_H*/