From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 08:15:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C17F01065674; Sun, 30 May 2010 08:15:55 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-29.mx.aerioconnect.net [216.240.47.89]) by mx1.freebsd.org (Postfix) with ESMTP id 9F0278FC13; Sun, 30 May 2010 08:15:55 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o4U8FsWe018667; Sun, 30 May 2010 01:15:54 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 666D42D601B; Sun, 30 May 2010 01:15:53 -0700 (PDT) Message-ID: <4C021EC4.9070406@elischer.org> Date: Sun, 30 May 2010 01:16:04 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Garrett Cooper References: <4C0108F1.5050004@pathscale.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: FreeBSD-Hackers , =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= , dfr@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 08:15:55 -0000 On 5/29/10 12:53 PM, Garrett Cooper wrote: > 2010/5/29 "C. Bergström": >> >> PathScale is slowly open sourcing and porting some of our core software [...] >> Questions/comments - Say hi on #pathscale - irc.freenode.net or email me >> directly > > Hi Christopher, > Have you spoken with Doug Rabson about this? He has work which is > to going to meet a similar end as PathDB could provide. yes, Doug has a habit of going quiet for a while and coming back with a fully working piece of amazing work so I'm taking the fact that I haven't heard a peep out of him for a while as a good sign. It may be too late but it would be worth talking with him. > Also, it might be a good time and a good idea to consider going to > one of the BSD conferences and presenting PathDB as a useful > alternative to gdb (especially if folks provide some degree of From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 09:35:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CB141065676 for ; Sun, 30 May 2010 09:35:20 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-pz0-f174.google.com (mail-pz0-f174.google.com [209.85.222.174]) by mx1.freebsd.org (Postfix) with ESMTP id 687598FC13 for ; Sun, 30 May 2010 09:35:13 +0000 (UTC) Received: by pzk4 with SMTP id 4so192902pzk.7 for ; Sun, 30 May 2010 02:35:12 -0700 (PDT) Received: by 10.141.89.4 with SMTP id r4mr2100794rvl.79.1275212112763; Sun, 30 May 2010 02:35:12 -0700 (PDT) Received: from [192.168.20.101] ([119.42.85.48]) by mx.google.com with ESMTPS id h11sm3488927rvm.9.2010.05.30.02.35.08 (version=SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 02:35:11 -0700 (PDT) Message-ID: <4C023290.5050706@pathscale.com> Date: Sun, 30 May 2010 16:40:32 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 To: James Mansion References: <4C0108F1.5050004@pathscale.com> <20100529160155.GA3519@anja> <4C013E24.6020204@pathscale.com> <4C015CE5.1050602@feral.com> <4C01647A.4060903@pathscale.com> <4C022FDE.3060404@mansionfamily.plus.com> In-Reply-To: <4C022FDE.3060404@mansionfamily.plus.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org, Matthew Jacob Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 09:35:20 -0000 James Mansion wrote: > C. Bergström wrote: >> Apologies.. I didn't really expect anyone to know about it. To me >> the best way to describe it is similar to gdb, but much cleaner >> codebase. > Might be more effective to offer this to the llvm community, I would > have thought. While it would be interesting to integrate it with llvm we have our own compiler infrastructure already. fwiw.. I did ping Chris on irc and waiting on feedback for what they'd need in a debugger.. It's probably a piece of the puzzle they simply haven't had a chance to take a look at yet.. From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 09:58:10 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D4F91065675 for ; Sun, 30 May 2010 09:58:10 +0000 (UTC) (envelope-from james@mansionfamily.plus.com) Received: from relay.ptn-ipout02.plus.net (relay.ptn-ipout02.plus.net [212.159.7.36]) by mx1.freebsd.org (Postfix) with ESMTP id BE9198FC08 for ; Sun, 30 May 2010 09:58:09 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAPfMAUxUXebr/2dsb2JhbACeNHG8R4UWBA Received: from outmx07.plus.net ([84.93.230.235]) by relay.ptn-ipout02.plus.net with ESMTP; 30 May 2010 10:29:03 +0100 Received: from mansionfamily.plus.com ([80.229.150.39] helo=pd600.barnhouse) by outmx07.plus.net with esmtp (Exim) id 1OIepL-0007aG-6A; Sun, 30 May 2010 10:29:03 +0100 Received: from [192.168.0.132] (unknown [192.168.0.132]) by pd600.barnhouse (Postfix) with ESMTP id CD11E3A278B; Sun, 30 May 2010 10:29:07 +0100 (BST) Message-ID: <4C022FDE.3060404@mansionfamily.plus.com> Date: Sun, 30 May 2010 10:29:02 +0100 From: James Mansion User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= References: <4C0108F1.5050004@pathscale.com> <20100529160155.GA3519@anja> <4C013E24.6020204@pathscale.com> <4C015CE5.1050602@feral.com> <4C01647A.4060903@pathscale.com> In-Reply-To: <4C01647A.4060903@pathscale.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org, Matthew Jacob Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 09:58:10 -0000 C. Bergström wrote: > Apologies.. I didn't really expect anyone to know about it. To me the > best way to describe it is similar to gdb, but much cleaner codebase. Might be more effective to offer this to the llvm community, I would have thought. From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 11:58:17 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6A321065679 for ; Sun, 30 May 2010 11:58:17 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 5284C8FC17 for ; Sun, 30 May 2010 11:58:16 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OIh9i-0006uy-TR for freebsd-hackers@freebsd.org; Sun, 30 May 2010 13:58:14 +0200 Received: from 193.33.173.33 ([193.33.173.33]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 30 May 2010 13:58:14 +0200 Received: from c.kworr by 193.33.173.33 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sun, 30 May 2010 13:58:14 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org connect(): No such file or directory From: Volodymyr Kostyrko Date: Sun, 30 May 2010 14:58:05 +0300 Lines: 53 Message-ID: References: <4BDD28E2.8010201@rawbw.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 193.33.173.33 User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; ru-RU; rv:1.9.1.9) Gecko/20100428 Thunderbird/3.0.4 In-Reply-To: <4BDD28E2.8010201@rawbw.com> Cc: freebsd-ports@freebsd.org Subject: Re: GSoC: Making ports work with clang X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 11:58:17 -0000 02.05.2010 10:25, Yuri wrote: > Having tried clang++ I have a feeling that it's not quite ready to be a > generic c++ compiler. > It crashes a lot, fails on many quite simple c++ patterns. Very immature. > Don't you feel it's too early to start project like you are going to > given the state of clang with c++? This is just a question of time. Two month ago was the time after three month ago. And clang proceeds rapidly. > You will just keep stumbling upon various problems with various ports > and maybe will make 30% of c++ ports build with it at best. Let me spit out my own stats on ports. I have 574 ports installed, E17, xfce, postgresql etc. 1. 23% of _my_ ports doesn't require any compiler. They are fonts, script, meta-ports etc. 2. 60% of _my_ ports compiles and works with current devel/clang. This means I have only something like 17% ports to deal with. Above this ports major breakages were: 1. __dso not found after link. Some symbols seems to be omitted from libraries and linking of plugins fails badly. 2. Assembler errors. Xorg has some in x11-servers/xorg-server x11-drivers/xf86-video-vesa x11-drivers/xf86-video-ati, everything else compiles and works. 3. Big bunch of compile errors or config errors. This means incorrectly written code, like not correctly declaring variables. This also means some automake stupidities like testing c++ compiler with c style code - for example clang++ refuses to compile "int main(void) {}". 4. Some ports specify that thay need at least gcc 3.3. 5. Some ports needs --dumpspecs. And only small number of ports really require some real investigations. audio/libmad - distorted sound lang/python26 - compiling any gir dumps core textproc/expat2 - dbus dumps core at launch I think there are more. For example on clang-compiled ports gtk apps have problems with rendering final frame, but this is not a gtk failure, rather some lib gtk depends on. And this all data is not current. It's one month old. Since then dumpspecs was implemented. And maybe some other problems begone - I just have not enough time to look at this thoroughly. -- Sphinx of black quartz judge my vow. From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 12:51:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28AE5106566B; Sun, 30 May 2010 12:51:22 +0000 (UTC) (envelope-from hinokind@gmail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id 7E69B8FC0A; Sun, 30 May 2010 12:51:21 +0000 (UTC) Received: by ewy1 with SMTP id 1so666494ewy.13 for ; Sun, 30 May 2010 05:51:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:to:cc:subject :references:date:mime-version:content-transfer-encoding:from :message-id:in-reply-to:user-agent; bh=CaIdvbOOj6dMFDYpd/5PHOXGICAaSzRjfPL1UXPc+2E=; b=rnWcogW4ckR8NNRHJeWY/1M9B/CD5mNOyu5B2D+PkghW9UYRj7eO+nCpFsJM3E+gjM pPiu0DX9Wab80BHRwgTrvcdzA2qERsqK/XqRy+KV+E0k1iuPi9XhiV8akJuPffcrKs8G LgL34kzVy8p5eJUmJlVwSXk7fLR+ulqPU38GQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:to:cc:subject:references:date:mime-version :content-transfer-encoding:from:message-id:in-reply-to:user-agent; b=QDN8GAJYG7Fw7TwY3WrglcRuu4I4M62THmh4HTbN1w99z9FdXBVoKdNFSFJ05Ln2hc 1bOnle8QQsW6Iv7Kfy3POOyLYDuxZYPVmRya/A02XVvbZwdRovgW4LZBouWOrKVe+101 4SuZEW1f34i7TdhtEE+QHFuVP/UXOLSQCWPGs= Received: by 10.213.32.195 with SMTP id e3mr3859446ebd.14.1275223880278; Sun, 30 May 2010 05:51:20 -0700 (PDT) Received: from klevas (hst-17-80.splius.lt [77.79.17.80]) by mx.google.com with ESMTPS id 16sm2377647ewy.7.2010.05.30.05.51.18 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 05:51:19 -0700 (PDT) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Volodymyr Kostyrko" , freebsd-hackers@freebsd.org References: <4BDD28E2.8010201@rawbw.com> Date: Sun, 30 May 2010 15:51:17 +0300 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: =?utf-8?B?QW5kcml1cyBNb3JrxatuYXM=?= Message-ID: In-Reply-To: User-Agent: Opera Mail/10.10 (FreeBSD) Cc: freebsd-ports@freebsd.org Subject: Re: GSoC: Making ports work with clang X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 12:51:22 -0000 On Sun, 30 May 2010 14:58:05 +0300, Volodymyr Kostyrko wrote: > 1. __dso not found after link. Some symbols seems to be omitted from > libraries and linking of plugins fails badly. Known problem with known fix. > 2. Assembler errors. Xorg has some in x11-servers/xorg-server > x11-drivers/xf86-video-vesa x11-drivers/xf86-video-ati, everything else > compiles and works. Assembler errors often aren't similar to each other, so fixing them may be very easy or difficult. Hopefully we will fix them for big stuff like xorg (not really as part of this GSoC project). > 3. Big bunch of compile errors or config errors. This means incorrectly > written code, like not correctly declaring variables. This also means > some automake stupidities like testing c++ compiler with c style code - > for example clang++ refuses to compile "int main(void) {}". $ cat main.cc int main(void) {} $ clang main.cc -o test && ./test && echo "No, it works." No, it works. Other than that, yes, many problems are related to insane configure scripts. > 4. Some ports specify that thay need at least gcc 3.3. This is another of those insane configure scripts, testing for specific version of specific compiler, rather than testing if it can compile anything. > 5. Some ports needs --dumpspecs. It's a bit uglier than "some ports": $ grep dumpspecs /usr/ports/Mk/bsd.gecko.mk GECKO_PTHREAD_LIBS!=${CC} -dumpspecs | ${GREP} -m 1 pthread: | ${SED} -e 's|^.*%{\!pg: %{pthread:|| ; s|}.*$$||' || ${TRUE} > audio/libmad - distorted sound > lang/python26 - compiling any gir dumps core > textproc/expat2 - dbus dumps core at launch Python and expat shout i386 in my face, clang/llvm tends to not like i386 too much. But I think few miscompilations were fixed recently, so some of these may already be working fine. > And this all data is not current. It's one month old. Since then > dumpspecs was implemented. And maybe some other problems begone - I just > have not enough time to look at this thoroughly. Some problems from a month ago are definitely gone, but I don't think dumpspecs is one of them. -- Andrius From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 13:53:58 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D40A1065673; Sun, 30 May 2010 13:53:58 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id EB40C8FC08; Sun, 30 May 2010 13:53:57 +0000 (UTC) Received: from macfeast.lan (0x573b9942.cpe.ge-1-2-0-1101.ronqu1.customer.tele.dk [87.59.153.66]) by csmtp2.one.com (Postfix) with ESMTP id 085F91B013C7D; Sun, 30 May 2010 13:36:46 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: multipart/signed; boundary=Apple-Mail-149-566662397; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: Date: Sun, 30 May 2010 15:36:45 +0200 Message-Id: <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> References: <4BDD28E2.8010201@rawbw.com> To: =?utf-8?Q?Andrius_Mork=C5=ABnas?= X-Mailer: Apple Mail (2.1078) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org Subject: Re: GSoC: Making ports work with clang X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 13:53:58 -0000 --Apple-Mail-149-566662397 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Den 30/05/2010 kl. 14.51 skrev Andrius Mork=C5=ABnas: > On Sun, 30 May 2010 14:58:05 +0300, Volodymyr Kostyrko = wrote: >> 1. __dso not found after link. Some symbols seems to be omitted from >> libraries and linking of plugins fails badly. > Known problem with known fix. >=20 >> 2. Assembler errors. Xorg has some in x11-servers/xorg-server >> x11-drivers/xf86-video-vesa x11-drivers/xf86-video-ati, everything = else >> compiles and works. > Assembler errors often aren't similar to each other, so fixing them = may > be very easy or difficult. Hopefully we will fix them for big stuff = like > xorg (not really as part of this GSoC project). >=20 >> 3. Big bunch of compile errors or config errors. This means = incorrectly >> written code, like not correctly declaring variables. This also means >> some automake stupidities like testing c++ compiler with c style code = - >> for example clang++ refuses to compile "int main(void) {}". > $ cat main.cc > int main(void) {} > $ clang main.cc -o test && ./test && echo "No, it works." > No, it works. >=20 > Other than that, yes, many problems are related to insane configure > scripts. >=20 >> 4. Some ports specify that thay need at least gcc 3.3. > This is another of those insane configure scripts, testing for = specific > version of specific compiler, rather than testing if it can compile > anything. >=20 >> 5. Some ports needs --dumpspecs. > It's a bit uglier than "some ports": > $ grep dumpspecs /usr/ports/Mk/bsd.gecko.mk > GECKO_PTHREAD_LIBS!=3D${CC} -dumpspecs | ${GREP} -m 1 pthread: | = ${SED} -e 's|^.*%{\!pg: %{pthread:|| ; s|}.*$$||' || ${TRUE} >=20 >> audio/libmad - distorted sound >> lang/python26 - compiling any gir dumps core >> textproc/expat2 - dbus dumps core at launch > Python and expat shout i386 in my face, clang/llvm tends to not like > i386 too much. But I think few miscompilations were fixed recently, > so some of these may already be working fine. >=20 >> And this all data is not current. It's one month old. Since then >> dumpspecs was implemented. And maybe some other problems begone - I = just >> have not enough time to look at this thoroughly. > Some problems from a month ago are definitely gone, but I don't think > dumpspecs is one of them. Andrius, would it make sense to create e.g. a wiki page tracking the = status and current known problems with compiling ports with clang? Just = like there's a wiki page ClangBSD status. I think it would make it easier for lurkers to jump in and test things, = and help whittle away at the problems. Thanks, Erik= --Apple-Mail-149-566662397-- From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 14:05:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5ECEF106564A; Sun, 30 May 2010 14:05:59 +0000 (UTC) (envelope-from hinokind@gmail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id B8B8B8FC12; Sun, 30 May 2010 14:05:58 +0000 (UTC) Received: by ewy1 with SMTP id 1so674087ewy.13 for ; Sun, 30 May 2010 07:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:to:cc:subject :references:date:mime-version:content-transfer-encoding:from :message-id:in-reply-to:user-agent; bh=heUOhruh4KcHmeO+Tg4PCa8eCg5ADoekO5u802lsfWI=; b=rnwkvA/3lF3ZZ7D91G6BBqRV5NMBgDCCVzLCX/UGRMcZ6MSFTICOV9W6xfGq5rAcxp CBUiLZlELoUuKh1AcNDRMKcEWpSFfBbczM3ZtnjF3q7fMqRfo1K8LrOzY61qd0kQTSgY yO7VrcTrR8Fxa7gZdGkkhHpbSmpsx3OekAVRM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:to:cc:subject:references:date:mime-version :content-transfer-encoding:from:message-id:in-reply-to:user-agent; b=u6yHFsYNG4yTM5aWcvSYk7jQKSjKNkK1FI6RuZgfR7g4BK29GHzfFV0uF9FnVu1mlz Xvn85aPVEIz+/8o73QgJwzBrUbzmFCQCliqi26qr3e/zdk0Y344M7PQqW9DlIwG8uqfT X94E5t8dItU7zEwCVoOmei8pxRrqzTgL1b96s= Received: by 10.213.9.80 with SMTP id k16mr1658122ebk.84.1275228357583; Sun, 30 May 2010 07:05:57 -0700 (PDT) Received: from klevas (hst-17-80.splius.lt [77.79.17.80]) by mx.google.com with ESMTPS id 16sm2410825ewy.3.2010.05.30.07.05.56 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 07:05:57 -0700 (PDT) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Erik Cederstrand" References: <4BDD28E2.8010201@rawbw.com> <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> Date: Sun, 30 May 2010 17:05:55 +0300 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: =?utf-8?B?QW5kcml1cyBNb3JrxatuYXM=?= Message-ID: In-Reply-To: <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> User-Agent: Opera Mail/10.10 (FreeBSD) Cc: freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org Subject: Re: GSoC: Making ports work with clang X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 14:05:59 -0000 On Sun, 30 May 2010 16:36:45 +0300, Erik Cederstrand wrote: > Andrius, would it make sense to create e.g. a wiki page tracking the status and current known problems with compiling ports with clang? Just like there's a wiki page ClangBSD status. http://wiki.freebsd.org/PortsAndClang It doesn't have all the known problems, just some of them. > I think it would make it easier for lurkers to jump in and test things, and help whittle away at the problems. We will probably put some info there how we compile ports with clang in the near future. -- Andrius From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 14:09:37 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 156E81065672 for ; Sun, 30 May 2010 14:09:37 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id E6C868FC13 for ; Sun, 30 May 2010 14:09:36 +0000 (UTC) Received: by pwj4 with SMTP id 4so1536710pwj.13 for ; Sun, 30 May 2010 07:09:36 -0700 (PDT) Received: by 10.115.85.21 with SMTP id n21mr2447414wal.111.1275228576475; Sun, 30 May 2010 07:09:36 -0700 (PDT) Received: from [192.168.20.101] ([119.42.85.48]) by mx.google.com with ESMTPS id f11sm39996593wai.11.2010.05.30.07.09.34 (version=SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 07:09:36 -0700 (PDT) Message-ID: <4C0272E9.8020800@pathscale.com> Date: Sun, 30 May 2010 21:15:05 +0700 From: =?UTF-8?B?IkMuIEJlcmdzdHLDtm0i?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org References: <4BDD28E2.8010201@rawbw.com> <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> In-Reply-To: <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Permissive licensed toolchain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 14:09:37 -0000 Hi What's the real status of a fully permissive licensed toolchain? 1) Benchmarks - (I mean emperical evidence on FBSD and per target with no anecdotal comments or speculation.. I admit benchmarks can actually be misleading since many companies optimize for them specifically) 2) Has anyone tested clang++ with libc++ or stdcxx? (In my tests I hit some build problems with Apache stdcxx) 3) Which assembler is being used? 4) Which linker is being used? What's the best way to make a plan which will get feedback if someone wanted to try alternative approach to the above? Thanks ./C From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 15:10:48 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC8321065670 for ; Sun, 30 May 2010 15:10:48 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id C88688FC15 for ; Sun, 30 May 2010 15:10:48 +0000 (UTC) Received: by pxi7 with SMTP id 7so1512078pxi.13 for ; Sun, 30 May 2010 08:10:48 -0700 (PDT) Received: by 10.115.134.40 with SMTP id l40mr2482218wan.163.1275232247939; Sun, 30 May 2010 08:10:47 -0700 (PDT) Received: from [192.168.20.101] ([119.42.85.48]) by mx.google.com with ESMTPS id a23sm40422079wam.14.2010.05.30.08.10.46 (version=SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 08:10:47 -0700 (PDT) Message-ID: <4C028141.3030702@pathscale.com> Date: Sun, 30 May 2010 22:16:17 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 To: Matthias Apitz References: <4BDD28E2.8010201@rawbw.com> <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> <4C0272E9.8020800@pathscale.com> <20100530145729.GA3894@current.Sisis.de> In-Reply-To: <20100530145729.GA3894@current.Sisis.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: Permissive licensed toolchain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 15:10:49 -0000 Matthias Apitz wrote: > El día Sunday, May 30, 2010 a las 09:15:05PM +0700, "C. Bergström" escribió: > > >> Hi >> >> What's the real status of a fully permissive licensed toolchain? >> >> ... >> > > Please don't high-jack another tread with a new topic; your mail > contains: > > In-Reply-To: <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> > apologies.. I thought changing the topic would make a new thread.. (not all mailing lists track thread id) From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 15:41:55 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C4C351065672 for ; Sun, 30 May 2010 15:41:55 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from csmtp2.one.com (csmtp2.one.com [91.198.169.22]) by mx1.freebsd.org (Postfix) with ESMTP id 5E5758FC1E for ; Sun, 30 May 2010 15:41:55 +0000 (UTC) Received: from macfeast.lan (0x573b9942.cpe.ge-1-2-0-1101.ronqu1.customer.tele.dk [87.59.153.66]) by csmtp2.one.com (Postfix) with ESMTP id C5A661B01821F; Sun, 30 May 2010 15:41:53 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: multipart/signed; boundary=Apple-Mail-163-574170180; protocol="application/pkcs7-signature"; micalg=sha1 From: Erik Cederstrand In-Reply-To: <4C0272E9.8020800@pathscale.com> Date: Sun, 30 May 2010 17:41:53 +0200 Message-Id: <3A66374B-B6EE-4286-9575-BDA6261A0AFA@cederstrand.dk> References: <4BDD28E2.8010201@rawbw.com> <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> <4C0272E9.8020800@pathscale.com> To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.1078) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "=?iso-8859-1?Q?=22C._Bergstr=F6m=22?=" Subject: Re: Permissive licensed toolchain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 15:41:56 -0000 --Apple-Mail-163-574170180 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Den 30/05/2010 kl. 16.15 skrev C. Bergstr=F6m: > What's the real status of a fully permissive licensed toolchain? You mean ClangBSD? > 1) Benchmarks - (I mean emperical evidence on FBSD and per target with = no anecdotal comments or speculation.. I admit benchmarks can actually = be misleading since many companies optimize for them specifically) I'm working (slowly) on comparing FreeBSD and ClangBSD for various = benchmarks. > 3) Which assembler is being used? The same as FreeBSD: GNU as. There is ongoing work in the llvm-mc = project to provide an assembler for LLVM, but ELF support is low = priority for Apple. There was an experimental patch on the mailing list = a couple of weeks back. > 4) Which linker is being used? GNU ld. LLVM also provides a linker, but last time I checked it wasn't = functional on FreeBSD. It provides LTO that would be interesting to = benchmark. > What's the best way to make a plan which will get feedback if someone = wanted to try alternative approach to the above? Make it easily available, e.g. as a Subversion repo like ClangBSD, or an = install CD, and provide instructions for easy installation and test. I'm = testing ClangBSD in a VirtualBox VM which makes it easy to recover after = hosing the operating system. Erik= --Apple-Mail-163-574170180-- From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 15:42:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A85D9106570E for ; Sun, 30 May 2010 15:42:08 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms16-1.1blu.de (ms16-1.1blu.de [89.202.0.34]) by mx1.freebsd.org (Postfix) with ESMTP id 625878FC16 for ; Sun, 30 May 2010 15:42:08 +0000 (UTC) Received: from [88.217.55.75] (helo=current.Sisis.de) by ms16-1.1blu.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1OIk7o-0001Un-1k; Sun, 30 May 2010 17:08:28 +0200 Received: from current.Sisis.de (current [127.0.0.1]) by current.Sisis.de (8.14.3/8.14.3) with ESMTP id o4UF8Qv1004233; Sun, 30 May 2010 17:08:26 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by current.Sisis.de (8.14.3/8.14.3/Submit) id o4UEvqkm003926; Sun, 30 May 2010 16:57:52 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: current.Sisis.de: guru set sender to guru@unixarea.de using -f Date: Sun, 30 May 2010 16:57:30 +0200 From: Matthias Apitz To: =?iso-8859-1?Q?C=2E_Bergstr=F6m?= Message-ID: <20100530145729.GA3894@current.Sisis.de> References: <4BDD28E2.8010201@rawbw.com> <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> <4C0272E9.8020800@pathscale.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4C0272E9.8020800@pathscale.com> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 8.0-CURRENT (i386) X-Con-Id: 51246 X-Originating-IP: 88.217.55.75 Cc: freebsd-hackers@freebsd.org Subject: Re: Permissive licensed toolchain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Matthias Apitz List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 15:42:08 -0000 El día Sunday, May 30, 2010 a las 09:15:05PM +0700, "C. Bergström" escribió: > Hi > > What's the real status of a fully permissive licensed toolchain? > > ... Please don't high-jack another tread with a new topic; your mail contains: In-Reply-To: <45C1FA95-C9A3-41EA-9E3A-61E35C7F6AD1@cederstrand.dk> HIH matthias -- Matthias Apitz t +49-89-61308 351 - f +49-89-61308 399 - m +49-170-4527211 e - w http://www.unixarea.de/ ¡Ya basta! ¡Tropas de OTAN, fuera de Afghanistan! There's an end of it! NATO troups out of Afghanistan! Schluss jetzt endlich! NATO raus aus Afghanistan! From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 17:21:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53D8A1065672 for ; Sun, 30 May 2010 17:21:33 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 32ED88FC0A for ; Sun, 30 May 2010 17:21:32 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 382465B04; Sun, 30 May 2010 10:21:31 -0700 (PDT) To: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= In-reply-to: Your message of "Sun, 30 May 2010 01:27:12 +0700." <4C015C80.9050009@pathscale.com> References: <4C0108F1.5050004@pathscale.com> <20100529160155.GA3519@anja> <4C013E24.6020204@pathscale.com> <20100529165115.068AF5B58@mail.bitblocks.com> <4C0148A4.8060207@pathscale.com> <20100529175226.ECDC45B58@mail.bitblocks.com> <4C015C80.9050009@pathscale.com> Date: Sun, 30 May 2010 10:21:31 -0700 From: Bakul Shah Message-Id: <20100530172131.382465B04@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 17:21:35 -0000 [Added -hackers as this may be of some interest to others. Hope you don't mind] On Sun, 30 May 2010 01:27:12 +0700 =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= wrote: > ps. Tell me what you need to make it interesting and we'll try to make > it happen.. Ok, here are some "interesting" ideas! * Add a ups like interface and I will be very happy! http://ups.sourceforce.net/ Supposedly the following is needed to make it work on Linux. cvs -d:pserver:anonymous@ups.cvs.sourceforge.net:/cvsroot/ups login cvs -z3 -d:pserver:anonymous@ups.cvs.sourceforge.net:/cvsroot/ups co -P ups cd ups ./configure --enable-longlong I haven't tried this on linux but with a couple of patches it builds and runs fine on freebsd-i386. It has a built in C interpreter which is very handy; you can add C code at breakpoints for conditional bkpts or patch a variable etc. But the GUI is the best part -- I won't try explaining it, you have to experience it! Perhaps it can be used somehow? * multi-{thread,core,process,program,machine} debugging. A GUI can be very useful here as you can show state of each thread in a separate window, pop open a new window as threads or processes get created etc. Basically debugging distributed programming support! * A debugger language like say plan9 Acid's. With it for instance you can implement code coverage. See section 15 http://www.vitanuova.com/inferno/papers/acidpaper.html for the code coverage trick. Google "debugging with acid" to see more stuff. ups's c interpreter can be considered a debugger language. There is a good paper on a dataflow language for debugging from Brown U. http://www.cs.brown.edu/~sk/Publications/Papers/Published/mcskr-dataflow-lang-script-debug-journal/ * Better integration with testing. There are IDEs that integrate debugging with code development but I am not aware of any that integrates it well with testing. Testing is still a batch process. When a test fails, I want to dive right in, figure out what went wrong, fix it and rerun or continue! I admit I have the vaguest idea of even what this means. The dataflow lang paper refed may be relevant as it talks about automatic assertion checking etc. * There is a lot that can be done to improve debugging GUIs. For instance I'd love to see 3D use. Not for eye-candy but for better visualization. Things like as you zoom in, you see more details (zoom in on a function name and "enter" function body -- very Fantastic Voyage-ish (1966 movie) but for code!), control flow shows up as color change, the more a code path is visited the more it lights up or gets fatter, IPC is shown as a flash from one thread to another and so on. IIRC I have seen the color change idea in some verilog code coverage tools. -- bakul From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 17:28:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C5331065676 for ; Sun, 30 May 2010 17:28:50 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 581C68FC13 for ; Sun, 30 May 2010 17:28:50 +0000 (UTC) Received: by pvg16 with SMTP id 16so1457138pvg.13 for ; Sun, 30 May 2010 10:28:49 -0700 (PDT) Received: by 10.140.58.15 with SMTP id g15mr2412690rva.96.1275240529243; Sun, 30 May 2010 10:28:49 -0700 (PDT) Received: from [192.168.20.101] ([119.42.85.48]) by mx.google.com with ESMTPS id l29sm3601582rvb.4.2010.05.30.10.28.42 (version=SSLv3 cipher=RC4-MD5); Sun, 30 May 2010 10:28:47 -0700 (PDT) Message-ID: <4C02A195.7020109@pathscale.com> Date: Mon, 31 May 2010 00:34:13 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 To: Bakul Shah References: <4C0108F1.5050004@pathscale.com> <20100529160155.GA3519@anja> <4C013E24.6020204@pathscale.com> <20100529165115.068AF5B58@mail.bitblocks.com> <4C0148A4.8060207@pathscale.com> <20100529175226.ECDC45B58@mail.bitblocks.com> <4C015C80.9050009@pathscale.com> <20100530172131.382465B04@mail.bitblocks.com> In-Reply-To: <20100530172131.382465B04@mail.bitblocks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 17:28:50 -0000 Bakul Shah wrote: > [Added -hackers as this may be of some interest to others. > Hope you don't mind] > I don't mind at all.. > On Sun, 30 May 2010 01:27:12 +0700 =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= wrote: > > >> ps. Tell me what you need to make it interesting and we'll try to make >> it happen.. >> > > Ok, here are some "interesting" ideas! > > * Add a ups like interface and I will be very happy! > http://ups.sourceforce.net/ > (I think you meant http://ups.sourceforge.net/ ) ups is ugly based on the screenshoot on the homepage and it would be really cool if it had an ncurses based version.. (maybe it does?) Very nice feedback! Thanks ./C From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 17:53:30 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5CCE106566C for ; Sun, 30 May 2010 17:53:30 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id A956D8FC12 for ; Sun, 30 May 2010 17:53:30 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 44ECF5B04; Sun, 30 May 2010 10:53:29 -0700 (PDT) To: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= In-reply-to: Your message of "Mon, 31 May 2010 00:34:13 +0700." <4C02A195.7020109@pathscale.com> References: <4C0108F1.5050004@pathscale.com> <20100529160155.GA3519@anja> <4C013E24.6020204@pathscale.com> <20100529165115.068AF5B58@mail.bitblocks.com> <4C0148A4.8060207@pathscale.com> <20100529175226.ECDC45B58@mail.bitblocks.com> <4C015C80.9050009@pathscale.com> <20100530172131.382465B04@mail.bitblocks.com> <4C02A195.7020109@pathscale.com> Date: Sun, 30 May 2010 10:53:29 -0700 From: Bakul Shah Message-Id: <20100530175329.44ECF5B04@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 17:53:30 -0000 On Mon, 31 May 2010 00:34:13 +0700 =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= wrote: > Bakul Shah wrote: > > [Added -hackers as this may be of some interest to others. > > Hope you don't mind] > > > I don't mind at all.. > > On Sun, 30 May 2010 01:27:12 +0700 =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= > wrote: > > > > > >> ps. Tell me what you need to make it interesting and we'll try to make > >> it happen.. > >> > > > > Ok, here are some "interesting" ideas! > > > > * Add a ups like interface and I will be very happy! > > http://ups.sourceforce.net/ > > > (I think you meant http://ups.sourceforge.net/ ) Yes. I must've been thinking of Star Wars! > ups is ugly based on the screenshoot on the homepage and it would be > really cool if it had an ncurses based version.. (maybe it does?) Its visual interface is sparse (like twm or xfig) and reflects the era it was designed in. It is very mouse driven so ncurses wouldn't make sense. Its debugging capabilities are outstanding but you have to use it to see that. > Very nice feedback! Thanks. Hope it fired a few greycells! From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 19:10:47 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F340C106566C for ; Sun, 30 May 2010 19:10:46 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id A614E8FC12 for ; Sun, 30 May 2010 19:10:46 +0000 (UTC) Received: by vws10 with SMTP id 10so286382vws.13 for ; Sun, 30 May 2010 12:10:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=sTAqIJmgpFWESZO0ALlyWvaB7mSfZSCbiUN2L1ZL97U=; b=WTKgNrqxev0EvWnzfzdCSQKn6C3KBv/5UGYsbmZScYN+ruuXVUrMp6qc4PRCp/qQg4 MbT7HPWUzPXgHqEj9jAAzgtbw2LsQ3J62lxScAKZ5FU1+8IXRVfMF0bIWQCYrJj3w2TD ttuISba8NZq477oLxbDF3FHQn/YIUA+ZM8EjQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=dlZl52zDC4ZOagbGpa0bbJ+/3yxwRf6oIIi0Sq6xwLR9I5zX9+unZKHjv9YaPx3yix N7juRVLXabtVb52c6hosvN3hgJYBRT2SGLUbK1kQlwkMtTxKcuM6B/gfDfLvzLTZUBmv tZ9zLhA4kTfVOxzz+OWp0+9/u2EDtCVeH30ak= MIME-Version: 1.0 Received: by 10.224.61.148 with SMTP id t20mr1232266qah.249.1275246645801; Sun, 30 May 2010 12:10:45 -0700 (PDT) Received: by 10.229.88.140 with HTTP; Sun, 30 May 2010 12:10:45 -0700 (PDT) Date: Mon, 31 May 2010 00:40:45 +0530 Message-ID: From: Shrikanth Kamath To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: DTrace and CTF data X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 19:10:47 -0000 I have a query about the FBT provider in DTrace, does FBT provider not need the CTF info for the /kernel binary... I have this observation, when I disassemble say a 'kernel_funtion' , I see the function is not instrumented... (kgdb) disassemble kernel_function Dump of assembler code for function kernel_function: 0xc04aa05d : push %ebp 0xc04aa05e : mov %esp,%ebp 0xc04aa060 : sub $0x2c,%esp Now after I did a 'fbt::kernel_function:entry', if I disassemble the 'kernel_function'... (kgdb) disassemble kernel_function Dump of assembler code for function kernel_function: 0xc04aa05d : lock mov %esp,%ebp <== FBT instrumentation 0xc04aa060 : sub $0x2c,%esp If I do a 'objdump' on the kernel.debug binary I do not see any CTF sections showing up? -- Shrikanth R K From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 19:53:02 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB9BB106564A for ; Sun, 30 May 2010 19:53:02 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe13.swip.net [212.247.155.129]) by mx1.freebsd.org (Postfix) with ESMTP id 203418FC13 for ; Sun, 30 May 2010 19:53:01 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=AhDQJYhhhzEA:10 a=ygRHs6EKU7oA:10 a=M8b_wTzEtboA:10 a=MnI1ikcADjEx7bvsp0jZvQ==:17 a=vTFmDgYLpvtw9FmW2DQA:9 a=RsyZ5zk-IJ8qJ3UwVIMA:7 a=cgEiT6sBH8dpEG8uL33uf5sAVhsA:4 a=wPNLvfGTeEIA:10 a=zK40HWSAGXpK1S42oqQA:9 a=gnB4RclB4wQogc_v6mUA:7 a=PPrse_5yiIbdUBVEOCR7ouyfPNcA:4 a=WQ_67oLP1xdXm_gn:21 a=sLo_uR9Tib-Su8yZ:21 Received: from [188.126.201.140] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe13.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 1013166698 for freebsd-hackers@freebsd.org; Sun, 30 May 2010 21:53:00 +0200 To: freebsd-hackers@freebsd.org From: Hans Petter Selasky X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'( =?iso-8859-15?q?=3B=5FIjlA=3A=0A=09hGE=2E=2EEw?=, =?iso-8859-15?q?XAQ*o=23=5C/M=7ESC=3DS1-f9=7BEzRfT=27=7CHhll5Q=5Dha5Bt-s=7Co?= =?iso-8859-15?q?TlKMusi=3A1e=5BwJl=7Dkd=7DGR=0A=09Z0adGx-x=5F0zGbZj=27e?=(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' Date: Sun, 30 May 2010 21:50:15 +0200 MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_3FsAMIETJnfkjt6" Message-Id: <201005302150.15232.hselasky@c2i.net> Subject: [FreeBSD 8/9] [64-bit IOCTL] Using the USB stack from a 32-bit application under a 64-bit kernel. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 19:53:02 -0000 --Boundary-00=_3FsAMIETJnfkjt6 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hi, The USB team in FreeBSD has discussed this issue internally and would like some advice how to best resolve the 32 bit to 64 bit IOCTL conversion issue. Sometimes IOCTL requests contain userland pointers of type "void *". When compiled on a 64-bit OS, sizeof(void *) is 8 bytes and when compiled on a 32- bit OS sizeof(void *) is 4 bytes. This size difference makes it impossible to forward IOCTLs 1:1 from a 32-bit application running under a 64-bit kernel. This issue does not only apply to the USB subsystem, but also other kernel IOCTL interfaces. I suggest a more general solution: typedef uint64_t pvoid64; When an IOCTL needs to reference pointers in userspace, then they should always pad the pointer to 64-bit and use the pvoid64, which could have been a compiler type, so that we can easily convert to "void *" without using casts. When converting 32-bit userland pointers into 64-bit ones, there is no pointer conversion except for the unsigned expansion and resulting zero-padding. I assume this assumption will always be valid. Please find attached an example patch to make the USB stack in 8 and 9 fully 64-bit compatible. Any comments are welcome! --HPS --Boundary-00=_3FsAMIETJnfkjt6 Content-Type: text/plain; charset="iso-8859-1"; name="usb_64bit_patch.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="usb_64bit_patch.txt" --- src/lib/libusb/libusb20.c 2010-03-08 16:57:53.000000000 0000 +++ src/lib/libusb/libusb20.c 2010-03-08 16:57:53.000000000 0000 @@ -320,7 +320,7 @@ void libusb20_tr_set_buffer(struct libusb20_transfer *xfer, void *buffer, uint16_t frIndex) { - xfer->ppBuffer[frIndex] = buffer; + xfer->ppBuffer[frIndex] = libusb20_u64ptr(buffer); return; } @@ -386,7 +386,7 @@ void libusb20_tr_setup_bulk(struct libusb20_transfer *xfer, void *pBuf, uint32_t length, uint32_t timeout) { - xfer->ppBuffer[0] = pBuf; + xfer->ppBuffer[0] = libusb20_u64ptr(pBuf); xfer->pLength[0] = length; xfer->timeout = timeout; xfer->nFrames = 1; @@ -398,7 +398,7 @@ { uint16_t len; - xfer->ppBuffer[0] = psetup; + xfer->ppBuffer[0] = libusb20_u64ptr(psetup); xfer->pLength[0] = 8; /* fixed */ xfer->timeout = timeout; @@ -406,7 +406,7 @@ if (len != 0) { xfer->nFrames = 2; - xfer->ppBuffer[1] = pBuf; + xfer->ppBuffer[1] = libusb20_u64ptr(pBuf); xfer->pLength[1] = len; } else { xfer->nFrames = 1; @@ -417,7 +417,7 @@ void libusb20_tr_setup_intr(struct libusb20_transfer *xfer, void *pBuf, uint32_t length, uint32_t timeout) { - xfer->ppBuffer[0] = pBuf; + xfer->ppBuffer[0] = libusb20_u64ptr(pBuf); xfer->pLength[0] = length; xfer->timeout = timeout; xfer->nFrames = 1; @@ -431,7 +431,7 @@ /* should not happen */ return; } - xfer->ppBuffer[frIndex] = pBuf; + xfer->ppBuffer[frIndex] = libusb20_u64ptr(pBuf); xfer->pLength[frIndex] = length; return; } --- src/lib/libusb/libusb20_int.h 2010-03-08 16:57:53.000000000 0000 +++ src/lib/libusb/libusb20_int.h 2010-03-08 16:57:53.000000000 0000 @@ -31,6 +31,8 @@ #ifndef _LIBUSB20_INT_H_ #define _LIBUSB20_INT_H_ +#define libusb20_u64ptr(ptr) ((uint64_t)(uintptr_t)(ptr)) + struct libusb20_device; struct libusb20_backend; struct libusb20_transfer; @@ -148,7 +150,7 @@ /* * Pointer to a list of buffer pointers: */ - void **ppBuffer; + uint64_t *ppBuffer; /* * Pointer to frame lengths, which are updated to actual length * after the USB transfer completes: --- src/lib/libusb/libusb20_ugen20.c 2010-03-08 16:57:53.000000000 0000 +++ src/lib/libusb/libusb20_ugen20.c 2010-03-08 16:57:53.000000000 0000 @@ -763,8 +763,8 @@ xfer->maxPacketLen = temp.max_packet_length; /* setup buffer and length lists */ - fsep->ppBuffer = xfer->ppBuffer;/* zero copy */ - fsep->pLength = xfer->pLength; /* zero copy */ + fsep->ppBuffer = libusb20_u64ptr(xfer->ppBuffer); /* zero copy */ + fsep->pLength = libusb20_u64ptr(xfer->pLength); /* zero copy */ return (0); /* success */ } --- src/sys/dev/usb/usb_generic.c 2010-05-17 04:19:10.000000000 0000 +++ src/sys/dev/usb/usb_generic.c 2010-05-17 04:19:10.000000000 0000 @@ -76,6 +76,7 @@ #define UGEN_BULK_FS_BUFFER_SIZE (64*32) /* bytes */ #define UGEN_BULK_HS_BUFFER_SIZE (1024*32) /* bytes */ #define UGEN_HW_FRAMES 50 /* number of milliseconds per transfer */ +#define UGEN_HW_PTR(u64) ((void *)(uintptr_t)(u64)) /* function prototypes */ @@ -134,7 +135,6 @@ TUNABLE_INT("hw.usb.ugen.debug", &ugen_debug); #endif - /* prototypes */ static int @@ -1039,7 +1039,7 @@ struct usb_device_request *req; struct usb_xfer *xfer; struct usb_fs_endpoint fs_ep; - void *uaddr; /* userland pointer */ + uint64_t uaddr; /* userland pointer */ void *kaddr; usb_frlength_t offset; usb_frlength_t rem; @@ -1077,11 +1077,13 @@ xfer->error = USB_ERR_INVAL; goto complete; } - error = copyin(fs_ep.ppBuffer, + + /* read frame buffer pointer */ + error = copyin(UGEN_HW_PTR(fs_ep.ppBuffer), &uaddr, sizeof(uaddr)); - if (error) { + if (error) return (error); - } + /* reset first frame */ usbd_xfer_set_frame_offset(xfer, 0, 0); @@ -1089,7 +1091,8 @@ req = xfer->frbuffers[0].buffer; - error = copyin(fs_ep.pLength, + /* read frame buffer length */ + error = copyin(UGEN_HW_PTR(fs_ep.pLength), &length, sizeof(length)); if (error) { return (error); @@ -1099,7 +1102,7 @@ goto complete; } if (length != 0) { - error = copyin(uaddr, req, length); + error = copyin(UGEN_HW_PTR(uaddr), req, length); if (error) { return (error); } @@ -1159,11 +1162,12 @@ for (; n != xfer->nframes; n++) { - error = copyin(fs_ep.pLength + n, + /* read frame buffer length */ + error = copyin(UGEN_HW_PTR(fs_ep.pLength + (4 * n)), &length, sizeof(length)); - if (error) { + if (error) break; - } + usbd_xfer_set_frame_len(xfer, n, length); if (length > rem) { @@ -1174,12 +1178,12 @@ if (!isread) { - /* we need to know the source buffer */ - error = copyin(fs_ep.ppBuffer + n, + /* read frame buffer pointer */ + error = copyin(UGEN_HW_PTR(fs_ep.ppBuffer + (8 * n)), &uaddr, sizeof(uaddr)); - if (error) { + if (error) break; - } + if (xfer->flags_int.isochronous_xfr) { /* get kernel buffer address */ kaddr = xfer->frbuffers[0].buffer; @@ -1193,7 +1197,7 @@ } /* move data */ - error = copyin(uaddr, kaddr, length); + error = copyin(UGEN_HW_PTR(uaddr), kaddr, length); if (error) { break; } @@ -1216,7 +1220,7 @@ struct usb_xfer *xfer; struct usb_fs_endpoint fs_ep; struct usb_fs_endpoint *fs_ep_uptr; /* userland ptr */ - void *uaddr; /* userland ptr */ + uint64_t uaddr; /* userland ptr */ void *kaddr; usb_frlength_t offset; usb_frlength_t rem; @@ -1281,12 +1285,12 @@ for (; n != xfer->nframes; n++) { - /* get initial length into "temp" */ - error = copyin(fs_ep.pLength + n, + /* get initial frame buffer length into "temp" */ + error = copyin(UGEN_HW_PTR(fs_ep.pLength + (4 * n)), &temp, sizeof(temp)); - if (error) { + if (error) return (error); - } + if (temp > rem) { /* the userland length has been corrupted */ DPRINTF("corrupt userland length " @@ -1307,12 +1311,12 @@ } if (isread) { - /* we need to know the destination buffer */ - error = copyin(fs_ep.ppBuffer + n, + /* read destination frame buffer pointer */ + error = copyin(UGEN_HW_PTR(fs_ep.ppBuffer + (8 * n)), &uaddr, sizeof(uaddr)); - if (error) { + if (error) return (error); - } + if (xfer->flags_int.isochronous_xfr) { /* only one frame buffer */ kaddr = USB_ADD_BYTES( @@ -1323,7 +1327,7 @@ } /* move data */ - error = copyout(kaddr, uaddr, length); + error = copyout(kaddr, UGEN_HW_PTR(uaddr), length); if (error) { return (error); } @@ -1334,12 +1338,11 @@ */ offset += temp; - /* update length */ + /* update frame buffer length */ error = copyout(&length, - fs_ep.pLength + n, sizeof(length)); - if (error) { + UGEN_HW_PTR(fs_ep.pLength + (4 * n)), sizeof(length)); + if (error) return (error); - } } complete: --- src/sys/dev/usb/usb_ioctl.h 2010-02-14 12:03:51.000000000 0000 +++ src/sys/dev/usb/usb_ioctl.h 2010-02-14 12:03:51.000000000 0000 @@ -131,9 +131,10 @@ * NOTE: isochronous USB transfer only use one buffer, but can have * multiple frame lengths ! */ - void **ppBuffer; /* pointer to userland buffers */ - uint32_t *pLength; /* pointer to frame lengths, updated - * to actual length */ + uint64_t ppBuffer; /* pointer to 64-bit userland buffer pointers */ + uint64_t pLength; /* pointer to 32-bit frame lengths, which + * get updated to the actual length after + * the transfer is complete. */ uint32_t nFrames; /* number of frames */ uint32_t aFrames; /* actual number of frames */ uint16_t flags; --Boundary-00=_3FsAMIETJnfkjt6-- From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 20:51:54 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E9217106566B for ; Sun, 30 May 2010 20:51:54 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id BBFA88FC13 for ; Sun, 30 May 2010 20:51:54 +0000 (UTC) Received: by pwj4 with SMTP id 4so1631998pwj.13 for ; Sun, 30 May 2010 13:51:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=+Wnsb5Qo5wY8aFCrysdctx+sZJeL3RNVXxY9XpDowoE=; b=d27YFA1bkcdjTdN/wIw8BFvSlnTWMHFxu/2mJnN5CCmSV+C3BPdrKiVw44HjYhYPvD MN9FVfsj4B0GjN8ol/RCAufR09NH8D427ffGAHZPZuhRolR4IPADGj04CQg/baEoCH2Y 6brimeba00oaTAUh4b+1g78l9PXKHXBW8qsEM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=SDF0hQisaPTVDhdgX1GJPsoZyTOi9wAIXe0ofdHuTKNu0OPtZZDr5YH2V18FntfKU5 qqkHyIbPcZ+AP2ejvCIBRFeV6uFKefo7dVIfOCNps089OMvmTxDZtuahrATho+kCV2p5 RM6rQvnCaThrxHMun4LrjkhZ1aj3HV66aY8Hw= MIME-Version: 1.0 Received: by 10.140.251.13 with SMTP id y13mr2546522rvh.116.1275252714258; Sun, 30 May 2010 13:51:54 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.141.40.4 with HTTP; Sun, 30 May 2010 13:51:54 -0700 (PDT) In-Reply-To: References: Date: Sun, 30 May 2010 13:51:54 -0700 X-Google-Sender-Auth: b8-yEXdhwipqDdqVCgaEc5qqFTY Message-ID: From: Artem Belevich To: Shrikanth Kamath Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: DTrace and CTF data X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 20:51:55 -0000 You may be confusing CTF info and the probes. Those are different things. CTF info just describes data types and variable/function location. DTrace later uses it to figure out where and how to install the probes. Probes are installed dynamically, at runtime when DTrace program is run. That's why you see it with kgdb on the live kernel once probe has been installed. DTrace does not modify your original kernel binary, so you will not see any probes there. --Artem On Sun, May 30, 2010 at 12:10 PM, Shrikanth Kamath wrote: > I have a query about the FBT provider in DTrace, does FBT provider not ne= ed > the CTF info for the /kernel binary... > > I have this observation, when I disassemble say a 'kernel_funtion' , I se= e > the function is not instrumented... > > (kgdb) disassemble kernel_function > Dump of assembler code for function kernel_function: > 0xc04aa05d : push =A0 %ebp > 0xc04aa05e : mov =A0 =A0%esp,%ebp > 0xc04aa060 : sub =A0 =A0$0x2c,%esp > > > Now after I did a 'fbt::kernel_function:entry', if I disassemble the > 'kernel_function'... > > (kgdb) disassemble kernel_function > Dump of assembler code for function kernel_function: > 0xc04aa05d : lock mov %esp,%ebp =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 <=3D=3D FBT > instrumentation > 0xc04aa060 : sub =A0 =A0$0x2c,%esp > > If I do a 'objdump' on the kernel.debug binary I do not see any CTF secti= ons > showing up? > > > -- > Shrikanth R K > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 22:20:05 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54C73106566B for ; Sun, 30 May 2010 22:20:05 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id CE1198FC12 for ; Sun, 30 May 2010 22:20:04 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o4UMK95J083544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 31 May 2010 01:20:09 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o4UMJu9n051444; Mon, 31 May 2010 01:19:56 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o4UMJtMs051443; Mon, 31 May 2010 01:19:55 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 31 May 2010 01:19:55 +0300 From: Kostik Belousov To: Hans Petter Selasky Message-ID: <20100530221955.GN83316@deviant.kiev.zoral.com.ua> References: <201005302150.15232.hselasky@c2i.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UXm1vU+X4IBWGr3e" Content-Disposition: inline In-Reply-To: <201005302150.15232.hselasky@c2i.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: [FreeBSD 8/9] [64-bit IOCTL] Using the USB stack from a 32-bit application under a 64-bit kernel. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 22:20:05 -0000 --UXm1vU+X4IBWGr3e Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, May 30, 2010 at 09:50:15PM +0200, Hans Petter Selasky wrote: > Hi, >=20 > The USB team in FreeBSD has discussed this issue internally and would lik= e=20 > some advice how to best resolve the 32 bit to 64 bit IOCTL conversion iss= ue. >=20 > Sometimes IOCTL requests contain userland pointers of type "void *". When= =20 > compiled on a 64-bit OS, sizeof(void *) is 8 bytes and when compiled on a= 32- > bit OS sizeof(void *) is 4 bytes. This size difference makes it impossibl= e to=20 > forward IOCTLs 1:1 from a 32-bit application running under a 64-bit kerne= l. >=20 > This issue does not only apply to the USB subsystem, but also other kerne= l=20 > IOCTL interfaces. I suggest a more general solution: >=20 > typedef uint64_t pvoid64; >=20 > When an IOCTL needs to reference pointers in userspace, then they should= =20 > always pad the pointer to 64-bit and use the pvoid64, which could have be= en a=20 > compiler type, so that we can easily convert to "void *" without using ca= sts. >=20 >=20 > When converting 32-bit userland pointers into 64-bit ones, there is no po= inter=20 > conversion except for the unsigned expansion and resulting zero-padding. = I=20 > assume this assumption will always be valid. >=20 >=20 > Please find attached an example patch to make the USB stack in 8 and 9 fu= lly=20 > 64-bit compatible. >=20 >=20 > Any comments are welcome! >=20 >=20 > --HPS > --- src/sys/dev/usb/usb_ioctl.h 2010-02-14 12:03:51.000000000 0000 > +++ src/sys/dev/usb/usb_ioctl.h 2010-02-14 12:03:51.000000000 0000 > @@ -131,9 +131,10 @@ > * NOTE: isochronous USB transfer only use one buffer, but can have > * multiple frame lengths ! > */ > - void **ppBuffer; /* pointer to userland buffers */ > - uint32_t *pLength; /* pointer to frame lengths, updated > - * to actual length */ > + uint64_t ppBuffer; /* pointer to 64-bit userland buffer pointers */ > + uint64_t pLength; /* pointer to 32-bit frame lengths, which > + * get updated to the actual length after > + * the transfer is complete. */ > uint32_t nFrames; /* number of frames */ > uint32_t aFrames; /* actual number of frames */ > uint16_t flags; Doesn't this change the existing ABI for 32bit platforms ? You may take a look at the sys/net/bpf.c, where the similar issue is handled for bpf ioctls. To keep the ABI intact, you would need to define the 32bit ABI structures and define compat ioctls, then handle the ioctls by converting the structures and calling the native handler. BIOCSRTIMEOUT32 is a good example. --UXm1vU+X4IBWGr3e Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwC5IsACgkQC3+MBN1Mb4g34gCg3UIvYPAQHJhjmWKZDxs0XHD/ rnsAn3JDJB7ZI3nuOHgmPLkpUp8rGTpk =QEGc -----END PGP SIGNATURE----- --UXm1vU+X4IBWGr3e-- From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 22:25:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 500B81065674 for ; Sun, 30 May 2010 22:25:59 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe14.swip.net [212.247.155.161]) by mx1.freebsd.org (Postfix) with ESMTP id D144C8FC08 for ; Sun, 30 May 2010 22:25:58 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=2DJHWa-kBsMA:10 a=ygRHs6EKU7oA:10 a=Q9fys5e9bTEA:10 a=M8b_wTzEtboA:10 a=MnI1ikcADjEx7bvsp0jZvQ==:17 a=HDyF1E8vuEUzaRvr4OgA:9 a=i6fWCQaUSqujUsJnSCkA:7 a=GwLyV8Ce67O-N1nXbWj3w7BM7GYA:4 a=PUjeQqilurYA:10 Received: from [188.126.201.140] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe14.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 628261508; Mon, 31 May 2010 00:25:56 +0200 From: Hans Petter Selasky To: Kostik Belousov Date: Mon, 31 May 2010 00:23:11 +0200 User-Agent: KMail/1.12.4 (FreeBSD/8.0-STABLE; KDE/4.3.4; amd64; ; ) References: <201005302150.15232.hselasky@c2i.net> <20100530221955.GN83316@deviant.kiev.zoral.com.ua> In-Reply-To: <20100530221955.GN83316@deviant.kiev.zoral.com.ua> X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'(; _IjlA: hGE..Ew, XAQ*o#\/M~SC=S1-f9{EzRfT'|Hhll5Q]ha5Bt-s|oTlKMusi:1e[wJl}kd}GR Z0adGx-x_0zGbZj'e(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201005310023.11672.hselasky@c2i.net> Cc: freebsd-hackers@freebsd.org Subject: Re: [FreeBSD 8/9] [64-bit IOCTL] Using the USB stack from a 32-bit application under a 64-bit kernel. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 22:25:59 -0000 On Monday 31 May 2010 00:19:55 Kostik Belousov wrote: Hi, > Doesn't this change the existing ABI for 32bit platforms ? Yes, it changes the ABI. > > You may take a look at the sys/net/bpf.c, where the similar > issue is handled for bpf ioctls. To keep the ABI intact, you > would need to define the 32bit ABI structures and define > compat ioctls, then handle the ioctls by converting the structures > and calling the native handler. BIOCSRTIMEOUT32 is a good example. > The problem in the case of USB is that we have to resolve "void **" and not only "void *", in which case your solution would work. This is the case to reduce the copying between the kernel and user-space. --HPS From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 22:32:40 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D730D1065676 for ; Sun, 30 May 2010 22:32:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 4E6FF8FC14 for ; Sun, 30 May 2010 22:32:39 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o4UMWmek084226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 31 May 2010 01:32:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o4UMWZAf051550; Mon, 31 May 2010 01:32:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o4UMWYtd051549; Mon, 31 May 2010 01:32:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 31 May 2010 01:32:34 +0300 From: Kostik Belousov To: Hans Petter Selasky Message-ID: <20100530223234.GO83316@deviant.kiev.zoral.com.ua> References: <201005302150.15232.hselasky@c2i.net> <20100530221955.GN83316@deviant.kiev.zoral.com.ua> <201005310023.11672.hselasky@c2i.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JOUqYYSeXsqzCnuM" Content-Disposition: inline In-Reply-To: <201005310023.11672.hselasky@c2i.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: [FreeBSD 8/9] [64-bit IOCTL] Using the USB stack from a 32-bit application under a 64-bit kernel. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 22:32:40 -0000 --JOUqYYSeXsqzCnuM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 31, 2010 at 12:23:11AM +0200, Hans Petter Selasky wrote: > On Monday 31 May 2010 00:19:55 Kostik Belousov wrote: >=20 > Hi, >=20 > > Doesn't this change the existing ABI for 32bit platforms ? >=20 > Yes, it changes the ABI. We do trying hard to keep ABI stable both between and on branches. >=20 > >=20 > > You may take a look at the sys/net/bpf.c, where the similar > > issue is handled for bpf ioctls. To keep the ABI intact, you > > would need to define the 32bit ABI structures and define > > compat ioctls, then handle the ioctls by converting the structures > > and calling the native handler. BIOCSRTIMEOUT32 is a good example. > >=20 >=20 > The problem in the case of USB is that we have to resolve "void **" and n= ot=20 > only "void *", in which case your solution would work. This is the case t= o=20 > reduce the copying between the kernel and user-space. You have to do one more copyin() then. Newer kernel/user interfaces tend to use ints and explicitely sized integer types, like sys/user.h:struct kinfo_vmentry, so your proposal is sound and agrees with best practice, but for new interfaces. For the old interfaces, please do not break ABI. --JOUqYYSeXsqzCnuM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwC54IACgkQC3+MBN1Mb4hKMwCfQouIFzE3FZjPDLc/DrQbg8oy r1YAoNxwiUFZfLEBc4Ku1rsjtv+C2Q/k =U/Fw -----END PGP SIGNATURE----- --JOUqYYSeXsqzCnuM-- From owner-freebsd-hackers@FreeBSD.ORG Sun May 30 22:59:13 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E5521065678 for ; Sun, 30 May 2010 22:59:13 +0000 (UTC) (envelope-from andy@fud.org.nz) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1C2478FC16 for ; Sun, 30 May 2010 22:59:12 +0000 (UTC) Received: by vws10 with SMTP id 10so468892vws.13 for ; Sun, 30 May 2010 15:59:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.157.197 with SMTP id c5mr2812326vcx.0.1275258748420; Sun, 30 May 2010 15:32:28 -0700 (PDT) Sender: andy@fud.org.nz Received: by 10.220.90.77 with HTTP; Sun, 30 May 2010 15:32:28 -0700 (PDT) In-Reply-To: <20100530221955.GN83316@deviant.kiev.zoral.com.ua> References: <201005302150.15232.hselasky@c2i.net> <20100530221955.GN83316@deviant.kiev.zoral.com.ua> Date: Mon, 31 May 2010 10:32:28 +1200 X-Google-Sender-Auth: tMIhuUddZdyXZgecvnXG631ktLE Message-ID: From: Andrew Thompson To: Kostik Belousov Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Hans Petter Selasky Subject: Re: [FreeBSD 8/9] [64-bit IOCTL] Using the USB stack from a 32-bit application under a 64-bit kernel. X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 22:59:13 -0000 On 31 May 2010 10:19, Kostik Belousov wrote: > On Sun, May 30, 2010 at 09:50:15PM +0200, Hans Petter Selasky wrote: >> Hi, >> >> The USB team in FreeBSD has discussed this issue internally and would li= ke >> some advice how to best resolve the 32 bit to 64 bit IOCTL conversion is= sue. >> >> Sometimes IOCTL requests contain userland pointers of type "void *". Whe= n >> compiled on a 64-bit OS, sizeof(void *) is 8 bytes and when compiled on = a 32- >> bit OS sizeof(void *) is 4 bytes. This size difference makes it impossib= le to >> forward IOCTLs 1:1 from a 32-bit application running under a 64-bit kern= el. >> >> This issue does not only apply to the USB subsystem, but also other kern= el >> IOCTL interfaces. I suggest a more general solution: >> >> typedef uint64_t pvoid64; >> >> When an IOCTL needs to reference pointers in userspace, then they should >> always pad the pointer to 64-bit and use the pvoid64, which could have b= een a >> compiler type, so that we can easily convert to "void *" without using c= asts. >> >> >> When converting 32-bit userland pointers into 64-bit ones, there is no p= ointer >> conversion except for the unsigned expansion and resulting zero-padding.= I >> assume this assumption will always be valid. >> >> >> Please find attached an example patch to make the USB stack in 8 and 9 f= ully >> 64-bit compatible. >> >> >> Any comments are welcome! >> >> >> --HPS >> --- src/sys/dev/usb/usb_ioctl.h =A0 =A0 =A0 2010-02-14 12:03:51.00000000= 0 0000 >> +++ src/sys/dev/usb/usb_ioctl.h =A0 =A0 =A0 2010-02-14 12:03:51.00000000= 0 0000 >> @@ -131,9 +131,10 @@ >> =A0 =A0 =A0 =A0* NOTE: isochronous USB transfer only use one buffer, but= can have >> =A0 =A0 =A0 =A0* multiple frame lengths ! >> =A0 =A0 =A0 =A0*/ >> - =A0 =A0 void =A0**ppBuffer; =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* pointer to = userland buffers */ >> - =A0 =A0 uint32_t *pLength; =A0 =A0 =A0 =A0 =A0 =A0 =A0/* pointer to fr= ame lengths, updated >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0* to actual length */ >> + =A0 =A0 uint64_t ppBuffer; =A0 =A0 =A0 =A0 =A0 =A0 =A0/* pointer to 64= -bit userland buffer pointers */ >> + =A0 =A0 uint64_t pLength; =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* pointer to 32= -bit frame lengths, which >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0* get updated to the actual length after >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0* the transfer is complete. */ >> =A0 =A0 =A0 uint32_t nFrames; =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* number of f= rames */ >> =A0 =A0 =A0 uint32_t aFrames; =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* actual numb= er of frames */ >> =A0 =A0 =A0 uint16_t flags; > > Doesn't this change the existing ABI for 32bit platforms ? > > You may take a look at the sys/net/bpf.c, where the similar > issue is handled for bpf ioctls. To keep the ABI intact, you > would need to define the 32bit ABI structures and define > compat ioctls, then handle the ioctls by converting the structures > and calling the native handler. BIOCSRTIMEOUT32 is a good example. This has been done for other usb ioctls but the above struct has a pointer to an array of pointers in userland which makes it difficult. It isnt copied in with the ioctl so doesnt get the chance to be fixed up. so far, http://people.freebsd.org/~thompsa/linux_usb_amd64_2.diff Andrew From owner-freebsd-hackers@FreeBSD.ORG Mon May 31 16:44:17 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E4151065675 for ; Mon, 31 May 2010 16:44:17 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id AE5608FC26 for ; Mon, 31 May 2010 16:44:16 +0000 (UTC) Received: by gyh20 with SMTP id 20so3686683gyh.13 for ; Mon, 31 May 2010 09:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=JgWi0APcs6OzOX/TeXvKnSzbpdlqEt7h3MfZJuYImnk=; b=da6EL/x/YQKJpoIz5wBfcorU7i7XYKKoBUxkVe12wgrTlslUwrmdMOkDUB/fzXy8MM 3Ac4zUftSVAsUg37AeG5hXVPmJNHMd8L6FwhLNUlqQpKRbuLbF8jwf00WlMp9aub7FEY FF9UcqzWN6k29EmcKUsg2HXccyjKdsI6hRoEw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=hXAqdltH3hlEEp96ZV+7RoQVpNdwL3L/LYKpqEvJdlxyr1yOIiu9IBUSR10lIr1vGw /xh78wbPRASfTQag6P00cisBF1ll4Od+4WOy7nXj+SQHwM0Ulwzcp9O8SdSD54dk/gOt rnH64L+W8zwvSTZckcWIr/W/rmUyYSoThcdZA= MIME-Version: 1.0 Received: by 10.224.59.12 with SMTP id j12mr1818876qah.94.1275324255430; Mon, 31 May 2010 09:44:15 -0700 (PDT) Received: by 10.229.88.140 with HTTP; Mon, 31 May 2010 09:44:15 -0700 (PDT) In-Reply-To: References: Date: Mon, 31 May 2010 22:14:15 +0530 Message-ID: From: Shrikanth Kamath To: Artem Belevich Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: DTrace and CTF data X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 May 2010 16:44:17 -0000 Thanks Artem, but I was actually asking about the CTF info, I always thought the probes used the CTF info for *knowing* where to instrument. So if I ran the CTFCONVERT utility on a *binary*, it should append CTF sections to it should it not ? If the CTFCONVERT ran successfully I should be able to see a .SUNW_ctf section when I run objdump --section-headers. But since this was not there in the binary but still FBT was able to instrument the *entry* of the function I was curious does FBT provider need the CTF info for function entry/return ? -- Shrikanth R K On Mon, May 31, 2010 at 2:21 AM, Artem Belevich wrote: > You may be confusing CTF info and the probes. Those are different things. > > CTF info just describes data types and variable/function location. > DTrace later uses it to figure out where and how to install the > probes. > > Probes are installed dynamically, at runtime when DTrace program is > run. That's why you see it with kgdb on the live kernel once probe has > been installed. > DTrace does not modify your original kernel binary, so you will not > see any probes there. > > --Artem > > > > On Sun, May 30, 2010 at 12:10 PM, Shrikanth Kamath > wrote: > > I have a query about the FBT provider in DTrace, does FBT provider not > need > > the CTF info for the /kernel binary... > > > > I have this observation, when I disassemble say a 'kernel_funtion' , I > see > > the function is not instrumented... > > > > (kgdb) disassemble kernel_function > > Dump of assembler code for function kernel_function: > > 0xc04aa05d : push %ebp > > 0xc04aa05e : mov %esp,%ebp > > 0xc04aa060 : sub $0x2c,%esp > > > > > > Now after I did a 'fbt::kernel_function:entry', if I disassemble the > > 'kernel_function'... > > > > (kgdb) disassemble kernel_function > > Dump of assembler code for function kernel_function: > > 0xc04aa05d : lock mov %esp,%ebp <== > FBT > > instrumentation > > 0xc04aa060 : sub $0x2c,%esp > > > > If I do a 'objdump' on the kernel.debug binary I do not see any CTF > sections > > showing up? > > > > > > -- > > Shrikanth R K > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to " > freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-hackers@FreeBSD.ORG Mon May 31 17:52:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12EAD1065679 for ; Mon, 31 May 2010 17:52:41 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D74BD8FC17 for ; Mon, 31 May 2010 17:52:40 +0000 (UTC) Received: by pwj1 with SMTP id 1so211205pwj.13 for ; Mon, 31 May 2010 10:52:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=7aQAQ/ycwWooWaznAAfHXJrkJqGzRwiqrqvbexVMFC0=; b=HtC/xbq4WFCHnFGanqrzD05VTI5/HraHy1ZqQne9tQJ8WZ4oF6AA4tVikAHK8zObpb STqjZsV2MW68gS2o/oy4PnUzvUKYwuRyho/oBCNcExvV2m16HRTMH40WnFOc3/9FBjo6 byqhP4bm+sRf9HcuOLNzWzBTjuoP8yE58FI1w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=w/P9VXr3f8QXmL/bLiULN9Gx1u+TpvFKyvpl2xQ0GP10d5LFpVt8QHSmkA22aRfJkA B60yRMvDIi91Kr+yu6CzvyhUFfxPzue2UI3T1NaYkg2CxUwGcsLWlSwhbKaOsCv5aKy2 9Oy+hYIwxhInyLA7Giismu1gBmffXmtvc4CVU= MIME-Version: 1.0 Received: by 10.141.91.9 with SMTP id t9mr3560127rvl.53.1275328360183; Mon, 31 May 2010 10:52:40 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.141.40.4 with HTTP; Mon, 31 May 2010 10:52:40 -0700 (PDT) In-Reply-To: References: Date: Mon, 31 May 2010 10:52:40 -0700 X-Google-Sender-Auth: o_frld8eM13G9cKndZPS9cdHAJs Message-ID: From: Artem Belevich To: Shrikanth Kamath Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: DTrace and CTF data X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 May 2010 17:52:41 -0000 >>> If I do a 'objdump' on the kernel.debug binary I do not see any CTF sec= tions showing up? Reference to 'objdump' combined with the fact that 80% of the message was dedicated to disassembly suggested that by "CTF section" you actually meant "probe". > If the CTFCONVERT ran successfully I should be able to see a=A0.SUNW_ctf > section when I run objdump --section-headers. Now it's clear that you indeed are looking for .SUNW_ctf section. My apologies for assuming too much. I should've asked to clarify the question first. > So if I ran the CTFCONVERT=A0utility on a *binary*, it should append CTF = sections to it should it not ? It should, provided it's run properely and that there's something to append. For what it's worth, .SUNW_ctf *is* present in my kernel built with WITH_CTF=3D1: $ objdump --section-headers /boot/kernel/kernel |grep SUNW 24 .SUNW_ctf 00044fe5 0000000000000000 0000000000000000 0062f01c 2= **2 That said, FBT provider may figure out function location based on plain old ELF symbols. For that CTF may not be needed. However, it may not be able to provide function and argument types if CTF info is not present. What does "dtrace -l -f kernel_function -v" show? Does it display proper info about argument types for entry/exit? --Artem On Mon, May 31, 2010 at 9:44 AM, Shrikanth Kamath w= rote: > Thanks Artem, but I was actually asking about the CTF info, I always thou= ght > the > probes used the CTF info for *knowing* where to instrument. So if I ran t= he > CTFCONVERT > utility on a *binary*, it should append CTF sections to it should it not = ? > If the CTFCONVERT ran successfully I should be able to see a=A0.SUNW_ctf > section when I run objdump --section-headers. But since this was not ther= e > in the binary > but still FBT was able to instrument the *entry* of the function I was > curious does FBT provider > need the CTF info for function entry/return ? > > -- > Shrikanth R K > > On Mon, May 31, 2010 at 2:21 AM, Artem Belevich wrote: >> >> You may be confusing CTF info and the probes. Those are different things= . >> >> CTF info just describes data types and variable/function location. >> DTrace later uses it to figure out where and how to install the >> probes. >> >> Probes are installed dynamically, at runtime when DTrace program is >> run. That's why you see it with kgdb on the live kernel once probe has >> been installed. >> DTrace does not modify your original kernel binary, so you will not >> see any probes there. >> >> --Artem >> >> >> >> On Sun, May 30, 2010 at 12:10 PM, Shrikanth Kamath >> wrote: >> > I have a query about the FBT provider in DTrace, does FBT provider not >> > need >> > the CTF info for the /kernel binary... >> > >> > I have this observation, when I disassemble say a 'kernel_funtion' , I >> > see >> > the function is not instrumented... >> > >> > (kgdb) disassemble kernel_function >> > Dump of assembler code for function kernel_function: >> > 0xc04aa05d : push =A0 %ebp >> > 0xc04aa05e : mov =A0 =A0%esp,%ebp >> > 0xc04aa060 : sub =A0 =A0$0x2c,%esp >> > >> > >> > Now after I did a 'fbt::kernel_function:entry', if I disassemble the >> > 'kernel_function'... >> > >> > (kgdb) disassemble kernel_function >> > Dump of assembler code for function kernel_function: >> > 0xc04aa05d : lock mov %esp,%ebp =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 <=3D=3D >> > FBT >> > instrumentation >> > 0xc04aa060 : sub =A0 =A0$0x2c,%esp >> > >> > If I do a 'objdump' on the kernel.debug binary I do not see any CTF >> > sections >> > showing up? >> > >> > >> > -- >> > Shrikanth R K >> > _______________________________________________ >> > freebsd-hackers@freebsd.org mailing list >> > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> > To unsubscribe, send any mail to >> > "freebsd-hackers-unsubscribe@freebsd.org" >> > > > From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 1 19:21:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED0A81065677 for ; Tue, 1 Jun 2010 19:21:33 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id CA46D8FC0A for ; Tue, 1 Jun 2010 19:21:33 +0000 (UTC) Received: by pxi7 with SMTP id 7so2713736pxi.13 for ; Tue, 01 Jun 2010 12:21:32 -0700 (PDT) Received: by 10.141.213.28 with SMTP id p28mr5081917rvq.19.1275420092314; Tue, 01 Jun 2010 12:21:32 -0700 (PDT) Received: from [192.168.20.101] ([119.42.84.23]) by mx.google.com with ESMTPS id 23sm5068010pzk.6.2010.06.01.12.21.29 (version=SSLv3 cipher=RC4-MD5); Tue, 01 Jun 2010 12:21:30 -0700 (PDT) Message-ID: <4C055F08.8070508@pathscale.com> Date: Wed, 02 Jun 2010 02:27:04 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 To: Ed Maste References: <4C0108F1.5050004@pathscale.com> <20100601190909.GA13911@sandvine.com> In-Reply-To: <20100601190909.GA13911@sandvine.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Jun 2010 19:21:34 -0000 Ed Maste wrote: > On Sat, May 29, 2010 at 07:30:41PM +0700, "C. Bergstr?m" wrote: > > >> PathScale is slowly open sourcing and porting some of our core software >> technology and thought the BSD community might be interested in PathDB. >> Months ago we gave a few FBSD developers private access to the source, >> but never received any feedback. Now we're asking more people to please >> test and tell us what you think. >> > > Very intereresting. I have an interest in seeing a BSD-licensed > compelling replacement for GDB. > > >> Source >> git clone git://git.pathscale.com/path64/debugger.git >> > > I had a look at this, but it does not seem to be ready to build > standalone: > > - it wants hg installed? > - it is expecting directories like ../buildmeister to exist > > Does it depend on infrastructure / libs outside of debugger/src, or > should it be possible to build it in isolation? > Hi Ed, I'm answering this same question quite a bit in private and hope you don't mind me adding the list on cc. (I hope those on the list don't mind as well) It currently doens't build on any BSD, but some of these patches below will help get you more progress http://www.netbsd.org/~joerg/pdb-g++.diff http://www.netbsd.org/~joerg/readline.cc.diff http://www.netbsd.org/~joerg/pathscale.diff # For hg to git http://pastie.org/private/cninatdq83krfy9dxagfnw # For gcc-4.1+ build support Jörg Sonnenberger has identified some areas we need to abstract which will get things more portable. I'm trying to get someone who is familiar with the codebase to do this and report back once progress has been made. A few other people interested in the project have showed up in #pathscale - irc.freenode.net if you're interested in more specifics. Thanks ./C From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 2 08:57:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF4611065670 for ; Wed, 2 Jun 2010 08:57:45 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 408DA8FC16 for ; Wed, 2 Jun 2010 08:57:44 +0000 (UTC) Received: by fxm5 with SMTP id 5so4528932fxm.13 for ; Wed, 02 Jun 2010 01:57:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:content-type :content-transfer-encoding; bh=6z9zAtUVYKiq7ZyRNmroHkWODf/bqd0gRpT3PzIcasA=; b=u2ZVItDseaU8AsgTyTRoDjNcb6nD7wdiofLYbWJqFMllhAU/+zN57js3T6WCYpvW6P sY5NBdCJ1QzzNBwfBPxEVSJ52q0xBvZInKT1kSOCVk/36hy2o2hcqmW1uI5qonLtXlF4 OWjFv4e5vY5o7k0X/QEN78Z7EsGQuDg57/dYc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=L99Te77HIdZWyyUYc3XVe2z/wRsiCj0n/cqJ4wd0Y7xYV8Ym9m12cF5eN1idMN9yhW EmQvBxyw2dxjAuoCiancaUOcKpUeZgukg2YZvk1BIYQU/DpDifFlYTTIGp6oF+Zj1oEK 5FPEuW6SxEmzqNEkW9+1gZpUvyHrspudvbfoc= Received: by 10.102.17.5 with SMTP id 5mr1140587muq.42.1275469064021; Wed, 02 Jun 2010 01:57:44 -0700 (PDT) Received: from localhost.localdomain ([213.152.137.43]) by mx.google.com with ESMTPS id 25sm5957182mul.24.2010.06.02.01.57.43 (version=SSLv3 cipher=RC4-MD5); Wed, 02 Jun 2010 01:57:43 -0700 (PDT) Message-ID: <4C061D1E.3090108@gmail.com> Date: Wed, 02 Jun 2010 12:58:06 +0400 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100330 Fedora/3.0.4-1.fc12 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: gvinum & raid5 on 8.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2010 08:57:45 -0000 Hi, all. I have problem with perfomance on raid5 volume. *** AHCI mode enabled in BIOS settings ahci.ko module is loaded /8.0/-/RELEASE/ FreeBSD /8.0/-/RELEASE/ #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys//GENERIC amd64/ # gpart show => 0 8192 md0 BSD (4.0M) 0 16 - free - (8.0K) 16 8176 1 !0 (4.0M) => 0 8192 ufsid/4b497774c23db769 BSD (4.0M) 0 16 - free - (8.0K) 16 8176 1 !0 (4.0M) => 34 1953525101 ada0 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-vinum (932G) => 34 1953525101 ada1 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-vinum (932G) => 34 1953525101 ada2 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-vinum (932G) => 34 1953525101 ada3 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953524973 2 freebsd-vinum (932G) # gvinum printconfig drive disk_3 device /dev/ada3p2 drive disk_2 device /dev/ada2p2 drive disk_1 device /dev/ada1p2 drive disk_0 device /dev/ada0p2 volume raid5 plex name raid5.p0 org raid5 512s vol raid5 sd name raid5.p0.s3 drive disk_3 len 1953524224s driveoffset 265s plex raid5.p0 plexoffset 1536s sd name raid5.p0.s2 drive disk_2 len 1953524224s driveoffset 265s plex raid5.p0 plexoffset 1024s sd name raid5.p0.s1 drive disk_1 len 1953524224s driveoffset 265s plex raid5.p0 plexoffset 512s sd name raid5.p0.s0 drive disk_0 len 1953524224s driveoffset 265s plex raid5.p0 plexoffset 0s # mount | grep raid5 /dev/gvinum/raid5 on /mnt/resc (ufs, local) # cd /mnt/resc # dd if=/dev/zero of=file count=16 bs=512k 16+0 records in 16+0 records out 8388608 bytes transferred in 44.871144 secs (186949 bytes/sec) please, any solutions -- Vladimir Ermakov From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 2 10:13:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67C241065677 for ; Wed, 2 Jun 2010 10:13:21 +0000 (UTC) (envelope-from lulf@pvv.ntnu.no) Received: from hylle01.itea.ntnu.no (hylle01.itea.ntnu.no [IPv6:2001:700:300:3::100]) by mx1.freebsd.org (Postfix) with ESMTP id B33608FC1B for ; Wed, 2 Jun 2010 10:13:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hylle01.itea.ntnu.no (Postfix) with ESMTP id D901731E04C; Wed, 2 Jun 2010 12:13:18 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at hylle01.itea.ntnu.no Received: from snapcase (lynx.stud.ntnu.no [IPv6:2001:700:300:3::180]) by hylle01.itea.ntnu.no (Postfix) with ESMTP id 70F4731E035; Wed, 2 Jun 2010 12:13:18 +0200 (CEST) Date: Wed, 2 Jun 2010 12:13:18 +0200 From: Ulf Lilleengen To: venom Message-ID: <20100602101318.GA4351@snapcase> References: <4C061D1E.3090108@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C061D1E.3090108@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: gvinum & raid5 on 8.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2010 10:13:21 -0000 On Wed, Jun 02, 2010 at 12:58:06PM +0400, venom wrote: > > Hi, all. > > I have problem with perfomance on raid5 volume. > I have gotten a report from others experiencing this as well. It might be a performance regression, but make sure that your parity is ok: sysctl kern.geom.vinum.debug=4 gvinum checkparity raid5.p0 If you're able to, please apply the following patch in order for the list commands to track your rebuild status with the 'gvinum list -v ' command: http://people.freebsd.org/~lulf/gvinum_setflag.diff Any errors during checkparity should come out in the console. -- Ulf Lilleengen From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 2 13:46:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3D301065674 for ; Wed, 2 Jun 2010 13:46:04 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 723B38FC13 for ; Wed, 2 Jun 2010 13:46:04 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id l26so525266fgb.13 for ; Wed, 02 Jun 2010 06:46:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=FJ6pdT+fto0ot7H4+AIlqXDtX0H3nvK3SiSHf7cQ9Zg=; b=bKTcoKVU8rg5h22HlcQ6W/pgm6xCTkgBkbwR1yCR3yjZ9nf7LdO+wQ6qjuzkIcGlJb 6Ux9wxwivGIxjK2mQLqk2vpStslCMziRKX9Inkj0HPUsVC6nb0qoE4ccj5wxLKhQ44lq 3vqcCSoylJsneQkDwAP7JNhG7tUpJrUwsSgyI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; b=X6+WTLEXlR4MCxcZC8WGlGCP8u61aUaZIH5oTiiYScUiwMTHt8dXQjk2oBp2YalurD LrjcHm/R3l7qdJzQwrHMSFvPyR12fCCGFrZmreNgw0YzG3mt94d+U3jzfON96RfckLsh GtVMZeeN6GEFcJKAIkwvVcCIhhAtZch7XxknU= Received: by 10.103.80.8 with SMTP id h8mr3016095mul.90.1275486363116; Wed, 02 Jun 2010 06:46:03 -0700 (PDT) Received: from localhost.localdomain ([213.152.137.43]) by mx.google.com with ESMTPS id 25sm6410518mul.24.2010.06.02.06.46.01 (version=SSLv3 cipher=RC4-MD5); Wed, 02 Jun 2010 06:46:02 -0700 (PDT) Message-ID: <4C0660B0.7030701@gmail.com> Date: Wed, 02 Jun 2010 17:46:24 +0400 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100330 Fedora/3.0.4-1.fc12 Thunderbird/3.0.4 MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org References: <4C061D1E.3090108@gmail.com> <20100602101318.GA4351@snapcase> In-Reply-To: <20100602101318.GA4351@snapcase> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: gvinum & raid5 on 8.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2010 13:46:05 -0000 On 06/02/2010 02:13 PM, Ulf Lilleengen wrote: > sysctl kern.geom.vinum.debug=4 > gvinum checkparity raid5.p0 > > source is patched, module is rebuilded and reloaded console output *** # sysctl kern.geom.vinum.debug=4 kern.geom.vinum.debug: 4 -> 4 # gvinum checkparity raid5.p0 # gvinum list -v raid5.p0 Plex raid5.p0: Size: 2000408281088 bytes (1907738 MB) Subdisks: 3 State: up Organization: raid5 Stripe size: 512 kB Flags: 0 Part of volume raid5 Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'check' Jun 2 13:33:19 kernel: GEOM_VINUM[0]: parity incorrect at offset 0x0 Jun 2 13:33:19 kernel: GEOM_VINUM[0]: parity check on raid5.p0 failed at 0x0 errno 35 Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada1p2' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r0' is already known Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada2p2' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r1' is already known Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada3p2' Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r2' is already known # dd if=/dev/zero of=file count=16 bs=512k 16+0 records in 16+0 records out 8388608 bytes transferred in 38.693080 secs (216799 bytes/sec) ** -- Vladimir Ermakov From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 2 15:05:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8ACF7106566B for ; Wed, 2 Jun 2010 15:05:24 +0000 (UTC) (envelope-from lulf@pvv.ntnu.no) Received: from hylle02.itea.ntnu.no (hylle02.itea.ntnu.no [IPv6:2001:700:300:3::101]) by mx1.freebsd.org (Postfix) with ESMTP id D59DD8FC08 for ; Wed, 2 Jun 2010 15:05:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hylle02.itea.ntnu.no (Postfix) with ESMTP id 1DEEBA8011; Wed, 2 Jun 2010 17:05:21 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at hylle02.itea.ntnu.no Received: from nobby.geeknest.org (lynx.stud.ntnu.no [IPv6:2001:700:300:3::180]) by hylle02.itea.ntnu.no (Postfix) with ESMTP id 72F96A8010; Wed, 2 Jun 2010 17:05:20 +0200 (CEST) Date: Wed, 2 Jun 2010 17:04:05 +0200 From: Ulf Lilleengen To: venom Message-ID: <20100602150404.GA25564@nobby.geeknest.org> References: <4C061D1E.3090108@gmail.com> <20100602101318.GA4351@snapcase> <4C0660B0.7030701@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C0660B0.7030701@gmail.com> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: gvinum & raid5 on 8.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jun 2010 15:05:24 -0000 On Wed, Jun 02, 2010 at 05:46:24PM +0400, venom wrote: > On 06/02/2010 02:13 PM, Ulf Lilleengen wrote: > > sysctl kern.geom.vinum.debug=4 > > gvinum checkparity raid5.p0 > > > > > > source is patched, module is rebuilded and reloaded > As you can see, the parity is incorrect. Try running 'gvinum rebuildparity raid5.p0'. > > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'check' > Jun 2 13:33:19 kernel: GEOM_VINUM[0]: parity incorrect at offset 0x0 ^^ > Jun 2 13:33:19 kernel: GEOM_VINUM[0]: parity check on raid5.p0 failed > at 0x0 errno 35 > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada1p2' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r0' is already known > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada2p2' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r1' is already known > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: event 'drive tasted' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: tasted drive on 'ada3p2' > Jun 2 13:33:19 kernel: GEOM_VINUM[2]: drive 'r2' is already known > > > > # dd if=/dev/zero of=file count=16 bs=512k > 16+0 records in > 16+0 records out > 8388608 bytes transferred in 38.693080 secs (216799 bytes/sec) > > > ** > > -- > Vladimir Ermakov > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Ulf Lilleengen From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 06:34:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11D68106564A for ; Thu, 3 Jun 2010 06:34:04 +0000 (UTC) (envelope-from samflanker@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id 9268B8FC17 for ; Thu, 3 Jun 2010 06:34:03 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so186570fga.13 for ; Wed, 02 Jun 2010 23:34:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=pzTZn3ho3cAZs46Ja78wXkh44SY7cErOhWJKJCHcnIw=; b=B07tuyG70ZB0sZZVxUGB3SxhX1JnohEXLeGJxoLQhzVpfhQXHcg97CWotb6dc811Uu sz6yqbB60i/d3r+73CPqOBn9qirLfu7XLJ9Q548he6jM0Md/Rj0V7DNvPQY7iP7Q1UDa MCFyfCPBcOHAXFNVn2yFM81QsHTsTRozaFCNg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:cc:subject:references :in-reply-to:content-type:content-transfer-encoding; b=YOOJMKnKJAA/TVHxpGDrnFuFXAWs+4utOIIQZcKTwps49riPm/v0IqqURVQasAOknH R1uiwnCRDnrxq+awpohz3ZbdluIu1q8Z9Iz6HJJv3RFKOb4CiqdW766zWTLs3XhrV6qw KKaykd0Pbu+gEyfVKnlGeN3bsW819yjwJADHE= Received: by 10.102.174.2 with SMTP id w2mr3235605mue.116.1275546842287; Wed, 02 Jun 2010 23:34:02 -0700 (PDT) Received: from localhost.localdomain ([213.152.137.43]) by mx.google.com with ESMTPS id 12sm13273676muq.3.2010.06.02.23.34.00 (version=SSLv3 cipher=RC4-MD5); Wed, 02 Jun 2010 23:34:00 -0700 (PDT) Message-ID: <4C074CF0.4030707@gmail.com> Date: Thu, 03 Jun 2010 10:34:24 +0400 From: venom User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100330 Fedora/3.0.4-1.fc12 Thunderbird/3.0.4 MIME-Version: 1.0 CC: freebsd-hackers@freebsd.org References: <4C061D1E.3090108@gmail.com> <20100602101318.GA4351@snapcase> <4C0660B0.7030701@gmail.com> <20100602150404.GA25564@nobby.geeknest.org> In-Reply-To: <20100602150404.GA25564@nobby.geeknest.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: gvinum & raid5 on 8.0-RELEASE X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 06:34:04 -0000 On 06/02/2010 07:04 PM, Ulf Lilleengen wrote: > On Wed, Jun 02, 2010 at 05:46:24PM +0400, venom wrote: > >> On 06/02/2010 02:13 PM, Ulf Lilleengen wrote: >> >>> sysctl kern.geom.vinum.debug=4 >>> gvinum checkparity raid5.p0 >>> >>> >>> >> source is patched, module is rebuilded and reloaded >> >> > As you can see, the parity is incorrect. Try running 'gvinum > rebuildparity raid5.p0'. > # gvinum checkparity raid5.p0 # dmesg | tail GEOM_VINUM[0]: parity incorrect at offset 0x1d1b5500000 GEOM_VINUM[0]: parity incorrect at offset 0x1d1b9900000 GEOM_VINUM[0]: parity incorrect at offset 0x1d1bd000000 GEOM_VINUM[0]: parity incorrect at offset 0x1d1c1400000 GEOM_VINUM[0]: parity incorrect at offset 0x1d1c1600000 GEOM_VINUM[0]: parity incorrect at offset 0x1d1c1900000 GEOM_VINUM[1]: parity operation on raid5.p0 finished GEOM_VINUM[2]: event 'save config' GEOM_VINUM[2]: event 'check' GEOM_VINUM[0]: plex raid5.p0 is busy with syncing or parity build # gvinum list -v raid5.p0 Plex raid5.p0: Size: 2000408281088 bytes (1907738 MB) Subdisks: 3 State: up Synced: 0 bytes (0%) Organization: raid5 Stripe size: 512 kB Flags: 64 Part of volume raid5 # ps auxf | grep gv_worker root 953 0.0 0.0 0 16 ?? DL 1:31PM 432:08.71 [gv_worker] # top -S -n | grep gv_worker 953 root 1 76 - 0K 16K - 0 432:09 0.00% gv_worker -- Vladimir Ermakov From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 08:06:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EFA71065672 for ; Thu, 3 Jun 2010 08:06:15 +0000 (UTC) (envelope-from v.haisman@sh.cvut.cz) Received: from service2.sh.cvut.cz (ns2.sh.cvut.cz [IPv6:2001:718:2::241]) by mx1.freebsd.org (Postfix) with ESMTP id 982DA8FC24 for ; Thu, 3 Jun 2010 08:06:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by service2.sh.cvut.cz (Postfix) with ESMTP id 6EC6D139A7A; Thu, 3 Jun 2010 10:06:13 +0200 (CEST) Received: from service2.sh.cvut.cz ([127.0.0.1]) by localhost (service2.sh.cvut.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 04787-09; Thu, 3 Jun 2010 10:06:12 +0200 (CEST) Received: from shell.sh.cvut.cz (shell.sh.cvut.cz [IPv6:2001:718:2::212]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by service2.sh.cvut.cz (Postfix) with ESMTP id D838F139A6C; Thu, 3 Jun 2010 10:06:11 +0200 (CEST) Received: by shell.sh.cvut.cz (Postfix, from userid 50017) id 9EFD5B839; Thu, 3 Jun 2010 10:06:11 +0200 (CEST) To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Date: Thu, 03 Jun 2010 10:06:11 +0200 From: =?UTF-8?Q?V=C3=A1clav_Haisman?= Message-ID: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> X-Sender: v.haisman@sh.cvut.cz User-Agent: RoundCube Webmail/0.4-beta X-Spam-Status: No, hits=-3.8 tagged_above=-255.0 required=5.0 tests=ALL_TRUSTED, CRM114_HAM_00 X-Spam-Level: Subject: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 08:06:15 -0000 Hi, is it possible to obtain some sort of a thread ID that identifies a thread within a process other than pthread_self()? Something like gettid() on Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does not identify the thread well enough. GDB on FreeBSD seems to know about threads and does not seem to use the same ID as is pthread_t. -- VH From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 14:44:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D71691065673 for ; Thu, 3 Jun 2010 14:44:56 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email1.allantgroup.com (email1.emsphone.com [199.67.51.115]) by mx1.freebsd.org (Postfix) with ESMTP id 9D6D88FC15 for ; Thu, 3 Jun 2010 14:44:56 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email1.allantgroup.com (8.14.0/8.14.0) with ESMTP id o53EirCA017823 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 3 Jun 2010 09:44:53 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.4/8.14.4) with ESMTP id o53EiqLi080474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 3 Jun 2010 09:44:53 -0500 (CDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.4/8.14.3/Submit) id o53EiqLb080473; Thu, 3 Jun 2010 09:44:52 -0500 (CDT) (envelope-from dan) Date: Thu, 3 Jun 2010 09:44:52 -0500 From: Dan Nelson To: V clav Haisman Message-ID: <20100603144452.GC85961@dan.emsphone.com> References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> X-OS: FreeBSD 8.0-STABLE User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: clamav-milter 0.96 at email1.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (email1.allantgroup.com [199.67.51.78]); Thu, 03 Jun 2010 09:44:53 -0500 (CDT) X-Scanned-By: MIMEDefang 2.45 Cc: freebsd-hackers@freebsd.org Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 14:44:56 -0000 In the last episode (Jun 03), Václav Haisman said: > is it possible to obtain some sort of a thread ID that identifies a thread > within a process other than pthread_self()? Something like gettid() on > Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does > not identify the thread well enough. GDB on FreeBSD seems to know about > threads and does not seem to use the same ID as is pthread_t. The return value of pthread_self() is a pointer to the (private) "struct pthread" for the current thread, and should uniquely identify a thread. Do you have a testcase that shows otherwise? GDB might just enumerate the currently active threads starting from 1. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 14:51:45 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EE751065705 for ; Thu, 3 Jun 2010 14:51:45 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 3D6878FC0A for ; Thu, 3 Jun 2010 14:51:44 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id o53EphnF020627; Thu, 3 Jun 2010 10:51:43 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.2 (mail.netplex.net [204.213.176.10]); Thu, 03 Jun 2010 10:51:43 -0400 (EDT) Date: Thu, 3 Jun 2010 10:51:43 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: =?UTF-8?Q?V=C3=A1clav_Haisman?= In-Reply-To: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> Message-ID: References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-851401618-1275576703=:28157" Cc: freebsd-hackers@freebsd.org Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 14:51:45 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---559023410-851401618-1275576703=:28157 Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 3 Jun 2010, V=C3=A1clav Haisman wrote: > > Hi, > is it possible to obtain some sort of a thread ID that identifies a threa= d > within a process other than pthread_self()? Something like gettid() on > Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does no= t > identify the thread well enough. GDB on FreeBSD seems to know about threa= ds > and does not seem to use the same ID as is pthread_t. "identifies a thread" well enough for what? pthread_t is suppose to be opaque. Whether it is an int, pointer, or whatever, it is implementation-defined and not suppose provide any more information than available through the standard pthread interfaces. There are some non-portable interfaces in though. --=20 DE ---559023410-851401618-1275576703=:28157-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 15:19:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0EF51065674 for ; Thu, 3 Jun 2010 15:19:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2AB0B8FC15 for ; Thu, 3 Jun 2010 15:19:34 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o53FJUKm025767 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 3 Jun 2010 18:19:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o53FJHtF015863; Thu, 3 Jun 2010 18:19:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o53FJFPP015862; Thu, 3 Jun 2010 18:19:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 3 Jun 2010 18:19:15 +0300 From: Kostik Belousov To: Dan Nelson Message-ID: <20100603151915.GI83316@deviant.kiev.zoral.com.ua> References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> <20100603144452.GC85961@dan.emsphone.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YA5SKL3iPwSPv+gD" Content-Disposition: inline In-Reply-To: <20100603144452.GC85961@dan.emsphone.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, V clav Haisman Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 15:19:35 -0000 --YA5SKL3iPwSPv+gD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 03, 2010 at 09:44:52AM -0500, Dan Nelson wrote: > In the last episode (Jun 03), V??clav Haisman said: > > is it possible to obtain some sort of a thread ID that identifies a thr= ead > > within a process other than pthread_self()? Something like gettid() on > > Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does > > not identify the thread well enough. GDB on FreeBSD seems to know about > > threads and does not seem to use the same ID as is pthread_t. >=20 > The return value of pthread_self() is a pointer to the (private) "struct > pthread" for the current thread, and should uniquely identify a thread. = Do > you have a testcase that shows otherwise? GDB might just enumerate the > currently active threads starting from 1. There is thr_self(2) undocumented syscall: int thr_self(long *id); --YA5SKL3iPwSPv+gD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwHx/IACgkQC3+MBN1Mb4jchQCfdER+RknWl4m/1gXPD2chD1Zj IugAnR+fMYzH4RlXvKHU89VezQICr7Cd =c3bg -----END PGP SIGNATURE----- --YA5SKL3iPwSPv+gD-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 19:15:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE526106564A for ; Thu, 3 Jun 2010 19:15:27 +0000 (UTC) (envelope-from v.haisman@sh.cvut.cz) Received: from service1.sh.cvut.cz (service1.sh.cvut.cz [IPv6:2001:718:2::214]) by mx1.freebsd.org (Postfix) with ESMTP id 26DB68FC15 for ; Thu, 3 Jun 2010 19:15:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by service1.sh.cvut.cz (Postfix) with ESMTP id 6A87E1252D7; Thu, 3 Jun 2010 21:15:26 +0200 (CEST) X-Spam-Score: -100.63 X-Spam-Level: X-Spam-Status: No, score=-100.63 tagged_above=-255 required=5 tests=[ALL_TRUSTED=-1.44, AWL=0.384, CRM114_HAM_00=, RATWARE_GECKO_BUILD=1.426, SMTPAUTH_SHDOMAIN=-100] Received: from service1.sh.cvut.cz ([127.0.0.1]) by localhost (service1.sh.cvut.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ebbZ2cXuz54T; Thu, 3 Jun 2010 21:15:22 +0200 (CEST) Received: from [10.0.0.1] (13.101.broadband5.iol.cz [88.100.101.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: v.haisman@sh.cvut.cz) by service1.sh.cvut.cz (Postfix) with ESMTP id A8DE412546E; Thu, 3 Jun 2010 21:15:22 +0200 (CEST) Message-ID: <4C07FF45.5070703@sh.cvut.cz> Date: Thu, 03 Jun 2010 21:15:17 +0200 From: =?UTF-8?B?VsOhY2xhdiBIYWlzbWFu?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.4) Gecko/20100526 Thunderbird/3.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> In-Reply-To: X-Enigmail-Version: 1.1 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enig314A951EB8F701BE6C665A06" Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 19:15:27 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig314A951EB8F701BE6C665A06 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Daniel Eischen wrote, On 3.6.2010 16:51: > On Thu, 3 Jun 2010, V=C3=A1clav Haisman wrote: >=20 >> >> Hi, >> is it possible to obtain some sort of a thread ID that identifies a th= read >> within a process other than pthread_self()? Something like gettid() on= >> Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does= not >> identify the thread well enough. GDB on FreeBSD seems to know about th= reads >> and does not seem to use the same ID as is pthread_t. >=20 > "identifies a thread" well enough for what? pthread_t is suppose > to be opaque. Whether it is an int, pointer, or whatever, it is > implementation-defined and not suppose provide any more information > than available through the standard pthread interfaces. There are > some non-portable interfaces in though. Exactly, pthread_t is opaque and it thus it is only by a chance that I ca= n print its value, it is a pointer on FreeBSD. It could as well be a struct= and then I could not possibly do anything to print the thread identity on scr= een. * 4 Thread 28426ec0 (LWP 100172) 0x0804b9f6 in TestThread::run () As shown above, GDB can show some sort of ID, the LWP bit. How does it ge= t it? Is it possible to get the LWP ID from inside the process without debugging it? --------------enig314A951EB8F701BE6C665A06 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREIAAYFAkwH/0wACgkQr2CghdezFMnIPACg3HRywOn0bVrX/U7RcXOBP7A8 dfYAn1L0ytWSeyeuuQiqW9x6QEOssOQM =HBhB -----END PGP SIGNATURE----- --------------enig314A951EB8F701BE6C665A06-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 19:16:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 70C80106564A for ; Thu, 3 Jun 2010 19:16:31 +0000 (UTC) (envelope-from v.haisman@sh.cvut.cz) Received: from service1.sh.cvut.cz (service1.sh.cvut.cz [IPv6:2001:718:2::214]) by mx1.freebsd.org (Postfix) with ESMTP id 20EFA8FC21 for ; Thu, 3 Jun 2010 19:16:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by service1.sh.cvut.cz (Postfix) with ESMTP id 6419F12546E; Thu, 3 Jun 2010 21:16:30 +0200 (CEST) X-Spam-Score: -100.665 X-Spam-Level: X-Spam-Status: No, score=-100.665 tagged_above=-255 required=5 tests=[ALL_TRUSTED=-1.44, AWL=0.349, CRM114_HAM_00=, RATWARE_GECKO_BUILD=1.426, SMTPAUTH_SHDOMAIN=-100] Received: from service1.sh.cvut.cz ([127.0.0.1]) by localhost (service1.sh.cvut.cz [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id obXoxEfAPTlC; Thu, 3 Jun 2010 21:16:27 +0200 (CEST) Received: from [10.0.0.1] (13.101.broadband5.iol.cz [88.100.101.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: v.haisman@sh.cvut.cz) by service1.sh.cvut.cz (Postfix) with ESMTP id 22D101252D7; Thu, 3 Jun 2010 21:16:27 +0200 (CEST) Message-ID: <4C07FF8C.1090800@sh.cvut.cz> Date: Thu, 03 Jun 2010 21:16:28 +0200 From: =?UTF-8?B?VsOhY2xhdiBIYWlzbWFu?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.4) Gecko/20100526 Thunderbird/3.1 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> <20100603144452.GC85961@dan.emsphone.com> <20100603151915.GI83316@deviant.kiev.zoral.com.ua> In-Reply-To: <20100603151915.GI83316@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.1 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------enigDF6A748632350A2092CB5F17" Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 19:16:31 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigDF6A748632350A2092CB5F17 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Kostik Belousov wrote, On 3.6.2010 17:19: > On Thu, Jun 03, 2010 at 09:44:52AM -0500, Dan Nelson wrote: >> In the last episode (Jun 03), V??clav Haisman said: >>> is it possible to obtain some sort of a thread ID that identifies a t= hread >>> within a process other than pthread_self()? Something like gettid() = on >>> Linux? Apparently, on FreeBSD the pthread_t is a pointer type and do= es >>> not identify the thread well enough. GDB on FreeBSD seems to know ab= out >>> threads and does not seem to use the same ID as is pthread_t. >> >> The return value of pthread_self() is a pointer to the (private) "stru= ct >> pthread" for the current thread, and should uniquely identify a thread= =2E Do >> you have a testcase that shows otherwise? GDB might just enumerate th= e >> currently active threads starting from 1. >=20 > There is thr_self(2) undocumented syscall: > int thr_self(long *id); Thanks, I'll try it. Is the returned ID the LWP ID that GDB shows? -- VH --------------enigDF6A748632350A2092CB5F17 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEAREIAAYFAkwH/4wACgkQr2CghdezFMlsxwCgg9LhvOwppMs6TawP3QKvyjmF hUgAnjGjMRXzvxEjccqXpAmUYE0Muu/D =qw+X -----END PGP SIGNATURE----- --------------enigDF6A748632350A2092CB5F17-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 19:55:23 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41F3F1065672 for ; Thu, 3 Jun 2010 19:55:23 +0000 (UTC) (envelope-from pluknet@gmail.com) Received: from mail-ww0-f54.google.com (mail-ww0-f54.google.com [74.125.82.54]) by mx1.freebsd.org (Postfix) with ESMTP id C9B3E8FC15 for ; Thu, 3 Jun 2010 19:55:22 +0000 (UTC) Received: by wwb22 with SMTP id 22so508088wwb.13 for ; Thu, 03 Jun 2010 12:55:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=2TK1KCtbY2qg1ngXJjxScS2NGxevz+gXRYPALeGXK+k=; b=V2DwvFPrtTdplrFEbzXV9LyrSlAdzpEkFuQwgSi444OpLyNMTd8kMiz/4NRmwqJ61Z 9AfmKJL0r+IZmCa7pSHDnuKhSqxBTOFyGOQEG+B9JzkiUKRQBRbywfAphAJVcclb1i44 KN6KeIVKy/8s3bjjsp1ceKuiJ3mfAtvNMGFa4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=CbEOo2dA1Dk/LjvrHrHEDnnDG6w+PpL2L+7wRkdqAfpC/jvUy3BaaGS9iXrsmJ88ZD H1jDIvmxvqBRpDuHne2AWkjDY3VGvumt99AamILZz4/Xa8ee8VJjA7IvMQymwtDw0iPj FvknyfJINSsJnX2YM6CGOyOAX675immobAs78= MIME-Version: 1.0 Received: by 10.216.187.131 with SMTP id y3mr1574725wem.34.1275594921712; Thu, 03 Jun 2010 12:55:21 -0700 (PDT) Received: by 10.216.51.78 with HTTP; Thu, 3 Jun 2010 12:55:21 -0700 (PDT) In-Reply-To: <4C07FF8C.1090800@sh.cvut.cz> References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> <20100603144452.GC85961@dan.emsphone.com> <20100603151915.GI83316@deviant.kiev.zoral.com.ua> <4C07FF8C.1090800@sh.cvut.cz> Date: Thu, 3 Jun 2010 23:55:21 +0400 Message-ID: From: pluknet To: =?ISO-8859-1?Q?V=E1clav_Haisman?= Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 19:55:23 -0000 2010/6/3 V=E1clav Haisman : > Kostik Belousov wrote, On 3.6.2010 17:19: >> On Thu, Jun 03, 2010 at 09:44:52AM -0500, Dan Nelson wrote: >>> In the last episode (Jun 03), V??clav Haisman said: >>>> is it possible to obtain some sort of a thread ID that identifies a th= read >>>> within a process other than pthread_self()? =A0Something like gettid()= on >>>> Linux? =A0Apparently, on FreeBSD the pthread_t is a pointer type and d= oes >>>> not identify the thread well enough. =A0GDB on FreeBSD seems to know a= bout >>>> threads and does not seem to use the same ID as is pthread_t. >>> >>> The return value of pthread_self() is a pointer to the (private) "struc= t >>> pthread" for the current thread, and should uniquely identify a thread.= =A0Do >>> you have a testcase that shows otherwise? =A0GDB might just enumerate t= he >>> currently active threads starting from 1. >> >> There is thr_self(2) undocumented syscall: >> int thr_self(long *id); > Thanks, I'll try it. Is the returned ID the LWP ID that GDB shows? > thr_self() does its work as well as ddb and procstat do: using td->td_tid. --=20 wbr, pluknet From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 20:09:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 556A4106566B for ; Thu, 3 Jun 2010 20:09:33 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id EB9438FC12 for ; Thu, 3 Jun 2010 20:09:32 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id o53K9Vv4011816; Thu, 3 Jun 2010 16:09:31 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.2 (mail.netplex.net [204.213.176.10]); Thu, 03 Jun 2010 16:09:31 -0400 (EDT) Date: Thu, 3 Jun 2010 16:09:31 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: =?UTF-8?B?VsOhY2xhdiBIYWlzbWFu?= In-Reply-To: <4C07FF45.5070703@sh.cvut.cz> Message-ID: References: <6e716c850fa84621482b71826dae55d5@shell.sh.cvut.cz> <4C07FF45.5070703@sh.cvut.cz> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-1804928587-1275595771=:29332" Cc: freebsd-hackers@freebsd.org Subject: Re: How to get a thread ID? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 20:09:33 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. ---559023410-1804928587-1275595771=:29332 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 3 Jun 2010, V=C3=A1clav Haisman wrote: > Daniel Eischen wrote, On 3.6.2010 16:51: >> On Thu, 3 Jun 2010, V=C3=A1clav Haisman wrote: >> >>> >>> Hi, >>> is it possible to obtain some sort of a thread ID that identifies a thr= ead >>> within a process other than pthread_self()? Something like gettid() on >>> Linux? Apparently, on FreeBSD the pthread_t is a pointer type and does = not >>> identify the thread well enough. GDB on FreeBSD seems to know about thr= eads >>> and does not seem to use the same ID as is pthread_t. >> >> "identifies a thread" well enough for what? pthread_t is suppose >> to be opaque. Whether it is an int, pointer, or whatever, it is >> implementation-defined and not suppose provide any more information >> than available through the standard pthread interfaces. There are >> some non-portable interfaces in though. > Exactly, pthread_t is opaque and it thus it is only by a chance that I ca= n > print its value, it is a pointer on FreeBSD. It could as well be a struct= and > then I could not possibly do anything to print the thread identity on scr= een. > > * 4 Thread 28426ec0 (LWP 100172) 0x0804b9f6 in TestThread::run () > > As shown above, GDB can show some sort of ID, the LWP bit. How does it ge= t > it? Is it possible to get the LWP ID from inside the process without > debugging it? I think the value that it is showing (28426ec0) is the hex value of the pthread_t (pointer). You can also use pthread_[gs]etname_np() from if you want to associate an string identifier with a thread. --=20 DE ---559023410-1804928587-1275595771=:29332-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 3 22:03:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DBB3F106564A; Thu, 3 Jun 2010 22:03:56 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id EC2808FC0A; Thu, 3 Jun 2010 22:03:55 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA20852; Fri, 04 Jun 2010 01:03:52 +0300 (EEST) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OKIW0-000137-7F; Fri, 04 Jun 2010 01:03:52 +0300 Message-ID: <4C0826C7.7070900@icyb.net.ua> Date: Fri, 04 Jun 2010 01:03:51 +0300 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: Doug Rabson References: <4BEBA334.6080101@icyb.net.ua> <4BEC040E.9080303@FreeBSD.org> <4BFE2ED6.1070402@freebsd.org> <4BFE8C07.9010303@icyb.net.ua> In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jun 2010 22:03:56 -0000 on 28/05/2010 17:49 Doug Rabson said the following: > > > On 27 May 2010 16:13, Andriy Gapon > wrote: > > on 27/05/2010 17:40 Doug Rabson said the following: > > > > Excellent work - thanks for looking into this. I still think its > easier > > to debug this code in userland using a shim that redirects the zfsboot > > i/o calls to simple read system calls... > > Absolutely! That should much easier. > Do you have such a shim that you could share? > I'd be much obliged for it. And not only I, I think. > Thanks! > > > Attached. I thought I sent it to the list before but perhaps I only sent > to one of the participants in the last gang block thread. Thanks a lot! I am sure that I will find it useful more than once. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 10:57:59 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 307941065670 for ; Fri, 4 Jun 2010 10:57:59 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id DFA0C8FC13 for ; Fri, 4 Jun 2010 10:57:58 +0000 (UTC) Received: from server.mypc.hu (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id 8658114DB824; Fri, 4 Jun 2010 12:57:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at server.mypc.hu Received: from server.mypc.hu ([127.0.0.1]) by server.mypc.hu (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id O+I2PhaLHs6H; Fri, 4 Jun 2010 12:57:54 +0200 (CEST) Received: from [192.168.1.105] (catv-80-99-92-167.catv.broadband.hu [80.99.92.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id 5B75314DB7A7; Fri, 4 Jun 2010 12:57:54 +0200 (CEST) Message-ID: <4C08DC5A.4020409@FreeBSD.org> Date: Fri, 04 Jun 2010 12:58:34 +0200 From: Gabor Kovesdan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Edwin Groothuis , d@delphij.net Subject: libc symbol versioning difficulties with iconv integration X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 10:57:59 -0000 Hello folks, I'm trying to integrate the result of my last SoC work to the base system but I'm facing some difficulties with libc symbol versioning. I placed the iconv code into an iconv subdirectory inside src/lib/libc and I added a Makefile and a symbol map, just like another parts of libc do but when I try to compile this stuff, I get this error in the linking phase: building shared library libc.so.7 /usr/bin/ld: libc.so.7: undefined versioned symbol namefts_open@FBSD_1.0 /usr/bin/ld: failed to set dynamic section sizes: Bad value *** Error code 1 I have no idea what's going wrong because I did everything exactly in the same way as another components do. I don't know why does it break at fts_open(), which is unrelated to iconv, not even used in the iconv code. If I just unhook the iconv part fromt he build, everything goes fine. Any ideas? Patch is here: http://kovesdan.org/patches/iconv-libc.diff Thanks, -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 15:12:21 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C48361065678; Fri, 4 Jun 2010 15:12:21 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 7F5338FC19; Fri, 4 Jun 2010 15:12:21 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id o54EudUL000431; Fri, 4 Jun 2010 10:56:39 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.2 (mail.netplex.net [204.213.176.10]); Fri, 04 Jun 2010 10:56:40 -0400 (EDT) Date: Fri, 4 Jun 2010 10:56:39 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Gabor Kovesdan In-Reply-To: <4C08DC5A.4020409@FreeBSD.org> Message-ID: References: <4C08DC5A.4020409@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Edwin Groothuis , FreeBSD Hackers , d@delphij.net Subject: Re: libc symbol versioning difficulties with iconv integration X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 15:12:21 -0000 On Fri, 4 Jun 2010, Gabor Kovesdan wrote: > Hello folks, > > I'm trying to integrate the result of my last SoC work to the base system but > I'm facing some difficulties with libc symbol versioning. I placed the iconv > code into an iconv subdirectory inside src/lib/libc and I added a Makefile > and a symbol map, just like another parts of libc do but when I try to > compile this stuff, I get this error in the linking phase: > > building shared library libc.so.7 > /usr/bin/ld: libc.so.7: undefined versioned symbol namefts_open@FBSD_1.0 > /usr/bin/ld: failed to set dynamic section sizes: Bad value > *** Error code 1 > > I have no idea what's going wrong because I did everything exactly in the > same way as another components do. I don't know why does it break at > fts_open(), which is unrelated to iconv, not even used in the iconv code. If > I just unhook the iconv part fromt he build, everything goes fine. Any ideas? > > Patch is here: http://kovesdan.org/patches/iconv-libc.diff I don't really see anything wrong with your Makefile or Symbol.map. Do you have any other local changes that could be causing problems? -- DE From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 15:34:08 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC34F106566B; Fri, 4 Jun 2010 15:34:08 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay04.stack.nl [IPv6:2001:610:1108:5010::107]) by mx1.freebsd.org (Postfix) with ESMTP id 710CE8FC16; Fri, 4 Jun 2010 15:34:08 +0000 (UTC) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mx1.stack.nl (Postfix) with ESMTP id 7201F1DD656; Fri, 4 Jun 2010 17:34:07 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 5F9311721B; Fri, 4 Jun 2010 17:34:07 +0200 (CEST) Date: Fri, 4 Jun 2010 17:34:07 +0200 From: Jilles Tjoelker To: Gabor Kovesdan Message-ID: <20100604153407.GA45024@stack.nl> References: <4C08DC5A.4020409@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4C08DC5A.4020409@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Edwin Groothuis , FreeBSD Hackers , d@delphij.net Subject: Re: libc symbol versioning difficulties with iconv integration X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 15:34:08 -0000 On Fri, Jun 04, 2010 at 12:58:34PM +0200, Gabor Kovesdan wrote: > I'm trying to integrate the result of my last SoC work to the base > system but I'm facing some difficulties with libc symbol versioning. I > placed the iconv code into an iconv subdirectory inside src/lib/libc and > I added a Makefile and a symbol map, just like another parts of libc do > but when I try to compile this stuff, I get this error in the linking phase: > building shared library libc.so.7 > /usr/bin/ld: libc.so.7: undefined versioned symbol namefts_open@FBSD_1.0 > /usr/bin/ld: failed to set dynamic section sizes: Bad value > *** Error code 1 > I have no idea what's going wrong because I did everything exactly in > the same way as another components do. I don't know why does it break at > fts_open(), which is unrelated to iconv, not even used in the iconv > code. If I just unhook the iconv part fromt he build, everything goes > fine. Any ideas? > Patch is here: http://kovesdan.org/patches/iconv-libc.diff There is a .include in iconv/Makefile.inc, what happens if you take that out? -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 20:06:26 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77BF41065680 for ; Fri, 4 Jun 2010 20:06:26 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe05.swip.net [212.247.154.129]) by mx1.freebsd.org (Postfix) with ESMTP id 0B1918FC08 for ; Fri, 4 Jun 2010 20:06:25 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.0 c=1 a=2caWB_DcjroA:10 a=kj9zAlcOel0A:10 a=M8b_wTzEtboA:10 a=MnI1ikcADjEx7bvsp0jZvQ==:17 a=VvkIwr1RqJHVyPaNJ4YA:9 a=-QfE4PvsRBdJVuQu3Bjla5WSkPAA:4 a=CjuIK1q_8ugA:10 Received: from [188.126.201.140] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe05.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 1302790072 for freebsd-hackers@freebsd.org; Fri, 04 Jun 2010 22:06:24 +0200 To: freebsd-hackers@freebsd.org From: Hans Petter Selasky X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'( =?iso-8859-15?q?=3B=5FIjlA=3A=0A=09hGE=2E=2EEw?=, =?iso-8859-15?q?XAQ*o=23=5C/M=7ESC=3DS1-f9=7BEzRfT=27=7CHhll5Q=5Dha5Bt-s=7Co?= =?iso-8859-15?q?TlKMusi=3A1e=5BwJl=7Dkd=7DGR=0A=09Z0adGx-x=5F0zGbZj=27e?=(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' Date: Fri, 4 Jun 2010 22:03:39 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201006042203.39515.hselasky@c2i.net> Subject: lib32 and Makefiles X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 20:06:26 -0000 Hi, What is the recommended way to detect a lib32 build on FreeBSD 8/9 from inside the Makefile? --HPS From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 19:43:42 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 123C1106566B for ; Fri, 4 Jun 2010 19:43:42 +0000 (UTC) (envelope-from bounces@nabble.com) Received: from kuber.nabble.com (kuber.nabble.com [216.139.236.158]) by mx1.freebsd.org (Postfix) with ESMTP id CB1788FC1A for ; Fri, 4 Jun 2010 19:43:41 +0000 (UTC) Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1OKcnt-00043M-AO for freebsd-hackers@freebsd.org; Fri, 04 Jun 2010 12:43:41 -0700 Message-ID: <28784339.post@talk.nabble.com> Date: Fri, 4 Jun 2010 12:43:41 -0700 (PDT) From: Stefan Miklosovic To: freebsd-hackers@freebsd.org In-Reply-To: <201003051233.42861.jpaetzel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: miklosovic.freebsd@gmail.com References: <20100304213329.GJ57205@bunrab.catwhisker.org> <201003050801.00440.jhb@freebsd.org> <201003051233.42861.jpaetzel@freebsd.org> X-Mailman-Approved-At: Fri, 04 Jun 2010 20:18:54 +0000 Subject: Re: Scripting sysinstall(8) to create & use multiple slices on a disk? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 19:43:42 -0000 Hi, could you please send me those installers you have written? I would like to write up mine and I dont have an idea how to do that. Your scripts would really helped me a lot. Be so kind and please send me to like attachment or to miklosovic.freebsd@gmail.com god bless you :) jpaetzel wrote: > > On Friday 05 March 2010 07:01:00 John Baldwin wrote: >> On Thursday 04 March 2010 4:33:29 pm David Wolfskill wrote: >> > For reasons that may well be idiosyncratic, I like to set up FreeBSD >> > machines to have at least 2 bootable slices -- e.g., one can act as a >> > fallback if an attempted software upgrade proves to have been >> ill-timed. >> > >> > In the past, I've done this manually; while a bit tedious & fairly >> > "target-rich" with opportunities for human error, it's something that >> is >> > typically done infrequently (i.e., once) in the life of a machine (or >> at >> > least its boot drive). >> > >> > At work, the IT folks use a scripted sysinstall(8) to set machines up; >> > to increase the probability that I'll be able to get 3 "special" >> > machines set up the way I want, I'm trying to set up a sysinstall >> config >> > file to make this as painless as possible. >> > >> > I managed to get a copy of the config script IT uses, so I had a >> > starting-point ... but they were setting the machines up with >> > >> > partition=exclusive >> > >> > which doesn't seem like a good choice for what I'm doing. :-} >> > >> > >> > After my first attempt failed, I poked around on the Net & found >> > >> > > le- >> >> slices-4387807/>, >> >> > (dated 18-11-08, 10:40 PM ), in which Peter Steele describes something >> > >> > similar to what I was about to try next, and writes: >> > | My intent here is to create three slices-one 6GB in size, another 1GB >> > | in size, and the third sized to consume the remaining free space. >> When >> > | I run this through sysinstall, it complains that it can't find the >> > | space for the partitions. It even complains that it can't find any >> > | free space. Because the slices don't get created, the subsequent >> label >> > | assignments fail as well. What is the proper commands for creating >> > | multiple slices in install.cfg? >> > >> > In a foillowup, he writes: >> > | After a lot of experimenting, my impression is that sysinstall simply >> > | doesn't support multiple slice installations. It works to a point, >> but >> > | I get some unexpected errors, e.g. >> > | >> > | Unable to make device node for /dev/ad0s1a in /dev >> > >> > which doesn't seem very encouraging. >> > >> > >> > Would someone please either confirm the limitation or provide a >> > suitable excerpt from a sysinstall config script to demonstrate >> > that it is actually possible? (Or show me where it's spelled out in >> the >> > man page....) >> > >> > (I'm using 7.x sysinstall, if that matters.) >> >> If you are doing a fully scripted install you may be better off just >> using >> a dedicated shell script to format your disks and mount them and then use >> the various *-install.sh scripts from the release distributions to >> install >> the code. You could still do this via sysinstall by sticking your shell >> script in /stand in the MFS root and having your sysinstall script just >> run that script. You might want to build a custom mfsroot to add some >> more >> useful tools though. >> >> I really think sysinstall needs to support a disk "backdoor" whereby the >> user can either manually partition disks and then mount them at /mnt (or >> have a script do it), and tell sysinstall to just skip the disk stuff and >> assume /mnt is mounted. > > David, > > I second the ditching sysinstall for a shell script idea. A shell script > that > replaces sysinstall is nearly as short as the install.cfg and a lot easier > to > figure out. I've written a half dozen auto installers for FreeBSD, from > trivial to complex and would be more than willing to help you get > something > set up. I can send you code if you want as well. > > -- > Thanks, > > Josh Paetzel > FreeBSD -- The power to serve > > > -- View this message in context: http://old.nabble.com/Scripting-sysinstall%288%29-to-create---use-multiple-slices-on-a-disk--tp27786883p28784339.html Sent from the freebsd-hackers mailing list archive at Nabble.com. From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 22:32:42 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E0E91065672 for ; Fri, 4 Jun 2010 22:32:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 08D2E8FC15 for ; Fri, 4 Jun 2010 22:32:41 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o54MWmPo073616 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 5 Jun 2010 01:32:48 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o54MWZ8d023020; Sat, 5 Jun 2010 01:32:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o54MWYPl023019; Sat, 5 Jun 2010 01:32:34 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 5 Jun 2010 01:32:32 +0300 From: Kostik Belousov To: Hans Petter Selasky Message-ID: <20100604223232.GQ83316@deviant.kiev.zoral.com.ua> References: <201006042203.39515.hselasky@c2i.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ViugPjSMr9TemhdD" Content-Disposition: inline In-Reply-To: <201006042203.39515.hselasky@c2i.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.7 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_20, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org Subject: Re: lib32 and Makefiles X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 22:32:42 -0000 --ViugPjSMr9TemhdD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 04, 2010 at 10:03:39PM +0200, Hans Petter Selasky wrote: > Hi, >=20 > What is the recommended way to detect a lib32 build on FreeBSD 8/9 > from inside the Makefile? For the C code, the proper way is to #ifdef COMPAT_32BIT. I was not able to find a good way to determine this inside Makefile. Probably, looking for the presence of LIB32* variables (see src/Makefile.inc1) is the most satisfactory way for now. --ViugPjSMr9TemhdD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwJfwAACgkQC3+MBN1Mb4ilkACfcqCq33i/umKnDyAzNRQ/8xC4 1ekAn0hWHomn1S8izy9Mo1g3Louc6z06 =vRS5 -----END PGP SIGNATURE----- --ViugPjSMr9TemhdD-- From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 4 22:39:56 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C19D106564A for ; Fri, 4 Jun 2010 22:39:56 +0000 (UTC) (envelope-from gabor@FreeBSD.org) Received: from server.mypc.hu (server.mypc.hu [87.229.73.95]) by mx1.freebsd.org (Postfix) with ESMTP id 044FA8FC1A for ; Fri, 4 Jun 2010 22:39:55 +0000 (UTC) Received: from server.mypc.hu (localhost [127.0.0.1]) by server.mypc.hu (Postfix) with ESMTP id 0E84F14DB852; Sat, 5 Jun 2010 00:39:54 +0200 (CEST) X-Virus-Scanned: amavisd-new at server.mypc.hu Received: from server.mypc.hu ([127.0.0.1]) by server.mypc.hu (server.mypc.hu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 404REeUP5rVk; Sat, 5 Jun 2010 00:39:51 +0200 (CEST) Received: from [192.168.1.105] (catv-80-99-92-167.catv.broadband.hu [80.99.92.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by server.mypc.hu (Postfix) with ESMTPSA id 8588114DB844; Sat, 5 Jun 2010 00:39:51 +0200 (CEST) Message-ID: <4C0980DF.6030300@FreeBSD.org> Date: Sat, 05 Jun 2010 00:40:31 +0200 From: Gabor Kovesdan User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Jilles Tjoelker References: <4C08DC5A.4020409@FreeBSD.org> <20100604153407.GA45024@stack.nl> In-Reply-To: <20100604153407.GA45024@stack.nl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: Edwin Groothuis , FreeBSD Hackers , d@delphij.net Subject: Re: libc symbol versioning difficulties with iconv integration X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jun 2010 22:39:56 -0000 El 2010. 06. 04. 17:34, Jilles Tjoelker escribió: >> Patch is here: http://kovesdan.org/patches/iconv-libc.diff >> > There is a .include in iconv/Makefile.inc, what happens > if you take that out? > OMG, I overlooked that. It remained there from the original Makefile that was used to build it as a standalone lib. Now it works, thanks! -- Gabor Kovesdan FreeBSD Volunteer EMAIL: gabor@FreeBSD.org .:|:. gabor@kovesdan.org WEB: http://people.FreeBSD.org/~gabor .:|:. http://kovesdan.org From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 00:31:51 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A53B1065673 for ; Sat, 5 Jun 2010 00:31:51 +0000 (UTC) (envelope-from wooh@wooh.hu) Received: from mail.netidea.hu (netwarehouse.netidea.hu [195.228.254.126]) by mx1.freebsd.org (Postfix) with ESMTP id 2715B8FC13 for ; Sat, 5 Jun 2010 00:31:50 +0000 (UTC) Received: from radon (localhost [127.0.0.1]) by mail.netidea.hu (Postfix) with ESMTP id 88B921294BB; Sat, 5 Jun 2010 01:58:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at netidea.hu Received: from mail.netidea.hu ([127.0.0.1]) by radon (mail.netidea.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HREpN8Vp6J8A; Sat, 5 Jun 2010 01:58:05 +0200 (CEST) Received: from jamms-MacBook.local (catv-89-132-136-110.catv.broadband.hu [89.132.136.110]) by mail.netidea.hu (Postfix) with ESMTP id 722821294B3; Sat, 5 Jun 2010 01:58:05 +0200 (CEST) Message-ID: <4C09932B.6040808@wooh.hu> Date: Sat, 05 Jun 2010 01:58:35 +0200 From: Adam PAPAI User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-questions@freebsd.org, freebsd-hackers@freebsd.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 00:31:51 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi List, A week ago I started to benchmark Linux vs. FreeBSD on a Dell Poweredge 1850. CPU: 2 x 3.4Ghz Xeon (Dual Core) Memory: 8GB (4x2) Disk: 1 x SEAGATE ST373454LC D404 (SCSI) FreeBSD kazoku 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #0: Tue May 25 20:54:11 UTC 2010 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 The tests with seqrewr, seqrd, rndrd, and so on is still going on, so I can only publish the seqwr result. (The PostgreSQL will be tested as well) (soft-updates are on) /dev/da0s1d on /usr (ufs, local, soft-updates) Tested with: sysbench --num-threads=$a --file-block-size=$bs --test=fileio - --file-total-size=2G --file-fsync-all=no --file-test-mode=seqwr run My first results (seqwr with 1,2,4,8,6,32 threads) can be found here. http://tech-blog.wooh.hu/~wooh/fbsd_vs_debian_seqwr.html Why FreeBSD is supreme with 1 and 2 thread. And why is it 2 and 3 times slower with 4-8-16-32 threads compared to Debian? The first two tests (1 thread and 2 thread) showed me that FreeBSD is supreme in I/O, but later tests showed me, that it can produce horrible I/O. How can I tune my disk to make it faster? Is it possible? What is the reason of the really slow I/O with more than 4 threads? What do you recommend me to do? Why is it damn slow with 8K blocksize? I have more than 15 FreeBSD servers in production environment and I don't want to change operating system due to I/O issues. I changed my OpenBSD servers to FreeBSD 3 years ago... :) When all tests are ready I'll publish all the results, including the postgresql benchmarks as well. Best Regards, - -- Adam PAPAI -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMCZMrAAoJEGq0EWvh5uiI10MIAM1iZxFZ5xssKmawHl56Ruin zHHgb4Nc15waTLdzFGfllAayDlZqvvpoSpOVbp8qDZYlkTbYPF6aMjkehqMvQUEo nFs7WN2VaCSOhUUQSwjqfGdnMLW9H5uyW/ZkYvgoOjQjz/vewDV6Fi+ZfGmt5Zqw gV1ZlXFdAUOUW6c90ODOPxn+7XCA5UC2sUMPB+1iNxrTiiS6C2YQ0Vy1fCXvrhU3 51n0ES/7JBF4sk5dH1VNEU/8AeQRBOoKPuAHhZKRZZ1x+1dMkDhwdD+KUHGrRGJd fUAZmMhjE6fRG86FbwK5jrZizHZYpE3PfpZe6tI3SIvw7NbUNrRsCMSiel+0FBg= =k3Sw -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 00:43:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3D0E1065679 for ; Sat, 5 Jun 2010 00:43:35 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3D1D38FC08 for ; Sat, 5 Jun 2010 00:43:34 +0000 (UTC) Received: by gwaa12 with SMTP id a12so132104gwa.13 for ; Fri, 04 Jun 2010 17:43:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received :in-reply-to:references:from:date:x-google-sender-auth:message-id :subject:to:cc:content-type; bh=dnfMTTudZMSHLIXjFDMyCOTiX7XxBeK4V4inaSB/h+8=; b=QNoqOJeLofy9yzfr3rOVvjwnxZcSG6QtqbwYYt+zKnGQ52SqJKx3XQwSqbUxZzxtOL vnvzJx6VJIkQ7otDi6faWxcH/SEc3qMH73OJ7YqCsD4EtyhvxR7dlTFOO0mG7gZdMrqD qiYmZHdqs+F2wyUdyljJDa65Pd6QZtTrrKqNo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=gMOGVJ13guVBx2wMRLL/I7cnUXvW8nZv/Z31NfjSl2K3e0y8WvT0yYqKpFoIc8imLl DS/F0loXj9UUs6MmGHTXoWdXjH4whVUmq0vf2f3wdpAfI0nO3cNgBydwXNAm8KvwwKSb 1M9BuMcfK77BaBTnTdQjqvA3Jj5BFwS4DJGtI= Received: by 10.150.48.2 with SMTP id v2mr11057137ybv.173.1275698614119; Fri, 04 Jun 2010 17:43:34 -0700 (PDT) MIME-Version: 1.0 Sender: mozolevsky@gmail.com Received: by 10.151.84.11 with HTTP; Fri, 4 Jun 2010 17:43:14 -0700 (PDT) In-Reply-To: <4C09932B.6040808@wooh.hu> References: <4C09932B.6040808@wooh.hu> From: Igor Mozolevsky Date: Sat, 5 Jun 2010 01:43:14 +0100 X-Google-Sender-Auth: pKssWYACNvJkvYWrJACZ7WvDmng Message-ID: To: Adam PAPAI Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 00:43:35 -0000 On 5 June 2010 00:58, Adam PAPAI wrote: > How can I tune my disk to make it faster? Is it possible? What is the > reason of the really slow I/O with more than 4 threads? What do you > recommend me to do? Why is it damn slow with 8K blocksize? Does linux still have async disk writes by default? Igor From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 01:36:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31303106564A; Sat, 5 Jun 2010 01:36:25 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [204.109.60.94]) by mx1.freebsd.org (Postfix) with ESMTP id 134558FC14; Sat, 5 Jun 2010 01:36:24 +0000 (UTC) Received: from core.draftnet (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id A85485C03; Sat, 5 Jun 2010 01:36:29 +0000 (UTC) From: Bruce Cran To: freebsd-hackers@freebsd.org Date: Sat, 5 Jun 2010 02:36:17 +0100 User-Agent: KMail/1.13.3 (FreeBSD/9.0-CURRENT; KDE/4.4.3; amd64; ; ) References: <4C09932B.6040808@wooh.hu> In-Reply-To: <4C09932B.6040808@wooh.hu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201006050236.17697.bruce@cran.org.uk> Cc: Adam PAPAI , freebsd-questions@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 01:36:25 -0000 On Saturday 05 June 2010 00:58:35 Adam PAPAI wrote: > Why FreeBSD is supreme with 1 and 2 thread. And why is it 2 and 3 times > slower with 4-8-16-32 threads compared to Debian? The first two tests (1 > thread and 2 thread) showed me that FreeBSD is supreme in I/O, but later > tests showed me, that it can produce horrible I/O. > > How can I tune my disk to make it faster? Is it possible? What is the > reason of the really slow I/O with more than 4 threads? What do you > recommend me to do? Why is it damn slow with 8K blocksize? Some quick tests show that ufs does do rather poorly on my system too. I have the following filesystems setup: /var : ufs with softupdates /usr/obj : zfs with checksums disabled /usr/src : zfs with compression enabled /home : zfs with compression disabled and checksums enabled I ran a test with a blocksize of 8KB and 16 threads. /var : 25.2MB/s /usr/obj : 64.8MB/s /usr/src : 386.3MB/s /home : 60.3MB/s -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 04:16:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 674211065670; Sat, 5 Jun 2010 04:16:15 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 13AF88FC19; Sat, 5 Jun 2010 04:16:14 +0000 (UTC) Received: by iwn5 with SMTP id 5so2183067iwn.13 for ; Fri, 04 Jun 2010 21:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=ILvFO3arqbsAjuHT5LZyTRVuYXC0iVjYy9w1sSAfBus=; b=jO9UCsv8B7Zrusy7QyM5KNLoGm0y1QWduRvFhwQK5PajkoAVgf0xG79v7pO41i7zEK ZiGF+pcZk1TDXG7/ndpY81XbJe4ONiddMO/dpDzROP2t20y/vhKC8+I0xOEtyLnceKiY v8dBkNP8iJ+LUbv2b/EIw9Fg/Q/cjGG/w+9wI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=W1bI38BrU29hDCydH5YtZufNAft94Vl7Dt79v3+TCOHTt1/Rv9lRnDAmE3adzWUWN4 jLxMgiNJcJ0X/o/USd+/cWEzD3zULh7uRhE21ZV8qcjo6wnuGwCcOFbRv20TjUeE6hHU 4wIIBJEm965oUKYXlC9KxpeutKkUL6P5crKY8= Received: by 10.231.178.162 with SMTP id bm34mr14469160ibb.86.1275711374293; Fri, 04 Jun 2010 21:16:14 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-128-180.dsl.klmzmi.sbcglobal.net [99.181.128.180]) by mx.google.com with ESMTPS id b3sm8506163ibf.19.2010.06.04.21.16.12 (version=SSLv3 cipher=RC4-MD5); Fri, 04 Jun 2010 21:16:13 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C09CF8B.5030006@dataix.net> Date: Sat, 05 Jun 2010 00:16:11 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: Sean Bruno References: <1274739973.31299.23.camel@localhost.localdomain> <4BFBD838.40208@dataix.net> <1274798852.4715.1.camel@localhost.localdomain> <1274894969.2481.47.camel@localhost.localdomain> <4BFD9325.50905@dataix.net> In-Reply-To: <4BFD9325.50905@dataix.net> X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers , "sbruno@freebsd.org" Subject: Re: Exposing Zone Sleeps X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 04:16:15 -0000 On 05/26/2010 17:31, jhell wrote: > On 05/26/2010 13:29, Sean Bruno wrote: >> On Tue, 2010-05-25 at 07:47 -0700, Sean Bruno wrote: >>>> Hi Sean, >>>> >>>> Nice work on this. I applied this to stable/8 r208530 and I am in the >>>> process of compiling the kernel right now. Everything else has built & >>>> runs as expected "i386". Attached is the adjusted patch which was one >>>> modification to the line number for uz_sleeps in sys/vm/uma_int.h. >>>> >>>> 8 files changed, 106 insertions(+), 7 deletions(-) >>>> >>>> For those wishing to apply this patch and test for them self: >>>> >>>> cd /usr/src >>>> patch >>> cd /usr/src/include >>>> make obj && make depend && make includes && make install >>>> cd /usr/src/lib/libmemstat >>>> make obj && make depend && make includes && make install >>>> cd /usr/src/usr.bin/vmstat >>>> make obj && make depend && make install >>>> cd /usr/src >>>> make kernel KERNCONF=YOUR_KERN_CONF >>>> reboot >>>> >>>> Can't wait to see some results from this & I will report back with >>>> either negative results of the build & run or positive results from the >>>> stats collected. >>>> >>>> If there is anything needed feel free to let me know and I will do what >>>> is possible ASAP. >>>> >>>> Thanks again, >>>> >>>> - -- >>>> >>>> jhell >>> >>> Excellent. Please check the output of vmstat -z and the appropriate >>> sysctl. I changed the display a bit to keep it from wrapping on a >>> standard terminal. >>> >>> Sean >>> >>> P.S. My intention it to MFC this to all releases. >>> >> >> I do have a concern related to the removal of an #ifdef DDB in this >> patch. Any comments? >> >> Sean > > This was in your original patch sent to the list. I questioned it too > but as far as testing it goes it has caused no harm that I can see here > but I will add those back in along with the improvements from Garrett, > then regenerate the patch and send it back to the list. > > Regards, > Following up on this, it has been ~ 1 -> 2 weeks with this patch on 4 machines I have had in question and have not noticed any negative effects or gains from what this patch provides. On a stable/8 i386 system there were absolutely no SLEEP stat bumps recorded at all. On a second note one negative impact that I had observed is the negative ability to upgrade a stable/7 system to a stable/8 system with this patch applied due to mismatched symbols or other related symptoms that would suggest that the patch would also have to be applied to the older system and compiled before the upgrade. This all had more to do with 'vm_uma_zone' being undeclared. Although this is a nice feature I do not see any benefits for applying it to a stable/7 or stable/8 systems as those are already clean. But applying this to HEAD for debugging of sleep issues that may creep into the code would be useful as a type of alert until the next release at which it should be turned off and ejected from the tree unless it was separated into the src/tools/tools directory as a debugging tool. With all luck & kind regards, -- jhell From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 07:27:02 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 971961065670; Sat, 5 Jun 2010 07:27:02 +0000 (UTC) (envelope-from wooh@wooh.hu) Received: from mail.netidea.hu (netwarehouse.netidea.hu [195.228.254.126]) by mx1.freebsd.org (Postfix) with ESMTP id 4FF1B8FC0A; Sat, 5 Jun 2010 07:27:01 +0000 (UTC) Received: from radon (localhost [127.0.0.1]) by mail.netidea.hu (Postfix) with ESMTP id 9A9C7129359; Sat, 5 Jun 2010 09:26:29 +0200 (CEST) X-Virus-Scanned: amavisd-new at netidea.hu Received: from mail.netidea.hu ([127.0.0.1]) by radon (mail.netidea.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fDpAUVYMULTC; Sat, 5 Jun 2010 09:26:29 +0200 (CEST) Received: from jamms-MacBook.local (catv-89-132-136-110.catv.broadband.hu [89.132.136.110]) by mail.netidea.hu (Postfix) with ESMTP id 095A1129356; Sat, 5 Jun 2010 09:26:29 +0200 (CEST) Message-ID: <4C09FC43.8070804@wooh.hu> Date: Sat, 05 Jun 2010 09:26:59 +0200 From: Adam PAPAI User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bruce Cran References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> In-Reply-To: <201006050236.17697.bruce@cran.org.uk> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 07:27:02 -0000 On 6/5/10 3:36 AM, Bruce Cran wrote: > Some quick tests show that ufs does do rather poorly on my system too. I have > the following filesystems setup: > > /var : ufs with softupdates > /usr/obj : zfs with checksums disabled > /usr/src : zfs with compression enabled > /home : zfs with compression disabled and checksums enabled > > I ran a test with a blocksize of 8KB and 16 threads. > > /var : 25.2MB/s > /usr/obj : 64.8MB/s > /usr/src : 386.3MB/s > /home : 60.3MB/s > It seems I have to test it with zfs as well. Tomorrow I'm gonna test it. -- Adam PAPAI From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 11:04:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40EE510656D5; Sat, 5 Jun 2010 11:04:12 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [204.109.60.94]) by mx1.freebsd.org (Postfix) with ESMTP id 1F1BC8FC20; Sat, 5 Jun 2010 11:04:11 +0000 (UTC) Received: from unknown (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id AAF085C84; Sat, 5 Jun 2010 11:04:16 +0000 (UTC) Date: Sat, 5 Jun 2010 12:04:05 +0100 From: Bruce Cran To: Stefan Miklosovic Message-ID: <20100605120405.00007954@unknown> In-Reply-To: References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> X-Mailer: Claws Mail 3.7.4cvs1 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org, Adam PAPAI Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 11:04:12 -0000 On Sat, 5 Jun 2010 12:50:15 +0200 Stefan Miklosovic wrote: > > /var : ufs with softupdates > > /usr/obj : zfs with checksums disabled > > /usr/src : zfs with compression enabled > > /home : zfs with compression disabled and checksums enabled > > > > I ran a test with a blocksize of 8KB and 16 threads. > > > > /var : 25.2MB/s > > /usr/obj : 64.8MB/s > > /usr/src : 386.3MB/s > > /home : 60.3MB/s > > Do I understand it well? It seems that zfs with compression enabled on > /usr/src with 8KB block size and 16 threads performs 386.3MB/s which > is about 6 times better than debian5? I am thinking about this image > http://tech-blog.wooh.hu/~wooh/debian_vs_freebsd_io_16_seqwr.png Yes - on one run it even hit 500MB/s. I suspect, however, that the benchmark isn't accurate because it won't be writing typical data. Instead it's probably using a buffer that compresses very well. -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 11:17:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48D5B106566B; Sat, 5 Jun 2010 11:17:57 +0000 (UTC) (envelope-from wooh@wooh.hu) Received: from mail.netidea.hu (netwarehouse.netidea.hu [195.228.254.126]) by mx1.freebsd.org (Postfix) with ESMTP id 00BC48FC16; Sat, 5 Jun 2010 11:17:56 +0000 (UTC) Received: from radon (localhost [127.0.0.1]) by mail.netidea.hu (Postfix) with ESMTP id 463A5129293; Sat, 5 Jun 2010 13:17:24 +0200 (CEST) X-Virus-Scanned: amavisd-new at netidea.hu Received: from mail.netidea.hu ([127.0.0.1]) by radon (mail.netidea.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5KpeR6XT2xRC; Sat, 5 Jun 2010 13:17:23 +0200 (CEST) Received: from jamms-MacBook.local (catv-89-134-206-111.catv.broadband.hu [89.134.206.111]) by mail.netidea.hu (Postfix) with ESMTP id 71A7F12925C; Sat, 5 Jun 2010 13:17:23 +0200 (CEST) Message-ID: <4C0A3262.8010507@wooh.hu> Date: Sat, 05 Jun 2010 13:17:54 +0200 From: Adam PAPAI User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <20100605120405.00007954@unknown> In-Reply-To: <20100605120405.00007954@unknown> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 11:17:57 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 6/5/10 1:04 PM, Bruce Cran wrote: > On Sat, 5 Jun 2010 12:50:15 +0200 > Stefan Miklosovic wrote: > >>> /var : ufs with softupdates >>> /usr/obj : zfs with checksums disabled >>> /usr/src : zfs with compression enabled >>> /home : zfs with compression disabled and checksums enabled >>> >>> I ran a test with a blocksize of 8KB and 16 threads. >>> >>> /var : 25.2MB/s >>> /usr/obj : 64.8MB/s >>> /usr/src : 386.3MB/s >>> /home : 60.3MB/s >> >> Do I understand it well? It seems that zfs with compression enabled on >> /usr/src with 8KB block size and 16 threads performs 386.3MB/s which >> is about 6 times better than debian5? I am thinking about this image >> http://tech-blog.wooh.hu/~wooh/debian_vs_freebsd_io_16_seqwr.png > > Yes - on one run it even hit 500MB/s. I suspect, however, that the > benchmark isn't accurate because it won't be writing typical data. > Instead it's probably using a buffer that compresses very well. Hm.. My ZFS tests showed me the same results. With compression it's pretty fast. An application benchmark will give us typical data write, so I'll run PgSQL benchmarks on the ZFS pool as well. - -- Adam PAPAI NETIDEA Informatikai Szolgáltató Kft. http://www.netidea.hu E-mail: wooh@wooh.hu Phone: +36 30 33-55-735 (Hungary) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMCjJiAAoJEGq0EWvh5uiIKFMH/1dP4OZGAMiBNSoRqGFfnZ5B /vtf5do2t3JRbjfYi2HyNn8gXss4xRDPouVmftl2OglIXA77hMIyIcjyoWnHGTBc M1WnnNDz1wIb8EYSl9MYKAjQA1wGsYd4UImd1MqOtZfSuOht6hTLoSiAnC1xMLtk 9vgFUtMok8XclPqL08J/dWs39+HwhSaooRnLEx7IYLSgFis7vQtJjOaWWG3LUADw QsivcCSjBBoQ7LD9WXN5prmlwt+CMBU/F1yyMaJXa0bNI7AM+hh5Mix03P4HAKEz 4Z92lcmLXzSVnllA0tAJvAwEPtk4laP6yzM9egStDNvxONLueQVLXfY8gvukQ2k= =MVI2 -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 11:17:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED9CA1065670 for ; Sat, 5 Jun 2010 11:17:12 +0000 (UTC) (envelope-from miklosovic.freebsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 814628FC13 for ; Sat, 5 Jun 2010 11:17:12 +0000 (UTC) Received: by fxm20 with SMTP id 20so1301668fxm.13 for ; Sat, 05 Jun 2010 04:17:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=acEZEmBuS0Z+yJD3czmBz58nMSgDqaEqoeDfVuB/PJg=; b=oeMFHYNZ948X8RQLibS3W1YAZksGSF3BLJhEqk7vfe8dKlIcNw1QlGC+A2OLm62fWU 2bB8h3Onju1EfHg2DumzCKmrK6WL6JOFFxBwmbztGunrYIX6LJDJMInpItK/jMoTyCpD lzWemWr/270YqDlGTARzxPsjRN0ysaffREYp8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nxf+JxqZ15vM6svNhaNwuQR2+JLDqxJ3G58KzOnrGOO1aR1AFTDLQCaV/fC0xuT9JI s4OwO4xcKlg/tqetq6n/MMlJ81xdW/96WdV9LoyZfp34nYFn9r8Ejghn3GuXF5/C1FBa TwRy/1GSmEJQ+eP6FKCqIWUT99olReRErOZcA= MIME-Version: 1.0 Received: by 10.103.78.31 with SMTP id f31mr4246974mul.79.1275735015688; Sat, 05 Jun 2010 03:50:15 -0700 (PDT) Received: by 10.103.172.11 with HTTP; Sat, 5 Jun 2010 03:50:15 -0700 (PDT) In-Reply-To: <201006050236.17697.bruce@cran.org.uk> References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> Date: Sat, 5 Jun 2010 12:50:15 +0200 Message-ID: From: Stefan Miklosovic To: Bruce Cran Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 05 Jun 2010 11:31:12 +0000 Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org, Adam PAPAI Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 11:17:13 -0000 > /var : ufs with softupdates > /usr/obj : zfs with checksums disabled > /usr/src : zfs with compression enabled > /home : zfs with compression disabled and checksums enabled > > I ran a test with a blocksize of 8KB and 16 threads. > > /var : 25.2MB/s > /usr/obj : 64.8MB/s > /usr/src : 386.3MB/s > /home : 60.3MB/s Do I understand it well? It seems that zfs with compression enabled on /usr/src with 8KB block size and 16 threads performs 386.3MB/s which is about 6 times better than debian5? I am thinking about this image http://tech-blog.wooh.hu/~wooh/debian_vs_freebsd_io_16_seqwr.png what is your system specs? From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 13:41:52 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0B611065675; Sat, 5 Jun 2010 13:41:52 +0000 (UTC) (envelope-from mozolevsky@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1DEEB8FC18; Sat, 5 Jun 2010 13:41:51 +0000 (UTC) Received: by vws18 with SMTP id 18so292996vws.13 for ; Sat, 05 Jun 2010 06:41:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received :in-reply-to:references:from:date:x-google-sender-auth:message-id :subject:to:cc:content-type; bh=SpsVcLzNlkM5HruhAmPivxnscu2UgHQMEuLzIFy1LtA=; b=VwRIzrmnm3q/femSJtmgJVspyyb121B+Sq909cYy4lIM9a3srjxUzJEpU4pXlMRFrF XnteLUmLsM1BqCvbGSDobRZAdhPtTPCgynQaNTGH0I+rEP+paAIStQg+qoLeWWm3M3qB opofK2ii3ETeF0zvBXsfDrPbeO+pHX8IqRfdE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=jeG2+8eBELICIxWUX9kiSsZ+DPe47TCmRexGmmaZiCwI6iTAxAn5O0oxWbP8L68jFm V+8+AmJFd9CznD+CwSZfWbtLqEHDVP0yG77bH0AG8zW+DVLFjCEqT65wR59J3f/F8GR1 ng87dQslAVOB6uJdfwN1xnTMuhJ0MRcyMeQ+I= Received: by 10.229.188.70 with SMTP id cz6mr3200115qcb.161.1275745310598; Sat, 05 Jun 2010 06:41:50 -0700 (PDT) MIME-Version: 1.0 Sender: mozolevsky@gmail.com Received: by 10.229.236.15 with HTTP; Sat, 5 Jun 2010 06:41:29 -0700 (PDT) In-Reply-To: <4C0A3262.8010507@wooh.hu> References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <20100605120405.00007954@unknown> <4C0A3262.8010507@wooh.hu> From: Igor Mozolevsky Date: Sat, 5 Jun 2010 14:41:29 +0100 X-Google-Sender-Auth: B9ULtr5oBQ2QqXcR8_qw2Wi8XY4 Message-ID: To: Adam PAPAI Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, freebsd-questions@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 13:41:52 -0000 >>>> /usr/src : zfs with compression enabled >>>> /usr/src : 386.3MB/s >>> Do I understand it well? It seems that zfs with compression enabled on >>> /usr/src with 8KB block size and 16 threads performs 386.3MB/s which >>> is about 6 times better than debian5? I am thinking about this image >>> http://tech-blog.wooh.hu/~wooh/debian_vs_freebsd_io_16_seqwr.png >> >> Yes - on one run it even hit 500MB/s. I suspect, however, that the >> benchmark isn't accurate because it won't be writing typical data. >> Instead it's probably using a buffer that compresses very well. > > Hm.. My ZFS tests showed me the same results. With compression it's > pretty fast. That's hardly a surprise - you take the source code, compress it into virtual non-existence leaving hardly anything to be written to the disk... Obviously if compression speed >> IO speed and the result of the compression is a significant reduction in size, you have a massive gain in writing that data to the disk. -- Igor From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 16:45:28 2010 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7216F1065673 for ; Sat, 5 Jun 2010 16:45:28 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outj.internet-mail-service.net [216.240.47.233]) by mx1.freebsd.org (Postfix) with ESMTP id 445AD8FC20 for ; Sat, 5 Jun 2010 16:45:28 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o55GjQXq008686; Sat, 5 Jun 2010 09:45:27 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 763EC2D6013; Sat, 5 Jun 2010 09:45:24 -0700 (PDT) Message-ID: <4C0A7F2F.3030105@elischer.org> Date: Sat, 05 Jun 2010 09:45:35 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Adam PAPAI References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> In-Reply-To: <4C09FC43.8070804@wooh.hu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: FreeBSD Hackers Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 16:45:28 -0000 On 6/5/10 12:26 AM, Adam PAPAI wrote: > On 6/5/10 3:36 AM, Bruce Cran wrote: >> Some quick tests show that ufs does do rather poorly on my system too. I have >> the following filesystems setup: >> >> /var : ufs with softupdates >> /usr/obj : zfs with checksums disabled >> /usr/src : zfs with compression enabled >> /home : zfs with compression disabled and checksums enabled >> >> I ran a test with a blocksize of 8KB and 16 threads. >> >> /var : 25.2MB/s >> /usr/obj : 64.8MB/s >> /usr/src : 386.3MB/s >> /home : 60.3MB/s >> > > It seems I have to test it with zfs as well. Tomorrow I'm gonna test it. > > Then the linux people will insist you use btrfs to compare apples to apples ... etc... etc.. From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 16:55:12 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A4A61065680 for ; Sat, 5 Jun 2010 16:55:12 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 546428FC19 for ; Sat, 5 Jun 2010 16:55:12 +0000 (UTC) Received: from [172.16.135.100] (lportal.in1.lcl [172.16.1.9]) by ns1.feral.com (8.14.3/8.14.3) with ESMTP id o55GtBUu043006 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Sat, 5 Jun 2010 09:55:11 -0700 (PDT) (envelope-from mj@feral.com) Message-ID: <4C0A816A.9040403@feral.com> Date: Sat, 05 Jun 2010 09:55:06 -0700 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> In-Reply-To: <4C0A7F2F.3030105@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.3 (ns1.feral.com [192.168.221.1]); Sat, 05 Jun 2010 09:55:11 -0700 (PDT) Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 16:55:12 -0000 All of these tests have been apples vs. oranges for years. The following seems to be true, though: a) FreeBSD sequential write performance in UFS has always been less than optimal. b) Linux sequential write performance in just about any filesystem has always been "impressive". But that "impressive" has come at some not so obvious costs. First of all, Linux is probably the most aggressive cluster/write-behind OS I've even seen. You can suck down all available memory with writebehind using dd. This means that some stats are "impressive", and others are "painful". A desktop that becomes completely unresponsive while you're doing this dd is one personal outcome. Also, you have to be careful what you're asking for in comparing the two platforms, or any platforms for that matter. What do you want to optimize for? Apparent responsiveness as a desktop? A specific workload (nfs, cifs) that completes N quatloos per fortnight? From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 17:30:53 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05EC81065677 for ; Sat, 5 Jun 2010 17:30:53 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.8]) by mx1.freebsd.org (Postfix) with ESMTP id 8FB8D8FC13 for ; Sat, 5 Jun 2010 17:30:52 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-044-227.pools.arcor-ip.net [88.66.44.227]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0Lzpcx-1POerJ3jHv-01544q; Sat, 05 Jun 2010 19:30:51 +0200 Received: (qmail 96510 invoked from network); 5 Jun 2010 17:30:50 -0000 Received: from f8x64.laiers.local (192.168.4.188) by mx.laiers.local with SMTP; 5 Jun 2010 17:30:50 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Sat, 5 Jun 2010 19:30:49 +0200 User-Agent: KMail/1.13.3 (FreeBSD/8.0-RELEASE-p2; KDE/4.4.3; amd64; ; ) References: <4C09932B.6040808@wooh.hu> In-Reply-To: <4C09932B.6040808@wooh.hu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201006051930.49804.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+9+btBBK1a8iGXqHjfTNkC/z6u5DFT7+VNwln J5Yr96UDlRIWj4ElVtABmRW+KaMWTL8HqZlomxHg//imhjTa10 QMH5JuCZqJBLCAj92Qhag== Cc: Adam PAPAI , freebsd-questions@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 17:30:53 -0000 On Saturday 05 June 2010 01:58:35 Adam PAPAI wrote: > Why FreeBSD is supreme with 1 and 2 thread. And why is it 2 and 3 times > slower with 4-8-16-32 threads compared to Debian? The first two tests (1 > thread and 2 thread) showed me that FreeBSD is supreme in I/O, but later > tests showed me, that it can produce horrible I/O. > > How can I tune my disk to make it faster? Is it possible? What is the > reason of the really slow I/O with more than 4 threads? What do you > recommend me to do? Why is it damn slow with 8K blocksize? You may find this interesting: http://info.iet.unipi.it/~luigi/geom_sched/ Regards, Max From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 17:41:25 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0181B1065670 for ; Sat, 5 Jun 2010 17:41:25 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6A3838FC14 for ; Sat, 5 Jun 2010 17:41:24 +0000 (UTC) Received: by vws18 with SMTP id 18so589816vws.13 for ; Sat, 05 Jun 2010 10:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=9+CvwVtHCnnEfbJUVvjbwhzaNEhVwFT3x9zV5eV2qXs=; b=FhAc0EUlVMYwRG+k+SJDzazc/vicVJ05HkeJuxhFQ1sz3KDM713/FPG3KNAxSojhcO aSZbIO5pjIK+M+T5F393LiTIlRx6qJ+YqxtJgQCf72qL+nkOYplaYs/YaDe6WAA65GVc 3Z4MxIVOfYfPuFDwMB97+Iz7Jo5TSO4XvN7rI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Bx+QCUTzZ5oYHmnmGcdlSOwwp+41/aPDHTOXfNU5dM0lQd4P61ugg/8jBbwq70YJ+a wMJfGLZj5AQyMU2XJdxMfTnH89goTK1QYncC0gK9DLnp32iN5fT0tK5Yet/ZXMI1us+Y /BMoObo+8yWpgGILVDDQ2t4A0b5ntMM0bxHE4= MIME-Version: 1.0 Received: by 10.224.102.202 with SMTP id h10mr6118602qao.49.1275759683300; Sat, 05 Jun 2010 10:41:23 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.229.183.213 with HTTP; Sat, 5 Jun 2010 10:41:23 -0700 (PDT) In-Reply-To: <4C0A816A.9040403@feral.com> References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> Date: Sat, 5 Jun 2010 19:41:23 +0200 X-Google-Sender-Auth: UMPiUyfhhuEiV4Ia2QSAv_vYguA Message-ID: From: Attilio Rao To: Matthew Jacob Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 17:41:25 -0000 2010/6/5 Matthew Jacob > > All of these tests have been apples vs. oranges for years. > > The following seems to be true, though: > > a) FreeBSD sequential write performance in UFS has always been less than = optimal. > > b) Linux sequential write performance in just about any filesystem has al= ways been "impressive". But that "impressive" has come at some not so obvio= us costs. First of all, Linux is probably the most aggressive cluster/write= -behind OS I've even seen. You can suck down all available memory with writ= ebehind using dd. This means that some stats are "impressive", and others a= re "painful". A desktop that becomes completely unresponsive while you're d= oing this dd is one personal outcome. > > Also, you have to be careful what you're asking for in comparing the two = platforms, or any platforms for that matter. What do you want to optimize f= or? Apparent responsiveness as a desktop? A specific workload (nfs, cifs) t= hat completes N quatloos per fortnight? Besides anything, I'm much more concerned about the loss of performance within FreeBSD itself. I wouldn't expect a so high pessimization when the number of threads increases (without considering the big performance loss with the 8k blocksize, pretty much reproducible). I'm trying to drive, privately, the tester to pmc/lock profiling analysis in order to start collecting some useful datas. While I think that we might pay a lot of attention to ZFS, I think we might not leave alone FFS. Having a fast, well supported, native filesystem might be a great thing for us. Comparing with other operating systems, as you smartly point out, might not be got as 'undefeatable truths' but have cons and prons that needs to be fully understood before to make false claims. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 17:51:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB07D1065676; Sat, 5 Jun 2010 17:51:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 451278FC1B; Sat, 5 Jun 2010 17:51:30 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o55Hparv062661 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 5 Jun 2010 20:51:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o55HpNut039882; Sat, 5 Jun 2010 20:51:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o55HpNhS039881; Sat, 5 Jun 2010 20:51:23 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 5 Jun 2010 20:51:23 +0300 From: Kostik Belousov To: Attilio Rao Message-ID: <20100605175123.GY83316@deviant.kiev.zoral.com.ua> References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kl6COkrTq67Sn9pw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, Matthew Jacob Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 17:51:31 -0000 --kl6COkrTq67Sn9pw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 05, 2010 at 07:41:23PM +0200, Attilio Rao wrote: > 2010/6/5 Matthew Jacob > > > > All of these tests have been apples vs. oranges for years. > > > > The following seems to be true, though: > > > > a) FreeBSD sequential write performance in UFS has always been less tha= n optimal. > > > > b) Linux sequential write performance in just about any filesystem has = always been "impressive". But that "impressive" has come at some not so obv= ious costs. First of all, Linux is probably the most aggressive cluster/wri= te-behind OS I've even seen. You can suck down all available memory with wr= itebehind using dd. This means that some stats are "impressive", and others= are "painful". A desktop that becomes completely unresponsive while you're= doing this dd is one personal outcome. > > > > Also, you have to be careful what you're asking for in comparing the tw= o platforms, or any platforms for that matter. What do you want to optimize= for? Apparent responsiveness as a desktop? A specific workload (nfs, cifs)= that completes N quatloos per fortnight? >=20 > Besides anything, I'm much more concerned about the loss of > performance within FreeBSD itself. I wouldn't expect a so high > pessimization when the number of threads increases (without > considering the big performance loss with the 8k blocksize, pretty > much reproducible). I'm trying to drive, privately, the tester to > pmc/lock profiling analysis in order to start collecting some useful > datas. Are the benchmarks create threads that write to the same file ? If yes, then this behaviour is well understood. > While I think that we might pay a lot of attention to ZFS, I think we > might not leave alone FFS. Having a fast, well supported, native > filesystem might be a great thing for us. >=20 > Comparing with other operating systems, as you smartly point out, > might not be got as 'undefeatable truths' but have cons and prons that > needs to be fully understood before to make false claims. >=20 > Thanks, > Attilio >=20 >=20 > -- > Peace can only be achieved by understanding - A. Einstein > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --kl6COkrTq67Sn9pw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwKjpsACgkQC3+MBN1Mb4gJZwCcDri6qMQ/Lq+HbhDZo1Wu49Eg MbsAoPBpV4igzN33oT8Jhm9GK/bIbQ8Z =erbj -----END PGP SIGNATURE----- --kl6COkrTq67Sn9pw-- From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 20:12:43 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B195106564A for ; Sat, 5 Jun 2010 20:12:43 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 679018FC13 for ; Sat, 5 Jun 2010 20:12:43 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id C79345B52 for ; Sat, 5 Jun 2010 13:12:42 -0700 (PDT) To: freebsd-hackers@freebsd.org Date: Sat, 05 Jun 2010 13:12:42 -0700 From: Bakul Shah Message-Id: <20100605201242.C79345B52@mail.bitblocks.com> Subject: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 20:12:43 -0000 Consider: $ yes | cat -n | (read a; echo $a; head -1) 1 y 2 y $ yes | cat -n | (head -1; read a; echo $a) 1 y 456 y As you can see, head reads far more than it should. This is fine most of the time but often it results in surprising output: # print ps header and all lines with sh in it $ ps|(head -1; grep sh) PID TT STAT TIME COMMAND # print first and last two lines $ look xa | (head -2; tail -2) xanthaline xanthamic Not quite what you expected, right? Yes, you can use read and echo N times but this is not as convenient as using head: $ look xa | (read a; echo $a; read a; echo $a; tail -2) xanthaline xanthamic xarque Xaverian The "fix" is to make sure head reads no more than $N bytes where $N is the number of *remaining* lines to be read. Yes this slows head down some but makes it more useful. [Ideally all commands that quit after partially reading their input ought to behave like this but that would slow down their common use far too much] Comments? Thanks to Rob Warnock for pointing out the head problem. From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 20:32:11 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1CB33106566B for ; Sat, 5 Jun 2010 20:32:11 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id A1DAF8FC08 for ; Sat, 5 Jun 2010 20:32:10 +0000 (UTC) Received: (qmail 8834 invoked by uid 399); 5 Jun 2010 20:32:09 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 5 Jun 2010 20:32:09 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C0AB448.2040104@FreeBSD.org> Date: Sat, 05 Jun 2010 13:32:08 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100330 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bakul Shah References: <20100605201242.C79345B52@mail.bitblocks.com> In-Reply-To: <20100605201242.C79345B52@mail.bitblocks.com> X-Enigmail-Version: 1.0.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 20:32:11 -0000 On 06/05/10 13:12, Bakul Shah wrote: > Consider: > > $ yes | cat -n | (read a; echo $a; head -1) > 1 y > 2 y > > $ yes | cat -n | (head -1; read a; echo $a) > 1 y > 456 y It's not at all clear to me what you are trying to accomplish here. If what you want is to read only the first line of the output of yes, then what you'd want to do is: yes | cat -n | head -1 | (read a; echo $a) 1 y > As you can see, head reads far more than it should. This is > fine most of the time but often it results in surprising > output: > > # print ps header and all lines with sh in it > $ ps|(head -1; grep sh) > PID TT STAT TIME COMMAND I don't understand why you think this would work. There is no input to the grep command. The only reason it exits at all is that you are executing in a subshell. > # print first and last two lines > $ look xa | (head -2; tail -2) > xanthaline > xanthamic Same problem here. There is no input to the tail command. Doug -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 20:48:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87F2C1065672 for ; Sat, 5 Jun 2010 20:48:46 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 48C9A8FC08 for ; Sat, 5 Jun 2010 20:48:46 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 6E56B5B52; Sat, 5 Jun 2010 13:48:45 -0700 (PDT) To: Doug Barton In-reply-to: Your message of "Sat, 05 Jun 2010 13:32:08 PDT." <4C0AB448.2040104@FreeBSD.org> References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> Comments: In-reply-to Doug Barton message dated "Sat, 05 Jun 2010 13:32:08 -0700." Date: Sat, 05 Jun 2010 13:48:45 -0700 From: Bakul Shah Message-Id: <20100605204845.6E56B5B52@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 20:48:46 -0000 On Sat, 05 Jun 2010 13:32:08 PDT Doug Barton wrote: > On 06/05/10 13:12, Bakul Shah wrote: > > Consider: > > > > $ yes | cat -n | (read a; echo $a; head -1) > > 1 y > > 2 y > > > > $ yes | cat -n | (head -1; read a; echo $a) > > 1 y > > 456 y > > It's not at all clear to me what you are trying to accomplish here. If > what you want is to read only the first line of the output of yes, then > what you'd want to do is: > > yes | cat -n | head -1 | (read a; echo $a) > 1 y > > > As you can see, head reads far more than it should. This is > > fine most of the time but often it results in surprising > > output: > > > > # print ps header and all lines with sh in it > > $ ps|(head -1; grep sh) > > PID TT STAT TIME COMMAND > > I don't understand why you think this would work. There is no input to > the grep command. The only reason it exits at all is that you are > executing in a subshell. > > > # print first and last two lines > > $ look xa | (head -2; tail -2) > > xanthaline > > xanthamic > > Same problem here. There is no input to the tail command. In general this is not true. Without running the following can you guess its output? $ look '' | (head -2; head -2) Will it produce A a or A a aa aal or A a sive abrastol or something else? Yes, we can always find a work around for a given case but the issue is that head buffers up more than it needs to. From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 21:02:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09902106566B for ; Sat, 5 Jun 2010 21:02:19 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 8D3B48FC0A for ; Sat, 5 Jun 2010 21:02:18 +0000 (UTC) Received: (qmail 12831 invoked by uid 399); 5 Jun 2010 21:02:17 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 5 Jun 2010 21:02:17 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C0ABB58.6030009@FreeBSD.org> Date: Sat, 05 Jun 2010 14:02:16 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100330 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bakul Shah References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <20100605204845.6E56B5B52@mail.bitblocks.com> In-Reply-To: <20100605204845.6E56B5B52@mail.bitblocks.com> X-Enigmail-Version: 1.0.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 21:02:19 -0000 On 06/05/10 13:48, Bakul Shah wrote: > Without running the following can you guess its output? > > $ look '' | (head -2; head -2) Again, it's not clear to me what you expect is going to happen with the second 'head -2' there. I agree that the actual output of your example is wacky and unexpected, but what I'm trying to get you say is what YOU think should happen. The examples that you pasted in your previous post did not and could not do what you said you wanted them to do, so I don't quite understand what the bug is. Put more simply, if you generate wacky command lines you should not be surprised when they produce wacky results. :) Doug On two occasions I have been asked [by members of Parliament!], "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. -- Charles Babbage -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 21:37:32 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 994B4106567C; Sat, 5 Jun 2010 21:37:32 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 7BD858FC19; Sat, 5 Jun 2010 21:37:32 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 816035B30; Sat, 5 Jun 2010 14:37:31 -0700 (PDT) To: Doug Barton In-reply-to: Your message of "Sat, 05 Jun 2010 14:02:16 PDT." <4C0ABB58.6030009@FreeBSD.org> References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <20100605204845.6E56B5B52@mail.bitblocks.com> <4C0ABB58.6030009@FreeBSD.org> Comments: In-reply-to Doug Barton message dated "Sat, 05 Jun 2010 14:02:16 -0700." Date: Sat, 05 Jun 2010 14:37:31 -0700 From: Bakul Shah Message-Id: <20100605213731.816035B30@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 21:37:32 -0000 On Sat, 05 Jun 2010 14:02:16 PDT Doug Barton wrote: > On 06/05/10 13:48, Bakul Shah wrote: > > Without running the following can you guess its output? > > > > $ look '' | (head -2; head -2) > > Again, it's not clear to me what you expect is going to happen with the > second 'head -2' there. I agree that the actual output of your example > is wacky and unexpected, but what I'm trying to get you say is what YOU > think should happen. The examples that you pasted in your previous post > did not and could not do what you said you wanted them to do, so I don't > quite understand what the bug is. There is no bug per se. What I am saying that it would be *less surprising* if $ look '' | (head -2; head -2) behaved the same as $ look '' | head -4 [And yes, I would use head -4 if I wanted four lines but the example was to illustrate the issue that head buffers more than it needs to]. It would be less surprising and more useful if $ ps | (head -1; grep ssh) showed PID TT STAT TIME COMMAND The change in head behaviour I am suggesting wouldn't break anything that already works but make it more useful for what you call 'wacky commands lines'! > Put more simply, if you generate wacky command lines you should not be > surprised when they produce wacky results. :) I didn't realize that use of ;(|) constitutes wackiness :-) They are simply exercising the power of shell! We have used these commands for so long that we take them for granted and we learn to avoid their use in such ways. When Rob Warnock first mentioned this, my initial reaction was the same as yours. But thinking more about it, I felt head can be made more useful with this change and felt it was worth bringing it to people's attention. But we can let it rest. From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 21:43:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DDDF1065670 for ; Sat, 5 Jun 2010 21:43:00 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 7672A8FC0C for ; Sat, 5 Jun 2010 21:43:00 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id D72AA5B30; Sat, 5 Jun 2010 14:42:59 -0700 (PDT) To: Mike Meyer In-reply-to: Your message of "Sat, 05 Jun 2010 17:02:42 EDT." <20100605170242.3bd614d3@bhuda.mired.org> References: <20100605201242.C79345B52@mail.bitblocks.com> <20100605170242.3bd614d3@bhuda.mired.org> Comments: In-reply-to Mike Meyer message dated "Sat, 05 Jun 2010 17:02:42 -0400." Date: Sat, 05 Jun 2010 14:42:59 -0700 From: Bakul Shah Message-Id: <20100605214259.D72AA5B30@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 21:43:00 -0000 On Sat, 05 Jun 2010 17:02:42 EDT Mike Meyer wrote: > As a general rule, programs don't expect to share their input with > other programs, nor do they make guarantees about what is and isn't > read from that input under those conditions. I'd say that shell > scripts that depend on what some command does with it's unprocessed > input are buggy. Fair enough. Thanks! From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 22:32:56 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED0F31065670 for ; Sat, 5 Jun 2010 22:32:56 +0000 (UTC) (envelope-from eng.mufic@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8692E8FC1C for ; Sat, 5 Jun 2010 22:32:56 +0000 (UTC) Received: by wyf28 with SMTP id 28so2260118wyf.13 for ; Sat, 05 Jun 2010 15:32:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=tLN7AIqJZgwVw4BTSdT7J4yxN3aLINPUccpPmkvex4s=; b=d6hduRj5JF2L+L6w+8v3Ni4vPM5jOLeKuNKuy06c+g+AWJq9Lk4Wi36xpXkbY/t+rD nmoUMA268bPK0W59GyhV0izYkE8J8Bx9ZA7XK1BTiUChEcso1BzL8xz5v1RZ3tcXnhj8 8urGXowinTIQ9tNgW3XUTs2GVf+ArFkduax9g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=M4rla7wNDJwwseFqNGd4GWGf2mOC5MtLqplxF4KiiFYAm/QdIpGV338YQmJ/474xc1 ddIK+60p12qTI7dgAR2J0EEiP7gpHQOPaNfoQMtHt0Y2RQcuETyEHannie16V8Zx/XDK fIxZ7E9Oc+R5ZtHX/WGn2ztytzYZWNnvmLMo8= MIME-Version: 1.0 Received: by 10.216.89.199 with SMTP id c49mr1003971wef.29.1275777175461; Sat, 05 Jun 2010 15:32:55 -0700 (PDT) Received: by 10.216.13.6 with HTTP; Sat, 5 Jun 2010 15:32:55 -0700 (PDT) Date: Sun, 6 Jun 2010 01:32:55 +0300 Message-ID: From: Mohammed Farrag To: hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Aurora Scheduler X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 22:32:57 -0000 Hi group, I am working on developing a new freebsd kernel theme for embedded. I was searching for the best scheduler. I think the best one is Aurora Scheduler developed by NASA. I found that it can be included in the configuration file in the freebsd but the problem was "Is it available for use by users"? the other one was where I can find its technique. Good luck, Mohammed Farrag From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 23:09:02 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC9AC106566B for ; Sat, 5 Jun 2010 23:09:02 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outq.internet-mail-service.net [216.240.47.240]) by mx1.freebsd.org (Postfix) with ESMTP id AFB2F8FC18 for ; Sat, 5 Jun 2010 23:09:02 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o55N9194024049; Sat, 5 Jun 2010 16:09:01 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 627E12D6012; Sat, 5 Jun 2010 16:08:59 -0700 (PDT) Message-ID: <4C0AD916.6070903@elischer.org> Date: Sat, 05 Jun 2010 16:09:10 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Mohammed Farrag References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: hackers@freebsd.org Subject: Re: Aurora Scheduler X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 23:09:02 -0000 On 6/5/10 3:32 PM, Mohammed Farrag wrote: > Hi group, > I am working on developing a new freebsd kernel theme for embedded. I was > searching for the best scheduler. I think the best one is Aurora Scheduler > developed by NASA. > I found that it can be included in the configuration file in the freebsd but > the problem was "Is it available for use by users"? the other one was where > I can find its technique. I'm somewhat confused. The aurora scheduler is for scheduling events such as meetings etc. I think. If I've got a teh wrong idea can you provide a pointer? > Good luck, > Mohammed Farrag > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 5 23:10:11 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2D8C106571D for ; Sat, 5 Jun 2010 23:10:11 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3C2458FC1B for ; Sat, 5 Jun 2010 23:10:06 +0000 (UTC) Received: by iwn5 with SMTP id 5so2783726iwn.13 for ; Sat, 05 Jun 2010 16:10:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=WwokK29ZJlPydi+r72YjCm6w50q70EUfs1E4JJQ5mQo=; b=Gz/RdA02/TDEgufPL4Qo/xChOMEpTdcG+/ZfrmUnp6h1DZ7epDIZ4CeBrpbSNCLB9D rvrp56v60ZPp7dXlq1qzl3c9vo0oyyNSMi/F533j815ZIew45XwOsmhV05JazfHQcBZ5 pCrnraySELK1W3QSDbDW1vkStQwWrwIy/6p7A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=eGQ6B2kFFfJga1Wzr3+/GmbHWeo/RtMHhO71+uTbW/lLUsTwHoEzbwOvMucWTLOAkq VfOzq2wFFV484OvJsvGOTGfNqCCRwRMPMaZQSdueXOj/pvEBoYaMWwb/3Kx/wPOGymoK EruwGm9/egHTvWTnD4KDYErO6kHMiQKONNvU4= Received: by 10.231.149.203 with SMTP id u11mr8906160ibv.6.1275779406461; Sat, 05 Jun 2010 16:10:06 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-128-180.dsl.klmzmi.sbcglobal.net [99.181.128.180]) by mx.google.com with ESMTPS id f1sm12536210ibg.15.2010.06.05.16.10.04 (version=SSLv3 cipher=RC4-MD5); Sat, 05 Jun 2010 16:10:05 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C0AD94B.4020602@dataix.net> Date: Sat, 05 Jun 2010 19:10:03 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: Mohammed Farrag References: In-Reply-To: X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: hackers@freebsd.org Subject: Re: Aurora Scheduler X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Jun 2010 23:10:12 -0000 On 06/05/2010 18:32, Mohammed Farrag wrote: > Hi group, > I am working on developing a new freebsd kernel theme for embedded. I was > searching for the best scheduler. I think the best one is Aurora Scheduler > developed by NASA. > I found that it can be included in the configuration file in the freebsd but > the problem was "Is it available for use by users"? the other one was where > I can find its technique. > Good luck, > Mohammed Farrag > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" SCHED_ULE or SCHED_4BSD Thats your two options with ULE being the default in recent 8.0+ systems. Use ( sysctl kern.sched.name ) to determine which one you are using. Unless I misunderstood the intention of your message I would say that you are a bit confused about what the Aurora Scheduler is/means as compared to what FreeBSD uses as sched_ule(4) or sched_4bsd(4). Regards, -- jhell