From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 00:16:52 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F8981065672 for ; Sun, 6 Jun 2010 00:16:52 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2207D8FC1E for ; Sun, 6 Jun 2010 00:16:51 +0000 (UTC) Received: by iwn5 with SMTP id 5so2811964iwn.13 for ; Sat, 05 Jun 2010 17:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:subject:x-enigmail-version:openpgp :content-type:content-transfer-encoding; bh=8+k/CDGKH4T9lVsc8pwCyIAU1Eftvj10+k6gS1cPsRA=; b=EQ7J1m2Q0c9lg2ovLhFW6FnJFePkxYv3NT2VbV7sjllzfVZKWNBp6aSee1046d/XOS vAT0Yes3CIfWr/wrAQee2OM1I1lbEQ6flyGt/cFgDGJLCIUX8276Qt4zV9IxT3mSQ/s9 SzLGKfrMaaiBEBcUyuKTDaGGTxnYIXVGUUM1M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=uxUY+cQAHfQf+vZm0RJbHtkOstvMhx8Cs48jc3boPUwuzu0lJ2EdWaBaZS958IaGs9 B26WEAeyWpvoqzbgdZ9qM3w7hNsBtFJ8qi4YgGgk+8dwOmeK0gWzJfllj6UsdPX5h0te /5r8JNrGETkFrXqRAW5GvUXvUekemDluAg1gI= Received: by 10.231.186.161 with SMTP id cs33mr15449239ibb.65.1275783411253; Sat, 05 Jun 2010 17:16:51 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-128-180.dsl.klmzmi.sbcglobal.net [99.181.128.180]) by mx.google.com with ESMTPS id d9sm12782288ibl.10.2010.06.05.17.16.50 (version=SSLv3 cipher=RC4-MD5); Sat, 05 Jun 2010 17:16:50 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C0AE8F1.8040606@dataix.net> Date: Sat, 05 Jun 2010 20:16:49 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: FreeBSD Hackers X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Fwd: Re: Aurora Scheduler X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 00:16:52 -0000 -------- Original Message -------- Subject: Re: Aurora Scheduler Date: Sat, 05 Jun 2010 20:15:49 -0400 From: jhell To: Mohammed Farrag On 06/05/2010 19:23, Mohammed Farrag wrote: > Hi all, > thanx for ur reply. First Aurora shows the schedule in a series of > graphical displays that allow the user to see the resource allocations and > the temporal relationships among the elements. This display also allows the > user to edit the schedule directly and easily. Aurora focuses on resource > requirements and temporal scheduling in combination. > see that : > http://www.stottlerhenke.com/products/aurora/?gclid=CNn85aj0iaICFVKX2Aodam_wVQ#challenge > and http://sbir.nasa.gov/SBIR/successes/ss/10-020text.html > it is used in very large projects and used to manage resources. I was > asking about the algorithm it use in scheduling process and we can apply it > to FreeBSD processes for scheduling. > Second, I know that it is not available in native FreeBSD so I intend to > build it manually but I just need the algorithm it use to schedule processes > based on resources and complex constraints. > Good Luck, > Mohammed Farrag > I'm afraid that you are confused about what the scheduler in the FreeBSD kernel does and what the Aurora Scheduler is. They are not relational to one another in any such way. http://sbir.nasa.gov/SBIR/successes/ss/10-020text.html There is not a similiar scheduling program in FreeBSD but there is Evolution, Kontact and other such things to manage your time. Good Luck -- jhell From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 00:21:46 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B541E106566C for ; Sun, 6 Jun 2010 00:21:46 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5CFDE8FC22 for ; Sun, 6 Jun 2010 00:21:46 +0000 (UTC) Received: by vws18 with SMTP id 18so970142vws.13 for ; Sat, 05 Jun 2010 17:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=qpbn9Qv9FXjrL1CTvyo871ji4pigx1TQ+Be8EePLOV4=; b=EGKER+cR8HlmBv8WQqqXsBV3gYvTkOrpdP/hq3brx+0xzyJsJQ+SukeKhE+XQtTtqI Mdgf7gS28tA9yjQ4YvogwV1DDDSbbDQvT8X7p1dZz9Nf/WWPM39DyshYAum4OaLorGbr GhBUQcA1S1gq5nSeaZOnu+TXSps4eCi5cJd2k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=ofHB/iscTGHc9/QEfJYeoJXKpbUnyWWpqzielfLK7BJy3oosBJp0tQKhPoHn8D/eWv RugCxVWToDeeYQwk5nm3lM3+jLGYeGqxwUgXhCZ7Y3I6z8/PGaMJRLx+r5v+fmfcJqxd du35G3lcn9qIxwGysVokNYHr82Pqqxez9eCJc= MIME-Version: 1.0 Received: by 10.224.86.200 with SMTP id t8mr6062702qal.50.1275783705330; Sat, 05 Jun 2010 17:21:45 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.229.183.213 with HTTP; Sat, 5 Jun 2010 17:21:45 -0700 (PDT) In-Reply-To: <20100605175123.GY83316@deviant.kiev.zoral.com.ua> References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> <20100605175123.GY83316@deviant.kiev.zoral.com.ua> Date: Sun, 6 Jun 2010 02:21:45 +0200 X-Google-Sender-Auth: bggLgAtXACRcNkfe2uaVi_nIo74 Message-ID: From: Attilio Rao To: Kostik Belousov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Matthew Jacob Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 00:21:46 -0000 2010/6/5 Kostik Belousov > > On Sat, Jun 05, 2010 at 07:41:23PM +0200, Attilio Rao wrote: > > 2010/6/5 Matthew Jacob > > > > > > All of these tests have been apples vs. oranges for years. > > > > > > The following seems to be true, though: > > > > > > a) FreeBSD sequential write performance in UFS has always been less t= han optimal. > > > > > > b) Linux sequential write performance in just about any filesystem ha= s always been "impressive". But that "impressive" has come at some not so o= bvious costs. First of all, Linux is probably the most aggressive cluster/w= rite-behind OS I've even seen. You can suck down all available memory with = writebehind using dd. This means that some stats are "impressive", and othe= rs are "painful". A desktop that becomes completely unresponsive while you'= re doing this dd is one personal outcome. > > > > > > Also, you have to be careful what you're asking for in comparing the = two platforms, or any platforms for that matter. What do you want to optimi= ze for? Apparent responsiveness as a desktop? A specific workload (nfs, cif= s) that completes N quatloos per fortnight? > > > > Besides anything, I'm much more concerned about the loss of > > performance within FreeBSD itself. I wouldn't expect a so high > > pessimization when the number of threads increases (without > > considering the big performance loss with the 8k blocksize, pretty > > much reproducible). I'm trying to drive, privately, the tester to > > pmc/lock profiling analysis in order to start collecting some useful > > datas. > Are the benchmarks create threads that write to the same file ? > If yes, then this behaviour is well understood. Actually I still don't know as I just sent an e-mail to the tester and he didn't followup still. However I'm not entirely sure this is a full bottleneck which may be reconduit to missing of byte-range locking. I want to dig more and better understand what's going on exactly. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 01:56:31 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF9C3106566C; Sun, 6 Jun 2010 01:56:31 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id 6B0538FC14; Sun, 6 Jun 2010 01:56:31 +0000 (UTC) Received: from mobileKamikaze.norad (HSI-KBW-078-042-098-160.hsi3.kabel-badenwuerttemberg.de [78.42.98.160]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id 36B4B8A1EEB; Sun, 6 Jun 2010 03:56:29 +0200 (CEST) Message-ID: <4C0B004D.9090706@bsdforen.de> Date: Sun, 06 Jun 2010 03:56:29 +0200 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.9) Gecko/20100331 Thunderbird/3.0.4 MIME-Version: 1.0 To: Bakul Shah References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <20100605204845.6E56B5B52@mail.bitblocks.com> <4C0ABB58.6030009@FreeBSD.org> <20100605213731.816035B30@mail.bitblocks.com> In-Reply-To: <20100605213731.816035B30@mail.bitblocks.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Doug Barton Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 01:56:31 -0000 On 05/06/2010 23:37, Bakul Shah wrote: > On Sat, 05 Jun 2010 14:02:16 PDT Doug Barton wrote: > It would be less surprising and more useful if > > $ ps | (head -1; grep ssh) > > showed > > PID TT STAT TIME COMMAND > > > The change in head behaviour I am suggesting wouldn't break > anything that already works but make it more useful for what > you call 'wacky commands lines'! I know this is besides the point you want to make, but I just cannot resist: # ps x | sed -e 1P -e '/ssh/\!d' -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 05:23:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 321E11065677 for ; Sun, 6 Jun 2010 05:23:16 +0000 (UTC) (envelope-from shrikanth07@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id DDF0D8FC08 for ; Sun, 6 Jun 2010 05:23:15 +0000 (UTC) Received: by vws18 with SMTP id 18so1230926vws.13 for ; Sat, 05 Jun 2010 22:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=KARo0KPMl50tFtO5b+eI8P4DkdT5pllvbWjkFkKEfuc=; b=UTO70qfYPNj50ph3aDKr2Wh1WdJW8O5NL7LDCakQaZ44scPZ9CuF4bKItoCr0ZFRXj pHOF2bWdixnqKLcTr9wOXHCiYVPaANsf2HqDRBUv4YAqZtxpJSM+GZg/NBqNPWPOfdH1 16NVVaWtc034KXtG7TCbt/jKZ8lXT4ggwPoPE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=vxjDBZ5bsbBQtx0MQtN3xe5Tn+to4pky0E9v9AQXQ4O5+p9TTx+wwmp5fIJs9lAJDE yNqJaj+UnibWkRanhqcVxg+VRCX+XyTluW07AA6Gp5sr+e7vNmJkbJ0Ip9oBF3rEA3LH Eg5w2+KKIRa4GOvauE15eOmep5Q2hCZDUB7pc= MIME-Version: 1.0 Received: by 10.229.181.203 with SMTP id bz11mr3738284qcb.94.1275801794921; Sat, 05 Jun 2010 22:23:14 -0700 (PDT) Received: by 10.229.88.140 with HTTP; Sat, 5 Jun 2010 22:23:14 -0700 (PDT) Date: Sun, 6 Jun 2010 10:53:14 +0530 Message-ID: From: Shrikanth Kamath To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ctfconvert : failed to resolve types X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 05:23:16 -0000 I have a small snippet trying to study ctfconvert... struct abc { int length; int bit; union { char key[0]; char *key_ptr[0]; } keys; }; int main() { int a =1; struct abc member = { 16, 5 }; printf("Sizeof abc structure [%d] \n", sizeof(struct abc)); } % gcc -g ctfconvert_prob.c % ctfconvert a.out I get the following error on running "ctfconvert"... ctfconvert_prob.c: failed to resolve the following types: struct 362 <16a>: failed to size member "keys" of type __anon__ (297 <129>) ERROR: ctfconvert_prob.c: failed to resolve types How can ctfconvert be equipped to resolve this data type? -- Shrikanth R K From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 07:18:11 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FA9A106567A for ; Sun, 6 Jun 2010 07:18:11 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id D60478FC0A for ; Sun, 6 Jun 2010 07:18:10 +0000 (UTC) Received: by pwj1 with SMTP id 1so1419294pwj.13 for ; Sun, 06 Jun 2010 00:18:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=SRcqNAltN731eckdAwCuvi6qpz0tXKbxuW8zMc7WS+8=; b=WBO2bWI6EI/RwHViCe9+r+Zjc48LBvTkFkrxStJfhp5BlxLcPZ/ys7SdTo/05W7yb2 zaAjBs6tMw81iFmw//hJAJH7ufwpwwHhCdqIcnIp8JJn2Z0iyy46GO0yeL4M6EUQEIXj X+7rTPBPiMzNvDE/kQ0HSKQi++vme/4jZrwAs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=uJrVqGyKAf7k5NXOXHnuhao+7hYpWgzm6IQphOAJOkWE6qzLfCQ62Kjwvh+cUUBdKq sT5W7TakJZCQzrZ/97VEz8Y4tALvK4XfWyS5UHqVvIVppzPoey2ZnNbk2kMAEHt2zji7 jmIqDryAbfrG9qa/B9wM41tWTkGCB1ZhBaQhQ= MIME-Version: 1.0 Received: by 10.141.214.40 with SMTP id r40mr10633721rvq.11.1275808690005; Sun, 06 Jun 2010 00:18:10 -0700 (PDT) Sender: artemb@gmail.com Received: by 10.141.40.4 with HTTP; Sun, 6 Jun 2010 00:18:09 -0700 (PDT) In-Reply-To: References: Date: Sun, 6 Jun 2010 00:18:09 -0700 X-Google-Sender-Auth: gNo7E6eGF8Wj95SoMKtD2nTecTU Message-ID: From: Artem Belevich To: Shrikanth Kamath Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: ctfconvert : failed to resolve types X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 07:18:11 -0000 Interesting. Looking at dwarf parsing sources in DTrace, and there are comments that suggest that ctfconvert should've been able to deal with zero-sized arrays. Look at die_sou_resolve() in cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c One observation, if you add a real member to the union that otherwise contains only zero-sized arrays, then ctfconvert is happy. This suggests that it can deal with zero-sized arrays as such, but got surprised that we've managed to define zero-sized data type that is not an array. It sounds like a corner case to me. After all, who could have thought that someone may need zero-sized struct or union one day? One possible way to deal with that would be to move zero-sized arrays out of the union. ctfconvert is happy and the size of your data structure should not change -- you still should be able to use zero-sized array members to append real data, only now they are members of the structure itself, not of the member union. This seems to work: struct abc { int length; int bit; char key[0]; char *key_ptr[0]; }; --Artem On Sat, Jun 5, 2010 at 10:23 PM, Shrikanth Kamath w= rote: > I have a small snippet trying to study ctfconvert... > > struct abc { > =A0 =A0int length; > =A0 =A0int bit; > =A0 =A0union { > =A0 =A0 =A0 =A0char key[0]; > =A0 =A0 =A0 =A0char *key_ptr[0]; > =A0 =A0} keys; > }; > > int main() > { > =A0 =A0int a =3D1; > =A0 =A0struct abc member =3D { 16, 5 }; > =A0 =A0printf("Sizeof abc structure [%d] \n", sizeof(struct abc)); > } > > % gcc -g ctfconvert_prob.c > % ctfconvert a.out > > I get the following error on running "ctfconvert"... > ctfconvert_prob.c: failed to resolve the following types: > struct 362 <16a>: failed to size member "keys" of type __anon__ (297 <129= >) > ERROR: ctfconvert_prob.c: failed to resolve types > > How can ctfconvert be equipped to resolve this data type? > > -- > Shrikanth R K > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 10:49:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E326106566B; Sun, 6 Jun 2010 10:49:50 +0000 (UTC) (envelope-from wooh@wooh.hu) Received: from mail.netidea.hu (netwarehouse.netidea.hu [195.228.254.126]) by mx1.freebsd.org (Postfix) with ESMTP id 25E5A8FC0C; Sun, 6 Jun 2010 10:49:49 +0000 (UTC) Received: from radon (localhost [127.0.0.1]) by mail.netidea.hu (Postfix) with ESMTP id 54492128EB8; Sun, 6 Jun 2010 12:49:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at netidea.hu Received: from mail.netidea.hu ([127.0.0.1]) by radon (mail.netidea.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7wdMnPlgl7AA; Sun, 6 Jun 2010 12:49:13 +0200 (CEST) Received: from jamms-MacBook.local (catv-80-99-63-6.catv.broadband.hu [80.99.63.6]) by mail.netidea.hu (Postfix) with ESMTP id 3C75E128C9E; Sun, 6 Jun 2010 12:49:13 +0200 (CEST) Message-ID: <4C0B7D4B.4080402@wooh.hu> Date: Sun, 06 Jun 2010 12:49:47 +0200 From: Adam PAPAI User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-questions@freebsd.org, freebsd-hackers@freebsd.org References: <4C09932B.6040808@wooh.hu> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 10:49:50 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 6/5/10 2:43 AM, Igor Mozolevsky wrote: > On 5 June 2010 00:58, Adam PAPAI wrote: > >> How can I tune my disk to make it faster? Is it possible? What is the >> reason of the really slow I/O with more than 4 threads? What do you >> recommend me to do? Why is it damn slow with 8K blocksize? > > Does linux still have async disk writes by default? Anyway, I looked after the default ext3 values: Debian mounts the ext3 with "defaults" option. This means: rw, suid, dev, exec, auto, nouser, and async. Well it means I have to test it with UFS (async) and Debian (sync). These test will take some time but I hope it worth the effort. Hm... - -- Adam PAPAI -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMC31LAAoJEGq0EWvh5uiIu3MH/i7KWfcYj2zXSsqbUK2W4dKi B0+pD861FBtxmS+O4c4jzR5vJYeVVyVfZ4DLpHs0tqr6u2QZWgTD5c9GXxRNn9Hg pVIL8/iL9BGtjNZdbjKU2RlE+QOb4LUuxqTWtz3poH4e6CQlAMOzvBcmbK41eWVn nr2/jlS8n7TFk74ewAH9NXABrhIaOtCjBf5YWWA9AnKhqjdlAM7gxC6QcbsGTLlR 5zvq6UfGuAMECOV98FDlm3k20LydLT0/Mdw9jth9+50v1NMnAddYjfZ/7Ci2KzZo uUN1VRcOhxmw6oliMPu/+Z324d6Xrp1vXpDQN8tSzME1d3O3CswPDfs3ocpjmkU= =VEsH -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 14:35:20 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EB291065678; Sun, 6 Jun 2010 14:35:20 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 4BB898FC19; Sun, 6 Jun 2010 14:35:20 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 3C8CA1FFC34; Sun, 6 Jun 2010 14:35:19 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id B1DB0844C6; Sun, 6 Jun 2010 16:33:09 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Doug Barton References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> Date: Sun, 06 Jun 2010 16:33:09 +0200 In-Reply-To: <4C0AB448.2040104@FreeBSD.org> (Doug Barton's message of "Sat, 05 Jun 2010 13:32:08 -0700") Message-ID: <86r5kk6xju.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 14:35:20 -0000 Doug Barton writes: > Bakul Shah writes: > > $ ps|(head -1; grep sh) > > PID TT STAT TIME COMMAND > I don't understand why you think this would work. There is no input to > the grep command. The only reason it exits at all is that you are > executing in a subshell. The output from ps is piped to the subshell, so all processes within that subshell get their stdin from that output. Bakul's error is in assuming that head will *consume* only the first line of input, whereas the only guarantee actually given is that it will *print* only the first line of input. This particular one-liner can be implemented reliably as follows: % ps | (read header; echo $header; grep sh) PID TT STAT TIME COMMAND 3415 0 Ss 0:00.31 -zsh (zsh) 3476 0 S+ 0:00.00 -zsh (zsh) because read is a shell built-in, and the shell guarantees that it will not consume more input than necessary. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 17:20:22 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07FD01065674 for ; Sun, 6 Jun 2010 17:20:22 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id B579E8FC0A for ; Sun, 6 Jun 2010 17:20:21 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id o56HKKcI069661 for ; Sun, 6 Jun 2010 10:20:21 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id o56HKKBu069660; Sun, 6 Jun 2010 10:20:20 -0700 (PDT) Date: Sun, 6 Jun 2010 10:20:20 -0700 (PDT) From: Matthew Dillon Message-Id: <201006061720.o56HKKBu069660@apollo.backplane.com> To: freebsd-hackers@freebsd.org References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 17:20:22 -0000 :All of these tests have been apples vs. oranges for years. : :The following seems to be true, though: : :a) FreeBSD sequential write performance in UFS has always been less than :optimal. If there's no read activity sequential write performance should be maximal with UFS. The keyphrase here is "no read activity". UFS's main problem, easily demonstrated by running something like blogbench --iterations=100, is that read I/O is given such a huge precedence over write I/O it can cause the write I/O to come to a complete grinding halt once the system caches are blown out and the reads start having to go to disk. Another big issue with filesystem benchmarks is the data footprint size of the benchmark. Many benchmarks do not have a sufficiently large data footprint and wind up simply testing how much memory the kernel is willing to give over to cache the benchmark's tests, instead of testing disk performance. Bonnie++ is a really good example of the latter problem. That said, all the BSDs have stall issues with parallel read & write activity on the same file. It essentially comes down to the vnode lock held during writes which can cause reads on the same file to stall even when those reads could be satisfied from the VM/BUF cache. Linux might appear to work better in such benchmarks because Linux essentially allows infininte write buffering, up to the point where system memory is exhausted, and the BSDs do not. Infinite write buffering might make a benchmark look good but it creates horrible stalls and inconsistencies on production systems. I noticed that FreeBSD's ZFS implementation issues VOP_WRITE's with a shared lock instead of an exclusive lock, thus avoiding this particular problem. It would be possible to do this with UFS too with some work to prevent file size changes from colliding during concurrent writes, or even using a separate serializer for modifying/write operations so read operations can continue to run concurrently. blogbench is a good way to test read/write interference during the system-cache phase of blogbench's operation (that would be the first 500-800 or so blogs on a 4G system). If working properly both read and write operations should be maximal during this phase. That is, the disk should be 100% saturated with writes while all reads are still fully satisfiable from the buffer cache / VM system, and at the same time the read rate should not suffer (not be seen to stall). It would be interesting to see a blogbench comparison between UFS and ZFS on the same hw/disk. -Matt From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 20:45:56 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 844FA1065673 for ; Sun, 6 Jun 2010 20:45:56 +0000 (UTC) (envelope-from wooh@wooh.hu) Received: from mail.netidea.hu (netwarehouse.netidea.hu [195.228.254.126]) by mx1.freebsd.org (Postfix) with ESMTP id 39D448FC1C for ; Sun, 6 Jun 2010 20:45:55 +0000 (UTC) Received: from radon (localhost [127.0.0.1]) by mail.netidea.hu (Postfix) with ESMTP id 1100112922B for ; Sun, 6 Jun 2010 22:45:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at netidea.hu Received: from mail.netidea.hu ([127.0.0.1]) by radon (mail.netidea.hu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GQ450ByFY0sT for ; Sun, 6 Jun 2010 22:45:18 +0200 (CEST) Received: from jamms-MacBook.local (catv-89-132-136-110.catv.broadband.hu [89.132.136.110]) by mail.netidea.hu (Postfix) with ESMTP id 2DC341291D4 for ; Sun, 6 Jun 2010 22:45:18 +0200 (CEST) Message-ID: <4C0C0901.8060605@wooh.hu> Date: Sun, 06 Jun 2010 22:45:53 +0200 From: Adam PAPAI User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> <201006061720.o56HKKBu069660@apollo.backplane.com> In-Reply-To: <201006061720.o56HKKBu069660@apollo.backplane.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 20:45:56 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 6/6/10 7:20 PM, Matthew Dillon wrote: > It would be interesting to see a blogbench comparison between UFS > and ZFS on the same hw/disk. I'll do it, just tell me how do you want to run the tests. The system params are: 8GB Memory 2x72GB SCSI HDD 2x3.4Ghz Xeon Overall: Dell Poweredge 1850. With no raid installed. I'm waiting the benchmark options to run. - -- Adam PAPAI -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMDAkBAAoJEGq0EWvh5uiIPs8H+wUm/AMo7KBOViRG+OMZKl65 RgeJRBco/FvAjZc7Qu8EwfL3QqkZzZcDZmYODwvfhu4Hmw5QFrbu/QjUG3OFFBjH 6Kgyod2VsIRNo74LDTcdHIDwqCKfWYpH1sSkzr73ewgVAMtwen/6ob1hqW9i7jp8 4Fg5emWcJxJ6vh5t/5cgNhrpFdlZ1g4Zcd4yOPdgC04RoRC5Pla4QK6if6KdN4Qe V9MmiJBdekdCwyaoL7sgXh6QLeXqEHd6D2BDYxS40BCH1OxTVnSb+AedqAeIO+OE 0470hgJnHb4500lu4UfdZtvj4DOUkDn/DiA7VU1we6iZCSitFQNxGi6fK2WyXaw= =sERi -----END PGP SIGNATURE----- From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 21:18:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B6A01065670 for ; Sun, 6 Jun 2010 21:18:04 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id BBA578FC1D for ; Sun, 6 Jun 2010 21:18:03 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id o56LI2of072583 for ; Sun, 6 Jun 2010 14:18:03 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id o56LI23k072582; Sun, 6 Jun 2010 14:18:02 -0700 (PDT) Date: Sun, 6 Jun 2010 14:18:02 -0700 (PDT) From: Matthew Dillon Message-Id: <201006062118.o56LI23k072582@apollo.backplane.com> To: freebsd-hackers@freebsd.org References: <4C09932B.6040808@wooh.hu> <201006050236.17697.bruce@cran.org.uk> <4C09FC43.8070804@wooh.hu> <4C0A7F2F.3030105@elischer.org> <4C0A816A.9040403@feral.com> <201006061720.o56HKKBu069660@apollo.backplane.com> <4C0C0901.8060605@wooh.hu> Subject: Re: sysbench / fileio - Linux vs. FreeBSD X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 21:18:04 -0000 :> It would be interesting to see a blogbench comparison between UFS :> and ZFS on the same hw/disk. : : :I'll do it, just tell me how do you want to run the tests. : :The system params are: : :8GB Memory :2x72GB SCSI HDD :2x3.4Ghz Xeon :Overall: Dell Poweredge 1850. With no raid installed. : :I'm waiting the benchmark options to run. : :- -- :Adam PAPAI With 8G of ram blogbench should blow out the system caches at around blog 1000-1600, though it also depends on the maximum number of vnodes supported by the system. One of the two (VM pages or vnode limit) will be hit. All you need to do is run blogbench with enough iterations to ensure that the run eventually blows out the system caches. 200 or 300 should do the job. It's easy to tell when the system cache gets blown out from looking at the output. Run something like the following script for a few hours. You want to get at least four full runs under your belt for each filesystem to factor out edge cases. For the filesystem setup it would be cool to test both the single-drive case and a simple non-redundant interleaved or mirrored setup (double the read bandwidth). With UFS use default parameters with softupdates turned on (I'd say also without SUJ). With ZFS I don't know how best to tune it, try to find a ZFS setup that performs decently. However, be sure to turn off compression and dedup (if those fs options are available), because blogbench basically just writes all-zeros which is highly compressable/collapsable and would skew the results badly. -Matt #!/bin/csh # # /build is the filesystem under test. set i = 0 while(1) set name = `printf "bench%05d" $i` echo $name if ( ! -d /build/blogs/$name ) then mkdir -p /build/blogs/$name blogbench --iterations=200 -d /build/blogs/$name sleep 120 endif @ i = $i + 1 end From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 21:58:38 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DDB41065675 for ; Sun, 6 Jun 2010 21:58:38 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 3585B8FC0A for ; Sun, 6 Jun 2010 21:58:38 +0000 (UTC) Received: (qmail 4518 invoked by uid 399); 6 Jun 2010 21:58:37 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 6 Jun 2010 21:58:37 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C0C1A0B.4090409@FreeBSD.org> Date: Sun, 06 Jun 2010 14:58:35 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100330 Thunderbird/3.0.4 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> In-Reply-To: <86r5kk6xju.fsf@ds4.des.no> X-Enigmail-Version: 1.0.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 21:58:38 -0000 On 06/06/10 07:33, Dag-Erling Smørgrav wrote: > Doug Barton writes: >> Bakul Shah writes: >>> $ ps|(head -1; grep sh) >>> PID TT STAT TIME COMMAND >> I don't understand why you think this would work. There is no input to >> the grep command. The only reason it exits at all is that you are >> executing in a subshell. > > The output from ps is piped to the subshell, so all processes within > that subshell get their stdin from that output. > > Bakul's error is in assuming that head will *consume* only the first > line of input, whereas the only guarantee actually given is that it will > *print* only the first line of input. If you have a 2 line file named foo that looks like this: one two Then do: cat foo | (cat ; echo 'blah' ; cat) you get: one two blah which seems to indicate to me that unless the first command is a shell builtin that the first command is going to receive all of the stdin, and the second command none of it. In any case my point above is still accurate, there was no stdin for Bakul's grep command because head swallowed it. However, even if there are non-builtins that act differently the more basic point is still valid, this is not a technique that should be relied on without testing to make sure that it's going to do what you think it's going to do. :) Doug -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 22:15:39 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7557106564A; Sun, 6 Jun 2010 22:15:39 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 91F4C8FC18; Sun, 6 Jun 2010 22:15:39 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 777761FFC33; Sun, 6 Jun 2010 22:15:38 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id F10FE844A7; Mon, 7 Jun 2010 00:13:28 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Doug Barton References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> Date: Mon, 07 Jun 2010 00:13:28 +0200 In-Reply-To: <4C0C1A0B.4090409@FreeBSD.org> (Doug Barton's message of "Sun, 06 Jun 2010 14:58:35 -0700") Message-ID: <86d3w3yflj.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 22:15:39 -0000 Doug Barton writes: > If you have a 2 line file named foo that looks like this: > one > two > > Then do: cat foo | (cat ; echo 'blah' ; cat) > > you get: > one > two > blah > > > which seems to indicate to me that unless the first command is a shell > builtin that the first command is going to receive all of the stdin, > and the second command none of it. The second command will receive whatever is left after the first is done. Otherwise, read(1) loops wouldn't work. You chose a poor example, since cat(1) consumes *everything*. Try using dd(1) instead, with varying block sizes and counts: /usr/src/sys% ls | (dd bs=3D32 count=3D1 ; echo "** hello **" ; dd bs=3D32 = count=3D1) Makefile amd64/ arm/ boot/ bsm/ 1+0 records in 1+0 records out 32 bytes transferred in 0.000156 secs (204913 bytes/sec) ** hello ** cam/ cddl/ compat/ conf/ contrib1+0 records in 1+0 records out 32 bytes transferred in 0.000134 secs (238822 bytes/sec) The reason why head(1) doesn't work as expected is that it uses buffered I/O with a fairly large buffer, so it consumes more than it needs. The only way to make it behave as the OP expected is to use unbuffered I/O and never read more bytes than the number of lines left, since the worst case is input consisting entirely of empty lines. We could add an option to do just that, but the same effect can be achieved more portably with read(1) loops: % nhead() { for n in $(jot $1) ; do read line ; print $line ; done }=20=20= =20 % jot 15 | (nhead 5 >/dev/null; nhead 3; echo hi; nhead 3) 6 7 8 hi 9 10 11 DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 6 22:21:24 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AF111065679 for ; Sun, 6 Jun 2010 22:21:24 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id 19F3C8FC19 for ; Sun, 6 Jun 2010 22:21:23 +0000 (UTC) Received: (qmail 2760 invoked by uid 399); 6 Jun 2010 22:21:23 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 6 Jun 2010 22:21:23 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4C0C1F62.8050206@FreeBSD.org> Date: Sun, 06 Jun 2010 15:21:22 -0700 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100330 Thunderbird/3.0.4 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> In-Reply-To: <86d3w3yflj.fsf@ds4.des.no> X-Enigmail-Version: 1.0.1 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jun 2010 22:21:24 -0000 On 06/06/10 15:13, Dag-Erling Smørgrav wrote: > The second command will receive whatever is left after the first is > done. Otherwise, read(1) loops wouldn't work. You chose a poor > example, since cat(1) consumes*everything*. Fair enough. My point remains though, using this technique is liable to lead to unpredictable results. :) Doug -- ... and that's just a little bit of history repeating. -- Propellerheads Improve the effectiveness of your Internet presence with a domain name makeover! http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 7 00:06:09 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C986106566C; Mon, 7 Jun 2010 00:06:09 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 2CBE18FC0C; Mon, 7 Jun 2010 00:06:08 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 97C6F5B5A; Sun, 6 Jun 2010 17:06:07 -0700 (PDT) To: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= In-reply-to: Your message of "Mon, 07 Jun 2010 00:13:28 +0200." <86d3w3yflj.fsf@ds4.des.no> References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> Comments: In-reply-to =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= message dated "Mon, 07 Jun 2010 00:13:28 +0200." Date: Sun, 06 Jun 2010 17:06:07 -0700 From: Bakul Shah Message-Id: <20100607000607.97C6F5B5A@mail.bitblocks.com> Cc: freebsd-hackers@freebsd.org, Doug Barton , Rob Warnock Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2010 00:06:09 -0000 On Mon, 07 Jun 2010 00:13:28 +0200 =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= wrote: > > The reason why head(1) doesn't work as expected is that it uses buffered > I/O with a fairly large buffer, so it consumes more than it needs. The > only way to make it behave as the OP expected is to use unbuffered I/O > and never read more bytes than the number of lines left, since the worst > case is input consisting entirely of empty lines. We could add an > option to do just that, but the same effect can be achieved more > portably with read(1) loops: Except read doesn't do it quite right: $ ps | (read a; echo $a ; grep zsh) PID TT STAT TIME COMMAND 1196 p0 Is 0:02.23 -zsh (zsh) 1209 p1 Is 0:00.35 -zsh (zsh) Alignment of column titles is messed up. Using egrep we can get the right alignment but egrep also shows up. $ ps | egrep 'TIME|zsh' PID TT STAT TIME COMMAND 1196 p0 Is 0:02.23 -zsh (zsh) 1209 p1 Is 0:00.35 -zsh (zsh) 71945 p2 DL+ 0:00.01 egrep TIME|zsh A small point but it is not trivial to get it exactly right. head -n directly expresses what one wants. But there is a deeper point. Several people pointed out alternatives for the examples given but in general you can't use a single command to replace a sequence of commands where each operates on part of the shared input in a different way. The reason we can't do this is buffering for efficiency. Usually there is no further use for the buffered but unconsumed input & it can be safely thrown away. So this is almost always the right thing to do but not when there *is* further use for the unconsumed input. Some programs already do the right thing (dd, for instance, as you pointed out). Some other commands do give you this option in a limited way. "man grep" & you will find: -m NUM, --max-count=NUM Stop reading a file after NUM matching lines. If the input is standard input from a regular file, and NUM matching lines are >>>> output, grep ensures that the standard input is positioned to >>>> just after the last matching line before exiting, regardless of the presence of trailing context lines. This enables a calling process to resume a search. So for instance $ < /usr/share/dict/words (grep -m 1 ''; grep -m 1 '') A a But pipe the file in and see what you get: $ cat /usr/share/dict/words | (grep -m 1 ''; grep -m 1 '') A nterectasia Grep does the right thing for files but not pipes! Now I do understand *why* this happens but still, it is annoying. So I believe there is value in providing an option to read *as much as needed* but not more. It will be slower but will handle the cases we are discussing. This will enhance *composability* -- supposedly part of the unix philosophy. The slow-but-read-just-as-much-as-needed option to be used when you need certain kind of composability and there is no other way. And yes, now do I think this is useful not just for head but also any other program that quits before reading to the end! [cc'ed Rob in case he wishes to chime in] From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 7 03:37:49 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 890601065675; Mon, 7 Jun 2010 03:37:49 +0000 (UTC) (envelope-from rpw3@rpw3.org) Received: from a64.rpw3.org (fast.rpw3.org [66.93.131.53]) by mx1.freebsd.org (Postfix) with ESMTP id 706958FC1D; Mon, 7 Jun 2010 03:37:49 +0000 (UTC) Received: by a64.rpw3.org (Postfix, from userid 101) id BC49A39AF1; Sun, 6 Jun 2010 20:20:47 -0700 (PDT) Date: Sun, 6 Jun 2010 20:20:47 -0700 From: Rob Warnock To: Bakul Shah Message-ID: <20100607032047.GA63703@a64.rpw3.org> References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> <20100607000607.97C6F5B5A@mail.bitblocks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100607000607.97C6F5B5A@mail.bitblocks.com> User-Agent: Mutt/1.4.2.3i X-Mailman-Approved-At: Mon, 07 Jun 2010 03:57:16 +0000 Cc: Dag-Erling Sm??rgrav , Doug Barton , freebsd-hackers@freebsd.org Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2010 03:37:49 -0000 Bakul Shah wrote: +---------- | [cc'ed Rob in case he wishes to chime in] +--------- No, I think you covered most of it quite well [including the bit about "grep -m" not working properly in the case of pipes, sockets, special files, etc.], thanks. Yes, I know that "grep(1)" only promises that "-m" works "If the input is standard input from a regular file", but IMHO we would all benefit from having an option for "grep" (and "head") that would force using "read(fd,&c,1)" when the user really, really wanted it (them) to. -Rob ----- Rob Warnock 627 26th Avenue San Mateo, CA 94403 (650)572-2607 From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 7 11:07:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D824E106567B; Mon, 7 Jun 2010 11:07:59 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 970B88FC13; Mon, 7 Jun 2010 11:07:59 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 89BD81FFC35; Mon, 7 Jun 2010 11:07:58 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id EEAE1844CC; Mon, 7 Jun 2010 13:05:48 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Bakul Shah References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> <20100607000607.97C6F5B5A@mail.bitblocks.com> Date: Mon, 07 Jun 2010 13:05:48 +0200 In-Reply-To: <20100607000607.97C6F5B5A@mail.bitblocks.com> (Bakul Shah's message of "Sun, 06 Jun 2010 17:06:07 -0700") Message-ID: <86631vksqb.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org, Doug Barton , Rob Warnock Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2010 11:07:59 -0000 Bakul Shah writes: > Except read doesn't do it quite right: > > $ ps | (read a; echo $a ; grep zsh) > PID TT STAT TIME COMMAND yeah, I forgot that it drops leading whitespace... DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 7 13:58:10 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B360106567B for ; Mon, 7 Jun 2010 13:58:10 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id CEA998FC26 for ; Mon, 7 Jun 2010 13:58:09 +0000 (UTC) Received: by ewy1 with SMTP id 1so510181ewy.33 for ; Mon, 07 Jun 2010 06:58:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type:content-transfer-encoding; bh=0XayazpbcZoQ69X5G3Xl2jDxHLN8OAx+EyXobuBxNiw=; b=gl8lZmEQn0P56P4xSbpOfn3U0Lvgcn/ocZkZTGMesnKaskDS0S2XiQ9mK2JmGZbvoV 7rwykD0CaOYP7XfTXPDlaaQPYO19LuCBOYw3M+OUDj5mPdUN1G2RK+B2A0d3kVjBraZg j39F7yl7gh8eEKkyZE+dgFCi3TvUS9rvyARjM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding; b=sDQLF3yztv6MHxKwvA0dp4WhcgVvralXkdMunETrcvC29zp5FFjTpQK/PmifoOGkb1 ZpDhyLW4Y7Sg0t3tZYjVfhachjtODVCVdbKLEiQyq0JfgHJP/6Akoofc66y4Ao6veEEb bfaqjwJlQ+LNuPCW/7/BgWXVTawbJP3hqbQ6E= Received: by 10.213.113.193 with SMTP id b1mr10807452ebq.24.1275919088649; Mon, 07 Jun 2010 06:58:08 -0700 (PDT) Received: from gumby.homeunix.com (bb-87-81-140-128.ukonline.co.uk [87.81.140.128]) by mx.google.com with ESMTPS id 16sm2670710ewy.11.2010.06.07.06.58.05 (version=SSLv3 cipher=RC4-MD5); Mon, 07 Jun 2010 06:58:06 -0700 (PDT) Date: Mon, 7 Jun 2010 14:58:03 +0100 From: RW To: freebsd-hackers@freebsd.org Message-ID: <20100607145803.45443930@gumby.homeunix.com> In-Reply-To: <4C0AE8F1.8040606@dataix.net> References: <4C0AE8F1.8040606@dataix.net> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: Aurora Scheduler X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2010 13:58:10 -0000 On Sat, 05 Jun 2010 20:16:49 -0400 jhell wrote: > > > I'm afraid that you are confused about what the scheduler in the > FreeBSD kernel does and what the Aurora Scheduler is. They are not > relational to one another in any such way. > ... > There is not a similiar scheduling program in FreeBSD but there is > Evolution, Kontact and other such things to manage your time. I suspect he's talking about using algorithms from project management software in an embedded kernel to help it schedule around real-time deadlines and so on. From owner-freebsd-hackers@FreeBSD.ORG Mon Jun 7 23:14:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BED181065678; Mon, 7 Jun 2010 23:14:08 +0000 (UTC) (envelope-from prvs=1767699700=brian@FreeBSD.org) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 74BB08FC0A; Mon, 7 Jun 2010 23:14:08 +0000 (UTC) Received: from pd5ml2no-ssvc.prod.shaw.ca ([10.0.153.164]) by pd5mo1no-svcs.prod.shaw.ca with ESMTP; 07 Jun 2010 16:59:06 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=1 a=2caWB_DcjroA:10 a=VphdPIyG4kEA:10 a=ORa4HqFjfvEA:10 a=IkcTkHD0fZMA:10 a=MJPcHhXccCG8eBs0us8XwA==:17 a=mpmaN9d1AAAA:8 a=MMwg4So0AAAA:8 a=6I5d2MoRAAAA:8 a=Hue02uaqDtnA88f1WkUA:9 a=59bl0h-LIdcIDY4r8Pt48I7mqrcA:4 a=QEXdDO2ut3YA:10 a=9klllKCzMBAA:10 a=WJ3hkfHDukgA:10 a=SV7veod9ZcQA:10 Received: from unknown (HELO store.lan.Awfulhak.org) ([70.79.162.198]) by pd5ml2no-dmz.prod.shaw.ca with ESMTP; 07 Jun 2010 16:59:05 -0600 Received: from store.lan.Awfulhak.org (localhost.localdomain [127.0.0.1]) by localhost (Email Security Appliance) with SMTP id 450BAC433AF_C0D79B9B; Mon, 7 Jun 2010 22:59:05 +0000 (GMT) Received: from gw.Awfulhak.org (gw.lan.Awfulhak.org [172.16.0.1]) by store.lan.Awfulhak.org (Sophos Email Appliance) with ESMTP id D5210C460F6_C0D79B0F; Mon, 7 Jun 2010 22:58:56 +0000 (GMT) Received: from localhost (brian@gw.lan.Awfulhak.org [172.16.0.1]) by gw.Awfulhak.org (8.14.4/8.14.4) with ESMTP id o57MwtJw030260; Mon, 7 Jun 2010 15:58:56 -0700 (PDT) (envelope-from brian@FreeBSD.org) Date: Mon, 7 Jun 2010 15:59:00 -0700 From: Brian Somers To: Dag-Erling =?UTF-8?B?U23DuHJncmF2?= Message-ID: <20100607155900.76080dca@Awfulhak.org> In-Reply-To: <86631vksqb.fsf@ds4.des.no> References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> <20100607000607.97C6F5B5A@mail.bitblocks.com> <86631vksqb.fsf@ds4.des.no> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.20.1; i386-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@FreeBSD.org, Barton , Rob Warnock , Doug Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2010 23:14:08 -0000 On Mon, 07 Jun 2010 13:05:48 +0200, Dag-Erling Sm=C3=B8rgrav w= rote: > Bakul Shah writes: > > Except read doesn't do it quite right: > > > > $ ps | (read a; echo $a ; grep zsh) > > PID TT STAT TIME COMMAND >=20 > yeah, I forgot that it drops leading whitespace... Well, leading $IFS $ ps | (IFS=3D read a; echo "$a"; grep zsh) works a lot better. --=20 Brian Somers Don't _EVER_ lose your sense of humour ! From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 8 09:01:41 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B02C1065673 for ; Tue, 8 Jun 2010 09:01:41 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id 390F78FC1C for ; Tue, 8 Jun 2010 09:01:40 +0000 (UTC) Received: from mobileKamikaze.norad (unknown [46.114.141.201]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id 82F078A1F22 for ; Tue, 8 Jun 2010 11:01:39 +0200 (CEST) Message-ID: <4C0E06F1.6030607@bsdforen.de> Date: Tue, 08 Jun 2010 11:01:37 +0200 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.9) Gecko/20100331 Thunderbird/3.0.4 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <20100605201242.C79345B52@mail.bitblocks.com> <4C0AB448.2040104@FreeBSD.org> <86r5kk6xju.fsf@ds4.des.no> <4C0C1A0B.4090409@FreeBSD.org> <86d3w3yflj.fsf@ds4.des.no> <20100607000607.97C6F5B5A@mail.bitblocks.com> <86631vksqb.fsf@ds4.des.no> <20100607155900.76080dca@Awfulhak.org> In-Reply-To: <20100607155900.76080dca@Awfulhak.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: Re: head behaviour X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 09:01:41 -0000 On 08/06/2010 00:59, Brian Somers wrote: > On Mon, 07 Jun 2010 13:05:48 +0200, Dag-Erling Smørgrav wrote: >> Bakul Shah writes: >>> Except read doesn't do it quite right: >>> >>> $ ps | (read a; echo $a ; grep zsh) >>> PID TT STAT TIME COMMAND >> >> yeah, I forgot that it drops leading whitespace... > > Well, leading $IFS > > $ ps | (IFS= read a; echo "$a"; grep zsh) > > works a lot better. As does using sed, which is the right tool for this kind of job anyway. -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 8 13:13:50 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E012A1065677 for ; Tue, 8 Jun 2010 13:13:50 +0000 (UTC) (envelope-from tss@iki.fi) Received: from dovecot.org (dovecot.org [82.118.211.50]) by mx1.freebsd.org (Postfix) with ESMTP id 9FBF28FC16 for ; Tue, 8 Jun 2010 13:13:50 +0000 (UTC) Received: from [10.134.134.32] (pickles.tp.telepac.pt [194.65.5.228]) by dovecot.org (Postfix) with ESMTP id 1E8ABFA89B4 for ; Tue, 8 Jun 2010 15:51:49 +0300 (EEST) From: Timo Sirainen To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Date: Tue, 08 Jun 2010 13:51:47 +0100 Message-ID: <1276001507.6691.388.camel@kurkku.sapo.corppt.com> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit Subject: close() failing with ECONNRESET X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jun 2010 13:13:51 -0000 I see that since FreeBSD 6.3 close() can fail with: > [ECONNRESET] The underlying object was a stream socket that was > shut down by the peer before all pending data was > delivered. Could someone explain what this is useful for? I'm not aware of any other OS that does this. Is this really something that many programs care about? I'd think there are only very few, and those exceptions could use some other syscall before close() to find out about it. Instead now you're forcing everyone else to change their code from: if (close(fd) < 0) log(..); to if (close(fd) < 0 && errno != ECONNRESET) log(..); or to write some wrapper to close(). From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 07:42:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 444F41065673 for ; Wed, 9 Jun 2010 07:42:00 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 250858FC19 for ; Wed, 9 Jun 2010 07:42:00 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id o597fwK6073055 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 9 Jun 2010 00:41:59 -0700 (PDT) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id o597fwVw073054; Wed, 9 Jun 2010 00:41:58 -0700 (PDT) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA10143; Wed, 9 Jun 10 00:38:10 PDT Date: Wed, 09 Jun 2010 00:35:11 -0700 From: perryh@pluto.rain.com To: tss@iki.fi Message-Id: <4c0f442f.N55MZNOTsFpdxLGx%perryh@pluto.rain.com> References: <1276001507.6691.388.camel@kurkku.sapo.corppt.com> In-Reply-To: <1276001507.6691.388.camel@kurkku.sapo.corppt.com> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: close() failing with ECONNRESET X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 07:42:00 -0000 Timo Sirainen wrote: > I see that since FreeBSD 6.3 close() can fail with: > > > [ECONNRESET] The underlying object was a stream socket that was > > shut down by the peer before all pending data was > > delivered. > > Could someone explain what this is useful for? Consistency with SIGPIPE, perhaps? From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 08:33:28 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91816106567B for ; Wed, 9 Jun 2010 08:33:28 +0000 (UTC) (envelope-from webmaster@kibab.com) Received: from mx0.deglitch.com (backbone.deglitch.com [78.110.53.255]) by mx1.freebsd.org (Postfix) with ESMTP id 4E2548FC18 for ; Wed, 9 Jun 2010 08:33:28 +0000 (UTC) Received: from localhost (unknown [195.94.237.82]) by mx0.deglitch.com (Postfix) with ESMTPA id 705A18FC4E for ; Wed, 9 Jun 2010 12:15:02 +0400 (MSD) Date: Wed, 9 Jun 2010 12:14:53 +0400 From: Ilya Bakulin To: freebsd-hackers@freebsd.org Message-ID: <20100609121453.095d92b4@kibab.com> Organization: Deglitch Networks X-Mailer: Claws Mail 3.7.6 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/GjjPT62J.LwN4AqkF9Kd14d"; protocol="application/pgp-signature" Subject: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 08:33:28 -0000 --Sig_/GjjPT62J.LwN4AqkF9Kd14d Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi hackers! While discussing my project's implementation details with my mentor, Alexander Leidinger, we've found that one of the ideas needs to be discusse= d with community, to find out possible use cases. That is, if it should be possible to spoof non-existing features. For example, if currently running kernel doesn't support FreeBSD 5.0 compat layer, "kern.features.compat_freebsd5" will be absent when querying=20 features list. The question is -- are there any cases when we want "kern.features.compat_freebsd5" be present? If some feature is not in kernel, then presenting its existence to the userland is useless and may be even harmful, if, for example, some application relies on this f= eature. Or there are some scenarios where such cheat is useful? --=20 Regards, Ilya Bakulin http://kibab.com xmpp://kibab612@jabber.ru --Sig_/GjjPT62J.LwN4AqkF9Kd14d Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkwPTYcACgkQo9vlj1oadwinjACgqlgRxr7FxcbwfuSfauxLSANf mgsAoJhc97LJcb7TLQMb0oZY49xOVDn/ =ZWP2 -----END PGP SIGNATURE----- --Sig_/GjjPT62J.LwN4AqkF9Kd14d-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 09:45:57 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0636D106566B for ; Wed, 9 Jun 2010 09:45:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 7650F8FC18 for ; Wed, 9 Jun 2010 09:45:56 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o599k6mU079223 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jun 2010 12:46:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o599jqXe034367; Wed, 9 Jun 2010 12:45:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o599jqbi034366; Wed, 9 Jun 2010 12:45:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Jun 2010 12:45:52 +0300 From: Kostik Belousov To: perryh@pluto.rain.com Message-ID: <20100609094552.GE83316@deviant.kiev.zoral.com.ua> References: <1276001507.6691.388.camel@kurkku.sapo.corppt.com> <4c0f442f.N55MZNOTsFpdxLGx%perryh@pluto.rain.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6B9BisSCoMvigduU" Content-Disposition: inline In-Reply-To: <4c0f442f.N55MZNOTsFpdxLGx%perryh@pluto.rain.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-hackers@freebsd.org, tss@iki.fi Subject: Re: close() failing with ECONNRESET X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 09:45:57 -0000 --6B9BisSCoMvigduU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 09, 2010 at 12:35:11AM -0700, perryh@pluto.rain.com wrote: > Timo Sirainen wrote: >=20 > > I see that since FreeBSD 6.3 close() can fail with: > > > > > [ECONNRESET] The underlying object was a stream socket that was > > > shut down by the peer before all pending data was > > > delivered. > > > > Could someone explain what this is useful for? Note that any return from close(2) that does not set errno to EBADF closes the supplied file descriptor. Mentioned errno value supplies caller with the information that not "all pending data was delivered". >=20 > Consistency with SIGPIPE, perhaps? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" --6B9BisSCoMvigduU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwPYtAACgkQC3+MBN1Mb4hH/gCg4ze0bLXxApA1dTt2JF1g9dT/ pxoAniY32ks/6UJpj6A4y3b789qQCIx2 =4Nz6 -----END PGP SIGNATURE----- --6B9BisSCoMvigduU-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 10:43:16 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 386D41065677 for ; Wed, 9 Jun 2010 10:43:16 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id B48A58FC24 for ; Wed, 9 Jun 2010 10:43:15 +0000 (UTC) Received: by bwz2 with SMTP id 2so2390789bwz.13 for ; Wed, 09 Jun 2010 03:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:subject :organization:references:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=GFUAiYOwIVMAFPHZh1vgtQL6rlAZpjAU/7IXkAl9REE=; b=sDxV6j7UQqmHhCGKbzkcuHtYNenLVERRfgXCRPjdqAVVwpv8rs1o4sgSouR4Kzrkyd 6KaSTEPnmk8gWn75fH3m7KGRR5aXZipiA/OD8VgNKQ/jigVlV9ltrhAqqvw1+Wll+hjV Lj+JslfIpRJ7ihcYVMvU7QZIizcY7FpCyJI+w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:organization:references:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=E2SjRoneglhr+RI5dH7m3lmXgp2MUiaqAmPlPt8dIDh339HJZzqNrmkdt3NZwBbHUX BPMFFlD8vxe20Bw4Vm78z3SEzUOorT2HsiZYauWutPYJoeJM6VstMYgJdzkoS6r0oETa g/t6zogzt8pDXPDhJlTp/P2mfsLBNFp+NGpzY= Received: by 10.204.83.85 with SMTP id e21mr1277919bkl.42.1276080194374; Wed, 09 Jun 2010 03:43:14 -0700 (PDT) Received: from localhost (ua1.etadirect.net [91.198.140.16]) by mx.google.com with ESMTPS id v14sm29020728bkz.8.2010.06.09.03.43.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 09 Jun 2010 03:43:13 -0700 (PDT) From: Mikolaj Golub To: Kostik Belousov Organization: TOA Ukraine References: <1276001507.6691.388.camel@kurkku.sapo.corppt.com> <4c0f442f.N55MZNOTsFpdxLGx%perryh@pluto.rain.com> <20100609094552.GE83316@deviant.kiev.zoral.com.ua> Date: Wed, 09 Jun 2010 13:43:10 +0300 In-Reply-To: <20100609094552.GE83316@deviant.kiev.zoral.com.ua> (Kostik Belousov's message of "Wed, 9 Jun 2010 12:45:52 +0300") Message-ID: <86vd9s32rl.fsf@zhuzha.ua1> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-hackers@freebsd.org, tss@iki.fi, perryh@pluto.rain.com Subject: Re: close() failing with ECONNRESET X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 10:43:16 -0000 On Wed, 9 Jun 2010 12:45:52 +0300 Kostik Belousov wrote: KB> On Wed, Jun 09, 2010 at 12:35:11AM -0700, perryh@pluto.rain.com wrote: >> Timo Sirainen wrote: >> >> > I see that since FreeBSD 6.3 close() can fail with: >> > >> > > [ECONNRESET] The underlying object was a stream socket that was >> > > shut down by the peer before all pending data was >> > > delivered. >> > >> > Could someone explain what this is useful for? KB> Note that any return from close(2) that does not set errno to EBADF KB> closes the supplied file descriptor. Mentioned errno value supplies KB> caller with the information that not "all pending data was delivered". We have kern/146845 about close(2) returning ECONNRESET for tcp connections. Looking at the code (which I am not very familiar with though) and running some tests make me think that currently ECONNRESET may be only returned by close(2) after shutdown()/close() on our side and simultaneous close() on the other side (and in this case this is wrong). -- Mikolaj Golub From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 13:14:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B9951065680 for ; Wed, 9 Jun 2010 13:14:01 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4A82D8FC28 for ; Wed, 9 Jun 2010 13:14:00 +0000 (UTC) Received: by iwn7 with SMTP id 7so2346655iwn.13 for ; Wed, 09 Jun 2010 06:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=QZuzM3APgptd80cwQbgBFrVZuntXK8uHaM4WEYVYjL0=; b=Zwkl82YiWv1KHkDp+idizNWzTzpSOzltXg1dbTiGkMK4+o72o7pLVbNafjfz4HKqxV E6XgrcQvq1jKXSbcY2vr0GMb05giuDZEfPm1pLSeWaqSYsUDeq5CDsrei3QlwASB4pUm eWBJxdBtUuvFFj3sjWfyCNIPyzN6GF0DxRO/Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=QWQ4q8A6Wmuc6BbIfJoOEW9MArKIvOdwwouvFx6wOJXxMlTUSwMTo4xyps11yvrBPC W4vt+au/RwY1EajU84/R7tOsuDD5CuVAzFzNwOcJS2vAtmQXhBycaeazv6Zc0ox+M8E8 9ITsLK32p2ywoIbbvGMtobWJXOn8IJzdSYIsk= Received: by 10.231.184.75 with SMTP id cj11mr3867289ibb.51.1276089240134; Wed, 09 Jun 2010 06:14:00 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-128-180.dsl.klmzmi.sbcglobal.net [99.181.128.180]) by mx.google.com with ESMTPS id b3sm30919006ibf.1.2010.06.09.06.13.57 (version=SSLv3 cipher=RC4-MD5); Wed, 09 Jun 2010 06:13:58 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C0F9394.9030202@dataix.net> Date: Wed, 09 Jun 2010 09:13:56 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: Ilya Bakulin References: <20100609121453.095d92b4@kibab.com> In-Reply-To: <20100609121453.095d92b4@kibab.com> X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 13:14:01 -0000 On 06/09/2010 04:14, Ilya Bakulin wrote: > Hi hackers! > > While discussing my project's implementation details with my mentor, > Alexander Leidinger, we've found that one of the ideas needs to be discussed with community, > to find out possible use cases. > That is, if it should be possible to spoof non-existing features. For > example, if currently running kernel doesn't support FreeBSD 5.0 compat > layer, "kern.features.compat_freebsd5" will be absent when querying > features list. The question is -- are there any cases when we want > "kern.features.compat_freebsd5" be present? If some feature is not in > kernel, then presenting its existence to the userland is useless > and may be even harmful, if, for example, some application relies on this feature. > Or there are some scenarios where such cheat is useful? > I can not think of any viable reason why one would want to "spoof" this when it is not available. To have this be present and set to a value of '0' would make sense for telling end-user program that its not available. Since a generic kernel is already built with the compatN layers that it supports I believe it would be best to look at __FreeBSD_version which should be kern.osreldate to see if that layer of compatibility is available at all and then leave it up to the end-user to enable it if they have built their own private kernel. Another thought that had come to mind was to make that sysctl an array of values for the layers of compatN that exist in the kernel. 1). Address [0] of the array is set to 1 if any compatN is available. 2). Further corresponding addresses 1 - N would correspond to _FREEBSD1 to FreeBSDN. "This is why COMPAT_FREEBSD32 is bad!" 3). If a compat version becomes unsupported set its array value to null. This would allow someone to be able to check against address [0] which should always exist and discontinue further checks to see what versions are available. Although it suggests subverting actually doing the work on the software writers part and checking against __FreeBSD_version to see what versions are available in a 'GENERIC' kernel. Just a thought but I would rather see an array of values than 3 - 4 separate sysctls. Kind regards & happy hacking, -- jhell From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 13:26:28 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3524D1065674 for ; Wed, 9 Jun 2010 13:26:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A35B78FC08 for ; Wed, 9 Jun 2010 13:26:27 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o59DPvvx003539 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Jun 2010 16:25:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o59DPhU2035122; Wed, 9 Jun 2010 16:25:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o59DPhqo035121; Wed, 9 Jun 2010 16:25:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 9 Jun 2010 16:25:43 +0300 From: Kostik Belousov To: jhell Message-ID: <20100609132543.GI83316@deviant.kiev.zoral.com.ua> References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IoysBk3hhUmaUGAr" Content-Disposition: inline In-Reply-To: <4C0F9394.9030202@dataix.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_50, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Ilya Bakulin , freebsd-hackers@freebsd.org Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 13:26:28 -0000 --IoysBk3hhUmaUGAr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 09, 2010 at 09:13:56AM -0400, jhell wrote: > On 06/09/2010 04:14, Ilya Bakulin wrote: > > Hi hackers! > >=20 > > While discussing my project's implementation details with my mentor, > > Alexander Leidinger, we've found that one of the ideas needs to be disc= ussed with community, > > to find out possible use cases. > > That is, if it should be possible to spoof non-existing features. For > > example, if currently running kernel doesn't support FreeBSD 5.0 compat > > layer, "kern.features.compat_freebsd5" will be absent when querying=20 > > features list. The question is -- are there any cases when we want > > "kern.features.compat_freebsd5" be present? If some feature is not in > > kernel, then presenting its existence to the userland is useless > > and may be even harmful, if, for example, some application relies on th= is feature. > > Or there are some scenarios where such cheat is useful? > >=20 >=20 > I can not think of any viable reason why one would want to "spoof" this > when it is not available. Many ports are doing wrong thing there, checking for run-time features at the build-time, turning on/off some functionality depending on its presence on the build host. --IoysBk3hhUmaUGAr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkwPllcACgkQC3+MBN1Mb4gU0gCg4T5LhRub/JYdxmgr5GpWOFrx 24QAnRvonnOInT6zfXqgFccAMyVCQYFb =K4nL -----END PGP SIGNATURE----- --IoysBk3hhUmaUGAr-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 14:54:21 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BBB9106570A for ; Wed, 9 Jun 2010 14:54:21 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 290CA8FC34 for ; Wed, 9 Jun 2010 14:54:20 +0000 (UTC) Received: by iwn7 with SMTP id 7so2464491iwn.13 for ; Wed, 09 Jun 2010 07:54:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:openpgp:content-type:content-transfer-encoding; bh=dCJ87RA0cOhaUCjuvtGBpYpWgavBAQ3VR0fBW7OkSfA=; b=BQ3QeKr6c4y+Bcja3lkjBXzvinxWw8pSArr6VTTrP8NXDbUUYOWT7cdTeB3iPLIuAa tU8P0ZWHWNMLriVJLlmRElDhjsa+fsQzSh/t0J/SoRJbzeS6cESgb6W8Y9SZOscR8gx6 GroV4+nqQ3Fn6A8OvLyeoeF4CWvM5TLZkTQg8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:openpgp:content-type :content-transfer-encoding; b=ZSpmEHbznZnsBnoKliINn545urwoIkPL4KEl0fm3D135Vnez2H9RVTT3LAKR80xqy/ eDa7AHGPwrJ40n27jv9QcsJ4of01BjJar4l+UJ7jM5+tfP5FQGFhZcE1fL+GRwDVFjjI wfc6QJIt8CbaxTquDrVMikrXEdgS2fV5cfcgY= Received: by 10.231.169.9 with SMTP id w9mr1551135iby.58.1276095259964; Wed, 09 Jun 2010 07:54:19 -0700 (PDT) Received: from centel.dataix.local (adsl-99-181-128-180.dsl.klmzmi.sbcglobal.net [99.181.128.180]) by mx.google.com with ESMTPS id t28sm31265972ibg.0.2010.06.09.07.54.17 (version=SSLv3 cipher=RC4-MD5); Wed, 09 Jun 2010 07:54:19 -0700 (PDT) Sender: "J. Hellenthal" Message-ID: <4C0FAB18.7010902@dataix.net> Date: Wed, 09 Jun 2010 10:54:16 -0400 From: jhell User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.9) Gecko/20100515 Thunderbird MIME-Version: 1.0 To: Kostik Belousov References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> In-Reply-To: <20100609132543.GI83316@deviant.kiev.zoral.com.ua> X-Enigmail-Version: 1.0.1 OpenPGP: id=89D8547E Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Ilya Bakulin , freebsd-hackers@freebsd.org Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 14:54:21 -0000 On 06/09/2010 09:25, Kostik Belousov wrote: > On Wed, Jun 09, 2010 at 09:13:56AM -0400, jhell wrote: >> On 06/09/2010 04:14, Ilya Bakulin wrote: >>> Hi hackers! >>> >>> While discussing my project's implementation details with my mentor, >>> Alexander Leidinger, we've found that one of the ideas needs to be discussed with community, >>> to find out possible use cases. >>> That is, if it should be possible to spoof non-existing features. For >>> example, if currently running kernel doesn't support FreeBSD 5.0 compat >>> layer, "kern.features.compat_freebsd5" will be absent when querying >>> features list. The question is -- are there any cases when we want >>> "kern.features.compat_freebsd5" be present? If some feature is not in >>> kernel, then presenting its existence to the userland is useless >>> and may be even harmful, if, for example, some application relies on this feature. >>> Or there are some scenarios where such cheat is useful? >>> >> >> I can not think of any viable reason why one would want to "spoof" this >> when it is not available. > Many ports are doing wrong thing there, checking for run-time features at > the build-time, turning on/off some functionality depending on its > presence on the build host. That would lead me to believe that the elimination of this sysctl would be better suited to solve the outcome of cases like this. And leads me to believe that it still rests on the end-user to tell whether or not they have that compatibility layer compiled in. Like I stated more towards the end of my last message "I believe" checking __FreeBSD_version should suffice and leave the final result up to the end-user as GENERIC will have the COMPAT_FREEBSD{N} layers compiled in that it needs or can support or are recommended. Being that this is a broad scenario and many different compilations of kernels could be used I still do not see a need to test for every one of them if an adequate means already exists. GENERIC in any case should be the kernel that is depended on and testing against __FreeBSD_version for what COMPAT versions are supported. Regards, -- jhell From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 15:50:28 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BD93106564A for ; Wed, 9 Jun 2010 15:50:28 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 43F0C8FC12 for ; Wed, 9 Jun 2010 15:50:28 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FA9A.dip.t-dialin.net [217.84.250.154]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id AD0C284400A; Wed, 9 Jun 2010 17:50:22 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 079CF514C; Wed, 9 Jun 2010 17:50:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1276098618; bh=ocmJ+GwDMLlccL4ymrg+FajJpB7POADU5HliUfSOVSk=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=z9DJ7U+WxpMQTjqobEGO2Ktmvkccg0DIPKV6fqngvRM94tQOmiStPfVbOEB10b2Yl YW/jYqKRM2zWA3oSBSsBaPcy/Q6Ox+G6/0tU8Mg3lyeqdojmzOi+hiq967Wq3pkwfu rHw5QYSE6AS9AP5gqdt88QRiYLTxr1BbKPn0PgzBRVbJcPJk/Lb9WDjtrAwBefSeKK 1SwDjVOgPc+bRgJvB0ZIJYBercLdQzE/Koo/b8iDm2q+l8zg9sq0QplurGWWeaoobr Lzh8Oi3E866zEahyfXNl19jYz0Opa3bMCRTi6L62JO/gGp4dIOct777DhfiTJmjxNK WKdAwPIVPrMxg== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o59FoGb4065681; Wed, 9 Jun 2010 17:50:16 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 09 Jun 2010 17:50:15 +0200 Message-ID: <20100609175015.18145hz1u8arpnwo@webmail.leidinger.net> Date: Wed, 09 Jun 2010 17:50:15 +0200 From: Alexander Leidinger To: Kostik Belousov References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> In-Reply-To: <20100609132543.GI83316@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: AD0C284400A.A62EA X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.1, required 6, autolearn=disabled, ALL_TRUSTED -1.00, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1276703425.68725@OXEOwFsM5zxTPqgcASyjeA X-EBL-Spam-Status: No X-Mailman-Approved-At: Wed, 09 Jun 2010 16:01:04 +0000 Cc: freebsd-hackers@freebsd.org, Ilya Bakulin Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 15:50:28 -0000 Quoting Kostik Belousov (from Wed, 9 Jun 2010 16:25:43 +0300): >> I can not think of any viable reason why one would want to "spoof" this >> when it is not available. > Many ports are doing wrong thing there, checking for run-time features at > the build-time, turning on/off some functionality depending on its > presence on the build host. We heard that "there are some ports", but we do not know a concrete example. Anyone here with a concrete example of such a port (maybe more than one)? The big question here is: what is going on at build time regarding those features? As you describe the problem, we have to make a change to the port anyway. And IMO it does not matter much if we change it to detect it at run-time (then we do not need the spoofing feature), or if we change it to look for the sysctl at build time. And the preferred way would be to detect at run-time then. What we search for is a good real-life example where spoofing a non-existing feature would be helpful. So far we where able to come up with cases where this would hurt, but not help. Bye, Alexander. -- If the master dies and the disciple grieves, the lives of both have been wasted. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 17:37:33 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DE89106566B for ; Wed, 9 Jun 2010 17:37:33 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id D1C958FC0C for ; Wed, 9 Jun 2010 17:37:32 +0000 (UTC) Received: by wyb33 with SMTP id 33so958418wyb.13 for ; Wed, 09 Jun 2010 10:37:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=iOnKka2rObKP5WTyURMnNAEgJJ+OcnxbUO8fS5KnevM=; b=GWNCkwtX1wha1oq/zpqOsvT1K65eRGCxPVOGcvgjC1lxvoQU4rud2mRZqJu5/oa3vI Q0erTyKq2JlXgGwTTjOUYk7szNZkG/Lkz2wCbjSv8hjMOa6L5otMyaihHtuCZesSczoU yuWj2YfftV/k9VM/X/B6MERnWnrCj/k/H5eD0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=tuawCngHD0vEhfvKrqQKxU2Fsdyp+RNP0NaXkVsV03Dcb2tI6IBqPKVPx/byYR7IyK PLN8l3ZmI4N52fN9PhqbPwxm2zqOSmupsGk6TRcCWZXaYuBfVIBDZPdA98pjfduewni2 o8ZriK/XMS/U9Wprm16gfxr7AidRJ9AiQemdc= Received: by 10.227.155.204 with SMTP id t12mr5277276wbw.29.1276103580101; Wed, 09 Jun 2010 10:13:00 -0700 (PDT) Received: from [192.168.0.204] (deviant.freebsdgirl.com [173.8.183.73]) by mx.google.com with ESMTPS id b14sm2687418wbb.5.2010.06.09.10.12.56 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 09 Jun 2010 10:12:58 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1078) Content-Type: text/plain; charset=us-ascii From: Garrett Cooper In-Reply-To: <20100609132543.GI83316@deviant.kiev.zoral.com.ua> Date: Wed, 9 Jun 2010 10:12:54 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> To: Kostik Belousov X-Mailer: Apple Mail (2.1078) X-Mailman-Approved-At: Wed, 09 Jun 2010 17:47:29 +0000 Cc: freebsd-hackers@freebsd.org, Ilya Bakulin Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 17:37:33 -0000 On Jun 9, 2010, at 6:25 AM, Kostik Belousov wrote: > On Wed, Jun 09, 2010 at 09:13:56AM -0400, jhell wrote: >> On 06/09/2010 04:14, Ilya Bakulin wrote: >>> Hi hackers! >>>=20 >>> While discussing my project's implementation details with my mentor, >>> Alexander Leidinger, we've found that one of the ideas needs to be = discussed with community, >>> to find out possible use cases. >>> That is, if it should be possible to spoof non-existing features. = For >>> example, if currently running kernel doesn't support FreeBSD 5.0 = compat >>> layer, "kern.features.compat_freebsd5" will be absent when querying=20= >>> features list. The question is -- are there any cases when we want >>> "kern.features.compat_freebsd5" be present? If some feature is not = in >>> kernel, then presenting its existence to the userland is useless >>> and may be even harmful, if, for example, some application relies on = this feature. >>> Or there are some scenarios where such cheat is useful? >>>=20 >>=20 >> I can not think of any viable reason why one would want to "spoof" = this >> when it is not available. > Many ports are doing wrong thing there, checking for run-time features = at > the build-time, turning on/off some functionality depending on its > presence on the build host. It's present in the ports Makefiles as well as in many autoconf scripts. = It's bad because it causes problems with cross-build and other related = scenarios, where you can't assume that the host system is going to match = the target system. Would be nice if the overrides could be built into ports with sensible = defaults and everything would default to a value instead of being = hardwired to strictly detect said functionality... but that's what = ultimately requires work to fix. -Garrett= From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 18:11:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4582106567B for ; Wed, 9 Jun 2010 18:11:59 +0000 (UTC) (envelope-from rhfb@akira.stdio.com) Received: from akira.stdio.com (akira.stdio.com [204.152.114.29]) by mx1.freebsd.org (Postfix) with SMTP id 861168FC19 for ; Wed, 9 Jun 2010 18:11:46 +0000 (UTC) Received: from akira (localhost [127.0.0.1]) by akira.stdio.com (Postfix) with SMTP id 9769650815 for ; Wed, 9 Jun 2010 13:52:40 -0400 (EDT) From: rhfb@akira.stdio.com To: freebsd-hackers@freebsd.org Message-Id: <20100609175244.9769650815@akira.stdio.com> Date: Wed, 9 Jun 2010 13:52:40 -0400 (EDT) Subject: NFSD lockup running ESXi 4 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 18:11:59 -0000 I have an AMD64 FreeBSD 8.0 running 8-Stable from around 2010/04/25 19:13:08. ZFS disk, Nfsd flags "-t -n 16", private network exclusive for nfs network, not using jumbo frames, HZ=1000, Device_Polling, Zero_Copy_Sockets, and the following sysctl options: net.inet.tcp.recvspace=232140 net.inet.tcp.sendspace=232140 net.inet.tcp.slowstart_flightsize=159 net.inet.tcp.mssdflt=1460 FreeBSD 6 TB zpool, nfs from Three ESXi 4 (newest patch level 193498) working reliably for months. Added a new ESXi, patched to the newest (Post Update 1) patch level 256968. Added a bunch of VM's, booted them all into the 2008 R2 Server install DVD. Then when attempting to do the installs (in parallel/simultaneously) I started getting the NFS server locking up. NFSD would wedge at 100% CPU in "rc_lo" which I presume is rc_lock? Once wedged, /etc/rc.d/nfsd restart can't kill nfsd. So a reboot is required. A Reboot causes all my active VM's with pending disk writes to have disk errors in the VM (10 second default timeout for disk writes in the VM.) This was very reproducable. Has anyone noticed this problem? Is this an ESXi problem with the newest updates? Is this a problem with NFS on FreeBSD 8? From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 18:39:51 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3C91106567A for ; Wed, 9 Jun 2010 18:39:51 +0000 (UTC) (envelope-from ncrogers@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id AA3818FC1B for ; Wed, 9 Jun 2010 18:39:51 +0000 (UTC) Received: by iwn7 with SMTP id 7so2745487iwn.13 for ; Wed, 09 Jun 2010 11:39:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=/SvenOO0hjjCsq1vB7thmVPGazMIVc66UOTkxsDrOSo=; b=riOYJ1oDSMRH78p509q49G3KUqX9LSi4MhYc97rqKCX2hkpiL5D5ESspq1HqpOwayi QLgC0+tlS3voqtDCzZF0LnSyG9va8tL4cELPNck3KedL5C6QHVCSPj9Z2BAzGY1koySe pH4cE3tnVrLuQKwKMnaTfsLwaDZD6+LSP4dHQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=kyRRYDd0zjB7TXQdT1MS+tk4bDRJIj78GkDVFigdxeHj+GjH6OU+ju7yzwvN8Oytlv TJGU/UthWWI7NgTg/vLYaYWih25ii1pyWHCXTGUJoDfEvl5vC3hD6xtOkfx2tyPndaQf DToy8kMg+zUdcoU49RlH15+GpSBBH127hXBJU= MIME-Version: 1.0 Received: by 10.231.188.156 with SMTP id da28mr8271189ibb.196.1276107463361; Wed, 09 Jun 2010 11:17:43 -0700 (PDT) Received: by 10.231.190.131 with HTTP; Wed, 9 Jun 2010 11:17:43 -0700 (PDT) Date: Wed, 9 Jun 2010 14:17:43 -0400 Message-ID: From: Nick Rogers To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 18:39:52 -0000 I have an 8.0-RELEASE system with 4000 "permanent" ARP entries due to having a network interface (em(4)) configured with 4000 aliases. The "arp -na" command takes what I consider to be an extremely long time to finish (up to 30s on an otherwise unloaded system). I am able to replicate this in a test environment by installing 8.0-RELEASE-amd64 on a VMWare VM w/ 1GB of RAM and a 2GHz CPU. The 4000 aliases/entries is arbitrary, but nicely illustrates the performance problem. The performance is much worse on a real/loaded system. I realize the 4k aliases on an interface is unusual but I have been effectively using this configuration in my network to try and keep my end-users's each on his/her own broadcast domain. The box is a router and I allocate addresses to each user and put each on his/her own subnet with a netmask of /30. If you would like more info on this I can provide it, but it has worked effectively in FreeBSD 6.0-7.2. The slow performance of "arp -na" is an issue for me because I have a web/CGI tool that runs various reports, many of them relying on acquiring the current "ARP table", and the performance of arp(8) makes the web interface extremely slow. I believe the problem was introduced between 7.2 and 8.0, when, as far as I understand, parts of the ARP subsystem were improved. In 7.2, the aliases configured on an interface were not considered ARP entries (at least according to arp(8)), but as of 8.0 they are marked as "permanent" ARP entries and displayed by arp(8), which seems to attribute to the performance problem. I ran the following perl script to setup my test system. This script was run after installing 8.0-RELEASE and adding the bash, perl, and p5-NetAddr-IP packages via pkg_add -r. #!/usr/bin/perl use strict; use diagnostics; use NetAddr::IP; my $interface = 'em1'; my $cidr = '10.0.0.1/18'; # configure the interface with 4000 or so aliases foreach my $na (@{NetAddr::IP->new($cidr)->splitref(30)}) { my $ip = $na->addr(); my $mask = $na->mask(); my $bcast = $na->broadcast()->addr(); my $cmd = "ifconfig $interface inet alias $ip netmask $mask broadcast $bcast"; print STDERR "$cmd\n"; system($cmd); } The results are as follows: [root@ ~]# uname -a FreeBSD .localdomain 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 [root@ ~]# ifconfig -a | wc -l 4113 [root@ ~]# ifconfig -a | head -15 em0: flags=8843 metric 0 mtu 1500 options=9b ether 00:0c:29:65:4d:3e inet 172.16.16.244 netmask 0xffffff00 broadcast 172.16.16.255 media: Ethernet autoselect (1000baseT ) status: active em1: flags=8843 metric 0 mtu 1500 options=9b ether 00:0c:29:65:4d:48 inet 10.0.0.0 netmask 0xfffffffc broadcast 10.0.0.3 inet 10.0.0.4 netmask 0xfffffffc broadcast 10.0.0.7 inet 10.0.0.8 netmask 0xfffffffc broadcast 10.0.0.11 inet 10.0.0.12 netmask 0xfffffffc broadcast 10.0.0.15 inet 10.0.0.16 netmask 0xfffffffc broadcast 10.0.0.19 inet 10.0.0.20 netmask 0xfffffffc broadcast 10.0.0.23 [root@ ~]# time ifconfig -a > /dev/null real 0m0.032s user 0m0.023s sys 0m0.008s [root@ ~]# arp -na | wc -l 4100 [root@ ~]# arp -na | tail -15 ? (10.0.5.80) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.5.48) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.5.16) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.244) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.212) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.180) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.148) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.116) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.84) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.52) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (10.0.1.20) at 00:0c:29:65:4d:48 on em1 permanent [ethernet] ? (172.16.16.1) at 00:50:56:c0:00:08 on em0 [ethernet] ? (172.16.16.2) at 00:50:56:ea:ea:1a on em0 [ethernet] ? (172.16.16.254) at 00:50:56:f2:75:00 on em0 [ethernet] ? (172.16.16.244) at 00:0c:29:65:4d:3e on em0 permanent [ethernet] [root@ ~]# uptime 7:28PM up 42 mins, 2 users, load averages: 0.00, 0.00, 0.00 [root@ ~]# time arp -na > /dev/null real 0m12.761s user 0m2.959s sys 0m9.753s [root@ ~]# Notice that "arp -na" takes about 13s to execute even though there is no other load. This can get a lot worse by a few orders of magnitude on a loaded machine in a production environment, and seems to scale up linearly when more aliases are added to the interface (permanent ARP entries created). I tried the same scenario on 8.1-BETA1 and it still takes a very long time for arp(8) to complete. I was able to isolate the performance bottleneck to a small piece of the arp(8) code. It seems that looking up the interface for an ARP entry is a very heavy operation when that entry corresponds to an alias assigned to the interface. Permanent ARP entries that do not correspond with an interface alias do not seem to cause arp(8) to puke on the interface lookup. The following commands and code diff illustrates how arp(8) can be modified to run a lot faster in this scenario, but obviously the associated interface is no longer printed for each entry. [root@ /usr/src/usr.sbin/arp]# uname -a FreeBSD .localdomain 8.1-BETA1 FreeBSD 8.1-BETA1 #0: Thu May 27 15:03:30 UTC 2010 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 [root@ /usr/src/usr.sbin/arp]# time /usr/sbin/arp -na | wc -l 4100 real 0m14.903s user 0m3.133s sys 0m11.519s [root@ /usr/src/usr.sbin/arp]# pwd /usr/src/usr.sbin/arp [root@ /usr/src/usr.sbin/arp]# !diff diff -ruN arp.c.orig arp.c --- arp.c.orig 2010-06-05 18:25:24.000000000 +0000 +++ arp.c 2010-06-05 18:28:19.000000000 +0000 @@ -562,7 +562,7 @@ const char *host; struct hostent *hp; struct iso88025_sockaddr_dl_data *trld; - char ifname[IF_NAMESIZE]; + //char ifname[IF_NAMESIZE]; int seg; if (nflag == 0) @@ -591,8 +591,8 @@ } } else printf("(incomplete)"); - if (if_indextoname(sdl->sdl_index, ifname) != NULL) - printf(" on %s", ifname); + //if (if_indextoname(sdl->sdl_index, ifname) != NULL) + //printf(" on %s", ifname); if (rtm->rtm_rmx.rmx_expire == 0) printf(" permanent"); else { [root@ /usr/src/usr.sbin/arp]# make clean && make rm -f arp arp.o arp.4.gz arp.8.gz arp.4.cat.gz arp.8.cat.gz Warning: Object directory not changed from original /usr/src/usr.sbin/arp cc -O2 -pipe -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c arp.c cc -O2 -pipe -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -o arp arp.o gzip -cn arp.4 > arp.4.gz gzip -cn arp.8 > arp.8.gz [root@ /usr/src/usr.sbin/arp]# time ./arp -na | wc -l 4099 real 0m0.036s user 0m0.015s sys 0m0.021s [root@ /usr/src/usr.sbin/arp]# Notice that 0.036s without the interface lookup is a heck of a lot faster than 14.903s when doing the interface lookup. Is there something that can be done to speedup the call to if_indextoname(), or would it be worthwhile for me to submit a patch that adds the ability to skip the interface lookup as an arp(8) option? From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:21:08 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9398C106564A for ; Wed, 9 Jun 2010 21:21:08 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id 247528FC13 for ; Wed, 9 Jun 2010 21:21:07 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-064-181-139.pools.arcor-ip.net [88.64.181.139]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0MVJte-1OmRA42nFe-00YkyT; Wed, 09 Jun 2010 23:21:06 +0200 Received: (qmail 97654 invoked from network); 9 Jun 2010 21:21:06 -0000 Received: from f8x64.laiers.local (192.168.4.188) by laiers.local with SMTP; 9 Jun 2010 21:21:06 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Wed, 9 Jun 2010 23:21:05 +0200 User-Agent: KMail/1.13.3 (FreeBSD/8.0-RELEASE-p2; KDE/4.4.3; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_BXAEMBK+l1397lG" Message-Id: <201006092321.05453.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+CUuBeog1IHg8LArdDJd1tqYDZkR22M65/3ul kc52RViAyefya7DbEpqZYccn65ZHoloM6Ahlh3H5hlYRr8a0VQ +u5h58uw5zeR1mvr204Sw== Cc: Nick Rogers Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:21:08 -0000 --Boundary-00=_BXAEMBK+l1397lG Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hello Nick, On Wednesday 09 June 2010 20:17:43 Nick Rogers wrote: > Is there something that can be done to speedup the call to > if_indextoname(), or would it be worthwhile for me to submit a patch that > adds the ability to skip the interface lookup as an arp(8) option? how about the attached: # time arp -an > /dev/null 0.171u 0.462s 0:00.63 100.0% 21+1538k 0+0io 0pf+0w # time arp.patched -an > /dev/null 0.005u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w # arp -an | wc -l 1095 I'll commit this soon unless somebody objects. Thanks for your report. Max --Boundary-00=_BXAEMBK+l1397lG Content-Type: text/x-patch; charset="ISO-8859-1"; name="arp_cache_ifname.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="arp_cache_ifname.diff" diff --git a/usr.sbin/arp/arp.c b/usr.sbin/arp/arp.c index a0e228c..cc4d383 100644 --- a/usr.sbin/arp/arp.c +++ b/usr.sbin/arp/arp.c @@ -555,6 +555,9 @@ search(u_long addr, action_fn *action) /* * Display an arp entry */ +static char lifname[IF_NAMESIZE]; +static long lifindex = -1; + static void print_entry(struct sockaddr_dl *sdl, struct sockaddr_inarp *addr, struct rt_msghdr *rtm) @@ -562,7 +565,6 @@ print_entry(struct sockaddr_dl *sdl, const char *host; struct hostent *hp; struct iso88025_sockaddr_dl_data *trld; - char ifname[IF_NAMESIZE]; int seg; if (nflag == 0) @@ -591,8 +593,12 @@ print_entry(struct sockaddr_dl *sdl, } } else printf("(incomplete)"); - if (if_indextoname(sdl->sdl_index, ifname) != NULL) - printf(" on %s", ifname); + if (sdl->sdl_index != lifindex && + if_indextoname(sdl->sdl_index, lifname) != NULL) { + lifindex = sdl->sdl_index; + printf(" on %s", lifname); + } else if (sdl->sdl_index == lifindex) + printf(" on %s", lifname); if (rtm->rtm_rmx.rmx_expire == 0) printf(" permanent"); else { --Boundary-00=_BXAEMBK+l1397lG-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:30:32 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD43F106566C for ; Wed, 9 Jun 2010 21:30:32 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 5FD9D8FC0A for ; Wed, 9 Jun 2010 21:30:32 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-064-181-139.pools.arcor-ip.net [88.64.181.139]) by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis) id 0MVq96-1Okn7f0sJ6-00XW4W; Wed, 09 Jun 2010 23:30:31 +0200 Received: (qmail 97750 invoked from network); 9 Jun 2010 21:30:31 -0000 Received: from f8x64.laiers.local (192.168.4.188) by ns1.laiers.local with SMTP; 9 Jun 2010 21:30:31 -0000 From: Max Laier Organization: FreeBSD To: freebsd-hackers@freebsd.org Date: Wed, 9 Jun 2010 23:30:30 +0200 User-Agent: KMail/1.13.3 (FreeBSD/8.0-RELEASE-p2; KDE/4.4.3; amd64; ; ) References: <201006092321.05453.max@love2party.net> In-Reply-To: <201006092321.05453.max@love2party.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201006092330.30750.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+cA2b5ZCfniAOoylcTsVb6NyD4mlMnExNj37F AHJg3AdVOcr9i6L88w+PBivVvKxCWRYumlxGXCl6uhpkb62lY5 X8+1Dn/xEZlegytPBX8iw== Cc: Nick Rogers Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:30:32 -0000 On Wednesday 09 June 2010 23:21:05 Max Laier wrote: > Hello Nick, > > On Wednesday 09 June 2010 20:17:43 Nick Rogers wrote: > > Is there something that can be done to speedup the call to > > if_indextoname(), or would it be worthwhile for me to submit a patch that > > adds the ability to skip the interface lookup as an arp(8) option? > > how about the attached: > > # time arp -an > /dev/null > 0.171u 0.462s 0:00.63 100.0% 21+1538k 0+0io 0pf+0w > # time arp.patched -an > /dev/null > 0.005u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w > > # arp -an | wc -l > 1095 > > I'll commit this soon unless somebody objects. ... make that s/long/int64_t/ ... I keep forgetting that long is 32bit sometimes. From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:42:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FF19106566C for ; Wed, 9 Jun 2010 21:42:04 +0000 (UTC) (envelope-from raysonlogin@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2061D8FC0C for ; Wed, 9 Jun 2010 21:42:03 +0000 (UTC) Received: by gyh20 with SMTP id 20so5638979gyh.13 for ; Wed, 09 Jun 2010 14:42:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=G3cz+g+htt9znLYZlPuBaK1Vvwm/MB91PRuGZSWULYU=; b=iurHMAVhNS9RBX7VWB4o3skZ79WRk9NMrSi0XTy5ORhd7C2aJ3LComoQI74Gb9rdMc KguT8Y8xN91+KsfAqFwGnCtMhBJR4Fq76OIsAcpGKULFpyD4aEI9NHccATf0gh3CMZe5 ACGx5t8eWOBfxuE49LtQS+/M74TIzKN8Xxd7U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=oOOueu5isyOBsCZWSMBc2hzlYJNMkE0V3QjvBWXPkHR9GJ+TPv/F0WhJbIgXw7lhCI GQafwuGAe9+PxYtsudKAVUmAX5uZEMtPPEdOIjsSbDXpd8NLZtH4gt2er3kIfV6daO/g N/tGbzcNR2GIHtpWPOgUJAJ8wEvt+JfMoPg2o= MIME-Version: 1.0 Received: by 10.224.23.131 with SMTP id r3mr1740128qab.378.1276119722685; Wed, 09 Jun 2010 14:42:02 -0700 (PDT) Received: by 10.220.84.145 with HTTP; Wed, 9 Jun 2010 14:42:02 -0700 (PDT) In-Reply-To: <4C055F08.8070508@pathscale.com> References: <4C0108F1.5050004@pathscale.com> <20100601190909.GA13911@sandvine.com> <4C055F08.8070508@pathscale.com> Date: Wed, 9 Jun 2010 17:42:02 -0400 Message-ID: From: Rayson Ho To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:42:04 -0000 Related to debuggers... just saw the LLDB (from the LLVM project) announcement today: http://lldb.llvm.org/ All of the code in the LLDB project is available under the standard LLVM License, an open source "BSD-style" license. Rayson 2010/6/1 "C. Bergstr=F6m" : > Ed Maste wrote: >> >> On Sat, May 29, 2010 at 07:30:41PM +0700, "C. Bergstr?m" wrote: >> >> >>> >>> PathScale is slowly open sourcing and porting some of our core software >>> technology and thought the BSD community might be interested in PathDB. >>> =A0Months ago we gave a few FBSD developers private access to the sourc= e, but >>> never received any feedback. =A0Now we're asking more people to please = test >>> and tell us what you think. >>> >> >> Very intereresting. =A0I have an interest in seeing a BSD-licensed >> compelling replacement for GDB. >> >> >>> >>> Source >>> git clone git://git.pathscale.com/path64/debugger.git >>> >> >> I had a look at this, but it does not seem to be ready to build >> standalone: >> >> - it wants hg installed? >> - it is expecting directories like ../buildmeister to exist >> >> Does it depend on infrastructure / libs outside of debugger/src, or >> should it be possible to build it in isolation? >> > > Hi Ed, > > I'm answering this same question quite a bit in private and hope you don'= t > mind me adding the list on cc. (I hope those on the list don't mind as we= ll) > > It currently doens't build on any BSD, but some of these patches below wi= ll > help get you more progress > > http://www.netbsd.org/~joerg/pdb-g++.diff > http://www.netbsd.org/~joerg/readline.cc.diff > http://www.netbsd.org/~joerg/pathscale.diff =A0# For hg to git > http://pastie.org/private/cninatdq83krfy9dxagfnw =A0# For gcc-4.1+ build > support > > J=F6rg Sonnenberger has identified some areas we need to abstract which w= ill > get things more portable. =A0I'm trying to get someone who is familiar wi= th > the codebase to do this and report back once progress has been made. > > A few other people interested in the project have showed up in #pathscale= - > irc.freenode.net if you're interested in more specifics. > > Thanks > > ./C > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:47:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80D2E1065673 for ; Wed, 9 Jun 2010 21:47:27 +0000 (UTC) (envelope-from cbergstrom@pathscale.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5ECC78FC08 for ; Wed, 9 Jun 2010 21:47:27 +0000 (UTC) Received: by pvb32 with SMTP id 32so787160pvb.13 for ; Wed, 09 Jun 2010 14:47:27 -0700 (PDT) Received: by 10.142.6.31 with SMTP id 31mr360892wff.11.1276120046872; Wed, 09 Jun 2010 14:47:26 -0700 (PDT) Received: from [192.168.20.101] ([119.42.85.224]) by mx.google.com with ESMTPS id 23sm5131711pzk.6.2010.06.09.14.47.24 (version=SSLv3 cipher=RC4-MD5); Wed, 09 Jun 2010 14:47:25 -0700 (PDT) Message-ID: <4C100D4B.6000804@pathscale.com> Date: Thu, 10 Jun 2010 04:53:15 +0700 From: =?ISO-8859-1?Q?=22C=2E_Bergstr=F6m=22?= User-Agent: Thunderbird 2.0.0.22 (X11/20090909) MIME-Version: 1.0 To: Rayson Ho References: <4C0108F1.5050004@pathscale.com> <20100601190909.GA13911@sandvine.com> <4C055F08.8070508@pathscale.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: Announcing PathDB X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:47:27 -0000 Rayson Ho wrote: > Related to debuggers... just saw the LLDB (from the LLVM project) > announcement today: > > and like any other llvm project it's glued to Mac with little regard for portability.. Oh wait.. so is pathdb, but we'll at least make an effort to fix that.. We also updated the license, but I doubt anyone cares... ./C (Rayson don't intentionally hijack a thread... You can just as easily start a new one) From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:56:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 986C7106564A for ; Wed, 9 Jun 2010 21:56:29 +0000 (UTC) (envelope-from ncrogers@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id E80E18FC13 for ; Wed, 9 Jun 2010 21:56:28 +0000 (UTC) Received: by iwn7 with SMTP id 7so2986290iwn.13 for ; Wed, 09 Jun 2010 14:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=nq+FU1sLVfsDEECnSuMkbNay8KlkVdAJzLUcBCRa2Ho=; b=t7ZdoOj4wb6Mi6pSAdjxCohwpECMM46otBoZazP2VNEeWzveoYKoic6k3qjZDYwwJW xxpA+FeseSr6J6RYJYa7Iwiu2wQJ/l1Js0aGJccBro+dkmYUHTSGPwIggzN88RedjXMW HLA2+PyuTqD3053HqCzTlgsIPJQrUEn/hJKnY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Y6BRuFXzhJwjkizKLYL4g5ZwNctk5f08pWIXc2UwmfuxOUF1d1ITSaaRpA7v/N30x0 LFKxV16LxlQ7CXMtiiIwXw6wN1Rwcj9dCRVX0FU6n/Nf9IbY8ZNFmmgsDuU60joN58Ri buB/I1+9h+JfNnMoOoUcXEGKwxHVIvpilc+AI= MIME-Version: 1.0 Received: by 10.231.186.161 with SMTP id cs33mr3679892ibb.65.1276120587207; Wed, 09 Jun 2010 14:56:27 -0700 (PDT) Received: by 10.231.190.131 with HTTP; Wed, 9 Jun 2010 14:56:27 -0700 (PDT) In-Reply-To: <201006092330.30750.max@love2party.net> References: <201006092321.05453.max@love2party.net> <201006092330.30750.max@love2party.net> Date: Wed, 9 Jun 2010 17:56:27 -0400 Message-ID: From: Nick Rogers To: Max Laier Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:56:29 -0000 That worked great: [root@ /usr/src/usr.sbin/arp]# time /usr/sbin/arp -na | wc -l 4100 real 0m14.850s user 0m3.187s sys 0m11.416s [root@ /usr/src/usr.sbin/arp]# time ./arp -na | wc -l 4100 real 0m0.046s user 0m0.022s sys 0m0.023s Thanks a lot, I have been trying to get a meaningful response on this issue for about a week now. Guess I finally got the right list. Thanks again! Can this change make it into 8.1-RELEASE? On Wed, Jun 9, 2010 at 5:30 PM, Max Laier wrote: > On Wednesday 09 June 2010 23:21:05 Max Laier wrote: > > Hello Nick, > > > > On Wednesday 09 June 2010 20:17:43 Nick Rogers wrote: > > > Is there something that can be done to speedup the call to > > > if_indextoname(), or would it be worthwhile for me to submit a patch > that > > > adds the ability to skip the interface lookup as an arp(8) option? > > > > how about the attached: > > > > # time arp -an > /dev/null > > 0.171u 0.462s 0:00.63 100.0% 21+1538k 0+0io 0pf+0w > > # time arp.patched -an > /dev/null > > 0.005u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w > > > > # arp -an | wc -l > > 1095 > > > > I'll commit this soon unless somebody objects. > > ... make that s/long/int64_t/ ... I keep forgetting that long is 32bit > sometimes. > From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 21:56:44 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 638111065670 for ; Wed, 9 Jun 2010 21:56:44 +0000 (UTC) (envelope-from prvs=1769b78111=brian@FreeBSD.org) Received: from idcmail-mo2no.shaw.ca (idcmail-mo2no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id 285D38FC18 for ; Wed, 9 Jun 2010 21:56:43 +0000 (UTC) Received: from pd5ml2no-ssvc.prod.shaw.ca ([10.0.153.164]) by pd6mo1no-svcs.prod.shaw.ca with ESMTP; 09 Jun 2010 15:41:43 -0600 X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.0 c=1 a=mI6YO6ZdSLUA:10 a=VphdPIyG4kEA:10 a=MJPcHhXccCG8eBs0us8XwA==:17 a=XhyYmjemAAAA:8 a=MMwg4So0AAAA:8 a=6I5d2MoRAAAA:8 a=pH9-y6s6C3R8sN3pzFEA:9 a=PfOooPt5BuJR3buouWJKG21W8yEA:4 a=CjuIK1q_8ugA:10 a=WJ3hkfHDukgA:10 a=SV7veod9ZcQA:10 a=UK7eymURZsQolFD8HA8A:9 a=JswPGclowjEDmHJ2xqQRFI7YW_UA:4 Received: from unknown (HELO store.lan.Awfulhak.org) ([70.79.162.198]) by pd5ml2no-dmz.prod.shaw.ca with ESMTP; 09 Jun 2010 15:41:43 -0600 Received: from store.lan.Awfulhak.org (localhost.localdomain [127.0.0.1]) by localhost (Email Security Appliance) with SMTP id 0EA4DC433AF_C100A97B; Wed, 9 Jun 2010 21:41:43 +0000 (GMT) Received: from gw.Awfulhak.org (gw.lan.Awfulhak.org [172.16.0.1]) by store.lan.Awfulhak.org (Sophos Email Appliance) with ESMTP id BFFE4C460F6_C100A94F; Wed, 9 Jun 2010 21:41:40 +0000 (GMT) Received: from dev.lan.Awfulhak.org (brian@dev.lan.Awfulhak.org [172.16.0.5]) by gw.Awfulhak.org (8.14.4/8.14.4) with ESMTP id o59Lfe13041715; Wed, 9 Jun 2010 14:41:40 -0700 (PDT) (envelope-from brian@FreeBSD.org) Date: Wed, 9 Jun 2010 14:41:34 -0700 From: Brian Somers To: rhfb@akira.stdio.com Message-ID: <20100609144134.40c7393d@dev.lan.Awfulhak.org> In-Reply-To: <20100609175244.9769650815@akira.stdio.com> References: <20100609175244.9769650815@akira.stdio.com> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.20.1; i386-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/ZHWSRWOJPjYYgrh__P5RSBx"; protocol="application/pgp-signature" Cc: freebsd-hackers@FreeBSD.org Subject: Re: NFSD lockup running ESXi 4 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 21:56:44 -0000 --Sig_/ZHWSRWOJPjYYgrh__P5RSBx Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Wed, 9 Jun 2010 13:52:40 -0400 (EDT) rhfb@akira.stdio.com wrote: > I have an AMD64 FreeBSD 8.0 running 8-Stable from around 2010/04/25 19:13= :08. >=20 > ZFS disk, Nfsd flags "-t -n 16", private network exclusive for nfs networ= k, > not using jumbo frames, HZ=3D1000, Device_Polling, Zero_Copy_Sockets, and= the > following sysctl options: > net.inet.tcp.recvspace=3D232140 > net.inet.tcp.sendspace=3D232140 > net.inet.tcp.slowstart_flightsize=3D159 > net.inet.tcp.mssdflt=3D1460 >=20 > FreeBSD 6 TB zpool, nfs from Three ESXi 4 (newest patch level 193498) > working reliably for months. >=20 > Added a new ESXi, patched to the newest (Post Update 1) patch level 25696= 8. > Added a bunch of VM's, booted them all into the 2008 R2 Server install DV= D. > Then when attempting to do the installs (in parallel/simultaneously) I st= arted > getting the NFS server locking up. NFSD would wedge at 100% CPU in "rc_l= o" > which I presume is rc_lock? Once wedged, /etc/rc.d/nfsd restart can't ki= ll > nfsd. So a reboot is required. A Reboot causes all my active VM's with > pending disk writes to have disk errors in the VM (10 second default time= out > for disk writes in the VM.) This was very reproducable. >=20 > Has anyone noticed this problem? Is this an ESXi problem with the newest > updates? Is this a problem with NFS on FreeBSD 8? I don't know if it's relevant, but I've been having nfs issues on -current. I believe they were caused by gam_server, a gnome program running on an NFS client machine that had /usr/ports nfs mounted and was doing a portupgr= ade. Nothing gnomeish should have been anywhere near /usr/ports, but analysis showed huge numbers of NFS stats against /usr/ports/distfiles/*, restat'ing the same files over and over. nfsd was going crazy on the server and gam_server was clocking up wads of CPU time on the client. FreeBSD-9 kernels prior to around June 6 were freezing on me. It may have been because of the nfsd activity, but I didn't investigate the freeze... Perhaps looking for changes that might might affect nfsd stability in the w= eek prior to June 6 might discover a fix? --=20 Brian Somers Don't _EVER_ lose your sense of humour ! --Sig_/ZHWSRWOJPjYYgrh__P5RSBx Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQCVAwUBTBAKkw7tvOdmanQhAQKGjAP8CCGA5Rn65QmLFMZj1MEqlQjlHt8NeTM3 +HcIfvsMCYVrvDka/1e5MpN42cby+XTEfpW1IE2Ja2Y4xQ0Cv4C0txqi5S+uxzGM Z1Q0kw1ZB43JhI6sQHZcefsquwg6gHnmLPGJJkujxrRvmhVyKd5Zx7hTe+7lz/KS s7Ydpe3b3Gs= =a4IS -----END PGP SIGNATURE----- --Sig_/ZHWSRWOJPjYYgrh__P5RSBx-- From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 9 22:25:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 37DAF106568B for ; Wed, 9 Jun 2010 22:25:15 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id BEB978FC1A for ; Wed, 9 Jun 2010 22:25:14 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-064-181-139.pools.arcor-ip.net [88.64.181.139]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0LosLT-1OuQss1zyd-00gluz; Thu, 10 Jun 2010 00:25:13 +0200 Received: (qmail 98812 invoked from network); 9 Jun 2010 22:25:13 -0000 Received: from f8x64.laiers.local (192.168.4.188) by router.laiers.local with SMTP; 9 Jun 2010 22:25:13 -0000 From: Max Laier Organization: FreeBSD To: Nick Rogers Date: Thu, 10 Jun 2010 00:25:12 +0200 User-Agent: KMail/1.13.3 (FreeBSD/8.0-RELEASE-p2; KDE/4.4.3; amd64; ; ) References: <201006092330.30750.max@love2party.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201006100025.12881.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18YHhi2dXdkcGNTWqxgMe4G7QZFwuTbHaHCQGy ITzNpd7vUc/oRLrSrQ6RW0VogVRyqqMBwhl69em6DCuTcALioe ihd3VoKW0HhJtgEKOo3ag== Cc: freebsd-hackers@freebsd.org Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jun 2010 22:25:15 -0000 On Wednesday 09 June 2010 23:56:27 Nick Rogers wrote: > Thanks a lot, I have been trying to get a meaningful response on this issue > for about a week now. Guess I finally got the right list. Thanks again! I saw your post before on -net etc, just never found the time ... and then I forgot. So it was more the right timing, than the right list :-) > Can this change make it into 8.1-RELEASE? It looks like it might be too late, I'll ask once the HEAD commit is done, but I wouldn't count on it. > On Wed, Jun 9, 2010 at 5:30 PM, Max Laier wrote: > > On Wednesday 09 June 2010 23:21:05 Max Laier wrote: > > > Hello Nick, > > > > > > On Wednesday 09 June 2010 20:17:43 Nick Rogers wrote: > > > > Is there something that can be done to speedup the call to > > > > if_indextoname(), or would it be worthwhile for me to submit a patch > > > > that > > > > > > adds the ability to skip the interface lookup as an arp(8) option? > > > > > > how about the attached: > > > > > > # time arp -an > /dev/null > > > 0.171u 0.462s 0:00.63 100.0% 21+1538k 0+0io 0pf+0w > > > # time arp.patched -an > /dev/null > > > 0.005u 0.000s 0:00.00 0.0% 0+0k 0+0io 0pf+0w > > > > > > # arp -an | wc -l > > > > > > 1095 > > > > > > I'll commit this soon unless somebody objects. > > > > ... make that s/long/int64_t/ ... I keep forgetting that long is 32bit > > sometimes. > > !DSPAM:4c100e18981892214220659! From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 08:05:27 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E4BF1065672 for ; Thu, 10 Jun 2010 08:05:26 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id E74BE8FC18 for ; Thu, 10 Jun 2010 08:05:24 +0000 (UTC) Received: by iwn7 with SMTP id 7so3546875iwn.13 for ; Thu, 10 Jun 2010 01:05:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=c0hEfUZFRybzFOAKJBZf/pAenG8ZzN7Un3OA/1uxQgA=; b=E53Y/g1Wj/Ssq1Y5olsDIE2Xk4LnV0thnrVeUJGnREtaFCY6FogpVQo5/8NLiUHBkK fx+WB4wcKfPoJjWQyhHggyQXw27UUbRJdUvZ82THxo8XGJ/Wlvzfk28vS7HsSvlMhXLs A005UnN8JRRFJC8CawG0JushdAh1xVo3xFstE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=KJ1tOdLLKbg9VJF+B3R8sjyRAEjO0kEyAoZsKRtj4uR8Fv1EKQ5JdU2tanqzl7lZyQ KkJMyzC6Rsbs8pU6PSNYfm4aPZjOxHgt3Fda/LX94xxseLdW8LMHRzVCNDHA+iRq3Ki7 SRoflfYCormtlNX+lA+1+ny2HysxihVBkzDk4= MIME-Version: 1.0 Received: by 10.231.111.209 with SMTP id t17mr9018196ibp.182.1276157124260; Thu, 10 Jun 2010 01:05:24 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.156.76 with HTTP; Thu, 10 Jun 2010 01:05:24 -0700 (PDT) In-Reply-To: <201006092321.05453.max@love2party.net> References: <201006092321.05453.max@love2party.net> Date: Thu, 10 Jun 2010 16:05:24 +0800 X-Google-Sender-Auth: aCYHrrFlVRFs-aFMx7wHpcE9lSo Message-ID: From: Adrian Chadd To: Max Laier Content-Type: text/plain; charset=ISO-8859-1 Cc: Nick Rogers , freebsd-hackers@freebsd.org Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 08:05:28 -0000 On 10 June 2010 05:21, Max Laier wrote: >> Is there something that can be done to speedup the call to >> if_indextoname(), or would it be worthwhile for me to submit a patch that >> adds the ability to skip the interface lookup as an arp(8) option? > > how about the attached: > This patch only short-circuits the lookup for a specific case - lots of entries with the same ifname. What about other situations - say, alternating interfaces in ARP? I read if_indextoname(). The problem is that each call is sucking down the ifaddrs table and looking for the index. Is this likely to be a problem in other situations, rather than just /usr/sbin/arp ? Adrian From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 08:18:06 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20CD8106567A for ; Thu, 10 Jun 2010 08:18:06 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 94F338FC19 for ; Thu, 10 Jun 2010 08:18:05 +0000 (UTC) Received: by fxm7 with SMTP id 7so485745fxm.13 for ; Thu, 10 Jun 2010 01:18:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:in-reply-to:references:reply-to:x-mailer:mime-version :content-type:content-transfer-encoding; bh=nTzdZYGBxhgIDv+FjpuygmAf6MBJgEsX3D6p3MPD2Co=; b=B9d3wdFgBdYQQngHUG1DG4gvSiKGNIssGnO4NUsxz393PsSVGHXij49TRc46riQMDw r/9r7FikN3vzCWiRhZ95UppjuTOML/oNzqvpZ/NYAsFScrBmPdJMdCbv4mWeRNaAfI0p kh/es3d7Ye//kq4SwvYz52CLacQQqo7I+k+Nk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; b=TMWBmEOGJDHBJPQ4L6N5OIymowaGOj3Y34e+t/Wg31zjGsEqQitYTQkzcl4wSWbWT8 D0bORrnJnvqRqfxk7fmRl0NGD+nPi+HFkdb5CLvY0M102KFbHlzikxPUeivxWO6D6T4F 1W2i87wapexFswo2vb94kAFh6a//+4XT3nJFE= Received: by 10.223.97.149 with SMTP id l21mr3957145fan.91.1276157884191; Thu, 10 Jun 2010 01:18:04 -0700 (PDT) Received: from ernst.jennejohn.org (p578E22C4.dip.t-dialin.net [87.142.34.196]) by mx.google.com with ESMTPS id j23sm7647546faa.14.2010.06.10.01.18.03 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 10 Jun 2010 01:18:03 -0700 (PDT) Date: Thu, 10 Jun 2010 10:18:01 +0200 From: Gary Jennejohn To: Garrett Cooper Message-ID: <20100610101801.742fac25@ernst.jennejohn.org> In-Reply-To: References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Bakulin , Ilya Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 08:18:06 -0000 On Wed, 9 Jun 2010 10:12:54 -0700 Garrett Cooper wrote: > On Jun 9, 2010, at 6:25 AM, Kostik Belousov wrote: > > > On Wed, Jun 09, 2010 at 09:13:56AM -0400, jhell wrote: > >> On 06/09/2010 04:14, Ilya Bakulin wrote: > >>> Hi hackers! > >>> > >>> While discussing my project's implementation details with my mentor, > >>> Alexander Leidinger, we've found that one of the ideas needs to be discussed with community, > >>> to find out possible use cases. > >>> That is, if it should be possible to spoof non-existing features. For > >>> example, if currently running kernel doesn't support FreeBSD 5.0 compat > >>> layer, "kern.features.compat_freebsd5" will be absent when querying > >>> features list. The question is -- are there any cases when we want > >>> "kern.features.compat_freebsd5" be present? If some feature is not in > >>> kernel, then presenting its existence to the userland is useless > >>> and may be even harmful, if, for example, some application relies on this feature. > >>> Or there are some scenarios where such cheat is useful? > >>> > >> > >> I can not think of any viable reason why one would want to "spoof" this > >> when it is not available. > > Many ports are doing wrong thing there, checking for run-time features at > > the build-time, turning on/off some functionality depending on its > > presence on the build host. > > It's present in the ports Makefiles as well as in many autoconf scripts. It's bad because it causes problems with cross-build and other related scenarios, where you can't assume that the host system is going to match the target system. > I don't find one single file in the ports tree which uses kern.features. But I just checked what's in the tree, not what may be in the ports themselves, i.e., I didn't extract/configure any ports. -- Gary Jennejohn From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 11:30:24 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BC781065672; Thu, 10 Jun 2010 11:30:24 +0000 (UTC) (envelope-from eng.mufic@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AA85A8FC16; Thu, 10 Jun 2010 11:30:23 +0000 (UTC) Received: by bwz2 with SMTP id 2so2910375bwz.13 for ; Thu, 10 Jun 2010 04:30:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=lw/VUaVVD5KCcvIzjDpTxCr3l/7J+zKq0cWUKmpUZ0c=; b=esB/Z1Nvl0PuVIVbNAcya9O83o7Tn9jalHx5G6tFksIAesXHJewLeYgDuAzn9iHkZM x0H9oJU3LGTWmGdAXQftzRL6DjKMVkXyhydJQmqnSwTe+A8KRJD2d/LUM/yzJxnjHrA0 aC62WDrjGcqNji0oURdDvAvH9M85aJfbFIIxs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=eLcZsa34WIUX+lZGv2AG/0JCVi+BHvub0ual1WlNqhyk+myoO0enoBnHXHVXyUJLie y+V0nGEid9ZXZka73bTYv3RQy/rewDVspqMRQHklTwHGwaoGBSd+fdHfxw3SRTjfJ1Vz elqPYqpvVNieZMbE7OZ0doXqpzMnti/ogOpEo= MIME-Version: 1.0 Received: by 10.204.47.6 with SMTP id l6mr39189bkf.90.1276169422286; Thu, 10 Jun 2010 04:30:22 -0700 (PDT) Sender: eng.mufic@gmail.com Received: by 10.216.13.6 with HTTP; Thu, 10 Jun 2010 04:30:22 -0700 (PDT) Date: Thu, 10 Jun 2010 14:30:22 +0300 X-Google-Sender-Auth: wWyXFHkno6iQrjG5ZvHYX921U-o Message-ID: From: Mohammed Farrag To: freebsd-current@freebsd.org, hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: New FreeBSD Kenel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 11:30:24 -0000 Hi all, I would like to share with you my work in Summer of Code for Producing a New FreeBSD Kernel Theme in the field of Reducing Size of Kernel for Embedded and I will be glad to receive your comments. Download the attached file please and I will be happy to connect with you or download it http://www.4shared.com/document/U237Abgs/description.html Good Luck, Mohammed Farrag From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 11:36:01 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92C55106566C for ; Thu, 10 Jun 2010 11:36:01 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 4C9318FC08 for ; Thu, 10 Jun 2010 11:36:01 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OMg3A-0007bu-Dm for freebsd-hackers@freebsd.org; Thu, 10 Jun 2010 13:35:56 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 10 Jun 2010 13:35:56 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 10 Jun 2010 13:35:56 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org connect(): No such file or directory From: Ivan Voras Date: Thu, 10 Jun 2010 13:35:47 +0200 Lines: 19 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 X-Enigmail-Version: 1.0.1 Subject: Proposal - increase SYSV SEMMNI and SEMMNS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 11:36:01 -0000 Currently, the defaults are: kern.ipc.semmni: 10 kern.ipc.semmns: 60 but there are several nontrivial pieces of software that would use much more; two examples are PostgreSQL (the database) and Mono (the .net clone). I'd like to increase these defaults to: kern.ipc.semmni: 50 kern.ipc.semmns: 300 If I'm reading it correctly, each semaphore takes around 12 bytes (times SEMMNS), allocated with kernel malloc at boot time. This change is not as critical as before since both are now loader tunables, but increasing the defaults would be a convenience move, to save some reconfiguration-and-reboot hassle when installing software. From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 11:47:10 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88F661065672 for ; Thu, 10 Jun 2010 11:47:10 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3FC418FC08 for ; Thu, 10 Jun 2010 11:47:09 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1OMgDx-0005WW-7f for freebsd-hackers@freebsd.org; Thu, 10 Jun 2010 13:47:05 +0200 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 10 Jun 2010 13:47:05 +0200 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 10 Jun 2010 13:47:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org connect(): No such file or directory From: Ivan Voras Date: Thu, 10 Jun 2010 13:46:53 +0200 Lines: 13 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100518 Thunderbird/3.0.4 In-Reply-To: X-Enigmail-Version: 1.0.1 Cc: freebsd-current@freebsd.org Subject: Re: New FreeBSD Kenel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 11:47:10 -0000 On 06/10/10 13:30, Mohammed Farrag wrote: > Hi all, > > I would like to share with you my work in Summer of Code for Producing a > New FreeBSD Kernel Theme in the field of Reducing Size of Kernel for > Embedded and I will be glad to receive your comments. Download the attached > file please and I will be happy to connect with you > or download it http://www.4shared.com/document/U237Abgs/description.html The description is too long and convoluted. Are you trying to say you want to create infrastructure which will allow a minimal kernel to load modules with device drivers dynamically, depending on detected hardware in the system? From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 11:53:00 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A964B106567B for ; Thu, 10 Jun 2010 11:53:00 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id 3AA578FC1B for ; Thu, 10 Jun 2010 11:53:00 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-064-181-139.pools.arcor-ip.net [88.64.181.139]) by mrelayeu.kundenserver.de (node=mrbap2) with ESMTP (Nemesis) id 0MQLgi-1OqMI241y0-00UdZH; Thu, 10 Jun 2010 13:52:59 +0200 Received: (qmail 12354 invoked from network); 10 Jun 2010 11:52:58 -0000 Received: from f8x64.laiers.local (192.168.4.188) by router.laiers.local with SMTP; 10 Jun 2010 11:52:58 -0000 From: Max Laier Organization: FreeBSD To: Adrian Chadd Date: Thu, 10 Jun 2010 13:52:57 +0200 User-Agent: KMail/1.13.3 (FreeBSD/8.0-RELEASE-p2; KDE/4.4.3; amd64; ; ) References: <201006092321.05453.max@love2party.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201006101352.57499.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+enEYcVBFB1LJRsZ+HbrIalbWhTVL+nm/IEyu hACNDO+1NqMznwPZHxvCyx921lH7Iar/SRwzOmwSDq4tiID9WF yG/Z5cb6pzJyrecbrwkWg== Cc: Nick Rogers , freebsd-hackers@freebsd.org Subject: Re: arp(8) performance w/ many aliases assigned to an interface X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 11:53:00 -0000 On Thursday 10 June 2010 10:05:24 Adrian Chadd wrote: > On 10 June 2010 05:21, Max Laier wrote: > >> Is there something that can be done to speedup the call to > >> if_indextoname(), or would it be worthwhile for me to submit a patch > >> that adds the ability to skip the interface lookup as an arp(8) option? > > > > how about the attached: > This patch only short-circuits the lookup for a specific case - lots > of entries with the same ifname. What about other situations - say, > alternating interfaces in ARP? Yes, it is bandaid, but I believe it covers most if not all common cases. I didn't want to start with a full cache, to avoid a big memory footprint. > I read if_indextoname(). The problem is that each call is sucking down > the ifaddrs table and looking for the index. Is this likely to be a > problem in other situations, rather than just /usr/sbin/arp ? I was also thinking about building a sysctl to export index to name information directly, but I don't think it's worth the hassle. Other (long running) programs that need index->name mapping over and over again already do caching that is specific to their particular usecase. See e.g. routed's if.c Regards, Max From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 12:19:29 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92E361065676; Thu, 10 Jun 2010 12:19:29 +0000 (UTC) (envelope-from eng.mufic@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id EE16E8FC19; Thu, 10 Jun 2010 12:19:28 +0000 (UTC) Received: by wyb33 with SMTP id 33so1731333wyb.13 for ; Thu, 10 Jun 2010 05:19:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=BPvQgAu7SUjpDJtzJKPEPYwTXZxM2/R7Sq4mbEc0MCs=; b=gmzJcwzGq5ZrwkmmoxNGb79zqcgy60+VVxkFEpGGCwpm/7EkPR+kqMA2c+sChjA82p xcyYGMCu43WHessseexAIfJ1txVftbm93XRtRu3tbLOegrVNyX8QhwOFeqiH1livzTm5 qOrHs2saK44k4Q0me2hst6IVauIvST+L+s1XE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=YaFe9HGnGRwGC+w4XnNFC/YpEOs4Gc0NiK3OrwMOunubpZEoGP+FS0q8obgWk4N0Oi HeE0fXQFL1xEsvDoQn2T+N3sj4jUhKMcvZO+eWMSeTGSsDJto/rBszZgPPnMI6f91TeX TODZN44qpWmdw/ONXACH88p1JSNLozlsoB98A= MIME-Version: 1.0 Received: by 10.216.187.204 with SMTP id y54mr60425wem.1.1276170816975; Thu, 10 Jun 2010 04:53:36 -0700 (PDT) Received: by 10.216.13.6 with HTTP; Thu, 10 Jun 2010 04:53:36 -0700 (PDT) In-Reply-To: References: Date: Thu, 10 Jun 2010 14:53:36 +0300 Message-ID: From: Mohammed Farrag To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: New FreeBSD Kenel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 12:19:29 -0000 exactly and I explained with an example how I would do that practically. From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 13:26:53 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9F901065673; Thu, 10 Jun 2010 13:26:53 +0000 (UTC) (envelope-from romain@blogreen.org) Received: from marvin.blogreen.org (unknown [IPv6:2a01:e35:2f7d:58c0::2]) by mx1.freebsd.org (Postfix) with ESMTP id 3B4A68FC08; Thu, 10 Jun 2010 13:26:53 +0000 (UTC) Received: by marvin.blogreen.org (Postfix, from userid 1001) id 5ACC168E3; Thu, 10 Jun 2010 15:26:52 +0200 (CEST) Date: Thu, 10 Jun 2010 15:26:52 +0200 From: Romain =?iso-8859-1?Q?Tarti=E8re?= To: Ivan Voras Message-ID: <20100610132652.GA54859@FreeBSD.org> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M9NhX3UHpAaciwkO" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://romain.blogreen.org/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org Subject: Re: Proposal - increase SYSV SEMMNI and SEMMNS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 13:26:53 -0000 --M9NhX3UHpAaciwkO Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 10, 2010 at 01:35:47PM +0200, Ivan Voras wrote: > kern.ipc.semmni: 10 -> 50 > kern.ipc.semmns: 60 -> 300 Looks like a good idea. > This change is not as critical as before [...] but increasing the > defaults would be a convenience move, to save some > reconfiguration-and-reboot hassle when installing software. In the same vein, and while they can be tuned without a reboot, some ports require to increase kern.ipc.shmmax and kern.ipc.shmall for them to work correctly (e.g. multimedia/totem). Maybe all these 'convenience increases of IPC related defaults' can be done together? Thanks! Romain --=20 Romain Tarti=E8re http://people.FreeBSD.org/~romain/ pgp: 8234 9A78 E7C0 B807 0B59 80FF BA4D 1D95 5112 336F (ID: 0x5112336F) (plain text =3Dnon-HTML=3D PGP/GPG encrypted/signed e-mail much appreciated) --M9NhX3UHpAaciwkO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQGcBAEBAgAGBQJMEOgaAAoJELpNHZVREjNv5j4L/iIMjfa9JQk6jjsFXSWKZb1Q zSJiPGqxUPeUaZWqaS7cgnHNKi5KFJYfU3ybtBn9nAr5jw6hO7sU/fXIJwzzVQYr 1xBCItI6x8a4xBO5uWz3rwPge50Lz+lmoRa/rLIySkiQsRQu6KLyUWZa1dZd1tjO 4QjjeCLyfXUReaFsJOJz0UJxtIAf0GeXSAqYJLc9Ut9J3XUJW7r0At/XM1l6fm4z i40FOUu7ZvopJjUoNpeT2Exn96tDOUfAa2kWFXL22suAgKGVhTZt5/vml0PFZqOi YrboQTpmWF9BWgtVfeximn7qf2i05H1ot0PLjxwT+t2hTqc2wCbMNdUV9Mzdm0t4 ZU/Cz08RsEbTVDwhY6zh+UtwJP/fFX5r0wqPbHCKPCqPK1JKJjtfxsxy4zns6FWt Ypoq+fZG3qubsQ5WWa3Uz9IN3BQ5LuhYCtnGpf+9/ck6mTluKBP3SEGkdvT9NbH/ 0DLqc+TvFDCvOs7L7oKbYtPAgZ6AJme7wq3e+j0VBQ== =qeOG -----END PGP SIGNATURE----- --M9NhX3UHpAaciwkO-- From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 14:01:14 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE88F1065673 for ; Thu, 10 Jun 2010 14:01:13 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 85B208FC19 for ; Thu, 10 Jun 2010 14:01:13 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FE15.dip.t-dialin.net [217.84.254.21]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 005A284400A; Thu, 10 Jun 2010 16:01:09 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 492C7512E; Thu, 10 Jun 2010 16:01:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1276178466; bh=+i+ubqlLfB/2gxl1goNnFQdhpexWMaDz7gbi7SM8gsk=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=p8Ogb8QYokXYgbeRWRfQM5gqskiLG9vp1xfQy1K8fwZWXFCPNuZ/UDqe8Gdln5J0O FtszqKVTywMBeyjBXwfeVbFXmQr/H/05rzgubw9i+oUa3zT41A+l1vaGdjTNj9KfH0 kwbprLpPR04/nYw3XyFAayjbb4M87WRedaboC8jRQUtE0nHdxkJIPPit+fei8B+fNs 42Oal4s91ZuhrQaMGTG5vL5J30uvVOues7BPW8837Nvjw+YLflDxRScuPrWYFkAlb3 3TIfPgCyjkZ3NvU9wgpeUzsRUO5Gifja8jGdrkTwNtCsGukuKPy0lSo5/zhbNnsWeG pJ1He355OKMIQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o5AE15IY035992; Thu, 10 Jun 2010 16:01:05 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 10 Jun 2010 16:01:04 +0200 Message-ID: <20100610160104.16285uq2jhuio34s@webmail.leidinger.net> Date: Thu, 10 Jun 2010 16:01:04 +0200 From: Alexander Leidinger To: jhell References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> <4C0FAB18.7010902@dataix.net> In-Reply-To: <4C0FAB18.7010902@dataix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 005A284400A.A8B86 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.1, required 6, autolearn=disabled, ALL_TRUSTED -1.00, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1276783270.91309@JbfyCxaJuohDnSCKz4293Q X-EBL-Spam-Status: No X-Mailman-Approved-At: Thu, 10 Jun 2010 14:10:58 +0000 Cc: Kostik Belousov , Ilya Bakulin , freebsd-hackers@freebsd.org Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 14:01:14 -0000 Quoting jhell (from Wed, 09 Jun 2010 10:54:16 -0400): > That would lead me to believe that the elimination of this sysctl would > be better suited to solve the outcome of cases like this. > > And leads me to believe that it still rests on the end-user to tell > whether or not they have that compatibility layer compiled in. > > Like I stated more towards the end of my last message "I believe" > checking __FreeBSD_version should suffice and leave the final result up > to the end-user as GENERIC will have the COMPAT_FREEBSD{N} layers > compiled in that it needs or can support or are recommended. > > Being that this is a broad scenario and many different compilations of > kernels could be used I still do not see a need to test for every one of > them if an adequate means already exists. GENERIC in any case should be > the kernel that is depended on and testing against __FreeBSD_version for > what COMPAT versions are supported. Maybe there is a misconception of what is being done. We will have something like (out of the top of my head, do not nail me on the spelling or a particular feature): kern.features.compat_4 kern.features.compat_6 kern.features.softupdates kern.features.ufs_snapshots As soon as something is active in the kernel (directly compiled in or via a module), the corresponding kern.features.XXX entry will show up (with a value of 1). The spoofing feature which Ilya was talking about in this thread will for sure allow to mask an existing feature away. __FreeBSD_version is not adequate at all for things which require a feature to be present in the kernel (can be removed in a custom kernel) and are required at run-time (build-time checks do not help here, just think about building and installing a program, and then changing the kernel config and installing a new kernel). Bye, Alexander. -- "Don't worry. Clemenza is OK. It's Paulie." -- Santino Corleone, "Chapter 4", page 93 http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 14:01:18 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BE841065676 for ; Thu, 10 Jun 2010 14:01:18 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id C3CAC8FC08 for ; Thu, 10 Jun 2010 14:01:17 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FE15.dip.t-dialin.net [217.84.254.21]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id E5C3A84405D; Thu, 10 Jun 2010 16:01:12 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id F23EF512F; Thu, 10 Jun 2010 16:01:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1276178470; bh=HcPrudxpxLqS+U0NedjPixqJKzBYsz/XKYvgqn5aND0=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=sLvxMRZV/YQB8F7+lVk2pC61pB2KqSQ0iQXEROb02fEmnW9Yox1CodzjBdTl6/uDt ePcfGgqUT0zlQ7nMJ8djkgUfmFS0RGCibDxrAEVI3T268xFwrVYu4R/ApVb/8mycS0 5NxmAmlyXXhjH+R0Y0en26BJWLtL8zgYp8+ELy3mLTnephpT9fYRuw0dvt3FRIADZR R3J2m5IAvO+bJkSBPtBbtn+G+fycigiQDyQzAEEHenHtmtATH+ofs6+Ty1oFEW92RS +gVlW0gc5UWDJhwb+bHdp6efPYT43lTr52GC2Cq1vMD8v/ck427KJOfaOobTy2x5cA t0y1t0fTB2u0A== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o5AE19n8036646; Thu, 10 Jun 2010 16:01:09 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 10 Jun 2010 16:01:09 +0200 Message-ID: <20100610160109.19585782fyr3buw4@webmail.leidinger.net> Date: Thu, 10 Jun 2010 16:01:09 +0200 From: Alexander Leidinger To: gljennjohn@googlemail.com References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> <20100610101801.742fac25@ernst.jennejohn.org> In-Reply-To: <20100610101801.742fac25@ernst.jennejohn.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: E5C3A84405D.A82A7 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=1.5, required 6, autolearn=disabled, ALL_TRUSTED -1.00, BR_SPAMMER_URI 2.00, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10, J_CHICKENPOX_48 0.60) X-EBL-MailScanner-SpamScore: s X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1276783275.65343@5yqbKXSQyv3MiHqpVh3vCw X-EBL-Spam-Status: No X-Mailman-Approved-At: Thu, 10 Jun 2010 15:49:03 +0000 Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Garrett Cooper , Ilya Bakulin Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 14:01:18 -0000 Quoting Gary Jennejohn (from Thu, 10 Jun 2010 10:18:01 +0200): > On Wed, 9 Jun 2010 10:12:54 -0700 > Garrett Cooper wrote: > >> On Jun 9, 2010, at 6:25 AM, Kostik Belousov wrote: >> >> > On Wed, Jun 09, 2010 at 09:13:56AM -0400, jhell wrote: >> >> On 06/09/2010 04:14, Ilya Bakulin wrote: >> >>> Hi hackers! >> >>> >> >>> While discussing my project's implementation details with my mentor, >> >>> Alexander Leidinger, we've found that one of the ideas needs to >> be discussed with community, >> >>> to find out possible use cases. >> >>> That is, if it should be possible to spoof non-existing features. For >> >>> example, if currently running kernel doesn't support FreeBSD 5.0 compat >> >>> layer, "kern.features.compat_freebsd5" will be absent when querying >> >>> features list. The question is -- are there any cases when we want >> >>> "kern.features.compat_freebsd5" be present? If some feature is not in >> >>> kernel, then presenting its existence to the userland is useless >> >>> and may be even harmful, if, for example, some application >> relies on this feature. >> >>> Or there are some scenarios where such cheat is useful? >> >>> >> >> >> >> I can not think of any viable reason why one would want to "spoof" this >> >> when it is not available. >> > Many ports are doing wrong thing there, checking for run-time features at >> > the build-time, turning on/off some functionality depending on its >> > presence on the build host. >> >> It's present in the ports Makefiles as well as in many autoconf >> scripts. It's bad because it causes problems with cross-build and >> other related scenarios, where you can't assume that the host >> system is going to match the target system. >> > > I don't find one single file in the ports tree which uses kern.features. That's not a surprise, currently there is nothing in kern.features to check for. One of the goals of the GSoC project is to add features to check for. We will have something like (out of the top of my head, do not nail me on the spelling or a particular feature): kern.features.compat_4 kern.features.compat_6 kern.features.softupdates kern.features.ufs_snapshots As soon as something is active in the kernel (directly compiled in or via a module), the corresponding kern.features.XXX entry will show up (with a value of 1). The spoofing feature which Ilya was talking about in this thread will for sure allow to mask an existing feature away. The big question is: where would we need a spoofing feature where a non-existing feature (= no runtime support in the kernel) needs to show up with a value of 1 (let's call this "spoof-on")? We identified the following obvious cases: - the software *needs* the feature at build-time -> we should not "spoof-on" a non-existing feature (build error) - the software needs the feature at run-time but not at build-time and the port/configure checks at build-time -> the port needs to be changed to test at run-time, spoof-on will hurt and is not needed (the feature is not used yet, so instead of changing the build to check for the feature systl the time is better spend to check at run-time -> no need for spoof-on) - the software needs the feature at run-time and checks at run-time -> spoof-on will hurt What we search for is a port which does not fall into one of the above cases and where spoof-on does not hurt. Bye, Alexander. -- One man's folly is another man's wife. -- Helen Rowland http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 10 22:34:29 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E6AE1065674 for ; Thu, 10 Jun 2010 22:34:29 +0000 (UTC) (envelope-from fergleiser@yahoo.com) Received: from web31706.mail.mud.yahoo.com (web31706.mail.mud.yahoo.com [68.142.201.186]) by mx1.freebsd.org (Postfix) with SMTP id 1CE1B8FC19 for ; Thu, 10 Jun 2010 22:34:28 +0000 (UTC) Received: (qmail 32424 invoked by uid 60001); 10 Jun 2010 22:07:48 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1276207668; bh=fXinuKNkTP+9L6ism50p9D4Xza6LhMiFLDlmM9wwCh4=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=RUhGkofivkqrYxEc8nonk/HhuyTU+YtWolYcj18I5Zd4fRG3RMSEQMrqeHWsNxXUolSQJztek5GB6xdQUjbq+UGNGAfzMJIqHpkBwXLRyF9xQBkOiuDCWyZJWLNEkGv4xKSzUqK/OgEXO+OrSFhh5FCDRpSXc8GvDQAWHWVhKkg= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type; b=441oXeNpKaQWgMwNGbzlXvoYG63U01IyG8Vs0PdSItKkL9jeCOnW+bOKn+ICrNcWsclGAaeJfKCSa/S0OTQWiq7f0Rk2WRE3sQC3mOUy0irxtu8rS9kjJ+qUxOXmUl/G+JrxoNEFwp8Tp8h1O60kqoyC95xX94kqXLfVce0PoCg=; Message-ID: <384709.32358.qm@web31706.mail.mud.yahoo.com> X-YMail-OSG: Uv68Oq0VM1nMI.TBHC79vvwKY0p4ik260myMql6m9xwZoUQ WPOvYsbqU_j83bitLYEjxwrAy9GOr_FxAS52fHd6ypCuH8.CaxAgBIan95HH laMMOmZeNje_O_tHeP6ylecFyAgvPyeWDQyjL0Y9QKDpwxlX8JAqrseeS6_j QKd9nt9ahbBZ8kgzuAlTeht2C7iFcTdTts_s9Cis0coNTv54900c28Zyj4kD e9RYDXS9WqZeVxMu_SYOhKDxuqWRiFnMQy2YRZ749hvVHQiESHLn89v8cXi4 DUmeo2Ri9gTBvKux4S8weH0H.SB4EHoSyqQQTEQMkCwd9QEVJMSFHqGA- Received: from [190.191.255.130] by web31706.mail.mud.yahoo.com via HTTP; Thu, 10 Jun 2010 15:07:48 PDT X-Mailer: YahooMailRC/397.8 YahooMailWebService/0.8.103.269680 Date: Thu, 10 Jun 2010 15:07:48 -0700 (PDT) From: Fernando Gleiser To: hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Subject: Network stack virtualization docs? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jun 2010 22:34:29 -0000 I'm looking for some technical docs about FreeBSD's network stac virt. I saw a paper from the wiki and little else. The paper seems to focus mostly on the paralielization of the network stack more than the framework for a virtualized network. I want to compare it against OpenSolaris' crossbow, which has a way to virtualize almost anything: virtual nics, virtual switches and virtual wires, so you can build a "wan in a box". Is there anything like that for FreeBSD? any pointers to good docs would be apreciated Fer From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 00:19:15 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E10241065678 for ; Fri, 11 Jun 2010 00:19:15 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outf.internet-mail-service.net [216.240.47.229]) by mx1.freebsd.org (Postfix) with ESMTP id C355F8FC19 for ; Fri, 11 Jun 2010 00:19:15 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o5B0JEI0028150; Thu, 10 Jun 2010 17:19:14 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (64.1.209.194.ptr.us.xo.net [64.1.209.194]) by idiom.com (Postfix) with ESMTP id 7D2702D6016; Thu, 10 Jun 2010 17:19:13 -0700 (PDT) Message-ID: <4C118119.5050303@elischer.org> Date: Thu, 10 Jun 2010 17:19:37 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 MIME-Version: 1.0 To: Fernando Gleiser References: <384709.32358.qm@web31706.mail.mud.yahoo.com> In-Reply-To: <384709.32358.qm@web31706.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 X-Mailman-Approved-At: Fri, 11 Jun 2010 01:27:26 +0000 Cc: virtualization@freebsd.org Subject: Re: Network stack virtualization docs? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 00:19:16 -0000 On 6/10/10 3:07 PM, Fernando Gleiser wrote: > I'm looking for some technical docs about FreeBSD's network stac virt. I saw a paper from the wiki and little else. The paper seems to focus mostly on the paralielization of the network stack more than the framework for a virtualized network. > > I want to compare it against OpenSolaris' crossbow, which has a way to virtualize almost anything: virtual nics, virtual switches and virtual wires, so you can build a "wan in a box". Is there anything like that for FreeBSD? any pointers to good docs would be apreciated > > > Fer > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" firstly you should probably post to virtualization@freebsd.org (CC'd) there have been various papers presented since 2003 on the topic but they are probably all out of date. the mechanism currently used is descripbed a bit in the 'how to virtualize a module' document. click on the 'download' link here: http://p4db.freebsd.org/fileViewer.cgi?FSPC=//depot/projects/vimage/porting_to_vimage.txt&REV=18 It describes how things are virtualized. once you have read that come back to this list with questions. From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 04:28:59 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73F681065673 for ; Fri, 11 Jun 2010 04:28:59 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 33F7C8FC15 for ; Fri, 11 Jun 2010 04:28:58 +0000 (UTC) Received: by iwn7 with SMTP id 7so940017iwn.13 for ; Thu, 10 Jun 2010 21:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=eSEYJ7q+aQy6ni34FSp911YqcgpF6WBmELx/4OWN4XQ=; b=XVkYPlJt0tgWoTQILSGiDucqcmQB6WEzVStToR+Bbjpg9e5sXC5LsG0B51XVE8m596 T1qBIr5QH7GdN/Y7GY4NuSYXm6ILO2iRSr5o1iP4AbxkPkbkfad3lEGVC0Z/knrtdyii qX3ToRjIv6TTdjWwGJo0RQMf69mKBB2XU9wNk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=nA1P9tYPi5Ba4XIvMkzvmSIOCzp3skTXCcU7iznYIfUguKGkgOW6WvjRKV5qa44N5V HHzg0RTWCQIQ2K723GXwwpJQRWhomL/39xCFnegE4MCi9UF25qol13Ene5R+ouxpSSbe 7QEcZRR2t7CMDL407KnfeMuP+HE1AjDpe1tmM= MIME-Version: 1.0 Received: by 10.231.157.205 with SMTP id c13mr1248977ibx.53.1276230538331; Thu, 10 Jun 2010 21:28:58 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.231.156.76 with HTTP; Thu, 10 Jun 2010 21:28:58 -0700 (PDT) In-Reply-To: <20100610160109.19585782fyr3buw4@webmail.leidinger.net> References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> <20100610101801.742fac25@ernst.jennejohn.org> <20100610160109.19585782fyr3buw4@webmail.leidinger.net> Date: Fri, 11 Jun 2010 12:28:58 +0800 X-Google-Sender-Auth: PthJ03921zmJt0kenjAioawZiO4 Message-ID: From: Adrian Chadd To: Alexander Leidinger Content-Type: text/plain; charset=ISO-8859-1 Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Ilya Bakulin , Garrett Cooper Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 04:28:59 -0000 How about exposing a simple userspace API for doing this, rather than doing it via sysctl? That way you could "simply" tie alternative overrides in as needed for builds (eg, environment variables setting overrides; and/or pointing to a configuration file with such) but not affect any runtime detection the rest of the system is doing. Adrian From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 08:00:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B28481065675; Fri, 11 Jun 2010 08:00:36 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2001:470:9a47::1]) by mx1.freebsd.org (Postfix) with ESMTP id 2938D8FC0C; Fri, 11 Jun 2010 08:00:35 +0000 (UTC) Received: from acme.spoerlein.net (localhost.spoerlein.net [IPv6:::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id o5B80Y0T025325 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 11 Jun 2010 10:00:35 +0200 (CEST) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1276243235; bh=BOZa3FCsMu3jCe2N0yzsT3wOprj98w69zzR9F9BWLQA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=pqychLypwQnSxqm9U72SOcOiHbpeayVfvnVNJgALOhLI9edX/pmnB+9yoFWnj8d+A GQBp3cCYybFa/PREgf6Ii1l2FHN+USY4JF33JU9aYjU3Sp3kRzo7CL1TuYCjtIstb5 /x94Gj+BjQYFF0ieDxP3M9Ohy47/ocNh5zHfFHKg= Received: (from uqs@localhost) by acme.spoerlein.net (8.14.4/8.14.4/Submit) id o5B80YqV025324; Fri, 11 Jun 2010 10:00:34 +0200 (CEST) (envelope-from uqs@spoerlein.net) Date: Fri, 11 Jun 2010 10:00:34 +0200 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Romain =?utf-8?B?VGFydGnDqHJl?= Message-ID: <20100611080034.GL39829@acme.spoerlein.net> Mail-Followup-To: Romain =?utf-8?B?VGFydGnDqHJl?= , Ivan Voras , freebsd-hackers@freebsd.org References: <20100610132652.GA54859@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20100610132652.GA54859@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-hackers@freebsd.org, Ivan Voras Subject: Re: Proposal - increase SYSV SEMMNI and SEMMNS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 08:00:36 -0000 On Thu, 10.06.2010 at 15:26:52 +0200, Romain Tartière wrote: > On Thu, Jun 10, 2010 at 01:35:47PM +0200, Ivan Voras wrote: > > kern.ipc.semmni: 10 -> 50 > > kern.ipc.semmns: 60 -> 300 > > Looks like a good idea. > > > This change is not as critical as before [...] but increasing the > > defaults would be a convenience move, to save some > > reconfiguration-and-reboot hassle when installing software. > > In the same vein, and while they can be tuned without a reboot, some ports > require to increase kern.ipc.shmmax and kern.ipc.shmall for them to work > correctly (e.g. multimedia/totem). > > Maybe all these 'convenience increases of IPC related defaults' can be > done together? Please just do it. We should ship with sensible defaults and not require tinkering for these sorts of things. I'm not sure if these defaults are already per-arch, but the embedded folks might want to retain the low defaults? Uli From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 09:08:35 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 639721065678 for ; Fri, 11 Jun 2010 09:08:35 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id F1D678FC18 for ; Fri, 11 Jun 2010 09:08:34 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FC95.dip.t-dialin.net [217.84.252.149]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 4790784400A; Fri, 11 Jun 2010 11:08:31 +0200 (CEST) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 1B39B51EF; Fri, 11 Jun 2010 11:08:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1276247308; bh=ESYvyCrmvNF8fhUwqQmmY3jf2mun8ksn70Zx/3UvZrw=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=0xqf1YMPDHSIx2r5R0E3Ze0weXgz1fLRYIYYTINZ19H+2MpvO+8Xrjz6tu8WmbO4G J8fFVl1pSB4f+XHHEH/u3FRgch14g+5RZXa2OSFfTeLjhaM9OWU+6yJ5MAeObR14EK kCQglK/AO1zZbuN8Uh5jfls+K3KP3hddSj7QsQfqlNwuzJ8R6G4fPEQTZdfkYP54Qp Wkrf1y9/bTFLshIcaNZFRbJqew0cRz7el9Na81IXXY26Sjihu7LC28jBXsnFchCIS2 CHszW3sRqBpO/UKq4l3S6TsguVmcmZauBSj5FVefoRpaMHME8iw+tpJAAn5tOgU1x9 dLia/lcy0c4jQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.4/8.13.8/Submit) id o5B98RaY065406; Fri, 11 Jun 2010 11:08:27 +0200 (CEST) (envelope-from Alexander@Leidinger.net) Received: from pslux.ec.europa.eu (pslux.ec.europa.eu [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 11 Jun 2010 11:08:27 +0200 Message-ID: <20100611110827.94483dvv07ske0ao@webmail.leidinger.net> Date: Fri, 11 Jun 2010 11:08:27 +0200 From: Alexander Leidinger To: Adrian Chadd References: <20100609121453.095d92b4@kibab.com> <4C0F9394.9030202@dataix.net> <20100609132543.GI83316@deviant.kiev.zoral.com.ua> <20100610101801.742fac25@ernst.jennejohn.org> <20100610160109.19585782fyr3buw4@webmail.leidinger.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 4790784400A.A6A85 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.1, required 6, autolearn=disabled, ALL_TRUSTED -1.00, DKIM_SIGNED 0.10, DKIM_VALID -0.10, DKIM_VALID_AU -0.10) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1276852112.72435@CDTtea5FgK2PSJANK4i97g X-EBL-Spam-Status: No X-Mailman-Approved-At: Fri, 11 Jun 2010 11:10:16 +0000 Cc: Kostik Belousov , freebsd-hackers@freebsd.org, Garrett Cooper , Ilya Bakulin Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 09:08:35 -0000 Quoting Adrian Chadd (from Fri, 11 Jun 2010 12:28:58 +0800): > How about exposing a simple userspace API for doing this, rather than > doing it via sysctl? > > That way you could "simply" tie alternative overrides in as needed for > builds (eg, environment variables setting overrides; and/or pointing > to a configuration file with such) but not affect any runtime > detection the rest of the system is doing. There is a framework (ok, one macro: FEATURE()) for the kern.features.X sysctl's. This exposes the features in sysctl when they are there. The goals of the GSoC project are to add more FEATURE()'s to the kernel, and to develop a way to spoof features. A simple way of doing the spoofing part would be some bsd.XXX.mk for ports so that you can maybe set an env-variable to spoof features. This way ports can have a look at it at build-time. This is not an option if you want to know if a feature is there at run-time (spoofing-off a feature just hides the sysctl, it does not disable the feature or prevents the use of it, it is just an administrative way of telling "please respect my wish, do not use XXX", and as such we wheren't able to come up with an use for spoof-on). Using an userland program -- maybe "featurectl" or "ftctl" or whatever -- does not hide the sysctl's, so any program which decides to use the sysctl's instead, will still see the administratively hidden features. If you want to make features (sysctl) hidden in a jail (not from within the jail but from outside the jail), you have to do something in the kernel anyway, so you do not really need an additional userland program (it's not a problem with sysctl to do it, the question is if spoof-on can only cause harm or not). So far I've seen only responses which I would describe as: - "rumors are there are some ports that maybe could use this" - "I do not have an answer for you, but maybe you could do X" Thank you to all such answers, but as this is not some just-for-fun project (Google is paying money to the students for their work), I will tell Ilya to not care about spoof-on, if nobody is showing us a specific example of where spoof-on would make sense (a port where this makes sense would be the best way, an hypotetical example will have to pass a likelyness-analysis and an are-there-good-alternatives-check). As the GSoC is also having a deadline, I will set the deadline for providing such ports/examples to the end of this month. Bye, Alexander. -- You are only young once, but you can stay immature indefinitely. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 10:06:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 724EB1065672; Fri, 11 Jun 2010 10:06:36 +0000 (UTC) (envelope-from mail25@bzerk.org) Received: from ei.bzerk.org (tunnel490.ipv6.xs4all.nl [IPv6:2001:888:10:1ea::2]) by mx1.freebsd.org (Postfix) with ESMTP id D8B048FC1D; Fri, 11 Jun 2010 10:06:35 +0000 (UTC) Received: from ei.bzerk.org (BOFH@localhost [127.0.0.1]) by ei.bzerk.org (8.14.3/8.14.3) with ESMTP id o5BA6UCK090227; Fri, 11 Jun 2010 12:06:30 +0200 (CEST) (envelope-from mail25@bzerk.org) Received: (from bulk@localhost) by ei.bzerk.org (8.14.3/8.14.3/Submit) id o5BA6UCV090226; Fri, 11 Jun 2010 12:06:30 +0200 (CEST) (envelope-from mail25@bzerk.org) Date: Fri, 11 Jun 2010 12:06:29 +0200 From: Ruben de Groot To: Romain Tarti??re , Ivan Voras , freebsd-hackers@freebsd.org Message-ID: <20100611100629.GA89804@ei.bzerk.org> Mail-Followup-To: Ruben de Groot , Romain Tarti??re , Ivan Voras , freebsd-hackers@freebsd.org References: <20100610132652.GA54859@FreeBSD.org> <20100611080034.GL39829@acme.spoerlein.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100611080034.GL39829@acme.spoerlein.net> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on ei.bzerk.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.0.1 (ei.bzerk.org [127.0.0.1]); Fri, 11 Jun 2010 12:06:34 +0200 (CEST) X-Mailman-Approved-At: Fri, 11 Jun 2010 11:10:31 +0000 Cc: Subject: Re: Proposal - increase SYSV SEMMNI and SEMMNS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 10:06:36 -0000 On Fri, Jun 11, 2010 at 10:00:34AM +0200, Ulrich Sp??rlein typed: > On Thu, 10.06.2010 at 15:26:52 +0200, Romain Tarti??re wrote: > > On Thu, Jun 10, 2010 at 01:35:47PM +0200, Ivan Voras wrote: > > > kern.ipc.semmni: 10 -> 50 > > > kern.ipc.semmns: 60 -> 300 > > > > Looks like a good idea. > > > > > This change is not as critical as before [...] but increasing the > > > defaults would be a convenience move, to save some > > > reconfiguration-and-reboot hassle when installing software. > > > > In the same vein, and while they can be tuned without a reboot, some ports > > require to increase kern.ipc.shmmax and kern.ipc.shmall for them to work > > correctly (e.g. multimedia/totem). > > > > Maybe all these 'convenience increases of IPC related defaults' can be > > done together? > > Please just do it. We should ship with sensible defaults and not > require tinkering for these sorts of things. I'm not sure if these > defaults are already per-arch, but the embedded folks might want to > retain the low defaults? Definately! And I also think application based tuning (postgres, mono) should be done per application, not by default. Ruben From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 11 12:25:03 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C399A106567B; Fri, 11 Jun 2010 12:25:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 9130D8FC1A; Fri, 11 Jun 2010 12:25:03 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 429C946C13; Fri, 11 Jun 2010 08:25:03 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 798388A04F; Fri, 11 Jun 2010 08:25:02 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Fri, 11 Jun 2010 07:56:10 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100609121453.095d92b4@kibab.com> <20100610160109.19585782fyr3buw4@webmail.leidinger.net> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201006110756.10287.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 11 Jun 2010 08:25:02 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Kostik Belousov , Alexander Leidinger , Adrian Chadd , Garrett Cooper , Ilya Bakulin Subject: Re: GSoC: registration of optional kernel features via sysctl: a question to the community X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jun 2010 12:25:03 -0000 On Friday 11 June 2010 12:28:58 am Adrian Chadd wrote: > How about exposing a simple userspace API for doing this, rather than > doing it via sysctl? > > That way you could "simply" tie alternative overrides in as needed for > builds (eg, environment variables setting overrides; and/or pointing > to a configuration file with such) but not affect any runtime > detection the rest of the system is doing. man 3 feature_present I don't see much practical use for spoofing. If there really was a desire for such a thing, then perhaps you could implement it in libc keyed off environment variables. It would perhaps be useful to have a feature_present(1) for ports that uses the API instead of using the sysctls directly, but I really don't think there are any useful cases for spoofing. At Y! we used kern.features.pae in kmod.mk to automatically enable PAE in standalone kernel module builds if the running kernel was using PAE since PAE alters the kernel ABI. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 01:17:04 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67764106566C; Sat, 12 Jun 2010 01:17:04 +0000 (UTC) (envelope-from eng.mufic@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 75BEF8FC14; Sat, 12 Jun 2010 01:17:03 +0000 (UTC) Received: by wyb33 with SMTP id 33so1716734wyb.13 for ; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=peLaGOkkf5eQAmDVAFzErJAH8FejGLUZZZBy02yI1c4=; b=Yjt3im6rL10UvlgEpsT5ziZ6zt3L7eIWqoJqXk3FwfSXFaMYOXNTrbPCZhLVPjuMxu 3vTFHbd31dWuJXuxMz/1vWPnWJCZXbA1mKskWTec2nCoFXZd4W8d9K3KMAnpcb5cZiqD mR0reI7RR+GMVvcoIxy8432FcC84LtoxqUG+o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=wLBwR964LhuqL1+UFwF7QCGJawW/bEfDyuCs1VJdbmOqm/4tUzc3BPUxucH9LnVSIi ehu/C/P+suomc+0Vr3qELRnLEZ2khQ4cq0VyT58e6kgpEdui0igtxwi4AOJLEDpoPvDs +dlb0Q0E6BCgpYENi56Qp4JVm1si6UZn0L3AI= MIME-Version: 1.0 Received: by 10.216.88.73 with SMTP id z51mr121574wee.27.1276305422047; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) Sender: eng.mufic@gmail.com Received: by 10.216.13.6 with HTTP; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) Date: Sat, 12 Jun 2010 04:17:02 +0300 X-Google-Sender-Auth: 06JQQG-3su7cZI4nAPmZxdJMvfs Message-ID: From: Mohammed Farrag To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org, freebsd-current@freebsd.org Subject: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 01:17:04 -0000 Hi, I hope that I could get some comments or suggestions about the work I did in developing a new Embedded FreeBSD Kernel Theme. I will be glad to see your replies. download the work file from here http://www.4shared.com/document/U237Abgs/description.html With best wishes, Mohammed Farrag From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 01:17:04 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67764106566C; Sat, 12 Jun 2010 01:17:04 +0000 (UTC) (envelope-from eng.mufic@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 75BEF8FC14; Sat, 12 Jun 2010 01:17:03 +0000 (UTC) Received: by wyb33 with SMTP id 33so1716734wyb.13 for ; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=peLaGOkkf5eQAmDVAFzErJAH8FejGLUZZZBy02yI1c4=; b=Yjt3im6rL10UvlgEpsT5ziZ6zt3L7eIWqoJqXk3FwfSXFaMYOXNTrbPCZhLVPjuMxu 3vTFHbd31dWuJXuxMz/1vWPnWJCZXbA1mKskWTec2nCoFXZd4W8d9K3KMAnpcb5cZiqD mR0reI7RR+GMVvcoIxy8432FcC84LtoxqUG+o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; b=wLBwR964LhuqL1+UFwF7QCGJawW/bEfDyuCs1VJdbmOqm/4tUzc3BPUxucH9LnVSIi ehu/C/P+suomc+0Vr3qELRnLEZ2khQ4cq0VyT58e6kgpEdui0igtxwi4AOJLEDpoPvDs +dlb0Q0E6BCgpYENi56Qp4JVm1si6UZn0L3AI= MIME-Version: 1.0 Received: by 10.216.88.73 with SMTP id z51mr121574wee.27.1276305422047; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) Sender: eng.mufic@gmail.com Received: by 10.216.13.6 with HTTP; Fri, 11 Jun 2010 18:17:02 -0700 (PDT) Date: Sat, 12 Jun 2010 04:17:02 +0300 X-Google-Sender-Auth: 06JQQG-3su7cZI4nAPmZxdJMvfs Message-ID: From: Mohammed Farrag To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: hackers@freebsd.org, freebsd-current@freebsd.org Subject: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 01:17:04 -0000 Hi, I hope that I could get some comments or suggestions about the work I did in developing a new Embedded FreeBSD Kernel Theme. I will be glad to see your replies. download the work file from here http://www.4shared.com/document/U237Abgs/description.html With best wishes, Mohammed Farrag From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 02:58:34 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8959F1065674; Sat, 12 Jun 2010 02:58:34 +0000 (UTC) (envelope-from chargen@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AAA6F8FC14; Sat, 12 Jun 2010 02:58:33 +0000 (UTC) Received: by bwz2 with SMTP id 2so990798bwz.13 for ; Fri, 11 Jun 2010 19:58:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=i8R1KympWNHCH/hYA+14xO7Zu8nH8//SX+Et+c0Z0yc=; b=wK4GeLzpMXuH31K6J3YANYYaN+ZCD3WPAO3I+1LZlurNeydHGoUMgckHY+lx60eNYE hz175XEleL9hwYiPdphvsnPnbXuif97JZAPOOaoeo50m28YcF/nR+LxFl7APkA1EHH3m mQJeMH3p8fUWozgLCBRMVGIwOEeH2rWn8GffE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=v/G0H6mzRUoay14mADi14eIb+c3sF4p0Pmo76BizLKeTTLPNn+gZnpYQCkRS0dxxtq yBiTnPLkw2N3s/eDOih7SbWRrSRQf68dHcH/0dbOVIjlQRHCrWBcNBZmMyLoD2aE2CE6 7Za+4FICT9fDSod+sAn6DmvsLqdtWG3w1xfWc= Received: by 10.204.160.145 with SMTP id n17mr1903145bkx.67.1276309984175; Fri, 11 Jun 2010 19:33:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.134.65 with HTTP; Fri, 11 Jun 2010 19:32:44 -0700 (PDT) In-Reply-To: References: From: Chargen Date: Sat, 12 Jun 2010 04:32:44 +0200 Message-ID: To: Mohammed Farrag X-Mailman-Approved-At: Sat, 12 Jun 2010 04:20:00 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 02:58:34 -0000 Sir, first of all: I'm no expert in this field but I've seen your document and I'm still wondering why you should impose such a design. I suppose it's well thought of but still I'm a bit opposed to binary configuration files because I think has to be some kind of dependency on how to generate these kind of files (correct me if I'm wrong?) as far as your document goes: "We will unload all the drivers that indicated with zeros in the module metadata file. That would make the OS to be a few of Megabytes." unload? what is the logic here? I'm sorry but what is the real gain here, can you please elaborate? On Sat, Jun 12, 2010 at 03:17, Mohammed Farrag wrote: > Hi, > I hope that I could get some comments or suggestions about the work I did > in developing a new Embedded FreeBSD Kernel Theme. I will be glad to see > your replies. > download the work file from here > http://www.4shared.com/document/U237Abgs/description.html > With best wishes, > Mohammed Farrag > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 02:58:34 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8959F1065674; Sat, 12 Jun 2010 02:58:34 +0000 (UTC) (envelope-from chargen@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id AAA6F8FC14; Sat, 12 Jun 2010 02:58:33 +0000 (UTC) Received: by bwz2 with SMTP id 2so990798bwz.13 for ; Fri, 11 Jun 2010 19:58:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type; bh=i8R1KympWNHCH/hYA+14xO7Zu8nH8//SX+Et+c0Z0yc=; b=wK4GeLzpMXuH31K6J3YANYYaN+ZCD3WPAO3I+1LZlurNeydHGoUMgckHY+lx60eNYE hz175XEleL9hwYiPdphvsnPnbXuif97JZAPOOaoeo50m28YcF/nR+LxFl7APkA1EHH3m mQJeMH3p8fUWozgLCBRMVGIwOEeH2rWn8GffE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; b=v/G0H6mzRUoay14mADi14eIb+c3sF4p0Pmo76BizLKeTTLPNn+gZnpYQCkRS0dxxtq yBiTnPLkw2N3s/eDOih7SbWRrSRQf68dHcH/0dbOVIjlQRHCrWBcNBZmMyLoD2aE2CE6 7Za+4FICT9fDSod+sAn6DmvsLqdtWG3w1xfWc= Received: by 10.204.160.145 with SMTP id n17mr1903145bkx.67.1276309984175; Fri, 11 Jun 2010 19:33:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.134.65 with HTTP; Fri, 11 Jun 2010 19:32:44 -0700 (PDT) In-Reply-To: References: From: Chargen Date: Sat, 12 Jun 2010 04:32:44 +0200 Message-ID: To: Mohammed Farrag X-Mailman-Approved-At: Sat, 12 Jun 2010 04:20:12 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org, freebsd-current@freebsd.org Subject: Re: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 02:58:34 -0000 Sir, first of all: I'm no expert in this field but I've seen your document and I'm still wondering why you should impose such a design. I suppose it's well thought of but still I'm a bit opposed to binary configuration files because I think has to be some kind of dependency on how to generate these kind of files (correct me if I'm wrong?) as far as your document goes: "We will unload all the drivers that indicated with zeros in the module metadata file. That would make the OS to be a few of Megabytes." unload? what is the logic here? I'm sorry but what is the real gain here, can you please elaborate? On Sat, Jun 12, 2010 at 03:17, Mohammed Farrag wrote: > Hi, > I hope that I could get some comments or suggestions about the work I did > in developing a new Embedded FreeBSD Kernel Theme. I will be glad to see > your replies. > download the work file from here > http://www.4shared.com/document/U237Abgs/description.html > With best wishes, > Mohammed Farrag > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 04:15:36 2010 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0D291065672; Sat, 12 Jun 2010 04:15:36 +0000 (UTC) (envelope-from matt.thyer@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 25E868FC18; Sat, 12 Jun 2010 04:15:35 +0000 (UTC) Received: by vws20 with SMTP id 20so1119562vws.13 for ; Fri, 11 Jun 2010 21:15:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=LN3NfMhtWvuLfehKHAAnXVpwzU9semH9KRkVgwk6EEY=; b=Ab4rThIkLo4zS6Zdhmy+dFvdw7ftTS6uvFAYmpOLIX4wEHVRQTLA14fS6PYXreg8ON N8L/httuqvR6YVuL36v/eN+k7BuEvS9xzGGxN2RmCaoNZTpaakqMc3bRwYFqwsK8Fjqv RNkuzMRL3vmqavEtDjsHKbi1Km4DOrMr3SKkY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=kClJ3JLdiUqpL/USkD0zvUdFnbCI1JHDsxdkaHzYVjBZW9WugL4QQyAK+1DbW4bUxM IfkfHv6zg0zAQJNFsnTjJxnVxZePrtqGtoTb5R7ZJQOawhtPDqy5i3m3w6hr2TaqO6Av xd25AfzCeNWJ2HDeIOkfus3XbbZ93dns1E1Do= MIME-Version: 1.0 Received: by 10.220.123.9 with SMTP id n9mr239921vcr.269.1276314562598; Fri, 11 Jun 2010 20:49:22 -0700 (PDT) Received: by 10.220.194.4 with HTTP; Fri, 11 Jun 2010 20:49:22 -0700 (PDT) In-Reply-To: References: Date: Sat, 12 Jun 2010 13:19:22 +0930 Message-ID: From: Matt Thyer To: Chargen Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 12 Jun 2010 04:25:56 +0000 Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org, Mohammed Farrag , freebsd-current@freebsd.org Subject: Re: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 04:15:36 -0000 On 12 June 2010 12:02, Chargen wrote: > Sir, first of all: I'm no expert in this field but I've seen your document > and I'm still wondering why you should impose such a design. > > I suppose it's well thought of but still I'm a bit opposed to binary > configuration files because I think has to be some kind of dependency on how > to generate these kind of files (correct me if I'm wrong?) > > as far as your document goes: > "We will unload all the drivers that indicated with zeros in the module > metadata file. That would make the OS to be a few of Megabytes." > > unload? what is the logic here? > > I'm sorry but what is the real gain here, > > can you please elaborate? > FreeBSD is already a very modular system and the traditional way (a traditional way) to build for embedded systems is to follow the NanoBSD build method (tools included in the source tree) with a stripped down kernel in which you only load the modules your hardware requires using the FreeBSD loader (or after the initial boot). However my Soekris net4801 board still takes about 2.5 minutes to boot and I think time could be saved by parallel probing of hardware where possible. Much work has been done on fast boot times in the Linux world including an impressive demonstration by an Intel team for car instrumentation panels (on Youtube... Google for fastest Linux boot). I'd vote for more work on FreeBSD's existing boot method rather than an entirely new implementation. What problem are you trying to solve Mohammed ? From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 04:15:36 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0D291065672; Sat, 12 Jun 2010 04:15:36 +0000 (UTC) (envelope-from matt.thyer@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 25E868FC18; Sat, 12 Jun 2010 04:15:35 +0000 (UTC) Received: by vws20 with SMTP id 20so1119562vws.13 for ; Fri, 11 Jun 2010 21:15:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=LN3NfMhtWvuLfehKHAAnXVpwzU9semH9KRkVgwk6EEY=; b=Ab4rThIkLo4zS6Zdhmy+dFvdw7ftTS6uvFAYmpOLIX4wEHVRQTLA14fS6PYXreg8ON N8L/httuqvR6YVuL36v/eN+k7BuEvS9xzGGxN2RmCaoNZTpaakqMc3bRwYFqwsK8Fjqv RNkuzMRL3vmqavEtDjsHKbi1Km4DOrMr3SKkY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=kClJ3JLdiUqpL/USkD0zvUdFnbCI1JHDsxdkaHzYVjBZW9WugL4QQyAK+1DbW4bUxM IfkfHv6zg0zAQJNFsnTjJxnVxZePrtqGtoTb5R7ZJQOawhtPDqy5i3m3w6hr2TaqO6Av xd25AfzCeNWJ2HDeIOkfus3XbbZ93dns1E1Do= MIME-Version: 1.0 Received: by 10.220.123.9 with SMTP id n9mr239921vcr.269.1276314562598; Fri, 11 Jun 2010 20:49:22 -0700 (PDT) Received: by 10.220.194.4 with HTTP; Fri, 11 Jun 2010 20:49:22 -0700 (PDT) In-Reply-To: References: Date: Sat, 12 Jun 2010 13:19:22 +0930 Message-ID: From: Matt Thyer To: Chargen Content-Type: text/plain; charset=ISO-8859-1 X-Mailman-Approved-At: Sat, 12 Jun 2010 04:26:05 +0000 Cc: freebsd-hackers@freebsd.org, hackers@freebsd.org, Mohammed Farrag , freebsd-current@freebsd.org Subject: Re: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 04:15:36 -0000 On 12 June 2010 12:02, Chargen wrote: > Sir, first of all: I'm no expert in this field but I've seen your document > and I'm still wondering why you should impose such a design. > > I suppose it's well thought of but still I'm a bit opposed to binary > configuration files because I think has to be some kind of dependency on how > to generate these kind of files (correct me if I'm wrong?) > > as far as your document goes: > "We will unload all the drivers that indicated with zeros in the module > metadata file. That would make the OS to be a few of Megabytes." > > unload? what is the logic here? > > I'm sorry but what is the real gain here, > > can you please elaborate? > FreeBSD is already a very modular system and the traditional way (a traditional way) to build for embedded systems is to follow the NanoBSD build method (tools included in the source tree) with a stripped down kernel in which you only load the modules your hardware requires using the FreeBSD loader (or after the initial boot). However my Soekris net4801 board still takes about 2.5 minutes to boot and I think time could be saved by parallel probing of hardware where possible. Much work has been done on fast boot times in the Linux world including an impressive demonstration by an Intel team for car instrumentation panels (on Youtube... Google for fastest Linux boot). I'd vote for more work on FreeBSD's existing boot method rather than an entirely new implementation. What problem are you trying to solve Mohammed ? From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 09:00:19 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B73091065672; Sat, 12 Jun 2010 09:00:19 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [204.109.60.94]) by mx1.freebsd.org (Postfix) with ESMTP id 74DDD8FC08; Sat, 12 Jun 2010 09:00:19 +0000 (UTC) Received: from unknown (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id 1C9E85C03; Sat, 12 Jun 2010 09:00:31 +0000 (UTC) Date: Sat, 12 Jun 2010 10:00:06 +0100 From: Bruce Cran To: Matt Thyer Message-ID: <20100612100006.000073fa@unknown> In-Reply-To: References: X-Mailer: Claws Mail 3.7.4cvs1 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Chargen , freebsd-hackers@freebsd.org, Farrag , Mohammed, freebsd-current@freebsd.org Subject: Re: I need reply in Embedded FreeBSD Kernel Theme X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 09:00:19 -0000 On Sat, 12 Jun 2010 13:19:22 +0930 Matt Thyer wrote: > However my Soekris net4801 board still takes about 2.5 minutes to boot > and I think time could be saved by parallel probing of hardware where > possible. > > Much work has been done on fast boot times in the Linux world > including an impressive demonstration by an Intel team for car > instrumentation panels (on Youtube... Google for fastest Linux boot). It's on the list of ideas for 9.0: see http://wiki.freebsd.org/FreeBSD9#head-960c3f5a8747af95199367a8c84030dfe2b99f1b -- Bruce Cran From owner-freebsd-hackers@FreeBSD.ORG Sat Jun 12 15:36:43 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 53B5E1065673 for ; Sat, 12 Jun 2010 15:36:43 +0000 (UTC) (envelope-from tdamas@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id F0A808FC13 for ; Sat, 12 Jun 2010 15:36:42 +0000 (UTC) Received: by vws20 with SMTP id 20so1899792vws.13 for ; Sat, 12 Jun 2010 08:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:from:date :message-id:subject:to:content-type; bh=8aR6iS7cNYk/6dHzll5JZJgwL4tCzMoRSsACdXH0a+0=; b=I+8OmoHOzgOCfIq6DMUEEquwvVxKo8pGjTGwuQIHsD3QOtbWW4kb6ZRc6zFUtL6IyG 7JZrcNmocA+qwhmH7GniTPJS8wEJmnsJcP+gH3EO1Ae2ilFUR3uM/zYNJv+gbwiOntqY YEUcEPImI4eLfgJ/6xOz+noZ1pL9bJ7Hgsyxs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=ZcVIz48+6gvw+mW+tO3RdGDORw/VPEzHsgKzMEsDo85pEy6yN3tTU6ZKNOvYh0lzyt aWGgpoKdLH9l6hQmsu8YoAULQu3uoE5MwMDBkERqkU0n0xOVWvk3ktjtd0xe8NYkF6QP QfGz8JEzRy7g2NV97d2QSG0NjvjMKO1iGdtaU= Received: by 10.220.127.2 with SMTP id e2mr761186vcs.137.1276357002109; Sat, 12 Jun 2010 08:36:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.220.192.73 with HTTP; Sat, 12 Jun 2010 08:36:22 -0700 (PDT) From: Thiago Damas Date: Sat, 12 Jun 2010 12:36:22 -0300 Message-ID: To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: strange zfs behavior X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jun 2010 15:36:43 -0000 Hi, I'm testing some configuration using ZFS with 4 disks seagate: ad4: 953869MB at ata2-master UDMA100 SATA 3Gb/s ad6: 953869MB at ata3-master UDMA100 SATA 3Gb/s ad8: 953869MB at ata4-master UDMA100 SATA 3Gb/s ad10: 953869MB at ata5-master UDMA100 SATA 3Gb/s The system its amd64 8.1-BETA1 (tested too in 8.0-p3). My only tuning its (in /boot/loader.conf): vm.kmem_size_scale="2" vfs.zfs.txg.timeout=5 The machine has 4Gb RAM, and SATA controller its LSI53C1020/1030 (adaptec 1020) At first, I used the following: zpool create -f -m /storage tank mirror /dev/ad4 /dev/ad6 mirror /dev/ad8 /dev/ad10 and I noticed ad10 slower than others (svc_t) svc_t: http://i48.tinypic.com/34s1ndd.gif http://i45.tinypic.com/m9x6ra.gif wait: http://i47.tinypic.com/2uqksv5.gif http://i49.tinypic.com/200qza9.gif Now, I swapped the configuration: zpool create -f -m /storage tank mirror /dev/ad10 /dev/ad8 mirror /dev/ad6 /dev/ad4 and now I have ad4 slower than others svc_t: http://i49.tinypic.com/2uxtqww.gif http://i50.tinypic.com/10dbcix.gif wait: http://i46.tinypic.com/331f5lf.gif http://i46.tinypic.com/2lc7c5k.gif Will always the last disk in zfs configuration perform like that? Any comments?