From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 14:04:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 405D11065673; Sun, 21 Mar 2010 14:04:13 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by mx1.freebsd.org (Postfix) with ESMTP id 8E77E8FC15; Sun, 21 Mar 2010 14:04:12 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so157839fga.13 for ; Sun, 21 Mar 2010 07:04:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=PrrFva0kbQYNEA6MUrI01eUuqXd4VQBIKirXR/fKBj4=; b=d3VbPi3SyUt/Q7szCjvpA+EBTqdOpWJtRKCj0TxnUfoUgX53r4MSnEweoStSnvXRAJ 05iTulqA0CyEWNaq2qvhdLUMJkrPd2rrK//TAL/4qfel0zlPhre05ILXoez4TXbJKp+Q l3hFfc9kCgOgBRpjAnyAjjwgwveSeK74Xa2v4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=njKgfKaDyDThztxFPemi0xSFgbR2fLi5MWzlnEsckerPTzlYj6CgrPBMc91NvcPdU2 wrzL01tQLdlBj9hnNP7+2DoYhMELKQpo3UZWyFc9KAVjLSPjHRKtHugOyPVuVLZZl9QW MjzjPhXVU7wPII9xEnvg2n3IcgtOXuJCVrdak= Received: by 10.87.62.1 with SMTP id p1mr4192349fgk.42.1269180251307; Sun, 21 Mar 2010 07:04:11 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 16sm920679fxm.0.2010.03.21.07.04.10 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 07:04:10 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA62757.7090400@FreeBSD.org> Date: Sun, 21 Mar 2010 16:04:07 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Julian Elischer References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> In-Reply-To: <4BA532FF.6040407@elischer.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Scott Long , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 14:04:13 -0000 Julian Elischer wrote: > In the Fusion-io driver we find that the limiting factor is not the > size of MAXPHYS, but the fact that we can not push more than > 170k tps through geom. (in my test machine. I've seen more on some > beefier machines), but that is only a limit on small transacrtions, > or in the case of large transfers the DMA engine tops out before a > bigger MAXPHYS would make any difference. Yes, GEOM is quite CPU-hungry on high request rates due to number of context switches. But impact probably may be reduced from two sides: by reducing overhead per request, or by reducing number of requests. Both ways may give benefits. If common opinion is not to touch defaults now - OK, agreed. (Note, Scott, I have agreed :)) But returning to the original question, does somebody knows real situation when increased MAXPHYS still causes problems? At least to make it safe. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 14:05:23 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B060E1065679; Sun, 21 Mar 2010 14:05:23 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.154]) by mx1.freebsd.org (Postfix) with ESMTP id DEE0B8FC24; Sun, 21 Mar 2010 14:05:22 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so158042fga.13 for ; Sun, 21 Mar 2010 07:05:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=BRJ41r/r91Fq55DvNWe6LznhUMxpmLSNAg11fbo4tys=; b=c7H3BUIot8BPnUVeIqVVf7kEIL8dDdTg2RDfY4J93sKY8LqtXpFWKmHOHvrFltqe3U pZLwFk8Hdtqo9lbFzkQBFzLHqkBYpoRZce88Iu0h29lTwWn4FcRpiqEAeWjNRmW8LhDc x35+Fcfg7G6l3fiWo5i8TfxJWGp8gddNpZD0c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=ofxrZSgSLrIcBX/hC8/q0gtZtPNeYhHv/QdrtnOOAtiJ599BsnD2ZTQcWI+MT+f3b/ saHVVWMwQfiScHRaHYzoce6uMewIxujxrRjnUo8ULVF3njrT2APHUYPGVTnepz9BSHom LCN97/uW2jCcRpmoU18mkJHArX0H4KBzLH7RQ= Received: by 10.87.2.15 with SMTP id e15mr1915544fgi.22.1269180321552; Sun, 21 Mar 2010 07:05:21 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 16sm921439fxm.0.2010.03.21.07.05.20 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 07:05:20 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA6279E.3010201@FreeBSD.org> Date: Sun, 21 Mar 2010 16:05:18 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Ivan Voras References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> In-Reply-To: <1269134585.00231959.1269122405@10.7.7.3> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 14:05:23 -0000 Ivan Voras wrote: > Julian Elischer wrote: >> You can get better throughput by using TSC for timing because the geom >> and devstat code does a bit of timing.. Geom can be told to turn off >> it's timing but devstat can't. The 170 ktps is with TSC as timer, >> and geom timing turned off. > > I see. I just ran randomio on a gzero device and with 10 userland > threads (this is a slow 2xquad machine) I get g_up and g_down saturated > fast with ~~ 120 ktps. Randomio uses gettimeofday() for measurements. I've just got 140Ktps from two real Intel X25-M SSDs on ICH10R AHCI controller and single Core2Quad CPU. So at least on synthetic tests it is potentially reachable even with casual hardware, while it completely saturated quad-core CPU. > Hmm, it looks like it could be easy to spawn more g_* threads (and, > barring specific class behaviour, it has a fair chance of working out of > the box) but the incoming queue will need to also be broken up for > greater effect. According to "notes", looks there is a good chance to obtain races, as some places expect only one up and one down thread. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 14:56:36 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A90AB106566B; Sun, 21 Mar 2010 14:56:36 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 464E08FC27; Sun, 21 Mar 2010 14:56:34 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA00267; Sun, 21 Mar 2010 16:56:33 +0200 (EET) (envelope-from avg@icyb.net.ua) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1NtMZt-000AgC-3A; Sun, 21 Mar 2010 16:56:33 +0200 Message-ID: <4BA633A0.2090108@icyb.net.ua> Date: Sun, 21 Mar 2010 16:56:32 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.24 (X11/20100321) MIME-Version: 1.0 To: Alexander Motin References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> In-Reply-To: <4BA6279E.3010201@FreeBSD.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 14:56:36 -0000 on 21/03/2010 16:05 Alexander Motin said the following: > Ivan Voras wrote: >> Hmm, it looks like it could be easy to spawn more g_* threads (and, >> barring specific class behaviour, it has a fair chance of working out of >> the box) but the incoming queue will need to also be broken up for >> greater effect. > > According to "notes", looks there is a good chance to obtain races, as > some places expect only one up and one down thread. I haven't given any deep thought to this issue, but I remember us discussing them over beer :-) I think one idea was making sure (somehow) that requests traveling over the same edge of a geom graph (in the same direction) do it using the same queue/thread. Another idea was to bring some netgraph-like optimization where some (carefully chosen) geom vertices pass requests by a direct call instead of requeuing. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 15:46:57 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26798106566B; Sun, 21 Mar 2010 15:46:57 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-5.mx.aerioconnect.net [216.240.47.65]) by mx1.freebsd.org (Postfix) with ESMTP id ABC138FC24; Sun, 21 Mar 2010 15:46:56 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2LFktDv015391; Sun, 21 Mar 2010 08:46:55 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 63C4B2D601C; Sun, 21 Mar 2010 08:46:55 -0700 (PDT) Message-ID: <4BA63F72.5000806@elischer.org> Date: Sun, 21 Mar 2010 08:46:58 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Alexander Motin References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA62757.7090400@FreeBSD.org> In-Reply-To: <4BA62757.7090400@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Scott Long , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 15:46:57 -0000 Alexander Motin wrote: > Julian Elischer wrote: >> In the Fusion-io driver we find that the limiting factor is not the >> size of MAXPHYS, but the fact that we can not push more than >> 170k tps through geom. (in my test machine. I've seen more on some >> beefier machines), but that is only a limit on small transacrtions, >> or in the case of large transfers the DMA engine tops out before a >> bigger MAXPHYS would make any difference. > > Yes, GEOM is quite CPU-hungry on high request rates due to number of > context switches. But impact probably may be reduced from two sides: by > reducing overhead per request, or by reducing number of requests. Both > ways may give benefits. > > If common opinion is not to touch defaults now - OK, agreed. (Note, > Scott, I have agreed :)) But returning to the original question, does > somebody knows real situation when increased MAXPHYS still causes > problems? At least to make it safe. well I know we havn't tested our bsd driver yet with MAXPHYS > 128KB at this time.. Must try that some time :-) From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 15:51:23 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2363B106566B; Sun, 21 Mar 2010 15:51:23 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-5.mx.aerioconnect.net [216.240.47.65]) by mx1.freebsd.org (Postfix) with ESMTP id E698B8FC21; Sun, 21 Mar 2010 15:51:22 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2LFpLad015549; Sun, 21 Mar 2010 08:51:22 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 748D62D6013; Sun, 21 Mar 2010 08:51:21 -0700 (PDT) Message-ID: <4BA6407C.3020103@elischer.org> Date: Sun, 21 Mar 2010 08:51:24 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Andriy Gapon References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> <4BA633A0.2090108@icyb.net.ua> In-Reply-To: <4BA633A0.2090108@icyb.net.ua> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Alexander Motin , freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 15:51:23 -0000 Andriy Gapon wrote: > on 21/03/2010 16:05 Alexander Motin said the following: >> Ivan Voras wrote: >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>> barring specific class behaviour, it has a fair chance of working out of >>> the box) but the incoming queue will need to also be broken up for >>> greater effect. >> According to "notes", looks there is a good chance to obtain races, as >> some places expect only one up and one down thread. > > I haven't given any deep thought to this issue, but I remember us discussing > them over beer :-) > I think one idea was making sure (somehow) that requests traveling over the same > edge of a geom graph (in the same direction) do it using the same queue/thread. > Another idea was to bring some netgraph-like optimization where some (carefully > chosen) geom vertices pass requests by a direct call instead of requeuing. > yeah, like the 1:1 single provider case. (which we an most of our custommers mostly use on our cards). i.e. no slicing or dicing, and just the raw flash card presented as /dev/fio0 From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:13:07 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 052F3106564A; Sun, 21 Mar 2010 16:13:07 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id A9AA88FC21; Sun, 21 Mar 2010 16:13:04 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LGD1Tn036769; Sun, 21 Mar 2010 10:13:01 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BA6279E.3010201@FreeBSD.org> Date: Sun, 21 Mar 2010 10:13:01 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> To: Alexander Motin X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:13:07 -0000 On Mar 21, 2010, at 8:05 AM, Alexander Motin wrote: > Ivan Voras wrote: >> Julian Elischer wrote: >>> You can get better throughput by using TSC for timing because the = geom >>> and devstat code does a bit of timing.. Geom can be told to turn off >>> it's timing but devstat can't. The 170 ktps is with TSC as timer, >>> and geom timing turned off. >>=20 >> I see. I just ran randomio on a gzero device and with 10 userland >> threads (this is a slow 2xquad machine) I get g_up and g_down = saturated >> fast with ~~ 120 ktps. Randomio uses gettimeofday() for measurements. >=20 > I've just got 140Ktps from two real Intel X25-M SSDs on ICH10R AHCI > controller and single Core2Quad CPU. So at least on synthetic tests it > is potentially reachable even with casual hardware, while it = completely > saturated quad-core CPU. >=20 >> Hmm, it looks like it could be easy to spawn more g_* threads (and, >> barring specific class behaviour, it has a fair chance of working out = of >> the box) but the incoming queue will need to also be broken up for >> greater effect. >=20 > According to "notes", looks there is a good chance to obtain races, as > some places expect only one up and one down thread. >=20 I agree that more threads just creates many more race complications. = Even if it didn't, the storage driver is a serialization point; it = doesn't matter if you have a dozen g_* threads if only one of them can = be in the top half of the driver at a time. No amount of fine-grained = locking is going to help this. I'd like to go in the opposite direction. The queue-dispatch-queue = model of GEOM is elegant and easy to extend, but very wasteful for the = simple case, where the simple case is one or two simple partition = transforms (mbr, bsdlabel) and/or a simple stripe/mirror transform. = None of these need a dedicated dispatch context in order to operate. = What I'd like to explore is compiling the GEOM stack at creation time = into a linear array of operations that happen without a g_down/g_up = context switch. As providers and consumers taste each other and build a = stack, that stack gets compiled into a graph, and that graph gets = executed directly from the calling context, both from the dev_strategy() = side on the top and the bio_done() on the bottom. GEOM classes that = need a detached context can mark themselves as such, doing so will = prevent a graph from being created, and the current dispatch model will = be retained. I expect that this will reduce i/o latency by a great margin, thus = directly addressing the performance problem that FusionIO makes an = example of. I'd like to also explore having the g_bio model not require = a malloc at every stage in the stack/graph; even though going through = UMA is fairly fast, it still represents overhead that can be eliminated. = It also represents an out-of-memory failure case that can be prevented. I might try to work on this over the summer. It's really a research = project in my head at this point, but I'm hopeful that it'll show = results. Scott From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:18:59 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3F20106566B; Sun, 21 Mar 2010 16:18:59 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 4F1AA8FC17; Sun, 21 Mar 2010 16:18:59 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LGIu2N036802; Sun, 21 Mar 2010 10:18:56 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BA633A0.2090108@icyb.net.ua> Date: Sun, 21 Mar 2010 10:18:56 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <082B2047-44AE-45DB-985B-D8928EBB4871@samsco.org> References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> <4BA633A0.2090108@icyb.net.ua> To: Andriy Gapon X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: Alexander Motin , freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:18:59 -0000 m On Mar 21, 2010, at 8:56 AM, Andriy Gapon wrote: > on 21/03/2010 16:05 Alexander Motin said the following: >> Ivan Voras wrote: >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>> barring specific class behaviour, it has a fair chance of working = out of >>> the box) but the incoming queue will need to also be broken up for >>> greater effect. >>=20 >> According to "notes", looks there is a good chance to obtain races, = as >> some places expect only one up and one down thread. >=20 > I haven't given any deep thought to this issue, but I remember us = discussing > them over beer :-) > I think one idea was making sure (somehow) that requests traveling = over the same > edge of a geom graph (in the same direction) do it using the same = queue/thread. > Another idea was to bring some netgraph-like optimization where some = (carefully > chosen) geom vertices pass requests by a direct call instead of = requeuing. >=20 Ah, I see that we were thinking about similar things. Another tactic, = and one that is easier to prototype and implement than moving GEOM to a graph, is to = allow separate but related bio's to be chained. If a caller, like maybe physio or the = bufdaemon or=20 even a middle geom transform, knows that it's going to send multiple = bio's at once, it chains them together into a single request, and that request gets = pipelined through the stack. Each layer operates on the entire chain before requeueing to = the next layer. Layers/classes that can't operate this way will get the bio serialized = automatically for them, breaking the chain, but those won't be the common cases. This will = bring cache locality benefits, and is something that know benefits high-transaction load = network applications. Scott From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:30:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2041106564A; Sun, 21 Mar 2010 16:30:53 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2001:470:9a47::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9DF258FC0A; Sun, 21 Mar 2010 16:30:53 +0000 (UTC) Received: from acme.spoerlein.net (localhost.spoerlein.net [IPv6:::1]) by acme.spoerlein.net (Postfix) with ESMTPS id E440D5C84; Sun, 21 Mar 2010 17:30:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1269189053; bh=heyuYa2JPG0lu7GUHvqXqkH+knPoUivhy4eDx2zeqFU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=lgKgFcyaynWP13u3Jc0RVgSAvrsfPi8iyUqvKWr6M+HGGUlsQdwMR5EMZrftjYVcG q26bX0RQ3mPHYuuBsU+wetU5XqPA/ifWmeB3myY8BxMR1vjmRhstr6JDwvcipDzBGl 5Rzrb4HputN8l8Vuy5t0+QLf/oG8JlvHcRrSR3oQ= Received: (from uqs@localhost) by acme.spoerlein.net (8.14.4/8.14.4/Submit) id o2LGUqKH088946; Sun, 21 Mar 2010 17:30:52 +0100 (CET) (envelope-from uqs@spoerlein.net) Date: Sun, 21 Mar 2010 17:30:52 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Scott Long Message-ID: <20100321163051.GT99813@acme.spoerlein.net> Mail-Followup-To: Scott Long , Matthew Dillon , Alexander Motin , freebsd-current@freebsd.org, freebsd-arch@freebsd.org References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-current@freebsd.org, Alexander Motin , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:30:54 -0000 On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: > Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an > odd number less than 512k. For the purpose of benchmarking against these > OS's, having comparable capabilities is essential; Linux easily beats FreeBSD > in the silly-i/o-test because of the MAXPHYS difference (though FreeBSD typically > stomps linux in real I/O because of vastly better latency and caching algorithms). > I'm fine with raising MAXPHYS in production once the problems are addressed. Hi Scott, while I'm sure that most of the FreeBSD admins are aware of "silly" benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some pointers regarding your statement that FreeBSD triumphs for real-world I/O loads? Can this be simulated using iozone, bonnie, etc? More importantly, is there a way to do this file system independently? Regards, Uli From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:32:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0A6E1106566C; Sun, 21 Mar 2010 16:32:43 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 9CDF78FC22; Sun, 21 Mar 2010 16:32:42 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LGWdqu036885; Sun, 21 Mar 2010 10:32:39 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BA52179.9030903@FreeBSD.org> Date: Sun, 21 Mar 2010 10:32:39 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <39C5864C-8A6B-4137-8743-D7B9F30F5939@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> To: Alexander Motin X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:32:43 -0000 On Mar 20, 2010, at 1:26 PM, Alexander Motin wrote: > Scott Long wrote: >> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>> Diminishing returns get hit pretty quickly with larger MAXPHYS = values. >>> As long as the I/O can be pipelined the reduced transaction rate >>> becomes less interesting when the transaction rate is less than a >>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>> considering whether it is an issue or not. 256K is actually quite >>> a reasonable value. Even 128K is reasonable. >>=20 >> I agree completely. I did quite a bit of testing on this in 2008 and = 2009. >> I even added some hooks into CAM to support this, and I thought that = I had >> discussed this extensively with Alexander at the time. Guess it was = yet another >> wasted conversation with him =3D-( I'll repeat it here for the = record. >=20 > AFAIR at that time you've agreed that 256K gives improvements, and 64K > of DFLTPHYS limiting most SCSI SIMs is too small. That's why you've > implemented that hooks in CAM. I have not forgot that conversation = (pity > that it quietly died for SCSI SIMs). I agree that too high value could > be just a waste of resources. As you may see I haven't blindly = committed > it, but asked public opinion. If you think 256K is OK - let it be = 256K. > If you think that 256K needed only for media servers - OK, but lets = make > it usable there. >=20 I think that somewhere in the range of 128-512k is appropriate for a = given platform. Maybe big-iron gets 512k and notebooks and embedded systems get 128k? = It's partially a platform architecture issue, and partially a platform = application issue. Ultimately, it should be possible to have up to 1M, and maybe even more. = I don't know how best to make that selectable, or whether it should just be the = default. >> Besides the nswbuf sizing problem, there is a real problem that a lot = of drivers >> have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are >> particular values, and they've sized their data structures = accordingly. Before >> these values are changed, an audit needs to be done OF EVERY SINGLE >> STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS >> in the ata driver, testing that your machine boots, and then = committing the change >> to source control. Some drivers will have non-obvious restrictions = based on >> the number of SG elements allowed in a particular command format. = MPT >> comes to mind (its multi message SG code seems to be broken when I = tried >> testing large MAXPHYS on it), but I bet that there are others. >=20 > As you should remember, we have made it in such way, that all = unchecked > drivers keep using DFLTPHYS, which is not going to be changed ever. So > there is no problem. I would more worry about non-CAM storages and = above > stuff, like some rare GEOM classes. And that's why I say that everything needs to be audited. Are there CAM = drivers that default to being silent on cpi->maxio, but still look at DFLTPHYS = and MAXPHYS? Are there non-CAM drivers that look at MAXPHYS, or that silently assume = that MAXPHYS will never be more than 128k? Scott From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:39:20 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 46AB3106566C; Sun, 21 Mar 2010 16:39:20 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id DA6AB8FC0C; Sun, 21 Mar 2010 16:39:19 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LGdAUv036941; Sun, 21 Mar 2010 10:39:10 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: Scott Long In-Reply-To: <20100321163051.GT99813@acme.spoerlein.net> Date: Sun, 21 Mar 2010 10:39:10 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <9524C333-F191-4F7A-A5FA-BD52498169C0@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <20100321163051.GT99813@acme.spoerlein.net> To: =?iso-8859-1?Q?Ulrich_Sp=F6rlein?= X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: freebsd-current@freebsd.org, Alexander Motin , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:39:20 -0000 On Mar 21, 2010, at 10:30 AM, Ulrich Sp=F6rlein wrote: > On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: >> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of = an >> odd number less than 512k. For the purpose of benchmarking against = these >> OS's, having comparable capabilities is essential; Linux easily beats = FreeBSD >> in the silly-i/o-test because of the MAXPHYS difference (though = FreeBSD typically >> stomps linux in real I/O because of vastly better latency and caching = algorithms). >> I'm fine with raising MAXPHYS in production once the problems are = addressed. >=20 > Hi Scott, >=20 > while I'm sure that most of the FreeBSD admins are aware of "silly" > benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some > pointers regarding your statement that FreeBSD triumphs for real-world > I/O loads? Can this be simulated using iozone, bonnie, etc? More > importantly, is there a way to do this file system independently? >=20 iozone and bonnie tend to be good at testing serialized I/O latency; = each read and write is serialized without any buffering. My experience = is that they give mixed results, sometimes they favor freebsd, sometime = linux, sometimes it's a wash, all because they are so sensitive to = latency. And that's where is also gets hard to have a "universal" = benchmark; what are you really trying to model, and how does that model = reflect your actual workload? Are you running a single-instance, single = threaded application that is sensitive to latency? Are you running a = multi-instance/multi-threaded app that is sensitive to bandwidth? Are = you operating on a single file, or on a large tree of files, or on a raw = device? Are you sharing a small number of relatively stable file = descriptors, or constantly creating and deleting files and truncating = space?= From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 16:53:49 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E89B1065670 for ; Sun, 21 Mar 2010 16:53:49 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [188.72.220.29]) by mx1.freebsd.org (Postfix) with ESMTP id 14D738FC16 for ; Sun, 21 Mar 2010 16:53:48 +0000 (UTC) Received: from acme.spoerlein.net (localhost.spoerlein.net [IPv6:::1]) by acme.spoerlein.net (Postfix) with ESMTPS id F211D5C84; Sun, 21 Mar 2010 17:53:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1269190397; bh=o2mmxpb8Ol6Va50nCfh3b6uJrRt/9DfdyoTAD/Y1P7E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=fdfvNQoZhGZYBjsrCRzcwHTFuWtrjlPR/nwEToJ6WwliOMvFz0s6o9ZQ9wZhWIaue STSEHXx4+CCJFhWAKhDVbhB6oZtB0tmrUAgmS9AuszZsqauCyvRsFqWM6GGeFJLbI1 Zmo3+kP9OT6gXzJGI4kPw2YvYdPwmb0dYcw/ZeX4= Received: (from uqs@localhost) by acme.spoerlein.net (8.14.4/8.14.4/Submit) id o2LGrGoJ089397; Sun, 21 Mar 2010 17:53:16 +0100 (CET) (envelope-from uqs@spoerlein.net) Date: Sun, 21 Mar 2010 17:53:16 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Scott Long Message-ID: <20100321165316.GU99813@acme.spoerlein.net> Mail-Followup-To: Scott Long , freebsd-current@freebsd.org, freebsd-arch@freebsd.org References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <20100321163051.GT99813@acme.spoerlein.net> <9524C333-F191-4F7A-A5FA-BD52498169C0@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9524C333-F191-4F7A-A5FA-BD52498169C0@samsco.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 16:53:49 -0000 [CC trimmed] On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote: > On Mar 21, 2010, at 10:30 AM, Ulrich Spörlein wrote: > > On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: > >> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an > >> odd number less than 512k. For the purpose of benchmarking against these > >> OS's, having comparable capabilities is essential; Linux easily beats FreeBSD > >> in the silly-i/o-test because of the MAXPHYS difference (though FreeBSD typically > >> stomps linux in real I/O because of vastly better latency and caching algorithms). > >> I'm fine with raising MAXPHYS in production once the problems are addressed. > > > > Hi Scott, > > > > while I'm sure that most of the FreeBSD admins are aware of "silly" > > benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some > > pointers regarding your statement that FreeBSD triumphs for real-world > > I/O loads? Can this be simulated using iozone, bonnie, etc? More > > importantly, is there a way to do this file system independently? > > > > iozone and bonnie tend to be good at testing serialized I/O latency; each read and write is serialized without any buffering. My experience is that they give mixed results, sometimes they favor freebsd, sometime linux, sometimes it's a wash, all because they are so sensitive to latency. And that's where is also gets hard to have a "universal" benchmark; what are you really trying to model, and how does that model reflect your actual workload? Are you running a single-instance, single threaded application that is sensitive to latency? Are you running a multi-instance/multi-threaded app that is sensitive to bandwidth? Are you operating on a single file, or on a large tree of files, or on a raw device? Are you sharing a small number of relatively stable file descriptors, or constantly creating and deleting files and truncating space? All true, that's why I wanted to know from you, which real world situations you encountered where FreeBSD did/does outperform Linux in regards to I/O throughput and/or latency (depending on scenario, of course). I hope you don't mind, Uli From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 17:04:01 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D2641065673; Sun, 21 Mar 2010 17:04:01 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id D20C58FC1C; Sun, 21 Mar 2010 17:04:00 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so188509fga.13 for ; Sun, 21 Mar 2010 10:03:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=meK1mei+IjxUsyPxLAGaHWPeu8K/ID5pXFAbI25Lnt0=; b=E5w+64ki2goCY5cMnXr88PjVoJj1GqRddGK3Fk0N8GlvlMIHtTJWwJtI2sYnxH9y+h DtLJhRyvXqUPZPIed6Jr0OtxwZjFkxCBkUAgEJdhuoshfE7NODaZCWcuuEav3i2x4Sx0 hTO3A75NSPcywoYZRTOUGcZQzK4aXasKv+AzE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=Ts/oIeOHaw9ejSXRrQv95SFPfAZ0DHS18EnQHpmqo3XMF0bCqW5MOAE+i+uIIM2Kmq tgrzZBdXoW8ipbiRLL4NjAwDnThHQZhJrAcqi6eU7TSSBl4GKBC5kCwN2QT2/sBYwB4r Hlv/zOk0d6VPcteHzDJdYX62Is6FmGrMJXcoA= Received: by 10.86.22.2 with SMTP id 2mr511612fgv.17.1269191039808; Sun, 21 Mar 2010 10:03:59 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 16sm1041828fxm.8.2010.03.21.10.03.58 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 10:03:59 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA6517C.3050509@FreeBSD.org> Date: Sun, 21 Mar 2010 19:03:56 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Scott Long References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <39C5864C-8A6B-4137-8743-D7B9F30F5939@samsco.org> In-Reply-To: <39C5864C-8A6B-4137-8743-D7B9F30F5939@samsco.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 17:04:01 -0000 Scott Long wrote: > On Mar 20, 2010, at 1:26 PM, Alexander Motin wrote: >> As you should remember, we have made it in such way, that all unchecked >> drivers keep using DFLTPHYS, which is not going to be changed ever. So >> there is no problem. I would more worry about non-CAM storages and above >> stuff, like some rare GEOM classes. > > And that's why I say that everything needs to be audited. Are there CAM drivers > that default to being silent on cpi->maxio, but still look at DFLTPHYS and MAXPHYS? If some (most of) drivers silent on cpi->maxio - they will be limited by safe level of DFLTPHYS, which should not be changed ever. There should be no problem. > Are there non-CAM drivers that look at MAXPHYS, or that silently assume that > MAXPHYS will never be more than 128k? That is a question. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 17:10:52 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D9AC1065670; Sun, 21 Mar 2010 17:10:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id EED398FC1E; Sun, 21 Mar 2010 17:10:51 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2LHAmOJ037217; Sun, 21 Mar 2010 11:10:48 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: Scott Long In-Reply-To: <20100321165316.GU99813@acme.spoerlein.net> Date: Sun, 21 Mar 2010 11:10:48 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <574BA80D-61C3-4E3A-A5D3-898ABC605AED@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <20100321163051.GT99813@acme.spoerlein.net> <9524C333-F191-4F7A-A5FA-BD52498169C0@samsco.org> <20100321165316.GU99813@acme.spoerlein.net> To: =?iso-8859-1?Q?Ulrich_Sp=F6rlein?= X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 17:10:52 -0000 On Mar 21, 2010, at 10:53 AM, Ulrich Sp=F6rlein wrote: > [CC trimmed] > On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote: >> On Mar 21, 2010, at 10:30 AM, Ulrich Sp=F6rlein wrote: >>> On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote: >>>> Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of = an >>>> odd number less than 512k. For the purpose of benchmarking against = these >>>> OS's, having comparable capabilities is essential; Linux easily = beats FreeBSD >>>> in the silly-i/o-test because of the MAXPHYS difference (though = FreeBSD typically >>>> stomps linux in real I/O because of vastly better latency and = caching algorithms). >>>> I'm fine with raising MAXPHYS in production once the problems are = addressed. >>>=20 >>> Hi Scott, >>>=20 >>> while I'm sure that most of the FreeBSD admins are aware of "silly" >>> benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some >>> pointers regarding your statement that FreeBSD triumphs for = real-world >>> I/O loads? Can this be simulated using iozone, bonnie, etc? More >>> importantly, is there a way to do this file system independently? >>>=20 >>=20 >> iozone and bonnie tend to be good at testing serialized I/O latency; = each read and write is serialized without any buffering. My experience = is that they give mixed results, sometimes they favor freebsd, sometime = linux, sometimes it's a wash, all because they are so sensitive to = latency. And that's where is also gets hard to have a "universal" = benchmark; what are you really trying to model, and how does that model = reflect your actual workload? Are you running a single-instance, single = threaded application that is sensitive to latency? Are you running a = multi-instance/multi-threaded app that is sensitive to bandwidth? Are = you operating on a single file, or on a large tree of files, or on a raw = device? Are you sharing a small number of relatively stable file = descriptors, or constantly creating and deleting files and truncating = space? >=20 > All true, that's why I wanted to know from you, which real world > situations you encountered where FreeBSD did/does outperform Linux in > regards to I/O throughput and/or latency (depending on scenario, of > course). I have some tests that spawn N number of threads and then do sequential = and random i/o either into a filesystem or a raw disk. FreeBSD gets = more work done with fewer I/O's than linux when you're operating through = the filesystem, thanks to softupdates and the block layer. Linux has a = predictive cache that often times will generate too much i/o in a vain = attempt to aggressively prefetch blocks. So even then it's hard to = measure in a simple way; linux will do more i/o, but less of it will be = useful to the application, thereby increasing latency and increasing = application runtime. Sorry I can't be more specific, but you're asking = for something that I explicitly say I can't provide. Scott From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 19:14:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FF47106564A; Sun, 21 Mar 2010 19:14:43 +0000 (UTC) (envelope-from julianelischer@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id CC95A8FC08; Sun, 21 Mar 2010 19:14:42 +0000 (UTC) Received: by gwj15 with SMTP id 15so2605465gwj.13 for ; Sun, 21 Mar 2010 12:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=+toFUVuysJTKGI6mHO1fI+e5Mp83VbA6/PyJPof96cQ=; b=Ru4G6j89dTAYDrOuyslHkZIUVhvNHFmLuARg6oPrMxRCCvsPtE1SR7jZlVQeeJuowW cEcUIMv8UblwNejsoJ4YQkZQAVaQ7vG8b/K4yO7AIcSe4Dg3OkP80zBey/4E8/6Wm/Cl 58JioF4/6DuutmarEQxvWKcf0VX+vpcMksJsc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=l2GIF0+dBVWiMr3zvPQWT12pQbqwsnSHnGEA+j8QmdftSbp3J4o9yg8p970o1B2FgG E5SYzlhIM+vAOsgdZNpaI5no9gGLocSusJo3Om+bAXD9nmvs2AXtB+/r9P4rgZjoE0Mn +23tT7MCrUDo9dJfU7WXgsFL7MFwPoixPHmM8= Received: by 10.90.18.33 with SMTP id 33mr3236754agr.12.1269197000835; Sun, 21 Mar 2010 11:43:20 -0700 (PDT) Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by mx.google.com with ESMTPS id 7sm1276334ywc.4.2010.03.21.11.43.19 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 11:43:20 -0700 (PDT) Sender: Julian Elischer Message-ID: <4BA668C9.5080407@elischer.org> Date: Sun, 21 Mar 2010 11:43:21 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Scott Long References: <1269109391.00231800.1269099002@10.7.7.3> <1269120182.00231865.1269108002@10.7.7.3> <1269120188.00231888.1269109203@10.7.7.3> <1269123795.00231922.1269113402@10.7.7.3> <1269130981.00231933.1269118202@10.7.7.3> <1269130986.00231939.1269119402@10.7.7.3> <1269134581.00231948.1269121202@10.7.7.3> <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Motin , freebsd-current@freebsd.org, Ivan Voras , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 19:14:43 -0000 Scott Long wrote: > I agree that more threads just creates many more race > complications. Even if it didn't, the storage driver is a > serialization point; it doesn't matter if you have a dozen g_* > threads if only one of them can be in the top half of the driver at > a time. No amount of fine-grained locking is going to help this. Well that depends on the driver and device.. We have multiple linux threads coming in the top under some setups so it wouldn't be a problem. > > I'd like to go in the opposite direction. The queue-dispatch-queue > model of GEOM is elegant and easy to extend, but very wasteful for > the simple case, where the simple case is one or two simple > partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror > transform. None of these need a dedicated dispatch context in > order to operate. What I'd like to explore is compiling the GEOM > stack at creation time into a linear array of operations that > happen without a g_down/g_up context switch. As providers and > consumers taste each other and build a stack, that stack gets > compiled into a graph, and that graph gets executed directly from > the calling context, both from the dev_strategy() side on the top > and the bio_done() on the bottom. GEOM classes that need a > detached context can mark themselves as such, doing so will prevent > a graph from being created, and the current dispatch model will be > retained. I've considered similar ideas. Or providing a non-queuing options for some simple transformations. > > I expect that this will reduce i/o latency by a great margin, thus > directly addressing the performance problem that FusionIO makes an > example of. I'd like to also explore having the g_bio model not > require a malloc at every stage in the stack/graph; even though > going through UMA is fairly fast, it still represents overhead that > can be eliminated. It also represents an out-of-memory failure > case that can be prevented. > > I might try to work on this over the summer. It's really a > research project in my head at this point, but I'm hopeful that > it'll show results. > > Scott > > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current To > unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sun Mar 21 19:52:55 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85DAA106564A for ; Sun, 21 Mar 2010 19:52:55 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail34.syd.optusnet.com.au (mail34.syd.optusnet.com.au [211.29.133.218]) by mx1.freebsd.org (Postfix) with ESMTP id 121C68FC0A for ; Sun, 21 Mar 2010 19:52:54 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c122-106-253-149.belrs3.nsw.optusnet.com.au [122.106.253.149]) by mail34.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o2LJqfsa013770 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 22 Mar 2010 06:52:52 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id o2LJqefc041158; Mon, 22 Mar 2010 06:52:40 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id o2LJqehP041157; Mon, 22 Mar 2010 06:52:40 +1100 (EST) (envelope-from peter) Date: Mon, 22 Mar 2010 06:52:40 +1100 From: Peter Jeremy To: Scott Long Message-ID: <20100321195240.GD45042@server.vk2pj.dyndns.org> References: <20100318.161117.658811636873842325.imp@bsdimp.com> <20100318.165725.480410072667175878.imp@bsdimp.com> <15D5A6D4-1594-4667-AE51-0E26950C81DA@samsco.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="J4XPiPrVK1ev6Sgr" Content-Disposition: inline In-Reply-To: <15D5A6D4-1594-4667-AE51-0E26950C81DA@samsco.org> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) X-CMAE-Score: 0 X-CMAE-Analysis: v=1.1 cv=DbJUOfM3lOzWBa8hRFFTIG5vmyMkKjbvT8Z585enDlE= c=1 sm=1 a=-o65W9qwAAAA:8 a=_H2LwnzFedx3IeB9fvgA:9 a=o7dPn8cotQ2W6MfXNmUA:7 a=0wTo5USnZ28Vc58ojebj1-v6jhYA:4 a=CjuIK1q_8ugA:10 a=HRJsq0TPXRMA:10 a=SCxxJzy2hwNx5BCsgHAA:9 a=58VVNLhxG5tGEyKuDwiTqIxm7TcA:4 a=y9B6laZwkeQwSvSRL7oYmA==:117 Cc: freebsd-arch@FreeBSD.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Mar 2010 19:52:55 -0000 --J4XPiPrVK1ev6Sgr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Mar-18 19:43:57 -0600, Scott Long wrote: >My understanding was that Atom wasn't super-scalar at all and has no >branch prediction or out-of-order logic. It's basically an 80486 >with a modern instruction set. Not quite. It only has in-order pipelines (though it does have a some very limited instruction re-ordering capablities) but is super-scalar (it can issue up to two instructions per cycle) and does have branch prediction. On the latter point, Intel states "branch predictors in Intel Atom processors do not distinguish between different branch types. Sometimes mixing different branch types can cause confusion in the branch prediction hardware." On my netbook, I'm seeing about 15% misprediction whilst idle but this drops to ~3% when I start OOo. Overall it looks like it's around 7%. I suspect predict_true/predict_false is unlikely to help in most cases. What would probably be more useful for Atom would be gcc scheduling support. This is available in gcc 4.3 (ie GPL3) but not in gcc 4.2. I've had a look at dumping the gcc 4.3 Atom scheduler into my gcc 4.2 but the infrastructure has changed sufficiently that this would be a non-trivial task. (And since it would not be committable, I don't think it's worth my time). Likewise, implementing scheduling from scratch in gcc 4.2 would be a non-trivial task. --=20 Peter Jeremy --J4XPiPrVK1ev6Sgr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkumeQgACgkQ/opHv/APuIdP0QCdHMNmba1vEvHN+duXYwV3aPdW pa0AoJSBchL2+HyeEAtKwB1QJV0frby+ =NSmF -----END PGP SIGNATURE----- --J4XPiPrVK1ev6Sgr-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 01:19:28 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D35E9106566C for ; Mon, 22 Mar 2010 01:19:28 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7A98D8FC15 for ; Mon, 22 Mar 2010 01:19:28 +0000 (UTC) Received: by vws1 with SMTP id 1so188952vws.13 for ; Sun, 21 Mar 2010 18:19:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=CCBCoeSdGBY5/LyK2DKG78COsETPwjg37vGlqWHzStQ=; b=PSE7OuLEr2Jxa5KYNcqCDtnRyIqlGqcND5j2JU4zlyC9HC/UVRHzbqrBZkBrP2I1pf 25PehvLVVukCOgIlpx5+mE0yQr22sATFpyMqKXxxm3giWGvwziIFD1tD/eblNtSc6avv /lezHJlrjuyCklvHuo+MB/EgppyeDNh8U0/l8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=cXocjCDcbsteYexf+YwfKw9zmsEdzaZpuvYOF+XMjMcMjFAF39EFsoCbv7jw3VO2eV byCTnhU431KFGd6lJZQ/Ww31YqgYoH4rnYDdEX5yE+KbN1OwxpUkKCo/Dy3kJuTdqvZe 1KlH1LgyaN6uCa/iJphGt4wTALo+KEjf5thsc= Received: by 10.220.126.222 with SMTP id d30mr2983568vcs.198.1269219280436; Sun, 21 Mar 2010 17:54:40 -0700 (PDT) Received: from ppp-21.206.dialinfree.com (ppp-21.206.dialinfree.com [209.172.21.206]) by mx.google.com with ESMTPS id 21sm30691517vws.2.2010.03.21.17.54.36 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 17:54:39 -0700 (PDT) Sender: "J. Hellenthal" Date: Sun, 21 Mar 2010 20:54:59 -0400 From: jhell To: Alexander Motin In-Reply-To: <4BA62757.7090400@FreeBSD.org> Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA62757.7090400@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD-Current , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 01:19:29 -0000 On Sun, 21 Mar 2010 10:04, mav@ wrote: > Julian Elischer wrote: >> In the Fusion-io driver we find that the limiting factor is not the >> size of MAXPHYS, but the fact that we can not push more than >> 170k tps through geom. (in my test machine. I've seen more on some >> beefier machines), but that is only a limit on small transacrtions, >> or in the case of large transfers the DMA engine tops out before a >> bigger MAXPHYS would make any difference. > > Yes, GEOM is quite CPU-hungry on high request rates due to number of > context switches. But impact probably may be reduced from two sides: by > reducing overhead per request, or by reducing number of requests. Both > ways may give benefits. > > If common opinion is not to touch defaults now - OK, agreed. (Note, > Scott, I have agreed :)) But returning to the original question, does > somebody knows real situation when increased MAXPHYS still causes > problems? At least to make it safe. > > I played with it on one re-compile of a kernel and for the sake of it DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash dump to be performed upon request (reboot -d) due to the boundary being hit for DMA which is 65536. Obviously this would have to be adjusted in ata-dma.c. I suppose that there would have to be a better way to get the real allowable boundary from the running system instead of setting it statically. Other then the above I do not see a reason why not... It is HEAD and this is the type of experimental stuff it was meant for. Regards, -- jhell From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 02:00:03 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E831106564A; Mon, 22 Mar 2010 02:00:03 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8A4548FC18; Mon, 22 Mar 2010 02:00:02 +0000 (UTC) Received: by gyg13 with SMTP id 13so858788gyg.13 for ; Sun, 21 Mar 2010 19:00:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=lnt2qNgAuxpKpdjwMqwtOPKBUNHtTgq0InbXtTpmM/M=; b=Nl87VGlHgHZ3E9fLZDedJ1RIY5ohRk94naKPVyg0anuva6N8yvee9Q92Cd51mGA3D5 yQCjg269ytiPuA5cS6KKdh975ciD5fQ+piWe/onmyd6/h4T5Jddv4cC3Q+hifrb/pMnp F2VmuwIhaN+mWNx6T7p7MEee8CcBpYSxAdLoU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=KnwlFLO7KAHKOiNCAsTxvOI9uoNy/UamB7ZaqvDPGZ/I9i15lq62RJrdbfSAnLNjj6 mgBo5i/ujKqBpsTK87mj1+stO34yVpx7P6HqA7lgPmjxEJVLWOyV4CNUmrhZLlRp6S5m nicdZkC2Rx+PpHnDB5LzknO8XL7qN1HgEyzO0= Received: by 10.90.40.17 with SMTP id n17mr3603308agn.3.1269223201919; Sun, 21 Mar 2010 19:00:01 -0700 (PDT) Received: from ppp-23.181.dialinfree.com (ppp-23.181.dialinfree.com [209.172.23.181]) by mx.google.com with ESMTPS id 22sm2697042iwn.4.2010.03.21.18.59.58 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 19:00:01 -0700 (PDT) Sender: "J. Hellenthal" Date: Sun, 21 Mar 2010 22:00:21 -0400 From: jhell To: Alexander Motin In-Reply-To: Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA62757.7090400@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD-Current , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 02:00:03 -0000 On Sun, 21 Mar 2010 20:54, jhell@ wrote: > > On Sun, 21 Mar 2010 10:04, mav@ wrote: >> Julian Elischer wrote: >>> In the Fusion-io driver we find that the limiting factor is not the >>> size of MAXPHYS, but the fact that we can not push more than >>> 170k tps through geom. (in my test machine. I've seen more on some >>> beefier machines), but that is only a limit on small transacrtions, >>> or in the case of large transfers the DMA engine tops out before a >>> bigger MAXPHYS would make any difference. >> >> Yes, GEOM is quite CPU-hungry on high request rates due to number of >> context switches. But impact probably may be reduced from two sides: by >> reducing overhead per request, or by reducing number of requests. Both >> ways may give benefits. >> >> If common opinion is not to touch defaults now - OK, agreed. (Note, >> Scott, I have agreed :)) But returning to the original question, does >> somebody knows real situation when increased MAXPHYS still causes >> problems? At least to make it safe. >> >> > > I played with it on one re-compile of a kernel and for the sake of it > DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash dump to > be performed upon request (reboot -d) due to the boundary being hit for DMA > which is 65536. Obviously this would have to be adjusted in ata-dma.c. > > I suppose that there would have to be a better way to get the real allowable > boundary from the running system instead of setting it statically. > > Other then the above I do not see a reason why not... It is HEAD and this is > the type of experimental stuff it was meant for. > > Regards, > > I should have also said that I also repeated the above without setting DFLTPHYS and setting MAXPHYS to 256. Regards, -- jhell From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 05:53:20 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E5782106566B; Mon, 22 Mar 2010 05:53:20 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f209.google.com (mail-fx0-f209.google.com [209.85.220.209]) by mx1.freebsd.org (Postfix) with ESMTP id 3FDCF8FC0C; Mon, 22 Mar 2010 05:53:19 +0000 (UTC) Received: by fxm1 with SMTP id 1so4966687fxm.33 for ; Sun, 21 Mar 2010 22:53:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=LjlcvBytxdFQIJQwccw31J8lb75cHk6eQciz2IhvwG8=; b=SepHsS2rn5zJs84hWeqxl4zmqXBpYpATOv7BBjkTEYM8gF5vlXiXnRCh5u1/VRzgfL tCugsD+97NuC2I5uyvINHcFTjtjYhFwiQRVIeKJGVRvBkNyju4QWFDT4GXEHL/YKJPf8 CszrVTT3nxcAXPWSPyKS/jkDOCE4K4PB8ohu4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=vCkXmbo8zXDVgMBKWi0I26CY5GGVhFdjYp2dV1Zudo11qqLLCoABj9b9j/KoCVlBOt lq95kS7wr2tu9ZywI7dhtWBv7+Qa8uzJomtwNNNHyTjBOGtrN37ssghh3JIxEiLzdvBa WyBzJoUTlPc8ucrTc5Tu0gxZqABaDGeG4UWRA= Received: by 10.223.5.75 with SMTP id 11mr3756400fau.46.1269237199101; Sun, 21 Mar 2010 22:53:19 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 14sm3294631fxm.13.2010.03.21.22.53.17 (version=SSLv3 cipher=RC4-MD5); Sun, 21 Mar 2010 22:53:18 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA705CB.9090705@FreeBSD.org> Date: Mon, 22 Mar 2010 07:53:15 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: jhell References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA62757.7090400@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 05:53:21 -0000 jhell wrote: > On Sun, 21 Mar 2010 20:54, jhell@ wrote: >> I played with it on one re-compile of a kernel and for the sake of it >> DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash >> dump to be performed upon request (reboot -d) due to the boundary >> being hit for DMA which is 65536. Obviously this would have to be >> adjusted in ata-dma.c. >> >> I suppose that there would have to be a better way to get the real >> allowable boundary from the running system instead of setting it >> statically. >> >> Other then the above I do not see a reason why not... It is HEAD and >> this is the type of experimental stuff it was meant for. > > I should have also said that I also repeated the above without setting > DFLTPHYS and setting MAXPHYS to 256. It was bad idea to increase DFLTPHYS. It is not intended to be increased. About DMA boundary, I do not very understand the problem. Yes, legacy ATA has DMA boundary of 64K, but there is no problem to submit S/G list of several segments. How long ago have you tried it, on which controller and which diagnostics do you have? -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 08:23:46 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 351A01065673; Mon, 22 Mar 2010 08:23:46 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id E88F98FC1A; Mon, 22 Mar 2010 08:23:45 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 27AF56435; Mon, 22 Mar 2010 08:23:44 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.3/8.14.3) with ESMTP id o2M8NhGQ005755; Mon, 22 Mar 2010 08:23:43 GMT (envelope-from phk@critter.freebsd.dk) To: Andriy Gapon From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 21 Mar 2010 16:56:32 +0200." <4BA633A0.2090108@icyb.net.ua> Date: Mon, 22 Mar 2010 08:23:43 +0000 Message-ID: <5754.1269246223@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Alexander Motin , freebsd-current@FreeBSD.org, Ivan Voras , freebsd-arch@FreeBSD.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 08:23:46 -0000 In message <4BA633A0.2090108@icyb.net.ua>, Andriy Gapon writes: >on 21/03/2010 16:05 Alexander Motin said the following: >> Ivan Voras wrote: >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>> barring specific class behaviour, it has a fair chance of working out of >>> the box) but the incoming queue will need to also be broken up for >>> greater effect. >> >> According to "notes", looks there is a good chance to obtain races, as >> some places expect only one up and one down thread. > >I haven't given any deep thought to this issue, but I remember us discussing >them over beer :-) The easiest way to obtain more parallelism, is to divide the mesh into multiple independent meshes. This will do you no good if you have five disks in a RAID-5 config, but if you have two disks each mounted on its own filesystem, you can run a g_up & g_down for each of them. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 09:57:07 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EFA21065673 for ; Mon, 22 Mar 2010 09:57:07 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 0AA508FC26 for ; Mon, 22 Mar 2010 09:57:06 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o2M9v1Ah049053 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 22 Mar 2010 11:57:01 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id o2M9v1Ln066249; Mon, 22 Mar 2010 11:57:01 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id o2M9v1Qb066248; Mon, 22 Mar 2010 11:57:01 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 22 Mar 2010 11:57:01 +0200 From: Kostik Belousov To: arch@freebsd.org, amd64@freebsd.org Message-ID: <20100322095701.GE2415@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7CZp05NP8/gJM8Cl" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.8 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Subject: XMM register usage in the kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 09:57:07 -0000 --7CZp05NP8/gJM8Cl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, apparently, my Core i7 930 does not support AES-NI. I read too much about this thing before realizing that I do not have access to supported CPU, so I had a time to prototype support for using XMM registers and, in some degree, a coprocessor features, in the kernel mode for AMD64. The i386 could be done in a similar manner, but would be less clean, it seems. The KPI consists of two functions, fpu_kern_enter() and fpu_kern_leave(). The pair should brace the region of code that intend to use coprocessor. Caller shall provide the context save area, I expect that usually this memory will be allocated as part of larger subsystem data structure. fpu_kern_enter() allows the nesting, assuming fpu_kern_leave() is called proper number of times to pop the context stack. I deliberately ignored the issue of handling the coprocessor exceptions in the kernel mode. My assumption is that this facility is for the things like AES-NI or similar, where exceptions are not generated if proper programming is ensured. Prototyped patch is at http://people.freebsd.org/~kib/misc/amd64_kern_fpu.1.patch Example usage can be seen at http://people.freebsd.org/~kib/misc/kernfpu (note that example does not follow the guidelines above, usermode can easily crash kernel module, but the goal was to give an example of KPI usage). Awaiting for feedback, thanks. --7CZp05NP8/gJM8Cl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkunPu0ACgkQC3+MBN1Mb4gFfQCg3o5rkj8YTcDGSLmkblbIzmK/ IUAAn2A35KYd5esZWz92+2yrURfE4sZM =R42e -----END PGP SIGNATURE----- --7CZp05NP8/gJM8Cl-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 11:06:57 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA33D106567B for ; Mon, 22 Mar 2010 11:06:57 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9FD0F8FC31 for ; Mon, 22 Mar 2010 11:06:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o2MB6vB4014965 for ; Mon, 22 Mar 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o2MB6vIK014963 for freebsd-arch@FreeBSD.org; Mon, 22 Mar 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Mar 2010 11:06:57 GMT Message-Id: <201003221106.o2MB6vIK014963@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 11:06:57 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 11:34:20 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9F6D106564A; Mon, 22 Mar 2010 11:34:20 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 488A28FC1D; Mon, 22 Mar 2010 11:34:20 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FF67.dip.t-dialin.net [217.84.255.103]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 2FD2F844B26; Mon, 22 Mar 2010 12:34:12 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 7907251B5; Mon, 22 Mar 2010 12:34:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269257648; bh=3P/QKcutTni5aiOOTSuHpYaLoMmfe8J8M93A+9gcekY=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=MmtWMdAM1uWYDfbAgkkTw7eqptZfkOyHUitN2YlQcuQAvkKa++yrZ9npxZEtKXchu a2rKc2jryVXoXZveR2p+cazpRFI8C9Kp58NmwydAd5zTaq/dtXsnaYgRlaTc+J+SQh 8vFq2taAJ1KomFjnRIOpAO+JsXdEBVVTYFVNcTsfViSKx6aTb8yGYwwCY4FC8OA0SK 7Nlkv9oNxkdWMmkZz1VsrRlKciibEEmbSLsZWNRUyjw1gxHeeq4hwP/JgnZtTfy/GS oi0sN3HpIpItKyHg3fXUwY5Rl323Jy5Mp2YDgUZ8m9ZQFGFUMF4KUHfArZ+bLQ/FVd H2QO8R4xhFFOw== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2MBY8nF032060; Mon, 22 Mar 2010 12:34:08 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 22 Mar 2010 12:34:08 +0100 Message-ID: <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> Date: Mon, 22 Mar 2010 12:34:08 +0100 From: Alexander Leidinger To: John Baldwin References: <20100310113422.95932h10tv1qh2o0@webmail.leidinger.net> <201003100812.29749.jhb@freebsd.org> In-Reply-To: <201003100812.29749.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 2FD2F844B26.BB2C7 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269862454.66797@2XXAxS3+6owyTeDP5PLY2g X-EBL-Spam-Status: No Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 11:34:20 -0000 Redirecting from stable@ to arch@... Quoting John Baldwin (from Wed, 10 Mar 2010 08:12:29 -0500): > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >> Quoting "Robert N. M. Watson" (from Tue, 9 Mar >> 2010 16:39:09 +0000): >> >> > >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >> > >> >>> From this you can see that sys.mk is included and parsed before > 'Makefile', >> >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. >> >> >> >> I think we need to find a different solution for this. The need to >> >> specify WITH_CTF at the command line is very error prone. :( >> > >> > You are neither the first person to have made this observation, nor >> > the first person to have failed to propose a solution in the form of >> > a patch :-). Ok, here is the proposal in form of a patch. :-) http://www.leidinger.net/test/ctf.diff > Unfortunately the ctf stuff breaks static binaries. I think that if > that were > fixed we would simply enable it by default and be done. The patch is: - enabling CTF stuff by default for the kernel - allows to disable the CTF stuff for the kernel by defining NO_CTF - *not* enabling the CTF stuff by default for libs and progs (if someone tells me how to distinguish the build for static stuff from dynamic stuff, I can have a look to enable it for the dynamic case) - allows to enable the CTF stuff for the userland by defining WITH_CTF as before I have not tested what this patch is doing to bootblocks or the loader (= stuff within /sys/ which is not the kernel or kernel modules). In case it hurts, we can add NO_CTF to the corresponding Makefiles. I do not have a scratch system around ATM, so any report from installing a bootblock (gpt/zfs/normal/whatever) after a buildworld/installworld is welcome. In case there are people which want to moan that I moved a conditional from make to shell: - the shell stuff is using build-ins - feel free to provide patches which makes it work with make-conditionals (I failed, and I tried several things before switching to shell stuff) In case people moan about the inverse logic I use in the shell conditional: - the current way is working for all cases and does not require to ignore an error in make ("make CTFCONVERT=false" correctly errors out) - test your proposal before you moan (worst case: all parmutations of WITH_CTF, NO_CTF, CTFCONVERT=false) - if the majority wants to have the '-' in front of a similar positive-logic command, I do not object, but this will ignore real errors from ctf{convert|merge} (whatever those could be) Bye, Alexander. -- The real problem with hunting elephants carrying the decoys. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 11:40:22 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67F76106566B; Mon, 22 Mar 2010 11:40:22 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout3.freenet.de (mout3.freenet.de [IPv6:2001:748:100:40::2:5]) by mx1.freebsd.org (Postfix) with ESMTP id F05B68FC1C; Mon, 22 Mar 2010 11:40:21 +0000 (UTC) Received: from [195.4.92.23] (helo=13.mx.freenet.de) by mout3.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NtfzY-00006b-7Q; Mon, 22 Mar 2010 12:40:20 +0100 Received: from p57ae04cc.dip0.t-ipconnect.de ([87.174.4.204]:10019 helo=ernst.jennejohn.org) by 13.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NtfzY-0005Bb-0v; Mon, 22 Mar 2010 12:40:20 +0100 Date: Mon, 22 Mar 2010 12:40:18 +0100 From: Gary Jennejohn To: Alexander Motin Message-ID: <20100322124018.7430f45e@ernst.jennejohn.org> In-Reply-To: <4BA6517C.3050509@FreeBSD.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <39C5864C-8A6B-4137-8743-D7B9F30F5939@samsco.org> <4BA6517C.3050509@FreeBSD.org> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Scott Long , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 11:40:22 -0000 On Sun, 21 Mar 2010 19:03:56 +0200 Alexander Motin wrote: > Scott Long wrote: > > Are there non-CAM drivers that look at MAXPHYS, or that silently assume that > > MAXPHYS will never be more than 128k? > > That is a question. > I only did a quick&dirty grep looking for MAXPHYS in /sys. Some drivers redefine MAXPHYS to be 512KiB. Some use their own local MAXPHYS which is usually 128KiB. Some look at MAXPHYS to figure out other things; the details escape me. There's one driver which actually uses 100*MAXPHYS for something, but I didn't check the details. Lots of them were non-CAM drivers AFAICT. -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 12:49:40 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E97541065672; Mon, 22 Mar 2010 12:49:39 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 8F6438FC13; Mon, 22 Mar 2010 12:49:39 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FF67.dip.t-dialin.net [217.84.255.103]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id B0E49844B26; Mon, 22 Mar 2010 13:49:32 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id B6DFA51BD; Mon, 22 Mar 2010 13:49:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269262169; bh=miJRW5cP0VaYHI4A2iGYb0DMJ9f7jCJcF67EwyWNwqI=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=ril+FxhmXGESKr4Za6mQCSymfbUrc9XBXsD9zGFHNes2J1PIiOB+QUdZ1r3AL9+Jb wyUYVJVWbVjmZjDwNno1QKujzbFPm/HIUMTdaX2EtXrJ+bLxn1rgEz2mUMAO65sq5U Mx5HtrRF0RiGNIwStS9YPuFg0N/3XypjWxvg6iJFSu4yubdWOGybi4FLTjU6ctLf9v oM1uXvwSAbJ66NHWIMyM9SHSrF32BZEdj7cdDUXfyoEjyUV9dOQmEFyUwtrPhHjJBw 8JMorRcHCIDAcTC1nU6oL0rPjmPM0xJO8vj71sMxFxj/l1YM0YY6xVoXfaoyjbBp3M t7ikQld7icwcA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2MCnSxq048872; Mon, 22 Mar 2010 13:49:28 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 22 Mar 2010 13:49:28 +0100 Message-ID: <20100322134928.83045xnbh4nkqzok@webmail.leidinger.net> Date: Mon, 22 Mar 2010 13:49:28 +0100 From: Alexander Leidinger To: Scott Long References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> In-Reply-To: <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: B0E49844B26.A3C70 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269866973.60709@WBDkZuiQeW9/cn/c0e+A1Q X-EBL-Spam-Status: No Cc: FreeBSD-Current , Alexander Motin , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 12:49:40 -0000 Quoting Scott Long (from Sat, 20 Mar 2010 12:17:33 -0600): > code was actually taking advantage of the larger I/O's. The > improvement really > depends on the workload, of course, and I wouldn't expect it to be noticeable > for most people unless they're running something like a media server. I don't think this is limited to media servers, think about situations where you process a large amount of data seuqntially... (seuqntial access case in a big data-warehouse scenario or a 3D render farm which get's the huge amount of data from a shared resource ("how many render-clients can I support at the same time with my disk infrastructure"-scenario) or some of the bigtable/nosql stuff which seems to be more and more popular at some sites). There are enough situations where sequential file access is the key performance metric so that I wouldn't say that only media servers depend upon large sequential I/O's. Bye, Alexander. -- That's life. What's life? A magazine. How much does it cost? Two-fifty. I only have a dollar. That's life. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 13:40:34 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A792106564A for ; Mon, 22 Mar 2010 13:40:34 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 5829A8FC0A for ; Mon, 22 Mar 2010 13:40:34 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 884851FFC58; Mon, 22 Mar 2010 13:40:32 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 558948449F; Mon, 22 Mar 2010 14:40:32 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Daniel Eischen References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> <4BA3E9DF.2050303@delphij.net> Date: Mon, 22 Mar 2010 14:40:32 +0100 In-Reply-To: (Daniel Eischen's message of "Fri, 19 Mar 2010 17:24:32 -0400 (EDT)") Message-ID: <86y6hkiien.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: Scott Long , d@delphij.net, Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 13:40:34 -0000 Daniel Eischen writes: > Perhaps I was wrong, but I thought Scott's question was more general: > is there a desire for a special installation suitable to small > appliances (usually flash-based)? Not all "small appliances" are flash-based; the Geode-based soekris can use 2.5" disks and has enough memory to run a stock install. The only trouble I've had is with /etc/periodic/daily/470.status-named, which uses an ungodly amount of memory. The AMD Geode is more or less equivalent to a Pentium MMX, and does not have SSE. Soekris net4801-48: CPU: Geode(TM) Integrated Processor by National Semi (266.65-MHz 586-class = CPU) Origin =3D "Geode by NSC" Id =3D 0x540 Stepping =3D 0 Features=3D0x808131 real memory =3D 134217728 (128 MB) avail memory =3D 126160896 (120 MB) Soekris net5501-70: CPU: Geode(TM) Integrated Processor by AMD PCS (499.91-MHz 586-class CPU) Origin =3D "AuthenticAMD" Id =3D 0x5a2 Stepping =3D 2 Features=3D0x88a93d AMD Features=3D0xc0400000 real memory =3D 536870912 (512 MB) avail memory =3D 510726144 (487 MB) DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 14:51:03 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EEBA31065676; Mon, 22 Mar 2010 14:51:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BE70A8FC1D; Mon, 22 Mar 2010 14:51:03 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7237346B35; Mon, 22 Mar 2010 10:51:03 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A67E98A025; Mon, 22 Mar 2010 10:51:02 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, gary.jennejohn@freenet.de Date: Mon, 22 Mar 2010 08:39:12 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <4BA4E7A9.3070502@FreeBSD.org> <4BA6517C.3050509@FreeBSD.org> <20100322124018.7430f45e@ernst.jennejohn.org> In-Reply-To: <20100322124018.7430f45e@ernst.jennejohn.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003220839.12907.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 22 Mar 2010 10:51:02 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Alexander Motin , FreeBSD-Current , Scott Long Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 14:51:04 -0000 On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: > On Sun, 21 Mar 2010 19:03:56 +0200 > Alexander Motin wrote: > > > Scott Long wrote: > > > Are there non-CAM drivers that look at MAXPHYS, or that silently assume that > > > MAXPHYS will never be more than 128k? > > > > That is a question. > > > > I only did a quick&dirty grep looking for MAXPHYS in /sys. > > Some drivers redefine MAXPHYS to be 512KiB. Some use their own local > MAXPHYS which is usually 128KiB. > > Some look at MAXPHYS to figure out other things; the details escape me. > > There's one driver which actually uses 100*MAXPHYS for something, but I > didn't check the details. > > Lots of them were non-CAM drivers AFAICT. The problem is the drivers that _don't_ reference MAXPHYS. The driver author at the time "knew" that MAXPHYS was 128k, so he did the MAXPHYS-dependent calculation and just put the result in the driver (e.g. only supporting up to 32 segments (32 4k pages == 128k) in a bus dma tag as a magic number to bus_dma_tag_create() w/o documenting that the '32' was derived from 128k or what the actual hardware limit on nsegments is). These cannot be found by a simple grep, they require manually inspecting each driver. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 14:51:06 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9458E106567A; Mon, 22 Mar 2010 14:51:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 63DE78FC1F; Mon, 22 Mar 2010 14:51:06 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 17E9346B6C; Mon, 22 Mar 2010 10:51:06 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 282598A028; Mon, 22 Mar 2010 10:51:05 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Mon, 22 Mar 2010 09:41:10 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201003100812.29749.jhb@freebsd.org> <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> In-Reply-To: <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003220941.10525.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 22 Mar 2010 10:51:05 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 14:51:06 -0000 On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: > Redirecting from stable@ to arch@... > > Quoting John Baldwin (from Wed, 10 Mar 2010 08:12:29 -0500): > > > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: > >> Quoting "Robert N. M. Watson" (from Tue, 9 Mar > >> 2010 16:39:09 +0000): > >> > >> > > >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: > >> > > >> >>> From this you can see that sys.mk is included and parsed before > > 'Makefile', > >> >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. > >> >> > >> >> I think we need to find a different solution for this. The need to > >> >> specify WITH_CTF at the command line is very error prone. :( > >> > > >> > You are neither the first person to have made this observation, nor > >> > the first person to have failed to propose a solution in the form of > >> > a patch :-). > > Ok, here is the proposal in form of a patch. :-) > http://www.leidinger.net/test/ctf.diff > > > Unfortunately the ctf stuff breaks static binaries. I think that if > > that were > > fixed we would simply enable it by default and be done. > > The patch is: > - enabling CTF stuff by default for the kernel > - allows to disable the CTF stuff for the kernel by defining NO_CTF > - *not* enabling the CTF stuff by default for libs and progs > (if someone tells me how to distinguish the build for static > stuff from dynamic stuff, I can have a look to enable it for > the dynamic case) > - allows to enable the CTF stuff for the userland by defining > WITH_CTF as before I think this patch looks very interesting. I think in some ways it would be nice to make CTF "opt-in" though instead of "opt-out". I think the current patch would enable CTF when building ports, for example. I think instead it should default to not building CTF, but require an ENABLE_CTF (instead of NO_CTF) to be set, and set that in bsd.kern.mk if WITH_CTF is defined. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 14:55:32 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DBBE1065676; Mon, 22 Mar 2010 14:55:32 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 347668FC23; Mon, 22 Mar 2010 14:55:31 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MEtPqV065712; Mon, 22 Mar 2010 08:55:25 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <201003220941.10525.jhb@freebsd.org> Date: Mon, 22 Mar 2010 08:55:25 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8E9F405D-0140-4C67-B7BD-94714E2DD109@samsco.org> References: <201003100812.29749.jhb@freebsd.org> <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: Alexander Leidinger , "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 14:55:32 -0000 On Mar 22, 2010, at 7:41 AM, John Baldwin wrote: > On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: >> Redirecting from stable@ to arch@... >>=20 >> Quoting John Baldwin (from Wed, 10 Mar 2010 = 08:12:29=20 > -0500): >>=20 >>> On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >>>> Quoting "Robert N. M. Watson" (from Tue, 9 = Mar >>>> 2010 16:39:09 +0000): >>>>=20 >>>>>=20 >>>>> On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >>>>>=20 >>>>>>> =46rom this you can see that sys.mk is included and parsed = before >>> 'Makefile', >>>>>>> so the WITH_CTF=3Dyes is not set until after sys.mk has been = parsed. >>>>>>=20 >>>>>> I think we need to find a different solution for this. The need = to >>>>>> specify WITH_CTF at the command line is very error prone. :( >>>>>=20 >>>>> You are neither the first person to have made this observation, = nor >>>>> the first person to have failed to propose a solution in the form = of >>>>> a patch :-). >>=20 >> Ok, here is the proposal in form of a patch. :-) >> http://www.leidinger.net/test/ctf.diff >>=20 >>> Unfortunately the ctf stuff breaks static binaries. I think that if = =20 >>> that were >>> fixed we would simply enable it by default and be done. >>=20 >> The patch is: >> - enabling CTF stuff by default for the kernel >> - allows to disable the CTF stuff for the kernel by defining NO_CTF >> - *not* enabling the CTF stuff by default for libs and progs >> (if someone tells me how to distinguish the build for static >> stuff from dynamic stuff, I can have a look to enable it for >> the dynamic case) >> - allows to enable the CTF stuff for the userland by defining >> WITH_CTF as before >=20 > I think this patch looks very interesting. I think in some ways it = would be=20 > nice to make CTF "opt-in" though instead of "opt-out". I think the = current=20 > patch would enable CTF when building ports, for example. I think = instead it=20 > should default to not building CTF, but require an ENABLE_CTF (instead = of=20 > NO_CTF) to be set, and set that in bsd.kern.mk if WITH_CTF is defined. >=20 I have a patch at Yahoo that makes WITH_CTF settable from the kernel = config file, thus making it opt-in. I'd prefer this as well to opt-out. = Give me a little bit to dig it up and polish it for review. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 15:01:47 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DD8C106566B; Mon, 22 Mar 2010 15:01:47 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id BC4C58FC0A; Mon, 22 Mar 2010 15:01:46 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MF1hIU086287; Mon, 22 Mar 2010 09:01:43 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <8E9F405D-0140-4C67-B7BD-94714E2DD109@samsco.org> Date: Mon, 22 Mar 2010 09:01:43 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <33755363-49D9-46C4-9EEC-2951ED263C86@samsco.org> References: <201003100812.29749.jhb@freebsd.org> <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> <8E9F405D-0140-4C67-B7BD-94714E2DD109@samsco.org> To: John Baldwin , Alexander Leidinger X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 15:01:47 -0000 Actually, I'll withdraw my patch, I like Alexander's better, with the = exception that CTF should be opt-in, not opt-out. Scott On Mar 22, 2010, at 8:55 AM, Scott Long wrote: > On Mar 22, 2010, at 7:41 AM, John Baldwin wrote: >> On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: >>> Redirecting from stable@ to arch@... >>>=20 >>> Quoting John Baldwin (from Wed, 10 Mar 2010 = 08:12:29=20 >> -0500): >>>=20 >>>> On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >>>>> Quoting "Robert N. M. Watson" (from Tue, 9 = Mar >>>>> 2010 16:39:09 +0000): >>>>>=20 >>>>>>=20 >>>>>> On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >>>>>>=20 >>>>>>>> =46rom this you can see that sys.mk is included and parsed = before >>>> 'Makefile', >>>>>>>> so the WITH_CTF=3Dyes is not set until after sys.mk has been = parsed. >>>>>>>=20 >>>>>>> I think we need to find a different solution for this. The need = to >>>>>>> specify WITH_CTF at the command line is very error prone. :( >>>>>>=20 >>>>>> You are neither the first person to have made this observation, = nor >>>>>> the first person to have failed to propose a solution in the form = of >>>>>> a patch :-). >>>=20 >>> Ok, here is the proposal in form of a patch. :-) >>> http://www.leidinger.net/test/ctf.diff >>>=20 >>>> Unfortunately the ctf stuff breaks static binaries. I think that = if =20 >>>> that were >>>> fixed we would simply enable it by default and be done. >>>=20 >>> The patch is: >>> - enabling CTF stuff by default for the kernel >>> - allows to disable the CTF stuff for the kernel by defining NO_CTF >>> - *not* enabling the CTF stuff by default for libs and progs >>> (if someone tells me how to distinguish the build for static >>> stuff from dynamic stuff, I can have a look to enable it for >>> the dynamic case) >>> - allows to enable the CTF stuff for the userland by defining >>> WITH_CTF as before >>=20 >> I think this patch looks very interesting. I think in some ways it = would be=20 >> nice to make CTF "opt-in" though instead of "opt-out". I think the = current=20 >> patch would enable CTF when building ports, for example. I think = instead it=20 >> should default to not building CTF, but require an ENABLE_CTF = (instead of=20 >> NO_CTF) to be set, and set that in bsd.kern.mk if WITH_CTF is = defined. >>=20 >=20 > I have a patch at Yahoo that makes WITH_CTF settable from the kernel = config file, thus making it opt-in. I'd prefer this as well to opt-out. = Give me a little bit to dig it up and polish it for review. >=20 > Scott >=20 > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to = "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 15:13:15 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EABF106566C; Mon, 22 Mar 2010 15:13:15 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-fx0-f224.google.com (mail-fx0-f224.google.com [209.85.220.224]) by mx1.freebsd.org (Postfix) with ESMTP id 403A88FC0C; Mon, 22 Mar 2010 15:13:14 +0000 (UTC) Received: by fxm24 with SMTP id 24so1747017fxm.3 for ; Mon, 22 Mar 2010 08:13:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=DsRv+5AHwNnZAxbyt2GiWovNe3F9FVZXx5YY0zZIM3k=; b=cMFUq1MxzJL1ie5YQCMyJKtoccBfrg6XvNvcgOne9/4RllPgIR3Yrn3l+cVk0is0a/ dDcpF6X4W6p9ecQKE44WTPgxGfPQrWVNqGEuyPch1CX1NTem3GJfUnCWHqebvmojicuS A4jlQGYXlBLIrnutF645ShZ7mpeEOatO+ztaE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=vfmy+Ly6x0Efvy4ohd+N4UjfdonSCczYib5P6Knfvs6azXETqPnPeJED1YuhOD0OyX +Pco24Lzt0MoTieJahGIyCsSEErWLHpUGgvcwdlcu6Tw16gyCvE5MU/7RkIoM/YinWym PLBTfOHfP6je0B8POWS4APM3s6r2+VoiP+56E= Received: by 10.223.58.83 with SMTP id f19mr2826229fah.88.1269270793405; Mon, 22 Mar 2010 08:13:13 -0700 (PDT) Received: from centel.dataix.local (ppp-21.63.dialinfree.com [209.172.21.63]) by mx.google.com with ESMTPS id 13sm3894781fxm.14.2010.03.22.08.12.27 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 22 Mar 2010 08:13:12 -0700 (PDT) Sender: "J. Hellenthal" Date: Mon, 22 Mar 2010 11:11:55 -0400 From: jhell To: Alexander Motin In-Reply-To: <4BA705CB.9090705@FreeBSD.org> Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA62757.7090400@FreeBSD.org> <4BA705CB.9090705@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 15:13:15 -0000 On Mon, 22 Mar 2010 01:53, Alexander Motin wrote: In Message-Id: <4BA705CB.9090705@FreeBSD.org> > jhell wrote: >> On Sun, 21 Mar 2010 20:54, jhell@ wrote: >>> I played with it on one re-compile of a kernel and for the sake of it >>> DFLTPHYS=128 MAXPHYS=256 and found out that I could not cause a crash >>> dump to be performed upon request (reboot -d) due to the boundary >>> being hit for DMA which is 65536. Obviously this would have to be >>> adjusted in ata-dma.c. >>> >>> I suppose that there would have to be a better way to get the real >>> allowable boundary from the running system instead of setting it >>> statically. >>> >>> Other then the above I do not see a reason why not... It is HEAD and >>> this is the type of experimental stuff it was meant for. >> >> I should have also said that I also repeated the above without setting >> DFLTPHYS and setting MAXPHYS to 256. > > It was bad idea to increase DFLTPHYS. It is not intended to be increased. > I just wanted to see what I could break; when I increased DFLTPHYS it was just for that purpose. It booted and everything was running after. Wasn't long enough to do any damage. > About DMA boundary, I do not very understand the problem. Yes, legacy > ATA has DMA boundary of 64K, but there is no problem to submit S/G list > of several segments. How long ago have you tried it, on which controller > and which diagnostics do you have? > > atapci0@pci0:0:31:1: class=0x01018a card=0x01271028 chip=0x24cb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801DB/DBL (ICH4/ICH4-L) UltraATA/100 EIDE Controller' class = mass storage subclass = ATA I do not have any diagnostics but if any are requested I do have the kernel's that I have tuned to the above values readily available to run again. The first time I tuned MAXPHYS was roughly about 7 weeks ago. That was until I noticed I could not get a crash dump for a problem I was having a week later and had to revert back to its default setting of 128. The problem I had a week later was unrelated. Two days ago when I saw this thread I recalled having modified MAXPHYS but could not remember the problem it caused so I re-enabled it again to reproduce the problem for sureness. Anything else you need please address, Regards, -- jhell From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 16:19:49 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23849106564A; Mon, 22 Mar 2010 16:19:49 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from mail-gx0-f211.google.com (mail-gx0-f211.google.com [209.85.217.211]) by mx1.freebsd.org (Postfix) with ESMTP id BD6788FC1F; Mon, 22 Mar 2010 16:19:48 +0000 (UTC) Received: by gxk3 with SMTP id 3so994503gxk.13 for ; Mon, 22 Mar 2010 09:19:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=4kQo9/KiBiBLYVqfysneIeFu6+5+qsMJTr9lZrB0DLo=; b=xl5XQtSQwiLPK/zdSCeOm/L5erXKwLi8BNrgxczQYPoXBtupxOKVc3M0wD5WsdxER8 QwhcW3IZ77tKPs68ghbe2xv3uCyZAkB4drn2OJHzbwwlaBmA9xJuxGo/Fxcwi0wWj88w j1I6tJ8Bh/drdDXU2qhGpvBlPl9Q2ik/gKdWc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=mN8jORWhrg/qEYHrz+01St/nvznheNnXdYZp7aqfCNyHBkpDlmbkCL3MjsQc96K7z9 WeLd3FwZZlwHjpDez94qH+iL10cGDWaf7SucgFy2aa0OY+cm6XB2IoK88IxWrRGUF3zZ SD/HBR1cWqbF9SRIubpe87oQk14KG3Epp9ui0= MIME-Version: 1.0 Received: by 10.101.189.30 with SMTP id r30mr8435712anp.70.1269273165090; Mon, 22 Mar 2010 08:52:45 -0700 (PDT) In-Reply-To: <201003220839.12907.jhb@freebsd.org> References: <4BA4E7A9.3070502@FreeBSD.org> <4BA6517C.3050509@FreeBSD.org> <20100322124018.7430f45e@ernst.jennejohn.org> <201003220839.12907.jhb@freebsd.org> Date: Mon, 22 Mar 2010 11:52:44 -0400 Message-ID: <3c0b01821003220852r61ca0ae3o95bea1c23ddc34d9@mail.gmail.com> From: Alexander Sack To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD-Current , gary.jennejohn@freenet.de, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 16:19:49 -0000 On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin wrote: > On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: >> On Sun, 21 Mar 2010 19:03:56 +0200 >> Alexander Motin wrote: >> >> > Scott Long wrote: >> > > Are there non-CAM drivers that look at MAXPHYS, or that silently ass= ume > that >> > > MAXPHYS will never be more than 128k? >> > >> > That is a question. >> > >> >> I only did a quick&dirty grep looking for MAXPHYS in /sys. >> >> Some drivers redefine MAXPHYS to be 512KiB. =A0Some use their own local >> MAXPHYS which is usually 128KiB. >> >> Some look at MAXPHYS to figure out other things; the details escape me. >> >> There's one driver which actually uses 100*MAXPHYS for something, but I >> didn't check the details. >> >> Lots of them were non-CAM drivers AFAICT. > > The problem is the drivers that _don't_ reference MAXPHYS. =A0The driver = author > at the time "knew" that MAXPHYS was 128k, so he did the MAXPHYS-dependent > calculation and just put the result in the driver (e.g. only supporting u= p to > 32 segments (32 4k pages =3D=3D 128k) in a bus dma tag as a magic number = to > bus_dma_tag_create() w/o documenting that the '32' was derived from 128k = or > what the actual hardware limit on nsegments is). =A0These cannot be found= by a > simple grep, they require manually inspecting each driver. 100% awesome comment. On another kernel, I myself was guilty of this crime (I did have a nice comment though above the def). This has been a great thread since our application really needs some of the optimizations that are being thrown around here. We have found in real live performance testing that we are almost always either controller bound (i.e. adding more disks to spread IOPs has little to no effect in large array configurations on throughput, we suspect that is hitting the RAID controller's firmware limitations) or tps bound, i.e. I never thought going from 128k -> 256k per transaction would have a dramatic effect on throughput (but I never verified). Back to HBAs, AFAIK, every modern iteration of the most popular HBAs can easily do way more than a 128k scatter/gather I/O. Do you guys know of any *modern* (circa within the last 3-4 years) that can not do more than 128k at a shot? In other words, I've always thought the limit was kernel imposed and not what the memory controller on the card can do (I certainly never got the impression talking with some of the IHVs over the years that they were designing their hardware for a 128k limit - I sure hope not!). -aps From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 16:21:14 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0E0B1065672; Mon, 22 Mar 2010 16:21:14 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 7FE068FC26; Mon, 22 Mar 2010 16:21:14 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FF67.dip.t-dialin.net [217.84.255.103]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id C87A68454F4; Mon, 22 Mar 2010 17:21:07 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E2A7151D3; Mon, 22 Mar 2010 17:21:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269274865; bh=Lz/hhWC7H46mCvqPG0T5lJNEpPCfSd++IBGssQgt8u0=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=hphPN4gp1CNeS8kD9N+IJASW61wYkD3NQlOkFv1X92lKy+HpnCyQYV6GTG1aUbSYL 4CsWb0GAkoAL9JpISDJExhCAQYmZEp+NeK7wCfo2QOkv6feC6zW7bp16H9PbHXNsfc iugsUkA7glte+dhKD+lAepapr1Cp+DBOmQAQtgl2/xEXZzKSkqrcnJVSgZbtX0fsZD Uwf+Non2M/pqiWbYJEl9ecz85V+QPvkviln6y2aX5ap3frz+/65xHA2Vt6h9iGqWzA GIJc3sRUZp9aXw4iQjA5gxfbi/kqWKcW+enXj6A/VijBCGLQL87/YNytLzQyKGmZSJ 2eInPqy/TakJw== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2MGL4Sc096535; Mon, 22 Mar 2010 17:21:04 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Mon, 22 Mar 2010 17:21:04 +0100 Message-ID: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> Date: Mon, 22 Mar 2010 17:21:04 +0100 From: Alexander Leidinger To: John Baldwin References: <201003100812.29749.jhb@freebsd.org> <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> In-Reply-To: <201003220941.10525.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: C87A68454F4.835AB X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.84, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, J_CHICKENPOX_65 0.60) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269879669.19898@U6jF4bV8beRu6z8/rQjXFA X-EBL-Spam-Status: No Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 16:21:15 -0000 Quoting John Baldwin (from Mon, 22 Mar 2010 09:41:10 -0400): > On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: >> Redirecting from stable@ to arch@... >> >> Quoting John Baldwin (from Wed, 10 Mar 2010 08:12:29 > -0500): >> >> > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >> >> Quoting "Robert N. M. Watson" (from Tue, 9 Mar >> >> 2010 16:39:09 +0000): >> >> >> >> > >> >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >> >> > >> >> >>> From this you can see that sys.mk is included and parsed before >> > 'Makefile', >> >> >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. >> >> >> >> >> >> I think we need to find a different solution for this. The need to >> >> >> specify WITH_CTF at the command line is very error prone. :( >> >> > >> >> > You are neither the first person to have made this observation, nor >> >> > the first person to have failed to propose a solution in the form of >> >> > a patch :-). >> >> Ok, here is the proposal in form of a patch. :-) >> http://www.leidinger.net/test/ctf.diff >> >> > Unfortunately the ctf stuff breaks static binaries. I think that if >> > that were >> > fixed we would simply enable it by default and be done. >> >> The patch is: >> - enabling CTF stuff by default for the kernel >> - allows to disable the CTF stuff for the kernel by defining NO_CTF >> - *not* enabling the CTF stuff by default for libs and progs >> (if someone tells me how to distinguish the build for static >> stuff from dynamic stuff, I can have a look to enable it for >> the dynamic case) >> - allows to enable the CTF stuff for the userland by defining >> WITH_CTF as before > > I think this patch looks very interesting. I think in some ways it would be > nice to make CTF "opt-in" though instead of "opt-out". I think the current > patch would enable CTF when building ports, for example. I think instead it If you talk about kernel modules: yes, this should enable CTF there. If you talk about programs which use bsd.prog.mk or bsd.lib.mk: no, this will not enable CTF. The ports which use gmake will not be affected, I'm not sure about ports which use our make but not bsd.prog.mk or bsd.lib.mk. Anyone with an example of such a port which I could test? A quick query of portmgr (miwi) via IM didn't produce an obvious candidate port. > should default to not building CTF, but require an ENABLE_CTF (instead of > NO_CTF) to be set, and set that in bsd.kern.mk if WITH_CTF is defined. What about your previous "enabled by default" for kernel+world (yes, my patch is asymmetric in that it only enables the kernel part as the result of static userland stuff seems to have a problem), what's the reason for the switch to opt-in? Normally we use MK_xxx for things which are opt-in/opt-out. What about using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, what should the xxx part look like? Is bsd.kern.mk included in module builds too? To make sure I get your and Scott's points right: - all opt-in - enabled for the kernel/mods via "makeoptions WITH_CTF=yes" in the kernel config instead of enabling it by default (maybe in bsd.kern.mk?) Note: the NO_CTF part is existing stuff, I probably would have to fix other places too then. The current patch is a minimal patch to opt-out for kernel builds and opt-in for prog/lib parts. The ports area needs to be investigated (if nothing is affected, nothing needs to be taken into account). I have a look at getting some time this week to rework the patch according to the outcome of the discussion here. Bye, Alexander. -- This fortune would be seven words long if it were six words shorter. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 16:27:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93FC1106566B; Mon, 22 Mar 2010 16:27:19 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 5A26B8FC0C; Mon, 22 Mar 2010 16:27:19 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MGRFPX050285; Mon, 22 Mar 2010 10:27:15 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <3c0b01821003220852r61ca0ae3o95bea1c23ddc34d9@mail.gmail.com> Date: Mon, 22 Mar 2010 10:27:15 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <50456989-F196-4907-A170-85806A73D25F@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <4BA6517C.3050509@FreeBSD.org> <20100322124018.7430f45e@ernst.jennejohn.org> <201003220839.12907.jhb@freebsd.org> <3c0b01821003220852r61ca0ae3o95bea1c23ddc34d9@mail.gmail.com> To: Alexander Sack X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: FreeBSD-Current , gary.jennejohn@freenet.de, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 16:27:19 -0000 On Mar 22, 2010, at 9:52 AM, Alexander Sack wrote: > On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin wrote: >> On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: >>> On Sun, 21 Mar 2010 19:03:56 +0200 >>> Alexander Motin wrote: >>>=20 >>>> Scott Long wrote: >>>>> Are there non-CAM drivers that look at MAXPHYS, or that silently = assume >> that >>>>> MAXPHYS will never be more than 128k? >>>>=20 >>>> That is a question. >>>>=20 >>>=20 >>> I only did a quick&dirty grep looking for MAXPHYS in /sys. >>>=20 >>> Some drivers redefine MAXPHYS to be 512KiB. Some use their own = local >>> MAXPHYS which is usually 128KiB. >>>=20 >>> Some look at MAXPHYS to figure out other things; the details escape = me. >>>=20 >>> There's one driver which actually uses 100*MAXPHYS for something, = but I >>> didn't check the details. >>>=20 >>> Lots of them were non-CAM drivers AFAICT. >>=20 >> The problem is the drivers that _don't_ reference MAXPHYS. The = driver author >> at the time "knew" that MAXPHYS was 128k, so he did the = MAXPHYS-dependent >> calculation and just put the result in the driver (e.g. only = supporting up to >> 32 segments (32 4k pages =3D=3D 128k) in a bus dma tag as a magic = number to >> bus_dma_tag_create() w/o documenting that the '32' was derived from = 128k or >> what the actual hardware limit on nsegments is). These cannot be = found by a >> simple grep, they require manually inspecting each driver. >=20 > 100% awesome comment. On another kernel, I myself was guilty of this > crime (I did have a nice comment though above the def). >=20 > This has been a great thread since our application really needs some > of the optimizations that are being thrown around here. We have found > in real live performance testing that we are almost always either > controller bound (i.e. adding more disks to spread IOPs has little to > no effect in large array configurations on throughput, we suspect that > is hitting the RAID controller's firmware limitations) or tps bound, > i.e. I never thought going from 128k -> 256k per transaction would > have a dramatic effect on throughput (but I never verified). >=20 > Back to HBAs, AFAIK, every modern iteration of the most popular HBAs > can easily do way more than a 128k scatter/gather I/O. Do you guys > know of any *modern* (circa within the last 3-4 years) that can not do > more than 128k at a shot? >64K broken in MPT at the moment. The hardware can do it, the driver = thinks it can do it, but it fails. AAC hardware traditionally cannot, = but maybe the firmware has been improved in the past few years. I know = that there are other low-performance devices that can't do more than 64 = or 128K, but none are coming to mind at the moment. Still, it shouldn't = be a universal assumption that all hardware can do big I/O's. Another consideration is that some hardware can do big I/O's, but not = very efficiently. Not all DMA engines are created equal, and moving to = compound commands and excessively long S/G lists can be a pessimization. = For example, MFI hardware does a hinted prefetch on the segment list, = but once you exceed a certain limit, that prefetch doesn't work anymore = and the firmware has to take the slow path to execute the i/o. I = haven't quantified this penalty yet, but it's something that should be = thought about. >=20 > In other words, I've always thought the limit was kernel imposed and > not what the memory controller on the card can do (I certainly never > got the impression talking with some of the IHVs over the years that > they were designing their hardware for a 128k limit - I sure hope > not!). You'd be surprised at the engineering compromises and handicaps that are = committed at IHVs because of misguided marketters. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 16:30:44 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AB24106566C; Mon, 22 Mar 2010 16:30:44 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id A76358FC18; Mon, 22 Mar 2010 16:30:43 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MGUdOi050325; Mon, 22 Mar 2010 10:30:39 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> Date: Mon, 22 Mar 2010 10:30:39 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201003100812.29749.jhb@freebsd.org> <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> To: Alexander Leidinger X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 16:30:44 -0000 On Mar 22, 2010, at 10:21 AM, Alexander Leidinger wrote: > Quoting John Baldwin (from Mon, 22 Mar 2010 09:41:10 = -0400): >=20 >> On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: >>> Redirecting from stable@ to arch@... >>>=20 >>> Quoting John Baldwin (from Wed, 10 Mar 2010 = 08:12:29 >> -0500): >>>=20 >>> > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >>> >> Quoting "Robert N. M. Watson" (from Tue, 9 = Mar >>> >> 2010 16:39:09 +0000): >>> >> >>> >> > >>> >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >>> >> > >>> >> >>> =46rom this you can see that sys.mk is included and parsed = before >>> > 'Makefile', >>> >> >>> so the WITH_CTF=3Dyes is not set until after sys.mk has been = parsed. >>> >> >> >>> >> >> I think we need to find a different solution for this. The = need to >>> >> >> specify WITH_CTF at the command line is very error prone. :( >>> >> > >>> >> > You are neither the first person to have made this observation, = nor >>> >> > the first person to have failed to propose a solution in the = form of >>> >> > a patch :-). >>>=20 >>> Ok, here is the proposal in form of a patch. :-) >>> http://www.leidinger.net/test/ctf.diff >>>=20 >>> > Unfortunately the ctf stuff breaks static binaries. I think that = if >>> > that were >>> > fixed we would simply enable it by default and be done. >>>=20 >>> The patch is: >>> - enabling CTF stuff by default for the kernel >>> - allows to disable the CTF stuff for the kernel by defining NO_CTF >>> - *not* enabling the CTF stuff by default for libs and progs >>> (if someone tells me how to distinguish the build for static >>> stuff from dynamic stuff, I can have a look to enable it for >>> the dynamic case) >>> - allows to enable the CTF stuff for the userland by defining >>> WITH_CTF as before >>=20 >> I think this patch looks very interesting. I think in some ways it = would be >> nice to make CTF "opt-in" though instead of "opt-out". I think the = current >> patch would enable CTF when building ports, for example. I think = instead it >=20 > If you talk about kernel modules: yes, this should enable CTF there. > If you talk about programs which use bsd.prog.mk or bsd.lib.mk: no, = this will not enable CTF. >=20 > The ports which use gmake will not be affected, I'm not sure about = ports which use our make but not bsd.prog.mk or bsd.lib.mk. Anyone with = an example of such a port which I could test? A quick query of portmgr = (miwi) via IM didn't produce an obvious candidate port. >=20 >> should default to not building CTF, but require an ENABLE_CTF = (instead of >> NO_CTF) to be set, and set that in bsd.kern.mk if WITH_CTF is = defined. >=20 > What about your previous "enabled by default" for kernel+world (yes, = my patch is asymmetric in that it only enables the kernel part as the = result of static userland stuff seems to have a problem), what's the = reason for the switch to opt-in? >=20 > Normally we use MK_xxx for things which are opt-in/opt-out. What about = using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, = what should the xxx part look like? >=20 > Is bsd.kern.mk included in module builds too? >=20 > To make sure I get your and Scott's points right: > - all opt-in > - enabled for the kernel/mods via "makeoptions WITH_CTF=3Dyes" in the > kernel config instead of enabling it by default (maybe in > bsd.kern.mk?) >=20 > Note: the NO_CTF part is existing stuff, I probably would have to fix = other places too then. The current patch is a minimal patch to opt-out = for kernel builds and opt-in for prog/lib parts. The ports area needs to = be investigated (if nothing is affected, nothing needs to be taken into = account). >=20 I think it's still important to be opt-in for dtrace. If you're = reluctant, I'll submit my patch which is opt-in, and is only about 10 = lines of code change. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 17:41:03 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DF08D106566C; Mon, 22 Mar 2010 17:41:02 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 973488FC0C; Mon, 22 Mar 2010 17:41:02 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 300C446B46; Mon, 22 Mar 2010 13:41:02 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3B9D48A026; Mon, 22 Mar 2010 13:41:01 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Mon, 22 Mar 2010 13:38:28 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> In-Reply-To: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003221338.28130.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 22 Mar 2010 13:41:01 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 17:41:03 -0000 On Monday 22 March 2010 12:21:04 pm Alexander Leidinger wrote: > Quoting John Baldwin (from Mon, 22 Mar 2010 09:41:10 -0400): > > > On Monday 22 March 2010 7:34:08 am Alexander Leidinger wrote: > >> Redirecting from stable@ to arch@... > >> > >> Quoting John Baldwin (from Wed, 10 Mar 2010 08:12:29 > > -0500): > >> > >> > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: > >> >> Quoting "Robert N. M. Watson" (from Tue, 9 Mar > >> >> 2010 16:39:09 +0000): > >> >> > >> >> > > >> >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: > >> >> > > >> >> >>> From this you can see that sys.mk is included and parsed before > >> > 'Makefile', > >> >> >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. > >> >> >> > >> >> >> I think we need to find a different solution for this. The need to > >> >> >> specify WITH_CTF at the command line is very error prone. :( > >> >> > > >> >> > You are neither the first person to have made this observation, nor > >> >> > the first person to have failed to propose a solution in the form of > >> >> > a patch :-). > >> > >> Ok, here is the proposal in form of a patch. :-) > >> http://www.leidinger.net/test/ctf.diff > >> > >> > Unfortunately the ctf stuff breaks static binaries. I think that if > >> > that were > >> > fixed we would simply enable it by default and be done. > >> > >> The patch is: > >> - enabling CTF stuff by default for the kernel > >> - allows to disable the CTF stuff for the kernel by defining NO_CTF > >> - *not* enabling the CTF stuff by default for libs and progs > >> (if someone tells me how to distinguish the build for static > >> stuff from dynamic stuff, I can have a look to enable it for > >> the dynamic case) > >> - allows to enable the CTF stuff for the userland by defining > >> WITH_CTF as before > > > > I think this patch looks very interesting. I think in some ways it would be > > nice to make CTF "opt-in" though instead of "opt-out". I think the current > > patch would enable CTF when building ports, for example. I think instead it > > If you talk about kernel modules: yes, this should enable CTF there. > If you talk about programs which use bsd.prog.mk or bsd.lib.mk: no, > this will not enable CTF. > > The ports which use gmake will not be affected, I'm not sure about > ports which use our make but not bsd.prog.mk or bsd.lib.mk. Anyone > with an example of such a port which I could test? A quick query of > portmgr (miwi) via IM didn't produce an obvious candidate port. Just about any port that doesn't use gmake. Also, anyone who writes a foo.c and types 'make foo' to get automatic rules. I think we should only enable CTF in stages. To do that safely, it needs to be opt-in in sys.mk. We can then work on changing the defaults in certain places (e.g. bsd.kern.mk and bsd.kmod.mk) to enable it for kernel builds and modules. Eventually we could enable it by default in bsd.prog.mk and bsd.lib.mk if desired, but I think those should be future steps. I think your patch has promise as being a good approach, but it ctf needs to be off-by-default in sys.mk. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 17:50:49 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1917A1065675; Mon, 22 Mar 2010 17:50:49 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id C70828FC14; Mon, 22 Mar 2010 17:50:48 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MHojY3057191; Mon, 22 Mar 2010 11:50:45 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <201003221338.28130.jhb@freebsd.org> Date: Mon, 22 Mar 2010 11:50:45 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <1C19222B-71A9-4A46-BF62-DD91CCE2923D@samsco.org> References: <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> <201003221338.28130.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: Alexander Leidinger , "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 17:50:49 -0000 On Mar 22, 2010, at 11:38 AM, John Baldwin wrote: >=20 > Just about any port that doesn't use gmake. Also, anyone who writes a = foo.c=20 > and types 'make foo' to get automatic rules. I think we should only = enable=20 > CTF in stages. To do that safely, it needs to be opt-in in sys.mk. = We can=20 > then work on changing the defaults in certain places (e.g. bsd.kern.mk = and=20 > bsd.kmod.mk) to enable it for kernel builds and modules. Eventually = we could=20 > enable it by default in bsd.prog.mk and bsd.lib.mk if desired, but I = think=20 > those should be future steps. I think your patch has promise as being = a good=20 > approach, but it ctf needs to be off-by-default in sys.mk. Or just removed from sys.mk entirely and moved into appropriate bsd.*.mk = files as appropriate. I don't like that DTrace, which is encumbered as = part of the CDDL, is being treated as core FreeBSD functionality. It's = not. Scott From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 18:29:01 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D75F71065675; Mon, 22 Mar 2010 18:29:01 +0000 (UTC) (envelope-from js@saltmine.radix.net) Received: from saltmine.radix.net (saltmine.radix.net [207.192.128.40]) by mx1.freebsd.org (Postfix) with ESMTP id 72A628FC15; Mon, 22 Mar 2010 18:29:01 +0000 (UTC) Received: from saltmine.radix.net (localhost [127.0.0.1]) by saltmine.radix.net (8.12.2/8.12.2) with ESMTP id o2MIAIV4020347; Mon, 22 Mar 2010 14:10:18 -0400 (EDT) Received: (from root@localhost) by saltmine.radix.net (8.12.2/8.12.2/Submit) id o2MIAHrY020345; Mon, 22 Mar 2010 14:10:17 -0400 (EDT) Received: from mail1.radix.net (mail1.radix.net [207.192.128.31]) by saltmine.radix.net (8.12.2/8.12.2) with ESMTP id o2MGS2V4005671 for ; Mon, 22 Mar 2010 12:28:02 -0400 (EDT) Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by mail1.radix.net (8.13.4/8.13.4) with ESMTP id o2MGS2ur027054 for ; Mon, 22 Mar 2010 12:28:02 -0400 (EDT) Received: from hub.freebsd.org (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 9F2231A9325; Mon, 22 Mar 2010 16:27:34 +0000 (UTC) Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 5FFA5106578E; Mon, 22 Mar 2010 16:27:32 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93FC1106566B; Mon, 22 Mar 2010 16:27:19 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 5A26B8FC0C; Mon, 22 Mar 2010 16:27:19 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2MGRFPX050285; Mon, 22 Mar 2010 10:27:15 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <3c0b01821003220852r61ca0ae3o95bea1c23ddc34d9@mail.gmail.com> Date: Mon, 22 Mar 2010 10:27:15 -0600 Message-Id: <50456989-F196-4907-A170-85806A73D25F@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <4BA6517C.3050509@FreeBSD.org> <20100322124018.7430f45e@ernst.jennejohn.org> <201003220839.12907.jhb@freebsd.org> <3c0b01821003220852r61ca0ae3o95bea1c23ddc34d9@mail.gmail.com> To: Alexander Sack X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by saltmine.radix.net id o2MGS2V4005671 Status: O X-Status: X-Keywords: X-UID: 162 Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 18:29:01 -0000 On Mar 22, 2010, at 9:52 AM, Alexander Sack wrote: > On Mon, Mar 22, 2010 at 8:39 AM, John Baldwin wrote: >> On Monday 22 March 2010 7:40:18 am Gary Jennejohn wrote: >>> On Sun, 21 Mar 2010 19:03:56 +0200 >>> Alexander Motin wrote: >>> >>>> Scott Long wrote: >>>>> Are there non-CAM drivers that look at MAXPHYS, or that silently assume >> that >>>>> MAXPHYS will never be more than 128k? >>>> >>>> That is a question. >>>> >>> >>> I only did a quick&dirty grep looking for MAXPHYS in /sys. >>> >>> Some drivers redefine MAXPHYS to be 512KiB. Some use their own local >>> MAXPHYS which is usually 128KiB. >>> >>> Some look at MAXPHYS to figure out other things; the details escape me. >>> >>> There's one driver which actually uses 100*MAXPHYS for something, but I >>> didn't check the details. >>> >>> Lots of them were non-CAM drivers AFAICT. >> >> The problem is the drivers that _don't_ reference MAXPHYS. The driver author >> at the time "knew" that MAXPHYS was 128k, so he did the MAXPHYS-dependent >> calculation and just put the result in the driver (e.g. only supporting up to >> 32 segments (32 4k pages == 128k) in a bus dma tag as a magic number to >> bus_dma_tag_create() w/o documenting that the '32' was derived from 128k or >> what the actual hardware limit on nsegments is). These cannot be found by a >> simple grep, they require manually inspecting each driver. > > 100% awesome comment. On another kernel, I myself was guilty of this > crime (I did have a nice comment though above the def). > > This has been a great thread since our application really needs some > of the optimizations that are being thrown around here. We have found > in real live performance testing that we are almost always either > controller bound (i.e. adding more disks to spread IOPs has little to > no effect in large array configurations on throughput, we suspect that > is hitting the RAID controller's firmware limitations) or tps bound, > i.e. I never thought going from 128k -> 256k per transaction would > have a dramatic effect on throughput (but I never verified). > > Back to HBAs, AFAIK, every modern iteration of the most popular HBAs > can easily do way more than a 128k scatter/gather I/O. Do you guys > know of any *modern* (circa within the last 3-4 years) that can not do > more than 128k at a shot? >64K broken in MPT at the moment. The hardware can do it, the driver thinks it can do it, but it fails. AAC hardware traditionally cannot, but maybe the firmware has been improved in the past few years. I know that there are other low-performance devices that can't do more than 64 or 128K, but none are coming to mind at the moment. Still, it shouldn't be a universal assumption that all hardware can do big I/O's. Another consideration is that some hardware can do big I/O's, but not very efficiently. Not all DMA engines are created equal, and moving to compound commands and excessively long S/G lists can be a pessimization. For example, MFI hardware does a hinted prefetch on the segment list, but once you exceed a certain limit, that prefetch doesn't work anymore and the firmware has to take the slow path to execute the i/o. I haven't quantified this penalty yet, but it's something that should be thought about. > > In other words, I've always thought the limit was kernel imposed and > not what the memory controller on the card can do (I certainly never > got the impression talking with some of the IHVs over the years that > they were designing their hardware for a 128k limit - I sure hope > not!). You'd be surprised at the engineering compromises and handicaps that are committed at IHVs because of misguided marketters. Scott _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 18:50:41 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32E5A106566B; Mon, 22 Mar 2010 18:50:41 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id C8DA68FC1C; Mon, 22 Mar 2010 18:50:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2MIj5Zf028027; Mon, 22 Mar 2010 12:45:05 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 12:45:05 -0600 (MDT) Message-Id: <20100322.124505.787670930858384500.imp@bsdimp.com> To: scottl@samsco.org From: "M. Warner Losh" In-Reply-To: References: <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: mav@freebsd.org, freebsd-current@freebsd.org, ivoras@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 18:50:41 -0000 In message: Scott Long writes: : I'd like to go in the opposite direction. The queue-dispatch-queue : model of GEOM is elegant and easy to extend, but very wasteful for : the simple case, where the simple case is one or two simple : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror : transform. None of these need a dedicated dispatch context in order : to operate. What I'd like to explore is compiling the GEOM stack at : creation time into a linear array of operations that happen without : a g_down/g_up context switch. As providers and consumers taste each : other and build a stack, that stack gets compiled into a graph, and : that graph gets executed directly from the calling context, both : from the dev_strategy() side on the top and the bio_done() on the : bottom. GEOM classes that need a detached context can mark : themselves as such, doing so will prevent a graph from being : created, and the current dispatch model will be retained. I have a few things to say on this. First, I've done similar things at past companies for systems that are similar to geom's queueing environment. It is possible to convert the queueing nodes in the graph to filtering nodes in the graph. Another way to look at this is to say you're implementing direct dispatch into geom's stack. This can be both good and bad, but should reduce latency a lot. One problem that I see is that you are calling into the driver from a different set of contexts. The queueing stuff was there to protect the driver from LoRs due to its routines being called from many different contexts, sometimes with other locks held (fact of life often in the kernel). So this certainly is something worth exploring, especially if we have optimized paths for up/down for certain geom classes while still allowing the current robust, but slow, paths for the more complicated nodes in the tree. It remains to be see if there's going to be issues around locking order, but we've hit that with both geom and ifnet in the past, so caution (eg, running with WITNESS turned on early and often) is advised. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:00:13 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 592961065670; Mon, 22 Mar 2010 19:00:13 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 15B158FC19; Mon, 22 Mar 2010 19:00:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2MIqLSh028159; Mon, 22 Mar 2010 12:52:21 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 12:52:22 -0600 (MDT) Message-Id: <20100322.125222.690091871650369537.imp@bsdimp.com> To: des@des.no From: "M. Warner Losh" In-Reply-To: <86y6hkiien.fsf@ds4.des.no> References: <4BA3E9DF.2050303@delphij.net> <86y6hkiien.fsf@ds4.des.no> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: deischen@freebsd.org, scottl@samsco.org, d@delphij.net, julian@elischer.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:00:13 -0000 In message: <86y6hkiien.fsf@ds4.des.no> Dag-Erling Sm=F8rgrav writes: : Daniel Eischen writes: : > Perhaps I was wrong, but I thought Scott's question was more genera= l: : > is there a desire for a special installation suitable to small : > appliances (usually flash-based)? : = : Not all "small appliances" are flash-based; the Geode-based soekris c= an : use 2.5" disks and has enough memory to run a stock install. The onl= y : trouble I've had is with /etc/periodic/daily/470.status-named, which : uses an ungodly amount of memory. The AMD Geode is more or less : equivalent to a Pentium MMX, and does not have SSE. : = : Soekris net4801-48: : = : CPU: Geode(TM) Integrated Processor by National Semi (266.65-MHz 586-= class CPU) : Origin =3D "Geode by NSC" Id =3D 0x540 Stepping =3D 0 : Features=3D0x808131 : real memory =3D 134217728 (128 MB) : avail memory =3D 126160896 (120 MB) : = : Soekris net5501-70: : = : CPU: Geode(TM) Integrated Processor by AMD PCS (499.91-MHz 586-class = CPU) : Origin =3D "AuthenticAMD" Id =3D 0x5a2 Stepping =3D 2 : Features=3D0x88a93d : AMD Features=3D0xc0400000 : real memory =3D 536870912 (512 MB) : avail memory =3D 510726144 (487 MB) You don't really get into the small to flash install realm until you start hitting some of the !x86 embedded boxes. There, you have to have a hand-crafted firmware load for each box or a limited range of boxes. This is even true in linux land. You'll never use sysinstall or anything remotely like it to install onto a Linksys WRT55G. you're going to give the firmware loader a new firmware image via http or simething like that. So for the x86 case, there's not a good argument to be made for hyperoptimizing GENERIC. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:07:58 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDD4C106566B; Mon, 22 Mar 2010 19:07:57 +0000 (UTC) (envelope-from pisymbol@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 976828FC1B; Mon, 22 Mar 2010 19:07:57 +0000 (UTC) Received: by pvc7 with SMTP id 7so1725188pvc.13 for ; Mon, 22 Mar 2010 12:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=IAbwQDXKWD9NlEl/9OqmLYW9cBIGQ2VBgkLOBd+Tk4Y=; b=nhMIq7dphzai8YsLQY2aGid9Rt8VCoTK/veYPs7rDiu0MWDg1v4aNttBUygAelvpLb aPAaKk85PCqWL2aNBdNbVCy04T8PdvkB9kYXWYu6gpE0Ko3pWRS/WvH++XNuDSc0yVON mcMsvA0/pTu32F9rCm6OCnsdq2y0gA4rPNGJo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=QIyl9hwMBbqWg9rep6rl6Y4BrR6gG3k1vmhvrzJcBUUQB4CEpswh33WNWs5UO+2G/w 9URo3ZMrL5LmXt/IbFQ5ucHGbr/ouVIKs9CqoxaACn4XtlhH39Er7Kz1gqrC8de38dOO ou4ewQmsBFPrIKjg9ayOXpAccKERhk6KS+vmg= MIME-Version: 1.0 Received: by 10.115.103.40 with SMTP id f40mr1014095wam.38.1269284877017; Mon, 22 Mar 2010 12:07:57 -0700 (PDT) In-Reply-To: <20100322.124505.787670930858384500.imp@bsdimp.com> References: <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> <20100322.124505.787670930858384500.imp@bsdimp.com> Date: Mon, 22 Mar 2010 15:07:56 -0400 Message-ID: <3c0b01821003221207p4e4eecabqb4f448813bf5a8a8@mail.gmail.com> From: Alexander Sack To: "M. Warner Losh" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org, scottl@samsco.org, freebsd-current@freebsd.org, mav@freebsd.org, ivoras@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:07:58 -0000 On Mon, Mar 22, 2010 at 2:45 PM, M. Warner Losh wrote: > In message: > =A0 =A0 =A0 =A0 =A0 =A0Scott Long writes: > : I'd like to go in the opposite direction. =A0The queue-dispatch-queue > : model of GEOM is elegant and easy to extend, but very wasteful for > : the simple case, where the simple case is one or two simple > : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror > : transform. =A0None of these need a dedicated dispatch context in order > : to operate. =A0What I'd like to explore is compiling the GEOM stack at > : creation time into a linear array of operations that happen without > : a g_down/g_up context switch. =A0As providers and consumers taste each > : other and build a stack, that stack gets compiled into a graph, and > : that graph gets executed directly from the calling context, both > : from the dev_strategy() side on the top and the bio_done() on the > : bottom. =A0GEOM classes that need a detached context can mark > : themselves as such, doing so will prevent a graph from being > : created, and the current dispatch model will be retained. > > I have a few things to say on this. > > First, I've done similar things at past companies for systems that are > similar to geom's queueing environment. =A0It is possible to convert the > queueing nodes in the graph to filtering nodes in the graph. =A0Another > way to look at this is to say you're implementing direct dispatch into > geom's stack. =A0This can be both good and bad, but should reduce > latency a lot. > > One problem that I see is that you are calling into the driver from a > different set of contexts. =A0The queueing stuff was there to protect > the driver from LoRs due to its routines being called from many > different contexts, sometimes with other locks held (fact of life > often in the kernel). > > So this certainly is something worth exploring, especially if we have > optimized paths for up/down for certain geom classes while still > allowing the current robust, but slow, paths for the more complicated > nodes in the tree. =A0It remains to be see if there's going to be issues > around locking order, but we've hit that with both geom and ifnet in > the past, so caution (eg, running with WITNESS turned on early and > often) is advised. Am I going crazy or does this sound a lot like Sun/SVR's stream based network stack? (design and problems, stream stack locking was notoriously tricky for the exact issue mentioned above, different running contexts with different locking granularity/requirements). -aps From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:08:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4691A1065670; Mon, 22 Mar 2010 19:08:43 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 06DAC8FC28; Mon, 22 Mar 2010 19:08:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2MIxbod028271; Mon, 22 Mar 2010 12:59:37 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 12:59:37 -0600 (MDT) Message-Id: <20100322.125937.278730673160410010.imp@bsdimp.com> To: Alexander@Leidinger.net From: "M. Warner Losh" In-Reply-To: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> References: <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:08:43 -0000 In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> Alexander Leidinger writes: : Normally we use MK_xxx for things which are opt-in/opt-out. What about : using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, : what should the xxx part look like? Normally we *TEST* MK_XXX for things which are opt-in/opt-out and require the user to say WITH_XXX or WITHOUT_XXX if they don't like the default (or want to ensure they get option XXX, even if we turn it off by default in the future). The default then gets encoded in bsd.own.mk, and permeates the FreeBSD build system since we include that everywhere, directly or indirectly. The problem is that bsd.own.mk is not included in sys.mk, nor should it be. That's why we have the hacky combination of WITH_CTF and NO_CTF that's there today. : Is bsd.kern.mk included in module builds too? Yes. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:08:51 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97575106566B; Mon, 22 Mar 2010 19:08:51 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 576838FC25; Mon, 22 Mar 2010 19:08:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2MJ5B60028317; Mon, 22 Mar 2010 13:05:11 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 13:05:12 -0600 (MDT) Message-Id: <20100322.130512.864843819464264610.imp@bsdimp.com> To: Alexander@Leidinger.net From: "M. Warner Losh" In-Reply-To: <20100322.125937.278730673160410010.imp@bsdimp.com> References: <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> <20100322.125937.278730673160410010.imp@bsdimp.com> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:08:51 -0000 In message: <20100322.125937.278730673160410010.imp@bsdimp.com> M. Warner Losh writes: : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> : Alexander Leidinger writes: : : Normally we use MK_xxx for things which are opt-in/opt-out. What about : : using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, : : what should the xxx part look like? : : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and : require the user to say WITH_XXX or WITHOUT_XXX if they don't like the : default (or want to ensure they get option XXX, even if we turn it off : by default in the future). The default then gets encoded in : bsd.own.mk, and permeates the FreeBSD build system since we include : that everywhere, directly or indirectly. : : The problem is that bsd.own.mk is not included in sys.mk, nor should : it be. That's why we have the hacky combination of WITH_CTF and : NO_CTF that's there today. : : : Is bsd.kern.mk included in module builds too? : : Yes. One last thing I should have said was that the patch that was posted earlier in the thread looked ok, and likely couldn't be made significantly better due to the bsd.own.mk issue. Warner P.S. Sorry for the lame followup to my own post. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:20:20 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47E111065670; Mon, 22 Mar 2010 19:20:20 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 07D2F8FC0C; Mon, 22 Mar 2010 19:20:18 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 6E85E1FFC59; Mon, 22 Mar 2010 19:20:17 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 385C1844F3; Mon, 22 Mar 2010 20:20:17 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: "M. Warner Losh" References: <4BA3E9DF.2050303@delphij.net> <86y6hkiien.fsf@ds4.des.no> <20100322.125222.690091871650369537.imp@bsdimp.com> Date: Mon, 22 Mar 2010 20:20:17 +0100 In-Reply-To: <20100322.125222.690091871650369537.imp@bsdimp.com> (M. Warner Losh's message of "Mon, 22 Mar 2010 12:52:22 -0600 (MDT)") Message-ID: <867hp4i2oe.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: deischen@freebsd.org, scottl@samsco.org, d@delphij.net, julian@elischer.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:20:20 -0000 "M. Warner Losh" writes: > Dag-Erling Sm=C3=B8rgrav writes: > > Not all "small appliances" are flash-based; the Geode-based soekris can > > use 2.5" disks and has enough memory to run a stock install. The only > > trouble I've had is with /etc/periodic/daily/470.status-named, which > > uses an ungodly amount of memory. The AMD Geode is more or less > > equivalent to a Pentium MMX, and does not have SSE. > You don't really get into the small to flash install realm until you > start hitting some of the !x86 embedded boxes. There, you have to > have a hand-crafted firmware load for each box or a limited range of > boxes. I know, I know, I know. My point is that not all !SSE machines are tiny embedded systems that can't run GENERIC. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 19:42:54 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5A641065686; Mon, 22 Mar 2010 19:42:54 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 727BE8FC1F; Mon, 22 Mar 2010 19:42:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2MJV1DM028767; Mon, 22 Mar 2010 13:31:02 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 13:31:02 -0600 (MDT) Message-Id: <20100322.133102.756786594490272788.imp@bsdimp.com> To: des@des.no From: "M. Warner Losh" In-Reply-To: <867hp4i2oe.fsf@ds4.des.no> References: <86y6hkiien.fsf@ds4.des.no> <20100322.125222.690091871650369537.imp@bsdimp.com> <867hp4i2oe.fsf@ds4.des.no> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: deischen@freebsd.org, scottl@samsco.org, d@delphij.net, julian@elischer.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 19:42:54 -0000 In message: <867hp4i2oe.fsf@ds4.des.no> Dag-Erling Sm=F8rgrav writes: : "M. Warner Losh" writes: : > Dag-Erling Sm=F8rgrav writes: : > > Not all "small appliances" are flash-based; the Geode-based soekr= is can : > > use 2.5" disks and has enough memory to run a stock install. The= only : > > trouble I've had is with /etc/periodic/daily/470.status-named, wh= ich : > > uses an ungodly amount of memory. The AMD Geode is more or less : > > equivalent to a Pentium MMX, and does not have SSE. : > You don't really get into the small to flash install realm until yo= u : > start hitting some of the !x86 embedded boxes. There, you have to : > have a hand-crafted firmware load for each box or a limited range o= f : > boxes. : = : I know, I know, I know. My point is that not all !SSE machines are t= iny : embedded systems that can't run GENERIC. Agreed. Most systems in use today that are !SSE are pretty beefy. Warner From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 20:42:41 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 018441065676; Mon, 22 Mar 2010 20:42:41 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id B66208FC2B; Mon, 22 Mar 2010 20:42:40 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 7E9D66440; Mon, 22 Mar 2010 20:42:39 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.3/8.14.3) with ESMTP id o2MKgbeS034478; Mon, 22 Mar 2010 20:42:37 GMT (envelope-from phk@critter.freebsd.dk) To: Alexander Sack From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 22 Mar 2010 15:07:56 -0400." <3c0b01821003221207p4e4eecabqb4f448813bf5a8a8@mail.gmail.com> Date: Mon, 22 Mar 2010 20:42:37 +0000 Message-ID: <34477.1269290557@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: scottl@samsco.org, mav@freebsd.org, freebsd-current@freebsd.org, ivoras@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 20:42:41 -0000 In message <3c0b01821003221207p4e4eecabqb4f448813bf5a8a8@mail.gmail.com>, Alexa nder Sack writes: >Am I going crazy or does this sound a lot like Sun/SVR's stream based >network stack? That is a good and pertinent observation. I did investigate a number of optimizations to the g_up/g_down scheme I eventually adopted, but found none that gained anything justifying the complexity they brought. In some cases, the optimizations used more CPU cycles than the straight g_up/g_down path, but obviously, the circumstances are vastly different with CPUs having 10 times higher clock, multiple cores and SSD disks, so a fresh look at this tradeoff is in order. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 21:08:05 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C8C11065670; Mon, 22 Mar 2010 21:08:05 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id F2FA48FC12; Mon, 22 Mar 2010 21:08:04 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 8127846B03; Mon, 22 Mar 2010 17:08:04 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A881A8A01F; Mon, 22 Mar 2010 17:08:03 -0400 (EDT) From: John Baldwin To: "M. Warner Losh" Date: Mon, 22 Mar 2010 16:05:24 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201003220941.10525.jhb@freebsd.org> <20100322.125937.278730673160410010.imp@bsdimp.com> <20100322.130512.864843819464264610.imp@bsdimp.com> In-Reply-To: <20100322.130512.864843819464264610.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003221605.24538.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 22 Mar 2010 17:08:03 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Alexander@leidinger.net, rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 21:08:05 -0000 On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> > M. Warner Losh writes: > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> > : Alexander Leidinger writes: > : : Normally we use MK_xxx for things which are opt-in/opt-out. What about > : : using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, > : : what should the xxx part look like? > : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like the > : default (or want to ensure they get option XXX, even if we turn it off > : by default in the future). The default then gets encoded in > : bsd.own.mk, and permeates the FreeBSD build system since we include > : that everywhere, directly or indirectly. > : > : The problem is that bsd.own.mk is not included in sys.mk, nor should > : it be. That's why we have the hacky combination of WITH_CTF and > : NO_CTF that's there today. > : > : : Is bsd.kern.mk included in module builds too? > : > : Yes. > > One last thing I should have said was that the patch that was posted > earlier in the thread looked ok, and likely couldn't be made > significantly better due to the bsd.own.mk issue. I think the patch is a good approach, I just think it needs to default to not enabling CTF by default. Instead, various bsd.foo.mk should selectively enable it. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 23:55:24 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A850106564A for ; Mon, 22 Mar 2010 23:55:24 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id BD3768FC1D for ; Mon, 22 Mar 2010 23:55:23 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B643045CD8; Tue, 23 Mar 2010 00:36:13 +0100 (CET) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 2A9F845CA0; Tue, 23 Mar 2010 00:36:08 +0100 (CET) Date: Tue, 23 Mar 2010 00:36:07 +0100 From: Pawel Jakub Dawidek To: Poul-Henning Kamp Message-ID: <20100322233607.GB1767@garage.freebsd.pl> References: <4BA633A0.2090108@icyb.net.ua> <5754.1269246223@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LpQ9ahxlCli8rRTG" Content-Disposition: inline In-Reply-To: <5754.1269246223@critter.freebsd.dk> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Alexander Motin , freebsd-current@FreeBSD.org, Andriy Gapon , freebsd-arch@FreeBSD.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 23:55:24 -0000 --LpQ9ahxlCli8rRTG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 22, 2010 at 08:23:43AM +0000, Poul-Henning Kamp wrote: > In message <4BA633A0.2090108@icyb.net.ua>, Andriy Gapon writes: > >on 21/03/2010 16:05 Alexander Motin said the following: > >> Ivan Voras wrote: > >>> Hmm, it looks like it could be easy to spawn more g_* threads (and, > >>> barring specific class behaviour, it has a fair chance of working out= of > >>> the box) but the incoming queue will need to also be broken up for > >>> greater effect. > >>=20 > >> According to "notes", looks there is a good chance to obtain races, as > >> some places expect only one up and one down thread. > > > >I haven't given any deep thought to this issue, but I remember us discus= sing > >them over beer :-) >=20 > The easiest way to obtain more parallelism, is to divide the mesh into > multiple independent meshes. >=20 > This will do you no good if you have five disks in a RAID-5 config, but > if you have two disks each mounted on its own filesystem, you can run > a g_up & g_down for each of them. A class is suppose to interact with other classes only via GEOM, so I think it should be safe to choose g_up/g_down threads for each class individually, for example: /dev/ad0s1a (DEV) | g_up_0 + g_down_0 | ad0s1a (BSD) | g_up_1 + g_down_1 | ad0s1 (MBR) | g_up_2 + g_down_2 | ad0 (DISK) We could easly calculate g_down thread based on bio_to->geom->class and g_up thread based on bio_from->geom->class, so we know I/O requests for our class are always coming from the same threads. If we could make the same assumption for geoms it would allow for even better distribution. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --LpQ9ahxlCli8rRTG Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkun/uYACgkQForvXbEpPzSKuwCguhkymDDx/xptSf8zaU8rQI/5 qmAAoIMNpThV3uN+larfBnQ/ZI3pqElZ =QCWg -----END PGP SIGNATURE----- --LpQ9ahxlCli8rRTG-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 22 23:58:39 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D0BE1065672 for ; Mon, 22 Mar 2010 23:58:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id B36528FC0A for ; Mon, 22 Mar 2010 23:58:38 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id BA2AE45E8E; Tue, 23 Mar 2010 00:58:36 +0100 (CET) Received: from localhost (chello089077043238.chello.pl [89.77.43.238]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 7D5A745CD8 for ; Tue, 23 Mar 2010 00:58:31 +0100 (CET) Date: Tue, 23 Mar 2010 00:58:31 +0100 From: Pawel Jakub Dawidek To: freebsd-arch@FreeBSD.org Message-ID: <20100322235831.GC1767@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DIOMP1UsTsWJauNi" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Increasing IOCPARM_MAX. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2010 23:58:39 -0000 --DIOMP1UsTsWJauNi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. IOCPARM_MAX defines maximum size of a structure that can be passed directly to ioctl(2). Because of how ioctl command is build using _IO*() macros we have only 13 bits to encode structure size. So the structure can be up to 8kB-1. Currently we define IOCPARM_MAX as PAGE_SIZE. This is IMHO wrong for three main reasons: 1. It is confusing on archs with page size larger than 8kB (not really sure if we support such archs (sparc64?)), as even if PAGE_SIZE is bigger than 8kB, we won't be able to encode anything larger in ioctl command. 2. It is a waste. Why the structure can be only 4kB on most archs if we have 13 bits dedicated for that, not 12? 3. It shouldn't depend on architecture and page size. My ioctl command can work on one arch, but not on the other? I suggest increasing IOCPARM_MAX to 8kB and making it independed of PAGE_SIZE and architecture it is compiled for. This will allow to use all the bits on all the archs for size. Note that this doesn't mean we will copy more on every ioctl(2) call. No. We still copyin(9)/copyout(9) only exact number of bytes encoded in ioctl command. Practical use for this change is ZFS. zfs_cmd_t structure used for ZFS ioctls is larger than 4kB, I could really used this change. The patch is here: http://people.freebsd.org/~pjd/patches/ioccom.h.patch Are there any possible problems with such a change or should it be safe to commit as is? BTW. I see the comment saying IOCPARM_MAX should be multiple of page size, but I can't see why. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --DIOMP1UsTsWJauNi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuoBCYACgkQForvXbEpPzQEJwCg0MsFUQKlLqcyT61KUlDOf2PN pRkAoJiPEmyjURU3Uto2xlJLIQ2Q3M1U =wKYo -----END PGP SIGNATURE----- --DIOMP1UsTsWJauNi-- From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 00:05:26 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03C38106564A; Tue, 23 Mar 2010 00:05:26 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 96FA18FC08; Tue, 23 Mar 2010 00:05:25 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2N05MI0056853; Mon, 22 Mar 2010 18:05:22 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100322233607.GB1767@garage.freebsd.pl> Date: Mon, 22 Mar 2010 18:05:22 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <8D465FFB-0389-4321-84B9-E45292697D26@samsco.org> References: <4BA633A0.2090108@icyb.net.ua> <5754.1269246223@critter.freebsd.dk> <20100322233607.GB1767@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: freebsd-arch@FreeBSD.org, Poul-Henning Kamp , freebsd-current@FreeBSD.org, Alexander Motin , Andriy Gapon Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 00:05:26 -0000 On Mar 22, 2010, at 5:36 PM, Pawel Jakub Dawidek wrote: > On Mon, Mar 22, 2010 at 08:23:43AM +0000, Poul-Henning Kamp wrote: >> In message <4BA633A0.2090108@icyb.net.ua>, Andriy Gapon writes: >>> on 21/03/2010 16:05 Alexander Motin said the following: >>>> Ivan Voras wrote: >>>>> Hmm, it looks like it could be easy to spawn more g_* threads = (and, >>>>> barring specific class behaviour, it has a fair chance of working = out of >>>>> the box) but the incoming queue will need to also be broken up for >>>>> greater effect. >>>>=20 >>>> According to "notes", looks there is a good chance to obtain races, = as >>>> some places expect only one up and one down thread. >>>=20 >>> I haven't given any deep thought to this issue, but I remember us = discussing >>> them over beer :-) >>=20 >> The easiest way to obtain more parallelism, is to divide the mesh = into >> multiple independent meshes. >>=20 >> This will do you no good if you have five disks in a RAID-5 config, = but >> if you have two disks each mounted on its own filesystem, you can run >> a g_up & g_down for each of them. >=20 > A class is suppose to interact with other classes only via GEOM, so I > think it should be safe to choose g_up/g_down threads for each class > individually, for example: >=20 > /dev/ad0s1a (DEV) > | > g_up_0 + g_down_0 > | > ad0s1a (BSD) > | > g_up_1 + g_down_1 > | > ad0s1 (MBR) > | > g_up_2 + g_down_2 > | > ad0 (DISK) >=20 > We could easly calculate g_down thread based on bio_to->geom->class = and > g_up thread based on bio_from->geom->class, so we know I/O requests = for > our class are always coming from the same threads. >=20 > If we could make the same assumption for geoms it would allow for even > better distribution. The whole point of the discussion, sans PHK's interlude, is to reduce = the context switches and indirection, not to increase it. But if you = can show decreased latency/higher-iops benefits of increasing it, more = power to you. I would think that the results of DFly's experiment with = parallelism-via-more-queues would serve as a good warning, though. Scott From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 00:33:55 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5B701065674; Tue, 23 Mar 2010 00:33:55 +0000 (UTC) (envelope-from julianelischer@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.27]) by mx1.freebsd.org (Postfix) with ESMTP id 9BB0D8FC27; Tue, 23 Mar 2010 00:33:54 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 22so242747eye.3 for ; Mon, 22 Mar 2010 17:33:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=1Uo69i8YIN6B8Csn1RSX/w1aTfU+eevdLHR211XGyMo=; b=Dwtj/qalff3+JWVjtxkn/OrncJCZmKWeSbh5v8LcevTNUrtSBvdeeh4CCzt1SCe97O 3qORmcEXLRERa5GdphmErgaBbk3jhWzeAnLotyI3hBCnqSmsinAsZy509rZYDCYLVezs v59Owjy6wmKA0Z9ng2bQrDhHZ8gZDDLq98nN0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=CUKFo5btO5fAcBkxKA8ltygcvdT8yAWIBpXxDo0QKBw0W78G+eVAqeIZeN+pTo8wdf ZyN53A6vbqsfAA4wn54XPCw9DEboF7rZyUeCw1voGig/ra7ohUjQzenEA5zzZmmN4g3r YLbd+VutfFyClTJRravICrGKspFX/kKjI9IJM= Received: by 10.213.99.135 with SMTP id u7mr427406ebn.36.1269304433778; Mon, 22 Mar 2010 17:33:53 -0700 (PDT) Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by mx.google.com with ESMTPS id 14sm2427690ewy.10.2010.03.22.17.33.50 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 22 Mar 2010 17:33:53 -0700 (PDT) Sender: Julian Elischer Message-ID: <4BA80C6F.7050506@elischer.org> Date: Mon, 22 Mar 2010 17:33:51 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4BA633A0.2090108@icyb.net.ua> <5754.1269246223@critter.freebsd.dk> <20100322233607.GB1767@garage.freebsd.pl> In-Reply-To: <20100322233607.GB1767@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org, Poul-Henning Kamp , freebsd-current@FreeBSD.org, Alexander Motin , Andriy Gapon Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 00:33:56 -0000 Pawel Jakub Dawidek wrote: > On Mon, Mar 22, 2010 at 08:23:43AM +0000, Poul-Henning Kamp wrote: >> In message <4BA633A0.2090108@icyb.net.ua>, Andriy Gapon writes: >>> on 21/03/2010 16:05 Alexander Motin said the following: >>>> Ivan Voras wrote: >>>>> Hmm, it looks like it could be easy to spawn more g_* threads (and, >>>>> barring specific class behaviour, it has a fair chance of working out of >>>>> the box) but the incoming queue will need to also be broken up for >>>>> greater effect. >>>> According to "notes", looks there is a good chance to obtain races, as >>>> some places expect only one up and one down thread. >>> I haven't given any deep thought to this issue, but I remember us discussing >>> them over beer :-) >> The easiest way to obtain more parallelism, is to divide the mesh into >> multiple independent meshes. >> >> This will do you no good if you have five disks in a RAID-5 config, but >> if you have two disks each mounted on its own filesystem, you can run >> a g_up & g_down for each of them. > > A class is suppose to interact with other classes only via GEOM, so I > think it should be safe to choose g_up/g_down threads for each class > individually, for example: > > /dev/ad0s1a (DEV) > | > g_up_0 + g_down_0 > | > ad0s1a (BSD) > | > g_up_1 + g_down_1 > | > ad0s1 (MBR) > | > g_up_2 + g_down_2 > | > ad0 (DISK) > > We could easly calculate g_down thread based on bio_to->geom->class and > g_up thread based on bio_from->geom->class, so we know I/O requests for > our class are always coming from the same threads. > > If we could make the same assumption for geoms it would allow for even > better distribution. doesn't really help my problem however.. I just want to access the base provider directly with no geom thread involved. > From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 02:38:32 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAC84106566C; Tue, 23 Mar 2010 02:38:32 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 6AEF18FC1F; Tue, 23 Mar 2010 02:38:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2N2Zq0w034669; Mon, 22 Mar 2010 20:35:52 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 22 Mar 2010 20:35:53 -0600 (MDT) Message-Id: <20100322.203553.752311254955266835.imp@bsdimp.com> To: jhb@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <201003221605.24538.jhb@freebsd.org> References: <20100322.125937.278730673160410010.imp@bsdimp.com> <20100322.130512.864843819464264610.imp@bsdimp.com> <201003221605.24538.jhb@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Alexander@leidinger.net, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 02:38:32 -0000 In message: <201003221605.24538.jhb@freebsd.org> John Baldwin writes: : On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: : > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> : > M. Warner Losh writes: : > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> : > : Alexander Leidinger writes: : > : : Normally we use MK_xxx for things which are opt-in/opt-out. What about : > : : using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, : > : : what should the xxx part look like? : > : : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and : > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like the : > : default (or want to ensure they get option XXX, even if we turn it off : > : by default in the future). The default then gets encoded in : > : bsd.own.mk, and permeates the FreeBSD build system since we include : > : that everywhere, directly or indirectly. : > : : > : The problem is that bsd.own.mk is not included in sys.mk, nor should : > : it be. That's why we have the hacky combination of WITH_CTF and : > : NO_CTF that's there today. : > : : > : : Is bsd.kern.mk included in module builds too? : > : : > : Yes. : > : > One last thing I should have said was that the patch that was posted : > earlier in the thread looked ok, and likely couldn't be made : > significantly better due to the bsd.own.mk issue. : : I think the patch is a good approach, I just think it needs to default to not : enabling CTF by default. Instead, various bsd.foo.mk should selectively : enable it. I should have added that bit as well... Warner From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 08:25:57 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A080D106566B; Tue, 23 Mar 2010 08:25:57 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 296FE8FC1A; Tue, 23 Mar 2010 08:25:57 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id o2N8Ps1p032955; Tue, 23 Mar 2010 01:25:56 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id o2N8PsLY032954; Tue, 23 Mar 2010 01:25:54 -0700 (PDT) Date: Tue, 23 Mar 2010 01:25:54 -0700 (PDT) From: Matthew Dillon Message-Id: <201003230825.o2N8PsLY032954@apollo.backplane.com> To: freebsd-arch@freebsd.org, freebsd-current@freebsd.org References: <4BA633A0.2090108@icyb.net.ua> <5754.1269246223@critter.freebsd.dk> <20100322233607.GB1767@garage.freebsd.pl> <8D465FFB-0389-4321-84B9-E45292697D26@samsco.org> Cc: Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 08:25:57 -0000 :The whole point of the discussion, sans PHK's interlude, is to reduce the context switches and indirection, not to increase it. But if you can show decreased latency/higher-iops benefits of increasing it, more power to you. I would think that the results of DFly's experiment with parallelism-via-more-queues would serve as a good warning, though. : :Scott Well, I'm not sure what experiment you are refering to but I'll assume its the network threading, which works quite well actually. The protocol threads can be matched against the toeplitz function and in that case the entire packet stream operates lockless. Even without the matching we still get good benefits from batching (e.g. via ether_input_chain()) which drops the IPI and per-packet switch overhead basically to zero. We have other issues but the protocol threads aren't one of them. In anycase, the lesson to learn with batching to a thread is that you don't want the thread to immediately preempt the sender (if it happens to be on the same cpu), or to generate an instant IPI (if going between cpus). This creates a degenerate case where you wind up with a thread switch on each message or an excessive messaging interrupt rate... THAT is what seriously screws up performance. The key is to be able to batch multiple messages per thread switch when under load and to be able to maintain a pipeline. A single user-process test case will always have a bit more latency and can wind up being inefficient for a variety of other reasons (e.g. whether the target thread is on the same cpu or not), but that becomes less relevant when the machine is under load so its a self-correcting problem for the most part. Once the machine is under load batching becomes highly efficient. That is, latency != cpu cycle cost under load. When the threads have enough work to do they can pick up the next message without the cost of entering a sleep state or needing a wakeup (or needing to generate an actual IPI interrupt, etc). Plus you can run lockless and you get excellent cache locality. So as long as you ensure these optimal operations become the norm under load you win. Getting the threads to pipeline properly and avoid unnecessary tsleeps and wakeups is the hard part. -- But with regard to geom, I'd have to agree with you. You don't want to pipeline a single N-stage request through N threads. One thread, sure... that can be batched to reduce overhead. N-stages through N-threads just creates unnecessary latency, complicates your ability to maintain a pipeline, and has a multiplicative effect on thread activity that negates the advantage of having multiple cpus (and destroys cache locality as well). You could possibly use a different trick at least for some of the simpler transformations, and that is to replicate the control structures on a per-cpu basis. If you replicate the control structures on a per-cpu basis then you can parallelize independent operations running through the same set of devices and remove the bottlenecks. The set of transformations for a single BIO would be able to run lockless within a single thread and the control system as a whole would have one thread per cpu. (Of course, a RAID layer would require some rendezvous to deal with contention/conflicts, but that's easily dealt with). That would be my suggestion. We use that trick for our route tables in DFly, and also for listen socket PCBs to remove choke points, and a few other things like statistics gathering. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 08:28:15 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E461F1065679; Tue, 23 Mar 2010 08:28:14 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 5EB638FC19; Tue, 23 Mar 2010 08:28:14 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FE1C.dip.t-dialin.net [217.84.254.28]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 333E5844B3D; Tue, 23 Mar 2010 09:28:04 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id C3BEA526D; Tue, 23 Mar 2010 09:28:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269332880; bh=gt0EIbCtEuC3kGtT+4AyU9z/A0OQNdytztUQIZe2okE=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=tKhiNGc4NMFDQwZ9bM71xGUx7iWwu3YyOwmpj5WSEuGzEqExBOsYpKY/zS22nqfg0 nzauv7vUZFeGCMV/dLHBKpzmOB0oUgq4yhqdLy2jWK7RHNg4Y8LRW1kyva44D9aOn0 MFmL8vws5SWSnPwTd9hV/0S50WRM9Zmufv8JE3NirF8PUMn1cFQPuMI3zjmBgSbmEz 3QrjakDyZv2D69OyZnmtcqo1ZGmZROvLeWqFh57cE4cXPJ/m0clg2yIfSHjwQePEFA uOIvhvmiFtf0cwlllGwga+dcP/FZ81bnd0BLV5VQEPjrzwobhQR9j8wHc3C1c4cHRW xLaMvyARUSADg== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2N8S0qh020500; Tue, 23 Mar 2010 09:28:00 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 23 Mar 2010 09:28:00 +0100 Message-ID: <20100323092800.78582z12asne714w@webmail.leidinger.net> Date: Tue, 23 Mar 2010 09:28:00 +0100 From: Alexander Leidinger To: "M. Warner Losh" References: <20100322123408.16671ijbvmcyux80@webmail.leidinger.net> <201003220941.10525.jhb@freebsd.org> <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> <20100322.125937.278730673160410010.imp@bsdimp.com> In-Reply-To: <20100322.125937.278730673160410010.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 333E5844B3D.AC845 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269937687.50628@qjnLQo8EKfojd2DS5BllLA X-EBL-Spam-Status: No Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 08:28:15 -0000 Quoting "M. Warner Losh" (from Mon, 22 Mar 2010 12:59:37 -0600 (MDT)): > In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> > Alexander Leidinger writes: > : Normally we use MK_xxx for things which are opt-in/opt-out. What about > : using MK_xxx instead of ENABLE_CTF? If people are in favour of MK_xxx, > : what should the xxx part look like? > > Normally we *TEST* MK_XXX for things which are opt-in/opt-out and > require the user to say WITH_XXX or WITHOUT_XXX if they don't like the > default (or want to ensure they get option XXX, even if we turn it off > by default in the future). The default then gets encoded in > bsd.own.mk, and permeates the FreeBSD build system since we include > that everywhere, directly or indirectly. As I was understanding jhb, he proposed to use ENABLE_CTF in a way like we use the MK_XXX, and my question was targetted in this direction. The implementation (location/code) probably needs to be different from how we do it normally, but IMO at least the naming convention (MK_XXX and WITH_CTF/WITHOUT_CTF) can stay... except someone provides a good reason not to use MK_XXX, off course. I asked what name the XXX part should have because in case we want to handle the CTF stuff similar to WITHOUT_SENDMAIL, we need to be able to distinguish the build of e.g. libctf/ctfconvert/ctfmerge and the use of those programs to build binaries with CTF info included. Bye, Alexander. -- In God we trust; all else we walk through. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 08:56:27 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66C791065672; Tue, 23 Mar 2010 08:56:27 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 2B4338FC08; Tue, 23 Mar 2010 08:56:27 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 82E116434; Tue, 23 Mar 2010 08:56:25 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.3/8.14.3) with ESMTP id o2N8uOPm036867; Tue, 23 Mar 2010 08:56:24 GMT (envelope-from phk@critter.freebsd.dk) To: Pawel Jakub Dawidek From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 23 Mar 2010 00:36:07 +0100." <20100322233607.GB1767@garage.freebsd.pl> Date: Tue, 23 Mar 2010 08:56:24 +0000 Message-ID: <36866.1269334584@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: Alexander Motin , freebsd-current@FreeBSD.org, Andriy Gapon , freebsd-arch@FreeBSD.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 08:56:27 -0000 In message <20100322233607.GB1767@garage.freebsd.pl>, Pawel Jakub Dawidek write s: >A class is suppose to interact with other classes only via GEOM, so I >think it should be safe to choose g_up/g_down threads for each class >individually, for example: > > /dev/ad0s1a (DEV) > | > g_up_0 + g_down_0 > | > ad0s1a (BSD) > | > g_up_1 + g_down_1 > | > ad0s1 (MBR) > | > g_up_2 + g_down_2 > | > ad0 (DISK) Uhm, that way you get _more_ context switches than today, today g_down will typically push the requests all the way down through the stack without a context switch. (Similar for g_up) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 10:06:45 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B9EC106566C; Tue, 23 Mar 2010 10:06:45 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id B8BD78FC0C; Tue, 23 Mar 2010 10:06:44 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FE1C.dip.t-dialin.net [217.84.254.28]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 2C638844B3D; Tue, 23 Mar 2010 11:06:39 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 10F3E5278; Tue, 23 Mar 2010 11:06:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269338796; bh=JJ62aO33y5vB4dEGBA7TwOP4qYXKdM7szD1DiKecWHI=; h=Message-ID:Date:From:To:Cc:Subject:MIME-Version:Content-Type: Content-Transfer-Encoding; b=1ka2IbtZ1MneZrcoJcxdmZyTAUUIPEXHD9F3UJgezM26O9QIKaZhqWM0EtOSQ+cjr KGJRjnypIDwvYd+RNp1u+x3rpi1Dx/XZs0Ega/7lu4sKWJZDQ0AnrPhKgdGfuGQ/oj 5HfouHm9bKRDsItcW4QK4dH+xHeYuxvtFR45tHXAPVtk7ynJ+1Amq64BD6w13YTWxc 1t5/kJcgMTGQbfeMVUqJ4liftasiifAdCh6X+yAH4y3q5LczpZ8e10hr6QkIX7kdtf EUHe8ePEkS/drm1hHWBBIFMM613gYFdu5qsEGds6uB0K0hRU85Pn8YE10C5cKLfl+S jLGk5muyhy5mQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2NA6Z5n046969; Tue, 23 Mar 2010 11:06:35 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 23 Mar 2010 11:06:35 +0100 Message-ID: <20100323110635.12058acj188msrzk@webmail.leidinger.net> Date: Tue, 23 Mar 2010 11:06:35 +0100 From: Alexander Leidinger To: Scott Long MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 2C638844B3D.32685 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269943599.57558@Ng9kkUxt7VqS6OBLwY743w X-EBL-Spam-Status: No Cc: "Robert N. M. Watson" , freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review (was: Re: is dtrace usable?) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 10:06:45 -0000 Quoting Scott Long (from Mon, 22 Mar 2010 10:30:39 -0600): > I think it's still important to be opt-in for dtrace. If you're > reluctant, I'll submit my patch which is opt-in, and is only about > 10 lines of code change. Thank you for removing just one sentence in your reply, the one which states that I try to get time to rework the patch according to the outcome of this discussion. Bye, Alexander. -- BOFH excuse #182: endothermal recalibration http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 10:12:54 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BE0E106566C; Tue, 23 Mar 2010 10:12:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 788218FC15; Tue, 23 Mar 2010 10:12:53 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FE1C.dip.t-dialin.net [217.84.254.28]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 63AB6844B3D; Tue, 23 Mar 2010 11:12:46 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 6736B527A; Tue, 23 Mar 2010 11:12:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269339163; bh=rOsiNdV4exvdzLkWd+0LgzOYlJcIq/yVjp0zqsvMd0Q=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=M0w9a2ddrjRX012d8OcHvKi+ZdW+TAVJHRbFSWZGEAGRy986rNtQEbDVSmYUiqWe0 coFG5GCz4M2D4VaGrsI5XGgFLSaB3QUIofwf6rAmx8PXWbT8ZFAsDLPI9V2hKUayDs Xe/5rZpQ+SlbjD0FW1UqIXghO837YgViE3+AntG3kV1iQF0DvD0wUvQwYXBE12Bvqm LyDS2ZrHv2PU1o9RnqfOmxbGyvSEYcRfSmCT0fpqqNdyij0QkSNTYj5wEp/K8qOKiy eUPrNioxAo16HMi8oMNLXFyVjndONe6gKMYT9qbJOy8qW13qFeDumh85FIxOj10qXc XEm3EblvrLksQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2NAChKg048504; Tue, 23 Mar 2010 11:12:43 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 23 Mar 2010 11:12:43 +0100 Message-ID: <20100323111243.124121qxmpk2c4lc@webmail.leidinger.net> Date: Tue, 23 Mar 2010 11:12:43 +0100 From: Alexander Leidinger To: "M. Warner Losh" References: <20100322.125937.278730673160410010.imp@bsdimp.com> <20100322.130512.864843819464264610.imp@bsdimp.com> <201003221605.24538.jhb@freebsd.org> <20100322.203553.752311254955266835.imp@bsdimp.com> In-Reply-To: <20100322.203553.752311254955266835.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 63AB6844B3D.1F191 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1269943968.69742@VR3TSwwrIBsvANpb2/+w7Q X-EBL-Spam-Status: No Cc: rwatson@FreeBSD.org, jhb@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 10:12:54 -0000 Quoting "M. Warner Losh" (from Mon, 22 Mar 2010 20:35:53 -0600 (MDT)): > In message: <201003221605.24538.jhb@freebsd.org> > John Baldwin writes: > : On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: > : > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> > : > M. Warner Losh writes: > : > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> > : > : Alexander Leidinger writes: > : > : : Normally we use MK_xxx for things which are opt-in/opt-out. > What about > : > : : using MK_xxx instead of ENABLE_CTF? If people are in favour > of MK_xxx, > : > : : what should the xxx part look like? > : > : > : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and > : > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like the > : > : default (or want to ensure they get option XXX, even if we turn it off > : > : by default in the future). The default then gets encoded in > : > : bsd.own.mk, and permeates the FreeBSD build system since we include > : > : that everywhere, directly or indirectly. > : > : > : > : The problem is that bsd.own.mk is not included in sys.mk, nor should > : > : it be. That's why we have the hacky combination of WITH_CTF and > : > : NO_CTF that's there today. > : > : > : > : : Is bsd.kern.mk included in module builds too? > : > : > : > : Yes. > : > > : > One last thing I should have said was that the patch that was posted > : > earlier in the thread looked ok, and likely couldn't be made > : > significantly better due to the bsd.own.mk issue. > : > : I think the patch is a good approach, I just think it needs to > default to not > : enabling CTF by default. Instead, various bsd.foo.mk should selectively > : enable it. > > I should have added that bit as well... And here it is: http://www.leidinger.net/test/ctf2.diff Please pay attention to one XXX comment. Both cases I describe look possible, but I would like to get some more eyes on this issues to not overlook something. I haven't renamed the NO_CTF part yet. Bikeshed: ENABLE_CTF / ADD_CTF / MK_CTF / MK_CTFINFO / MK_CTFINC / ...? Cast your vote please. Bye, Alexander. -- Zapp: There's only one surefire way back into a woman's heart and parts beyond. I speak, of course, of Karaoke. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 14:28:03 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 995BB10657BE; Tue, 23 Mar 2010 14:28:03 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5765B8FC08; Tue, 23 Mar 2010 14:28:03 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E39E146B8E; Tue, 23 Mar 2010 10:28:02 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id EB73A8A025; Tue, 23 Mar 2010 10:28:01 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Tue, 23 Mar 2010 10:25:55 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100322.125937.278730673160410010.imp@bsdimp.com> <20100322.203553.752311254955266835.imp@bsdimp.com> <20100323111243.124121qxmpk2c4lc@webmail.leidinger.net> In-Reply-To: <20100323111243.124121qxmpk2c4lc@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003231025.55404.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 23 Mar 2010 10:28:02 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 14:28:03 -0000 On Tuesday 23 March 2010 6:12:43 am Alexander Leidinger wrote: > Quoting "M. Warner Losh" (from Mon, 22 Mar 2010 > 20:35:53 -0600 (MDT)): > > > In message: <201003221605.24538.jhb@freebsd.org> > > John Baldwin writes: > > : On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: > > : > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> > > : > M. Warner Losh writes: > > : > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> > > : > : Alexander Leidinger writes: > > : > : : Normally we use MK_xxx for things which are opt-in/opt-out. > > What about > > : > : : using MK_xxx instead of ENABLE_CTF? If people are in favour > > of MK_xxx, > > : > : : what should the xxx part look like? > > : > : > > : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and > > : > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like the > > : > : default (or want to ensure they get option XXX, even if we turn it off > > : > : by default in the future). The default then gets encoded in > > : > : bsd.own.mk, and permeates the FreeBSD build system since we include > > : > : that everywhere, directly or indirectly. > > : > : > > : > : The problem is that bsd.own.mk is not included in sys.mk, nor should > > : > : it be. That's why we have the hacky combination of WITH_CTF and > > : > : NO_CTF that's there today. > > : > : > > : > : : Is bsd.kern.mk included in module builds too? > > : > : > > : > : Yes. > > : > > > : > One last thing I should have said was that the patch that was posted > > : > earlier in the thread looked ok, and likely couldn't be made > > : > significantly better due to the bsd.own.mk issue. > > : > > : I think the patch is a good approach, I just think it needs to > > default to not > > : enabling CTF by default. Instead, various bsd.foo.mk should selectively > > : enable it. > > > > I should have added that bit as well... > > And here it is: > http://www.leidinger.net/test/ctf2.diff > > Please pay attention to one XXX comment. Both cases I describe look > possible, but I would like to get some more eyes on this issues to not > overlook something. I would maybe put a comment in front of the CFLAGS+= line for now and leave the rest of the XXX comment. I'm not sure of the best way to solve this yet. > I haven't renamed the NO_CTF part yet. Bikeshed: ENABLE_CTF / ADD_CTF > / MK_CTF / MK_CTFINFO / MK_CTFINC / ...? Cast your vote please. I think the naming stuff you have used is fine. I think it is better to use NO_CTF rather than MK_CTF because this is not set via bsd.own.mk but is a special case. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Tue Mar 23 20:40:03 2010 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61236106566B for ; Tue, 23 Mar 2010 20:40:03 +0000 (UTC) (envelope-from matthew.fleming@isilon.com) Received: from seaxch09.isilon.com (seaxch09.isilon.com [74.85.160.25]) by mx1.freebsd.org (Postfix) with ESMTP id 443C58FC1A for ; Tue, 23 Mar 2010 20:40:03 +0000 (UTC) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Tue, 23 Mar 2010 13:40:01 -0700 Message-ID: <06D5F9F6F655AD4C92E28B662F7F853E0388A6A9@seaxch09.desktop.isilon.com> In-Reply-To: <20100322095701.GE2415@deviant.kiev.zoral.com.ua> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: XMM register usage in the kernel Thread-Index: AcrJpg/RspJMph1HRuqVPYTQ/eBI4wBIrl4Q References: <20100322095701.GE2415@deviant.kiev.zoral.com.ua> From: "Matthew Fleming" To: "Kostik Belousov" , , Cc: Subject: RE: XMM register usage in the kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Mar 2010 20:40:03 -0000 > The KPI consists of two functions, fpu_kern_enter() and fpu_kern_leave(). > The pair should brace the region of code that intend to use coprocessor. > Caller shall provide the context save area, I expect that usually this > memory will be allocated as part of larger subsystem data structure. >=20 > fpu_kern_enter() allows the nesting, assuming fpu_kern_leave() is called > proper number of times to pop the context stack. Isilon would like it if something like this functionality ended up in CURRENT. We have some hacks that do something similar. We use SSE2 instructions to significantly speed up computing ... stuff. :-) Thanks, matthew From owner-freebsd-arch@FreeBSD.ORG Wed Mar 24 13:59:52 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF0F6106566C; Wed, 24 Mar 2010 13:59:52 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 727A28FC12; Wed, 24 Mar 2010 13:59:52 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FFF0.dip.t-dialin.net [217.84.255.240]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 49AAE844410; Wed, 24 Mar 2010 14:59:46 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 351F95363; Wed, 24 Mar 2010 14:59:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269439182; bh=83uFMz57JdsoBRVlrYulDHximPH6LEvE0XcTsMsLXBs=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=CgwW3CSZh31IkRG2xhupQe56J/ChU6NT9RhXzpxIQJ64BorMHFnyft9W3jnoiLcZT bsTP3ilrHyaFtbbD/DfpD+mK+Xn8LSDhb73bFvzD0Okv0b/1kwALJhokV3P5YsqiC9 w4sKeCTQTUY2c534h7bwCZzdZOi/o84t7bzVATn0qpNqzPIZKpR5reee2vc9+HxOuk d8qkoz3ARWqEp0Ut5wwwNpZZzMG4qoDbyQrldBiti/LtW5RviCheo53g8esYd5nVKc 0voPF2t8CdSaEBJTi+Yu5XklhAje7xia1Hlf1gTukB7hDQGf+K153ISYpKIzwM/s3G ITN0xkIZbT2Ng== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2ODxfv7098204; Wed, 24 Mar 2010 14:59:41 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 24 Mar 2010 14:59:41 +0100 Message-ID: <20100324145941.181387uohp3zdl1o@webmail.leidinger.net> Date: Wed, 24 Mar 2010 14:59:41 +0100 From: Alexander Leidinger To: John Baldwin References: <20100322.125937.278730673160410010.imp@bsdimp.com> <20100322.203553.752311254955266835.imp@bsdimp.com> <20100323111243.124121qxmpk2c4lc@webmail.leidinger.net> <201003231025.55404.jhb@freebsd.org> In-Reply-To: <201003231025.55404.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 49AAE844410.E57DF X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270043986.76109@nkmInldpzaCwFQkQSJ1Njw X-EBL-Spam-Status: No Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 13:59:53 -0000 Quoting John Baldwin (from Tue, 23 Mar 2010 10:25:55 -0400): > On Tuesday 23 March 2010 6:12:43 am Alexander Leidinger wrote: >> Quoting "M. Warner Losh" (from Mon, 22 Mar 2010 >> 20:35:53 -0600 (MDT)): >> >> > In message: <201003221605.24538.jhb@freebsd.org> >> > John Baldwin writes: >> > : On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: >> > : > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> >> > : > M. Warner Losh writes: >> > : > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> >> > : > : Alexander Leidinger writes: >> > : > : : Normally we use MK_xxx for things which are opt-in/opt-out. >> > What about >> > : > : : using MK_xxx instead of ENABLE_CTF? If people are in favour >> > of MK_xxx, >> > : > : : what should the xxx part look like? >> > : > : >> > : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and >> > : > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like > the >> > : > : default (or want to ensure they get option XXX, even if we turn it > off >> > : > : by default in the future). The default then gets encoded in >> > : > : bsd.own.mk, and permeates the FreeBSD build system since we include >> > : > : that everywhere, directly or indirectly. >> > : > : >> > : > : The problem is that bsd.own.mk is not included in sys.mk, nor should >> > : > : it be. That's why we have the hacky combination of WITH_CTF and >> > : > : NO_CTF that's there today. >> > : > : >> > : > : : Is bsd.kern.mk included in module builds too? >> > : > : >> > : > : Yes. >> > : > >> > : > One last thing I should have said was that the patch that was posted >> > : > earlier in the thread looked ok, and likely couldn't be made >> > : > significantly better due to the bsd.own.mk issue. >> > : >> > : I think the patch is a good approach, I just think it needs to >> > default to not >> > : enabling CTF by default. Instead, various bsd.foo.mk should selectively >> > : enable it. >> > >> > I should have added that bit as well... >> >> And here it is: >> http://www.leidinger.net/test/ctf2.diff >> >> Please pay attention to one XXX comment. Both cases I describe look >> possible, but I would like to get some more eyes on this issues to not >> overlook something. > > I would maybe put a comment in front of the CFLAGS+= line for now and leave > the rest of the XXX comment. I'm not sure of the best way to solve this yet. Done. I want to have a look if it is possible to do it similar to the LD_CTF_FLAG way later. Currently I have the problem that WITH_CTF is not picked up by kmod.mk if "makeoptions WITH_CTF=yes" is used in the kernel config. This means that all makeoptions do not propagate to module builds. Any ideas? Maybe letting makeoptions add lines like "KERNELSUBMAKEARGS+=" and using this for calls to make (I assume kernel modules are build by issuing calls to make in the corresponding directories)? >> I haven't renamed the NO_CTF part yet. Bikeshed: ENABLE_CTF / ADD_CTF >> / MK_CTF / MK_CTFINFO / MK_CTFINC / ...? Cast your vote please. > > I think the naming stuff you have used is fine. I think it is better to use > NO_CTF rather than MK_CTF because this is not set via bsd.own.mk but is a > special case. I agree. Bye, Alexander. -- If graphics hackers are so smart, why can't they get the bugs out of fresh paint? http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Wed Mar 24 14:15:45 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2D531065678; Wed, 24 Mar 2010 14:15:44 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id AF4DD8FC28; Wed, 24 Mar 2010 14:15:44 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 4516046B29; Wed, 24 Mar 2010 10:15:44 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 5D3B18A028; Wed, 24 Mar 2010 10:15:43 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Wed, 24 Mar 2010 10:05:55 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100322.125937.278730673160410010.imp@bsdimp.com> <201003231025.55404.jhb@freebsd.org> <20100324145941.181387uohp3zdl1o@webmail.leidinger.net> In-Reply-To: <20100324145941.181387uohp3zdl1o@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003241005.55239.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 24 Mar 2010 10:15:43 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 14:15:45 -0000 On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: > Quoting John Baldwin (from Tue, 23 Mar 2010 10:25:55 -0400): > > > On Tuesday 23 March 2010 6:12:43 am Alexander Leidinger wrote: > >> Quoting "M. Warner Losh" (from Mon, 22 Mar 2010 > >> 20:35:53 -0600 (MDT)): > >> > >> > In message: <201003221605.24538.jhb@freebsd.org> > >> > John Baldwin writes: > >> > : On Monday 22 March 2010 3:05:12 pm M. Warner Losh wrote: > >> > : > In message: <20100322.125937.278730673160410010.imp@bsdimp.com> > >> > : > M. Warner Losh writes: > >> > : > : In message: <20100322172104.14234yawbsev0sw8@webmail.leidinger.net> > >> > : > : Alexander Leidinger writes: > >> > : > : : Normally we use MK_xxx for things which are opt-in/opt-out. > >> > What about > >> > : > : : using MK_xxx instead of ENABLE_CTF? If people are in favour > >> > of MK_xxx, > >> > : > : : what should the xxx part look like? > >> > : > : > >> > : > : Normally we *TEST* MK_XXX for things which are opt-in/opt-out and > >> > : > : require the user to say WITH_XXX or WITHOUT_XXX if they don't like > > the > >> > : > : default (or want to ensure they get option XXX, even if we turn it > > off > >> > : > : by default in the future). The default then gets encoded in > >> > : > : bsd.own.mk, and permeates the FreeBSD build system since we include > >> > : > : that everywhere, directly or indirectly. > >> > : > : > >> > : > : The problem is that bsd.own.mk is not included in sys.mk, nor should > >> > : > : it be. That's why we have the hacky combination of WITH_CTF and > >> > : > : NO_CTF that's there today. > >> > : > : > >> > : > : : Is bsd.kern.mk included in module builds too? > >> > : > : > >> > : > : Yes. > >> > : > > >> > : > One last thing I should have said was that the patch that was posted > >> > : > earlier in the thread looked ok, and likely couldn't be made > >> > : > significantly better due to the bsd.own.mk issue. > >> > : > >> > : I think the patch is a good approach, I just think it needs to > >> > default to not > >> > : enabling CTF by default. Instead, various bsd.foo.mk should selectively > >> > : enable it. > >> > > >> > I should have added that bit as well... > >> > >> And here it is: > >> http://www.leidinger.net/test/ctf2.diff > >> > >> Please pay attention to one XXX comment. Both cases I describe look > >> possible, but I would like to get some more eyes on this issues to not > >> overlook something. > > > > I would maybe put a comment in front of the CFLAGS+= line for now and leave > > the rest of the XXX comment. I'm not sure of the best way to solve this yet. > > Done. I want to have a look if it is possible to do it similar to the > LD_CTF_FLAG way later. > > Currently I have the problem that WITH_CTF is not picked up by kmod.mk > if "makeoptions WITH_CTF=yes" is used in the kernel config. This means > that all makeoptions do not propagate to module builds. > > Any ideas? Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you should patch kern.post.mk to propogate WITH_CTF to modules. This is how it works for DEBUG now: .if defined(DEBUG) MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" .endif -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Mar 24 14:42:25 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61B3E1065670; Wed, 24 Mar 2010 14:42:25 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 071AD8FC19; Wed, 24 Mar 2010 14:42:24 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FFF0.dip.t-dialin.net [217.84.255.240]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id EA3C2844410; Wed, 24 Mar 2010 15:42:17 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id EBCC05368; Wed, 24 Mar 2010 15:42:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269441735; bh=FXGLtGkmQm3NSAk/1rg8ZQyDHRrfr/Iz7TI5o8tZPYg=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=AAzjJ2LKXKJCOHgOE9kP8lZtOPiipVKsrcbPr5sp/WCDT3Mtx6KvQqmsWc7YftAAB 9SR5flVq30y/qndVLanPKUkzvCFXZIGAb3/xVo0zaDFsNbvvzokiolhogL6FACasCT 3vgkC/LqUYJI5hffGLVtcu/xHluQx5r/lh8RYsXuAuW6IXqMgBAKgXXg3dNDh2cGic GC8WTbBJfKOx+SK5Jt0ZkG2KvNiOtknpv6PKqFtyCceQuUzEgCrlXxRVmjfMRn3oxM ZpyCOOmJyI2j1FNYYnvmwOkMpPtGq1hYlVdgwRGZNphxKJ9o4nOINEq0eF27SkkG11 jyNmaHb0Fh4wg== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2OEgEbB020460; Wed, 24 Mar 2010 15:42:14 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 24 Mar 2010 15:42:14 +0100 Message-ID: <20100324154214.16865r6wk0r22rcw@webmail.leidinger.net> Date: Wed, 24 Mar 2010 15:42:14 +0100 From: Alexander Leidinger To: John Baldwin References: <20100322.125937.278730673160410010.imp@bsdimp.com> <201003231025.55404.jhb@freebsd.org> <20100324145941.181387uohp3zdl1o@webmail.leidinger.net> <201003241005.55239.jhb@freebsd.org> In-Reply-To: <201003241005.55239.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: EA3C2844410.11495 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270046539.53453@biznikKceG804K8s2d5y1w X-EBL-Spam-Status: No Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 14:42:25 -0000 Quoting John Baldwin (from Wed, 24 Mar 2010 10:05:55 -0400): > On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: >> Currently I have the problem that WITH_CTF is not picked up by kmod.mk >> if "makeoptions WITH_CTF=yes" is used in the kernel config. This means >> that all makeoptions do not propagate to module builds. >> >> Any ideas? > > Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you should > patch kern.post.mk to propogate WITH_CTF to modules. This is how it works > for DEBUG now: > > .if defined(DEBUG) > MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" > .endif Do we want to be able to override WITH_CTF in modules (-> kern.pre.mk instead of kernl.post.mk)? While I'm here, do we want to have CONF_CFLAGS used in modules too? I would expect that they are used there (I use it to use -fno-builtin for my kernel build) and would put it into kern.post.mk. Bye, Alexander. -- BOFH excuse #258: That's easy to fix, but I can't be bothered http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Wed Mar 24 19:23:50 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1695B106566B; Wed, 24 Mar 2010 19:23:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DA6B08FC08; Wed, 24 Mar 2010 19:23:49 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 7366346B53; Wed, 24 Mar 2010 15:23:49 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id AC8C98A01F; Wed, 24 Mar 2010 15:23:48 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Wed, 24 Mar 2010 11:32:29 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100322.125937.278730673160410010.imp@bsdimp.com> <201003241005.55239.jhb@freebsd.org> <20100324154214.16865r6wk0r22rcw@webmail.leidinger.net> In-Reply-To: <20100324154214.16865r6wk0r22rcw@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003241132.29588.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 24 Mar 2010 15:23:48 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00, DATE_IN_PAST_03_06 autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 19:23:50 -0000 On Wednesday 24 March 2010 10:42:14 am Alexander Leidinger wrote: > Quoting John Baldwin (from Wed, 24 Mar 2010 10:05:55 -0400): > > > On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: > > >> Currently I have the problem that WITH_CTF is not picked up by kmod.mk > >> if "makeoptions WITH_CTF=yes" is used in the kernel config. This means > >> that all makeoptions do not propagate to module builds. > >> > >> Any ideas? > > > > Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you should > > patch kern.post.mk to propogate WITH_CTF to modules. This is how it works > > for DEBUG now: > > > > .if defined(DEBUG) > > MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" > > .endif > > Do we want to be able to override WITH_CTF in modules (-> kern.pre.mk > instead of kernl.post.mk)? No, I think it is fine to treat it the same as DEBUG. > While I'm here, do we want to have CONF_CFLAGS used in modules too? I > would expect that they are used there (I use it to use -fno-builtin > for my kernel build) and would put it into kern.post.mk. Hmm, probably yes. That should be a separate commit however. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 25 09:56:27 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AB8AE10656B9; Thu, 25 Mar 2010 09:56:27 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 20C398FC1F; Thu, 25 Mar 2010 09:56:27 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FB2C.dip.t-dialin.net [217.84.251.44]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id BDB9A8455F7; Thu, 25 Mar 2010 10:56:19 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 5A60D5412; Thu, 25 Mar 2010 10:56:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269510976; bh=A1Gd6pRdbcz1A2yfXSNvICIh7xEpcJWxZKYLCYHhMEk=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=qxgmbGH3QOiUJ2eCqY9w9lod+hg+QNdnjAGjfQ/VQrhWO+UvlGzxxtQ38BC/goLwa X/IfLWFw9LutcSsKeqDjg6RqkPBAVv2lSu8pfSaEiW6PDmbhMuzjoyPOEk1xHP2zbm DE4CwpGtQUcbkyE993k/tn/0UTg6iUYPFvFcV6/+ZXBtdloGQe/jQ/OzCr3DlmyN7K G4QXataKnh8SU7lHiYGQAiTGHbFhM04u/EPBsAmToiZ6ZR9kCUYfXImIHuCVYDghj8 OrlD7TiAWUAg/KMO2z7P+f1XB033UfMRmLFwRC+5WwFskS2TThdw98X7a6QeCVzP9x Pu1afEYFApplQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2P9uGjr011233; Thu, 25 Mar 2010 10:56:16 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 25 Mar 2010 10:56:15 +0100 Message-ID: <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> Date: Thu, 25 Mar 2010 10:56:15 +0100 From: Alexander Leidinger To: John Baldwin References: <20100322.125937.278730673160410010.imp@bsdimp.com> <201003241005.55239.jhb@freebsd.org> <20100324154214.16865r6wk0r22rcw@webmail.leidinger.net> <201003241132.29588.jhb@freebsd.org> In-Reply-To: <201003241132.29588.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: BDB9A8455F7.34F85 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270115780.20329@8xeTu+yLILD0hSbMokbJXw X-EBL-Spam-Status: No Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2010 09:56:27 -0000 Quoting John Baldwin (from Wed, 24 Mar 2010 11:32:29 -0400): > On Wednesday 24 March 2010 10:42:14 am Alexander Leidinger wrote: >> Quoting John Baldwin (from Wed, 24 Mar 2010 10:05:55 > -0400): >> >> > On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: >> >> >> Currently I have the problem that WITH_CTF is not picked up by kmod.mk >> >> if "makeoptions WITH_CTF=yes" is used in the kernel config. This means >> >> that all makeoptions do not propagate to module builds. >> >> >> >> Any ideas? >> > >> > Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you should >> > patch kern.post.mk to propogate WITH_CTF to modules. This is how it works >> > for DEBUG now: >> > >> > .if defined(DEBUG) >> > MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" >> > .endif >> >> Do we want to be able to override WITH_CTF in modules (-> kern.pre.mk >> instead of kernl.post.mk)? > > No, I think it is fine to treat it the same as DEBUG. DEBUG is in kern.pre.mk, and thus can be overriden. Currently I have WITH_CTF handling in kern.post.mk, and thus it can not be overriden. There are some MKMODULESENV things in post.mk already. On a related note, the LD_CTF_FLAGS thing does not seem to work for modules. Reading the man page of ld tells that -g is a noop (the switch is ignored). Thus I would prefer to remove the setting of -g for ld. Any objections? >> While I'm here, do we want to have CONF_CFLAGS used in modules too? I >> would expect that they are used there (I use it to use -fno-builtin >> for my kernel build) and would put it into kern.post.mk. > > Hmm, probably yes. That should be a separate commit however. I agree. Bye, Alexander. -- These days the necessities of life cost you about three times what they used to, and half the time they aren't even fit to drink. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Thu Mar 25 11:55:08 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 416E71065677; Thu, 25 Mar 2010 11:55:08 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 11A598FC26; Thu, 25 Mar 2010 11:55:08 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 8ED4546B98; Thu, 25 Mar 2010 07:55:07 -0400 (EDT) Received: from jhbbsd.localnet (localhost [IPv6:::1]) by bigwig.baldwin.cx (Postfix) with ESMTPA id ACE058A021; Thu, 25 Mar 2010 07:55:06 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Thu, 25 Mar 2010 07:49:57 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100322.125937.278730673160410010.imp@bsdimp.com> <201003241132.29588.jhb@freebsd.org> <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> In-Reply-To: <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003250749.57994.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 25 Mar 2010 07:55:06 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.1 required=4.2 tests=AWL,BAYES_00,NO_RELAYS autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2010 11:55:08 -0000 On Thursday 25 March 2010 5:56:15 am Alexander Leidinger wrote: > Quoting John Baldwin (from Wed, 24 Mar 2010 11:32:29 -0400): > > > On Wednesday 24 March 2010 10:42:14 am Alexander Leidinger wrote: > >> Quoting John Baldwin (from Wed, 24 Mar 2010 10:05:55 > > -0400): > >> > >> > On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: > >> > >> >> Currently I have the problem that WITH_CTF is not picked up by kmod.mk > >> >> if "makeoptions WITH_CTF=yes" is used in the kernel config. This means > >> >> that all makeoptions do not propagate to module builds. > >> >> > >> >> Any ideas? > >> > > >> > Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you should > >> > patch kern.post.mk to propogate WITH_CTF to modules. This is how it works > >> > for DEBUG now: > >> > > >> > .if defined(DEBUG) > >> > MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" > >> > .endif > >> > >> Do we want to be able to override WITH_CTF in modules (-> kern.pre.mk > >> instead of kernl.post.mk)? > > > > No, I think it is fine to treat it the same as DEBUG. > > DEBUG is in kern.pre.mk, and thus can be overriden. Currently I have > WITH_CTF handling in kern.post.mk, and thus it can not be overriden. > There are some MKMODULESENV things in post.mk already. Ok. > On a related note, the LD_CTF_FLAGS thing does not seem to work for > modules. Reading the man page of ld tells that -g is a noop (the > switch is ignored). Thus I would prefer to remove the setting of -g > for ld. Any objections? Ok. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Mar 25 13:41:20 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8599E106566B; Thu, 25 Mar 2010 13:41:20 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 43DA18FC19; Thu, 25 Mar 2010 13:41:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2PDbXS3080127; Thu, 25 Mar 2010 07:37:33 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 25 Mar 2010 07:37:36 -0600 (MDT) Message-Id: <20100325.073736.635700942514937954.imp@bsdimp.com> To: jhb@FreeBSD.org From: "M. Warner Losh" In-Reply-To: <201003250749.57994.jhb@freebsd.org> References: <201003241132.29588.jhb@freebsd.org> <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> <201003250749.57994.jhb@freebsd.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Alexander@leidinger.net, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2010 13:41:20 -0000 In message: <201003250749.57994.jhb@freebsd.org> John Baldwin writes: : On Thursday 25 March 2010 5:56:15 am Alexander Leidinger wrote: : > Quoting John Baldwin (from Wed, 24 Mar 2010 11:32:29 : -0400): : > : > > On Wednesday 24 March 2010 10:42:14 am Alexander Leidinger wrote: : > >> Quoting John Baldwin (from Wed, 24 Mar 2010 10:05:55 : > > -0400): : > >> : > >> > On Wednesday 24 March 2010 9:59:41 am Alexander Leidinger wrote: : > >> : > >> >> Currently I have the problem that WITH_CTF is not picked up by kmod.mk : > >> >> if "makeoptions WITH_CTF=yes" is used in the kernel config. This means : > >> >> that all makeoptions do not propagate to module builds. : > >> >> : > >> >> Any ideas? : > >> > : > >> > Hmmmm. That's odd because 'DEBUG=-g' does work. Ah, I think you : should : > >> > patch kern.post.mk to propogate WITH_CTF to modules. This is how it : works : > >> > for DEBUG now: : > >> > : > >> > .if defined(DEBUG) : > >> > MKMODULESENV+= DEBUG_FLAGS="${DEBUG}" : > >> > .endif : > >> : > >> Do we want to be able to override WITH_CTF in modules (-> kern.pre.mk : > >> instead of kernl.post.mk)? : > > : > > No, I think it is fine to treat it the same as DEBUG. : > : > DEBUG is in kern.pre.mk, and thus can be overriden. Currently I have : > WITH_CTF handling in kern.post.mk, and thus it can not be overriden. : > There are some MKMODULESENV things in post.mk already. : : Ok. : : > On a related note, the LD_CTF_FLAGS thing does not seem to work for : > modules. Reading the man page of ld tells that -g is a noop (the : > switch is ignored). Thus I would prefer to remove the setting of -g : > for ld. Any objections? : : Ok. In general, I think this is OK. I'd like to see the revised patch before it is committed. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Mar 25 14:07:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F044106566B; Thu, 25 Mar 2010 14:07:30 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id B70FC8FC1D; Thu, 25 Mar 2010 14:07:29 +0000 (UTC) Received: from outgoing.leidinger.net (pD954FB2C.dip.t-dialin.net [217.84.251.44]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 21D2F8455F7; Thu, 25 Mar 2010 15:07:25 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 20E175061; Thu, 25 Mar 2010 15:07:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269526042; bh=RuGcZJjVnvqn4QcU5kzTSRRn4z0EZMiORpRD0rA37w0=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=2WglHo6U6VPqoQIpG7+359UnpYPG3YO3ohUbk6sJJzSgW2WF489I6iEsMPWPwhQNU LE730N8CVyotY92HwaaY2dO+e6QcllsMGQH/tJvTijVzgn9btFIAAb7puULKyw5Mrm WG3MWywzYaxOqKZU6jyPk+c1DV3kcVHuv/VBnxVLCRTCxvJLE0aihPd5q9U4IgSwtU BCkWQ6k4sUB95VvkacYRFlyDIxW7COkSpowF3NrPDDMKVMcDfsWLSIHsarl5Ti1RTv XURCNrmVKENairbNuq7jpLE/3JMVw0oB7+uaeZ5SC2zRoRGQ7mwhap3fSKoHqOgW6R FNIrgK4JDhYbA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2PE7LW8035685; Thu, 25 Mar 2010 15:07:21 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 25 Mar 2010 15:07:21 +0100 Message-ID: <20100325150721.113962su1wkcpao8@webmail.leidinger.net> Date: Thu, 25 Mar 2010 15:07:21 +0100 From: Alexander Leidinger To: "M. Warner Losh" References: <201003241132.29588.jhb@freebsd.org> <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> <201003250749.57994.jhb@freebsd.org> <20100325.073736.635700942514937954.imp@bsdimp.com> In-Reply-To: <20100325.073736.635700942514937954.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 21D2F8455F7.763F7 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270130846.0547@FmRxKDipIwpKU5orzUwfng X-EBL-Spam-Status: No Cc: rwatson@FreeBSD.org, jhb@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2010 14:07:30 -0000 Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 07:37:36 -0600 (MDT)): > In general, I think this is OK. I'd like to see the revised patch > before it is committed. My current timeschedule: - tomorrow: review the patch again myself (ENOTIME today) - tomorrow: provide a patch for others to review - if nobody complains (problem in the patch or ENOTIME for review), commit it sometime next week Bye, Alexander. -- Leela: "There it is, the near-death star." http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 09:15:35 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DB48106566B; Fri, 26 Mar 2010 09:15:35 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 140548FC17; Fri, 26 Mar 2010 09:15:34 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2C1F3.dip.t-dialin.net [217.226.193.243]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 5F92C844054; Fri, 26 Mar 2010 10:15:28 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 309425112; Fri, 26 Mar 2010 10:15:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269594925; bh=5SQqbrXikZid6oSBpz9ejX7Q61lJhwT7PqBGXljn4mY=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=VQO0R8XvJrgsS79Pr1ktl++jAtoEG3AysJxa1+oufdmvOhpsefMD1L9l9JfojsWKs xAPRvVUtreR9y4Vmc4qkN012C2DYUexDrTrQXe//fjk/zZK54x9nQ+R6bS+R03Vtkb obYYcKcGu/Sk7SLmaAfEPoD8SCitgIdyyQ1ENTMLbgQZgwBUlgAVTfDwn09QmoDGY2 NuWw91QNHU/2RIlYxnji+jNOscKQmlbAmCKlo97W6yQtOgLRqfSHVQ7aCPMNTY4Bnn HvefK9U5h6BOlUH4JXGjn1osWQu9/ZfBko/nG6asVTmFST3NALjFqhSR0OAjsRLUgW J/reaqmwCx4sg== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2Q9FOgT001390; Fri, 26 Mar 2010 10:15:24 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 26 Mar 2010 10:15:24 +0100 Message-ID: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> Date: Fri, 26 Mar 2010 10:15:24 +0100 From: Alexander Leidinger To: "M. Warner Losh" References: <201003241132.29588.jhb@freebsd.org> <20100325105615.10186jr08a5oj00s@webmail.leidinger.net> <201003250749.57994.jhb@freebsd.org> <20100325.073736.635700942514937954.imp@bsdimp.com> In-Reply-To: <20100325.073736.635700942514937954.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 5F92C844054.10FDF X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270199730.81069@+FOTgcwzzgGo6Yfl8GGmbA X-EBL-Spam-Status: No Cc: rwatson@FreeBSD.org, jhb@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 09:15:35 -0000 Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 07:37:36 -0600 (MDT)): > In general, I think this is OK. I'd like to see the revised patch > before it is committed. And here it is: http://www.Leidinger.net/test/ctf3.diff My commit log would be something like this: ---snip--- WITH_CTF can now be specified in src.conf (not recommended, there are some problems with static executables), make.conf (would also affect ports which do not use GNU make and do not override the compile targets) or in the kernel config (via "makeoptions WITH_CTF=yes"). Additional (related) changes: - propagate WITH_CTF to module builds - do not add -g to the linker flags, it's a noop there anyway (at least according to the man page of ld) - do not add -g yo CFLAGS unconditionally we need to have a look if it is really needed (IMO not) or if there is a way to add it only when WITH_CTF is used Note: don't worry when you see ctfconvert lines appearing in your build, they are protected with a shell conditional and are not run as long as you don't have WITH_CTF defined. Reviewed by: imp, jhb, scottl (earlier version) Discussed on: arch@ ---snip--- If nobody comes up with problems (or wants more time for the review): I would like to commit this next week. Bye, Alexander. -- We are each only one drop in a great ocean -- but some of the drops sparkle! http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 13:37:48 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 302C0106564A; Fri, 26 Mar 2010 13:37:48 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 00E1C8FC0A; Fri, 26 Mar 2010 13:37:48 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A527446B7F; Fri, 26 Mar 2010 09:37:47 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id A2EF08A021; Fri, 26 Mar 2010 09:37:46 -0400 (EDT) From: John Baldwin To: Alexander Leidinger Date: Fri, 26 Mar 2010 07:50:23 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201003241132.29588.jhb@freebsd.org> <20100325.073736.635700942514937954.imp@bsdimp.com> <20100326101524.15695bisy2324t8g@webmail.leidinger.net> In-Reply-To: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003260750.23159.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 26 Mar 2010 09:37:46 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 13:37:48 -0000 On Friday 26 March 2010 5:15:24 am Alexander Leidinger wrote: > Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 > 07:37:36 -0600 (MDT)): > > > In general, I think this is OK. I'd like to see the revised patch > > before it is committed. > > And here it is: > http://www.Leidinger.net/test/ctf3.diff > > My commit log would be something like this: > ---snip--- > WITH_CTF can now be specified in src.conf (not recommended, there > are some problems with static executables), make.conf (would also > affect ports which do not use GNU make and do not override the > compile targets) or in the kernel config (via "makeoptions > WITH_CTF=yes"). > > Additional (related) changes: > - propagate WITH_CTF to module builds > - do not add -g to the linker flags, it's a noop there anyway > (at least according to the man page of ld) > - do not add -g yo CFLAGS unconditionally > we need to have a look if it is really needed (IMO not) or if there > is a way to add it only when WITH_CTF is used > > Note: don't worry when you see ctfconvert lines appearing in your build, > they are protected with a shell conditional and are not run as long as > you don't have WITH_CTF defined. Hmm, perhaps put an @ on the CTF* lines for now to remove them from make output since they are just noise in the common case? I expect at some point in the future that we will end up reverting this commit once we have working userland dtrace and will store CTF symbols in everything by default at which point the '@' can be removed, but until then I think the extra noise outways the gain. > Reviewed by: imp, jhb, scottl (earlier version) > Discussed on: arch@ > ---snip--- > > If nobody comes up with problems (or wants more time for the review): > I would like to commit this next week. > > Bye, > Alexander. > > -- > We are each only one drop in a great > ocean -- but some of the drops sparkle! > > http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 > http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 > -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 15:45:04 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B46DC1065672; Fri, 26 Mar 2010 15:45:04 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 7006E8FC0A; Fri, 26 Mar 2010 15:45:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2QFbCWC011171; Fri, 26 Mar 2010 09:37:12 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 26 Mar 2010 09:37:17 -0600 (MDT) Message-Id: <20100326.093717.857133809997909007.imp@bsdimp.com> To: Alexander@Leidinger.net From: "M. Warner Losh" In-Reply-To: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> References: <201003250749.57994.jhb@freebsd.org> <20100325.073736.635700942514937954.imp@bsdimp.com> <20100326101524.15695bisy2324t8g@webmail.leidinger.net> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 15:45:04 -0000 In message: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> Alexander Leidinger writes: : Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 : 07:37:36 -0600 (MDT)): : : > In general, I think this is OK. I'd like to see the revised patch : > before it is committed. : : And here it is: : http://www.Leidinger.net/test/ctf3.diff : : My commit log would be something like this: : ---snip--- : WITH_CTF can now be specified in src.conf (not recommended, there : are some problems with static executables), make.conf (would also : affect ports which do not use GNU make and do not override the : compile targets) or in the kernel config (via "makeoptions : WITH_CTF=yes"). : : Additional (related) changes: : - propagate WITH_CTF to module builds : - do not add -g to the linker flags, it's a noop there anyway : (at least according to the man page of ld) : - do not add -g yo CFLAGS unconditionally : we need to have a look if it is really needed (IMO not) or if there : is a way to add it only when WITH_CTF is used : : Note: don't worry when you see ctfconvert lines appearing in your : build, : they are protected with a shell conditional and are not run as long as : you don't have WITH_CTF defined. : : Reviewed by: imp, jhb, scottl (earlier version) : Discussed on: arch@ : ---snip--- : : If nobody comes up with problems (or wants more time for the review): : I would like to commit this next week. I agree with John about the @ sign. I'm a little worried about the WITH_CTF being special, but not documented as being special. Can you add a note to bsd.own.mk documenting it as such? Also, can you document WITH_CTF in build.9 please? I know that build.9 needs some updating, but we should at least try to keep up with new things. Many are missing now. This would also be a good place to mention WITH_CTF is special. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 16:12:52 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4938F1065674; Fri, 26 Mar 2010 16:12:52 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id DEF5B8FC20; Fri, 26 Mar 2010 16:12:51 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2C1F3.dip.t-dialin.net [217.226.193.243]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id A2653844054; Fri, 26 Mar 2010 17:12:44 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id B1B9A5074; Fri, 26 Mar 2010 17:12:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1269619961; bh=uaAKnfa+49et7kUbwI2eCzqIPsJid1OFqjjcP1prMXg=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=kMEuM66nant8VV2UHeNs9Au1Ly1rLgVXtwPOr3d7pLE7kSSuk4vcm9Uum9a6FUXLi CxhjGGACh1SaRVt2rXUjxw7ajHVdKSXOlykvDVR2OMwQFxL5ZyPSCcePhvIWikOq2V K6oT4KvxpoEdpSRuwwTHm1WvYuVpU/ERmF8Z2U7I84zbJIM8wrnjTsK6BF+3mOrf2Q eqbDitisjuN+nXK7WZCuCvP89117Ux7I73AwoPVTVvXrYRl8ITItEFmUrWsIbbW/pb B0lG0a1nYPbkd9xw3lCTlJl1PV0fAnkrWBBGo8ah+Azd6zpCxNX/zLU00LTQwuwkqg GRlDU+WCXZA9A== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2QGCfHq068172; Fri, 26 Mar 2010 17:12:41 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 26 Mar 2010 17:12:41 +0100 Message-ID: <20100326171241.16524bklcroedou8@webmail.leidinger.net> Date: Fri, 26 Mar 2010 17:12:41 +0100 From: Alexander Leidinger To: "M. Warner Losh" References: <201003250749.57994.jhb@freebsd.org> <20100325.073736.635700942514937954.imp@bsdimp.com> <20100326101524.15695bisy2324t8g@webmail.leidinger.net> <20100326.093717.857133809997909007.imp@bsdimp.com> In-Reply-To: <20100326.093717.857133809997909007.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: A2653844054.1EACD X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1270224767.57603@9fr1sFGZ9x/Xo+56skf6oQ X-EBL-Spam-Status: No Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 16:12:52 -0000 Quoting "M. Warner Losh" (from Fri, 26 Mar 2010 09:37:17 -0600 (MDT)): > In message: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> > Alexander Leidinger writes: > : Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 > : 07:37:36 -0600 (MDT)): > : > : > In general, I think this is OK. I'd like to see the revised patch > : > before it is committed. > : > : And here it is: > : http://www.Leidinger.net/test/ctf3.diff > : > : My commit log would be something like this: > : ---snip--- > : WITH_CTF can now be specified in src.conf (not recommended, there > : are some problems with static executables), make.conf (would also > : affect ports which do not use GNU make and do not override the > : compile targets) or in the kernel config (via "makeoptions > : WITH_CTF=yes"). > : > : Additional (related) changes: > : - propagate WITH_CTF to module builds > : - do not add -g to the linker flags, it's a noop there anyway > : (at least according to the man page of ld) > : - do not add -g yo CFLAGS unconditionally > : we need to have a look if it is really needed (IMO not) or if there > : is a way to add it only when WITH_CTF is used > : > : Note: don't worry when you see ctfconvert lines appearing in your > : build, > : they are protected with a shell conditional and are not run as long as > : you don't have WITH_CTF defined. > : > : Reviewed by: imp, jhb, scottl (earlier version) > : Discussed on: arch@ > : ---snip--- > : > : If nobody comes up with problems (or wants more time for the review): > : I would like to commit this next week. > > I agree with John about the @ sign. > > I'm a little worried about the WITH_CTF being special, but not > documented as being special. Can you add a note to bsd.own.mk > documenting it as such? > > Also, can you document WITH_CTF in build.9 please? I know that > build.9 needs some updating, but we should at least try to keep up > with new things. Many are missing now. This would also be a good > place to mention WITH_CTF is special. I will have a look at it next week. As I'm not a native english speaker, you can expect another patch for review (text suggestions for build.9 are welcome, I will take care about the mdoc markup then). Bye, Alexander. -- Multiple-function gadgets will not perform any function adequately. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 16:29:01 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C144F1065678; Fri, 26 Mar 2010 16:29:01 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 620A38FC34; Fri, 26 Mar 2010 16:29:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2QGNeJj011743; Fri, 26 Mar 2010 10:23:40 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 26 Mar 2010 10:23:44 -0600 (MDT) Message-Id: <20100326.102344.364422370204095721.imp@bsdimp.com> To: Alexander@Leidinger.net From: "M. Warner Losh" In-Reply-To: <20100326171241.16524bklcroedou8@webmail.leidinger.net> References: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> <20100326.093717.857133809997909007.imp@bsdimp.com> <20100326171241.16524bklcroedou8@webmail.leidinger.net> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 16:29:01 -0000 In message: <20100326171241.16524bklcroedou8@webmail.leidinger.net> Alexander Leidinger writes: : Quoting "M. Warner Losh" (from Fri, 26 Mar 2010 : 09:37:17 -0600 (MDT)): : : > In message: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> : > Alexander Leidinger writes: : > : Quoting "M. Warner Losh" (from Thu, 25 Mar 2010 : > : 07:37:36 -0600 (MDT)): : > : : > : > In general, I think this is OK. I'd like to see the revised patch : > : > before it is committed. : > : : > : And here it is: : > : http://www.Leidinger.net/test/ctf3.diff : > : : > : My commit log would be something like this: : > : ---snip--- : > : WITH_CTF can now be specified in src.conf (not recommended, there : > : are some problems with static executables), make.conf (would also : > : affect ports which do not use GNU make and do not override the : > : compile targets) or in the kernel config (via "makeoptions : > : WITH_CTF=yes"). : > : : > : Additional (related) changes: : > : - propagate WITH_CTF to module builds : > : - do not add -g to the linker flags, it's a noop there anyway : > : (at least according to the man page of ld) : > : - do not add -g yo CFLAGS unconditionally : > : we need to have a look if it is really needed (IMO not) or if there : > : is a way to add it only when WITH_CTF is used : > : : > : Note: don't worry when you see ctfconvert lines appearing in your : > : build, : > : they are protected with a shell conditional and are not run as long : > as : > : you don't have WITH_CTF defined. : > : : > : Reviewed by: imp, jhb, scottl (earlier version) : > : Discussed on: arch@ : > : ---snip--- : > : : > : If nobody comes up with problems (or wants more time for the : > review): : > : I would like to commit this next week. : > : > I agree with John about the @ sign. : > : > I'm a little worried about the WITH_CTF being special, but not : > documented as being special. Can you add a note to bsd.own.mk : > documenting it as such? : > : > Also, can you document WITH_CTF in build.9 please? I know that : > build.9 needs some updating, but we should at least try to keep up : > with new things. Many are missing now. This would also be a good : > place to mention WITH_CTF is special. : : I will have a look at it next week. As I'm not a native english : speaker, you can expect another patch for review (text suggestions for : build.9 are welcome, I will take care about the mdoc markup then). How does this text look (posted so everybody can kibitz): WITH_CTF the build process will run the DTrace CTF conversion tools on built objects. Please note that this WITH_ option is handled differently than all other WITH_ options (there's no WITHOUT_CTF, or corresponding MK_CTF in the build system). Warner From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 18:34:15 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 458DD1065670; Fri, 26 Mar 2010 18:34:15 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 884F28FC27; Fri, 26 Mar 2010 18:34:14 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 765D8A59F68; Sat, 27 Mar 2010 02:34:13 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id 1PuJzITjEXt2; Sat, 27 Mar 2010 02:34:07 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id E3C42A59F4D; Sat, 27 Mar 2010 02:34:03 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:x-enigmail-version:openpgp:content-type; b=OjD7vZIS2jjneeWc+1QdPx5tTkVK3BFoX8KNSGqvcfCqb8iuboRe4L0hvA9gr0QdA lAPmJVUfMoDqb1RrQw3EQ== Message-ID: <4BACFE18.7010309@delphij.net> Date: Fri, 26 Mar 2010 11:34:00 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: freebsd-arch@freebsd.org, ports@freebsd.org X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------000309060306060201000507" Cc: Alexander Logvinov , the_paya@gentoo.org Subject: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 18:34:15 -0000 This is a multi-part message in MIME format. --------------000309060306060201000507 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, The recent zlib import has added some assumption that _LARGEFILE_64_SOURCE is only defined on systems with System V style *64 interface. Moreover, I have added _FILE_OFFSET_BITS = 64 definition into zconf.h so that it would pick up the 64 bit interface properly. This unfortunately could cause some namespace pollution. As such, I would propose the attached changes to zlib headers: zconf.h: * If _LARGEFILE_64_SOURCE is defined, set __FreeBSD_LARGEFILE_64_SOURCE and undefine it, as it would break zlib.h * If _FILE_OFFSET_BITS is undefined, set __FreeBSD_FILE_OFFSET_BITS and define it as 64. zlib.h: * If __FreeBSD_LARGEFILE_64_SOURCE is defined and _LARGEFILE_64_SOURCE undefined, undefine __FreeBSD_LARGEFILE_64_SOURCE and define _LARGEFILE_64_SOURCE. * If __FreeBSD_FILE_OFFSET_BITS is defined and _FILE_OFFSET_BITS is defined, undefine both. This approach is kind of mess, though, but would avoid massive changes which I'd propose for next zlib release. Comments? Objections? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLrP4XAAoJEATO+BI/yjfBk2YH/Ag38kdtjxAk0l2kdgnHPnZ7 Wf9uk+0ixgE8X2uHfkOeiVO99Ma47aFU/thS1qgXRIWqP/iQEMqOiUayubYnsCJk K8quwzEuifM0hlIPzHxgzo5/e1O6GhUdIkJVJj+T//twG2BGXziYHMye/aph0iRa kW5DEq469jBoz62N8FDn4iatZoXT5boBc0bE3GQCKJhUADbpC84vjCCHfdVx50mu x5hEO88TNaWSn4AkPgs0xPBYQNM+w6t2g/CLNfylumIUVHcSs+v8sLKrxdqqvKNx hn97KmDagy5BVaWaAFAqFclgfAVbjfa8NIaOr8egxnuVHXTzEzjHFUD7fS22Oqo= =eOpg -----END PGP SIGNATURE----- --------------000309060306060201000507 Content-Type: text/plain; name="zlib.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zlib.diff" SW5kZXg6IGxpYi9saWJ6L3pjb25mLmgKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gbGliL2xpYnovemNv bmYuaAkocmV2aXNpb24gMjA1NjUxKQorKysgbGliL2xpYnovemNvbmYuaAkod29ya2luZyBj b3B5KQpAQCAtMzc1LDYgKzM3NSwxMyBAQAogIyAgZW5kaWYKICNlbmRpZgogCisjaWYgZGVm aW5lZChfX0ZyZWVCU0RfXykgJiYgZGVmaW5lZChfTEFSR0VGSUxFNjRfU09VUkNFKQorI2lm ICFkZWZpbmVkKF9fRnJlZUJTRF9MQVJHRUZJTEU2NF9TT1VSQ0UpCisjZGVmaW5lIF9fRnJl ZUJTRF9MQVJHRUZJTEU2NF9TT1VSQ0UKKyNlbmRpZgorI3VuZGVmIF9MQVJHRUZJTEU2NF9T T1VSQ0UKKyNlbmRpZgorCiAjaWZkZWYgX0xBUkdFRklMRTY0X1NPVVJDRQogIyAgaW5jbHVk ZSA8c3lzL3R5cGVzLmg+CiAjZW5kaWYKQEAgLTM5MSw2ICszOTgsOSBAQAogI2luY2x1ZGUg PHN5cy90eXBlcy5oPgogI2RlZmluZQl6X29mZl90CW9mZl90CiAjaWZuZGVmIF9GSUxFX09G RlNFVF9CSVRTCisjaWYgIWRlZmluZWQoX19GcmVlQlNEX0ZJTEVfT0ZGU0VUX0JJVFMpCisj ZGVmaW5lIF9fRnJlZUJTRF9GSUxFX09GRlNFVF9CSVRTCisjZW5kaWYKICNkZWZpbmUgX0ZJ TEVfT0ZGU0VUX0JJVFMgNjQKICNlbmRpZgogCkluZGV4OiBsaWIvbGliei96bGliLmgKPT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQotLS0gbGliL2xpYnovemxpYi5oCShyZXZpc2lvbiAyMDU2NTEpCisrKyBs aWIvbGliei96bGliLmgJKHdvcmtpbmcgY29weSkKQEAgLTE1OTcsNiArMTU5NywyMCBAQAog WkVYVEVSTiBjb25zdCB1TG9uZ2YgKiBaRVhQT1JUIGdldF9jcmNfdGFibGUgICAgT0YoKHZv aWQpKTsKIFpFWFRFUk4gaW50ICAgICAgICAgICAgWkVYUE9SVCBpbmZsYXRlVW5kZXJtaW5l IE9GKCh6X3N0cmVhbXAsIGludCkpOwogCisjaWYgZGVmaW5lZChfX0ZyZWVCU0RfTEFSR0VG SUxFNjRfU09VUkNFKQorI3VuZGVmIF9fRnJlZUJTRF9MQVJHRUZJTEU2NF9TT1VSQ0UKKyNp ZiAhZGVmaW5lZChfTEFSR0VGSUxFNjRfU09VUkNFKQorI2RlZmluZSBfTEFSR0VGSUxFNjRf U09VUkNFCisjZW5kaWYKKyNlbmRpZgorCisjaWYgZGVmaW5lZChfX0ZyZWVCU0RfRklMRV9P RkZTRVRfQklUUykKKyN1bmRlZiBfX0ZyZWVCU0RfRklMRV9PRkZTRVRfQklUUworI2lmIGRl ZmluZWQoX0ZJTEVfT0ZGU0VUX0JJVFMpCisjdW5kZWYgX0ZJTEVfT0ZGU0VUX0JJVFMKKyNl bmRpZgorI2VuZGlmCisKICNpZmRlZiBfX2NwbHVzcGx1cwogfQogI2VuZGlmCg== --------------000309060306060201000507-- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 21:36:06 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from [127.0.0.1] (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by hub.freebsd.org (Postfix) with ESMTP id 4899D106566B; Fri, 26 Mar 2010 21:36:06 +0000 (UTC) (envelope-from jkim@FreeBSD.org) From: Jung-uk Kim To: freebsd-arch@FreeBSD.org, d@delphij.net Date: Fri, 26 Mar 2010 17:35:52 -0400 User-Agent: KMail/1.6.2 References: <4BACFE18.7010309@delphij.net> In-Reply-To: <4BACFE18.7010309@delphij.net> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003261735.56279.jkim@FreeBSD.org> Cc: ports@freebsd.org, the_paya@gentoo.org, Alexander Logvinov Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 21:36:07 -0000 On Friday 26 March 2010 02:34 pm, Xin LI wrote: > Hi, > > The recent zlib import has added some assumption that > _LARGEFILE_64_SOURCE is only defined on systems with System V style > *64 interface. Moreover, I have added _FILE_OFFSET_BITS = 64 > definition into zconf.h so that it would pick up the 64 bit > interface properly. I guess you meant _LARGEFILE64_SOURCE. :-) Last night I had so much trouble rebuilding ports because many ports have bogus assumption that _LARGEFILE64_SOURCE is required for supporting large files. It seems _LARGEFILE64_SOURCE is only needed for OSes where sizeof(off_t) is 4 and it provides additional functions such as fseeko64() and ftello64(), which takes off64_t type as an argument or returns off64_t type. However, _FILE_OFFSET_BITS = 64 means sizeof(off_t) is 8 but you have to avoid off64_t, fseeko46 (), ftello64(), etc, etc... http://www.gnu.org/s/libc/manual/html_node/Feature-Test-Macros.html If I read them all correctly, it means (on 32-bit platforms): Case M1 M2 M3 T1 T2 F1 F2 Note ---------------------------------- 1 N N N 4 N N N 2 N N Y 8 N N N 3 N Y N 4 ? ? ? [1] 4 N Y Y 8 ? ? ? [1] 5 Y N N 4 ? ? ? [2] 6 Y N Y 8 N Y N [3] 7 Y Y N 4 Y Y Y [4] 8 Y Y Y 8 Y Y Y [5] M1: _LARGEFILE_SOURCE (N: undefined, Y: defined) M2: _LARGEFILE64_SOURCE (N: undefined, Y: defined) M3: _FILE_OFFSET_BITS (N: undefined or 32, Y: 64) T1: off_t (4: 32-bit, 8: 64-bit) T2: off64_t (N: unavail, Y: avail) F1: fseeko() and ftello() (N: unavail, Y: avail) F2: fseeko64() and ftello64() (N: invisible, Y: visible) [1] Impossible. Some people argue that _LARGEFILE64_SOURCE must imply _LARGEFILE_SOURCE. [2] Impossible. Some people argue that _LARGEFILE_SOURCE must imply _LARGEFILE64_SOURCE and/or _FILE_OFFSET_BITS=64. [3] All BSDs (including Darwin) and the future of Linux. ;-) [4] Transitional according to the GNU libc manual. [5] It is wrong but I think it exists. So, zlib developers wanted to distinguish #6 from #7 and #8. I think we can do something like Apple did: http://trac.macports.org/changeset/65036 Basically _LARGEFILE64_SOURCE is completely unnecessary and evil for all BSDs. > This unfortunately could cause some namespace pollution. As such, > I would propose the attached changes to zlib headers: > > zconf.h: > * If _LARGEFILE_64_SOURCE is defined, set > __FreeBSD_LARGEFILE_64_SOURCE and undefine it, as it would break > zlib.h > * If _FILE_OFFSET_BITS is undefined, set > __FreeBSD_FILE_OFFSET_BITS and define it as 64. > > zlib.h: > * If __FreeBSD_LARGEFILE_64_SOURCE is defined and > _LARGEFILE_64_SOURCE undefined, undefine > __FreeBSD_LARGEFILE_64_SOURCE and define _LARGEFILE_64_SOURCE. > * If __FreeBSD_FILE_OFFSET_BITS is defined and _FILE_OFFSET_BITS > is defined, undefine both. > > This approach is kind of mess, though, but would avoid massive > changes which I'd propose for next zlib release. > > Comments? Objections? Please don't do that. I think we just have to fix individual ports for now, something like this: http://trac.macports.org/changeset/64855 It seems the author of zlib suggested it is better solution, actually: http://trac.macports.org/ticket/24061 FYI, there is a web site dedicated for this issue: http://ac-archive.sourceforge.net/largefile/index.html Yes, I had to google a lot last night... :-( Jung-uk Kim From owner-freebsd-arch@FreeBSD.ORG Fri Mar 26 21:55:24 2010 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BAAB106564A for ; Fri, 26 Mar 2010 21:55:24 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.64.117]) by mx1.freebsd.org (Postfix) with ESMTP id 8EF558FC1A for ; Fri, 26 Mar 2010 21:55:23 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.3/8.14.3) with ESMTP id o2QLH66m043465 for ; Sat, 27 Mar 2010 00:17:06 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.3/8.14.3/Submit) id o2QLH66P043464 for arch@FreeBSD.org; Sat, 27 Mar 2010 00:17:06 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Sat, 27 Mar 2010 00:17:06 +0300 From: Gleb Smirnoff To: arch@FreeBSD.org Message-ID: <20100326211706.GI18894@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Subject: touch panel support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Mar 2010 21:55:24 -0000 Hello, I've got a display with touch panel, and I'd like to get in working in FreeBSD. The touch panel is supported by NetBSD's uep(4). So far, I have written uep(4) for FreeBSD, that successfully reads and parses data from the USB touch panel device. And then I've got a problem. Our mouse subsystem is not ready for touch panels. Our mouse(4) protocol does not support mouse driver passing _absolute_ coordinates to the mouse(4) subsystem. It only expects a relative movement of the mouse. But _absolute_ coordinates are principal idea of any touch panel. The lesser problem is lack of generic support for touch panel calibration. Both of these problems are solved in NetBSD. They've got a wsmux(4) device, just like our kbdmux(4), but for mice. This mouse multiplexer can also understand absolute coordinates from underlying mice drivers. NetBSD also has a generic support for calibration of touch panels. What is the FreeBSD future way to go: port things for NetBSD? Write something different? -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 00:02:17 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79D2C106566B; Sat, 27 Mar 2010 00:02:17 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3B00B8FC08; Sat, 27 Mar 2010 00:02:16 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 271ED1FFC22; Sat, 27 Mar 2010 00:02:16 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id EA32F8449F; Sat, 27 Mar 2010 01:02:15 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: d@delphij.net References: <4BACFE18.7010309@delphij.net> Date: Sat, 27 Mar 2010 01:02:15 +0100 In-Reply-To: <4BACFE18.7010309@delphij.net> (Xin LI's message of "Fri, 26 Mar 2010 11:34:00 -0700") Message-ID: <86wrwylji0.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: ports@freebsd.org, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 00:02:17 -0000 Xin LI writes: > The recent zlib import has added some assumption that > _LARGEFILE_64_SOURCE is only defined on systems with System V style *64 > interface. Moreover, I have added _FILE_OFFSET_BITS =3D 64 definition > into zconf.h so that it would pick up the 64 bit interface properly. This is wrong, FreeBSD has native 64-bit stat() etc. and does not need _LARGEFILE_WHATEVER. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 00:26:17 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DFF9106566B; Sat, 27 Mar 2010 00:26:17 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0FCA28FC19; Sat, 27 Mar 2010 00:26:17 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 6D6F9A5A1A8; Sat, 27 Mar 2010 08:26:15 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id WKz2Jlykemuk; Sat, 27 Mar 2010 08:26:09 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 61222A5A099; Sat, 27 Mar 2010 08:26:07 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=qxIvKDAWVhUzY15QzO9Fk1S59KeD0eCcPdveNKQ1EDMsyyPkQzbBpVRV8RKOtbi/r FTtV1wRrtc6yNbS4F/shA== Message-ID: <4BAD509B.3080805@delphij.net> Date: Fri, 26 Mar 2010 17:26:03 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <4BACFE18.7010309@delphij.net> <86wrwylji0.fsf@ds4.des.no> In-Reply-To: <86wrwylji0.fsf@ds4.des.no> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: ports@freebsd.org, d@delphij.net, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 00:26:17 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/26 17:02, Dag-Erling Smørgrav wrote: > Xin LI writes: >> The recent zlib import has added some assumption that >> _LARGEFILE_64_SOURCE is only defined on systems with System V style *64 >> interface. Moreover, I have added _FILE_OFFSET_BITS = 64 definition >> into zconf.h so that it would pick up the 64 bit interface properly. > > This is wrong, FreeBSD has native 64-bit stat() etc. and does not need > _LARGEFILE_WHATEVER. Yes we do not need that and it just cause compilation errors. The problem is that some third party software thinks that they need to define _LARGEFILE64_*, which will break zlib.h on FreeBSD :( I'm inclined to adopt a solution similar to what Apple (pointed out by Jkim@) is currently having by disabling the _LARGEFILE64_SOURCE if __FreeBSD__ is defined. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLrVCbAAoJEATO+BI/yjfBj4AIAMcUAjLIZpNW2sGD0/Z9XLU3 SBevqjvR9iwGANTXOKiB3aofvUygmTfG+8KrxlZTJ51O8vlYgA28eGT0iiDpfoLz yUtpAN1MIitPp/VtNwHpTpfJcfP+AX060G4MGdxUWCHjoJWhbWMv7OnLbquGdglZ 8bbvQR9EDxm8gM2OT9/b14WOsilYwFBpNlNvxl+Q9d0oWNIi08xWmwvC2aWF7L/6 oTFp2tCKXfi++RCxPU5v+q5SaogKjx1bw612UWvetEhHylcXQpQxqjtyL5WA5IAd bRYJquOjt3+3mziE0VLlQSQgUSCSYMXls6LqmQSfe3W1sU7wG1rFlZG5BfD/UyM= =DDry -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 00:46:28 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E663106566C; Sat, 27 Mar 2010 00:46:28 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 382B88FC12; Sat, 27 Mar 2010 00:46:27 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 35EA61FFC58; Sat, 27 Mar 2010 00:46:27 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 13390844DA; Sat, 27 Mar 2010 01:46:27 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: d@delphij.net References: <4BACFE18.7010309@delphij.net> <86wrwylji0.fsf@ds4.des.no> <4BAD509B.3080805@delphij.net> Date: Sat, 27 Mar 2010 01:46:26 +0100 In-Reply-To: <4BAD509B.3080805@delphij.net> (Xin LI's message of "Fri, 26 Mar 2010 17:26:03 -0700") Message-ID: <86ljdelhgd.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: ports@freebsd.org, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 00:46:28 -0000 Xin LI writes: > The problem is that some third party software thinks that they need to > define _LARGEFILE64_*, which will break zlib.h on FreeBSD :( Then that third-party software is broken and needs to be fixed. _LARGEFILE64_SOURCE is (supposed to be) used to expose the stat64() API. FreeBSD does not have stat64(). Any application that defines it and then calls stat() instead of stat64() is broken to begin with. Any application that defines it and then calls stat64() will not compile on FreeBSD. See sections 3.3.2 and 3.1 of this document: http://www.unix.org/version2/whatsnew/lfs20mar.html On Linux, it's a no-op, because while the kernel has separate 32-bit stat() and 64-bit stat64() syscalls, glibc aliases stat() to stat64(). DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 00:49:31 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBD69106566C; Sat, 27 Mar 2010 00:49:31 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 7CFAB8FC1E; Sat, 27 Mar 2010 00:49:31 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 6761B1FFC5C; Sat, 27 Mar 2010 00:49:30 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 46AB0844DA; Sat, 27 Mar 2010 01:49:30 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: d@delphij.net References: <4BACFE18.7010309@delphij.net> Date: Sat, 27 Mar 2010 01:49:30 +0100 In-Reply-To: <4BACFE18.7010309@delphij.net> (Xin LI's message of "Fri, 26 Mar 2010 11:34:00 -0700") Message-ID: <86hbo2lhb9.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: ports@freebsd.org, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 00:49:31 -0000 Xin LI writes: > The recent zlib import has added some assumption that > _LARGEFILE_64_SOURCE is only defined on systems with System V style > *64 interface. I forgot to address this point in your original message. Basically, the entire thread boils down to this: that assumption is correct. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 01:43:58 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCFA31065672; Sat, 27 Mar 2010 01:43:58 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 823058FC24; Sat, 27 Mar 2010 01:43:58 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 98D15A5643B; Sat, 27 Mar 2010 09:43:56 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id l4PnWyQCnquX; Sat, 27 Mar 2010 09:43:49 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id D6759A5514B; Sat, 27 Mar 2010 09:43:46 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=BCUummhlLxINy0+9mDgRxyHYy/F22D7yOlELcZTXq4LDGKbzg7s42G/eKpHpVOInW vEGdRnW8NVdYxIZY2zXLA== Message-ID: <4BAD62CF.6090901@delphij.net> Date: Fri, 26 Mar 2010 18:43:43 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: =?UTF-8?B?RGFnLUVybGluZyBTbcO4cmdyYXY=?= References: <4BACFE18.7010309@delphij.net> <86wrwylji0.fsf@ds4.des.no> <4BAD509B.3080805@delphij.net> <86ljdelhgd.fsf@ds4.des.no> In-Reply-To: <86ljdelhgd.fsf@ds4.des.no> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: ports@freebsd.org, d@delphij.net, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 01:43:59 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/26 17:46, Dag-Erling Smørgrav wrote: > Xin LI writes: >> The problem is that some third party software thinks that they need to >> define _LARGEFILE64_*, which will break zlib.h on FreeBSD :( > > Then that third-party software is broken and needs to be fixed. > > _LARGEFILE64_SOURCE is (supposed to be) used to expose the stat64() API. > FreeBSD does not have stat64(). Any application that defines it and > then calls stat() instead of stat64() is broken to begin with. Any > application that defines it and then calls stat64() will not compile on > FreeBSD. > > See sections 3.3.2 and 3.1 of this document: > > http://www.unix.org/version2/whatsnew/lfs20mar.html > > On Linux, it's a no-op, because while the kernel has separate 32-bit > stat() and 64-bit stat64() syscalls, glibc aliases stat() to stat64(). So... May I consider my import just exposed some existing bugs in other applications and we don't want to workaround these issues? I'm sort of feeling guilty for making the transition path hard, though... Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLrWLPAAoJEATO+BI/yjfBJAcH/0WLPx5wiV/5ue4ZmmdPojMi bxK0XneEwO56bJMOJHg6qxBqwwBm3egabq1abkRYLdOVwoXc9hiGAdVJjjymJ3lz xJWV23XpLHzso9z3Ev33virj32+Br++zsucdh5aEmC0YvdpvFDQUiU9LUNIErf/g bjqzrapugiEkrL8xD2Maq5F+OdeMPOV3HXMjU39RpyRKVTfIkG4tfL8wDmBD/KAI 7byS1syUqDP2uvIvHmO2R3lFrto6cjwRhn38Y51XOQpu/Wvrp6KEKX47/vFBUjwE JHPIGlbkoo3LezPjE+Sv6I4+MAsNncmyol5jKGAxmfe9wNjkHs3Br/AyPGbNCyI= =00ta -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 01:51:05 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5BCF106564A; Sat, 27 Mar 2010 01:51:05 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 3DF008FC08; Sat, 27 Mar 2010 01:51:04 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 981E61FFC5B; Sat, 27 Mar 2010 01:51:03 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 71C76844DA; Sat, 27 Mar 2010 02:51:03 +0100 (CET) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: d@delphij.net References: <4BACFE18.7010309@delphij.net> <86wrwylji0.fsf@ds4.des.no> <4BAD509B.3080805@delphij.net> <86ljdelhgd.fsf@ds4.des.no> <4BAD62CF.6090901@delphij.net> Date: Sat, 27 Mar 2010 02:51:03 +0100 In-Reply-To: <4BAD62CF.6090901@delphij.net> (Xin LI's message of "Fri, 26 Mar 2010 18:43:43 -0700") Message-ID: <868w9elego.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.95 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: ports@freebsd.org, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@freebsd.org Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 01:51:05 -0000 Xin LI writes: > So... May I consider my import just exposed some existing bugs in other > applications and we don't want to workaround these issues? Correct. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 09:18:51 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7148106564A; Sat, 27 Mar 2010 09:18:51 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout0.freenet.de (mout0.freenet.de [IPv6:2001:748:100:40::2:2]) by mx1.freebsd.org (Postfix) with ESMTP id 690928FC1C; Sat, 27 Mar 2010 09:18:51 +0000 (UTC) Received: from [195.4.92.25] (helo=15.mx.freenet.de) by mout0.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NvSAJ-0007Kf-9y; Sat, 27 Mar 2010 10:18:47 +0100 Received: from p57ae0231.dip0.t-ipconnect.de ([87.174.2.49]:20840 helo=ernst.jennejohn.org) by 15.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NvSAJ-0002A5-3A; Sat, 27 Mar 2010 10:18:47 +0100 Date: Sat, 27 Mar 2010 10:18:46 +0100 From: Gary Jennejohn To: "M. Warner Losh" Message-ID: <20100327101846.5b4d6b18@ernst.jennejohn.org> In-Reply-To: <20100326.102344.364422370204095721.imp@bsdimp.com> References: <20100326101524.15695bisy2324t8g@webmail.leidinger.net> <20100326.093717.857133809997909007.imp@bsdimp.com> <20100326171241.16524bklcroedou8@webmail.leidinger.net> <20100326.102344.364422370204095721.imp@bsdimp.com> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Alexander@Leidinger.net, rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: CTF patch for testing/review X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 09:18:51 -0000 On Fri, 26 Mar 2010 10:23:44 -0600 (MDT) "M. Warner Losh" wrote: > In message: <20100326171241.16524bklcroedou8@webmail.leidinger.net> > Alexander Leidinger writes: [entry in build(7)] > : I will have a look at it next week. As I'm not a native english > : speaker, you can expect another patch for review (text suggestions for > : build.9 are welcome, I will take care about the mdoc markup then). > Note, it's build(7), not build(9). At least, I find no build(9) on my -current system installed two days ago. > How does this text look (posted so everybody can kibitz): > > WITH_CTF the build process will run the DTrace CTF conversion > tools on built objects. Please note that this WITH_ > option is handled differently than all other WITH_ > options (there's no WITHOUT_CTF, or corresponding ==> there is - no contractions, please > MK_CTF in the build system). > Otherwise it looks OK to me. -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 09:27:32 2010 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88655106564A; Sat, 27 Mar 2010 09:27:32 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout5.freenet.de (mout5.freenet.de [IPv6:2001:748:100:40::2:7]) by mx1.freebsd.org (Postfix) with ESMTP id 2230F8FC08; Sat, 27 Mar 2010 09:27:32 +0000 (UTC) Received: from [195.4.92.12] (helo=2.mx.freenet.de) by mout5.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NvSIk-0003Fu-RO; Sat, 27 Mar 2010 10:27:30 +0100 Received: from p57ae0231.dip0.t-ipconnect.de ([87.174.2.49]:59680 helo=ernst.jennejohn.org) by 2.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NvSIk-0005SE-IO; Sat, 27 Mar 2010 10:27:30 +0100 Date: Sat, 27 Mar 2010 10:27:29 +0100 From: Gary Jennejohn To: Gleb Smirnoff Message-ID: <20100327102729.3bb8fba4@ernst.jennejohn.org> In-Reply-To: <20100326211706.GI18894@FreeBSD.org> References: <20100326211706.GI18894@FreeBSD.org> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org Subject: Re: touch panel support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 09:27:32 -0000 On Sat, 27 Mar 2010 00:17:06 +0300 Gleb Smirnoff wrote: > Hello, > > I've got a display with touch panel, and I'd like to get in working > in FreeBSD. The touch panel is supported by NetBSD's uep(4). So far, > I have written uep(4) for FreeBSD, that successfully reads and parses > data from the USB touch panel device. > > And then I've got a problem. Our mouse subsystem is not ready for > touch panels. Our mouse(4) protocol does not support mouse driver > passing _absolute_ coordinates to the mouse(4) subsystem. It only > expects a relative movement of the mouse. But _absolute_ coordinates > are principal idea of any touch panel. > > The lesser problem is lack of generic support for touch panel > calibration. > > Both of these problems are solved in NetBSD. They've got a wsmux(4) > device, just like our kbdmux(4), but for mice. This mouse multiplexer > can also understand absolute coordinates from underlying mice drivers. > NetBSD also has a generic support for calibration of touch panels. > > What is the FreeBSD future way to go: port things for NetBSD? Write > something different? > IMO we should go for porting what NetBSD has. -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 10:57:35 2010 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE8A6106564A for ; Sat, 27 Mar 2010 10:57:35 +0000 (UTC) (envelope-from rnoland@FreeBSD.org) Received: from gizmo.2hip.net (gizmo.2hip.net [64.74.207.195]) by mx1.freebsd.org (Postfix) with ESMTP id 5C7618FC08 for ; Sat, 27 Mar 2010 10:57:35 +0000 (UTC) Received: from [192.168.1.4] (adsl-154-185-142.bna.bellsouth.net [68.154.185.142]) (authenticated bits=0) by gizmo.2hip.net (8.14.3/8.14.3) with ESMTP id o2RAvUOT094626 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 27 Mar 2010 06:57:30 -0400 (EDT) (envelope-from rnoland@FreeBSD.org) From: Robert Noland To: gary.jennejohn@freenet.de In-Reply-To: <20100327102729.3bb8fba4@ernst.jennejohn.org> References: <20100326211706.GI18894@FreeBSD.org> <20100327102729.3bb8fba4@ernst.jennejohn.org> Content-Type: text/plain Organization: FreeBSD Date: Sat, 27 Mar 2010 05:57:24 -0500 Message-Id: <1269687444.35918.9.camel@balrog.2hip.net> Mime-Version: 1.0 X-Mailer: Evolution 2.26.3 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.9 required=5.0 tests=AWL, BAYES_00, FH_DATE_PAST_20XX, RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RDNS_DYNAMIC,SPF_SOFTFAIL autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on gizmo.2hip.net Cc: arch@FreeBSD.org, Gleb Smirnoff Subject: Re: touch panel support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 10:57:35 -0000 On Sat, 2010-03-27 at 10:27 +0100, Gary Jennejohn wrote: > On Sat, 27 Mar 2010 00:17:06 +0300 > Gleb Smirnoff wrote: > > > Hello, > > > > I've got a display with touch panel, and I'd like to get in working > > in FreeBSD. The touch panel is supported by NetBSD's uep(4). So far, > > I have written uep(4) for FreeBSD, that successfully reads and parses > > data from the USB touch panel device. > > > > And then I've got a problem. Our mouse subsystem is not ready for > > touch panels. Our mouse(4) protocol does not support mouse driver > > passing _absolute_ coordinates to the mouse(4) subsystem. It only > > expects a relative movement of the mouse. But _absolute_ coordinates > > are principal idea of any touch panel. > > > > The lesser problem is lack of generic support for touch panel > > calibration. > > > > Both of these problems are solved in NetBSD. They've got a wsmux(4) > > device, just like our kbdmux(4), but for mice. This mouse multiplexer > > can also understand absolute coordinates from underlying mice drivers. > > NetBSD also has a generic support for calibration of touch panels. > > > > What is the FreeBSD future way to go: port things for NetBSD? Write > > something different? > > > > IMO we should go for porting what NetBSD has. Well, input device stuff is tricky... From a console perspective, what we do now with kbdmux and moused seems like the right thing. If you look at X though, we would really like to have independent input devices, each with its own set of characteristics. For multi-seat installations, you basically want to allocate some set of input devices to each seat. In other cases, you might want your touchpad to work independently of your mouse, for example. I'm not certain what the best way to achieve this is, but our input layer is currently pretty frustrating to deal with, especially when you throw HAL or DeviceKit into the mix. We have to go to a lot of effort to figure out if mice are using moused or not, and if they should be advertised to X. robert. > -- > Gary Jennejohn > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" -- Robert Noland FreeBSD From owner-freebsd-arch@FreeBSD.ORG Sat Mar 27 12:37:36 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97235106566B; Sat, 27 Mar 2010 12:37:36 +0000 (UTC) (envelope-from ache@nagual.pp.ru) Received: from nagual.pp.ru (nagual.pp.ru [194.87.13.69]) by mx1.freebsd.org (Postfix) with ESMTP id E634A8FC0A; Sat, 27 Mar 2010 12:37:35 +0000 (UTC) Received: from nagual.pp.ru (ache@localhost [127.0.0.1]) by nagual.pp.ru (8.14.4/8.14.4) with ESMTP id o2RCOHNh095382; Sat, 27 Mar 2010 15:24:17 +0300 (MSK) (envelope-from ache@nagual.pp.ru) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nagual.pp.ru; s=default; t=1269692658; bh=HvEdz31Jhxu7hEtToYd75+KSRJjbjNPUWWbM5wZhCkY=; l=589; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=Unc76wVGupDZ90lplSPEmMfefnNz+XRyWBV96VMks7ek9UxkhYVoDt4skUmf3JLO/ +JWfJEuH90fNc7jo4vV8/LJYQ2JhR8v0G4HGcrhY8ZmUqGx0UioErEF1ZXS7meWM0H /vMR0LOsU1J+y7kPEguQXoMwqhEN1KQ+5na/R7Z8= Received: (from ache@localhost) by nagual.pp.ru (8.14.4/8.14.4/Submit) id o2RCOGmA095380; Sat, 27 Mar 2010 15:24:16 +0300 (MSK) (envelope-from ache) Date: Sat, 27 Mar 2010 15:24:14 +0300 From: Andrey Chernov To: d@delphij.net Message-ID: <20100327122413.GA95235@nagual.pp.ru> Mail-Followup-To: Andrey Chernov , d@delphij.net, Dag-Erling Sm??rgrav , ports@FreeBSD.ORG, the_paya@gentoo.org, Alexander Logvinov , freebsd-arch@FreeBSD.ORG References: <4BACFE18.7010309@delphij.net> <86wrwylji0.fsf@ds4.des.no> <4BAD509B.3080805@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BAD509B.3080805@delphij.net> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: ports@FreeBSD.ORG, Dag-Erling Sm??rgrav , Alexander Logvinov , the_paya@gentoo.org, freebsd-arch@FreeBSD.ORG Subject: Re: [RFC] Reduce namespace pollution on zlib.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Mar 2010 12:37:36 -0000 On Fri, Mar 26, 2010 at 05:26:03PM -0700, Xin LI wrote: > > This is wrong, FreeBSD has native 64-bit stat() etc. and does not need > > _LARGEFILE_WHATEVER. > > Yes we do not need that and it just cause compilation errors. > > The problem is that some third party software thinks that they need to > define _LARGEFILE64_*, which will break zlib.h on FreeBSD :( To keep ports code untouched you should not #undef anything like that names in the public zlib.h header, just remove/redefine *LARGEFILE* for zlib only in some private internal header. -- http://ache.pp.ru/