From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 11:07:07 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC3851065675 for ; Mon, 15 Mar 2010 11:07:07 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AFF518FC19 for ; Mon, 15 Mar 2010 11:07:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id o2FB77sm026798 for ; Mon, 15 Mar 2010 11:07:07 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id o2FB77KK026796 for freebsd-arch@FreeBSD.org; Mon, 15 Mar 2010 11:07:07 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 15 Mar 2010 11:07:07 GMT Message-Id: <201003151107.o2FB77KK026796@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-arch@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 11:07:07 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/120749 arch [request] Suggest upping the default kern.ps_arg_cache 1 problem total. From owner-freebsd-arch@FreeBSD.ORG Mon Mar 15 18:07:07 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95FAF106566C for ; Mon, 15 Mar 2010 18:07:07 +0000 (UTC) (envelope-from returns@c.ss36.on9mail.com) Received: from c.ss36.on9mail.com (c.ss36.on9mail.com [209.144.27.61]) by mx1.freebsd.org (Postfix) with ESMTP id 611908FC1B for ; Mon, 15 Mar 2010 18:07:07 +0000 (UTC) Received: by c.ss36.on9mail.com (Postfix, from userid 0) id 1D7275815E7; Mon, 15 Mar 2010 12:48:43 -0500 (CDT) To: freebsd-arch@freebsd.org Message-ID: <1268675323_SectionID-501087_HitID-1268672224201_SiteID-43865_EmailID-102469378_DB-1_SID-0@ss36.on9mail.com> DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ss36.on9mail.com; i=@ss36.on9mail.com; q=dns/txt; s=gmmailerd; t=1268675310; h=From; bh=47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=; l=0; b=XEcpEb0JK+FDkX6QFrI3orssd6UCiAYdaWHIG+y0hhlEGXF1JVE/UnFQ9cjhWQDKPN5NFLAzKM35UQSG4582UD7DMuNz9kaZQEzjNF6+2a/wes4KfTpE3ixXaxQNLggzRP7vQw0VbbD3bab8/KVbnoIR+m3mLWmjl0APRijcUUo= From: "Mike Hammond" Mime-Version: 1.0 Date: Mon, 15 Mar 2010 12:48:43 -0500 (CDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: If you have a hardwood floor, times have changed. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Mar 2010 18:07:07 -0000 =20 Happy New Years! www.FloorGuardian.com = January 2010 =EF=BF=BD =20 We protect your floors while the others just cover it! We offer floo= r protection.=EF=BF=BD You have invested serious money in your gym floors a= nd we offer serious protection.=EF=BF=BD We protect your floor for the dama= ge that tables, chairs and heels can inflict.=EF=BF=BD Installs in about an= hour for my gyms.=EF=BF=BD Come in 9 colors but we can also Pantone match = if needed.=EF=BF=BD Not one of those tacky tarp systems, Floor Guardian is = a velcro'd seamed, carpet based, 6' wide roll product that is easily rolled= and carted under all doorways and stored on end.=EF=BF=BD No lifting as wi= th tiles, and none of the tape waste or trip hazards of a tarp and way more= elegant in looks.=EF=BF=BD As an example a=EF=BF=BD 70'x96' gym would have= 16 rolls and the storage would be 32"x108" of floor space.=20 =EF=BF=BDWe also will take your tarp system as a trade = in- We clean them and then donate them to your choice of=EF=BF=BD a Boys an= d Girls Club.=EF=BF=BD Turn your Gym or Hall into more than just an athletic s= pace, use it for receptions, rehearsals, recitals, graduations, PTA meeting= s, fund raisers and registrations. Reduce the need to refinish, while warmi= ng and quieting down the room.=EF=BF=BD Have recess insides on rainy days!= =EF=BF=BD Kids love it.=EF=BF=BD 7 year full warranty. So tough, you can cl= ean it with regular bleach.=EF=BF=BD Hundreds of schools for the past 20 ye= ars have been protecting their floors using Floor Guardian without 1 compla= int.=EF=BF=BD We have the referrals to back it up.=EF=BF=BD Click here for = a free sample and brochure. =EF=BF=BD Please visit our website or call for = more info.=20 =20 =20 =20 =20 =20 =20 web - www.FloorGuardian.com =EF=BF=BD=EF=BF=BD|=EF=BF= =BD=EF=BF=BD Product Sample - Free Sample=EF=BF=BD |=EF=BF=BD=EF=BF=BD tel = - 206-255-1491=20 =20 =20 =20 =20 =0AMessage sent by: Floor Guardian, 4841 California Ave SW, Seattle, 9= 8166-4329, United States=0A=0ATo unsubscribe, click the link below.=0Ahttp:= //c.ss36.on9mail.com/RWCode/subscribe.asp?SID=3D0&SiteID=3D43865&Email=3Dfr= eebsd-arch@freebsd.org&HitID=3D1268672224201 From owner-freebsd-arch@FreeBSD.ORG Tue Mar 16 21:50:58 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8037106566B for ; Tue, 16 Mar 2010 21:50:58 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (unknown [IPv6:2001:470:1f09:679::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7D8838FC08 for ; Tue, 16 Mar 2010 21:50:58 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 3A36F8F57; Tue, 16 Mar 2010 21:51:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on muon.cran.org.uk X-Spam-Level: X-Spam-Status: No, score=-2.7 required=8.0 tests=AWL,BAYES_00,RDNS_DYNAMIC autolearn=no version=3.2.5 Received: from core.draftnet (87-194-158-129.bethere.co.uk [87.194.158.129]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA; Tue, 16 Mar 2010 21:51:14 +0000 (UTC) From: Bruce Cran To: freebsd-arch@freebsd.org Date: Tue, 16 Mar 2010 21:50:33 +0000 User-Agent: KMail/1.12.4 (FreeBSD/9.0-CURRENT; KDE/4.3.5; amd64; ; ) References: <201003121513.38721.max@love2party.net> <20100313200155.O22734@delplex.bde.org> In-Reply-To: <20100313200155.O22734@delplex.bde.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003162150.33852.bruce@cran.org.uk> Cc: Max Laier Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2010 21:50:58 -0000 On Saturday 13 March 2010 09:24:39 Bruce Evans wrote: > These macros may have useful 15-25 years ago for i386, i486 and Pentium1, > since CPU branch predictors were either nonexistent or not so good. > After that, CPU branch predictors became quite good. The macros should > have been mostly unused 15-25 years ago too, since they optimize for > unreadability and unwritability. Fortunately they are rarely used in > FreeBSD. They were imported from NetBSD in 2003 where they are used > more (306 instances in 2005 NetBSD /sys vs 28 instances in 2004 FreeBSD > /sys; there are 2208 instances of likely() in 2004 linux-2.6.10). The Cell powerpc processor doesn't have a dynamic branch prediction unit. Quite a few articles recommend using __builtin_expect to improve performance. -- Bruce Cran From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 10:18:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D806F106564A for ; Wed, 17 Mar 2010 10:18:19 +0000 (UTC) (envelope-from cole@opteqint.net) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id BEB1A8FC0C for ; Wed, 17 Mar 2010 10:18:19 +0000 (UTC) Received: by pwj4 with SMTP id 4so664361pwj.13 for ; Wed, 17 Mar 2010 03:18:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.142.247.23 with SMTP id u23mr271685wfh.319.1268819414158; Wed, 17 Mar 2010 02:50:14 -0700 (PDT) X-Originating-IP: [196.210.171.238] Date: Wed, 17 Mar 2010 11:50:14 +0200 Message-ID: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> From: Cole To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: PCI and Resouce question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 10:18:19 -0000 Hi. Sorry if this is not the correct list, I have tried posting to freebsd-hackers, and didnt get a response. Maybe im being stupid and not connecting something, or just not understanding, in which case sorry again. What im trying to achieve is writing to a few registers/memory addresses for the memory location of BAR 0 for a pci card. I do not want to modify the drivers source code, since it might become unavailable sometime in the future. So what im looking to do is write another kernel module, that can obtain the cards device_t structure, as well as the resource structure of the card. My aim is to obtain the resource structure so that I can then obtain the bus_tag and bus_handle, and use those to write to the BAR 0 memory space. My understanding is that I would use pci_find_device to obtain the device_t, then I think I can use bus_alloc_resource_any to obtain the resource, I see there used to be a bus_get_resource, but the man page for that no longer seems to exist, and references to it seem to have been removed. If there is another way or better way of doing this, please let me know. Also im not familiar with the specifics of bus_alloc_resouce_any, and I dont know if calling this function from another kernel module will give me the already allocated memory for this pci card, or will it create a new one and assign it to the BAR 0 register on the card? I also know I can get the softc structure from the device_t, but I dont wish to use that, as it might change without my knowledge, especially if the driver becomes closed in the future. I dont really do a lot of driver development, or this sort of programming, so any help would be appreciated, and once again, im sorry if this is all basic stuff and im not understanding something, or if im posting to the wrong list. Regards /Cole From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 10:49:55 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68259106564A for ; Wed, 17 Mar 2010 10:49:55 +0000 (UTC) (envelope-from kmsujit@gmail.com) Received: from mail-px0-f200.google.com (mail-px0-f200.google.com [209.85.216.200]) by mx1.freebsd.org (Postfix) with ESMTP id 3FDC88FC17 for ; Wed, 17 Mar 2010 10:49:54 +0000 (UTC) Received: by pxi38 with SMTP id 38so123680pxi.27 for ; Wed, 17 Mar 2010 03:49:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=FlQRahRTdsdpbtujSCWOmbmZEKzR5pnWPxlJrF+0m1Q=; b=DVJ2a+gD+INq5FwJ+H2a3doTUFhkZgkTiKttZteHUs3d5MvuXIL4y4Ba6ZVPpJTF8n hl20UDjLi2ngOuwWrTvHbb9/1IzDOGA8KSBl2dRFcLNcIrWzBivk7YCBWxvt9SahJiuH lK7lhWxxJQ1U4XVCRy/lhalj/lONMZMmII5m0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Fn7awXRctW4utGP6ZrwirdlZya9JQFoSx9K2S/Ppm3J6FCVGiXRij0aQ3qprOiy781 vB/fnjrvujMXPdOVIkfoysxboSkF6fYq8fLg8IDl2iRsNoooi3l+f4vxRvmijx3jpQEc WI8peRPgxoHImAa0PgcZHwYpUAqunad1bKQhk= MIME-Version: 1.0 Received: by 10.114.188.23 with SMTP id l23mr464667waf.40.1268821471426; Wed, 17 Mar 2010 03:24:31 -0700 (PDT) In-Reply-To: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> References: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> Date: Wed, 17 Mar 2010 15:54:31 +0530 Message-ID: <74fe56021003170324l7379e8f8mcd2da23887d78667@mail.gmail.com> From: Sujit K M To: Cole Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-arch@freebsd.org Subject: Re: PCI and Resouce question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 10:49:55 -0000 > What im trying to achieve is writing to a few registers/memory > addresses for the memory location of BAR 0 for a pci card. I do not > want to modify the drivers source code, since it might become > unavailable sometime in the future. So what im looking to do is write > another kernel module, that can obtain the cards device_t structure, > as well as the resource structure of the card. My aim is to obtain the > resource structure so that I can then obtain the bus_tag and > bus_handle, and use those to write to the BAR 0 memory space. You can still do this if you use port related functionality. But would not be done at the device level, but at the driver level. Also the related functionality like poll/select as the case may be will be utilized at an device level(device_t). > > My understanding is that I would use pci_find_device to obtain the > device_t, then I think I can use bus_alloc_resource_any to obtain the > resource, I see there used to be a bus_get_resource, but the man page > for that no longer seems to exist, and references to it seem to have > been removed. If there is another way or better way of doing this, > please let me know. Also im not familiar with the specifics of > bus_alloc_resouce_any, and I dont know if calling this function from > another kernel module will give me the already allocated memory for > this pci card, or will it create a new one and assign it to the BAR 0 > register on the card? This again is at higher level than the device_t related. Would be much more usefull in developing HA(High Availability). Thanks, Sujit From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 11:17:41 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5F18B1065670 for ; Wed, 17 Mar 2010 11:17:41 +0000 (UTC) (envelope-from gary.jennejohn@freenet.de) Received: from mout5.freenet.de (mout5.freenet.de [IPv6:2001:748:100:40::2:7]) by mx1.freebsd.org (Postfix) with ESMTP id 218938FC0A for ; Wed, 17 Mar 2010 11:17:41 +0000 (UTC) Received: from [195.4.92.23] (helo=13.mx.freenet.de) by mout5.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #1) id 1NrrFs-0000mE-9m; Wed, 17 Mar 2010 12:17:40 +0100 Received: from p57ae23ad.dip0.t-ipconnect.de ([87.174.35.173]:64644 helo=ernst.jennejohn.org) by 13.mx.freenet.de with esmtpa (ID gary.jennejohn@freenet.de) (port 25) (Exim 4.72 #2) id 1NrrFs-0003AY-1F; Wed, 17 Mar 2010 12:17:40 +0100 Date: Wed, 17 Mar 2010 12:17:39 +0100 From: Gary Jennejohn To: Cole Message-ID: <20100317121739.591b925e@ernst.jennejohn.org> In-Reply-To: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> References: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: PCI and Resouce question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gary.jennejohn@freenet.de List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 11:17:41 -0000 On Wed, 17 Mar 2010 11:50:14 +0200 Cole wrote: > Sorry if this is not the correct list, I have tried posting to > freebsd-hackers, and didnt get a response. Maybe im being stupid and > not connecting something, or just not understanding, in which case > sorry again. > I saw several responses to your post on hackers. --- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 12:41:06 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63531106566B for ; Wed, 17 Mar 2010 12:41:06 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 23E978FC12 for ; Wed, 17 Mar 2010 12:41:06 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id B728046B52; Wed, 17 Mar 2010 08:41:05 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id DA4BE8A01F; Wed, 17 Mar 2010 08:41:04 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 17 Mar 2010 08:40:57 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> In-Reply-To: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003170840.57911.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 17 Mar 2010 08:41:04 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Cole Subject: Re: PCI and Resouce question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 12:41:06 -0000 On Wednesday 17 March 2010 5:50:14 am Cole wrote: > Hi. > > Sorry if this is not the correct list, I have tried posting to > freebsd-hackers, and didnt get a response. Maybe im being stupid and > not connecting something, or just not understanding, in which case > sorry again. > > What im trying to achieve is writing to a few registers/memory > addresses for the memory location of BAR 0 for a pci card. I do not > want to modify the drivers source code, since it might become > unavailable sometime in the future. So what im looking to do is write > another kernel module, that can obtain the cards device_t structure, > as well as the resource structure of the card. My aim is to obtain the > resource structure so that I can then obtain the bus_tag and > bus_handle, and use those to write to the BAR 0 memory space. > > My understanding is that I would use pci_find_device to obtain the > device_t, then I think I can use bus_alloc_resource_any to obtain the > resource, I see there used to be a bus_get_resource, but the man page > for that no longer seems to exist, and references to it seem to have > been removed. If there is another way or better way of doing this, > please let me know. Also im not familiar with the specifics of > bus_alloc_resouce_any, and I dont know if calling this function from > another kernel module will give me the already allocated memory for > this pci card, or will it create a new one and assign it to the BAR 0 > register on the card? The PCI bus expects that only the driver calls bus_alloc_resource_any(), and that it only does this once. bus_get_resource() does not return the actual 'struct resource *r', so it will not do you any good. If you know for certain that the existing driver has not called 'bus_alloc_resource_any()', then you can call it from your module. Note that if the device driver is unloaded, the PCI bus may unmap the resource out from under your module (the PCI bus assumes that only the child device_t uses the resources, so it cleans up allocated resources when a driver unloads). Really, the clean way to handle something like this is to have your new code to share the device_t with the other driver. You can do this by using an approach similar to the vga_pci driver and changing the existing driver to attach to your device instead. So long as you propogate/emulate the proper PCI and bus methods, the child device driver can work fine with only a one- line change to add a new DRIVER_MODULE() that attaches to your device as the bus instead of pci. For, example, if your driver creates 'foo0' devices and the existing driver is called 'bar', you would create a new line: DRIVER_MODULE(bar, foo, ...); where the rest of the fields are copied from the existing 'DRIVER_MODULE(bar, pci)' line. Your foo driver will have to have a higher probe priority than the real driver, and you will have to load your driver before the real driver. Another alternative would be to make the real driver become the proxy driver. Attach your driver as a child of it. You will have to add bus and PCI methods to the real driver to satisfy the needs of your device. Mostly you just want this driver to have a custom methods for allocating resources where it shares a resource it has already allocated with your driver. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Mar 17 18:18:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCC961065670 for ; Wed, 17 Mar 2010 18:18:30 +0000 (UTC) (envelope-from cole@opteqint.net) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id A0F0C8FC1A for ; Wed, 17 Mar 2010 18:18:30 +0000 (UTC) Received: by pwj4 with SMTP id 4so1090326pwj.13 for ; Wed, 17 Mar 2010 11:18:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.142.210.21 with SMTP id i21mr675580wfg.265.1268849910071; Wed, 17 Mar 2010 11:18:30 -0700 (PDT) X-Originating-IP: [196.210.136.150] In-Reply-To: <201003170840.57911.jhb@freebsd.org> References: <9f206d1a1003170250r2727a55ajb636a0bdf1a2f137@mail.gmail.com> <201003170840.57911.jhb@freebsd.org> Date: Wed, 17 Mar 2010 20:18:30 +0200 Message-ID: <9f206d1a1003171118x6bf16f9crba0063c550c3b007@mail.gmail.com> From: Cole To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: PCI and Resouce question X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Mar 2010 18:18:30 -0000 Hi. Thanks for explaining it and making it clearer, and thanks to everyone who commented. It makes things a lot easier to understand I know now how to sort this out. Thanks again /Cole On 17 March 2010 14:40, John Baldwin wrote: > On Wednesday 17 March 2010 5:50:14 am Cole wrote: >> Hi. >> >> Sorry if this is not the correct list, I have tried posting to >> freebsd-hackers, and didnt get a response. Maybe im being stupid and >> not connecting something, or just not understanding, in which case >> sorry again. >> >> What im trying to achieve is writing to a few registers/memory >> addresses for the memory location of BAR 0 for a pci card. I do not >> want to modify the drivers source code, since it might become >> unavailable sometime in the future. So what im looking to do is write >> another kernel module, that can obtain the cards device_t structure, >> as well as the resource structure of the card. My aim is to obtain the >> resource structure so that I can then obtain the bus_tag and >> bus_handle, and use those to write to the BAR 0 memory space. >> >> My understanding is that I would use pci_find_device to obtain the >> device_t, then I think I can use bus_alloc_resource_any to obtain the >> resource, I see there used to be a bus_get_resource, but the man page >> for that no longer seems to exist, and references to it seem to have >> been removed. If there is another way or better way of doing this, >> please let me know. Also im not familiar with the specifics of >> bus_alloc_resouce_any, and I dont know if calling this function from >> another kernel module will give me the already allocated memory for >> this pci card, or will it create a new one and assign it to the BAR 0 >> register on the card? > > The PCI bus expects that only the driver calls bus_alloc_resource_any(), = and > that it only does this once. =A0bus_get_resource() does not return the ac= tual > 'struct resource *r', so it will not do you any good. =A0If you know for = certain > that the existing driver has not called 'bus_alloc_resource_any()', then = you > can call it from your module. =A0Note that if the device driver is unload= ed, the > PCI bus may unmap the resource out from under your module (the PCI bus as= sumes > that only the child device_t uses the resources, so it cleans up allocate= d > resources when a driver unloads). > > Really, the clean way to handle something like this is to have your new c= ode > to share the device_t with the other driver. =A0You can do this by using = an > approach similar to the vga_pci driver and changing the existing driver t= o > attach to your device instead. =A0So long as you propogate/emulate the pr= oper > PCI and bus methods, the child device driver can work fine with only a on= e- > line change to add a new DRIVER_MODULE() that attaches to your device as = the > bus instead of pci. =A0For, example, if your driver creates 'foo0' device= s and > the existing driver is called 'bar', you would create a new line: > > DRIVER_MODULE(bar, foo, ...); > > where the rest of the fields are copied from the existing 'DRIVER_MODULE(= bar, > pci)' line. =A0Your foo driver will have to have a higher probe priority = than > the real driver, and you will have to load your driver before the real dr= iver. > > Another alternative would be to make the real driver become the proxy dri= ver. > Attach your driver as a child of it. =A0You will have to add bus and PCI = methods > to the real driver to satisfy the needs of your device. =A0Mostly you jus= t want > this driver to have a custom methods for allocating resources where it sh= ares > a resource it has already allocated with your driver. > > -- > John Baldwin > From owner-freebsd-arch@FreeBSD.ORG Thu Mar 18 13:47:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01E031065673 for ; Thu, 18 Mar 2010 13:47:43 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 574168FC1C for ; Thu, 18 Mar 2010 13:47:41 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA06489 for ; Thu, 18 Mar 2010 15:47:40 +0200 (EET) (envelope-from avg@freebsd.org) Message-ID: <4BA22EFC.8020902@freebsd.org> Date: Thu, 18 Mar 2010 15:47:40 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.23 (X11/20100211) MIME-Version: 1.0 To: freebsd-arch@freebsd.org X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: g_vfs_open and bread(devvp, ...) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2010 13:47:43 -0000 Forwarding here in an attempt to increase exposure. -------- Original Message -------- Subject: g_vfs_open and bread(devvp, ...) Date: Wed, 17 Mar 2010 11:52:32 +0200 From: Andriy Gapon To: freebsd-fs@FreeBSD.org, freebsd-geom@freebsd.org CC: Poul-Henning Kamp , Bruce Evans I've given a fresh look to the issue of g_vfs_open and bread(devvp, ...) in filesystem code. This time I hope to present my reasoning more clearly than I did in my previous attempts. For this reason I am omitting historical references and dramatic examples/demonstrations but they are still available upon request (and in archives). I hope that my shortened notation in references to the code and data structures will not be confusing. bread() and the API family it belongs to is an interface to buffer cache system. Buffer cache system can be roughly divided into two parts: cache part and I/O path part. I think that it is understood that both parts must have the same notion of a block size when translating a block number to a byte offset. (If this point is not clear, I can follow up with another essay). In the case of cache code the translation is explicit and simple: A) offset = blkno * bo_bsize In the case of I/O code the translation is not that straightforward, because it can be altered/overridden by bop_strategy which can in turn hook vop_strategy, etc. Let's consider a simple case of a filesystem that: a) connects to a geom provider via g_vfs_open b) doesn't modify anything in devvp->v_bufobj (in particular bo_ops and bo_bsize) c) uses bread(devvp) (e.g. to access an equivalent of superblock, etc) Short overview of geom_vfs glue: 1) g_vfs_open sets devvp->v_bufobj.bo_ops to g_vfs_bufops, where bop_strategy is g_vfs_strategy 2) bo_bsize is set to pp->sectorsize 3) g_vfs_strategy doesn't perform any block-to-offset translation of its own, it expects b_iooffset to be correctly set and passes its value to bio_offset When a filesystem issues bread(devvp) the following happens in the I/O path: I) bread() calls breadn() II) in breadn(): bp->b_iooffset = dbtob(bp->b_blkno), that is b_iooffset is set to blkno * DEV_BSIZE (where DEV_BSIZE is 512) III) breadn() then calls bstrategy() which is a simple wrapper around BO_STRATEGY IV) g_vfs_strategy gets called and, as described in (3) above, it simply passes on b_iooffset value to bio_offset V) thus, a block size used for I/O operation is 512 (DEV_BSIZE) VI) on the other hand, as stated in (A) above, block size used in caching code is bo_bsize Thus, if bo_bsize != DEV_BSIZE, or alternatively said, pp->sectorsize != 512, we have a trouble of data getting cached with incorrect offsets. Additionally, from (V) above we must conclude that a filesystem must specify block numbers in DEV_BSIZE units to bread(devvp, blkno, ...) if the conditions (a), (b), (c) are met. In fact, all such filesystems already do that, because otherwise they would read incorrect data from the media. So, the problem is only with the caching of the data. As such, this issue has little practical effects, because only a small number of reads is done via devvp and only for sufficiently small chunks of data (I hope). fs/udf used to be greatly affected by this issue when it was reading directory nodes via devvp, but that was in the past (prior to 189082). Still I think that we should fix this issue for general code correctness/quality reasons. And also to avoid possible future bugs. As demonstrated by (V) and (VI) above, obvious and easiest fix is to (always) set bo_bsize to DEV_BSIZE in g_vfs_open(): --- a/sys/geom/geom_vfs.c +++ b/sys/geom/geom_vfs.c @@ -179,7 +179,7 @@ g_vfs_open(struct vnode *vp, struct g_consumer **cpp, const char *fsname, int wr bo = &vp->v_bufobj; bo->bo_ops = g_vfs_bufops; bo->bo_private = cp; - bo->bo_bsize = pp->sectorsize; + bo->bo_bsize = DEV_BSIZE; gp->softc = bo; return (error); I tested this change with ufs, udf and cd9660 and haven't observed any regressions. P.S. There might something to changing bread(devvp) convention, so that blkno is expected in sectorsize units. But setting bo_bsize to sectorsize is only a tiny portion of what needs to be changed to make it actually work. -- Andriy Gapon -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Thu Mar 18 22:11:05 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2BDDB106564A; Thu, 18 Mar 2010 22:11:05 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id DDF458FC27; Thu, 18 Mar 2010 22:11:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2IMB1nO068863; Thu, 18 Mar 2010 16:11:01 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Mar 2010 16:11:17 -0600 (MDT) Message-Id: <20100318.161117.658811636873842325.imp@bsdimp.com> To: rwatson@FreeBSD.org From: "M. Warner Losh" In-Reply-To: References: <201003121513.38721.max@love2party.net> <20100313200155.O22734@delplex.bde.org> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: max@love2party.net, freebsd-arch@FreeBSD.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2010 22:11:05 -0000 In message: Robert Watson writes: : : On Sat, 13 Mar 2010, Bruce Evans wrote: : : >> My point is: Handle with care!!! Trust your compiler/CPU : >> predictors/... - most of the time, they are smarter than you are ;) : > : > These macros may have useful 15-25 years ago for i386, i486 and : > Pentium1, since CPU branch predictors were either nonexistent or not : > so good. After that, CPU branch predictors became quite good. The : > macros should have been mostly unused 15-25 years ago too, since they : > optimize for unreadability and unwritability. Fortunately they are : > rarely used in FreeBSD. They were imported from NetBSD in 2003 where : > they are used more (306 instances in 2005 NetBSD /sys vs 28 instances : > in 2004 FreeBSD /sys; there are 2208 instances of likely() in 2004 : > linux-2.6.10). : : I think it would be reasonable to expect that people deploy branch : prediction macros (as with prefetch, etc) only where there's specific : measurements that indicate they are important to have there -- at the : very least, pmc data, but ideally also benchmarking data. They are more useful on architectures where you have branches that tell the CPU if they are likely or unlikely to be taken... Warner From owner-freebsd-arch@FreeBSD.ORG Thu Mar 18 22:25:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D036106566B; Thu, 18 Mar 2010 22:25:30 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from mrout2-b.corp.re1.yahoo.com (mrout2-b.corp.re1.yahoo.com [69.147.107.21]) by mx1.freebsd.org (Postfix) with ESMTP id C9C828FC17; Thu, 18 Mar 2010 22:25:29 +0000 (UTC) Received: from [IPv6:::1] (proxy10.corp.re1.yahoo.com [69.147.105.206]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id o2IMEn36080258; Thu, 18 Mar 2010 15:14:50 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100318.161117.658811636873842325.imp@bsdimp.com> Date: Thu, 18 Mar 2010 16:14:49 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201003121513.38721.max@love2party.net> <20100313200155.O22734@delplex.bde.org> <20100318.161117.658811636873842325.imp@bsdimp.com> To: "M. Warner Losh" X-Mailer: Apple Mail (2.1077) Cc: max@love2party.net, rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2010 22:25:30 -0000 On Mar 18, 2010, at 4:11 PM, M. Warner Losh wrote: > In message: > Robert Watson writes: > :=20 > : On Sat, 13 Mar 2010, Bruce Evans wrote: > :=20 > : >> My point is: Handle with care!!! Trust your compiler/CPU > : >> predictors/... - most of the time, they are smarter than you are = ;) > : > > : > These macros may have useful 15-25 years ago for i386, i486 and > : > Pentium1, since CPU branch predictors were either nonexistent or = not > : > so good. After that, CPU branch predictors became quite good. The > : > macros should have been mostly unused 15-25 years ago too, since = they > : > optimize for unreadability and unwritability. Fortunately they = are > : > rarely used in FreeBSD. They were imported from NetBSD in 2003 = where > : > they are used more (306 instances in 2005 NetBSD /sys vs 28 = instances > : > in 2004 FreeBSD /sys; there are 2208 instances of likely() in 2004 > : > linux-2.6.10). > :=20 > : I think it would be reasonable to expect that people deploy branch > : prediction macros (as with prefetch, etc) only where there's = specific > : measurements that indicate they are important to have there -- at = the > : very least, pmc data, but ideally also benchmarking data. >=20 > They are more useful on architectures where you have branches that > tell the CPU if they are likely or unlikely to be taken... >=20 And that's a very good point, one that Bruce really failed to address. = Not only is branch prediction useful for MIPS and ARM, I suspect that it's also = useful for Atom. Scott From owner-freebsd-arch@FreeBSD.ORG Thu Mar 18 23:02:15 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 54EC7106564A; Thu, 18 Mar 2010 23:02:15 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 140B98FC15; Thu, 18 Mar 2010 23:02:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id o2IMv7hp069371; Thu, 18 Mar 2010 16:57:08 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Mar 2010 16:57:25 -0600 (MDT) Message-Id: <20100318.165725.480410072667175878.imp@bsdimp.com> To: scottl@samsco.org From: "M. Warner Losh" In-Reply-To: References: <20100318.161117.658811636873842325.imp@bsdimp.com> X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: max@love2party.net, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2010 23:02:15 -0000 In message: Scott Long writes: : On Mar 18, 2010, at 4:11 PM, M. Warner Losh wrote: : > In message: : > Robert Watson writes: : > : : > : On Sat, 13 Mar 2010, Bruce Evans wrote: : > : : > : >> My point is: Handle with care!!! Trust your compiler/CPU : > : >> predictors/... - most of the time, they are smarter than you are ;) : > : > : > : > These macros may have useful 15-25 years ago for i386, i486 and : > : > Pentium1, since CPU branch predictors were either nonexistent or not : > : > so good. After that, CPU branch predictors became quite good. The : > : > macros should have been mostly unused 15-25 years ago too, since they : > : > optimize for unreadability and unwritability. Fortunately they are : > : > rarely used in FreeBSD. They were imported from NetBSD in 2003 where : > : > they are used more (306 instances in 2005 NetBSD /sys vs 28 instances : > : > in 2004 FreeBSD /sys; there are 2208 instances of likely() in 2004 : > : > linux-2.6.10). : > : : > : I think it would be reasonable to expect that people deploy branch : > : prediction macros (as with prefetch, etc) only where there's specific : > : measurements that indicate they are important to have there -- at the : > : very least, pmc data, but ideally also benchmarking data. : > : > They are more useful on architectures where you have branches that : > tell the CPU if they are likely or unlikely to be taken... : > : : And that's a very good point, one that Bruce really failed to : address. Not only is branch prediction useful for MIPS and ARM, I : suspect that it's also useful for Atom. The PMC work will tell us that... Warner From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 01:06:44 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4E90106564A for ; Fri, 19 Mar 2010 01:06:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 499168FC08 for ; Fri, 19 Mar 2010 01:06:44 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 593C8A57DA8; Fri, 19 Mar 2010 09:06:43 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id RKw3ilPQdmRi; Fri, 19 Mar 2010 09:06:36 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id EAC24A57C1A; Fri, 19 Mar 2010 09:06:34 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:x-enigmail-version:openpgp:content-type; b=Zr69CBKVu40RLeRKMm7ZBfXJadhBvx8PaJEC4iFq//+KRFvh1vce1A5LXsnUGDUEn rg3jm/Zdz4Adu0fTSywcQ== Message-ID: <4BA2CE17.2050105@delphij.net> Date: Thu, 18 Mar 2010 18:06:31 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: multipart/mixed; boundary="------------040004010806050003090308" Cc: Subject: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 01:06:44 -0000 This is a multi-part message in MIME format. --------------040004010806050003090308 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I think it doesn't really make sense to by default use MACHINE_CPU=i486 when the kernel is built with SSE by default today. Attached patch uses i686 SSE MMX by default, the user can always change the default setting by overriding CPUTYPE (they have to do it as SSE is enabled by default for several years). Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLos4XAAoJEATO+BI/yjfB7A0H/iXJmmBf8vbiT0a1gEW4SXfV sxfD/T0CuFvsDHahVAjMZVTg0pGOhmBijn4Q/tDzd4EyshLAJ9CdPNv/fPqz2CAj RJBlCrxudcRIGemtZAVGsZU3ZJ+XX38rIDXKhzYIgwfMJiZlSf5Fi41ACphllpWc +1KkdiMWkaSe33iRiOzvkfgDtdf1w5wK5vJJ7q/TJ0m6bZNYeHPDZBAp0557FWd4 Equ4xwqtUb/aoojyWbJ9z93tokXUZT+cHSOISFcYIFyp7BdYjdFDRwPxawYUr2Tu jS4QXkiWQxa1EswAOPwnY/jA7tpwPHTOM8DZKxeRVMfS4W/9tDkBMEI4wi1ljZk= =dSKH -----END PGP SIGNATURE----- --------------040004010806050003090308 Content-Type: text/plain; name="default-ssei686.diff" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="default-ssei686.diff" SW5kZXg6IHNoYXJlL21rL2JzZC5jcHUubWsKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gc2hhcmUvbWsv YnNkLmNwdS5tawkocmV2aXNpb24gMjA1MjgyKQorKysgc2hhcmUvbWsvYnNkLmNwdS5tawko d29ya2luZyBjb3B5KQpAQCAtNyw3ICs3LDcgQEAKIC5pZiAhZGVmaW5lZChDUFVUWVBFKSB8 fCBlbXB0eShDUFVUWVBFKQogX0NQVUNGTEFHUyA9CiAuIGlmICR7TUFDSElORV9BUkNIfSA9 PSAiaTM4NiIKLU1BQ0hJTkVfQ1BVID0gaTQ4NgorTUFDSElORV9DUFUgPSBzc2UgaTY4NiBt bXgKIC4gZWxpZiAke01BQ0hJTkVfQVJDSH0gPT0gImFtZDY0IgogTUFDSElORV9DUFUgPSBh bWQ2NCBzc2UyIHNzZQogLiBlbGlmICR7TUFDSElORV9BUkNIfSA9PSAiaWE2NCIK --------------040004010806050003090308-- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 01:44:05 2010 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAB83106566B; Fri, 19 Mar 2010 01:44:05 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 8A8EF8FC14; Fri, 19 Mar 2010 01:44:05 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2J1hvVc013954; Thu, 18 Mar 2010 19:43:57 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <20100318.165725.480410072667175878.imp@bsdimp.com> Date: Thu, 18 Mar 2010 19:43:57 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <15D5A6D4-1594-4667-AE51-0E26950C81DA@samsco.org> References: <20100318.161117.658811636873842325.imp@bsdimp.com> <20100318.165725.480410072667175878.imp@bsdimp.com> To: "M. Warner Losh" X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: max@love2party.net, rwatson@FreeBSD.org, freebsd-arch@FreeBSD.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 01:44:06 -0000 On Mar 18, 2010, at 4:57 PM, M. Warner Losh wrote: > In message: > Scott Long writes: > : On Mar 18, 2010, at 4:11 PM, M. Warner Losh wrote: > : > In message: = > : > Robert Watson writes: > : > :=20 > : > : On Sat, 13 Mar 2010, Bruce Evans wrote: > : > :=20 > : > : >> My point is: Handle with care!!! Trust your compiler/CPU > : > : >> predictors/... - most of the time, they are smarter than you = are ;) > : > : > > : > : > These macros may have useful 15-25 years ago for i386, i486 = and > : > : > Pentium1, since CPU branch predictors were either nonexistent = or not > : > : > so good. After that, CPU branch predictors became quite good. = The > : > : > macros should have been mostly unused 15-25 years ago too, = since they > : > : > optimize for unreadability and unwritability. Fortunately = they are > : > : > rarely used in FreeBSD. They were imported from NetBSD in = 2003 where > : > : > they are used more (306 instances in 2005 NetBSD /sys vs 28 = instances > : > : > in 2004 FreeBSD /sys; there are 2208 instances of likely() in = 2004 > : > : > linux-2.6.10). > : > :=20 > : > : I think it would be reasonable to expect that people deploy = branch > : > : prediction macros (as with prefetch, etc) only where there's = specific > : > : measurements that indicate they are important to have there -- = at the > : > : very least, pmc data, but ideally also benchmarking data. > : >=20 > : > They are more useful on architectures where you have branches that > : > tell the CPU if they are likely or unlikely to be taken... > : >=20 > :=20 > : And that's a very good point, one that Bruce really failed to > : address. Not only is branch prediction useful for MIPS and ARM, I > : suspect that it's also useful for Atom. >=20 > The PMC work will tell us that... >=20 My understanding was that Atom wasn't super-scalar at all and has no = branch prediction or out-of-order logic. It's basically an 80486 with a = modern instruction set. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 12:00:04 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8ABB11065675 for ; Fri, 19 Mar 2010 12:00:04 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 54BD08FC25 for ; Fri, 19 Mar 2010 12:00:04 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 046C046B1A; Fri, 19 Mar 2010 08:00:04 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3B8118A01F; Fri, 19 Mar 2010 08:00:03 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org, d@delphij.net Date: Fri, 19 Mar 2010 07:51:26 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <4BA2CE17.2050105@delphij.net> In-Reply-To: <4BA2CE17.2050105@delphij.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201003190751.26767.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 19 Mar 2010 08:00:03 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 12:00:04 -0000 On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: > Hi, > > I think it doesn't really make sense to by default use MACHINE_CPU=i486 > when the kernel is built with SSE by default today. > > Attached patch uses i686 SSE MMX by default, the user can always change > the default setting by overriding CPUTYPE (they have to do it as SSE is > enabled by default for several years). The kernel is only built with support for userland applications using SSE, it does not _use_ SSE. Similarly, the kernel is built with support for PG_NX provided on 64-bit processors, but it does not do so by failing to support older 32-bit processors. I think this change is premature. Users can already set CPUTYPE in make.conf. Also, most modern x86 server-class machines are 64-bit in which case they would be running FreeBSD/amd64 and using SSE already. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 14:42:23 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 47079106566B; Fri, 19 Mar 2010 14:42:23 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail08.syd.optusnet.com.au (mail08.syd.optusnet.com.au [211.29.132.189]) by mx1.freebsd.org (Postfix) with ESMTP id D69B78FC1B; Fri, 19 Mar 2010 14:42:22 +0000 (UTC) Received: from c122-106-146-195.carlnfd1.nsw.optusnet.com.au (c122-106-146-195.carlnfd1.nsw.optusnet.com.au [122.106.146.195]) by mail08.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o2JEgFBx016952 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 20 Mar 2010 01:42:17 +1100 Date: Sat, 20 Mar 2010 01:42:15 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Scott Long In-Reply-To: Message-ID: <20100320012752.B1181@delplex.bde.org> References: <201003121513.38721.max@love2party.net> <20100313200155.O22734@delplex.bde.org> <20100318.161117.658811636873842325.imp@bsdimp.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: max@love2party.net, rwatson@freebsd.org, freebsd-arch@freebsd.org Subject: Re: likely and unlikely X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 14:42:23 -0000 On Thu, 18 Mar 2010, Scott Long wrote: > On Mar 18, 2010, at 4:11 PM, M. Warner Losh wrote: >> In message: >> Robert Watson writes: >> : >> : On Sat, 13 Mar 2010, Bruce Evans wrote: >> : >> : >> My point is: Handle with care!!! Trust your compiler/CPU >> : >> predictors/... - most of the time, they are smarter than you are ;) >> : > >> : > These macros may have useful 15-25 years ago for i386, i486 and >> : > Pentium1, since CPU branch predictors were either nonexistent or not >> : > so good. After that, CPU branch predictors became quite good. The >> : > macros should have been mostly unused 15-25 years ago too, since they >> : > optimize for unreadability and unwritability. Fortunately they are >> : > rarely used in FreeBSD. They were imported from NetBSD in 2003 where >> : > they are used more (306 instances in 2005 NetBSD /sys vs 28 instances >> : > in 2004 FreeBSD /sys; there are 2208 instances of likely() in 2004 >> : > linux-2.6.10). >> : >> : I think it would be reasonable to expect that people deploy branch >> : prediction macros (as with prefetch, etc) only where there's specific >> : measurements that indicate they are important to have there -- at the >> : very least, pmc data, but ideally also benchmarking data. >> >> They are more useful on architectures where you have branches that >> tell the CPU if they are likely or unlikely to be taken... > > And that's a very good point, one that Bruce really failed to address. Not only > is branch prediction useful for MIPS and ARM, I suspect that it's also useful > for Atom. I addressed it indirectly :-) : The useful use of these macros is limited to: - the 1% or 0.1% of CPUs that are not amd64 or i386 - on these CPUs, the small percentage of code that runs in the kernel (since userland is out of scope for this discussion and much harder to optimize globally since it is much larger) - the small percentage of kernel code that is MD (since these optimizations are very MD and I hope no one one plans to tangle MI code with ifdefs for them) This gives a very small target for optimization and difficulties measuring the effect. Most systems shouldn't spend most of the time in the kernel. It is easy to think of specialized systems which are exceptions but hard to think of any that spend much time in their MD part, except in copyin/out which should already be optimized. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 16:12:18 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69E891065672; Fri, 19 Mar 2010 16:12:18 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 10AAB8FC17; Fri, 19 Mar 2010 16:12:18 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 1E2E7A57C1A; Sat, 20 Mar 2010 00:12:17 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id nBmr5xZC2GDJ; Sat, 20 Mar 2010 00:12:10 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 6CADFA67C50; Sat, 20 Mar 2010 00:12:09 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=n2Bdmvc7V0D4zmui/hINqG9t795F1bN5XIF3/y8/vDLznwaeqN5Cl+pOvBYygRRtM IQYUI07wo/2D4LtcW6QoQ== Message-ID: <4BA3A250.2080204@delphij.net> Date: Fri, 19 Mar 2010 09:12:00 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: John Baldwin References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> In-Reply-To: <201003190751.26767.jhb@freebsd.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: d@delphij.net, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 16:12:18 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/19 04:51, John Baldwin wrote: > On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: >> Hi, >> >> I think it doesn't really make sense to by default use MACHINE_CPU=i486 >> when the kernel is built with SSE by default today. >> >> Attached patch uses i686 SSE MMX by default, the user can always change >> the default setting by overriding CPUTYPE (they have to do it as SSE is >> enabled by default for several years). > > The kernel is only built with support for userland applications using SSE, it > does not _use_ SSE. Similarly, the kernel is built with support for PG_NX > provided on 64-bit processors, but it does not do so by failing to support > older 32-bit processors. I think this change is premature. Users can already > set CPUTYPE in make.conf. Also, most modern x86 server-class machines are > 64-bit in which case they would be running FreeBSD/amd64 and using SSE > already. Yes you are right. Sorry for the confusion. Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLo6JPAAoJEATO+BI/yjfBVZ4IAIcyFNbDKwtpKQyU9xg5T9B1 JeJXA7pyMnb5YydJy5KR/Kw5RTpRoMzp64CuMDfBLH1oEbxHaFGGlwqm+qwNnk2S a1CraA/9vmr4oOZbIso6jAhEBet1l8+O6hcMfhs9+8LnZpgkUbhikYOABQwVRaw7 xhMQYaEXseB5rltavgqlSQcDlmS3mctuyBuS1KGa0ONGxt4dT5giEWcmpvruouF9 MyGsElWA7iUW2c1EjPywGlPW3M2pSdzaIySDK4eE4w1jmtjl+gzQ0TMNdcNVpGo7 jKb4eZD1oxLAg5ljxbRQCTeK199TNPBYktAnaiIZa/FEDa7cHGURf7EsipDkTv4= =0n/e -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 18:49:27 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCA1C1065686 for ; Fri, 19 Mar 2010 18:49:27 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (out-0-26.mx.aerioconnect.net [216.240.47.86]) by mx1.freebsd.org (Postfix) with ESMTP id BF8848FC20 for ; Fri, 19 Mar 2010 18:49:27 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2JIaCnw022143; Fri, 19 Mar 2010 11:36:12 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id EB6102D601C; Fri, 19 Mar 2010 11:36:11 -0700 (PDT) Message-ID: <4BA3C41F.3000404@elischer.org> Date: Fri, 19 Mar 2010 11:36:15 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: John Baldwin References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> In-Reply-To: <201003190751.26767.jhb@freebsd.org> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: d@delphij.net, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 18:49:27 -0000 John Baldwin wrote: > On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: >> Hi, >> >> I think it doesn't really make sense to by default use MACHINE_CPU=i486 >> when the kernel is built with SSE by default today. >> >> Attached patch uses i686 SSE MMX by default, the user can always change >> the default setting by overriding CPUTYPE (they have to do it as SSE is >> enabled by default for several years). > > The kernel is only built with support for userland applications using SSE, it > does not _use_ SSE. Similarly, the kernel is built with support for PG_NX > provided on 64-bit processors, but it does not do so by failing to support > older 32-bit processors. I think this change is premature. Users can already > set CPUTYPE in make.conf. Also, most modern x86 server-class machines are > 64-bit in which case they would be running FreeBSD/amd64 and using SSE > already. > and a lot of low power boxes (e.g. soekris) are 586 class. From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 18:54:02 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7022D106566C; Fri, 19 Mar 2010 18:54:02 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 2B17E8FC19; Fri, 19 Mar 2010 18:54:01 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2JIrkLp001323; Fri, 19 Mar 2010 12:53:46 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <4BA3C41F.3000404@elischer.org> Date: Fri, 19 Mar 2010 12:53:45 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> To: Julian Elischer X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: d@delphij.net, freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 18:54:02 -0000 On Mar 19, 2010, at 12:36 PM, Julian Elischer wrote: > John Baldwin wrote: >> On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: >>> Hi, >>>=20 >>> I think it doesn't really make sense to by default use = MACHINE_CPU=3Di486 >>> when the kernel is built with SSE by default today. >>>=20 >>> Attached patch uses i686 SSE MMX by default, the user can always = change >>> the default setting by overriding CPUTYPE (they have to do it as SSE = is >>> enabled by default for several years). >> The kernel is only built with support for userland applications using = SSE, it does not _use_ SSE. Similarly, the kernel is built with support = for PG_NX provided on 64-bit processors, but it does not do so by = failing to support older 32-bit processors. I think this change is = premature. Users can already set CPUTYPE in make.conf. Also, most = modern x86 server-class machines are >> 64-bit in which case they would be running FreeBSD/amd64 and using = SSE >> already. >=20 >=20 > and a lot of low power boxes (e.g. soekris) are 586 class. >=20 Are these machines typically installed via a GENERIC kernel from = freebsd.org release CD's? Maybe there's a market to create a new = mini-distribution tailored for these devices. It would come with a = suitable kernel and install/setup tools. Scott From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 20:20:50 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 668131065672 for ; Fri, 19 Mar 2010 20:20:50 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 35E4A8FC13 for ; Fri, 19 Mar 2010 20:20:50 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id B0D0646B38; Fri, 19 Mar 2010 16:20:49 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id EA4038A01F; Fri, 19 Mar 2010 16:20:48 -0400 (EDT) From: John Baldwin To: Scott Long Date: Fri, 19 Mar 2010 16:20:29 -0400 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <4BA2CE17.2050105@delphij.net> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> In-Reply-To: <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003191620.29912.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Fri, 19 Mar 2010 16:20:48 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.7 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: d@delphij.net, Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 20:20:50 -0000 On Friday 19 March 2010 2:53:45 pm Scott Long wrote: > > On Mar 19, 2010, at 12:36 PM, Julian Elischer wrote: > > > John Baldwin wrote: > >> On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: > >>> Hi, > >>> > >>> I think it doesn't really make sense to by default use MACHINE_CPU=i486 > >>> when the kernel is built with SSE by default today. > >>> > >>> Attached patch uses i686 SSE MMX by default, the user can always change > >>> the default setting by overriding CPUTYPE (they have to do it as SSE is > >>> enabled by default for several years). > >> The kernel is only built with support for userland applications using SSE, it does not _use_ SSE. Similarly, the kernel is built with support for PG_NX provided on 64-bit processors, but it does not do so by failing to support older 32-bit processors. I think this change is premature. Users can already set CPUTYPE in make.conf. Also, most modern x86 server-class machines are > >> 64-bit in which case they would be running FreeBSD/amd64 and using SSE > >> already. > > > > > > and a lot of low power boxes (e.g. soekris) are 586 class. > > > > Are these machines typically installed via a GENERIC kernel from freebsd.org release CD's? Maybe there's a market to create a new mini-distribution tailored for these devices. It would come with a suitable kernel and install/setup tools. Supporting 486/586 in the kernel doesn't actually cost anything. We don't use SSE (except in a few isolated cases) in the kernel already because of the overhead that would be required to manage a separate FPU context for the kernel (it would severely impact the performance of all context switches as well as all user <--> kernel transitions such as interrupts, traps, etc. if we were to make widespread use of SSE). This is why we use -no-sse for the kernel compile even on amd64 where we know for certain that all supported CPUs support SSE. There really isn't a compelling reason to drop 486/586 support from GENERIC. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 20:31:00 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 758E21065676 for ; Fri, 19 Mar 2010 20:31:00 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 174CA8FC1C for ; Fri, 19 Mar 2010 20:30:59 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id o2JKFsMJ025701; Fri, 19 Mar 2010 16:15:54 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.2 (mail.netplex.net [204.213.176.10]); Fri, 19 Mar 2010 16:15:55 -0400 (EDT) Date: Fri, 19 Mar 2010 16:15:54 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Scott Long In-Reply-To: <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> Message-ID: References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: d@delphij.net, Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 20:31:00 -0000 On Fri, 19 Mar 2010, Scott Long wrote: > > On Mar 19, 2010, at 12:36 PM, Julian Elischer wrote: > >> John Baldwin wrote: >>> On Thursday 18 March 2010 9:06:31 pm Xin LI wrote: >>>> Hi, >>>> >>>> I think it doesn't really make sense to by default use MACHINE_CPU=i486 >>>> when the kernel is built with SSE by default today. >>>> >>>> Attached patch uses i686 SSE MMX by default, the user can always change >>>> the default setting by overriding CPUTYPE (they have to do it as SSE is >>>> enabled by default for several years). >>> The kernel is only built with support for userland applications using SSE, it does not _use_ SSE. Similarly, the kernel is built with support for PG_NX provided on 64-bit processors, but it does not do so by failing to support older 32-bit processors. I think this change is premature. Users can already set CPUTYPE in make.conf. Also, most modern x86 server-class machines are >>> 64-bit in which case they would be running FreeBSD/amd64 and using SSE >>> already. >> >> >> and a lot of low power boxes (e.g. soekris) are 586 class. >> > > Are these machines typically installed via a GENERIC kernel from > freebsd.org release CD's? Maybe there's a market to create a new > mini-distribution tailored for these devices. It would come with a > suitable kernel and install/setup tools. Well, we have nanobsd, but having a suitable install tool for small flash-based systems where you want a nanobsd-like setup (readonly filesystems) would be very nice. I try to write procedures for our embedded systems so others (neophytes) can create and burn them, but it might be easier for someone to get started with embedded systems if they could do it from a release using an install tool. -- DE From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 21:17:31 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD6B7106567D; Fri, 19 Mar 2010 21:17:31 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 70B5F8FC1D; Fri, 19 Mar 2010 21:17:31 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id ED80AA581DC; Sat, 20 Mar 2010 05:17:29 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id 9Dd+B5zw1BFy; Sat, 20 Mar 2010 05:17:23 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 7BA21A58206; Sat, 20 Mar 2010 05:17:22 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=KITHlzH+tD3krVIfuPFRwsPcyin0YebGVknYY3VgEgHbP0q+ubYVWCRIxFy17GyH4 X3sgJbPWbRmB5Wa+QKTVw== Message-ID: <4BA3E9DF.2050303@delphij.net> Date: Fri, 19 Mar 2010 14:17:19 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: Daniel Eischen References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> In-Reply-To: X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Scott Long , d@delphij.net, Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 21:17:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/19 13:15, Daniel Eischen wrote: [...] > Well, we have nanobsd, but having a suitable install tool > for small flash-based systems where you want a nanobsd-like > setup (readonly filesystems) would be very nice. I try > to write procedures for our embedded systems so others > (neophytes) can create and burn them, but it might be > easier for someone to get started with embedded systems > if they could do it from a release using an install tool. Some computation intense tasks would benefit from enabling certain optimizations which is not suitable for older processors. However, just like John said servers tends to use 64-bit platform more than 32-bit ones, so perhaps we can just dismiss the idea of enabling these optimizations on FreeBSD/i386 platform and focus on FreeBSD/amd64... Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLo+nfAAoJEATO+BI/yjfB4QUH/Rg9nVzjumkcrZbm+JWeDGoD +/he/sb31TDACTSHh5BWtVOPszHIKZ93AiMUWMsbONEm/EGttF9Vpxq7gJPLFFIK /wF5SgQUi2hrwfdHSF3WiAa5jtMEf7UdbWYyHj0ixXqQf3kFbL6RV82ivzmc0Csp 09gVVnWncDMg7ADkP9uYHTNkFhxpGHMSw/eRP0792dVqy5GZKWIbDsjME8GZE3Yd jC4TBmdd7ThfYVdabt9+g/GFmca3khGvuoSfpahJzkeu808E38fzRDYT5zmzATLk 9XLColO4e88Bq7LEBVdLW7jBxnUm7vzMuKD+6q8/GBl9hAbhHVu9xxTa5TWsEj0= =BqYV -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 21:24:43 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA84B1065676 for ; Fri, 19 Mar 2010 21:24:43 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 6570F8FC18 for ; Fri, 19 Mar 2010 21:24:43 +0000 (UTC) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.4/8.14.4/NETPLEX) with ESMTP id o2JLOWwM028421; Fri, 19 Mar 2010 17:24:32 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.2.2 (mail.netplex.net [204.213.176.10]); Fri, 19 Mar 2010 17:24:32 -0400 (EDT) Date: Fri, 19 Mar 2010 17:24:32 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: d@delphij.net In-Reply-To: <4BA3E9DF.2050303@delphij.net> Message-ID: References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> <4BA3E9DF.2050303@delphij.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Scott Long , Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 21:24:43 -0000 On Fri, 19 Mar 2010, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 2010/03/19 13:15, Daniel Eischen wrote: > [...] >> Well, we have nanobsd, but having a suitable install tool >> for small flash-based systems where you want a nanobsd-like >> setup (readonly filesystems) would be very nice. I try >> to write procedures for our embedded systems so others >> (neophytes) can create and burn them, but it might be >> easier for someone to get started with embedded systems >> if they could do it from a release using an install tool. > > Some computation intense tasks would benefit from enabling certain > optimizations which is not suitable for older processors. > > However, just like John said servers tends to use 64-bit platform more > than 32-bit ones, so perhaps we can just dismiss the idea of enabling > these optimizations on FreeBSD/i386 platform and focus on FreeBSD/amd64... Perhaps I was wrong, but I thought Scott's question was more general: is there a desire for a special installation suitable to small appliances (usually flash-based)? Sorry, I didn't mean to steal the thread. -- DE From owner-freebsd-arch@FreeBSD.ORG Fri Mar 19 21:42:45 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B71C2106564A; Fri, 19 Mar 2010 21:42:45 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id E94108FC19; Fri, 19 Mar 2010 21:42:44 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 9BC2AA58235; Sat, 20 Mar 2010 05:42:37 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id WQ7VG+GFH9f0; Sat, 20 Mar 2010 05:42:32 +0800 (CST) Received: from delta.delphij.net (drawbridge.ixsystems.com [206.40.55.65]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 066B7A5823E; Sat, 20 Mar 2010 05:42:29 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=j6BoJyaO79t5V2V+2QeYW8G1hryWMEQ5u8ATdmWlLgknGIJqwYmvj/5n84ySrDrV7 kcZK3d4bflP9d9UaDnR/g== Message-ID: <4BA3EFC1.60504@delphij.net> Date: Fri, 19 Mar 2010 14:42:25 -0700 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: Daniel Eischen References: <4BA2CE17.2050105@delphij.net> <201003190751.26767.jhb@freebsd.org> <4BA3C41F.3000404@elischer.org> <5BED0721-442C-44B3-8B23-3D94BE5354A9@samsco.org> <4BA3E9DF.2050303@delphij.net> In-Reply-To: X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Scott Long , d@delphij.net, Julian Elischer , freebsd-arch@freebsd.org Subject: Re: [PATCH] Utilize i686, SSE and MMX by default on FreeBSD/i386 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2010 21:42:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/19 14:24, Daniel Eischen wrote: > On Fri, 19 Mar 2010, Xin LI wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 2010/03/19 13:15, Daniel Eischen wrote: >> [...] >>> Well, we have nanobsd, but having a suitable install tool >>> for small flash-based systems where you want a nanobsd-like >>> setup (readonly filesystems) would be very nice. I try >>> to write procedures for our embedded systems so others >>> (neophytes) can create and burn them, but it might be >>> easier for someone to get started with embedded systems >>> if they could do it from a release using an install tool. >> >> Some computation intense tasks would benefit from enabling certain >> optimizations which is not suitable for older processors. >> >> However, just like John said servers tends to use 64-bit platform more >> than 32-bit ones, so perhaps we can just dismiss the idea of enabling >> these optimizations on FreeBSD/i386 platform and focus on >> FreeBSD/amd64... > > Perhaps I was wrong, but I thought Scott's question was more > general: is there a desire for a special installation suitable > to small appliances (usually flash-based)? At $work we have plans on providing a Flash based images that is specifically built for some need, as a continuation of the FreeNAS project. My understanding is that we want to provide images for more "general" purpose. I knew that there is still a lot of people using i386+PAE kernels on their Linux servers although they have > 4GB of RAM barely because they can "see" the memory in top(1) output. I personally don't think it's a very attrative idea. Perhaps, we can have a separate, FreeBSD based x86 embedded platform SDK, like what Windows CE has offered as their "Platform Builder", that can give one a menu to choose which part the engineer does not want/etc and just build an Flash image for burn into the boot media (ideally it would use pre-built binaries for some common-case platforms) by using src/tools/build/options? Sounds like a SoC 2010 proposal to me :) Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLo+/BAAoJEATO+BI/yjfB46EH/A9BkztAH5s6zxX+hbsfyQfr fMHPnx4KAPVy1ItWTWikUOvGw3eoSSfX1UH8hOXbzEZnwSWIJAPsLNWrOy7Usprx 4aSwvu3UsEsma5xBJeGjkhh+Bvird47T4OfEBMzsutvxEV/PJZvOE/TTpkIq+5sQ vlG+HPi2fuMm026zgfb52dtHoH+6KMPbYUU61Cp9XprgCif6eH1mNAWEPCxeoviE E71vOc1I8kS0xz5DvKsT2HG9Xcrrl8PMwboow62CBt/xZrwPYRYioh9a/hdZ6nzp fBb+ISxf3G7mSf7txpvdXJfPklqdL/8rxPrFyAQPqGWnBe5G/JoUT9yUbHyPjbQ= =PkRA -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 15:20:16 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 317931065670; Sat, 20 Mar 2010 15:20:16 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f224.google.com (mail-fx0-f224.google.com [209.85.220.224]) by mx1.freebsd.org (Postfix) with ESMTP id 8E86E8FC1C; Sat, 20 Mar 2010 15:20:15 +0000 (UTC) Received: by fxm24 with SMTP id 24so257789fxm.3 for ; Sat, 20 Mar 2010 08:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:subject:x-enigmail-version:content-type :content-transfer-encoding; bh=vvUN2xXU6OfXsVRUp5YJ/fHgtbXc4qLggljTqGUJHzM=; b=l2xWHqKVDMhy0h3pjpoYeVZYgw2weLpUcDINbSeH/syIdaZx2WxbWLRc1NBGeEXIBx LWYKuoJIGHbDha/poyUDwtgEFxzSI80tzBH/JorKCo2WzmZ2ZVWLBLtBneAhOLrHcL6j Q9/3xUi9fo44vT1Elrp1kp+f8WjMek7JanWrY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:subject :x-enigmail-version:content-type:content-transfer-encoding; b=lAPsAIn/5L3pYShmDw3PvEBdqm+Z8tjl8/0PklhBllDE5Y+boYGlNxe+HgFlVPgFzY 4NZWE+uejFZSI1mhGQZoLLUV067fyxmd3UD0rQ6pu6+chi+c46q1ppBU/6w6kJLT6M3g DYontRp3el1393VFELR+HRbBacEIaEEijtO3I= Received: by 10.223.132.197 with SMTP id c5mr890792fat.35.1269098412629; Sat, 20 Mar 2010 08:20:12 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 16sm182270fxm.12.2010.03.20.08.20.11 (version=SSLv3 cipher=RC4-MD5); Sat, 20 Mar 2010 08:20:12 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA4E7A9.3070502@FreeBSD.org> Date: Sat, 20 Mar 2010 17:20:09 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: FreeBSD-Current , freebsd-arch@freebsd.org X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: Subject: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 15:20:16 -0000 Hi. With set of changes done to ATA, CAM and GEOM subsystems last time we may now get use for increased MAXPHYS (maximum physical I/O size) kernel constant from 128K to some bigger value. Increasing it allows to improve performance and reduce processing overhead for large I/O operations. Potential downside is a bit increased kernel memory usage, but as soon as these values were not changing for more than 8 years, I don't think it should be significant now. Present state of things: - ahci(4) and siis(4) support any I/O sizes up to MAXPHYS; - ata(4) supports I/O sizes up to min(512K, MAXPHYS) for the most of controllers, and works correctly for the rest; - most of SCSI controller drivers still limited by DFLTPHYS, but parts needed to work on them one by one later are already in place. - ad(4), da(4), ada(4), cd(4), acd(4), afd(4), atapicam(4) drivers support any I/O sizes, supported by underlying hardware and reported by ata(4) and cam(4) subsystems; - gmirror(4), gstripe(4), graid3(4), gconcat(4) were tested and fixed to support any I/O sizes up to MAXPHYS; All above I have successfully tested last months with MAXPHYS of 1MB on i386 and amd64 platforms. So my questions are: - does somebody know any issues denying increasing MAXPHYS in HEAD? - are there any specific opinions about value? 512K, 1MB, MD? -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 17:53:19 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9E96106566B; Sat, 20 Mar 2010 17:53:19 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0B6468FC12; Sat, 20 Mar 2010 17:53:17 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id o2KHrHer003947; Sat, 20 Mar 2010 10:53:17 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id o2KHrH5x003946; Sat, 20 Mar 2010 10:53:17 -0700 (PDT) Date: Sat, 20 Mar 2010 10:53:17 -0700 (PDT) From: Matthew Dillon Message-Id: <201003201753.o2KHrH5x003946@apollo.backplane.com> To: Alexander Motin References: <4BA4E7A9.3070502@FreeBSD.org> Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 17:53:20 -0000 :All above I have successfully tested last months with MAXPHYS of 1MB on :i386 and amd64 platforms. : :So my questions are: :- does somebody know any issues denying increasing MAXPHYS in HEAD? :- are there any specific opinions about value? 512K, 1MB, MD? : :-- :Alexander Motin (nswbuf * MAXPHYS) of KVM is reserved for pbufs, so on i386 you might hit up against KVM exhaustion issues in unrelated subsystems. nswbuf typically maxes out at around 256. For i386 1MB is probably too large (256M of reserved KVM is a lot for i386). On amd64 there shouldn't be a problem. Diminishing returns get hit pretty quickly with larger MAXPHYS values. As long as the I/O can be pipelined the reduced transaction rate becomes less interesting when the transaction rate is less than a certain level. Off the cuff I'd say 2000 tps is a good basis for considering whether it is an issue or not. 256K is actually quite a reasonable value. Even 128K is reasonable. Nearly all the issues I've come up against in the last few years have been related more to pipeline algorithms breaking down and less with I/O size. The cluster_read() code is especially vulnerable to algorithmic breakdowns when fast media (such as a SSD) is involved. e.g. I/Os queued from the previous cluster op can create stall conditions in subsequent cluster ops before they can issue new I/Os to keep the pipeline hot. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 18:17:40 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5165E106564A; Sat, 20 Mar 2010 18:17:40 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 1AE4F8FC13; Sat, 20 Mar 2010 18:17:39 +0000 (UTC) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.14.3/8.14.3) with ESMTP id o2KIHYFi074281; Sat, 20 Mar 2010 12:17:34 -0600 (MDT) (envelope-from scottl@samsco.org) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Scott Long In-Reply-To: <201003201753.o2KHrH5x003946@apollo.backplane.com> Date: Sat, 20 Mar 2010 12:17:33 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> To: Matthew Dillon X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=-1.0 required=3.8 tests=ALL_TRUSTED autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 18:17:40 -0000 On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >=20 > :All above I have successfully tested last months with MAXPHYS of 1MB = on > :i386 and amd64 platforms. > : > :So my questions are: > :- does somebody know any issues denying increasing MAXPHYS in HEAD? > :- are there any specific opinions about value? 512K, 1MB, MD? > : > :--=20 > :Alexander Motin >=20 > (nswbuf * MAXPHYS) of KVM is reserved for pbufs, so on i386 you > might hit up against KVM exhaustion issues in unrelated subsystems. > nswbuf typically maxes out at around 256. For i386 1MB is probably > too large (256M of reserved KVM is a lot for i386). On amd64 there > shouldn't be a problem. >=20 Yes, this needs to be addressed. I've never gotten a clear answer from VM people like Peter Wemm and Alan Cox on what should be done. > Diminishing returns get hit pretty quickly with larger MAXPHYS = values. > As long as the I/O can be pipelined the reduced transaction rate > becomes less interesting when the transaction rate is less than a > certain level. Off the cuff I'd say 2000 tps is a good basis for > considering whether it is an issue or not. 256K is actually quite > a reasonable value. Even 128K is reasonable. >=20 I agree completely. I did quite a bit of testing on this in 2008 and = 2009. I even added some hooks into CAM to support this, and I thought that I = had discussed this extensively with Alexander at the time. Guess it was yet = another wasted conversation with him =3D-( I'll repeat it here for the record. What I call the silly-i/o-test, filling a disk up with the dd command, = yields performance improvements up to a MAXPHYS of 512K. Beyond that and it's negligible, and actually starts running into contention on the VM = page queues lock. There is some work to break down this lock, so it's worth revisiting in the future. For the non-silly-i/o-test, where I do real file i/o using various = sequential and random patterns, there was a modest improvement up to 256K, and a slight improvement up to 512K. This surprised me as I figured that most = filesystem i/o would be in UFS block sized chunks. Then I realized that the UFS = clustering code was actually taking advantage of the larger I/O's. The improvement = really depends on the workload, of course, and I wouldn't expect it to be = noticeable for most people unless they're running something like a media server. Besides the nswbuf sizing problem, there is a real problem that a lot of = drivers have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are particular values, and they've sized their data structures accordingly. = Before these values are changed, an audit needs to be done OF EVERY SINGLE STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS in the ata driver, testing that your machine boots, and then committing = the change to source control. Some drivers will have non-obvious restrictions = based on the number of SG elements allowed in a particular command format. MPT comes to mind (its multi message SG code seems to be broken when I tried testing large MAXPHYS on it), but I bet that there are others. Windows has a MAXPHYS equivalent of 1M. Linux has an equivalent of an odd number less than 512k. For the purpose of benchmarking against = these OS's, having comparable capabilities is essential; Linux easily beats = FreeBSD in the silly-i/o-test because of the MAXPHYS difference (though FreeBSD = typically stomps linux in real I/O because of vastly better latency and caching = algorithms). I'm fine with raising MAXPHYS in production once the problems are = addressed. > Nearly all the issues I've come up against in the last few years = have > been related more to pipeline algorithms breaking down and less = with > I/O size. The cluster_read() code is especially vulnerable to > algorithmic breakdowns when fast media (such as a SSD) is involved. > e.g. I/Os queued from the previous cluster op can create stall > conditions in subsequent cluster ops before they can issue new I/Os > to keep the pipeline hot. >=20 Yes, this is another very good point. It's time to start really = figuring out what SSD means for FreeBSD I/O. Scott From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 18:22:11 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26C28106566C; Sat, 20 Mar 2010 18:22:11 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id E618F8FC21; Sat, 20 Mar 2010 18:22:10 +0000 (UTC) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.14.4/8.14.1) with ESMTP id o2KIM4gf004252; Sat, 20 Mar 2010 11:22:08 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.14.4/8.13.4/Submit) id o2KIM4xw004251; Sat, 20 Mar 2010 11:22:04 -0700 (PDT) Date: Sat, 20 Mar 2010 11:22:04 -0700 (PDT) From: Matthew Dillon Message-Id: <201003201822.o2KIM4xw004251@apollo.backplane.com> To: "C. P. Ghost" References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 18:22:11 -0000 :Pardon my ignorance, but wouldn't so much KVM make small embedded :devices like Soekris boards with 128 MB of physical RAM totally unusable :then? On my net4801, running RELENG_8: : :vm.kmem_size: 40878080 : :hw.physmem: 125272064 :hw.usermen: 84840448 :hw.realmem: 134217728 KVM != physical memory. On i386 by default the kernel has 1G of KVM and userland has 3G. While the partition can be moved to increase available KVM on i386 (e.g. 2G/2G), it isn't recommended. So the KVM reserved for various things does not generally impact physical memory use. The number of swap buffers (nswbuf) is scaled to 1/4 nbufs with a maximum of 256. Systems with small amounts of memory should not be impacted. The issue w/ regards to KVM problems on i386 is mostly restricted to systems with 2G+ of ram where the kernel's various internal parameters are scaled to their maximum values or limits. On systems with less ram the kernel's internal parameters are usually scaled down sufficiently that there is very little chance of the kernel running out of KVM. -Matt From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 18:40:45 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C0CC106564A; Sat, 20 Mar 2010 18:40:45 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id B1B968FC17; Sat, 20 Mar 2010 18:40:44 +0000 (UTC) Received: by bwz8 with SMTP id 8so3886437bwz.3 for ; Sat, 20 Mar 2010 11:40:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.48.197 with SMTP id s5mr20992bkf.177.1269108838463; Sat, 20 Mar 2010 11:13:58 -0700 (PDT) X-Originating-IP: [93.203.28.245] In-Reply-To: <201003201753.o2KHrH5x003946@apollo.backplane.com> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> Date: Sat, 20 Mar 2010 19:13:58 +0100 Message-ID: From: "C. P. Ghost" To: Matthew Dillon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Alexander Motin , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 18:40:45 -0000 On Sat, Mar 20, 2010 at 6:53 PM, Matthew Dillon wrote: > > :All above I have successfully tested last months with MAXPHYS of 1MB on > :i386 and amd64 platforms. > : > :So my questions are: > :- does somebody know any issues denying increasing MAXPHYS in HEAD? > :- are there any specific opinions about value? 512K, 1MB, MD? > : > :-- > :Alexander Motin > > =A0 =A0(nswbuf * MAXPHYS) of KVM is reserved for pbufs, so on i386 you > =A0 =A0might hit up against KVM exhaustion issues in unrelated subsystems= . > =A0 =A0nswbuf typically maxes out at around 256. =A0For i386 1MB is proba= bly > =A0 =A0too large (256M of reserved KVM is a lot for i386). =A0On amd64 th= ere > =A0 =A0shouldn't be a problem. Pardon my ignorance, but wouldn't so much KVM make small embedded devices like Soekris boards with 128 MB of physical RAM totally unusable then? On my net4801, running RELENG_8: vm.kmem_size: 40878080 hw.physmem: 125272064 hw.usermen: 84840448 hw.realmem: 134217728 > =A0 =A0Diminishing returns get hit pretty quickly with larger MAXPHYS val= ues. > =A0 =A0As long as the I/O can be pipelined the reduced transaction rate > =A0 =A0becomes less interesting when the transaction rate is less than a > =A0 =A0certain level. =A0Off the cuff I'd say 2000 tps is a good basis fo= r > =A0 =A0considering whether it is an issue or not. =A0256K is actually qui= te > =A0 =A0a reasonable value. =A0Even 128K is reasonable. > > =A0 =A0Nearly all the issues I've come up against in the last few years h= ave > =A0 =A0been related more to pipeline algorithms breaking down and less wi= th > =A0 =A0I/O size. =A0The cluster_read() code is especially vulnerable to > =A0 =A0algorithmic breakdowns when fast media (such as a SSD) is involved= . > =A0 =A0e.g. =A0I/Os queued from the previous cluster op can create stall > =A0 =A0conditions in subsequent cluster ops before they can issue new I/O= s > =A0 =A0to keep the pipeline hot. Thanks, -cpghost. --=20 Cordula's Web. http://www.cordula.ws/ From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 19:26:53 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 462CD106566C; Sat, 20 Mar 2010 19:26:53 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 9B9AC8FC0A; Sat, 20 Mar 2010 19:26:52 +0000 (UTC) Received: by fxm22 with SMTP id 22so3930198fxm.14 for ; Sat, 20 Mar 2010 12:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=5o8zI4Z2F7aw1yG7OevShLO0BWzWkprMueYsuWfhHao=; b=pN/SPcPgEejIBP+5U2N5ZRH/MZ9vONhwU4R7Iuh1PJszurDVnEqp1fo6++Emokyw2b zamFMKRMLKHUaSlkKtTRj2Ns1Ad2UNXQTXsQoDf+JTb5y2C5wYHz9Nx3a/YFBMjwUci+ xkYv5sRQRRE9QpBsF7Jc1rolk/N/ELLNZwWVM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=UiWzDCsbVt38yvogHTTIARYir2WpEOKGZW83LRduE0rOLGBPjtJH24iCmr6sxR8oye 9CgF1r8zC4Q6LKvVb1mpP1S/+vVGADoFuUmx6YvlBzYaBLz8LVVy+zA09AcO7ZyFdoXR 0gNM7fHkNKBlqjAJqaP13UszNS2kxWMhEMqF0= Received: by 10.223.94.200 with SMTP id a8mr12097540fan.86.1269113211478; Sat, 20 Mar 2010 12:26:51 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 13sm2177639fxm.14.2010.03.20.12.26.50 (version=SSLv3 cipher=RC4-MD5); Sat, 20 Mar 2010 12:26:51 -0700 (PDT) Sender: Alexander Motin Message-ID: <4BA52179.9030903@FreeBSD.org> Date: Sat, 20 Mar 2010 21:26:49 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Scott Long References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> In-Reply-To: <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 19:26:53 -0000 Scott Long wrote: > On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >> Diminishing returns get hit pretty quickly with larger MAXPHYS values. >> As long as the I/O can be pipelined the reduced transaction rate >> becomes less interesting when the transaction rate is less than a >> certain level. Off the cuff I'd say 2000 tps is a good basis for >> considering whether it is an issue or not. 256K is actually quite >> a reasonable value. Even 128K is reasonable. > > I agree completely. I did quite a bit of testing on this in 2008 and 2009. > I even added some hooks into CAM to support this, and I thought that I had > discussed this extensively with Alexander at the time. Guess it was yet another > wasted conversation with him =-( I'll repeat it here for the record. AFAIR at that time you've agreed that 256K gives improvements, and 64K of DFLTPHYS limiting most SCSI SIMs is too small. That's why you've implemented that hooks in CAM. I have not forgot that conversation (pity that it quietly died for SCSI SIMs). I agree that too high value could be just a waste of resources. As you may see I haven't blindly committed it, but asked public opinion. If you think 256K is OK - let it be 256K. If you think that 256K needed only for media servers - OK, but lets make it usable there. > Besides the nswbuf sizing problem, there is a real problem that a lot of drivers > have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are > particular values, and they've sized their data structures accordingly. Before > these values are changed, an audit needs to be done OF EVERY SINGLE > STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS > in the ata driver, testing that your machine boots, and then committing the change > to source control. Some drivers will have non-obvious restrictions based on > the number of SG elements allowed in a particular command format. MPT > comes to mind (its multi message SG code seems to be broken when I tried > testing large MAXPHYS on it), but I bet that there are others. As you should remember, we have made it in such way, that all unchecked drivers keep using DFLTPHYS, which is not going to be changed ever. So there is no problem. I would more worry about non-CAM storages and above stuff, like some rare GEOM classes. > I'm fine with raising MAXPHYS in production once the problems are > addressed. That's why in my post I've asked people about any known problems. I've addressed several related issues in last months, and I am looking for more. To address problems, it would be nice to know about them first. -- Alexander Motin From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 20:03:00 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D2E78106564A for ; Sat, 20 Mar 2010 20:03:00 +0000 (UTC) (envelope-from alan.l.cox@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id A75998FC08 for ; Sat, 20 Mar 2010 20:03:00 +0000 (UTC) Received: by pvc7 with SMTP id 7so953742pvc.13 for ; Sat, 20 Mar 2010 13:03:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:reply-to:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=JLTq/EML72PMPz50E81sMrvNHgsmXT1cQDYCCtSQXDk=; b=HnER6hGt8kZuJBnGiw2zHIf1V7BOsziZqVvyFq0W7vSa7oAj9r8Y2VUXgg6duLver5 qfHlSsp0/p2FAXEPuUW8juPIJ/y9cSYuk8Lf+IVmyebzOuuprwCCeY6M+G2LJrz3+WDn ZkLIDPZmPmj6mziv27aPzmeWmfPg29s5bG/UE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; b=s0r0+32tgXk+eYcuBoLrkUVDCnqEdnfikuWQvK9C9HCgX9ifeInlmIlULAGI+sv35k h0+WIV4jx/mvI4Z+nyRLU025Rls7Jg6s0M1VtSSd+1waqzlbd0fCCYDwg3yU8MH+iyuT UEE3upWOS1WxQflMCtOWuQFNgAPKFHBpvI3VA= MIME-Version: 1.0 Received: by 10.143.24.41 with SMTP id b41mr474035wfj.98.1269113898971; Sat, 20 Mar 2010 12:38:18 -0700 (PDT) In-Reply-To: <4BA4E7A9.3070502@FreeBSD.org> References: <4BA4E7A9.3070502@FreeBSD.org> Date: Sat, 20 Mar 2010 14:38:18 -0500 Message-ID: From: Alan Cox To: Alexander Motin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: alc@freebsd.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 20:03:00 -0000 2010/3/20 Alexander Motin > Hi. > > With set of changes done to ATA, CAM and GEOM subsystems last time we > may now get use for increased MAXPHYS (maximum physical I/O size) kernel > constant from 128K to some bigger value. [snip] > All above I have successfully tested last months with MAXPHYS of 1MB on > i386 and amd64 platforms. > > So my questions are: > - does somebody know any issues denying increasing MAXPHYS in HEAD? > - are there any specific opinions about value? 512K, 1MB, MD? > > For now, I think it should machine-dependent. The virtual memory system should have no problems with MAXPHYS of 1MB on amd64 and ia64. Alan From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 20:41:36 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F8A9106566C; Sat, 20 Mar 2010 20:41:36 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outm.internet-mail-service.net [216.240.47.236]) by mx1.freebsd.org (Postfix) with ESMTP id 2CFD78FC08; Sat, 20 Mar 2010 20:41:36 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2KKfX20006239; Sat, 20 Mar 2010 13:41:33 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 3926E2D6013; Sat, 20 Mar 2010 13:41:32 -0700 (PDT) Message-ID: <4BA532FF.6040407@elischer.org> Date: Sat, 20 Mar 2010 13:41:35 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Alexander Motin References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> In-Reply-To: <4BA52179.9030903@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Scott Long , FreeBSD-Current , freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 20:41:36 -0000 Alexander Motin wrote: > Scott Long wrote: >> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>> Diminishing returns get hit pretty quickly with larger MAXPHYS values. >>> As long as the I/O can be pipelined the reduced transaction rate >>> becomes less interesting when the transaction rate is less than a >>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>> considering whether it is an issue or not. 256K is actually quite >>> a reasonable value. Even 128K is reasonable. >> I agree completely. I did quite a bit of testing on this in 2008 and 2009. >> I even added some hooks into CAM to support this, and I thought that I had >> discussed this extensively with Alexander at the time. Guess it was yet another >> wasted conversation with him =-( I'll repeat it here for the record. In the Fusion-io driver we find that the limiting factor is not the size of MAXPHYS, but the fact that we can not push more than 170k tps through geom. (in my test machine. I've seen more on some beefier machines), but that is only a limit on small transacrtions, or in the case of large transfers the DMA engine tops out before a bigger MAXPHYS would make any difference. Where it may make a difference is that Linux only pushes 128k at a time it looks like so many hardware engines have likely never been tested with greater. (not sure about Windows). Some drivers may also be written with the assumption that they will not see more. OF course they should be able to limit the transaction size down themselves if they are written well. > > AFAIR at that time you've agreed that 256K gives improvements, and 64K > of DFLTPHYS limiting most SCSI SIMs is too small. That's why you've > implemented that hooks in CAM. I have not forgot that conversation (pity > that it quietly died for SCSI SIMs). I agree that too high value could > be just a waste of resources. As you may see I haven't blindly committed > it, but asked public opinion. If you think 256K is OK - let it be 256K. > If you think that 256K needed only for media servers - OK, but lets make > it usable there. > >> Besides the nswbuf sizing problem, there is a real problem that a lot of drivers >> have incorrectly assumed over the years that MAXPHYS and DFLTPHYS are >> particular values, and they've sized their data structures accordingly. Before >> these values are changed, an audit needs to be done OF EVERY SINGLE >> STORAGE DRIVER. No exceptions. This isn't a case of changing MAXHYS >> in the ata driver, testing that your machine boots, and then committing the change >> to source control. Some drivers will have non-obvious restrictions based on >> the number of SG elements allowed in a particular command format. MPT >> comes to mind (its multi message SG code seems to be broken when I tried >> testing large MAXPHYS on it), but I bet that there are others. > > As you should remember, we have made it in such way, that all unchecked > drivers keep using DFLTPHYS, which is not going to be changed ever. So > there is no problem. I would more worry about non-CAM storages and above > stuff, like some rare GEOM classes. > >> I'm fine with raising MAXPHYS in production once the problems are >> addressed. > > That's why in my post I've asked people about any known problems. I've > addressed several related issues in last months, and I am looking for > more. To address problems, it would be nice to know about them first. > From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 21:00:29 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 602D01065676 for ; Sat, 20 Mar 2010 21:00:29 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 1858B8FC1B for ; Sat, 20 Mar 2010 21:00:28 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Nt5mV-0002y8-BF for freebsd-arch@freebsd.org; Sat, 20 Mar 2010 22:00:27 +0100 Received: from 93-138-28-199.adsl.net.t-com.hr ([93.138.28.199]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:00:27 +0100 Received: from ivoras by 93-138-28-199.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:00:27 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Sat, 20 Mar 2010 22:00:16 +0100 Lines: 26 Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-28-199.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) In-Reply-To: <4BA532FF.6040407@elischer.org> Cc: freebsd-current@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 21:00:29 -0000 Julian Elischer wrote: > Alexander Motin wrote: >> Scott Long wrote: >>> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>>> Diminishing returns get hit pretty quickly with larger MAXPHYS >>>> values. >>>> As long as the I/O can be pipelined the reduced transaction rate >>>> becomes less interesting when the transaction rate is less than a >>>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>>> considering whether it is an issue or not. 256K is actually quite >>>> a reasonable value. Even 128K is reasonable. >>> I agree completely. I did quite a bit of testing on this in 2008 and >>> 2009. >>> I even added some hooks into CAM to support this, and I thought that >>> I had >>> discussed this extensively with Alexander at the time. Guess it was >>> yet another >>> wasted conversation with him =-( I'll repeat it here for the record. > > In the Fusion-io driver we find that the limiting factor is not the > size of MAXPHYS, but the fact that we can not push more than > 170k tps through geom. (in my test machine. I've seen more on some > beefier machines), but that is only a limit on small transacrtions, Do the GEOM threads (g_up, g_down) go into saturation? Effectively all IO is serialized through them. From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 21:06:42 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E4441065672 for ; Sat, 20 Mar 2010 21:06:42 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 18CBE8FC13 for ; Sat, 20 Mar 2010 21:06:41 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Nt5sW-0005pS-Rh for freebsd-arch@freebsd.org; Sat, 20 Mar 2010 22:06:40 +0100 Received: from 93-138-28-199.adsl.net.t-com.hr ([93.138.28.199]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:06:40 +0100 Received: from ivoras by 93-138-28-199.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:06:40 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Sat, 20 Mar 2010 22:06:25 +0100 Lines: 11 Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-28-199.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) In-Reply-To: <4BA4E7A9.3070502@FreeBSD.org> Cc: freebsd-current@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 21:06:42 -0000 Alexander Motin wrote: > All above I have successfully tested last months with MAXPHYS of 1MB on > i386 and amd64 platforms. > > So my questions are: > - does somebody know any issues denying increasing MAXPHYS in HEAD? > - are there any specific opinions about value? 512K, 1MB, MD? Not really important and tangential to the topic: are there options to make it dynamic? Loader tunable? Sysctl? From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 21:30:30 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28C6D106564A; Sat, 20 Mar 2010 21:30:30 +0000 (UTC) (envelope-from julian@elischer.org) Received: from out-0.mx.aerioconnect.net (outn.internet-mail-service.net [216.240.47.237]) by mx1.freebsd.org (Postfix) with ESMTP id 083708FC16; Sat, 20 Mar 2010 21:30:29 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o2KLUTkH008746; Sat, 20 Mar 2010 14:30:29 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 9EC772D6018; Sat, 20 Mar 2010 14:30:28 -0700 (PDT) Message-ID: <4BA53E77.8010208@elischer.org> Date: Sat, 20 Mar 2010 14:30:31 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Ivan Voras References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 21:30:30 -0000 Ivan Voras wrote: > Julian Elischer wrote: >> Alexander Motin wrote: >>> Scott Long wrote: >>>> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>>>> Diminishing returns get hit pretty quickly with larger MAXPHYS >>>>> values. >>>>> As long as the I/O can be pipelined the reduced transaction rate >>>>> becomes less interesting when the transaction rate is less than a >>>>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>>>> considering whether it is an issue or not. 256K is actually quite >>>>> a reasonable value. Even 128K is reasonable. >>>> I agree completely. I did quite a bit of testing on this in 2008 >>>> and 2009. >>>> I even added some hooks into CAM to support this, and I thought that >>>> I had >>>> discussed this extensively with Alexander at the time. Guess it was >>>> yet another >>>> wasted conversation with him =-( I'll repeat it here for the record. >> >> In the Fusion-io driver we find that the limiting factor is not the >> size of MAXPHYS, but the fact that we can not push more than >> 170k tps through geom. (in my test machine. I've seen more on some >> beefier machines), but that is only a limit on small transacrtions, > > Do the GEOM threads (g_up, g_down) go into saturation? Effectively all > IO is serialized through them. basically.. You can get better throughput by using TSC for timing because the geom and devstat code does a bit of timing.. Geom can be told to turn off it's timing but devstat can't. The 170 ktps is with TSC as timer, and geom timing turned off. It could just be the shear weight of the work being done. Linux on the same machine using the same driver code (with different wrappers) gets 225k tps. > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Mar 20 21:50:12 2010 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E8F81065675 for ; Sat, 20 Mar 2010 21:50:12 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EC1AD8FC14 for ; Sat, 20 Mar 2010 21:50:11 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Nt6Yc-0002C6-AD for freebsd-arch@freebsd.org; Sat, 20 Mar 2010 22:50:10 +0100 Received: from 93-138-28-199.adsl.net.t-com.hr ([93.138.28.199]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:50:10 +0100 Received: from ivoras by 93-138-28-199.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 20 Mar 2010 22:50:10 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Sat, 20 Mar 2010 22:49:53 +0100 Lines: 44 Message-ID: References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <4BA53E77.8010208@elischer.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-28-199.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) In-Reply-To: <4BA53E77.8010208@elischer.org> Cc: freebsd-current@freebsd.org Subject: Re: Increasing MAXPHYS X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Mar 2010 21:50:12 -0000 Julian Elischer wrote: > Ivan Voras wrote: >> Julian Elischer wrote: >>> Alexander Motin wrote: >>>> Scott Long wrote: >>>>> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>>>>> Diminishing returns get hit pretty quickly with larger MAXPHYS >>>>>> values. >>>>>> As long as the I/O can be pipelined the reduced transaction rate >>>>>> becomes less interesting when the transaction rate is less than a >>>>>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>>>>> considering whether it is an issue or not. 256K is actually quite >>>>>> a reasonable value. Even 128K is reasonable. >>>>> I agree completely. I did quite a bit of testing on this in 2008 >>>>> and 2009. >>>>> I even added some hooks into CAM to support this, and I thought >>>>> that I had >>>>> discussed this extensively with Alexander at the time. Guess it >>>>> was yet another >>>>> wasted conversation with him =-( I'll repeat it here for the record. >>> >>> In the Fusion-io driver we find that the limiting factor is not the >>> size of MAXPHYS, but the fact that we can not push more than >>> 170k tps through geom. (in my test machine. I've seen more on some >>> beefier machines), but that is only a limit on small transacrtions, >> >> Do the GEOM threads (g_up, g_down) go into saturation? Effectively all >> IO is serialized through them. > > basically.. > > You can get better throughput by using TSC for timing because the geom > and devstat code does a bit of timing.. Geom can be told to turn off > it's timing but devstat can't. The 170 ktps is with TSC as timer, > and geom timing turned off. I see. I just ran randomio on a gzero device and with 10 userland threads (this is a slow 2xquad machine) I get g_up and g_down saturated fast with ~~ 120 ktps. Randomio uses gettimeofday() for measurements. Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect.