From owner-freebsd-arch@FreeBSD.ORG Sun Nov 19 14:07:44 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 30B1F16A40F for ; Sun, 19 Nov 2006 14:07:44 +0000 (UTC) (envelope-from julian@elischer.org) Received: from customer-domains.icp-qv1-irony13.iinet.net.au (customer-domains.icp-qv1-irony13.iinet.net.au [203.59.1.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E51B43D83 for ; Sun, 19 Nov 2006 14:07:26 +0000 (GMT) (envelope-from julian@elischer.org) Received: from 203-166-240-248.dyn.iinet.net.au (HELO [192.168.3.3]) ([203.166.240.248]) by customer-domains.icp-qv1-irony13.iinet.net.au with ESMTP; 19 Nov 2006 22:07:31 +0800 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAAADvzX0XLpvD4fmdsb2JhbAANjDsBAWs X-IronPort-AV: i="4.09,438,1157299200"; d="scan'208"; a="25002846:sNHT29452329" Message-ID: <45606524.9040000@elischer.org> Date: Sun, 19 Nov 2006 06:07:32 -0800 From: Julian Elischer User-Agent: Thunderbird 1.5.0.8 (Macintosh/20061025) MIME-Version: 1.0 To: Ivan Voras References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org Subject: Re: [Fwd: Re: Lockless algorithms [was Re: splxxx replacements?]] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Nov 2006 14:07:44 -0000 Ivan Voras wrote: > > There are patent issues and the GPL license, but this is just too > powerful an > algorithm to ignore. In many way, this brings out the ultimate in what > shared > memory system can do. > I don;t think that patent issues can be to much of an issue as RCU (as it is in my understanding) has been used in the kernel since the very earliest of days, even before there was SMP. Take the example of a file reference to an inode.. you want file X. you open it someone deletes it and creates a new one. you still have your copy to work on until you are finished with it.. when you finish, the reference count on the inode goes to 0 and it is freed. The other process can update the new version of the file without worrying about colliding with your copy. From my understanding this is RCU. (unless I'm thinking of a different RCU :-) From owner-freebsd-arch@FreeBSD.ORG Sun Nov 19 14:42:15 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D12F916A412 for ; Sun, 19 Nov 2006 14:42:15 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls405.htnet.hr (ls405.t-com.hr [195.29.150.135]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7F08343D5C for ; Sun, 19 Nov 2006 14:41:59 +0000 (GMT) (envelope-from ivoras@fer.hr) Received: from ls422.t-com.hr (ls422.t-com.hr [195.29.150.237]) by ls405.htnet.hr (Postfix) with ESMTP id 1B47A146B1D; Sun, 19 Nov 2006 15:42:08 +0100 (CET) Received: from ls422.t-com.hr (localhost.localdomain [127.0.0.1]) by ls422.t-com.hr (Qmlai) with ESMTP id ED303C90057; Sun, 19 Nov 2006 15:42:07 +0100 (CET) X-Envelope-Sender-Info: KDHLkYIFXCRHG7zIR7FVXkIRpTXt0bGIVSewDbCk7/lTw1kHEKzHYnX/8Zi03bEx X-Envelope-Sender: ivoras@fer.hr Received: from [10.0.0.100] (89-172-36-18.adsl.net.t-com.hr [89.172.36.18])by ls422.t-com.hr (Qmlai) with ESMTP id A70FE1308031; Sun, 19 Nov 2006 15:42:06 +0100 (CET) Message-ID: <45606D47.7000005@fer.hr> Date: Sun, 19 Nov 2006 15:42:15 +0100 From: Ivan Voras User-Agent: Thunderbird 1.5.0.8 (Windows/20061025) MIME-Version: 1.0 To: Julian Elischer References: <45606524.9040000@elischer.org> In-Reply-To: <45606524.9040000@elischer.org> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-imss-version: 2.044 X-imss-result: Passed X-imss-scores: Clean:35.54652 C:2 M:3 S:5 R:5 X-imss-settings: Baseline:1 C:1 M:1 S:1 R:1 (0.0000 0.0000) Cc: freebsd-arch@freebsd.org Subject: Re: [Fwd: Re: Lockless algorithms [was Re: splxxx replacements?]] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Nov 2006 14:42:15 -0000 Julian Elischer wrote: > I don;t think that patent issues can be to much of an issue as RCU (as > it is in my understanding) has been used in the kernel since the very > earliest of days, even before there was SMP. > > From my understanding this is RCU. (unless I'm thinking of a different > RCU :-) You're right, but I found this document: http://lse.sourceforge.net/locking/rcupdate.html and it mentions the algorithm/method was developed "for DYNIX/ptx, a UNIX operating system from Sequent Computer Systems Inc., now a part of IBM. Similar methods were also used for Tornado and K42 OS projects at University of Toronto and IBM Research." It seems to me that there's nothing to patent, since it's an obvious way to do lock-free update, but what do I know... I brought this up because I remembered PHK's thread on new callout mechanism and the discussion about scaling it to MP. From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 04:11:24 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6687C16A412 for ; Mon, 20 Nov 2006 04:11:24 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 715D343D6E for ; Mon, 20 Nov 2006 04:11:10 +0000 (GMT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.6/8.13.6) with ESMTP id kAK4BNNP064502 for ; Sun, 19 Nov 2006 20:11:23 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Sun, 19 Nov 2006 20:11:22 -0800 (PST) From: John Polstra To: arch@freebsd.org Cc: Subject: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 04:11:24 -0000 I have at least one motherboard on which MSI doesn't work. It's a Tyan S2721-533, also known as Thunder i7501 Pro. (When MSI is enabled and used, it often hangs the system solid.) It seems like we're going to need a quirk table to deal with this sort of thing. I have a couple of questions before I work on it. First, I assume that whether MSI works or not is at least partially a function of the motherboard and maybe the BIOS, as opposed to the chipset. Is that right? At least on this Tyan board, there seem to be some nice kenv variables (e.g., smbus.planar.product) already set that could help me recognize the board. Is that a reasonable approach? Second, is there already a similar quirk table that I could simply extend to deal with this, or does it make more sense to create a new one? Third, is there already an "initialize the whole PCI subsystem" function where I could handle the quirk table, or should I add a new SYSINIT? John From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 18:43:18 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DB71316A519 for ; Mon, 20 Nov 2006 18:43:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0765143DBD for ; Mon, 20 Nov 2006 18:41:05 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id kAKIf70A077243; Mon, 20 Nov 2006 13:41:09 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 20 Nov 2006 12:42:57 -0500 User-Agent: KMail/1.9.1 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611201242.58088.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Mon, 20 Nov 2006 13:41:10 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2207/Mon Nov 20 10:09:00 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: John Polstra Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 18:43:18 -0000 On Sunday 19 November 2006 23:11, John Polstra wrote: > I have at least one motherboard on which MSI doesn't work. It's a > Tyan S2721-533, also known as Thunder i7501 Pro. (When MSI is enabled > and used, it often hangs the system solid.) It seems like we're going > to need a quirk table to deal with this sort of thing. I have a > couple of questions before I work on it. > > First, I assume that whether MSI works or not is at least partially > a function of the motherboard and maybe the BIOS, as opposed to the > chipset. Is that right? At least on this Tyan board, there seem > to be some nice kenv variables (e.g., smbus.planar.product) already > set that could help me recognize the board. Is that a reasonable > approach? It's going to be a function of the chipset, as something in the chipset (presumably a Host -> PCI bridge) has to listen for writes to 0xfeeXXXXXX and convert them into APIC messages. There are two ways I planned on doing this: 1) Allow PCI-PCI bridges to be blacklisted, and the pcib_alloc_msi[x]() methods would compare the bridge's device id against a blacklist. This can matter if you have virtual PCI-PCI bridges that really a HT -> PCI bridge or the like. 2) Blacklist chipsets in the x86 MD code based on the device ID of the first Host -> PCI bridge at device 0.0.0. I haven't worked on this yet as I haven't had a broken system to test on yet. It would also probably be a good idea to see what chipsets Linux blacklists, though we might not have to blacklist as many (e.g. Linux blacklists the 7502 chipset because of issues with level triggered messages, but since we use edge-trigger it seems to work fine). > Second, is there already a similar quirk table that I could simply > extend to deal with this, or does it make more sense to create a new > one? No. > Third, is there already an "initialize the whole PCI subsystem" > function where I could handle the quirk table, or should I add > a new SYSINIT? See above. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 18:49:19 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C0CD116A518; Mon, 20 Nov 2006 18:49:19 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id E45F743D7C; Mon, 20 Nov 2006 18:48:56 +0000 (GMT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.6/8.13.6) with ESMTP id kAKInBDq085828; Mon, 20 Nov 2006 10:49:12 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200611201242.58088.jhb@freebsd.org> Date: Mon, 20 Nov 2006 10:49:11 -0800 (PST) From: John Polstra To: John Baldwin Cc: freebsd-arch@freebsd.org Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 18:49:19 -0000 On 20-Nov-2006 John Baldwin wrote: > On Sunday 19 November 2006 23:11, John Polstra wrote: >> First, I assume that whether MSI works or not is at least partially >> a function of the motherboard and maybe the BIOS, as opposed to the >> chipset. Is that right? At least on this Tyan board, there seem >> to be some nice kenv variables (e.g., smbus.planar.product) already >> set that could help me recognize the board. Is that a reasonable >> approach? > > It's going to be a function of the chipset, as something in the chipset > (presumably a Host -> PCI bridge) has to listen for writes to 0xfeeXXXXXX and > convert them into APIC messages. Hmm, this Tyan 2721 board has the Intel E7501 chipset. Isn't it supposed to support MSI? It doesn't work, and that's why I thought the motherboard might also be a factor. I'll paste in the pciconf output below. John hostb0@pci0:0:0: class=0x060000 card=0x00000000 chip=0x254c8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'E7501 Host Controller' class = bridge subclass = HOST-PCI pcib1@pci0:2:0: class=0x060400 card=0x00000000 chip=0x25438086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = 'E7500/E7501 HI_B Virtual PCI-to-PCI Bridge' class = bridge subclass = PCI-PCI pcib4@pci0:3:0: class=0x060400 card=0x00000000 chip=0x25458086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = 'E7500/E7501 HI_C Virtual PCI-to-PCI Bridge' class = bridge subclass = PCI-PCI none0@pci0:29:0: class=0x0c0300 card=0x24808086 chip=0x24828086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801CA/CAM (ICH3-S/ICH3-M) USB Controller' class = serial bus subclass = USB none1@pci0:29:1: class=0x0c0300 card=0x24808086 chip=0x24848086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801CA/CAM (ICH3-S/ICH3-M) USB Controller' class = serial bus subclass = USB pcib7@pci0:30:0: class=0x060400 card=0x00000000 chip=0x244e8086 rev=0x42 hdr=0x01 vendor = 'Intel Corporation' device = '82801BA/CA/DB/DBL/EB/ER/FB (ICH2/3/4/4/5/5/6), 6300ESB Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:31:0: class=0x060100 card=0x00000000 chip=0x24808086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801CA/CAM (ICH3-S/ICH3-M) LPC Interface' class = bridge subclass = PCI-ISA atapci0@pci0:31:1: class=0x01018a card=0x24808086 chip=0x248b8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801CA (ICH3) UltraATA/100 EIDE Controller' class = mass storage subclass = ATA none2@pci0:31:3: class=0x0c0500 card=0x24808086 chip=0x24838086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82801CA/CAM (ICH3-S/ICH3-M) SMBus Controller' class = serial bus subclass = SMBus none3@pci5:28:0: class=0x080020 card=0x14618086 chip=0x14618086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = 'P64H2 I/O Advanced Programmable Interrupt Controller' class = base peripheral subclass = interrupt controller pcib2@pci5:29:0: class=0x060400 card=0x00000050 chip=0x14608086 rev=0x04 hdr=0x01 vendor = 'Intel Corporation' device = '82870P2 P64H2 PCI/PCI-X Hub Controller' class = bridge subclass = PCI-PCI none4@pci5:30:0: class=0x080020 card=0x14618086 chip=0x14618086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = 'P64H2 I/O Advanced Programmable Interrupt Controller' class = base peripheral subclass = interrupt controller pcib3@pci5:31:0: class=0x060400 card=0x00000050 chip=0x14608086 rev=0x04 hdr=0x01 vendor = 'Intel Corporation' device = '82870P2 P64H2 PCI/PCI-X Hub Controller' class = bridge subclass = PCI-PCI em0@pci7:1:0: class=0x020000 card=0x10118086 chip=0x10108086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Dual Port Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet em1@pci7:1:1: class=0x020000 card=0x10118086 chip=0x10108086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Dual Port Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bge0@pci6:3:0: class=0x020000 card=0x164814e4 chip=0x164814e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5704 NetXtreme Dual Gigabit Adapter' class = network subclass = ethernet bge1@pci6:3:1: class=0x020000 card=0x164814e4 chip=0x164814e4 rev=0x10 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5704 NetXtreme Dual Gigabit Adapter' class = network subclass = ethernet none5@pci2:28:0: class=0x080020 card=0x14618086 chip=0x14618086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = 'P64H2 I/O Advanced Programmable Interrupt Controller' class = base peripheral subclass = interrupt controller pcib5@pci2:29:0: class=0x060400 card=0x00000050 chip=0x14608086 rev=0x04 hdr=0x01 vendor = 'Intel Corporation' device = '82870P2 P64H2 PCI/PCI-X Hub Controller' class = bridge subclass = PCI-PCI none6@pci2:30:0: class=0x080020 card=0x14618086 chip=0x14618086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = 'P64H2 I/O Advanced Programmable Interrupt Controller' class = base peripheral subclass = interrupt controller pcib6@pci2:31:0: class=0x060400 card=0x00000050 chip=0x14608086 rev=0x04 hdr=0x01 vendor = 'Intel Corporation' device = '82870P2 P64H2 PCI/PCI-X Hub Controller' class = bridge subclass = PCI-PCI fxp0@pci1:1:0: class=0x020000 card=0x10408086 chip=0x12298086 rev=0x10 hdr=0x00 vendor = 'Intel Corporation' device = '82550/1/7/8/9 EtherExpress PRO/100(B) Ethernet Adapter' class = network subclass = ethernet vgapci0@pci1:2:0: class=0x030000 card=0x80081002 chip=0x47521002 rev=0x27 hdr=0x00 vendor = 'ATI Technologies Inc' device = 'Rage XL PCI' class = display subclass = VGA From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 21:08:07 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2FAB916A4A0 for ; Mon, 20 Nov 2006 21:08:07 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D48A43D6E for ; Mon, 20 Nov 2006 21:07:35 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id kAKL7mBu078143; Mon, 20 Nov 2006 16:07:48 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: John Polstra Date: Mon, 20 Nov 2006 15:49:21 -0500 User-Agent: KMail/1.9.1 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611201549.21685.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Mon, 20 Nov 2006 16:07:48 -0500 (EST) X-Virus-Scanned: ClamAV 0.88.3/2208/Mon Nov 20 12:20:46 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.1.3 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on server.baldwin.cx Cc: freebsd-arch@freebsd.org Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 21:08:07 -0000 On Monday 20 November 2006 13:49, John Polstra wrote: > On 20-Nov-2006 John Baldwin wrote: > > On Sunday 19 November 2006 23:11, John Polstra wrote: > >> First, I assume that whether MSI works or not is at least partially > >> a function of the motherboard and maybe the BIOS, as opposed to the > >> chipset. Is that right? At least on this Tyan board, there seem > >> to be some nice kenv variables (e.g., smbus.planar.product) already > >> set that could help me recognize the board. Is that a reasonable > >> approach? > > > > It's going to be a function of the chipset, as something in the chipset > > (presumably a Host -> PCI bridge) has to listen for writes to 0xfeeXXXXXX and > > convert them into APIC messages. > > Hmm, this Tyan 2721 board has the Intel E7501 chipset. Isn't it > supposed to support MSI? It doesn't work, and that's why I thought > the motherboard might also be a factor. I'll paste in the pciconf > output below. I'm not sure if 7501 works or not. Scott might know if there are errata for it. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 22:45:53 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0B0FD16A412; Mon, 20 Nov 2006 22:45:53 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 35BB043D68; Mon, 20 Nov 2006 22:45:24 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3066D45684; Mon, 20 Nov 2006 23:45:42 +0100 (CET) Received: from localhost (dma35.neoplus.adsl.tpnet.pl [83.24.56.35]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D0CF345681; Mon, 20 Nov 2006 23:45:36 +0100 (CET) Date: Mon, 20 Nov 2006 23:45:32 +0100 From: Pawel Jakub Dawidek To: Attilio Rao Message-ID: <20061120224532.GA232@garage.freebsd.pl> References: <3bbf2fe10610181518k68356528i154267c0bd1b1a77@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IS0zKkzwUGydFO0o" Content-Disposition: inline In-Reply-To: <3bbf2fe10610181518k68356528i154267c0bd1b1a77@mail.gmail.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-current@freebsd.org, Robert Watson , freebsd-arch@freebsd.org Subject: Re: sx locks rewriting - needs testers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 22:45:53 -0000 --IS0zKkzwUGydFO0o Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Oct 19, 2006 at 12:18:16AM +0200, Attilio Rao wrote: > In my P4 branch: //depot/user/attilio/attilio_smpng/... > you can find a sx locks rewriting using the optimized semantic of > rwlocks; in the end this might result in a valuable performance > improvement. I'm using sx(9) locks for Solaris mutexes and rwlocks, as I need locks for ZFS that I can sleep while holding. =46rom my simple tests your locks are stable and ZFS is 10% faster. Nice work. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --IS0zKkzwUGydFO0o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFFYjAMForvXbEpPzQRAlHrAJ4qFZjGYWtc4//lZfylAYr3MUhNaQCbBG98 mIB6AXZ6rdipJyeonJ6bT/g= =4XU7 -----END PGP SIGNATURE----- --IS0zKkzwUGydFO0o-- From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 23:45:05 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9487616A60B for ; Mon, 20 Nov 2006 23:45:05 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.238]) by mx1.FreeBSD.org (Postfix) with ESMTP id C56BE43D5F for ; Mon, 20 Nov 2006 23:44:45 +0000 (GMT) (envelope-from jfvogel@gmail.com) Received: by wx-out-0506.google.com with SMTP id s18so1721567wxc for ; Mon, 20 Nov 2006 15:45:03 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=m2KgcGTly2E1OWat2hJF2iRZOUZYb3HodUODFlqc2nG+frSMHaeuNk6ZrRLLa1p4kgBmh+O25p2P8yAUwADkHHrkLjJv232o9dE/SsZZ0ZstXNQz0X2wu3x4+ZHe/U28kii9PrefWjOF06IN3YFJx053m62Juk0OaO5IthUZdfQ= Received: by 10.90.84.17 with SMTP id h17mr4195936agb.1164066303371; Mon, 20 Nov 2006 15:45:03 -0800 (PST) Received: by 10.35.118.6 with HTTP; Mon, 20 Nov 2006 15:45:03 -0800 (PST) Message-ID: <2a41acea0611201545s6a9848e2k952845f4ccedc04d@mail.gmail.com> Date: Mon, 20 Nov 2006 15:45:03 -0800 From: "Jack Vogel" To: "John Baldwin" In-Reply-To: <200611201549.21685.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200611201549.21685.jhb@freebsd.org> Cc: freebsd-arch@freebsd.org, John Polstra Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 23:45:05 -0000 On 11/20/06, John Baldwin wrote: > On Monday 20 November 2006 13:49, John Polstra wrote: > > On 20-Nov-2006 John Baldwin wrote: > > > On Sunday 19 November 2006 23:11, John Polstra wrote: > > >> First, I assume that whether MSI works or not is at least partially > > >> a function of the motherboard and maybe the BIOS, as opposed to the > > >> chipset. Is that right? At least on this Tyan board, there seem > > >> to be some nice kenv variables (e.g., smbus.planar.product) already > > >> set that could help me recognize the board. Is that a reasonable > > >> approach? > > > > > > It's going to be a function of the chipset, as something in the chipset > > > (presumably a Host -> PCI bridge) has to listen for writes to 0xfeeXXXXXX > and > > > convert them into APIC messages. > > > > Hmm, this Tyan 2721 board has the Intel E7501 chipset. Isn't it > > supposed to support MSI? It doesn't work, and that's why I thought > > the motherboard might also be a factor. I'll paste in the pciconf > > output below. > > I'm not sure if 7501 works or not. Scott might know if there are errata for > it. I've looked at the specs for that chipset, and yes, it appears to have MSI. You're right though, for anything to work surely needs MB support as well. MSI is only going to work on PCI-X and PCI-E you know. Earlier someone asserted quirks would be chipset based, you know one thing about Linux quirks is they don't tie them down to anything specific like that, its just some known issue with a way to detect it. I could imagine a motherboard maker that screws something up in their design so even if a chipset in theory supports MSI the thing still wont work, so I think we should be ready to handle that. When you say it doesnt work, what are you trying to use it with, the E1000s? Jack From owner-freebsd-arch@FreeBSD.ORG Mon Nov 20 23:59:22 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF43116A407 for ; Mon, 20 Nov 2006 23:59:22 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id A43A843D92 for ; Mon, 20 Nov 2006 23:59:02 +0000 (GMT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.6/8.13.6) with ESMTP id kAKNxKfN093518; Mon, 20 Nov 2006 15:59:20 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <2a41acea0611201545s6a9848e2k952845f4ccedc04d@mail.gmail.com> Date: Mon, 20 Nov 2006 15:59:20 -0800 (PST) From: John Polstra To: Jack Vogel Cc: freebsd-arch@freebsd.org Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Nov 2006 23:59:22 -0000 On 20-Nov-2006 Jack Vogel wrote: > On 11/20/06, John Baldwin wrote: >> I'm not sure if 7501 works or not. Scott might know if there are errata for >> it. > > I've looked at the specs for that chipset, and yes, it appears to have MSI. > You're right though, for anything to work surely needs MB support as well. > MSI is only going to work on PCI-X and PCI-E you know. > > Earlier someone asserted quirks would be chipset based, you know > one thing about Linux quirks is they don't tie them down to anything > specific like that, its just some known issue with a way to detect it. > I could imagine a motherboard maker that screws something up in > their design so even if a chipset in theory supports MSI the thing > still wont work, so I think we should be ready to handle that. > > When you say it doesnt work, what are you trying to use it with, the > E1000s? Yes, it's the 82546EB that's on the motherboard. When MSI is enabled and I try to do anything with its network interfaces, the system hangs solid (won't even echo console keystrokes) at least half the time. When it doesn't hang, I get TX watchdog timeouts on both interfaces. It works perfectly if I disable MSI. John From owner-freebsd-arch@FreeBSD.ORG Tue Nov 21 00:02:23 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3A7916A412 for ; Tue, 21 Nov 2006 00:02:23 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.228]) by mx1.FreeBSD.org (Postfix) with ESMTP id C5DC743D46 for ; Tue, 21 Nov 2006 00:02:02 +0000 (GMT) (envelope-from jfvogel@gmail.com) Received: by wx-out-0506.google.com with SMTP id s18so1726230wxc for ; Mon, 20 Nov 2006 16:02:21 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=ap3H8B44BHZ9IIjwnJnI++BWioiLmrY35ufCwUMYJsDoNjJtcuKCXwLjDhS13GHXNctlQiFLcZKYsk6wMSFceIDKb3MryuOrTHnd7DbHB0cT8Q4vCDThTINxjVTdcnydhQscpfhEb8Wfp8u/O1casgFXk8P4CevT1aF34lmcNfA= Received: by 10.90.94.2 with SMTP id r2mr4230492agb.1164067340854; Mon, 20 Nov 2006 16:02:20 -0800 (PST) Received: by 10.35.118.6 with HTTP; Mon, 20 Nov 2006 16:02:20 -0800 (PST) Message-ID: <2a41acea0611201602m6cf8d983lb4699bb1037efcdc@mail.gmail.com> Date: Mon, 20 Nov 2006 16:02:20 -0800 From: "Jack Vogel" To: "John Polstra" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <2a41acea0611201545s6a9848e2k952845f4ccedc04d@mail.gmail.com> Cc: freebsd-arch@freebsd.org Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Nov 2006 00:02:23 -0000 On 11/20/06, John Polstra wrote: > On 20-Nov-2006 Jack Vogel wrote: > > On 11/20/06, John Baldwin wrote: > >> I'm not sure if 7501 works or not. Scott might know if there are errata for > >> it. > > > > I've looked at the specs for that chipset, and yes, it appears to have MSI. > > You're right though, for anything to work surely needs MB support as well. > > MSI is only going to work on PCI-X and PCI-E you know. > > > > Earlier someone asserted quirks would be chipset based, you know > > one thing about Linux quirks is they don't tie them down to anything > > specific like that, its just some known issue with a way to detect it. > > I could imagine a motherboard maker that screws something up in > > their design so even if a chipset in theory supports MSI the thing > > still wont work, so I think we should be ready to handle that. > > > > When you say it doesnt work, what are you trying to use it with, the > > E1000s? > > Yes, it's the 82546EB that's on the motherboard. When MSI is > enabled and I try to do anything with its network interfaces, the > system hangs solid (won't even echo console keystrokes) at least > half the time. When it doesn't hang, I get TX watchdog timeouts on > both interfaces. It works perfectly if I disable MSI. I'll look into this. I have 546 NICs that aren't LOMs, I can check those on a MB that I know supports MSI, and from there see if I can find the Tyan or something that has that chipset and that LOM. Its a short week and I'm busy with some other projects, so be patient please :) Jack From owner-freebsd-arch@FreeBSD.ORG Tue Nov 21 00:06:31 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1EA4F16A415 for ; Tue, 21 Nov 2006 00:06:31 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from blake.polstra.com (blake.polstra.com [64.81.189.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id E525543D49 for ; Tue, 21 Nov 2006 00:06:11 +0000 (GMT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [64.81.189.67]) by blake.polstra.com (8.13.6/8.13.6) with ESMTP id kAL06UxK093653; Mon, 20 Nov 2006 16:06:30 -0800 (PST) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.5 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <2a41acea0611201602m6cf8d983lb4699bb1037efcdc@mail.gmail.com> Date: Mon, 20 Nov 2006 16:06:30 -0800 (PST) From: John Polstra To: Jack Vogel Cc: freebsd-arch@freebsd.org Subject: Re: Where do MSI quirks belong? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Nov 2006 00:06:31 -0000 On 21-Nov-2006 Jack Vogel wrote: > On 11/20/06, John Polstra wrote: >> Yes, it's the 82546EB that's on the motherboard. When MSI is >> enabled and I try to do anything with its network interfaces, the >> system hangs solid (won't even echo console keystrokes) at least >> half the time. When it doesn't hang, I get TX watchdog timeouts on >> both interfaces. It works perfectly if I disable MSI. > > I'll look into this. I have 546 NICs that aren't LOMs, I can check those on > a MB that I know supports MSI, and from there see if I can find the Tyan > or something that has that chipset and that LOM. Hey, thanks! > Its a short week and I'm busy with some other projects, so be patient > please :) No problem at all. I'm just running with MSI disabled for now. John From owner-freebsd-arch@FreeBSD.ORG Tue Nov 21 10:58:48 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 28AFD16A403 for ; Tue, 21 Nov 2006 10:58:48 +0000 (UTC) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (comp.chem.msu.su [158.250.32.97]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9998243D4C for ; Tue, 21 Nov 2006 10:58:25 +0000 (GMT) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.13.4/8.13.3) with ESMTP id kALAwhx1033800 for ; Tue, 21 Nov 2006 13:58:43 +0300 (MSK) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.13.4/8.13.3/Submit) id kALAwhRJ033799 for freebsd-arch@freebsd.org; Tue, 21 Nov 2006 13:58:43 +0300 (MSK) (envelope-from yar) Date: Tue, 21 Nov 2006 13:58:42 +0300 From: Yar Tikhiy To: freebsd-arch@freebsd.org Message-ID: <20061121105842.GD13289@comp.chem.msu.su> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="SUOF0GtieIMvvwua" Content-Disposition: inline User-Agent: Mutt/1.5.9i Subject: Limitations in fts(3) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Nov 2006 10:58:48 -0000 --SUOF0GtieIMvvwua Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi folks, Our FTSENT appears to have unjustified limits as it uses u_short for fts_pathlen and short for fts_level. Consequently, our fts(3) fails miserably on deep trees although there are plenty of resources (memory) available to it--see PR bin/104458. (Traversing unlimited trees using limited resources is a different topic.) Note that the fts_pathlen range is not limited by PATH_MAX because fts(3) constructs long paths by itself. To begin with, paths longer than PATH_MAX are quite useless because they can't be passed to syscalls. Bruce Evans proposed that we could have a new flag FTS_NOPATH to disable generating full pathnames at all. It can be good if used without FTS_NOCHDIR, as fts(3) will take the process to the proper directory so that it can access its entries by their relative pathnames. This way can relieve us of the fts_pathlen limitation without touching the structures from fts.h. However, there still is the fts_level limit. This field begs to be extended to int64_t, as we face all those yottabyte file systems. (But at least it could be int to match FTW.level from ftw(3).) NetBSD folks have already extended fts_pathlen and fts_namelen to u_int, but they forgot about fts_level. They also extended fts_number to int64_t. It looks like time for us to change our fts.h, too. Symbol versioning now even allows us to have both new and old fts(3) in the same libc if we want to. Here's a prototype patch extending fts fields. I tested it on i386, and it just worked. Of course, I'll test it on other arches if we decide this is the way to go. The string/array length fields become size_t because it's the only proper type for them. Note that the patch also extends flag fields to u_int in order to get rid of short types in fts.h entirely, since on modern arches they buy us neither memory economy nor computing speed. It's also a chance to extend fts_number to int64_t. According to Google Code, fts_bignum isn't used outside our src tree, so we can eventually drop it in favour of the extended fts_number. What do you think about all this? Thanks! -- Yar --SUOF0GtieIMvvwua Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=fts-patch --- include/fts.h.orig Fri Jan 7 03:06:20 2005 +++ include/fts.h Mon Nov 20 17:04:48 2006 @@ -44,8 +44,8 @@ dev_t fts_dev; /* starting device # */ char *fts_path; /* path for this descent */ int fts_rfd; /* fd for root */ - int fts_pathlen; /* sizeof(path) */ - int fts_nitems; /* elements in the sort array */ + size_t fts_pathlen; /* sizeof(path) */ + size_t fts_nitems; /* elements in the sort array */ int (*fts_compar) /* compare function */ (const struct _ftsent * const *, const struct _ftsent * const *); @@ -69,22 +69,15 @@ struct _ftsent *fts_cycle; /* cycle node */ struct _ftsent *fts_parent; /* parent directory */ struct _ftsent *fts_link; /* next file in directory */ - union { - struct { - long __fts_number; /* local numeric value */ - void *__fts_pointer; /* local address value */ - } __struct_ftsent; - int64_t __fts_bignum; - } __union_ftsent; -#define fts_number __union_ftsent.__struct_ftsent.__fts_number -#define fts_pointer __union_ftsent.__struct_ftsent.__fts_pointer -#define fts_bignum __union_ftsent.__fts_bignum + int64_t fts_number; /* local numeric value */ +#define fts_bignum fts_number /* XXX non-std, should go away */ + void *fts_pointer; /* local address value */ char *fts_accpath; /* access path */ char *fts_path; /* root path */ int fts_errno; /* errno for this node */ int fts_symfd; /* fd for symlink */ - u_short fts_pathlen; /* strlen(fts_path) */ - u_short fts_namelen; /* strlen(fts_name) */ + size_t fts_pathlen; /* strlen(fts_path) */ + size_t fts_namelen; /* strlen(fts_name) */ ino_t fts_ino; /* inode */ dev_t fts_dev; /* device */ @@ -92,7 +85,7 @@ #define FTS_ROOTPARENTLEVEL -1 #define FTS_ROOTLEVEL 0 - short fts_level; /* depth (-1 to N) */ + int64_t fts_level; /* depth (-1 to N) */ #define FTS_D 1 /* preorder directory */ #define FTS_DC 2 /* directory that causes cycles */ @@ -108,18 +101,18 @@ #define FTS_SL 12 /* symbolic link */ #define FTS_SLNONE 13 /* symbolic link without target */ #define FTS_W 14 /* whiteout object */ - u_short fts_info; /* user flags for FTSENT structure */ + u_int fts_info; /* user flags for FTSENT structure */ #define FTS_DONTCHDIR 0x01 /* don't chdir .. to the parent */ #define FTS_SYMFOLLOW 0x02 /* followed a symlink to get here */ #define FTS_ISW 0x04 /* this is a whiteout object */ - u_short fts_flags; /* private flags for FTSENT structure */ + u_int fts_flags; /* private flags for FTSENT structure */ #define FTS_AGAIN 1 /* read node again */ #define FTS_FOLLOW 2 /* follow symbolic link */ #define FTS_NOINSTR 3 /* no instructions */ #define FTS_SKIP 4 /* discard node */ - u_short fts_instr; /* fts_set() instructions */ + u_int fts_instr; /* fts_set() instructions */ struct stat *fts_statp; /* stat(2) information */ char *fts_name; /* file name */ --- lib/libc/gen/fts.c.orig Tue Jun 8 10:23:23 2004 +++ lib/libc/gen/fts.c Mon Nov 20 17:17:39 2006 @@ -56,15 +56,15 @@ #include #include "un-namespace.h" -static FTSENT *fts_alloc(FTS *, char *, int); +static FTSENT *fts_alloc(FTS *, char *, size_t); static FTSENT *fts_build(FTS *, int); static void fts_lfree(FTSENT *); static void fts_load(FTS *, FTSENT *); static size_t fts_maxarglen(char * const *); static void fts_padjust(FTS *, FTSENT *); static int fts_palloc(FTS *, size_t); -static FTSENT *fts_sort(FTS *, FTSENT *, int); -static u_short fts_stat(FTS *, FTSENT *, int); +static FTSENT *fts_sort(FTS *, FTSENT *, size_t); +static u_int fts_stat(FTS *, FTSENT *, int); static int fts_safe_changedir(FTS *, FTSENT *, int, char *); static int fts_ufslinks(FTS *, const FTSENT *); @@ -119,9 +119,8 @@ struct _fts_private *priv; FTS *sp; FTSENT *p, *root; - int nitems; FTSENT *parent, *tmp; - int len; + size_t len, nitems; /* Options check. */ if (options & ~FTS_OPTIONMASK) { @@ -228,7 +227,7 @@ FTS *sp; FTSENT *p; { - int len; + size_t len; char *cp; /* @@ -630,12 +629,12 @@ { struct dirent *dp; FTSENT *p, *head; - int nitems; FTSENT *cur, *tail; DIR *dirp; void *oldaddr; - size_t dnamlen; - int cderrno, descend, len, level, maxlen, nlinks, oflag, saved_errno, + size_t dnamlen, len, maxlen, nitems; + int64_t level; + int cderrno, descend, nlinks, oflag, saved_errno, nostat, doadjust; char *cp; @@ -745,7 +744,7 @@ if (!ISSET(FTS_SEEDOT) && ISDOT(dp->d_name)) continue; - if ((p = fts_alloc(sp, dp->d_name, (int)dnamlen)) == NULL) + if ((p = fts_alloc(sp, dp->d_name, dnamlen)) == NULL) goto mem1; if (dnamlen >= maxlen) { /* include space for NUL */ oldaddr = sp->fts_path; @@ -774,21 +773,6 @@ maxlen = sp->fts_pathlen - len; } - if (len + dnamlen >= USHRT_MAX) { - /* - * In an FTSENT, fts_pathlen is a u_short so it is - * possible to wraparound here. If we do, free up - * the current structure and the structures already - * allocated, then error out with ENAMETOOLONG. - */ - free(p); - fts_lfree(head); - (void)closedir(dirp); - cur->fts_info = FTS_ERR; - SET(FTS_STOP); - errno = ENAMETOOLONG; - return (NULL); - } p->fts_level = level; p->fts_parent = sp->fts_cur; p->fts_pathlen = len + dnamlen; @@ -889,7 +873,7 @@ return (head); } -static u_short +static u_int fts_stat(sp, p, follow) FTS *sp; FTSENT *p; @@ -991,7 +975,7 @@ fts_sort(sp, head, nitems) FTS *sp; FTSENT *head; - int nitems; + size_t nitems; { FTSENT **ap, *p; @@ -1023,7 +1007,7 @@ fts_alloc(sp, name, namelen) FTS *sp; char *name; - int namelen; + size_t namelen; { FTSENT *p; size_t len; @@ -1095,18 +1079,6 @@ { sp->fts_pathlen += more + 256; - /* - * Check for possible wraparound. In an FTS, fts_pathlen is - * a signed int but in an FTSENT it is an unsigned short. - * We limit fts_pathlen to USHRT_MAX to be safe in both cases. - */ - if (sp->fts_pathlen < 0 || sp->fts_pathlen >= USHRT_MAX) { - if (sp->fts_path) - free(sp->fts_path); - sp->fts_path = NULL; - errno = ENAMETOOLONG; - return (1); - } sp->fts_path = reallocf(sp->fts_path, sp->fts_pathlen); return (sp->fts_path == NULL); } --SUOF0GtieIMvvwua-- From owner-freebsd-arch@FreeBSD.ORG Wed Nov 22 19:00:33 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0DC4716A4FB for ; Wed, 22 Nov 2006 19:00:33 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB67A43D62 for ; Wed, 22 Nov 2006 18:59:50 +0000 (GMT) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id 114812C2ADC; Wed, 22 Nov 2006 13:00:20 -0600 (CST) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id RC5B1tk3wHvm; Wed, 22 Nov 2006 13:00:19 -0600 (CST) Received: from [216.63.78.18] (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id 4602B2C2AD1; Wed, 22 Nov 2006 13:00:19 -0600 (CST) Message-ID: <45649E42.70409@cs.rice.edu> Date: Wed, 22 Nov 2006 13:00:18 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.10) Gecko/20050817 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kip Macy , arch@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alan Cox Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 19:00:33 -0000 Kip Macy wrote: > Do you have any thoughts on when superpage support might go into -CURRENT? > I plan to do it over the Christmas holidays. I will merge it in three stages: First, the new buddy-style physical memory allocator (vm/vm_phys.c). Second, the superpage reservation system. Third, pmap support for individual architectures. Aside from the direct benefits of superpage support on TLB performance for applications, there are also some other side benefits: First, in the pmap for amd64, arm, and i386, we maintain a single pv list per superpage. So, substantial memory savings are possible for applications that share large amounts of memory between address spaces. Second, the new physical memory allocator tries to cluster allocations in the physical address space. This is beneficial on architectures like amd64 where we extensively use the kernel's direct map to access kernel data because we experience fewer TLB misses. Third, contigmalloc(9) is completely reimplemented. So, for example, M_NOWAIT actually has an effect. In particular, Giant is no longer required for M_NOWAIT allocations. Fourth, the new physical memory allocator supports sparse physical address spaces, like the Itaniums have. At present, -CURRENT cannot utilize the entire physical memory on many Itaniums. There is only one caveat. Idle-time page prezeroing is not supported. However, ever since the VM system emerged from the Giant kernel lock, I've seen little or no benefit from it. In any case, I've left the inferfaces in place so it could return if an effective implementation is found. Lastly, page coloring dies. However, the beneficial effects of page coloring are for the most part captured by superpages. Specifically, regardless of whether the pmap is able to promote a particular region of the address space to a superpage, e.g., due to heterogenous access rights of pages within the region, if the memory has been provided by the reservation system it will have "perfect" coloring. Regards, Alan From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 00:37:41 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 32A1216A412 for ; Thu, 23 Nov 2006 00:37:41 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.184]) by mx1.FreeBSD.org (Postfix) with ESMTP id F037043D55 for ; Thu, 23 Nov 2006 00:37:08 +0000 (GMT) (envelope-from kip.macy@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so661277nfc for ; Wed, 22 Nov 2006 16:37:39 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=EOlZYS4OR5OijwtrkCbP5OjLaae7K8tsccQ8PwviGJBXTJk6DVUfH5g1y9aa1Ma8qy3f4xpNMitHr7BTl6dvBmahH4ubAniv4PfIMKCmbNfBLX8CGNU/9rfbtxDkkYS5L3kIqqzKw2S/HTMOhP9mgPBXfvwvs/QM7sZQQQL8o9k= Received: by 10.82.175.2 with SMTP id x2mr1347658bue.1164242258626; Wed, 22 Nov 2006 16:37:38 -0800 (PST) Received: by 10.82.191.20 with HTTP; Wed, 22 Nov 2006 16:37:38 -0800 (PST) Message-ID: Date: Wed, 22 Nov 2006 16:37:38 -0800 From: "Kip Macy" To: "Alan Cox" In-Reply-To: <45649E42.70409@cs.rice.edu> MIME-Version: 1.0 References: <45649E42.70409@cs.rice.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 00:37:41 -0000 On 11/22/06, Alan Cox wrote: > > Kip Macy wrote: > > > Do you have any thoughts on when superpage support might go into > -CURRENT? > > > I plan to do it over the Christmas holidays. I will merge it in three > stages: First, the new buddy-style physical memory allocator > (vm/vm_phys.c). Second, the superpage reservation system. Third, pmap > support for individual architectures. That is great to hear. I'm waiting on pins and needles for this. With 4 page sizes and comparatively small TLBs, the T1 will likely benefit greatly from your work. -Kip From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 02:14:12 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BFA716A412; Thu, 23 Nov 2006 02:14:12 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from digger1.defence.gov.au (digger1.defence.gov.au [203.5.217.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2490E43D4C; Thu, 23 Nov 2006 02:13:38 +0000 (GMT) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from ednmsw501.dsto.defence.gov.au (ednmsw501.dsto.defence.gov.au [131.185.2.150]) by digger1.defence.gov.au with ESMTP id kAN2CFhJ024391; Thu, 23 Nov 2006 12:42:15 +1030 (CST) Received: from muttley2.dsto.defence.gov.au (unverified) by ednmsw501.dsto.defence.gov.au (Content Technologies SMTPRS 4.3.17) with ESMTP id ; Thu, 23 Nov 2006 12:43:55 +1030 Received: from ednex510.dsto.defence.gov.au (ednex510.dsto.defence.gov.au [131.185.2.170]) by muttley2.dsto.defence.gov.au (8.13.6/8.13.6) with ESMTP id kAN28Vve016328 ; Thu, 23 Nov 2006 12:38:31 +1030 (CST) Received: from obelix.dsto.defence.gov.au ([203.6.60.208]) by ednex510.dsto.defence.gov.au with Microsoft SMTPSVC (6.0.3790.1830); Thu, 23 Nov 2006 12:38:48 +1030 Received: from obelix.dsto.defence.gov.au (localhost [127.0.0.1]) by obelix.dsto.defence.gov.au (8.13.7/8.13.7) with ESMTP id kAN27mit051670; Thu, 23 Nov 2006 10:07:49 +0800 (WST) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: (from wilkinsa@localhost) by obelix.dsto.defence.gov.au (8.13.7/8.13.8/Submit) id kAN27mjK051661; Thu, 23 Nov 2006 10:07:48 +0800 (WST) (envelope-from wilkinsa) Date: Thu, 23 Nov 2006 10:07:48 +0800 From: "Wilkinson, Alex" To: freebsd-arch@freebsd.org, arch@freebsd.org Message-ID: <20061123020747.GZ2260@obelix.dsto.defence.gov.au> Mail-Followup-To: freebsd-arch@freebsd.org, arch@freebsd.org References: <45649E42.70409@cs.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <45649E42.70409@cs.rice.edu> User-Agent: Mutt/1.5.13 (2006-08-11) X-OriginalArrivalTime: 23 Nov 2006 02:08:48.0851 (UTC) FILETIME=[5054D230:01C70EA4] X-TM-AS-Product-Ver: SMEX-7.0.0.1345-3.6.1039-14784.003 X-TM-AS-Result: No--2.772200-0.000000-31 Cc: Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 02:14:12 -0000 0n Wed, Nov 22, 2006 at 01:00:18PM -0600, Alan Cox wrote: >Kip Macy wrote: > >>Do you have any thoughts on when superpage support might go into -CURRENT? erm, what is superpage ? -aW From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 02:14:12 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BFA716A412; Thu, 23 Nov 2006 02:14:12 +0000 (UTC) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from digger1.defence.gov.au (digger1.defence.gov.au [203.5.217.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2490E43D4C; Thu, 23 Nov 2006 02:13:38 +0000 (GMT) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: from ednmsw501.dsto.defence.gov.au (ednmsw501.dsto.defence.gov.au [131.185.2.150]) by digger1.defence.gov.au with ESMTP id kAN2CFhJ024391; Thu, 23 Nov 2006 12:42:15 +1030 (CST) Received: from muttley2.dsto.defence.gov.au (unverified) by ednmsw501.dsto.defence.gov.au (Content Technologies SMTPRS 4.3.17) with ESMTP id ; Thu, 23 Nov 2006 12:43:55 +1030 Received: from ednex510.dsto.defence.gov.au (ednex510.dsto.defence.gov.au [131.185.2.170]) by muttley2.dsto.defence.gov.au (8.13.6/8.13.6) with ESMTP id kAN28Vve016328 ; Thu, 23 Nov 2006 12:38:31 +1030 (CST) Received: from obelix.dsto.defence.gov.au ([203.6.60.208]) by ednex510.dsto.defence.gov.au with Microsoft SMTPSVC (6.0.3790.1830); Thu, 23 Nov 2006 12:38:48 +1030 Received: from obelix.dsto.defence.gov.au (localhost [127.0.0.1]) by obelix.dsto.defence.gov.au (8.13.7/8.13.7) with ESMTP id kAN27mit051670; Thu, 23 Nov 2006 10:07:49 +0800 (WST) (envelope-from wilkinsa@obelix.dsto.defence.gov.au) Received: (from wilkinsa@localhost) by obelix.dsto.defence.gov.au (8.13.7/8.13.8/Submit) id kAN27mjK051661; Thu, 23 Nov 2006 10:07:48 +0800 (WST) (envelope-from wilkinsa) Date: Thu, 23 Nov 2006 10:07:48 +0800 From: "Wilkinson, Alex" To: freebsd-arch@freebsd.org, arch@freebsd.org Message-ID: <20061123020747.GZ2260@obelix.dsto.defence.gov.au> Mail-Followup-To: freebsd-arch@freebsd.org, arch@freebsd.org References: <45649E42.70409@cs.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <45649E42.70409@cs.rice.edu> User-Agent: Mutt/1.5.13 (2006-08-11) X-OriginalArrivalTime: 23 Nov 2006 02:08:48.0851 (UTC) FILETIME=[5054D230:01C70EA4] X-TM-AS-Product-Ver: SMEX-7.0.0.1345-3.6.1039-14784.003 X-TM-AS-Result: No--2.772200-0.000000-31 Cc: Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 02:14:12 -0000 0n Wed, Nov 22, 2006 at 01:00:18PM -0600, Alan Cox wrote: >Kip Macy wrote: > >>Do you have any thoughts on when superpage support might go into -CURRENT? erm, what is superpage ? -aW From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 03:25:26 2006 Return-Path: X-Original-To: arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7730C16A403 for ; Thu, 23 Nov 2006 03:25:26 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BF5143D45 for ; Thu, 23 Nov 2006 03:24:53 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id 3153C61FD49; Thu, 23 Nov 2006 14:25:24 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id A04CB27417; Thu, 23 Nov 2006 14:25:22 +1100 (EST) Date: Thu, 23 Nov 2006 14:25:16 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Alan Cox In-Reply-To: <45649E42.70409@cs.rice.edu> Message-ID: <20061123124725.P35210@delplex.bde.org> References: <45649E42.70409@cs.rice.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kip Macy , arch@FreeBSD.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 03:25:26 -0000 On Wed, 22 Nov 2006, Alan Cox wrote: > There is only one caveat. Idle-time page prezeroing is not supported. > However, ever since the VM system emerged from the Giant kernel lock, I've > seen little or no benefit from it. ... That's probably because PREEMPTION is broken and the brokenness turns idle-time page prezeroing into a pessimization. Without PREEMPTION I see much the same benefits from idle-time page prezeroing as in RELENG_4 -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 of a RELENG_4 with a -current i386 SMP kernel and a ~5.2 userland on a Turion X2 2GHz: %%% vm.idlezero_enable=0: -------------------------------------------------------------- >>> elf make world completed on Thu Nov 23 12:01:56 EST 2006 (started Thu Nov 23 11:50:59 EST 2006) -------------------------------------------------------------- 656.36 real 815.59 user 194.92 sys 23572 maximum resident set size 1164 average shared memory size 1212 average unshared data size 128 average unshared stack size 14202993 page reclaims 6911 page faults 0 swaps 14686 block input operations 4647 block output operations 77645 messages sent 0 messages received 35459 signals received 838638 voluntary context switches 391631 involuntary context switches vm.idlezero_enable=1: -------------------------------------------------------------- >>> elf make world completed on Thu Nov 23 12:35:54 EST 2006 (started Thu Nov 23 12:25:07 EST 2006) -------------------------------------------------------------- 647.19 real 814.16 user 185.69 sys 23572 maximum resident set size 1168 average shared memory size 1220 average unshared data size 128 average unshared stack size 14202807 page reclaims 6958 page faults 0 swaps 14534 block input operations 4689 block output operations 77466 messages sent 0 messages received 35456 signals received 847575 voluntary context switches 397783 involuntary context switches %%% With idlezero enabled and PREEMPTION not enabled in the above, pgzero runs in actual idle time for 14 seconds and reduces both the real and sys times by 9 seconds (1.5% of real time and 5% of system time). With idlezero enabled and PREEMPTION enabled (details not shown), PREEMPTION doesn't actually work but pgzero depends on it working, so pgzero runs for much longer than in the above, with all the extra time stolen from non-idle time. In my makeworld benchmarks, this gives total benefits that are negative and about the same magnitude as the postive ones without PREEMPTION. PREEMPTION gives some other negative benefits for makeworld, but the others are smaller, at least without any userland idle priority threads that want to run all the time. The system for the above tests has a fairly large write bandwidtth (5GB/sec for movnt*) so it benefits from idle-time page prezeroing less than most systems. I've seen it taking and saving 3% of the time for makeworld (60 seconds out of 1800) on UP systems with similar CPU speeds but slower memory and pagezero() not optimized to use movnt*. UP systems benefit less than SMP ones since they have a lower percentage of idle time. Of course the possible savings are less if the system is less often idle, but makeworld -j4 on an SMP system leaves a lot of time idle, especially when it runs mkdep and perl serially. I don't know if pgzero is running mainly in bursts in the time left idle by mkdep in the above, but guess not since it limits itself to not zeroing very many pages to avoid thrashing caches. Perhaps it should not limit itself so much when the zeroing is nontemporal. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 03:26:41 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 45B8F16A416 for ; Thu, 23 Nov 2006 03:26:41 +0000 (UTC) (envelope-from gordon@tetlows.org) Received: from spiff.melthusia.org (spiff.melthusia.org [207.178.198.141]) by mx1.FreeBSD.org (Postfix) with ESMTP id 85D5E43D49 for ; Thu, 23 Nov 2006 03:26:08 +0000 (GMT) (envelope-from gordon@tetlows.org) Received: from [10.0.0.24] ([64.241.37.140]) (authenticated bits=0) by spiff.melthusia.org (8.13.6/8.13.6) with ESMTP id kAN3Q3aW071030 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 22 Nov 2006 19:26:05 -0800 (PST) (envelope-from gordon@tetlows.org) In-Reply-To: <20061123020747.GZ2260@obelix.dsto.defence.gov.au> References: <45649E42.70409@cs.rice.edu> <20061123020747.GZ2260@obelix.dsto.defence.gov.au> Mime-Version: 1.0 (Apple Message framework v752.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <17DE7E25-BCEE-46C7-9EB2-73A9D5C37CB1@tetlows.org> Content-Transfer-Encoding: 7bit From: Gordon Tetlow Date: Wed, 22 Nov 2006 19:25:55 -0800 To: "Wilkinson, Alex" X-Mailer: Apple Mail (2.752.3) Cc: freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 03:26:41 -0000 On Nov 22, 2006, at 6:07 PM, Wilkinson, Alex wrote: > 0n Wed, Nov 22, 2006 at 01:00:18PM -0600, Alan Cox wrote: > >> Kip Macy wrote: >> >>> Do you have any thoughts on when superpage support might go into - >>> CURRENT? > > erm, what is superpage ? http://www.cs.rice.edu/~jnavarro/superpages/ -gordon From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 04:06:02 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 24B1016A403; Thu, 23 Nov 2006 04:06:01 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-arch@freebsd.org Date: Thu, 23 Nov 2006 12:05:55 +0800 User-Agent: KMail/1.8.2 References: <45649E42.70409@cs.rice.edu> <20061123124725.P35210@delplex.bde.org> In-Reply-To: <20061123124725.P35210@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231205.55629.davidxu@freebsd.org> Cc: Kip Macy , Alan Cox , arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 04:06:02 -0000 On Thursday 23 November 2006 11:25, Bruce Evans wrote: > On Wed, 22 Nov 2006, Alan Cox wrote: > > There is only one caveat. Idle-time page prezeroing is not supported. > > However, ever since the VM system emerged from the Giant kernel lock, > > I've seen little or no benefit from it. ... > > That's probably because PREEMPTION is broken and the brokenness turns > idle-time page prezeroing into a pessimization. Without PREEMPTION I > see much the same benefits from idle-time page prezeroing as in RELENG_4 > -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > of a RELENG_4 with a -current i386 SMP kernel and a ~5.2 userland on > a Turion X2 2GHz: > > %%% > vm.idlezero_enable=0: > -------------------------------------------------------------- > > >>> elf make world completed on Thu Nov 23 12:01:56 EST 2006 > > (started Thu Nov 23 11:50:59 EST 2006) > -------------------------------------------------------------- > 656.36 real 815.59 user 194.92 sys > 23572 maximum resident set size > 1164 average shared memory size > 1212 average unshared data size > 128 average unshared stack size > 14202993 page reclaims > 6911 page faults > 0 swaps > 14686 block input operations > 4647 block output operations > 77645 messages sent > 0 messages received > 35459 signals received > 838638 voluntary context switches > 391631 involuntary context switches > > vm.idlezero_enable=1: > -------------------------------------------------------------- > > >>> elf make world completed on Thu Nov 23 12:35:54 EST 2006 > > (started Thu Nov 23 12:25:07 EST 2006) > -------------------------------------------------------------- > 647.19 real 814.16 user 185.69 sys > 23572 maximum resident set size > 1168 average shared memory size > 1220 average unshared data size > 128 average unshared stack size > 14202807 page reclaims > 6958 page faults > 0 swaps > 14534 block input operations > 4689 block output operations > 77466 messages sent > 0 messages received > 35456 signals received > 847575 voluntary context switches > 397783 involuntary context switches > %%% > > With idlezero enabled and PREEMPTION not enabled in the above, pgzero > runs in actual idle time for 14 seconds and reduces both the real and > sys times by 9 seconds (1.5% of real time and 5% of system time). > > With idlezero enabled and PREEMPTION enabled (details not shown), > PREEMPTION doesn't actually work but pgzero depends on it working, so > pgzero runs for much longer than in the above, with all the extra time > stolen from non-idle time. In my makeworld benchmarks, this gives > total benefits that are negative and about the same magnitude as the > postive ones without PREEMPTION. PREEMPTION gives some other negative > benefits for makeworld, but the others are smaller, at least without > any userland idle priority threads that want to run all the time. > > The system for the above tests has a fairly large write bandwidtth > (5GB/sec for movnt*) so it benefits from idle-time page prezeroing > less than most systems. I've seen it taking and saving 3% of the time > for makeworld (60 seconds out of 1800) on UP systems with similar CPU > speeds but slower memory and pagezero() not optimized to use movnt*. > UP systems benefit less than SMP ones since they have a lower percentage > of idle time. > > Of course the possible savings are less if the system is less often > idle, but makeworld -j4 on an SMP system leaves a lot of time idle, > especially when it runs mkdep and perl serially. I don't know if > pgzero is running mainly in bursts in the time left idle by mkdep in > the above, but guess not since it limits itself to not zeroing very > many pages to avoid thrashing caches. Perhaps it should not limit > itself so much when the zeroing is nontemporal. > > Bruce > _______________________________________________ I think on SMP, the BSD scheduler code does not preempt an idle thread in remote cpu, this might explain the reason why page_zero uses much time on SMP. the maybe_preempt() only works for current cpu. David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 04:06:02 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 24B1016A403; Thu, 23 Nov 2006 04:06:01 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-arch@freebsd.org Date: Thu, 23 Nov 2006 12:05:55 +0800 User-Agent: KMail/1.8.2 References: <45649E42.70409@cs.rice.edu> <20061123124725.P35210@delplex.bde.org> In-Reply-To: <20061123124725.P35210@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231205.55629.davidxu@freebsd.org> Cc: Kip Macy , Alan Cox , arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 04:06:02 -0000 On Thursday 23 November 2006 11:25, Bruce Evans wrote: > On Wed, 22 Nov 2006, Alan Cox wrote: > > There is only one caveat. Idle-time page prezeroing is not supported. > > However, ever since the VM system emerged from the Giant kernel lock, > > I've seen little or no benefit from it. ... > > That's probably because PREEMPTION is broken and the brokenness turns > idle-time page prezeroing into a pessimization. Without PREEMPTION I > see much the same benefits from idle-time page prezeroing as in RELENG_4 > -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > of a RELENG_4 with a -current i386 SMP kernel and a ~5.2 userland on > a Turion X2 2GHz: > > %%% > vm.idlezero_enable=0: > -------------------------------------------------------------- > > >>> elf make world completed on Thu Nov 23 12:01:56 EST 2006 > > (started Thu Nov 23 11:50:59 EST 2006) > -------------------------------------------------------------- > 656.36 real 815.59 user 194.92 sys > 23572 maximum resident set size > 1164 average shared memory size > 1212 average unshared data size > 128 average unshared stack size > 14202993 page reclaims > 6911 page faults > 0 swaps > 14686 block input operations > 4647 block output operations > 77645 messages sent > 0 messages received > 35459 signals received > 838638 voluntary context switches > 391631 involuntary context switches > > vm.idlezero_enable=1: > -------------------------------------------------------------- > > >>> elf make world completed on Thu Nov 23 12:35:54 EST 2006 > > (started Thu Nov 23 12:25:07 EST 2006) > -------------------------------------------------------------- > 647.19 real 814.16 user 185.69 sys > 23572 maximum resident set size > 1168 average shared memory size > 1220 average unshared data size > 128 average unshared stack size > 14202807 page reclaims > 6958 page faults > 0 swaps > 14534 block input operations > 4689 block output operations > 77466 messages sent > 0 messages received > 35456 signals received > 847575 voluntary context switches > 397783 involuntary context switches > %%% > > With idlezero enabled and PREEMPTION not enabled in the above, pgzero > runs in actual idle time for 14 seconds and reduces both the real and > sys times by 9 seconds (1.5% of real time and 5% of system time). > > With idlezero enabled and PREEMPTION enabled (details not shown), > PREEMPTION doesn't actually work but pgzero depends on it working, so > pgzero runs for much longer than in the above, with all the extra time > stolen from non-idle time. In my makeworld benchmarks, this gives > total benefits that are negative and about the same magnitude as the > postive ones without PREEMPTION. PREEMPTION gives some other negative > benefits for makeworld, but the others are smaller, at least without > any userland idle priority threads that want to run all the time. > > The system for the above tests has a fairly large write bandwidtth > (5GB/sec for movnt*) so it benefits from idle-time page prezeroing > less than most systems. I've seen it taking and saving 3% of the time > for makeworld (60 seconds out of 1800) on UP systems with similar CPU > speeds but slower memory and pagezero() not optimized to use movnt*. > UP systems benefit less than SMP ones since they have a lower percentage > of idle time. > > Of course the possible savings are less if the system is less often > idle, but makeworld -j4 on an SMP system leaves a lot of time idle, > especially when it runs mkdep and perl serially. I don't know if > pgzero is running mainly in bursts in the time left idle by mkdep in > the above, but guess not since it limits itself to not zeroing very > many pages to avoid thrashing caches. Perhaps it should not limit > itself so much when the zeroing is nontemporal. > > Bruce > _______________________________________________ I think on SMP, the BSD scheduler code does not preempt an idle thread in remote cpu, this might explain the reason why page_zero uses much time on SMP. the maybe_preempt() only works for current cpu. David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 06:28:29 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E0A2D16A4A0; Thu, 23 Nov 2006 06:28:29 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1497F43D55; Thu, 23 Nov 2006 06:27:56 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id 03DFF119229; Thu, 23 Nov 2006 17:28:27 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 7C27F27416; Thu, 23 Nov 2006 17:28:26 +1100 (EST) Date: Thu, 23 Nov 2006 17:28:25 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: David Xu In-Reply-To: <200611231205.55629.davidxu@freebsd.org> Message-ID: <20061123170941.T36132@delplex.bde.org> References: <45649E42.70409@cs.rice.edu> <20061123124725.P35210@delplex.bde.org> <200611231205.55629.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 06:28:30 -0000 On Thu, 23 Nov 2006, David Xu wrote: > On Thursday 23 November 2006 11:25, Bruce Evans wrote: >> On Wed, 22 Nov 2006, Alan Cox wrote: >>> There is only one caveat. Idle-time page prezeroing is not supported. >>> However, ever since the VM system emerged from the Giant kernel lock, >>> I've seen little or no benefit from it. ... >> >> That's probably because PREEMPTION is broken and the brokenness turns >> idle-time page prezeroing into a pessimization. Without PREEMPTION I >> see much the same benefits from idle-time page prezeroing as in RELENG_4 >> -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > > I think on SMP, the BSD scheduler code does not preempt an idle > thread in remote cpu, this might explain the reason why page_zero > uses much time on SMP. the maybe_preempt() only works > for current cpu. See my previous mail to freebsd-arch about this. IPI_PREEMPTION should result in preempting pgzero, but doesn't seem to, and if it did then you would wish it wouldn't since it has larger overheads than voluntarility giving up control. For user idle threads, there is nothing better than sending an IPI, but pgzero does a good job of limiting itself so preempting it just pessimizes it. Even non-IPI preemption will sometimes have higher overheads. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 06:28:29 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E0A2D16A4A0; Thu, 23 Nov 2006 06:28:29 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1497F43D55; Thu, 23 Nov 2006 06:27:56 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id 03DFF119229; Thu, 23 Nov 2006 17:28:27 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 7C27F27416; Thu, 23 Nov 2006 17:28:26 +1100 (EST) Date: Thu, 23 Nov 2006 17:28:25 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: David Xu In-Reply-To: <200611231205.55629.davidxu@freebsd.org> Message-ID: <20061123170941.T36132@delplex.bde.org> References: <45649E42.70409@cs.rice.edu> <20061123124725.P35210@delplex.bde.org> <200611231205.55629.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 06:28:30 -0000 On Thu, 23 Nov 2006, David Xu wrote: > On Thursday 23 November 2006 11:25, Bruce Evans wrote: >> On Wed, 22 Nov 2006, Alan Cox wrote: >>> There is only one caveat. Idle-time page prezeroing is not supported. >>> However, ever since the VM system emerged from the Giant kernel lock, >>> I've seen little or no benefit from it. ... >> >> That's probably because PREEMPTION is broken and the brokenness turns >> idle-time page prezeroing into a pessimization. Without PREEMPTION I >> see much the same benefits from idle-time page prezeroing as in RELENG_4 >> -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > > I think on SMP, the BSD scheduler code does not preempt an idle > thread in remote cpu, this might explain the reason why page_zero > uses much time on SMP. the maybe_preempt() only works > for current cpu. See my previous mail to freebsd-arch about this. IPI_PREEMPTION should result in preempting pgzero, but doesn't seem to, and if it did then you would wish it wouldn't since it has larger overheads than voluntarility giving up control. For user idle threads, there is nothing better than sending an IPI, but pgzero does a good job of limiting itself so preempting it just pessimizes it. Even non-IPI preemption will sometimes have higher overheads. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 06:53:40 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 7BE8516A403; Thu, 23 Nov 2006 06:53:40 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: Bruce Evans Date: Thu, 23 Nov 2006 14:53:33 +0800 User-Agent: KMail/1.8.2 References: <200611231205.55629.davidxu@freebsd.org> <20061123170941.T36132@delplex.bde.org> In-Reply-To: <20061123170941.T36132@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231453.34287.davidxu@freebsd.org> Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 06:53:41 -0000 On Thursday 23 November 2006 14:28, Bruce Evans wrote: > On Thu, 23 Nov 2006, David Xu wrote: > > On Thursday 23 November 2006 11:25, Bruce Evans wrote: > >> On Wed, 22 Nov 2006, Alan Cox wrote: > >>> There is only one caveat. Idle-time page prezeroing is not supported. > >>> However, ever since the VM system emerged from the Giant kernel lock, > >>> I've seen little or no benefit from it. ... > >> > >> That's probably because PREEMPTION is broken and the brokenness turns > >> idle-time page prezeroing into a pessimization. Without PREEMPTION I > >> see much the same benefits from idle-time page prezeroing as in RELENG_4 > >> -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > > > > I think on SMP, the BSD scheduler code does not preempt an idle > > thread in remote cpu, this might explain the reason why page_zero > > uses much time on SMP. the maybe_preempt() only works > > for current cpu. > > See my previous mail to freebsd-arch about this. IPI_PREEMPTION should > result in preempting pgzero, but doesn't seem to, and if it did then > you would wish it wouldn't since it has larger overheads than voluntarility > giving up control. For user idle threads, there is nothing better > than sending an IPI, but pgzero does a good job of limiting itself so > preempting it just pessimizes it. Even non-IPI preemption will sometimes > have higher overheads. > > Bruce Do you really believe IPI_PREEMPTION works as expected ? the problem is in sched_4bsd.c, the remote cpu running an idle cpu is only interrupted by IPI_AST not IPI_PREEMPT, IPI_AST has no effect for kernel thread, note that kick_other_cpu is only used for bound thread which is being resumed, kick_other_cpu does send IPI_PREEMPT, though pagezero thread is a bound thread in most time, but it is not the case that kick_other_cpu will be used, we want to preempt it not resuem it, so PREEMPTION and IPI_PREEPMTION do not work for pagezero thread running on remote cpu. David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 06:53:40 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 7BE8516A403; Thu, 23 Nov 2006 06:53:40 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: Bruce Evans Date: Thu, 23 Nov 2006 14:53:33 +0800 User-Agent: KMail/1.8.2 References: <200611231205.55629.davidxu@freebsd.org> <20061123170941.T36132@delplex.bde.org> In-Reply-To: <20061123170941.T36132@delplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231453.34287.davidxu@freebsd.org> Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 06:53:41 -0000 On Thursday 23 November 2006 14:28, Bruce Evans wrote: > On Thu, 23 Nov 2006, David Xu wrote: > > On Thursday 23 November 2006 11:25, Bruce Evans wrote: > >> On Wed, 22 Nov 2006, Alan Cox wrote: > >>> There is only one caveat. Idle-time page prezeroing is not supported. > >>> However, ever since the VM system emerged from the Giant kernel lock, > >>> I've seen little or no benefit from it. ... > >> > >> That's probably because PREEMPTION is broken and the brokenness turns > >> idle-time page prezeroing into a pessimization. Without PREEMPTION I > >> see much the same benefits from idle-time page prezeroing as in RELENG_4 > >> -- a speedup of a few percent for makeworld. E.g., for makeworld -j4 > > > > I think on SMP, the BSD scheduler code does not preempt an idle > > thread in remote cpu, this might explain the reason why page_zero > > uses much time on SMP. the maybe_preempt() only works > > for current cpu. > > See my previous mail to freebsd-arch about this. IPI_PREEMPTION should > result in preempting pgzero, but doesn't seem to, and if it did then > you would wish it wouldn't since it has larger overheads than voluntarility > giving up control. For user idle threads, there is nothing better > than sending an IPI, but pgzero does a good job of limiting itself so > preempting it just pessimizes it. Even non-IPI preemption will sometimes > have higher overheads. > > Bruce Do you really believe IPI_PREEMPTION works as expected ? the problem is in sched_4bsd.c, the remote cpu running an idle cpu is only interrupted by IPI_AST not IPI_PREEMPT, IPI_AST has no effect for kernel thread, note that kick_other_cpu is only used for bound thread which is being resumed, kick_other_cpu does send IPI_PREEMPT, though pagezero thread is a bound thread in most time, but it is not the case that kick_other_cpu will be used, we want to preempt it not resuem it, so PREEMPTION and IPI_PREEPMTION do not work for pagezero thread running on remote cpu. David Xu From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 07:09:16 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id AEC1816A4C8; Thu, 23 Nov 2006 07:09:14 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: Bruce Evans Date: Thu, 23 Nov 2006 15:09:08 +0800 User-Agent: KMail/1.8.2 References: <20061123170941.T36132@delplex.bde.org> <200611231453.34287.davidxu@freebsd.org> In-Reply-To: <200611231453.34287.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231509.08742.davidxu@freebsd.org> Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 07:09:16 -0000 On Thursday 23 November 2006 14:53, David Xu wrote: > Do you really believe IPI_PREEMPTION works as expected ? the problem > is in sched_4bsd.c, the remote cpu running an idle cpu is only > interrupted by IPI_AST not IPI_PREEMPT, IPI_AST has no effect for kernel > thread, note that kick_other_cpu is only used for bound thread which is > being resumed, kick_other_cpu does send IPI_PREEMPT, though pagezero > thread is a bound thread in most time, but it is not the case that > kick_other_cpu will be used, we want to preempt it not resuem it, so > PREEMPTION and IPI_PREEPMTION do not work for pagezero thread running > on remote cpu. > > David Xu the above may not be very clear. The forward_wakeup only selects idle cpu, so a thread is being added will not preempt remote cpu which is running pagezero thread. From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 07:09:16 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id AEC1816A4C8; Thu, 23 Nov 2006 07:09:14 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: Bruce Evans Date: Thu, 23 Nov 2006 15:09:08 +0800 User-Agent: KMail/1.8.2 References: <20061123170941.T36132@delplex.bde.org> <200611231453.34287.davidxu@freebsd.org> In-Reply-To: <200611231453.34287.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200611231509.08742.davidxu@freebsd.org> Cc: arch@freebsd.org, Kip Macy , Alan Cox , freebsd-arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 07:09:16 -0000 On Thursday 23 November 2006 14:53, David Xu wrote: > Do you really believe IPI_PREEMPTION works as expected ? the problem > is in sched_4bsd.c, the remote cpu running an idle cpu is only > interrupted by IPI_AST not IPI_PREEMPT, IPI_AST has no effect for kernel > thread, note that kick_other_cpu is only used for bound thread which is > being resumed, kick_other_cpu does send IPI_PREEMPT, though pagezero > thread is a bound thread in most time, but it is not the case that > kick_other_cpu will be used, we want to preempt it not resuem it, so > PREEMPTION and IPI_PREEPMTION do not work for pagezero thread running > on remote cpu. > > David Xu the above may not be very clear. The forward_wakeup only selects idle cpu, so a thread is being added will not preempt remote cpu which is running pagezero thread. From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 15:57:31 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 66D3416A4AB for ; Thu, 23 Nov 2006 15:57:31 +0000 (UTC) (envelope-from freebsd-arch@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2867343D6A for ; Thu, 23 Nov 2006 15:56:54 +0000 (GMT) (envelope-from freebsd-arch@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1GnGx2-00052F-Q8 for freebsd-arch@freebsd.org; Thu, 23 Nov 2006 16:57:25 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 23 Nov 2006 16:57:24 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 23 Nov 2006 16:57:24 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-arch@freebsd.org From: Ivan Voras Date: Thu, 23 Nov 2006 16:56:55 +0100 Lines: 18 Message-ID: References: <20061119041421.I16763@delplex.bde.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 1.5.0.4 (X11/20060625) In-Reply-To: Sender: news Subject: Re: What is the PREEMPTION option good for? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 15:57:31 -0000 Ivan Voras wrote: > Bruce Evans wrote: > >> Most of the difference is caused by pgzero becoming too active with >> PREEMPTION. > > Don't know about the other things but I've noticed pagezero is > suspiciously active on heavy loaded SMP web servers (even complained on > @stable a long time ago). I'll try disabling PREEMPTION and see how it goes. Ok, I couldn't run extensive tests because people were waiting to use the machine, so this should be considered anecdotal evidence. On a simple benchmark that repeatedly (for 1 minute) and concurrently (target=50 concurrent requests) hits a dynamic web page on a development machine (2 proc true SMP), the performance goes up from ~85 requests/sec. to ~105 requests/s by disabling PREEMPTION. This improvement looks suspiciously high to me, but I don't think I'll be going back :) pagezero is now not noticable in 'top' output. From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 19:03:27 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CE27116A416 for ; Thu, 23 Nov 2006 19:03:27 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6605443D55 for ; Thu, 23 Nov 2006 19:02:49 +0000 (GMT) (envelope-from uspoerlein@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so919948nfc for ; Thu, 23 Nov 2006 11:03:25 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:date:from:to:cc:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=J2LX+3IZyXW+PZRjoS5zH1aq5wuZj9ElkdxF7wjja9/QNIpfAMI3UQh//Yg+1Y/lPH0qc1mLnGWdM7sm27KIXgzqjFLjzei4R7S0lasrWNHQSu83tDVyEbPfhmApZDps+REAAPv065tpB9x+rUWaiuoMqO/HZoyQHdOK3+gnv6k= Received: by 10.49.43.2 with SMTP id v2mr4414557nfj.1164308605026; Thu, 23 Nov 2006 11:03:25 -0800 (PST) Received: from roadrunner.q.local ( [85.180.138.209]) by mx.google.com with ESMTP id r34sm5851693nfc.2006.11.23.11.03.24; Thu, 23 Nov 2006 11:03:24 -0800 (PST) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.8/8.13.8) with ESMTP id kANJ3Lo7009739; Thu, 23 Nov 2006 20:03:21 +0100 (CET) (envelope-from uspoerlein@gmail.com) Received: (from q@localhost) by roadrunner.q.local (8.13.8/8.13.8/Submit) id kAMKGPjs003604; Wed, 22 Nov 2006 21:16:25 +0100 (CET) (envelope-from uspoerlein@gmail.com) Date: Wed, 22 Nov 2006 21:16:25 +0100 From: Ulrich Spoerlein To: Alan Cox Message-ID: <20061122201625.GC1522@roadrunner.q.local> Mail-Followup-To: Alan Cox , Kip Macy , arch@freebsd.org References: <45649E42.70409@cs.rice.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45649E42.70409@cs.rice.edu> Cc: Kip Macy , arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 19:03:27 -0000 Alan Cox wrote: > Lastly, page coloring dies. However, the beneficial effects of page coloring are for the > most part captured by superpages. Specifically, regardless of whether the pmap is able to > promote a particular region of the address space to a superpage, e.g., due to heterogenous > access rights of pages within the region, if the memory has been provided by the reservation > system it will have "perfect" coloring. Hi Alan, what performance impact do you estimate for "older" processors? I know very little about superpages, so I assume that, e.g., earlier Pentiums don't support it? Where do you think the break off point lies? Ulrich Spoerlein -- A: Yes. >Q: Are you sure? > >A: Because it reverses the logical flow of conversation. > >>Q: Why is top posting frowned upon? From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 23:19:31 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18C2616A416 for ; Thu, 23 Nov 2006 23:19:31 +0000 (UTC) (envelope-from jb@what-creek.com) Received: from what-creek.com (what-creek.com [66.111.37.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id 640DF43D78 for ; Thu, 23 Nov 2006 23:18:42 +0000 (GMT) (envelope-from jb@what-creek.com) Received: by what-creek.com (Postfix, from userid 102) id 7FC8D140EC08; Thu, 23 Nov 2006 23:20:35 +0000 (GMT) Date: Thu, 23 Nov 2006 23:20:35 +0000 From: John Birrell To: arch@freebsd.org Message-ID: <20061123232035.GA56985@what-creek.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: Subject: Proposed change to make -j X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 23:19:31 -0000 Currently 'make -j' reports an error if the number of jobs isn't specified. I'd like to change make(1) to treat -j (without a number) as meaning "set the number of jobs to the number of processors". On sun4v, each processor isn't too powerful and system performance is only decent when you use all the processors - 32 in my case. I've been working on a parallel 'make release' process which would benefit from having -j set by default. At the moment I set MAKEFLAGS=j32 in my environment and this achieves the desired result, but -j would be more general. Thoughts? -- John Birrell From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 23:29:58 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B454416A416 for ; Thu, 23 Nov 2006 23:29:58 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from spunkymail-a9.dreamhost.com (sd-green-bigip-202.dreamhost.com [208.97.132.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE0EF43D46 for ; Thu, 23 Nov 2006 23:29:20 +0000 (GMT) (envelope-from rnsanchez@wait4.org) Received: from sauron.lan.box (unknown [200.180.165.63]) by spunkymail-a9.dreamhost.com (Postfix) with ESMTP id 3E90C20AEA; Thu, 23 Nov 2006 15:29:55 -0800 (PST) Date: Thu, 23 Nov 2006 21:29:52 -0200 From: Ricardo Nabinger Sanchez To: Ulrich Spoerlein Message-Id: <20061123212952.f332a7ad.rnsanchez@wait4.org> In-Reply-To: <20061122201625.GC1522@roadrunner.q.local> References: <45649E42.70409@cs.rice.edu> <20061122201625.GC1522@roadrunner.q.local> Organization: SYS_WAIT4 X-Mailer: Sylpheed version 2.3.0beta2 (GTK+ 2.10.6; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Kip Macy , Alan Cox , arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 23:29:58 -0000 On Wed, 22 Nov 2006 21:16:25 +0100 Ulrich Spoerlein wrote: > what performance impact do you estimate for "older" processors? I know > very little about superpages, so I assume that, e.g., earlier Pentiums > don't support it? Where do you think the break off point lies? Please correct me if I'm wrong, but if superpages means using 4 MB TLB pages, then it is quite old (~1993) and every pentium (i586 or newer) should support it. -- Ricardo Nabinger Sanchez Powered by FreeBSD "Left to themselves, things tend to go from bad to worse." From owner-freebsd-arch@FreeBSD.ORG Thu Nov 23 23:48:39 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3CF9816A417 for ; Thu, 23 Nov 2006 23:48:39 +0000 (UTC) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (mail.cs.rice.edu [128.42.1.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 18FB343D67 for ; Thu, 23 Nov 2006 23:48:00 +0000 (GMT) (envelope-from alc@cs.rice.edu) Received: from mail.cs.rice.edu (localhost.localdomain [127.0.0.1]) by mail.cs.rice.edu (Postfix) with ESMTP id AAB8F2C2AD2; Thu, 23 Nov 2006 17:48:37 -0600 (CST) X-Virus-Scanned: by amavis-2.4.0 at mail.cs.rice.edu Received: from mail.cs.rice.edu ([127.0.0.1]) by mail.cs.rice.edu (mail.cs.rice.edu [127.0.0.1]) (amavisd-new, port 10024) with LMTP id eRXYQ10fQAy8; Thu, 23 Nov 2006 17:48:37 -0600 (CST) Received: from [216.63.78.18] (adsl-216-63-78-18.dsl.hstntx.swbell.net [216.63.78.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.cs.rice.edu (Postfix) with ESMTP id EFC8D2C2AD1; Thu, 23 Nov 2006 17:48:36 -0600 (CST) Message-ID: <45663353.1040803@cs.rice.edu> Date: Thu, 23 Nov 2006 17:48:35 -0600 From: Alan Cox User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.10) Gecko/20050817 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ricardo Nabinger Sanchez References: <45649E42.70409@cs.rice.edu> <20061122201625.GC1522@roadrunner.q.local> <20061123212952.f332a7ad.rnsanchez@wait4.org> In-Reply-To: <20061123212952.f332a7ad.rnsanchez@wait4.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: Kip Macy , Ulrich Spoerlein , arch@freebsd.org Subject: Re: superpage plans X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 23:48:39 -0000 Ricardo Nabinger Sanchez wrote: >On Wed, 22 Nov 2006 21:16:25 +0100 >Ulrich Spoerlein wrote: > > > >>what performance impact do you estimate for "older" processors? I know >>very little about superpages, so I assume that, e.g., earlier Pentiums >>don't support it? Where do you think the break off point lies? >> >> > >Please correct me if I'm wrong, but if superpages means using 4 MB TLB pages, >then it is quite old (~1993) and every pentium (i586 or newer) should support >it. > > > Yes, you are correct. Regards, Alan From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 00:07:18 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B595116A417 for ; Fri, 24 Nov 2006 00:07:18 +0000 (UTC) (envelope-from infofarmer@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.186]) by mx1.FreeBSD.org (Postfix) with ESMTP id C451243D68 for ; Fri, 24 Nov 2006 00:06:39 +0000 (GMT) (envelope-from infofarmer@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so990114nfc for ; Thu, 23 Nov 2006 16:07:16 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=kszQ83hn1Dzmn8T7Qtt6urjkJiGfyY35RA+MR0WQuFpCniBP4MMsdHL5Fd0VJnmBqmZRLXaQIIClVcm0FVr6dmi2/kYwFCYFT3R+GfKBpZUm159a4PA3p6mrvERA6e39+Jg9evXDFBbTTYyZoBMY4herX/PodI1b9Ma2Jd+Tm0U= Received: by 10.78.69.7 with SMTP id r7mr9655033hua.1164326835771; Thu, 23 Nov 2006 16:07:15 -0800 (PST) Received: by 10.78.167.16 with HTTP; Thu, 23 Nov 2006 16:07:15 -0800 (PST) Message-ID: Date: Fri, 24 Nov 2006 03:07:15 +0300 From: "Andrew Pantyukhin" Sender: infofarmer@gmail.com To: "John Birrell" In-Reply-To: <20061123232035.GA56985@what-creek.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20061123232035.GA56985@what-creek.com> X-Google-Sender-Auth: 379c6526b9393844 Cc: arch@freebsd.org Subject: Re: Proposed change to make -j X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 00:07:18 -0000 On 11/24/06, John Birrell wrote: > Currently 'make -j' reports an error if the number of jobs > isn't specified. > > I'd like to change make(1) to treat -j (without a number) as > meaning "set the number of jobs to the number of processors". > > On sun4v, each processor isn't too powerful and system performance > is only decent when you use all the processors - 32 in my case. > > I've been working on a parallel 'make release' process which > would benefit from having -j set by default. At the moment I > set MAKEFLAGS=j32 in my environment and this achieves the desired > result, but -j would be more general. > > Thoughts? Sounds very nice. It wouldn't make much difference to me, but (number of cpu's + 1) seems like a good value, too. From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 00:10:07 2006 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7FC2416A417 for ; Fri, 24 Nov 2006 00:10:07 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0FEF243D4C for ; Fri, 24 Nov 2006 00:09:29 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id 8BF7864D060; Fri, 24 Nov 2006 11:10:03 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id 54DC627417; Fri, 24 Nov 2006 11:10:02 +1100 (EST) Date: Fri, 24 Nov 2006 11:10:00 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Ivan Voras In-Reply-To: Message-ID: <20061124104830.B39451@delplex.bde.org> References: <20061119041421.I16763@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-arch@FreeBSD.org Subject: Re: What is the PREEMPTION option good for? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 00:10:07 -0000 On Thu, 23 Nov 2006, Ivan Voras wrote: > Ivan Voras wrote: >> Bruce Evans wrote: >> >>> Most of the difference is caused by pgzero becoming too active with >>> PREEMPTION. >> >> Don't know about the other things but I've noticed pagezero is >> suspiciously active on heavy loaded SMP web servers (even complained on >> @stable a long time ago). I'll try disabling PREEMPTION and see how it goes. > > Ok, I couldn't run extensive tests because people were waiting to use > the machine, so this should be considered anecdotal evidence. On a > simple benchmark that repeatedly (for 1 minute) and concurrently > (target=50 concurrent requests) hits a dynamic web page on a development > machine (2 proc true SMP), the performance goes up from ~85 > requests/sec. to ~105 requests/s by disabling PREEMPTION. This > improvement looks suspiciously high to me, but I don't think I'll be > going back :) pagezero is now not noticable in 'top' output. I (or someone) will make pgzero not depend on PREEMPTION. I found what PREEMPTION is good for -- it is to not completely lose to RELENG_4 for interrupt latency. Without PREEMPTION, hitting caps lock results in the keyboard interrupt handler busy-waiting for 3-5 msec while programming the LED. The keyboard driver has always done this stupidly, but it doesn't matter much if interrupt priorities work right. Without PREEMPTION, interrupt priorities don't work right -- higher priority interrupt handlers don't run while the very low priority keyboard interrupt handler is running. This results in several thousand packets per second being dropped in my network blast tests. I'm testing bge mainly on a !SMP system with an rx load of 240kpps. My bge has an rx ring size of 1024, so drops shouldn't occur unless the latency is > 4mS, but it is apparently much higher than 3-5mS since I lose 1-3k packets when I hit caps lock. PREEMPTION is less needed for SMP systems since it takes something like 2 kernel threads busy-waiting for too long at the same time to give very large interrupt latencies. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 00:30:03 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ECFA916A407 for ; Fri, 24 Nov 2006 00:30:03 +0000 (UTC) (envelope-from jasone@FreeBSD.org) Received: from canonware.com (24-38-119-150-st.losaca.adelphia.net [24.38.119.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id B885243D49 for ; Fri, 24 Nov 2006 00:29:25 +0000 (GMT) (envelope-from jasone@FreeBSD.org) Received: from [192.168.168.201] (24-38-119-150-st.losaca.adelphia.net [24.38.119.150]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by canonware.com (Postfix) with ESMTP id DA8971298D3; Thu, 23 Nov 2006 16:30:27 -0800 (PST) Message-ID: <45663D0A.7010508@FreeBSD.org> Date: Thu, 23 Nov 2006 16:30:02 -0800 From: Jason Evans User-Agent: Thunderbird 1.5.0.8 (X11/20061105) MIME-Version: 1.0 To: John Birrell References: <20061123232035.GA56985@what-creek.com> In-Reply-To: <20061123232035.GA56985@what-creek.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: arch@freebsd.org Subject: Re: Proposed change to make -j X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 00:30:04 -0000 John Birrell wrote: > Currently 'make -j' reports an error if the number of jobs > isn't specified. > > I'd like to change make(1) to treat -j (without a number) as > meaning "set the number of jobs to the number of processors". > > On sun4v, each processor isn't too powerful and system performance > is only decent when you use all the processors - 32 in my case. > > I've been working on a parallel 'make release' process which > would benefit from having -j set by default. At the moment I > set MAKEFLAGS=j32 in my environment and this achieves the desired > result, but -j would be more general. For gmake, a bare -j says to not limit the number of simultaneous jobs. I don't think that it's a good idea to use a flag name that already has a different (and potentially useful) meaning for gmake. I have some concern that no matter what number of jobs you decide to run simultaneously when your proposed flag is specified (N, N+1, 2N, etc., where N is the number of CPUs), that choice will not be universally useful, since the best choice of argument to -j depends on what 'make' is building. Might it be better to use sysctl(8) in your command line? make -j `sysctl -n hw.ncpu` Jason From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 07:03:52 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F3D1516A407; Fri, 24 Nov 2006 07:03:51 +0000 (UTC) (envelope-from rink@rink.nu) Received: from mx0.rink.nu (thunderstone.rink.nu [80.112.228.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id D732643D49; Fri, 24 Nov 2006 07:03:09 +0000 (GMT) (envelope-from rink@rink.nu) Received: from localhost (localhost [127.0.0.1]) by mx0.rink.nu (Postfix) with ESMTP id A4B4117087; Fri, 24 Nov 2006 08:04:23 +0100 (CET) X-Virus-Scanned: amavisd-new at rink.nu Received: from mx0.rink.nu ([127.0.0.1]) by localhost (thunderstone.rink.nu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ofqgymHOyBGp; Fri, 24 Nov 2006 08:04:17 +0100 (CET) Received: by mx0.rink.nu (Postfix, from userid 1000) id 6337017083; Fri, 24 Nov 2006 08:04:17 +0100 (CET) Date: Fri, 24 Nov 2006 08:04:17 +0100 From: Rink Springer To: Jason Evans Message-ID: <20061124070417.GA63991@rink.nu> References: <20061123232035.GA56985@what-creek.com> <45663D0A.7010508@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=sha1; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline In-Reply-To: <45663D0A.7010508@FreeBSD.org> User-Agent: Mutt/1.5.13 (2006-08-11) Cc: arch@freebsd.org, John Birrell Subject: Re: Proposed change to make -j X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 07:03:52 -0000 --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 23, 2006 at 04:30:02PM -0800, Jason Evans wrote: > John Birrell wrote: > >Currently 'make -j' reports an error if the number of jobs > >isn't specified. > > > >I'd like to change make(1) to treat -j (without a number) as > >meaning "set the number of jobs to the number of processors". > > > >On sun4v, each processor isn't too powerful and system performance > >is only decent when you use all the processors - 32 in my case. > > > >I've been working on a parallel 'make release' process which > >would benefit from having -j set by default. At the moment I > >set MAKEFLAGS=3Dj32 in my environment and this achieves the desired > >result, but -j would be more general. >=20 > For gmake, a bare -j says to not limit the number of simultaneous jobs.= =20 > I don't think that it's a good idea to use a flag name that already=20 > has a different (and potentially useful) meaning for gmake. >=20 > I have some concern that no matter what number of jobs you decide to run= =20 > simultaneously when your proposed flag is specified (N, N+1, 2N, etc.,=20 > where N is the number of CPUs), that choice will not be universally=20 > useful, since the best choice of argument to -j depends on what 'make'=20 > is building. Might it be better to use sysctl(8) in your command line? >=20 > make -j `sysctl -n hw.ncpu` What about 'make -j0'? This current gives an error message... --=20 Rink P.W. Springer - http://rink.nu "It's you isn't it? THE BASTARD OPERATOR FROM HELL!" "In the flesh, on the phone and in your account..." - BOFH #3 --xHFwDpU9dbj6ez1V Content-Type: application/x-pkcs7-signature Content-Disposition: attachment; filename="smime.p7s" Content-Transfer-Encoding: base64 MIIJawYJKoZIhvcNAQcCoIIJXDCCCVgCAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3DQEHAaCC BuIwggObMIIDBKADAgECAhAiuN7bs9pg6t3I0n6G5OOTMA0GCSqGSIb3DQEBBQUAMGIxCzAJ BgNVBAYTAlpBMSUwIwYDVQQKExxUaGF3dGUgQ29uc3VsdGluZyAoUHR5KSBMdGQuMSwwKgYD VQQDEyNUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgSXNzdWluZyBDQTAeFw0wNjExMDgwOTI2 NTNaFw0wNzExMDgwOTI2NTNaMIHSMREwDwYDVQQEEwhTcHJpbmdlcjEaMBgGA1UEKhMRUmlu ayBQZXRlciBXeWNoZXIxIzAhBgNVBAMTGlJpbmsgUGV0ZXIgV3ljaGVyIFNwcmluZ2VyMRsw GQYJKoZIhvcNAQkBFgxtYWlsQHJpbmsubnUxHzAdBgkqhkiG9w0BCQEWEHJpbmtAZnJlZWJz ZC5vcmcxIDAeBgkqhkiG9w0BCQEWEXJpbmtAaWwuZm9udHlzLm5sMRwwGgYJKoZIhvcNAQkB Fg1yaW5rQHN0YWNrLm5sMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxryGDfel YzzENX7wodkbVY1NALfaiPfNEG10YjD8ZWdK9zkN26Tc878Shbqapq0KYFD8TACGfEhKoMvo qbf0PHAS/gNYr81Arqa9FRPUfzvtDE/cMbhvI+p7ufBITyYnPJp9MUD72iT+DohRR2ISVi3i NAEgDuSbYYNxctnvXqU6O6EPy3mzoFPDoiOQwBfVtFrjxBbND9BUK2bjtUyGt4x8I/Vulzrt qLPTokva+b97DHRgbCA/aLLYIrU6QoqOFJ8GrAbro/FZLYh4m1oJk3FEHVQOKkk7xzIaFmmP QGJRL8m6nrIZFTrQ+X2wmzfLD55K/UiqbekOuMiWbY9EbwIDAQABo10wWzBLBgNVHREERDBC gQxtYWlsQHJpbmsubnWBEHJpbmtAZnJlZWJzZC5vcmeBEXJpbmtAaWwuZm9udHlzLm5sgQ1y aW5rQHN0YWNrLm5sMAwGA1UdEwEB/wQCMAAwDQYJKoZIhvcNAQEFBQADgYEAIfIcieRjePBA wjZqvOdGpyPcNDnK/ubeQSTV5Y4AHWxm1sXhQxB/XrQ3RVdz1qDnBRL1AjkEBAl8e9+am4s6 D6TaSlmJeNXn6ZPJTQecisz3M+AKiMckShM3oAeUi0ktn1yNYR+hz5aQN612XT5OZRYznJVZ kPf1DiA2RVVyz+MwggM/MIICqKADAgECAgENMA0GCSqGSIb3DQEBBQUAMIHRMQswCQYDVQQG EwJaQTEVMBMGA1UECBMMV2VzdGVybiBDYXBlMRIwEAYDVQQHEwlDYXBlIFRvd24xGjAYBgNV BAoTEVRoYXd0ZSBDb25zdWx0aW5nMSgwJgYDVQQLEx9DZXJ0aWZpY2F0aW9uIFNlcnZpY2Vz IERpdmlzaW9uMSQwIgYDVQQDExtUaGF3dGUgUGVyc29uYWwgRnJlZW1haWwgQ0ExKzApBgkq hkiG9w0BCQEWHHBlcnNvbmFsLWZyZWVtYWlsQHRoYXd0ZS5jb20wHhcNMDMwNzE3MDAwMDAw WhcNMTMwNzE2MjM1OTU5WjBiMQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1 bHRpbmcgKFB0eSkgTHRkLjEsMCoGA1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElz c3VpbmcgQ0EwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBAMSmPFVzVftOucqZWh5owHUE cJ3f6f+jHuy9zfVb8hp2vX8MOmHyv1HOAdTlUAow1wJjWiyJFXCO3cnwK4Vaqj9xVsuvPAsH 5/EfkTYkKhPPK9Xzgnc9A74r/rsYPge/QIACZNenprufZdHFKlSFD0gEf6e20TxhBEAeZBly YLf7AgMBAAGjgZQwgZEwEgYDVR0TAQH/BAgwBgEB/wIBADBDBgNVHR8EPDA6MDigNqA0hjJo dHRwOi8vY3JsLnRoYXd0ZS5jb20vVGhhd3RlUGVyc29uYWxGcmVlbWFpbENBLmNybDALBgNV HQ8EBAMCAQYwKQYDVR0RBCIwIKQeMBwxGjAYBgNVBAMTEVByaXZhdGVMYWJlbDItMTM4MA0G CSqGSIb3DQEBBQUAA4GBAEiM0VCD6gsuzA2jZqxnD3+vrL7CF6FDlpSdf0whuPg2H6otnzYv wPQcUCCTcDz9reFhYsPZOhl+hLGZGwDFGguCdJ4lUJRix9sncVcljd2pnDmOjCBPZV+V2vf3 h9bGCE6u9uo05RAaWzVNd+NWIXiC3CEZNd4ksdMdRv9dX2VPMYICUTCCAk0CAQEwdjBiMQsw CQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkgTHRkLjEsMCoG A1UEAxMjVGhhd3RlIFBlcnNvbmFsIEZyZWVtYWlsIElzc3VpbmcgQ0ECECK43tuz2mDq3cjS fobk45MwCQYFKw4DAhoFAKCBsTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3 DQEJBTEPFw0wNjExMjQwNzA0MTdaMCMGCSqGSIb3DQEJBDEWBBSRajpbtxlqLtUTJ573Fdoy zcEswTBSBgkqhkiG9w0BCQ8xRTBDMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDANBgkqhkiG9w0BAQEFAASCAQCp qfM6SjDED4VnZV/VadgDa5bj9IKxBJP6Qx2lwNbGtaZup44sdoj9Q7ZKN4WgYUzFCRITqvMd 0l8FuEVhWd2D5NplwFh0mWtqTRlpfXvh0f3hYTUFTdg5DI9REX8qSsgiFvomVSrod3u4rEmT 1F+3IpSDJxI0NcoDTD7+TJQVs3EBRLQ8LpFuXGC0lFsKApuxYTi3LpjEtwfs7q4ufDpb9UvI laXn1pPQwcqO2iNDrv1tmIpgz4or/W3myjLvJwWLWO7wDXdtpB/KvIypa1+sN1KeAPDOdK3l qe64ek2TgKGeiVeXl5ye3h18MX77IUO05YL5Ji6kgGJzUdWF3jNH --xHFwDpU9dbj6ez1V-- From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 07:18:50 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C81C16A403; Fri, 24 Nov 2006 07:18:50 +0000 (UTC) (envelope-from ru@rambler-co.ru) Received: from relay0.rambler.ru (relay0.rambler.ru [81.19.66.187]) by mx1.FreeBSD.org (Postfix) with ESMTP id 037BB43D49; Fri, 24 Nov 2006 07:18:09 +0000 (GMT) (envelope-from ru@rambler-co.ru) Received: from relay0.rambler.ru (localhost [127.0.0.1]) by relay0.rambler.ru (Postfix) with ESMTP id C20665CED; Fri, 24 Nov 2006 10:18:48 +0300 (MSK) Received: from edoofus.park.rambler.ru (unknown [81.19.65.108]) by relay0.rambler.ru (Postfix) with ESMTP id A13F85CD8; Fri, 24 Nov 2006 10:18:48 +0300 (MSK) Received: (from ru@localhost) by edoofus.park.rambler.ru (8.13.8/8.13.8) id kAO7J1Iu038265; Fri, 24 Nov 2006 10:19:01 +0300 (MSK) (envelope-from ru) Date: Fri, 24 Nov 2006 10:19:01 +0300 From: Ruslan Ermilov To: Rink Springer Message-ID: <20061124071901.GA38235@rambler-co.ru> References: <20061123232035.GA56985@what-creek.com> <45663D0A.7010508@FreeBSD.org> <20061124070417.GA63991@rink.nu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline In-Reply-To: <20061124070417.GA63991@rink.nu> User-Agent: Mutt/1.5.13 (2006-08-11) X-Virus-Scanned: No virus found Cc: arch@freebsd.org, Jason Evans , John Birrell Subject: Re: Proposed change to make -j X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 07:18:50 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Nov 24, 2006 at 08:04:17AM +0100, Rink Springer wrote: > On Thu, Nov 23, 2006 at 04:30:02PM -0800, Jason Evans wrote: > > John Birrell wrote: > > >Currently 'make -j' reports an error if the number of jobs > > >isn't specified. > > > > > >I'd like to change make(1) to treat -j (without a number) as > > >meaning "set the number of jobs to the number of processors". > > > > > >On sun4v, each processor isn't too powerful and system performance > > >is only decent when you use all the processors - 32 in my case. > > > > > >I've been working on a parallel 'make release' process which > > >would benefit from having -j set by default. At the moment I > > >set MAKEFLAGS=3Dj32 in my environment and this achieves the desired > > >result, but -j would be more general. > >=20 > > For gmake, a bare -j says to not limit the number of simultaneous jobs.= =20 > > I don't think that it's a good idea to use a flag name that already=20 > > has a different (and potentially useful) meaning for gmake. > >=20 > > I have some concern that no matter what number of jobs you decide to ru= n=20 > > simultaneously when your proposed flag is specified (N, N+1, 2N, etc.,= =20 > > where N is the number of CPUs), that choice will not be universally=20 > > useful, since the best choice of argument to -j depends on what 'make'= =20 > > is building. Might it be better to use sysctl(8) in your command line? > >=20 > > make -j `sysctl -n hw.ncpu` >=20 > What about 'make -j0'? This current gives an error message... >=20 That would be an unportable extension which depends on the FreeBSD sysctl (hw.ncpu). What Jason suggested is just fine and UNIXish. Also, on most systems around here where I did testing (UP and SMP), I saw the imrpovement in wall clock time with -j going up to 2xNCPU, but on other systems I only saw it with -j up to NCPU, so there's no one true formula. It's a product of many factors, including properly written makefiles in a build set. Cheers, --=20 Ruslan Ermilov ru@FreeBSD.org FreeBSD committer --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFZpzlqRfpzJluFF4RAvMdAKCcrPwlR62A8kfk+faVO81pkIbOzACeNVHo 0GH0VTHSXo1//Dv5c7/fYew= =RyB7 -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 17:40:23 2006 Return-Path: X-Original-To: freebsd-arch@FreeBSD.ORG Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 984D916A403 for ; Fri, 24 Nov 2006 17:40:23 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6C2D243D5D for ; Fri, 24 Nov 2006 17:39:38 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (rytytm@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id kAOHeDni072174 for ; Fri, 24 Nov 2006 18:40:18 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id kAOHeDHB072173; Fri, 24 Nov 2006 18:40:13 +0100 (CET) (envelope-from olli) Date: Fri, 24 Nov 2006 18:40:13 +0100 (CET) Message-Id: <200611241740.kAOHeDHB072173@lurza.secnetix.de> From: Oliver Fromme To: freebsd-arch@FreeBSD.ORG X-Newsgroups: list.freebsd-arch User-Agent: tin/1.8.2-20060425 ("Shillay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 24 Nov 2006 18:40:19 +0100 (CET) Cc: Subject: New option for ls(1), du(1), df(1) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 17:40:23 -0000 Hi, I'm not sure if posting to this list is the correct way to proceed, but the title says "Architecture and design discussions", and I'm having a design issue, so here it goes ... The tools ls(1), du(1) and df(1) all have a "-h" option which means so-called "human-readable" output. However, personally I think the output isn't very human-readable either way. You often get a mixture of numbers with various combinations of "K", "M", "G", and it's difficult to recognize the dimensions quickly and compare them to each other. Just an example: Filesystem Size Used Avail Capacity Mounted on /dev/ad0s1a 989M 61M 849M 7% / /dev/ad0s1d 9.7G 31M 8.9G 0% /var /dev/ad0s1e 9.7G 1.5G 7.4G 17% /usr /dev/ad0s1f 122G 1.3G 111G 1% /home /dev/md0 290M 20K 267M 0% /tmp I've hacked a new option into those tools which uses thousands separators for displaying the numbers. With that option, the output looks like this (in US locale): Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad0s1a 1,012,974 62,418 869,520 7% / /dev/ad0s1d 10,154,158 31,474 9,310,352 0% /var /dev/ad0s1e 10,154,158 1,545,718 7,796,108 17% /usr /dev/ad0s1f 128,009,212 1,358,600 116,409,876 1% /home /dev/md0 297,326 20 273,520 0% /tmp With a German locale (my personal default), periods are used instead of commas. The patches for ls(1), du(1) and df(1) are available from this URL: http://www.secnetix.de/~olli/FreeBSD/ls-patch/ They apply cleanly to RELENG_6 and RELENG_5. Except for df(1) they also apply to HEAD (the df(1) patch requires a small modification because of the recent addition of the "kflag" variable in HEAD). Those patches are not ready yet (that's why i haven't send-pr'ed them). They're not checked against style(9), and the option that I used for the new feature is "-," which is probably against style(9), even though it is easy to remember. For the same reason I haven't updated the usage strings and the manpages yet (but I'll do that when there is consensus). I have a few questions ... 1. Which option letter should be used (if not "-,")? Currently, the following are unused in ls(1): -DEJKMNOQVXYejvyz They are also unused in du(1) and df(1), so one of those would be fine. 2. There's a bit of common code in my patches to the three utilities. Should that code be moved to libutil? That's also where humanize_number(3) already lives. It is also worth noting that probably many more tools might benefit from that feature. "netstat -i" and "ipfw show" come to mind immediately. Any comments are welcome. Best regards Oliver PS: I'm reading the list, so there's no need to put me on CC when replying. -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them. From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 18:42:59 2006 Return-Path: X-Original-To: freebsd-arch@FreeBSD.ORG Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B1C1916A412 for ; Fri, 24 Nov 2006 18:42:59 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [209.204.185.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id BD74C43D4C for ; Fri, 24 Nov 2006 18:42:16 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 246945B1E for ; Fri, 24 Nov 2006 10:42:59 -0800 (PST) To: freebsd-arch@FreeBSD.ORG In-reply-to: Your message of "Fri, 24 Nov 2006 18:40:13 +0100." <200611241740.kAOHeDHB072173@lurza.secnetix.de> Date: Fri, 24 Nov 2006 10:42:59 -0800 From: Bakul Shah Message-Id: <20061124184259.246945B1E@mail.bitblocks.com> Cc: Subject: Re: New option for ls(1), du(1), df(1) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 18:42:59 -0000 Oliver Fromme writes: ... > I've hacked a new option into those tools which uses > thousands separators for displaying the numbers. With > that option, the output looks like this (in US locale): > > Filesystem 1K-blocks Used Avail Capacity Mounted on > /dev/ad0s1a 1,012,974 62,418 869,520 7% / > /dev/ad0s1d 10,154,158 31,474 9,310,352 0% /var > /dev/ad0s1e 10,154,158 1,545,718 7,796,108 17% /usr > /dev/ad0s1f 128,009,212 1,358,600 116,409,876 1% /home > /dev/md0 297,326 20 273,520 0% /tmp ... Rather than add more code to quite a few programs I just use a separate program since this is useful in many places. $ df | hunit Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad0s3a 99,183 66,198 25,051 73% / devfs 1 1 0 100% /dev /dev/ad0s3e 190,167 29,468 145,486 17% /var /dev/ad0s3f 61,147,922 54,629,382 1,626,708 97% /usr Granted, columns don't line up right but that too belongs in a separate program. > It is also worth noting that probably many more tools > might benefit from that feature. "netstat -i" and > "ipfw show" come to mind immediately. Which is why a separate program is nicer. From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 19:19:54 2006 Return-Path: X-Original-To: freebsd-arch@FreeBSD.ORG Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7EEF616A40F for ; Fri, 24 Nov 2006 19:19:54 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from spunkymail-a8.dreamhost.com (sd-green-bigip-119.dreamhost.com [208.97.132.119]) by mx1.FreeBSD.org (Postfix) with ESMTP id 63AEE43D45 for ; Fri, 24 Nov 2006 19:19:11 +0000 (GMT) (envelope-from rnsanchez@wait4.org) Received: from sauron.lan.box (unknown [200.102.66.48]) by spunkymail-a8.dreamhost.com (Postfix) with ESMTP id E826E10A023; Fri, 24 Nov 2006 11:19:52 -0800 (PST) Date: Fri, 24 Nov 2006 17:19:49 -0200 From: Ricardo Nabinger Sanchez To: Oliver Fromme Message-Id: <20061124171949.cf649d76.rnsanchez@wait4.org> In-Reply-To: <200611241740.kAOHeDHB072173@lurza.secnetix.de> References: <200611241740.kAOHeDHB072173@lurza.secnetix.de> Organization: SYS_WAIT4 X-Mailer: Sylpheed version 2.3.0beta2 (GTK+ 2.10.6; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.ORG Subject: Re: New option for ls(1), du(1), df(1) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 19:19:54 -0000 On Fri, 24 Nov 2006 18:40:13 +0100 (CET) Oliver Fromme wrote: > I've hacked a new option into those tools which uses > thousands separators for displaying the numbers. Perhaps you want to hack humanize_humber(3) instead, which a great deal of tools use. -- Ricardo Nabinger Sanchez Powered by FreeBSD "Left to themselves, things tend to go from bad to worse." From owner-freebsd-arch@FreeBSD.ORG Fri Nov 24 22:15:52 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1382816A412 for ; Fri, 24 Nov 2006 22:15:52 +0000 (UTC) (envelope-from karels@karels.net) Received: from staring.karels.net (staring.karels.net [206.196.45.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33C7543D67 for ; Fri, 24 Nov 2006 22:15:01 +0000 (GMT) (envelope-from karels@karels.net) Received: from redrock.karels.net (root@redrock.karels.net [206.196.45.2]) by staring.karels.net (8.11.6/8.11.2) with ESMTP id kAOMFch15284; Fri, 24 Nov 2006 16:15:38 -0600 (CST) Received: from redrock.karels.net (karels@localhost.KARELS.NET [127.0.0.1]) by redrock.karels.net (8.11.6/8.11.2) with ESMTP id kAOMFYQ28380; Fri, 24 Nov 2006 16:15:38 -0600 (CST) Message-Id: <200611242215.kAOMFYQ28380@redrock.karels.net> To: Ricardo Nabinger Sanchez From: Mike Karels In-reply-to: Your message of Fri, 24 Nov 2006 17:19:49 -0200. <20061124171949.cf649d76.rnsanchez@wait4.org> Date: Fri, 24 Nov 2006 16:15:34 -0600 Cc: Oliver Fromme , freebsd-arch@freebsd.org Subject: Re: New option for ls(1), du(1), df(1) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: karels@karels.net List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 22:15:52 -0000 > > I've hacked a new option into those tools which uses > > thousands separators for displaying the numbers. > Perhaps you want to hack humanize_humber(3) instead, which a great deal of > tools use. Maybe humanize_number could look at an environment variable that describes what the human wants :-). Mike